• Journal of the Computational Structural Engineering Institute of Korea
• /
• v.36 no.4
• /
• pp.221-232
• /
• 2023

This paper presents a novel convolutional neural network (CNN)-based approach for predicting bolt clamping force in the early bolt loosening state of bolted structures. The approach entails tightening eight bolts with different clamping forces and generating frequency responses, which are then used to create a similarity map. This map quantifies the magnitude and shape similarity between the frequency responses and the initial model in a fully fastened state. Krylov subspace-based model order reduction is employed to efficiently handle the large amount of frequency response data. The CNN model incorporates a regression output layer to predict the clamping forces of the bolts. Its performance is evaluated by training the network by using various amounts of training data and convolutional layers. The input data for the model are derived from the magnitude and shape similarity map obtained from the frequency responses. The results demonstrate the diagnostic potential and effectiveness of the proposed approach in detecting early bolt loosening. Accurate bolt clamping force predictions in the early loosening state can thus be achieved by utilizing the frequency response data and CNN model. The findings afford valuable insights into the application of CNNs for assessing the integrity of bolted structures.

• Journal of Korea Multimedia Society
• /
• v.25 no.2
• /
• pp.247-256
• /
• 2022

Visual tracking of selected target objects is fundamental challenging problems in computer vision. Object tracking localize the region of target object with bounding box in the video. We propose a Siam-FPN based custom fully CNN to solve visual tracking problems by regressing the target area in an end-to-end manner. A method of preserving the feature information flow using a feature map connection structure was applied. In this way, information is preserved and emphasized across the network. To regress object region and to classify object, the region proposal network was connected with the Siamese network. The performance of the tracking algorithm was evaluated using the OTB-100 dataset. Success Plot and Precision Plot were used as evaluation matrix. As a result of the experiment, 0.621 in Success Plot and 0.838 in Precision Plot were achieved.

• Smart Structures and Systems
• /
• v.29 no.1
• /
• pp.141-151
• /
• 2022

In recent years, the industry and research communities have focused on developing autonomous crack inspection approaches, which mainly include image acquisition and crack detection. In these approaches, mobile devices such as cameras, drones or smartphones are utilized as sensing platforms to acquire structural images, and the deep learning (DL)-based methods are being developed as important crack detection approaches. However, the process of image acquisition and collection is time-consuming, which delays the inspection. Also, the present mobile devices such as smartphones can be not only a sensing platform but also a computing platform that can be embedded with deep neural networks (DNNs) to conduct on-site crack detection. Due to the limited computing resources of mobile devices, the size of the DNNs should be reduced to improve the computational efficiency. In this study, an architecture called pruned crack recognition network (PCR-Net) was developed for the detection of structural cracks. A dataset containing 11000 images was established based on the raw images from bridge inspections. A pruning method was introduced to reduce the size of the base architecture for the optimization of the model size. Comparative studies were conducted with image processing techniques (IPTs) and other DNNs for the evaluation of the performance of the proposed PCR-Net. Furthermore, a modularly designed framework that integrated the PCR-Net was developed to realize a DL-based crack detection application for smartphones. Finally, on-site crack detection experiments were carried out to validate the performance of the developed system of smartphone-based detection of structural cracks.

• Journal of the Korean Society of Surveying, Geodesy, Photogrammetry and Cartography
• /
• v.38 no.1
• /
• pp.23-33
• /
• 2020

Recently, high-resolution images can be easily acquired using UAV (Unmanned Aerial Vehicle), so that it is possible to produce small area observation and spatial information at low cost. In particular, research on the generation of cover maps in crop production areas is being actively conducted for monitoring the agricultural environment. As a result of comparing classification performance by applying RF(Random Forest), SVM(Support Vector Machine) and CNN(Convolutional Neural Network), deep learning classification method has many advantages in image classification. In particular, land cover classification using satellite images has the advantage of accuracy and time of classification using satellite image data set and pre-trained parameters. However, UAV images have different characteristics such as satellite images and spatial resolution, which makes it difficult to apply them. In order to solve this problem, we conducted a study on the application of deep learning algorithms that can be used for analyzing agricultural lands where UAV data sets and small-scale composite cover exist in Korea. In this study, we applied DeepLab V3 +, FC-DenseNet (Fully Convolutional DenseNets) and FRRN-B (Full-Resolution Residual Networks), the semantic image classification of the state-of-art algorithm, to UAV data set. As a result, DeepLab V3 + and FC-DenseNet have an overall accuracy of 97% and a Kappa coefficient of 0.92, which is higher than the conventional classification. The applicability of the cover classification using UAV images of small areas is shown.

• The Journal of the Acoustical Society of Korea
• /
• v.36 no.6
• /
• pp.407-412
• /
• 2017

In this paper, we propose a speech emotion recognition method using a deep neural network based on the attention mechanism. The proposed method consists of a combination of CNN (Convolution Neural Networks), GRU (Gated Recurrent Unit), DNN (Deep Neural Networks) and attention mechanism. The spectrogram of the speech signal contains characteristic patterns according to the emotion. Therefore, we modeled characteristic patterns according to the emotion by applying the tuned Gabor filters as convolutional filter of typical CNN. In addition, we applied the attention mechanism with CNN and FC (Fully-Connected) layer to obtain the attention weight by considering context information of extracted features and used it for emotion recognition. To verify the proposed method, we conducted emotion recognition experiments on six emotions. The experimental results show that the proposed method achieves higher performance in speech emotion recognition than the conventional methods.

• Journal of the Computational Structural Engineering Institute of Korea
• /
• v.36 no.5
• /
• pp.339-346
• /
• 2023

The uncertainties of microstructural features affect the properties of materials. Numerous pores that are randomly distributed in materials make it difficult to predict the properties of the materials. The distribution of pores in cementitious materials has a great influence on their mechanical properties. Existing studies focus on analyzing the statistical relationship between pore distribution and material responses, and the correlation between them is not yet fully determined. In this study, the mechanical response of cementitious materials is predicted through an image-based data approach using a convolutional neural network (CNN), and the correlation between pore distribution and material response is analyzed. The dataset for machine learning consists of high-resolution micro-CT images and the properties (tensile strength) of cementitious materials. The microstructures are characterized, and the mechanical properties are evaluated through 2D direct tension simulations using the phase-field fracture model. The attributes of input images are analyzed to identify the spot with the greatest influence on the prediction of material response through CNN. The correlation between pore distribution characteristics and material response is analyzed by comparing the active regions during the CNN process and the pore distribution.

• Journal of Intelligence and Information Systems
• /
• v.24 no.1
• /
• pp.205-225
• /
• 2018

Convolutional Neural Network (ConvNet) is one class of the powerful Deep Neural Network that can analyze and learn hierarchies of visual features. Originally, first neural network (Neocognitron) was introduced in the 80s. At that time, the neural network was not broadly used in both industry and academic field by cause of large-scale dataset shortage and low computational power. However, after a few decades later in 2012, Krizhevsky made a breakthrough on ILSVRC-12 visual recognition competition using Convolutional Neural Network. That breakthrough revived people interest in the neural network. The success of Convolutional Neural Network is achieved with two main factors. First of them is the emergence of advanced hardware (GPUs) for sufficient parallel computation. Second is the availability of large-scale datasets such as ImageNet (ILSVRC) dataset for training. Unfortunately, many new domains are bottlenecked by these factors. For most domains, it is difficult and requires lots of effort to gather large-scale dataset to train a ConvNet. Moreover, even if we have a large-scale dataset, training ConvNet from scratch is required expensive resource and time-consuming. These two obstacles can be solved by using transfer learning. Transfer learning is a method for transferring the knowledge from a source domain to new domain. There are two major Transfer learning cases. First one is ConvNet as fixed feature extractor, and the second one is Fine-tune the ConvNet on a new dataset. In the first case, using pre-trained ConvNet (such as on ImageNet) to compute feed-forward activations of the image into the ConvNet and extract activation features from specific layers. In the second case, replacing and retraining the ConvNet classifier on the new dataset, then fine-tune the weights of the pre-trained network with the backpropagation. In this paper, we focus on using multiple ConvNet layers as a fixed feature extractor only. However, applying features with high dimensional complexity that is directly extracted from multiple ConvNet layers is still a challenging problem. We observe that features extracted from multiple ConvNet layers address the different characteristics of the image which means better representation could be obtained by finding the optimal combination of multiple ConvNet layers. Based on that observation, we propose to employ multiple ConvNet layer representations for transfer learning instead of a single ConvNet layer representation. Overall, our primary pipeline has three steps. Firstly, images from target task are given as input to ConvNet, then that image will be feed-forwarded into pre-trained AlexNet, and the activation features from three fully connected convolutional layers are extracted. Secondly, activation features of three ConvNet layers are concatenated to obtain multiple ConvNet layers representation because it will gain more information about an image. When three fully connected layer features concatenated, the occurring image representation would have 9192 (4096+4096+1000) dimension features. However, features extracted from multiple ConvNet layers are redundant and noisy since they are extracted from the same ConvNet. Thus, a third step, we will use Principal Component Analysis (PCA) to select salient features before the training phase. When salient features are obtained, the classifier can classify image more accurately, and the performance of transfer learning can be improved. To evaluate proposed method, experiments are conducted in three standard datasets (Caltech-256, VOC07, and SUN397) to compare multiple ConvNet layer representations against single ConvNet layer representation by using PCA for feature selection and dimension reduction. Our experiments demonstrated the importance of feature selection for multiple ConvNet layer representation. Moreover, our proposed approach achieved 75.6% accuracy compared to 73.9% accuracy achieved by FC7 layer on the Caltech-256 dataset, 73.1% accuracy compared to 69.2% accuracy achieved by FC8 layer on the VOC07 dataset, 52.2% accuracy compared to 48.7% accuracy achieved by FC7 layer on the SUN397 dataset. We also showed that our proposed approach achieved superior performance, 2.8%, 2.1% and 3.1% accuracy improvement on Caltech-256, VOC07, and SUN397 dataset respectively compare to existing work.

• IEMEK Journal of Embedded Systems and Applications
• /
• v.11 no.3
• /
• pp.163-171
• /
• 2016

Lane detection is a widely researched topic. Although simple road detection is easily achieved by previous methods, lane detection becomes very difficult in several complex cases involving noisy edges. To address this, we use a Convolution neural network (CNN) for image enhancement. CNN is a deep learning method that has been very successfully applied in object detection and recognition. In this paper, we introduce a robust lane detection method based on a CNN combined with random sample consensus (RANSAC) algorithm. Initially, we calculate edges in an image using a hat shaped kernel, then we detect lanes using the CNN combined with the RANSAC. In the training process of the CNN, input data consists of edge images and target data is images that have real white color lanes on an otherwise black background. The CNN structure consists of 8 layers with 3 convolutional layers, 2 subsampling layers and multi-layer perceptron (MLP) of 3 fully-connected layers. Convolutional and subsampling layers are hierarchically arranged to form a deep structure. Our proposed lane detection algorithm successfully eliminates noise lines and was found to perform better than other formal line detection algorithms such as RANSAC

• Proceedings of the Korea Information Processing Society Conference
• /
• 2017.04a
• /
• pp.1031-1034
• /
• 2017

기존 번호판 검출 및 인식에 사용되어지는 방법은 사랑이 직접 번호판의 특정을 기술하여 검출을 진행한다. 본 연구에서는 학습 기반의 완전 컨볼루션 신경망을 이용하여 번호판을 검출하였고 신경망은 약 27MB의 용량만으로 110-FPS 정도의 성능을 얻었다. 학습을 위한 데이터는 한국 번호판의 모든 종류 및 주간, 야간의 환경을 포함한 대략 5000개를 직접 수집하였다 또한 5000개의 데이터를 회전 및 이동에 대한 무작위적인 변형을 주어 대략 15000개의 데이터로 확장하였다 확장된 데이터로 얻은 결과로 번호판 검출률 97%를 얻었다.

• Journal of Information Processing Systems
• /
• v.16 no.4
• /
• pp.809-819
• /
• 2020

Gender prediction accuracy increases as convolutional neural network (CNN) architecture evolves. This paper compares voting and ensemble schemes to utilize the already trained five CNN models to further improve gender prediction accuracy. The majority voting usually requires odd-numbered models while the proposed softmax-based voting can utilize any number of models to improve accuracy. The ensemble of CNN models combined with one more fully-connected layer requires further tuning or training of the models combined. With experiments, it is observed that the voting or ensemble of CNN models leads to further improvement of gender prediction accuracy and that especially softmax-based voters always show better gender prediction accuracy than majority voters. Also, compared with softmax-based voters, ensemble models show a slightly better or similar accuracy with added training of the combined CNN models. Softmax-based voting can be a fast and efficient way to get better accuracy without further training since the selection of the top accuracy models among available CNN pre-trained models usually leads to similar accuracy to that of the corresponding ensemble models.