• Journal of KIISE:Software and Applications
• /
• v.26 no.11
• /
• pp.1261-1269
• /
• 1999

다층 퍼셉트론은 다양한 응용 분야에 성공적으로 적용되고 있는 대표적인 신경회로망 모델이다. 그러나 다층 퍼셉트론의 학습에서 나타나는 플라토에 기인한 느린 학습 속도와 지역 극소는 실제 응용문제에 적용함에 있어서 가장 큰 문제로 지적되어왔다. 이 문제를 해결하기 위해 여러 가지 다양한 학습알고리즘들이 개발되어 왔으나, 계산의 비효율성으로 인해 실제 문제에는 적용하기 힘든 예가 많은 등, 현재까지 만족할 만한 해결책은 제시되지 못하고 있다. 본 논문에서는 다층퍼셉트론의 베이시스 함수로 사용되는 시그모이드 함수를 보다 일반화된 형태로 정의하여 사용함으로써 학습에 있어서의 플라토를 완화하고, 지역극소에 빠지는 것을 줄이는 접근방법을 소개한다. 본 방법은 기존의 변형된 가중치 수정식을 사용한 학습 속도 향상의 방법들과는 다른 접근 방법을 택함으로써 기존의 방법들과 함께 사용하는 것이 가능하다는 특징을 갖고 있다. 제안하는 방법의 성능을 확인하기 위하여 간단한 패턴 인식 문제들에의 적용 실험 및 기존의 학습 속도 향상 방법을 함께 사용하여 시계열 예측 문제에 적용한 실험을 수행하였고, 그 결과로부터 제안안 방법의 효율성을 확인할 수 있었다. Abstract A multilayer perceptron is the most well-known neural network model which has been successfully applied to various fields of application. Its slow learning caused by plateau and local minima of gradient descent learning, however, have been pointed as the biggest problems in its practical use. To solve such a problem, a number of researches on learning algorithms have been conducted, but it can be said that none of satisfying solutions have been presented so far because the problems such as computational inefficiency have still been existed in these algorithms. In this paper, we propose a new learning approach to minimize the effect of plateau and reduce the possibility of getting trapped in local minima by generalizing the sigmoidal function which is used as the basis function of a multilayer perceptron. Adapting a new approach that differs from the conventional methods with revised updating equation, the proposed method can be used together with the existing methods to improve the learning performance. We conducted some experiments to test the proposed method on simple problems of pattern recognition and a problem of time series prediction, compared our results with the results of the existing methods, and confirmed that the proposed method is efficient enough to apply to the real problems.

• KIPS Transactions on Computer and Communication Systems
• /
• v.10 no.7
• /
• pp.191-198
• /
• 2021

Since the size of training dataset become large and the model is getting deeper to achieve high accuracy in deep learning, the deep neural network training requires a lot of computation and it takes too much time with a single node. Therefore, distributed deep learning is proposed to reduce the training time by distributing computation across multiple nodes. In this study, we propose hybrid allreduce strategy that considers the characteristics of each layer and communication and computational overlapping technique for synchronization of distributed deep learning. Since the convolution layer has fewer parameters than the fully-connected layer as well as it is located at the upper, only short overlapping time is allowed. Thus, butterfly allreduce is used to synchronize the convolution layer. On the other hand, fully-connecter layer is synchronized using ring all-reduce. The empirical experiment results on PyTorch with our proposed scheme shows that the proposed method reduced the training time by up to 33% compared to the baseline PyTorch.

• Journal of the Korea Institute of Information Security & Cryptology
• /
• v.29 no.3
• /
• pp.491-501
• /
• 2019

In side-channel analysis, which exploit physical leakage from a cryptographic device, deep learning based attack has been significantly interested in recent years. However, most of the state-of-the-art methods have been focused on classifying side-channel information in a profiled scenario where attackers can obtain label of training data. In this paper, we propose a new method based on deep learning to improve non-profiling side-channel attack such as Differential Power Analysis and Correlation Power Analysis. The proposed method is a signal preprocessing technique that reduces the noise in a trace by modifying Auto-Encoder framework to the context of side-channel analysis. Previous work on Denoising Auto-Encoder was trained through randomly added noise by an attacker. In this paper, the proposed model trains Auto-Encoder through the noise from real data using the noise-reduced-label. Also, the proposed method permits to perform non-profiled attack by training only a single neural network. We validate the performance of the noise reduction of the proposed method on real traces collected from ChipWhisperer board. We demonstrate that the proposed method outperforms classic preprocessing methods such as Principal Component Analysis and Linear Discriminant Analysis.

• Journal of the Korean Society of Radiology
• /
• v.15 no.4
• /
• pp.547-554
• /
• 2021

In order to quickly and accurately diagnose pneumonia on a chest X-ray image, different batch sizes of 4, 8, 16, and 32 were applied to the same Xception deep learning model, and modeling was performed 3 times, respectively. As a result of the performance evaluation of deep learning modeling, in the case of modeling to which batch size 32 was applied, the results of accuracy, loss function value, mean square error, and learning time per epoch showed the best results. And in the accuracy evaluation of the Test Metric, the modeling applied with batch size 8 showed the best results, and the precision evaluation showed excellent results in all batch sizes. In the recall evaluation, modeling applied with batch size 16 showed the best results, and for F1-score, modeling applied with batch size 16 showed the best results. And the AUC score evaluation was the same for all batch sizes. Based on these results, deep learning modeling with batch size 32 showed high accuracy, stable artificial neural network learning, and excellent speed. It is thought that accurate and rapid lesion detection will be possible if a batch size of 32 is applied in an automatic diagnosis study for feature extraction and classification of pneumonia in chest X-ray images using deep learning in the future.

• KSCE Journal of Civil and Environmental Engineering Research
• /
• v.41 no.6
• /
• pp.751-760
• /
• 2021

Most bridges in Korea are over 20 years old, and many problems linked to their deterioration are being reported. The current practice for bridge inspection mainly depends on expert evaluation, which can be subjective. Recent studies have introduced data-driven methods using building information modeling, which can be more efficient and objective, but these methods require manual procedures that consume time and money. To overcome this, this study developed an image-based automaticbridge component classification network to reduce the time and cost required for converting the visual information of bridges to a digital model. The proposed method comprises two convolutional neural networks. The first network estimates the type of the bridge based on the superstructure, and the second network classifies the bridge components. In avalidation test, the proposed system automatically classified the components of 461 bridge images with 96.6 % of accuracy. The proposed approach is expected to contribute toward current bridge maintenance practice.

• Journal of Internet Computing and Services
• /
• v.23 no.5
• /
• pp.79-85
• /
• 2022

Anomaly detection is a method to detect and block abnormal data flows in general users' data sets. The previously known method is a method of detecting and defending an attack based on a signature using the signature of an already known attack. This has the advantage of a low false positive rate, but the problem is that it is very vulnerable to a zero-day vulnerability attack or a modified attack. However, in the case of anomaly detection, there is a disadvantage that the false positive rate is high, but it has the advantage of being able to identify, detect, and block zero-day vulnerability attacks or modified attacks, so related studies are being actively conducted. In this study, we want to deal with these anomaly detection mechanisms, and we propose a new mechanism that performs both anomaly detection and classification while supplementing the high false positive rate mentioned above. In this study, the experiment was conducted with five configurations considering the characteristics of various algorithms. As a result, the model showing the best accuracy was proposed as the result of this study. After detecting an attack by applying the Extra Tree and Three-layer ANN at the same time, the attack type is classified using the Extra Tree for the classified attack data. In this study, verification was performed on the NSL-KDD data set, and the accuracy was 99.8%, 99.1%, 98.9%, 98.7%, and 97.9% for Normal, Dos, Probe, U2R, and R2L, respectively. This configuration showed superior performance compared to other models.

• Journal of Broadcast Engineering
• /
• v.26 no.4
• /
• pp.429-440
• /
• 2021

With the recent development of deep convolutional neural network learning, deep learning techniques applied to single image super-resolution are showing good results. One of the existing deep learning-based super-resolution techniques is RDN(Residual Dense Network), in which the initial feature information is transmitted to the last layer using residual dense blocks, and subsequent layers are restored using input information of previous layers. However, if all hierarchical features are connected and learned and a large number of residual dense blocks are stacked, despite good performance, a large number of parameters and huge computational load are needed, so it takes a lot of time to learn a network and a slow processing speed, and it is not applicable to a mobile system. In this paper, we use the residual dense structure, which is a continuous memory structure that reuses previous information, and the residual dense channel attention block using the channel attention method that determines the importance according to the feature map of the image. We propose a method that can increase the depth to obtain a large receptive field and maintain a concise model at the same time. As a result of the experiment, the proposed network obtained PSNR as low as 0.205dB on average at 4× magnification compared to RDN, but about 1.8 times faster processing speed, about 10 times less number of parameters and about 1.74 times less computation.

• The Journal of the Korea institute of electronic communication sciences
• /
• v.17 no.6
• /
• pp.1137-1144
• /
• 2022

In this study, an artificial intelligence(AI) was developed to help with facial expression practice in order to express emotions. The developed AI used multimodal inputs consisting of sentences and facial images for deep neural networks (DNNs). The DNNs calculated similarities between the emotions predicted by the sentences and the emotions predicted by facial images. The user practiced facial expressions based on the situation given by sentences, and the AI provided the user with numerical feedback based on the similarity between the emotion predicted by sentence and the emotion predicted by facial expression. ResNet34 structure was trained on FER2013 public data to predict emotions from facial images. To predict emotions in sentences, KoBERT model was trained in transfer learning manner using the conversational speech dataset for emotion classification opened to the public by AIHub. The DNN that predicts emotions from the facial images demonstrated 65% accuracy, which is comparable to human emotional classification ability. The DNN that predicts emotions from the sentences achieved 90% accuracy. The performance of the developed AI was evaluated through experiments with changing facial expressions in which an ordinary person was participated.

• Journal of Korea Society of Industrial Information Systems
• /
• v.27 no.6
• /
• pp.13-23
• /
• 2022

In this paper, we propose a detailed component image classification algorithm by fashion item for unstructured data retrieval in the fashion field. Due to the COVID-19 environment, AI-based online shopping malls are increasing recently. However, there is a limit to accurate unstructured data search with existing keyword search and personalized style recommendations based on user surfing behavior. In this study, pre-processing using Mask R-CNN was conducted using images crawled from online shopping sites and then classified components for each fashion item through CNN. We obtain the accuaracy for collar of the shirt's as 93.28%, the pattern of the shirt as 98.10%, the 3 classese fit of the jeans as 91.73%, And, we further obtained one for the 4 classes fit of jeans as 81.59% and the color of the jeans as 93.91%. At the results for the decorated items, we also obtained the accuract of the washing of the jeans as 91.20% and the demage of jeans accuaracy as 92.96%.

• KIPS Transactions on Software and Data Engineering
• /
• v.12 no.1
• /
• pp.51-58
• /
• 2023

In this paper, we propose an organ segmentation technique for the automatic extraction of medical diagnostic indicators from X-ray images. In order to calculate diagnostic indicators of heart disease and spinal disease such as VHS(vertebral heart scale) and Cobb angle, it is necessary to accurately segment the thoracic spine, carina, and heart in a chest X-ray image. A deep neural network model in which the high-resolution representation of the image for each layer and the structure converted into a low-resolution feature map are connected in parallel was adopted. This structure enables the relative position information in the image to be effectively reflected in the segmentation process. It is shown that learning performance can be improved by combining the OCR module, in which pixel information and object information are mutually interacted in a multi-step process, and the channel attention module, which allows each channel of the network to be reflected as different weight values. In addition, a method of augmenting learning data is presented in order to provide robust performance against changes in the position, shape, and size of the subject in the X-ray image. The effectiveness of the proposed theory was evaluated through an experiment using 145 human chest X-ray images and 118 animal X-ray images.