• Journal of IKEEE
• /
• v.25 no.4
• /
• pp.644-649
• /
• 2021

Self-Imitation Learning is a simple off-policy actor-critic algorithm that makes an agent find an optimal policy by using past good experiences. In case that Self-Imitation Learning is combined with reinforcement learning algorithms that have actor-critic architecture, it shows performance improvement in various game environments. However, its applications are limited to reinforcement learning algorithms that have actor-critic architecture. In this paper, we propose a method of applying Self-Imitation Learning to Deep Q-Network which is a value-based deep reinforcement learning algorithm and train it in various game environments. We also show that Self-Imitation Learning can be applied to Deep Q-Network to improve the performance of Deep Q-Network by comparing the proposed algorithm and ordinary Deep Q-Network training results.

• IEMEK Journal of Embedded Systems and Applications
• /
• v.17 no.5
• /
• pp.297-307
• /
• 2022

The simple procedural segment selection algorithm commonly used in Dynamic Adaptive Streaming over HTTP (DASH) reveals severe weakness to provide high-quality streaming services in the integrated mobile networks of various wired and wireless links. A major issue could be how to properly cope with dynamically changing underlying network conditions. The key to meet it should be to make the segment selection algorithm much more adaptive to fluctuation of network traffics. This paper presents a system architecture that replaces the existing procedural segment selection algorithm with a deep reinforcement learning algorithm based on the Asynchronous Advantage Actor-Critic (A3C). The distributed A3C-based deep learning server is designed and implemented to allow multiple clients in different network conditions to stream videos simultaneously, collect learning data quickly, and learn asynchronously, resulting in greatly improved learning speed as the number of video clients increases. The performance analysis shows that the proposed algorithm outperforms both the conventional DASH algorithm and the Deep Q-Network algorithm in terms of the user's quality of experience and the speed of deep learning.

• Nuclear Engineering and Technology
• /
• v.55 no.2
• /
• pp.506-515
• /
• 2023

This study is to find a method to learn engine sound after the start-up of a diesel generator installed in nuclear power plant with an unsupervised deep learning algorithm (CNN autoencoder) and a new method to predict the failure of a diesel generator using it. In order to learn the sound of a diesel generator with a deep learning algorithm, sound data recorded before and after the start-up of two diesel generators was used. The sound data of 20 min and 2 h were cut into 7 s, and the split sound was converted into a spectrogram image. 1200 and 7200 spectrogram images were created from sound data of 20 min and 2 h, respectively. Using two different deep learning algorithms (CNN autoencoder and binary classification), it was investigated whether the diesel generator post-start sounds were learned as normal. It was possible to accurately determine the post-start sounds as normal and the pre-start sounds as abnormal. It was also confirmed that the deep learning algorithm could detect the virtual abnormal sounds created by mixing the unusual sounds with the post-start sounds. This study showed that the unsupervised anomaly detection algorithm has a good accuracy increased about 3% with comparing to the binary classification algorithm.

• Journal of the Korea Society of Computer and Information
• /
• v.24 no.3
• /
• pp.11-17
• /
• 2019

In this paper, we propose a method to recognize the moving direction in the indoor environment by using the sensing system equipped with passive infrared (PIR) sensors and a deep learning algorithm. A PIR sensor generates a signal that can be distinguished according to the direction of movement of the user. A sensing system with four PIR sensors deployed by $45^{\circ}$ increments is developed and installed in the ceiling of the room. The PIR sensor signals from 6 users with 10-time experiments for 8 directions were collected. We extracted the raw data sets and performed experiments varying the number of sensors fed into the deep learning algorithm. The proposed sensing system using deep learning algorithm can recognize the users' moving direction by 99.2 %. In addition, with only one PIR senor, the recognition accuracy reaches 98.4%.

• Corrosion Science and Technology
• /
• v.18 no.6
• /
• pp.253-257
• /
• 2019

A deep learning based real-time painting surface inspection algorithm is proposed herein, designed for developing an autonomous inspection drone. The painting surface inspection is usually conducted manually. However, the manual inspection has a limitation in obtaining accurate data for correct judgement on the surface because of human error and deviation of individual inspection experiences. The best method to replace manual surface inspection is the vision-based inspection method with a camera, using various image processing algorithms. Nevertheless, the visual inspection is difficult to apply to surface inspection due to diverse appearances of material, hue, and lightning effects. To overcome technical limitations, a deep learning-based pattern recognition algorithm is proposed, which is specialized for painting surface inspections. The proposed algorithm functions in real time on the embedded board mounted on an autonomous inspection drone. The inspection results data are stored in the database and used for training the deep learning algorithm to improve performance. The various experiments for pre-inspection of painting processes are performed to verify real-time performance of the proposed deep learning algorithm.

• The Journal of the Korea institute of electronic communication sciences
• /
• v.15 no.1
• /
• pp.85-92
• /
• 2020

Deep learning shows outstanding performance in image and video analysis, such as object classification, object detection and semantic segmentation. In this paper, it is analyzed that the performances of deep learning models can be affected by characteristics of train dataset. It is proposed as a method for selecting activation function and optimization algorithm of deep learning to classify facial expression. Classification performances are compared and analyzed by applying various algorithms of each component of deep learning model for CK+, MMI, and KDEF datasets. As results of simulation, it is shown that genetic algorithm can be an effective solution for optimizing components of deep learning model.

• Korean Journal of Radiology
• /
• v.22 no.4
• /
• pp.612-623
• /
• 2021

Objective: To evaluate the diagnostic performance of a deep learning algorithm for the automated detection of developmental dysplasia of the hip (DDH) on anteroposterior (AP) radiographs. Materials and Methods: Of 2601 hip AP radiographs, 5076 cropped unilateral hip joint images were used to construct a dataset that was further divided into training (80%), validation (10%), or test sets (10%). Three radiologists were asked to label the hip images as normal or DDH. To investigate the diagnostic performance of the deep learning algorithm, we calculated the receiver operating characteristics (ROC), precision-recall curve (PRC) plots, sensitivity, specificity, positive predictive value (PPV), and negative predictive value (NPV) and compared them with the performance of radiologists with different levels of experience. Results: The area under the ROC plot generated by the deep learning algorithm and radiologists was 0.988 and 0.988-0.919, respectively. The area under the PRC plot generated by the deep learning algorithm and radiologists was 0.973 and 0.618-0.958, respectively. The sensitivity, specificity, PPV, and NPV of the proposed deep learning algorithm were 98.0, 98.1, 84.5, and 99.8%, respectively. There was no significant difference in the diagnosis of DDH by the algorithm and the radiologist with experience in pediatric radiology (p = 0.180). However, the proposed model showed higher sensitivity, specificity, and PPV, compared to the radiologist without experience in pediatric radiology (p < 0.001). Conclusion: The proposed deep learning algorithm provided an accurate diagnosis of DDH on hip radiographs, which was comparable to the diagnosis by an experienced radiologist.

• Journal of Korean Neurosurgical Society
• /
• v.66 no.1
• /
• pp.53-62
• /
• 2023

Objective : Deep learning is a machine learning approach based on artificial neural network training, and object detection algorithm using deep learning is used as the most powerful tool in image analysis. We analyzed and evaluated the diagnostic performance of a deep learning algorithm to identify skull fractures in plain radiographic images and investigated its clinical applicability. Methods : A total of 2026 plain radiographic images of the skull (fracture, 991; normal, 1035) were obtained from 741 patients. The RetinaNet architecture was used as a deep learning model. Precision, recall, and average precision were measured to evaluate the deep learning algorithm's diagnostic performance. Results : In ResNet-152, the average precision for intersection over union (IOU) 0.1, 0.3, and 0.5, were 0.7240, 0.6698, and 0.3687, respectively. When the intersection over union (IOU) and confidence threshold were 0.1, the precision was 0.7292, and the recall was 0.7650. When the IOU threshold was 0.1, and the confidence threshold was 0.6, the true and false rates were 82.9% and 17.1%, respectively. There were significant differences in the true/false and false-positive/false-negative ratios between the anterior-posterior, towne, and both lateral views (p=0.032 and p=0.003). Objects detected in false positives had vascular grooves and suture lines. In false negatives, the detection performance of the diastatic fractures, fractures crossing the suture line, and fractures around the vascular grooves and orbit was poor. Conclusion : The object detection algorithm applied with deep learning is expected to be a valuable tool in diagnosing skull fractures.

• International Journal of Computer Science & Network Security
• /
• v.22 no.6
• /
• pp.364-373
• /
• 2022

Image morphing methods make seamless transition changes in the image and mask the meaningful information attached to it. This can be detected by traditional machine learning algorithms and new emerging deep learning algorithms. In this research work, scope of different Hybrid learning approaches having combination of Deep learning and Machine learning are being analyzed with the public dataset CASIA V1.0, CASIA V2.0 and DVMM to find the most efficient algorithm. The simulated results with CNN (Convolution Neural Network), Hybrid approach of CNN along with SVM (Support Vector Machine) and Hybrid approach of CNN along with Random Forest algorithm produced 96.92 %, 95.98 and 99.18 % accuracy respectively with the CASIA V2.0 dataset having 9555 images. The accuracy pattern of applied algorithms changes with CASIA V1.0 data and DVMM data having 1721 and 1845 set of images presenting minimal accuracy with Hybrid approach of CNN and Random Forest algorithm. It is confirmed that the choice of best algorithm to find image forgery depends on input data type. This paper presents the combination of best suited algorithm to detect image morphing with different input datasets.

• Journal of the Korea Society of Computer and Information
• /
• v.23 no.10
• /
• pp.23-31
• /
• 2018

Recently, there have been many studies on machine learning. Among them, studies on reinforcement learning are actively worked. In this study, we propose a controller to control bicycle using DDPG (Deep Deterministic Policy Gradient) algorithm which is the latest deep reinforcement learning method. In this paper, we redefine the compensation function of bicycle dynamics and neural network to learn agents. When using the proposed method for data learning and control, it is possible to perform the function of not allowing the bicycle to fall over and reach the further given destination unlike the existing method. For the performance evaluation, we have experimented that the proposed algorithm works in various environments such as fixed speed, random, target point, and not determined. Finally, as a result, it is confirmed that the proposed algorithm shows better performance than the conventional neural network algorithms NAF and PPO.