• Journal of Integrative Natural Science
• /
• v.11 no.4
• /
• pp.204-208
• /
• 2018

Comparison of different optimizer performance in photovoltaic power modeling using artificial neural deep learning techniques is described in this paper. Six different deep learning optimizers are tested for Long-Short-Term Memory networks in this study. The optimizers are namely Adam, Stochastic Gradient Descent, Root Mean Square Propagation, Adaptive Gradient, and some variants such as Adamax and Nadam. For comparing the optimization techniques, high and low fluctuated photovoltaic power output are examined and the power output is real data obtained from the site at Mokpo university. Using Python Keras version, we have developed the prediction program for the performance evaluation of the optimizations. The prediction error results of each optimizer in both high and low power cases shows that the Adam has better performance compared to the other optimizers.

• International Journal of Computer Science & Network Security
• /
• v.23 no.10
• /
• pp.115-128
• /
• 2023

Deep learning has been incorporating various optimization techniques motivated by new pragmatic optimizing algorithm advancements and their usage has a central role in Machine learning. In recent past, new avatars of various optimizers are being put into practice and their suitability and applicability has been reported on various domains. The resurgence of novelty starts from Stochastic Gradient Descent to convex and non-convex and derivative-free approaches. In the contemporary of these horizons of optimizers, choosing a best-fit or appropriate optimizer is an important consideration in deep learning theme as these working-horse engines determines the final performance predicted by the model. Moreover with increasing number of deep layers tantamount higher complexity with hyper-parameter tuning and consequently need to delve for a befitting optimizer. We empirically examine most popular and widely used optimizers on various data sets and networks-like MNIST and GAN plus others. The pragmatic comparison focuses on their similarities, differences and possibilities of their suitability for a given application. Additionally, the recent optimizer variants are highlighted with their subtlety. The article emphasizes on their critical role and pinpoints buttress options while choosing among them.

• International Journal of Internet, Broadcasting and Communication
• /
• v.13 no.1
• /
• pp.100-106
• /
• 2021

Over the past few decades, natural language processing research has not made much. However, the widespread use of deep learning and neural networks attracted attention for the application of neural networks in natural language processing. Sentiment analysis is one of the challenges of natural language processing. Emotions are things that a person thinks and feels. Therefore, sentiment analysis should be able to analyze the person's attitude, opinions, and inclinations in text or actual text. In the case of emotion analysis, it is a priority to simply classify two emotions: positive and negative. In this paper we propose the deep learning based sentimental analysis system according to various optimizer that is SGD, ADAM and RMSProp. Through experimental result RMSprop optimizer shows the best performance compared to others on IMDB data set. Future work is to find more best hyper parameter for sentimental analysis system.

• International Journal of Computer Science & Network Security
• /
• v.21 no.7
• /
• pp.324-330
• /
• 2021

Agriculture is one of the essential needs of human life on planet Earth. It is the source of food and earnings for many individuals around the world. The economy of many countries is associated with the agriculture sector. Lots of diseases exist that attack various fruits and crops. Apple Tree Leaves also suffer different types of pathological conditions that affect their production. These pathological conditions include apple scab, cedar apple rust, or multiple diseases, etc. In this paper, an automatic detection framework based on deep learning is investigated for apple leaves disease classification. Different pre-trained models, VGG16, ResNetV2, InceptionV3, and MobileNetV2, are considered for transfer learning. A combination of parameters like learning rate, batch size, and optimizer is analyzed, and the best combination of ResNetV2 with Adam optimizer provided the best classification accuracy of 94%.

• Korean Journal of Remote Sensing
• /
• v.38 no.6_1
• /
• pp.1329-1341
• /
• 2022

Destroying the marine environment and marine ecosystem and causing marine accidents, marine debris is generated every year, and among them, submerged marine debris is difficult to identify and collect because it is on the seabed. Therefore, deep-learning-based semantic segmentation was experimented on waste fish nets and waste ropes using underwater images to identify efficient collection and distribution. For segmentation, a high-resolution network (HRNet), a state-of-the-art deep learning technique, was used, and the performance of each optimizer was compared. In the segmentation result fish net, F1 score=(86.46%, 86.20%, 85.29%), IoU=(76.15%, 75.74%, 74.36%), For the rope F1 score=(80.49%, 80.48%, 77.86%), IoU=(67.35%, 67.33%, 63.75%) in the order of adaptive moment estimation (Adam), Momentum, and stochastic gradient descent (SGD). Adam's results were the highest in both fish net and rope. Through the research results, the evaluation of segmentation performance for each optimizer and the possibility of segmentation of marine debris in the latest deep learning technique were confirmed. Accordingly, it is judged that by applying the latest deep learning technique to the identification of submerged marine debris through underwater images, it will be helpful in estimating the distribution of marine sedimentation debris through more accurate and efficient identification than identification through the naked eye.

• Journal of the Korean Society of Radiology
• /
• v.17 no.5
• /
• pp.709-715
• /
• 2023

In this study, the performance evaluation of image quality for noise reduction was implemented using the U-net deep learning architecture in computed tomography (CT) images. In order to generate input data, the Gaussian noise was applied to ground truth (GT) data, and datasets were consisted of 8:1:1 ratio of train, validation, and test sets among 1300 CT images. The Adagrad, Adam, and AdamW were used as optimizer function, and 10, 50 and 100 times for number of epochs were applied. In addition, learning rates of 0.01, 0.001, and 0.0001 were applied using the U-net deep learning model to compare the output image quality. To analyze the quantitative values, the peak signal to noise ratio (PSNR) and coefficient of variation (COV) were calculated. Based on the results, deep learning model was useful for noise reduction. We suggested that optimized hyper parameters for noise reduction in CT images were AdamW optimizer function, 100 times number of epochs and 0.0001 learning rates.

• Korean Journal of Remote Sensing
• /
• v.38 no.5_3
• /
• pp.819-834
• /
• 2022

Artificial Intelligence (AI) techniques have been effectively used for image classification, object detection, and image segmentation. Along with the recent advancement of computing power, deep learning models can build deeper and thicker networks and achieve better performance by creating more appropriate feature maps based on effective activation functions and optimizer algorithms. This review paper examined technical and academic trends of Convolutional Neural Network (CNN) and Transformer models that are emerging techniques in remote sensing and suggested their utilization strategies and development directions. A timely supply of satellite images and real-time processing for deep learning to cope with disaster monitoring will be required for future work. In addition, a big data platform dedicated to satellite images should be developed and integrated with drone and Closed-circuit Television (CCTV) images.

• Journal of Advanced Information Technology and Convergence
• /
• v.8 no.2
• /
• pp.119-127
• /
• 2018

Training a very large deep neural network can be painfully slow and prone to overfitting. Many researches have done for overcoming the problem. In this paper, a combination of early stopping and ADAM based deep neural network was presented. This form of deep network is useful for handling the big data because it automatically stop the training before overfitting occurs. Also generalization ability is better than pure deep neural network model.

• International Journal of Computer Science & Network Security
• /
• v.23 no.11
• /
• pp.133-141
• /
• 2023

This work take deeper analysis of Adaptive Moment Estimation (Adam) and Adam with Weight Decay (AdamW) implementation in real world text classification problem (DGA Malware Detection). AdamW is introduced by decoupling weight decay from L2 regularization and implemented as improved optimizer. This work introduces a novel implementation of AdamW variant as AdamW+ by further simplifying weight decay implementation in AdamW. DGA malware detection LSTM models results for Adam, AdamW and AdamW+ are evaluated on various DGA families/ groups as multiclass text classification. Proposed AdamW+ optimizer results has shown improvement in all standard performance metrics over Adam and AdamW. Analysis of outcome has shown that novel optimizer has outperformed both Adam and AdamW text classification based problems.

• International Journal of Computer Science & Network Security
• /
• v.21 no.9
• /
• pp.223-229
• /
• 2021

Classification of different blood cell types is an essential task for human's medical treatment. The white blood cells have different types of cells. Counting total White Blood Cells (WBC) and differential of the WBC types are required by the physicians to diagnose the disease correctly. This paper used transfer learning methods to the pre-trained deep learning models to classify different WBCs. The best pre-trained model was Inception ResNetV2 with Adam optimizer that produced classification accuracy of 98.4% for the dataset comprising four types of WBCs.