Browse > Article
http://dx.doi.org/10.14400/JDC.2022.20.4.389

Optimal Algorithm and Number of Neurons in Deep Learning  

Jang, Ha-Young (Dept. of Computer Engineering, Kunsan University)
You, Eun-Kyung (Avionics Software Development Center)
Kim, Hyeock-Jin (Dept. of Computer Engineering, Chungwoon University)
Publication Information
Journal of Digital Convergence / v.20, no.4, 2022 , pp. 389-396 More about this Journal
Abstract
Deep Learning is based on a perceptron, and is currently being used in various fields such as image recognition, voice recognition, object detection, and drug development. Accordingly, a variety of learning algorithms have been proposed, and the number of neurons constituting a neural network varies greatly among researchers. This study analyzed the learning characteristics according to the number of neurons of the currently used SGD, momentum methods, AdaGrad, RMSProp, and Adam methods. To this end, a neural network was constructed with one input layer, three hidden layers, and one output layer. ReLU was applied to the activation function, cross entropy error (CEE) was applied to the loss function, and MNIST was used for the experimental dataset. As a result, it was concluded that the number of neurons 100-300, the algorithm Adam, and the number of learning (iteraction) 200 would be the most efficient in deep learning learning. This study will provide implications for the algorithm to be developed and the reference value of the number of neurons given new learning data in the future.
Keywords
Deep Learning; Stochastic Gradient Descent; Momentum; AdaGrad; RMSProp; Adam;
Citations & Related Records
Times Cited By KSCI : 4  (Citation Analysis)
연도 인용수 순위
1 S. Ruder. (2016). An overview of gradient descent optimization algorithms, arXiv preprint arXiv:1609.04747.
2 G. H. Joo, C. H. Park, & H. S. Im. (2020). Performance Evaluation of Machine Learning Optimizers. Journal of IKEEE, 24(3), 766-776.   DOI
3 D. Kingma & J. Ba. (2014). Adam: A Method for Stochastic optimization, arXiv preprint arXiv:1412.6980.
4 T. Mitchell. (1997). Machine Learning. NW: McGraw-Hill.
5 Y. J. Seo & K. S. Kim. (2021). Development of an Artificial Intelligence Model for Predicting the Policy and the Environment Affecting the Public Interest on the Real Estate. Journal of Korean Institute of Information Technology, 19(12), 135-141.   DOI
6 Y. M. Park & D. I. Jung. (2022). Development of vision system for quality inspection of automotive parts and comparison of machine learning models. The journal of Convergence on Culture Technology, 8(1), 409-415.
7 M. H. Jung & W. H. Kwon. (2021). Present status and future of AI-based drug discovery. Journal of the Korea Institute Of Information and Communication Engineering, 25(12), 1797-1808.   DOI
8 E. J. Lim, E. Y. Lee, & I. G. Lee. (2021). Behavior and Script Similarity-Based Cryptojacking Detection Framework Using Machine Learning. Journal of The Korea Institute of Information Security and Cryptology, 31(6), 1105-1114.   DOI
9 D. S. Yook, H. W. Lee, & I. C. Yoo. (2020). A survey on parallel training algorithms for deep neural networks. The Journal of The Acoustical Society of Korea, 39(6), 505-514.   DOI
10 J. P. Youn. (2021). A Study on the Prediction Method of Air-Compressor's Health Condition Using Convolutional Neural Network(CNN), Master's thesis, Soong-sil University, Seoul.
11 D. John, H. Elad & S. Yoram. (2011). Adaptive Subgradient Methods for Online Learning and Stochastic Optimization. Journal of Machine Learning Research, 12, 2121-2159.
12 https://terms.naver.com/entry.naver?docId=3386834&categoryId=58369&cid=58369.
13 M. J. Kang. (2020). Comparison of Gradient Descent for Deep Learning. Journal of Korea Academia-Industrial cooperation Society, 21(2), 189-194.   DOI
14 S. Y. Kim, W. K. Chung, & S. R. Shin. (2019). Acoustic Full-waveform Inversion using Adam Optimizer. Geophysics and Geophysical Exploration, 22(4), 202-209.   DOI
15 Y. LeCun, B. E. Boser, J. S. Denker & D. Henderson, R.E. Howard, W. E Hubbard & L.D. Jackel. (1990). Handwritten digit recognition with a back-propagation network, In Advances in neural information processing systems, 396-404.