Journal of Korea Multimedia Society (한국멀티미디어학회논문지)

Korea Multimedia Society (한국멀티미디어학회)
권호 표지
DOI QR코드

DOI QR Code

Optimized Neural Network Weights and Biases Using Particle Swarm Optimization Algorithm for Prediction Applications

  • Ahmadzadeh, Ezat (Department of Computer Engineering, Chosun University) ;
  • Lee, Jieun (Department of Computer Engineering, Chosun University) ;
  • Moon, Inkyu (Department of Computer Engineering, Chosun University)
  • Received : 2017.05.12
  • Accepted : 2017.07.04
  • Published : 2017.08.31
Download PDF
⟨ Previous Next ⟩

Abstract

Artificial neural networks (ANNs) play an important role in the fields of function approximation, prediction, and classification. ANN performance is critically dependent on the input parameters, including the number of neurons in each layer, and the optimal values of weights and biases assigned to each neuron. In this study, we apply the particle swarm optimization method, a popular optimization algorithm for determining the optimal values of weights and biases for every neuron in different layers of the ANN. Several regression models, including general linear regression, Fourier regression, smoothing spline, and polynomial regression, are conducted to evaluate the proposed method's prediction power compared to multiple linear regression (MLR) methods. In addition, residual analysis is conducted to evaluate the optimized ANN accuracy for both training and test datasets. The experimental results demonstrate that the proposed method can effectively determine optimal values for neuron weights and biases, and high accuracy results are obtained for prediction applications. Evaluations of the proposed method reveal that it can be used for prediction and estimation purposes, with a high accuracy ratio, and the designed model provides a reliable technique for optimization. The simulation results show that the optimized ANN exhibits superior performance to MLR for prediction purposes.

Keywords

References

  1. A. Maren, C. Harston, and R. Pap, Handbook of Neural Computing Applications, Academic Press, Cambridge, 2014.
  2. J. Chae, J. Lim, H. Kim, and J. Lee, “Study on Real-time Gesture Recognition based on Convolutional Neural Network for Game Applications,” Journal of Korea Multimedia Society, Vol. 20, No. 5, pp. 835-843, 2017. https://doi.org/10.9717/kmms.2017.20.5.835
  3. D. Graupe, Principles of Artificial Neural Networks, World Scientific, Singapore, 2013.
  4. F. Gharehchopoghand and E. Ahmadzadeh, "Artificial Neural Network Application in Letters Recognition for Farsi/Arabicmanuscripts," International Journal of Scientific and Technology Research, Vol. 1, Issue 8, pp. 49-54, 2012.
  5. E. Ahmadzadeh, W. Azar, and M. Masdari, "A New Approach to Persian and Arabic Handwritten Character Recognition with Hybrid of Artificial Neural Network and Genetic Algorithm," International Journal of Applied Information Systems, Vol. 6, pp. 11-15, 2014. https://doi.org/10.5120/ijais14-451090
  6. H. Hyungseob and U. Chong, "Neural Network Based Detection of Drowsiness with Eyes Open Using AR Modelling," IETE Technical Review, Vol. 33, pp. 518-524, 2016. https://doi.org/10.1080/02564602.2015.1118362
  7. K. Duand and M. Swamy, Particle Swarm Optimization, Springer International Publishing, Switzerland, 2016.
  8. M. Anantathanavitand and M. Munlin, "Using K-means Radius Particle Swarm Optimization for the Travelling Salesman Problem," IETE Technical Review, Vol. 33, Issue 2, pp. 172-180, 2016. https://doi.org/10.1080/02564602.2015.1057770
  9. Y. Shi, "Particle Swarm Optimization: Developments, Applications and Resources," Evolutionary Computation IEEE, Vol. 1, pp. 81-86, 2001.
  10. H. Tsai, Y. Tyan, Y. Wu, and Y. Lin, "Isolated Particle Swarm Optimization with Particle Migration and Global Best Adoption," Engineering Optimization, Vol. 44, Issue 12, pp. 1405-1424, 2012. https://doi.org/10.1080/0305215X.2012.654787
  11. J. Kennedy, Particle Swarm Optimization, Springer, Switzerland, 2011.
  12. Y. Shi, "Particle Swarm Optimization," IEEE Connections, Vol. 2, No.1, pp. 8-13, 2004.
  13. H. Han and U. Chong, "Neural Network Based Detection of Drowsiness with Eyes Open Using AR Modelling," IETE Technical Review, Vol. 33, Issue 5, pp. 518-524, 2016. https://doi.org/10.1080/02564602.2015.1118362
  14. M. Yaghini, M. Khoshraftar, and M. Fallahi, "A Hybrid Algorithm for Artificial Neural Network Training," Engineering Applications of Artificial Intelligence, Vol. 26, Issue 1, pp. 293-301, 2013. https://doi.org/10.1016/j.engappai.2012.01.023
  15. A. Askarzadehand and A. Rezazadeh, "Artificial Neural Network Training Using a New Efficient Optimization Algorithm," Applied Soft Computting, Vol. 13, Issue 2, pp. 1206-1213, 2013. https://doi.org/10.1016/j.asoc.2012.10.023
  16. M. Sayed, S. Gharghory, and H. Kamal, "Gain Tuning PI Controllers for Boiler Turbine Unit Using a New Hybridjump PSO," Journal of Electrical Systems and Information Technology, Vol 2, Issue 1, pp. 99-110, 2015. https://doi.org/10.1016/j.jesit.2015.03.009
  17. K. Sreejiniand and V. Govindan, "Improved Multiscale Matched Filter for Retina Vessel Segmentation Using PSO Algorithm," Egyptian Informatics Journal, Vol. 16, Issue 1, pp. 253-260, 2015. https://doi.org/10.1016/j.eij.2015.06.004
  18. T. Zhangand and X. You, "Improvement of the Training and Normalization Method of Artificial Neural Network in the Prediction of Indoor Environment," Proceeding of Engineering, Vol. 121, pp. 1245-1251, 2015. https://doi.org/10.1016/j.proeng.2015.09.152
  19. A. Suresh, K. Harish, and N. Radhika, "Particle Swarm Optimization over Back Propagation Neural Network," Proceeding of Computer Science, Vol. 46, pp. 268-275, 2015. https://doi.org/10.1016/j.procs.2015.02.020
  20. S. Mahapatra, R. Daniel, D. Dey, and S. Nayak, "Induction Motor Control Using PSO-ANFIS," Proceeding of Computer Science, Vol. 48, pp. 753-768, 2015. https://doi.org/10.1016/j.procs.2015.04.212
  21. A. Sahuand and S. Pattnaik, "Evolving Neuro Structure Using Adaptive PSO and Modified TLBO for Classification," Proceeding of Computer Science, Vol. 92, pp. 450-454, 2016. https://doi.org/10.1016/j.procs.2016.07.366
  22. C. Rajeswari, B. Sathiyabhama, S. Devendiran, and K. Manivannan, "Bearing Fault Diagnosis Using Wavelet Packet Transform, Hybrid PSO and Support Vector Machine," Proceeding of Engineering, Vol. 97, pp. 1772-1783, 2014. https://doi.org/10.1016/j.proeng.2014.12.329
  23. K. Anand, B. Barik, K. Tamilmannan, and P. Sathiya, "Artificial Neural Network Modeling Studies to Predict the Friction Welding Process Parameters of Incoloy 800H Joints," Engineering Science and Technology, an International Journal, Vol. 18, Issue 3, pp. 394-407, 2015. https://doi.org/10.1016/j.jestch.2015.02.001
  24. M. Khajehand and A. Barkhordar, "Modelling of Solid-phase Tea Waste Extraction for the Removal of Manganese from Food Samples by Using Artificial Neural Network Approach," Food Chemistry, Vol. 141, Issue 2, pp. 712-717, 2013. https://doi.org/10.1016/j.foodchem.2013.04.075
  25. S. Wang, Artificial Neural Network, Springer pp. 81-100, Switzerland, 2003.
  26. J. Cohen, P. Cohen, S. West, and L. Aiken, Applied Multiple Regression/Correlation Analysis for the Behavioral Sciences, Routledge, United Kingdom, Vol. 17, 2013.
  27. H. Detteand and V. Melas, “Optimal Designs for Estimating Individual Coefficients in Fourier Regression Models,” The Annals of Statistics, Vol. 31, No. 5, pp. 1669-1692, 2003. https://doi.org/10.1214/aos/1065705122
  28. S. Ozdemir, M. Peng, and Y. Xiao, "Polynomial Regression‐Based Privacy‐Preserving Data Aggregation for Wireless Sensor Networks," Wireless Communications and Mobile Computing, Vol. 15, Issue 4, pp. 615-628, 2015. https://doi.org/10.1002/wcm.2369
  29. C. Gu, Smoothing Spline ANOVA Models, Springer Science and Business Media, Vol. 297, Springer, Switzerland, 2013.
  30. R. Cookand and S. Weisberg, Residuals and Influence in Regression, Chapman and Hall, New York, 1982.