Journal of Korea Water Resources Association (한국수자원학회논문집)

Korea Water Resources Association (한국수자원학회)
권호 표지
DOI QR코드

DOI QR Code

Improvement of multi layer perceptron performance using combination of adaptive moments and improved harmony search for prediction of Daecheong Dam inflow

대청댐 유입량 예측을 위한 Adaptive Moments와 Improved Harmony Search의 결합을 이용한 다층퍼셉트론 성능향상

  • Lee, Won Jin (Department of Civil Engineering, Chungbuk National University) ;
  • Lee, Eui Hoon (School of Civil Engineering, Chungbuk National University)
  • 이원진 (충북대학교 토목공학과) ;
  • 이의훈 (충북대학교 토목공학부)
  • Received : 2022.10.31
  • Accepted : 2022.11.29
  • Published : 2023.01.31
Download PDF
⟨ Previous Next ⟩

Abstract

High-reliability prediction of dam inflow is necessary for efficient dam operation. Recently, studies were conducted to predict the inflow of dams using Multi Layer Perceptron (MLP). Existing studies used the Gradient Descent (GD)-based optimizer as the optimizer among MLP operators to find the optimal correlation between data. However, the GD-based optimizers have disadvantages in that the prediction performance is deteriorated due to the possibility of convergence to the local optimal value and the absence of storage space. This study improved the shortcomings of the GD-based optimizer by developing Adaptive moments combined with Improved Harmony Search (AdamIHS), which combines Adaptive moments among GD-based optimizers and Improved Harmony Search (IHS). In order to evaluate the learning and prediction performance of MLP using AdamIHS, Daecheong Dam inflow was learned and predicted and compared with the learning and prediction performance of MLP using GD-based optimizer. Comparing the learning results, the Mean Squared Error (MSE) of MLP, which is 5 hidden layers using AdamIHS, was the lowest at 11,577. Comparing the prediction results, the average MSE of MLP, which is one hidden layer using AdamIHS, was the lowest at 413,262. Using AdamIHS developed in this study, it will be possible to show improved prediction performance in various fields.

높은 신뢰도의 댐 유입량 예측은 효율적인 댐 운영을 위해 필요하다. 최근 다층퍼셉트론(Multi Layer Perceptron, MLP)을 활용하여 댐의 유입량을 예측하는 연구들이 진행되었다. 기존 연구들은 MLP의 연산자 중 자료 간의 최적 상관관계를 찾는 optimizer로 경사하강법(Gradient Descent, GD) 기반의 optimizer를 사용하였다. 하지만, GD 기반의 optimizer들은 지역 최적값으로의 수렴 가능성과 저장공간 부재로 인해 예측성능이 저하된다는 단점이 있다. 본 연구는 GD 기반 optimizer 중 Adaptive moments와 Improved Harmony Search (IHS)를 결합한 Adaptive moments combined with Improved Harmony Search (AdamIHS)를 개발하여 GD 기반 optimizer의 단점을 개선하였다. AdamIHS를 사용한 MLP의 학습 및 예측성능을 평가하기 위해 대청댐 유입량을 학습 및 예측하였으며, GD 기반 optimizer를 사용한 MLP의 학습 및 예측성능과 비교하였다. 학습결과를 비교하면, AdamIHS를 사용한 은닉층 5개인 MLP의 Mean Squared Error (MSE) 평균값이 11,577로 가장 낮았다. 예측결과를 비교하면, AdamIHS를 사용한 은닉층 1개인 MLP의 MSE 평균값이 413,262로 가장 낮았다. 본 연구에서 개발된 AdamIHS를 활용하면 다양한 분야에서 향상된 예측성능을 보여줄 수 있을 것이다.

Keywords

Acknowledgement

이 논문은 충북대학교 국립대학육성사업(2022)지원을 받아 작성되었음

References

  1. Agatonovic-Kustrin, S., and Beresford, R. (2000). "Basic concepts of artificial neural network (ANN) modeling and its application in pharmaceutical research." Journal of Pharmaceutical and Biomedical Analysis, Vol. 22, No. 5, pp. 717-727. https://doi.org/10.1016/S0731-7085(99)00272-1
  2. Amjady, N., and Keynia, F. (2011). "A new neural network approach to short term load forecasting of electrical power systems." Energies, Vol. 4, No. 3, pp. 488-503. https://doi.org/10.3390/en4030488
  3. Aqil, M., Kita, I., Yano, A., and Nishiyama, S. (2007). "A comparative study of artificial neural networks and neuro-fuzzy in continuous modeling of the daily and hourly behaviour of runoff." Journal of Hydrology, Vol. 337, No. 1-2, pp. 22-34. https://doi.org/10.1016/j.jhydrol.2007.01.013
  4. Babaei, M., Moeini, R., and Ehsanzadeh, E. (2019). "Artificial neural network and support vector machine models for inflow prediction of dam reservoir (case study: Zayandehroud dam reservoir)." Water Resources Management, Vol. 33, No. 6, pp. 2203-2218. https://doi.org/10.1007/s11269-019-02252-5
  5. Cho, Y.T., Lee, G.H., Hong, J.H., and Geem, G.W. (2022). "Prediction of photovoltaic generation using machine learning models with various weight optimization techniques." Journal of Korean Institute of Intelligent Systems, Vol. 32, No. 1, pp. 1-6. https://doi.org/10.5391/JKIIS.2022.32.1.1
  6. Choi, K. (2016). "Real-time artificial neural network for highdimensional medical image." The Korean Society of Radiology, Vol. 10, No. 8, pp. 637-643. https://doi.org/10.7742/jksr.2016.10.8.637
  7. Dorigo, M., Birattari, M., and Stutzle, T. (2006). "Ant colony optimization." IEEE Computational Intelligence Magazine, Vol. 1, No. 4, pp. 28-39. https://doi.org/10.1109/MCI.2006.329691
  8. Eom, J., and Jung, K. (2019). "Estimation of hourly dam inflow using time series data." Journal of the Korean Society of Hazard Mitigation, Vol. 19, No. 2, pp. 163-168. https://doi.org/10.9798/kosham.2019.19.2.163
  9. Geem, Z.W., Kim, J.H., and Loganathan, G.V. (2001). "A new heuristic optimization algorithm: harmony search." Simulation, Vol. 76, No. 2, pp. 60-68. https://doi.org/10.1177/003754970107600201
  10. Geem, Z.W., Kim, J.H., and Loganathan, G.V. (2002). "Application of harmony search algorithm to water resources problems." Conference of the Environmental and Water Resources Institute of ASCE, ASCE, Roanoke, VA, U.S.
  11. Gocken, M., Ozcalici, M., Boru, A., and Dosdogru, A.T. (2016). "Integrating metaheuristics and artificial neural networks for improved stock price prediction." Expert Systems with Applications, Vol. 44, pp. 320-331. https://doi.org/10.1016/j.eswa.2015.09.029
  12. Goldberg, D.E., and Holland, J.H. (1988). "Genetic algorithms and machine learning." Machine Learning, Vol. 3, pp. 95-99. https://doi.org/10.1023/a:1022602019183
  13. Granata, F., Gargano, R., and De Marinis, G. (2016). "Support vector regression for rainfall-runoff modeling in urban drainage: A comparison with the EPA's storm water management model." Water, Vol. 8, No. 3, 69.
  14. Han, H., Choi, C., Jung, J., and Kim, H.S. (2021). "Application of sequence to sequence learning based LSTM model (LSTM-s2s) for forecasting dam inflow." Journal of Korea Water Resources Association, Vol. 54, No. 3, pp. 157-166. https://doi.org/10.3741/JKWRA.2021.54.3.157
  15. Jeon, G., Park, J., Jung, J., and Yoon, H. (2021). "Structural response estimation using gated recurrent unit." Korean Society of Hazard Mitigation, Vol. 21, pp. 171-179. https://doi.org/10.9798/KOSHAM.2021.21.3.171
  16. Joo, G., Park, C., and Im, H. (2020). "Performance evaluation of machine learning optimizers." Journal of IKEEE, Vol. 24, No. 3, pp. 766-776. https://doi.org/10.7471/IKEEE.2020.24.3.766
  17. Jung, S., Lee, D., and Lee, K. (2018). "Prediction of river water level using deep-learning open library." Journal of Korean Society of Hazard Mitigation, Vol. 18, No. 1, pp. 1-11. https://doi.org/10.9798/KOSHAM.2018.18.1.1
  18. Kennedy, J., and Eberhart, R. (1995). "Particle swarm optimization." Proceedings of the IEEE International Conference on Neural Networks, Vol. 4, pp.1942-1948.
  19. Kim, D., Kim, Y., Jang, D., and Baek, C. (2019). "A study on the estimation of CSVR for flood damage analysis." Journal of the Korean Society Hazard Mitigation, Vol. 19, No. 6, pp. 303-311. https://doi.org/10.9798/kosham.2019.19.6.303
  20. Kim, L.S. (2004). "Firm bankruptcy prediction using artificial neural network models." The East-West Economics Association of Korea, No. 16, pp. 65-80.
  21. Lee, A., Geem, Z.W., and Suh, K.D. (2016a). "Determination of optimal initial weights of an artificial neural network by using the harmony search algorithm: application to breakwater armor stones." Applied Sciences, Vol. 6, No. 6, 164.
  22. Lee, E.H., Yoo, D.G., Choi, Y.H., and Kim, J.H. (2016b). "Development of the new meta-heuristic optimization algorithm inspired by a vision correction procedure: Vision correction algorithm," Journal of the Korea Academia-Industrial, Vol. 17, No. 3, pp. 117-126. https://doi.org/10.5762/KAIS.2016.17.3.117
  23. Lee, J., Sun, Y.G., Lee, S.M., Kim, S.H., Kim, Y., Lee, W., Sim, I., and Kim, J.Y. (2019). "Implementation of smart meter applying power consumption prediction based on GRU model." The Journal of The Institute of Internet, Broadcasting and Communication, Vol. 19, No. 5, pp. 93-99. https://doi.org/10.7236/JIIBC.2019.19.5.93
  24. Lee, W.Y., Ko, K.E., Geem, Z.W., and Sim, K.B. (2017). "Method that determining the Hyperparameter of CNN using HS algorithm." Journal of the Korean Institute of Intelligent Systems, Vol. 27, No. 1, pp. 22-28. https://doi.org/10.5391/JKIIS.2017.27.1.022
  25. Mahdavi, M., Fesanghary, M., and Damangir, E. (2007). "An improved harmony search algorithm for solving optimization problems." Applied Mathematics and Computation, Vol. 188, No. 2, pp. 1567-1579. https://doi.org/10.1016/j.amc.2006.11.033
  26. Mahsa, M., and Lee, T. (2018). "Comparison of optimization algorithms in deep learning-based neural networks for hydrological forecasting: Case study of Nam River daily runoff." Journal of the Korean Society of Hazard Mitigation, Vol. 18, No. 6, pp. 377-384. https://doi.org/10.9798/kosham.2018.18.6.377
  27. McCulloch, W.S., and Pitts, W. (1943). "A logical calculus of the ideas immanent in nervous activity." The Bulletin of Mathematical Biophysics, Vol. 5, No. 4, pp. 115-133. https://doi.org/10.1007/BF02478259
  28. Mok, J.Y., Choi, J.H., and Moon, Y.I. (2020). "Prediction of multipurpose dam inflow using deep learning." Journal of Korea Water Resources Association, Vol. 53, No. 2, pp. 97-105. https://doi.org/10.3741/JKWRA.2020.53.2.97
  29. Moon, B.S., and Lee, K.J. (2011). "Prediction of the inflow in Seum-jin Dam according to rainfall." Journal of the Korea Entertainment Industry Association, Vol. 5, No. 2, pp. 104-111. https://doi.org/10.21184/jkeia.2011.06.5.2.104
  30. Park, C., and Chung, I.M. (2020). "Evaluating the groundwater prediction using LSTM model." Journal of Korea Water Resources Association, Vol. 53, No. 4, pp. 273-283. https://doi.org/10.3741/JKWRA.2020.53.4.273
  31. Park, M.K., Yoon, Y.S., Lee, H.H., and Kim, J.H. (2018). "Application of recurrent neural network for inflow prediction into multi-purpose dam basin." Journal of Korea Water Resources Association, Vol. 51, No. 12, pp. 1217-1227. https://doi.org/10.3741/JKWRA.2018.51.12.1217
  32. Riad, S., Mania, J., Bouchaou, L., and Najjar, Y. (2004). "Predicting catchment flow in a semi-arid region via an artificial neural network technique." Hydrological Processes, Vol. 18, No. 13, pp. 2387-2393. https://doi.org/10.1002/hyp.1469
  33. Rosenblatt, F. (1958). "The perceptron: a probabilistic model for information storage and organization in the brain." Psychological Review, Vol. 65, No. 6, 386.
  34. Rumelhart, D.E., Hinton, G.E., and Williams, R.J. (1986). "Learning representations by back-propagating errors." Nature, Vol. 323, No. 6088, pp. 533-536. https://doi.org/10.1038/323533a0
  35. Ryu, Y.M., and Lee, E.H. (2022). "Application of neural networks to predict Daecheong Dam water levels." Journal of the Korean Society of Hazard Mitigation, Vol. 22, No. 1, pp. 67-78. https://doi.org/10.9798/KOSHAM.2022.22.1.67
  36. Sedki, A., Ouazar, D., and El Mazoudi, E. (2009). "Evolving neural network using real coded genetic algorithm for daily rainfall - runoff forecasting." Expert Systems with Applications, Vol. 36, No. 3, pp. 4523-4527. https://doi.org/10.1016/j.eswa.2008.05.024
  37. Seo, Y., Kim, S., Kisi, O., and Singh, V.P. (2015). "Daily water level forecasting using wavelet decomposition and artificial intelligence techniques." Journal of Hydrology, Vol. 520, pp. 224-243. https://doi.org/10.1016/j.jhydrol.2014.11.050
  38. Valipour, M., Banihabib, M.E., and Behbahani, S.M.R. (2012). "Parameters estimate of autogressive moving average and autogressive integrated moving average models and compare their ability for inflow forecasting." Journal of Mathematics and Statistics, Vol. 8, No. 3, pp. 330-338. https://doi.org/10.3844/jmssp.2012.330.338
  39. Valipour, M., Banihabib, M.E., and Behbahani, S.M.R. (2013). "Comparison of the ARMA, ARIMA, and the autogressive artificial neural network models in forecasting the monthly inflow of Dez dam resorvoir." Journal of Hydrology, Vol. 479, pp. 433-441. https://doi.org/10.1016/j.jhydrol.2012.11.017
  40. Yoo, Y., Kim, D., and Lee, J. (2020). "Performance analysis of various activation functions in super resolution model." Proceedings of the Korea Information Processing Society Conference, Vol. 27, No. 1, pp. 504-507.
  41. Zhang, D., Lindholm, G., and Ratnaweera, H. (2018). "Use long short-term memory to enhance Internet of Things for combined sewer overflow monitoring." Journal of hydrology, Vol. 556, pp. 409-418. https://doi.org/10.1016/j.jhydrol.2017.11.018
  42. Zhou, Z., Li, Z., Snowling, S., Baetz, B.W., and Na, D. (2019). "A random forest model for inflow prediction at wastewater treatment plants." Stochastic Environmental Research and Risk Assessment, Vol. 33, No. 10, pp. 1781-1792. https://doi.org/10.1007/s00477-019-01732-9