Journal of Korea Water Resources Association (한국수자원학회논문집)

Korea Water Resources Association (한국수자원학회)
권호 표지
DOI QR코드

DOI QR Code

Improvement of multi layer perceptron performance using combination of gradient descent and harmony search for prediction of ground water level

지하수위 예측을 위한 경사하강법과 화음탐색법의 결합을 이용한 다층퍼셉트론 성능향상

  • Lee, Won Jin (Department of Civil Engineering, Chungbuk National University) ;
  • Lee, Eui Hoon (School of Civil Engineering, Chungbuk National University)
  • 이원진 (충북대학교 토목공학과) ;
  • 이의훈 (충북대학교 토목공학부)
  • Received : 2022.08.26
  • Accepted : 2022.10.13
  • Published : 2022.11.30
Download PDF
⟨ Previous Next ⟩

Abstract

Groundwater, one of the resources for supplying water, fluctuates in water level due to various natural factors. Recently, research has been conducted to predict fluctuations in groundwater levels using Artificial Neural Network (ANN). Previously, among operators in ANN, Gradient Descent (GD)-based Optimizers were used as Optimizer that affect learning. GD-based Optimizers have disadvantages of initial correlation dependence and absence of solution comparison and storage structure. This study developed Gradient Descent combined with Harmony Search (GDHS), a new Optimizer that combined GD and Harmony Search (HS) to improve the shortcomings of GD-based Optimizers. To evaluate the performance of GDHS, groundwater level at Icheon Yullhyeon observation station were learned and predicted using Multi Layer Perceptron (MLP). Mean Squared Error (MSE) and Mean Absolute Error (MAE) were used to compare the performance of MLP using GD and GDHS. Comparing the learning results, GDHS had lower maximum, minimum, average and Standard Deviation (SD) of MSE than GD. Comparing the prediction results, GDHS was evaluated to have a lower error in all of the evaluation index than GD.

물을 공급하기 위한 자원 중 하나인 지하수는 다양한 자연적 요인에 의해 수위의 변동이 발생한다. 최근, 인공신경망을 이용하여 지하수위의 변동을 예측하는 연구가 진행되었다. 기존에는 인공신경망 연산자 중 학습에 영향을 미치는 Optimizer로 경사하강법(Gradient Descent, GD) 기반 Optimizer를 사용하였다. GD 기반 Optimizer는 초기 상관관계 의존성과 해의 비교 및 저장 구조 부재의 단점이 존재한다. 본 연구는 GD 기반 Optimizer의 단점을 개선하기 위해 GD와 화음탐색법(Harmony Search, HS)를 결합한 새로운 Optimizer인 Gradient Descent combined with Harmony Search(GDHS)를 개발하였다. GDHS의 성능을 평가하기 위해 다층퍼셉트론(Multi Layer Perceptron, MLP)을 이용하여 이천율현 관측소의 지하수위를 학습 및 예측하였다. GD 및 GDHS를 사용한 MLP의 성능을 비교하기 위해 Mean Squared Error(MSE) 및 Mean Absolute Error(MAE)를 사용하였다. 학습결과를 비교하면, GDHS는 GD보다 MSE의 최대값, 최소값, 평균값 및 표준편차가 작았다. 예측결과를 비교하면, GDHS는 GD보다 모든 평가지표에서 오차가 작은 것으로 평가되었다.

Keywords

Acknowledgement

본 연구는 2022년도 정부(교육부)의 재원으로 한국연구재단의 지원을 받아 수행된 연구입니다. 이에 감사드립니다(NRF-2019R1I1A3A01059929).

References

  1. Affandia, AK., Watanabe, K., and Tirtomihardjo, H. (2007). "Application of an artificial neural network to estimate groundwater level fluctuation." Journal of Spatial Hydrology, Vol. 7, No. 2, pp. 17-32.
  2. Aqil, M., Kita, I., Yano, A., and Nishiyama, S. (2007). "A comparative study of artificial neural networks and neuro-fuzzy in continuous modeling of the daily and hourly behaviour of runoff." Journal of Hydrology, Vol. 337, No. 1-2, pp. 22-34. https://doi.org/10.1016/j.jhydrol.2007.01.013
  3. Barthel, R., and Banzhaf, S. (2016). "Groundwater and surface water interaction at the regional-scale - a review with focus on regional integrated models." Water Resources Management, Vol. 30, No. 1, pp. 1-32. https://doi.org/10.1007/s11269-015-1163-z
  4. Batelaan, O., De Smedt, F., and Triest, L. (2003). "Regional groundwater discharge: phreatophyte mapping, groundwater modelling and impact analysis of land-use change." Journal of Hydrology, Vol. 275, No. 1-2, pp. 86-108. https://doi.org/10.1016/S0022-1694(03)00018-0
  5. Daliakopoulos, I.N., Coulibaly, P., and Tsanis, I.K. (2005). "Ground water level forecasting using artificial neural networks." Journal of Hydrology, Vol. 309, No. 1-4, pp. 229-240. https://doi.org/10.1016/j.jhydrol.2004.12.001
  6. Derbela, M., and Nouiri, I. (2020). "Intelligent approach to predict future groundwater level based on artificial neural networks (ANN)." Euro-Mediterranean Journal for Environmental Integration, Vol. 5, No. 3, pp. 1-11. https://doi.org/10.1007/s41207-019-0135-8
  7. Ebrahimi, H., and Rajaee, T. (2017). "Simulation of groundwater level variations using wavelet combined with neural network, linear regression and support vector machine." Global and Planetary Change, Vol. 148, pp. 181-191. https://doi.org/10.1016/j.gloplacha.2016.11.014
  8. Geem, Z.W., Kim, J.H., and Loganathan, G.V. (2001). "A new heuristic optimization algorithm: harmony search." Simulation, Vol. 76, No. 2, pp. 60-68. https://doi.org/10.1177/003754970107600201
  9. Goldberg, D.E., and Holland, J.H. (1988). "Genetic algorithms and machine learning." Machine Learning, Vol. 3, pp. 95-99. https://doi.org/10.1023/A:1022602019183
  10. Joo, G., Park, C., and Im, H. (2020). "Performance evaluation of machine learning optimizers." Journal of Korean Electrical and Electronics Engineers, Vol. 24, No. 3, pp. 766-776.
  11. Kennedy, J., and Eberhart, R. (1995). "Particle swarm optimization." Proceedings of the IEEE International Conference on Neural Networks, Indianapolis, IN, Vol. 4, pp. 1942-1948.
  12. Kim, I., and Lee, J. (2018). "Prediction model for spatial and temporal variation of groundwater level based on river stage." Journal of Hydrologic Engineering, Vol. 23, No. 6, 06018002. https://doi.org/10.1061/(ASCE)HE.1943-5584.0001658
  13. Kim, I., Lee, J., Kim, J., Lee, H., and Lee, J. (2021). "Analysis of groundwater level prediction performance with influencing factors by artificial neural network." Journal of the Korean Geotechnical Society, Vol. 37, No. 5, pp. 19-31. https://doi.org/10.7843/KGS.2021.37.5.19
  14. Kim, Y.N., and Lee, E.H. (2020). "Development of the meta-heuristic optimization algorithm: Exponential bandwidth harmony search with centralized global search." Journal of the Korea AcademiaIndustrial cooperation Society, Vol. 21, No. 2, pp. 8-18.
  15. Knotters, M., and Bierkens, M.F. (2000). "Physical basis of time series models for water table depths." Water Resources Research, Vol. 36, No. 1, pp. 181-188. https://doi.org/10.1029/1999WR900288
  16. Liu, Q., Jian, W., and Nie, W. (2021). "Rainstorm-induced landslides early warning system in mountainous cities based on groundwater level change fast prediction." Sustainable Cities and Society, Vol. 69, 102817. https://doi.org/10.1016/j.scs.2021.102817
  17. Mahdavi, M., Fesanghary, M., and Damangir, E. (2007). "An improved harmony search algorithm for solving optimization problems." Applied mathematics and computation, Vol. 188, No. 2, pp. 1567-1579. https://doi.org/10.1016/j.amc.2006.11.033
  18. Maxwell, R.M., Condon, L.E., and Kollet, S.J. (2015). "A highresolution simulation of groundwater and surface water over most of the continental US with the integrated hydrologic model ParFlow v3." Geoscientific Model Devel opment, Vol. 8, No. 3, pp. 923-937. https://doi.org/10.5194/gmd-8-923-2015
  19. McCulloch, W.S., and Pitts, W. (1943). "A logical calculus of the ideas immanent in nervous activity." The Bulletin of Mathematical Biophysics, Vol. 5, No. 4, pp. 115-133. https://doi.org/10.1007/BF02478259
  20. Ministry of Land, Infrastructure and Transport (MOLIT) (2011). Bok-stream basic plan (change) report, p. 34.
  21. Park, C., and Chung, I.M. (2020). "Evaluating the groundwater prediction using LSTM model." Journal of Korea Water Resources Association, Vol. 53, No. 4, pp. 273-283. https://doi.org/10.3741/JKWRA.2020.53.4.273
  22. Rosenblatt, F. (1958). "The perceptron: a probabilistic model for information storage and organization in the brain." Psychological Review, Vol. 65, No. 6, 386. https://doi.org/10.1037/h0042519
  23. Rumelhart, D.E., Hinton, G.E., and Williams, R.J. (1986). "Learning representations by back-propagating errors." Nature, Vol. 323, No. 6088, pp. 533-536. https://doi.org/10.1038/323533a0
  24. Ryu, Y.M., and Lee, E.H. (2022). "Application of neural networks to predict Daecheong Dam water levels." Journal of the Korean Society of Hazard Mitigation, Vol. 22, No. 1, pp. 67-78. https://doi.org/10.9798/KOSHAM.2022.22.1.67
  25. Sahoo, S., and Jha, M.K. (2013). "Groundwater-level prediction using multiple linear regression and artificial neural network techniques: a comparative assessment." Hydrogeology Journal, Vol. 21, No. 8, pp. 1865-1887. https://doi.org/10.1007/s10040-013-1029-5
  26. Sahoo, S., Russo, T.A., Elliott, J., and Foster, I. (2017). "Machine learning algorithms for modeling groundwater level changes in agricultural regions of the US." Water Resources Research, Vol. 53, No. 5, pp. 3878-3895. https://doi.org/10.1002/2016WR019933
  27. Sattari, M.T., Mirabbasi, R., Sushab, R.S., and Abraham, J. (2018). "Prediction of groundwater level in Ardebil plain using support vector regression and M5 tree model." Groundwater, Vol. 56, No. 4, pp. 636-646. https://doi.org/10.1111/gwat.12620
  28. Sedki, A., Ouazar, D., and El Mazoudi, E. (2009). "Evolving neural network using real coded genetic algorithm for daily rainfall - runoff forecasting." Expert Systems with Applications, Vol. 36, No. 3, pp. 4523-4527. https://doi.org/10.1016/j.eswa.2008.05.024
  29. Shin, M.J., Moon, S.H., Kang, K.G., Moon, D.C., and Koh, H.J. (2020). "Analysis of groundwater level variations caused by the changes in groundwater withdrawals using long short-term memory network." Hydrology, Vol. 7, No. 3, 64. https://doi.org/10.3390/hydrology7030064
  30. Suryanarayana, C., Sudheer, C., Mahammood, V., and Panigrahi, B.K. (2014). "An integrated wavelet-support vector machine for groundwater level prediction in Visakhapatnam, India." Neurocomputing, Vol. 145, pp. 324-335. https://doi.org/10.1016/j.neucom.2014.05.026
  31. Trichakis, I.C., Nikolos, I.K., and Karatzas, G.P. (2011). "Artificial neural network (ANN) based modeling for karstic groundwater level simulation." Water Resources Management, Vol. 25, No. 4, pp. 1143-1152. https://doi.org/10.1007/s11269-010-9628-6
  32. White, J.T., Knowling, M.J., and Moore, C.R. (2020). "Consequences of groundwater-model vertical discretization in risk-based decision-Making." Groundwater, Vol. 58, No. 5, pp. 695-709.
  33. Yoo, Y., Kim, D., and Lee, J. (2020). "Performance analysis of various activation functions in super resolution model." Proceedings of the Korea Information Processing Society Conference, KIPS, Vol. 27, No. 1, pp. 504-507.
  34. Yousefi, H., Zahedi, S., Niksokhan, M.H., and Momeni, M. (2019). "Ten-year prediction of groundwater level in Karaj plain (Iran) using MODFLOW2005-NWT in MATLAB." Environmental Earth Sciences, Vol. 78, No. 12, pp. 1-14. https://doi.org/10.1007/s12665-018-7995-0