Journal of Korea Water Resources Association (한국수자원학회논문집)

Korea Water Resources Association (한국수자원학회)
권호 표지
DOI QR코드

DOI QR Code

Development of new artificial neural network optimizer to improve water quality index prediction performance

수질 지수 예측성능 향상을 위한 새로운 인공신경망 옵티마이저의 개발

  • Ryu, Yong Min (Department of Civil Engineering, Chungbuk National University) ;
  • Kim, Young Nam (Department of Civil Engineering, Chungbuk National University) ;
  • Lee, Dae Won (Department of Civil Engineering, Chungbuk National University) ;
  • Lee, Eui Hoon (School of Civil Engineering, Chungbuk National University)
  • 류용민 (충북대학교 토목공학과) ;
  • 김영남 (충북대학교 토목공학과) ;
  • 이대원 (충북대학교 토목공학과) ;
  • 이의훈 (충북대학교 토목공학부)
  • Received : 2023.12.19
  • Accepted : 2024.01.22
  • Published : 2024.02.29
Download PDF
⟨ Previous Next ⟩

Abstract

Predicting water quality of rivers and reservoirs is necessary for the management of water resources. Artificial Neural Networks (ANNs) have been used in many studies to predict water quality with high accuracy. Previous studies have used Gradient Descent (GD)-based optimizers as an optimizer, an operator of ANN that searches parameters. However, GD-based optimizers have the disadvantages of the possibility of local optimal convergence and absence of a solution storage and comparison structure. This study developed improved optimizers to overcome the disadvantages of GD-based optimizers. Proposed optimizers are optimizers that combine adaptive moments (Adam) and Nesterov-accelerated adaptive moments (Nadam), which have low learning errors among GD-based optimizers, with Harmony Search (HS) or Novel Self-adaptive Harmony Search (NSHS). To evaluate the performance of Long Short-Term Memory (LSTM) using improved optimizers, the water quality data from the Dasan water quality monitoring station were used for training and prediction. Comparing the learning results, Mean Squared Error (MSE) of LSTM using Nadam combined with NSHS (NadamNSHS) was the lowest at 0.002921. In addition, the prediction rankings according to MSE and R2 for the four water quality indices for each optimizer were compared. Comparing the average of ranking for each optimizer, it was confirmed that LSTM using NadamNSHS was the highest at 2.25.

하천과 저수지의 수질을 예측하는 것은 수자원관리를 위해 필요하다. 높은 정확도의 수질 예측을 위해 많은 연구들에서 인공신경망이 활용되었다. 기존 연구들은 매개변수를 탐색하는 인공신경망의 연산자인 옵티마이저로 경사하강법 기반 옵티마이저를 사용하였다. 그러나 경사하강법 기반 옵티마이저는 지역 최적값으로의 수렴 가능성과 해의 저장 및 비교구조가 없다는 단점이 있다. 본 연구에서는 인공신경망을 이용한 수질 예측성능을 향상시키기 위해 개량형 옵티마이저를 개발하여 경사하강법 기반 옵티마이저의 단점을 개선하였다. 본 연구에서 제안한 옵티마이저는 경사하강법 기반 옵티마이저 중 학습오차가 낮은 Adaptive moments (Adam)과 Nesterov-accelerated adaptive moments (Nadam)를 Harmony Search(HS) 또는 Novel Self-adaptive Harmony Search (NSHS)와 결합한 옵티마이저이다. 개량형 옵티마이저의 학습 및 예측성능 평가를 위해 개량형 옵티마이저를 Long Short-Term Memory (LSTM)에 적용하여 국내의 다산 수질관측소의 수질인자인 수온, 용존산소량, 수소이온농도 및 엽록소-a를 학습 및 예측하였다. 학습결과를 비교하면, Nadam combined with NSHS (NadamNSHS)를 사용한 LSTM의 Mean Squared Error (MSE)가 0.002921로 가장 낮았다. 또한, 각 옵티마이저별 4개 수질인자에 대한 MSE 및 R2에 따른 예측순위를 비교하였다. 각 옵티마이저의 평균 순위를 비교하면, NadamNSHS를 사용한 LSTM이 2.25로 가장 높은 것을 확인하였다.

Keywords

Acknowledgement

이 논문은 충북대학교 국립대학육성사업(2023)지원을 받아 작성되었음

References

  1. Akkoyunlu, A., and Akiner, M.E. (2010). "Feasibility assessment of data-driven models in predicting pollution trends of Omerli Lake, Turkey." Water Resources Management, Vol. 24, No. 13, pp. 3419-3436. https://doi.org/10.1007/s11269-010-9613-0
  2. Babu, C.N., and Reddy, B.E. (2014). "A moving-average filter based hybrid ARIMA-ANN model for forecasting time series data." Applied Soft Computing, Vol. 23, pp. 27-38. https://doi.org/10.1016/j.asoc.2014.05.028
  3. Cai, Q., Zhang, D., Zheng, W., and Leung, S.C. (2015). "A new fuzzy time series forecasting model combined with ant colony optimization and auto-regression." Knowledge-Based Systems, Vol. 74, pp. 61-68. https://doi.org/10.1016/j.knosys.2014.11.003
  4. Cao, X., Liu, H., and Chen, N. (1997). "Classification of Cm I energy levels using PCA-BPN and PCA-NLM." Chemical Physics, Vol. 220, No. 3, pp. 289-297. https://doi.org/10.1016/S0301-0104(97)00139-0
  5. Chen, Y., Song, L., Liu, Y., Yang, L., and Li, D. (2020). "A review of the artificial neural network models for water quality prediction." Applied Sciences, Vol. 10, No. 17, 5776.
  6. Choi, Y.H., Eghdami, S., Ngo, T.T., Chaurasia, S.N., and Kim, J.H. (2019). "Comparison of parameter-setting-free and self-adaptive harmony search." Proceedings Harmony Search and Nature Inspired Optimization Algorithms, Singapore, pp. 105-112.
  7. Dogan, E., Ates, A., Yilmaz, C., and Eren, B. (2008). "Application of artificial neural networks to estimate wastewater treatment plant inlet biochemical oxygen demand." Environmental Progress, Vol. 27, No. 4, pp.439-446. https://doi.org/10.1002/ep.10295
  8. Faruk, D.O. (2010). "A hybrid neural network and ARIMA model for water quality time series prediction." Engineering Applications of Artificial Intelligence, Vol. 23, No. 4, pp. 586-594. https://doi.org/10.1016/j.engappai.2009.09.015
  9. Folorunso, T.A., Aibinu, A.M., Kolo, J.G., Sadiku, S.O., and Orire, A.M. (2018). "Effects of data normalization on water quality model in a recirculatory aquaculture system using artificial neural network." I-manager's Journal on Pattern Recognition, Vol. 5, No. 3, 21.
  10. Geem, Z.W. (2006). "Optimal cost design of water distribution networks using harmony search." Engineering Optimization, Vol. 38, No. 3, pp. 259-277. https://doi.org/10.1080/03052150500467430
  11. Geem, Z.W., Kim, J.H., and Loganathan, G.V. (2001). "A new heuristic optimization algorithm: Harmony search." Simulation, Vol. 76, No. 2, pp. 60-68. https://doi.org/10.1177/003754970107600201
  12. Hasan, H., and Tahir, N.M. (2010). "Feature selection of breast cancer based on principal component analysis." In 2010 6th International Colloquium on Signal Processing & its Applications, Malacca, Malaysia, pp. 1-4.
  13. Hochreiter, S., and Schmidhuber, J. (1997). "Long short-term memory." Neural Computation, Vol. 9, No. 8, pp. 1735-1780. https://doi.org/10.1162/neco.1997.9.8.1735
  14. Jin, L., Kuang, X., Huang, H., Qin, Z., and Wang, Y. (2005). "Study on the overfitting of the artificial neural network forecasting model." Acta Meteorologica Sinica, Vol. 19, No. 2, 216.
  15. Joo, D.S., Choi, D.J., and Park, H. (2000). "The effects of data preprocessing in the determination of coagulant dosing rate." Water Research, Vol. 34, No. 13, pp. 3295-3302. https://doi.org/10.1016/S0043-1354(00)00067-1
  16. Joo, G., Park, C., and Im, H. (2020). "Performance evaluation of machine learning optimizers." Journal of Institute of Korean Electrical and Engineers, Vol. 24, No, 3, pp. 766-776.
  17. Khatri, P., Gupta, K.K., and Gupta, R.K. (2021). "Drift compensation of commercial water quality sensors using machine learning to extend the calibration lifetime." Journal of Ambient Intelligence and Humanized Computing, Vol. 12, pp. 3091-3099. https://doi.org/10.1007/s12652-020-02469-y
  18. Lee, W.J., and Lee, E.H. (2022a). "Runoff prediction based on the discharge of pump stations in an urban stream using a modified multi-layer perceptron combined with meta-heuristic optimization." Water, Vol. 14, No. 1, 99.
  19. Lee, W.J., and Lee, E.H. (2022b). "Improvement of multi layer perceptron performance using combination of gradient descent and harmony search for prediction of ground water level." Journal of Korea Water Resources Association, Vol. 55, No. 11, pp. 903-911. https://doi.org/10.3741/JKWRA.2022.55.11.903
  20. Li, L., Jiang, P., Xu, H., Lin, G., Guo, D., and Wu, H. (2019). "Water quality prediction based on recurrent neural network and improved evidence theory: a case study of Qiantang River, China." Environmental Science and Pollution Research, Vol. 26, No. 19, pp. 19879-19896. https://doi.org/10.1007/s11356-019-05116-y
  21. Lu, H., and Ma, X. (2020). "Hybrid decision tree-based machine learning models for short-term water quality prediction." Chemosphere, Vol. 249, 126169.
  22. Luo, K. (2013). "A novel self-adaptive harmony search algorithm." Journal of Applied Mathematics, Vol. 2013, pp. 1-16. https://doi.org/10.1155/2013/653749
  23. Manjarres, D., Landa-Torres, I., Gil-Lopez, S., Del Ser, J., Bilbao, M.N., Salcedo-Sanz, S., and Geem, Z.W. (2013). "A survey on applications of the harmony search algorithm." Engineering Applications of Artificial Intelligence, Vol. 26, No. 8, pp. 1818-1831. https://doi.org/10.1016/j.engappai.2013.05.008
  24. McCulloch, W.S., and Pitts, W. (1943). "A logical calculus of the ideas immanent in nervous activity." The Bulletin of Mathematical Biophysics, Vol. 5, No. 4, pp. 115-133. https://doi.org/10.1007/BF02478259
  25. Mok, J.Y., Choi, J.H., and Moon, Y.I. (2020). "Prediction of multipurpose dam inflow using deep learning." Journal of Korea Water Resources Association, Vol. 53, No. 2, pp. 97-105.
  26. Nawi, N.M., Atomi, W.H., and Rehman, M.Z. (2013). "The effect of data pre-processing on optimized training of artificial neural networks." Procedia Technology, Vol. 11, pp. 32-39. https://doi.org/10.1016/j.protcy.2013.12.159
  27. Pan, Q.K., Suganthan, P.N., Tasgetiren, M.F., and Liang, J.J. (2010). "A self-adaptive global best harmony search algorithm for continuous optimization problems." Applied Mathematics and Computation, Vol. 216, pp. 830-848. https://doi.org/10.1016/j.amc.2010.01.088
  28. Park, S.Y., Choi, J.H., Wang, S., and Park, S.S. (2006). "Design of a water quality monitoring network in a large river system using the genetic algorithm." Ecological modelling, Vol. 199, No. 3, pp. 289-297. https://doi.org/10.1016/j.ecolmodel.2006.06.002
  29. Rosenblatt, F. (1958). "The perceptron: A probabilistic model for information storage and organization in the brain." Psychological Review, Vol. 65, No. 6, 386.
  30. Rumelhart, D.E., Hinton, G.E., and Williams, R.J. (1986). "Learning representations by back-propagating errors." Nature, Vol. 323, No. 6088, pp. 533-536. https://doi.org/10.1038/323533a0
  31. Ryu, Y.M., and Lee, E.H. (2022). "Application of neural networks to predict Daecheong Dam water levels." Journal of the Korean Society of Hazard Mitigation, Vol. 22, No. 1, pp. 67-78. https://doi.org/10.9798/KOSHAM.2022.22.1.67
  32. Sedki, A., Ouazar, D., and El Mazoudi, E. (2009). "Evolving neural network using real coded genetic algorithm for daily rainfall-runoff forecasting." Expert Systems with Applications, Vol. 36, No. 3, pp. 4523-4527. https://doi.org/10.1016/j.eswa.2008.05.024
  33. Wang, T.S., Tan, C.H., Chen, L., and Tsai, Y.C. (2008). "Applying artificial neural networks and remote sensing to estimate chlorophyll-a concentration in water body." Proceedings of the 2008 2nd International Symposium Intelligent Information Technology Application IITA, Shanghai, China, pp. 540-544.
  34. Wang, Y., Zhou, J., Chen, K., Wang, Y., and Liu, L. (2017). "Water quality prediction method based on LSTM neural network." Proceedings 2017 12th International Conference on Intelligent Systems and Knowledge Engineering, Nanjing, China, pp. 1-5.
  35. Xiang, Y., and Jiang, L. (2009). "Water quality prediction using LSSVM and particle swarm optimization." Proceedings 2009 Second International Workshop on Knowledge Discovery and Data Mining, Moscow, Russia, pp. 900-904.
  36. Zare, A., Bayat, V., and Daneshkare, A. (2011). "Forecasting nitrate concentration in groundwater using artificial neural network and linear regression models." International Agrophysics, Vol. 25, No. 2, pp. 187-192.
  37. Zhang, L., Zou, Z., and Shan, W. (2017). "Development of a method for comprehensive water quality forecasting and its application in Miyun reservoir of Beijing, China." Journal of Environmental Sciences, Vol. 56, pp. 240-246. https://doi.org/10.1016/j.jes.2016.07.017
  38. Zheng, L., Wang, H., Liu, C., Zhang, S., Ding, A., Xie, E., Li, J., and Wang, S. (2021). "Prediction of harmful algal blooms in large water bodies using the combined EFDC and LSTM models." Journal of Environmental Management, Vol. 295, 113060.