Journal of Korea Water Resources Association (한국수자원학회논문집)

Korea Water Resources Association (한국수자원학회)
권호 표지
DOI QR코드

DOI QR Code

Prediction of dam inflow based on LSTM-s2s model using luong attention

Attention 기법을 적용한 LSTM-s2s 모델 기반 댐유입량 예측 연구

  • Lee, Jonghyeok (Department of Civil and Environmental Engineering, Yonsei University) ;
  • Choi, Suyeon (Department of Civil and Environmental Engineering, Yonsei University) ;
  • Kim, Yeonjoo (Department of Civil and Environmental Engineering, Yonsei University)
  • 이종혁 (연세대학교 건설환경공학과) ;
  • 최수연 (연세대학교 건설환경공학과) ;
  • 김연주 (연세대학교 건설환경공학과)
  • Received : 2022.04.12
  • Accepted : 2022.05.30
  • Published : 2022.07.31
Download PDF
⟨ Previous Next ⟩

Abstract

With the recent development of artificial intelligence, a Long Short-Term Memory (LSTM) model that is efficient with time-series analysis is being used to increase the accuracy of predicting the inflow of dams. In this study, we predict the inflow of the Soyang River dam, using the LSTM model with the Sequence-to-Sequence (LSTM-s2s) and attention mechanism (LSTM-s2s with attention) that can further improve the LSTM performance. Hourly inflow, temperature, and precipitation data from 2013 to 2020 were used to train the model, and validate and test for evaluating the performance of the models. As a result, the LSTM-s2s with attention showed better performance than the LSTM-s2s in general as well as in predicting a peak value. Both models captured the inflow pattern during the peaks but detailed hourly variability is limitedly simulated. We conclude that the proposed LSTM-s2s with attention can improve inflow forecasting despite its limits in hourly prediction.

최근 인공지능의 발전으로 시계열 자료 분석에 효과적인 Long Short-Term Memory (LSTM) 모델이 댐유입량 예측의 정확도를 높이는 데 활용되고 있다. 본 연구에서는 그 중 LSTM의 성능을 더욱 향상할 수 있는 Sequence-to-Sequence (s2s) 구조에 Attention 기법을 LSTM 모델에 첨가하여 소양강댐 유역의 유입량을 예측하였다. 분석 데이터는 2013년부터 2020년까지의 유입량 시자료와 종관기상관측기온 및 강수량 자료를 학습, 검증, 평가로 나누어 훈련한 후, 모델의 성능 평가를 진행하였다. 분석 결과, LSTM-s2s 모델보다 attention까지 첨가한 모델이 일반적으로 더 좋은 성능을 보였으며, attention 첨가 모델이 첨두값도 더 잘 예측하는 모습을 보였다. 그리고 두 모델 모두 첨두값 발생 동안 유량 패턴을 잘 반영하였지만 세밀한 시간 단위 변화량에는 어려움이 있었다. 이를 통해 시간 단위 예측의 어려움에도 불구하고, LSTM-s2s에 attention까지 첨가한 모델이 기존 LSTM-s2s의 예측 성능을 향상할 수 있음을 알 수 있었다.

Keywords

Acknowledgement

본 연구는 정부의 재원으로 과학기술정보통신부/한국연구재단의 지원(2020R1A2C2007670)과 국토교통부/국토교통과학기술진흥원의 지원(22CTAP-C163540-02)을 받아 수행되었습니다.

References

  1. Bicknell, B.R., Imhoff, J.C., Kittle Jr, J.L., Donigian Jr, A.S., and Johanson, R.C. (1996). Hydrological simulation program-FORTRAN. User's Manual for Release 11. U.S. Environmental Protection Agency, Washington, D.C., U.S.
  2. Cho, K., and Kim, Y. (2022). "Improving streamflow prediction in the WRF-Hydro model with LSTM networks." Journal of Hydrology, Vol. 605, 127297. https://doi.org/10.1016/j.jhydrol.2021.127297
  3. Choi, H., Cho, K., and Bengio, Y. (2018). "Fine-grained attention mechanism for neural machine translation." Neurocomputing, Vol. 284, pp. 171-176. https://doi.org/10.1016/j.neucom.2018.01.007
  4. Choi, J.-H., Kim, J.-S., Gwon, J.-H., and Moon, Y.-I. (2016). "Weighting assessment on evaluation indicators of dam rehabilitation using the AHP analysis." Journal of Korea Water Resources Association, Vol. 49, No. 5, pp. 381-389. https://doi.org/10.3741/JKWRA.2016.49.5.381
  5. Costa-jussa, M.R. (2018). "From feature to paradigm: Deep learning in machine translation." The Journal of Artificial Intelligence Research, Vol. 61, pp. 947-974. https://doi.org/10.1613/jair.1.11198
  6. Dawson, C.W., and Wilby, R.L. (2001). "Hydrological modelling using artificial neural networks." Progress in Physical Geography, Vol. 25, No. 1, pp. 80-108. https://doi.org/10.1177/030913330102500104
  7. Devia, G.K., Ganasri, B.P., and Dwarakish, G.S. (2015). "A review on hydrological models." Aquatic Procedia, Vol. 4, pp. 1001-1007. https://doi.org/10.1016/j.aqpro.2015.02.126
  8. Fu, M., Fan, T., Ding, Z., Salih, S.Q., Al-Ansari, N., and Yaseen, Z.M. (2020). "Deep learning data-intelligence model based on adjusted forecasting window scale: Application in daily streamflow simulation." IEEE Access, Vol. 8, pp. 32632-32651. https://doi.org/10.1109/access.2020.2974406
  9. Han, H., Choi, C., Jung, J., and Kim, H.S. (2021). "Application of sequence to sequence learning based LSTM model (LSTM-s2s) for forecasting dam inflow." Journal of Korea Water Resources Association, Vol. 54, No. 3, pp. 157-166. https://doi.org/10.3741/JKWRA.2021.54.3.157
  10. He, X., Luo, J., Li, P., Zuo, G., and Xie, J. (2020). "A hybrid model based on variational mode decomposition and gradient boosting regression tree for monthly runoff forecasting." Water Resources Management, Vol. 34, No. 2, pp. 865-884. https://doi.org/10.1007/s11269-020-02483-x
  11. Hochreiter, S., and Schmidhuber, J. (1997). "Long short-term memory." Neural Computation, Vol. 9, No. 8, pp. 1735-1780. https://doi.org/10.1162/neco.1997.9.8.1735
  12. Jung, S., Lee, D., and Lee, K. (2017). "Prediction of river water level using deep-learning open library." Korean Society of Hazard Mitigation, Vol. 18, No. 1, pp. 1-11. https://doi.org/10.9798/kosham.2018.18.1.1
  13. Kao, I.-F., Zhou, Y., Chang, L.-C., and Chang, F.-J. (2020). "Exploring a long short-term memory based encoder-decoder framework for multi-step-ahead flood forecasting." Journal of Hydrology, Vol. 583, 124631. https://doi.org/10.1016/j.jhydrol.2020.124631
  14. Kim, B.K., Kim, S., Lee, E.T., and Kim, H.S. (2007). "Methodology for estimating ranges of SWAT model parameters: Application to Imha Lake inflow and suspended sediments." Journal of The Korean Society of Civil Engineers B, Vol. 27 No. 6B, pp. 661-668.
  15. Kwak, J.-W., Kim, H.-S., and Hong, I.-P. (2009). "A study of progressive parameter calibrations for rainfall-runoff models." Proceedings of the Korea Water Resources Association Conference, pp. 1499-1503.
  16. Kwon, O.-I., and Shim, M.-P. (1997). "Determination scheme of variable restricted water level during flood period of multipurpose dam." Journal of Korea Water Resources Association, Vol. 30, No. 6, pp. 709-720.
  17. Le, X.-H., Ho, H. V., Lee, G., and Jung, S. (2019). "Application of Long Short-Term Memory (LSTM) neural network for flood forecasting." Water, Vol. 11, No. 7, 1387. https://doi.org/10.3390/w11071387
  18. Legates, D.R., and McCabe Jr., G.J. (1999). "Evaluating the use of "goodness-of-fit" Measures in hydrologic and hydroclimatic model validation." Water Resources Research, Vol. 35, No. 1, pp. 233-241. https://doi.org/10.1029/1998WR900018
  19. Li, W., Guo, D., and Fang, X. (2018). "Multimodal architecture for video captioning with memory networks and an attention mechanism." Pattern Recognition Letters, Vol. 105, pp. 23-29. https://doi.org/10.1016/j.patrec.2017.10.012
  20. Li, Z., Liu, W., Zhang, X., and Zheng, F. (2009). "Impacts of land use change and climate variability on hydrology in an agricultural catchment on the loess plateau of China." Journal of Hydrology, Vol. 377, No. 1-2, pp. 35-42. https://doi.org/10.1016/j.jhydrol.2009.08.007
  21. Luong, M.-T., Pham, H., and Manning, C.D. (2015). "Effective approaches to attention-based neural machine translation." Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, ACL, Lisbon, Portugal, pp. 1412-1421.
  22. Marmanis, D., Schindler, K., Wegner, J. D., Galliani, S., Datcu, M., and Stilla, U. (2018). "Classification with an edge: Improving semantic image segmentation with boundary detection." ISPRS Journal of Photogrammetry and Remote Sensing, Vol. 135, pp. 158-172. https://doi.org/10.1016/j.isprsjprs.2017.11.009
  23. National Water Resources Management Infomation System (WAMIS) (2003). South Korea, accessed 10, January 2022, .
  24. Noh, J.-K., and Lee, J.-N. (2011). "Comparison of streamflow runoff model in Korea for applying to reservoir operation." Korean Journal of Agricultural Science, Vol. 38, No. 3, pp. 513-524. https://doi.org/10.7744/CNUJAS.2011.38.3.513
  25. Nourani, V. (2017). "An emotional ANN (EANN) approach to modeling rainfall-runoff process." Journal of Hydrology, Vol. 544, pp. 267-277. https://doi.org/10.1016/j.jhydrol.2016.11.033
  26. Nourani, V., Komasi, M., and Mano, A. (2009). "A multivariate ANN-wavelet approach for rainfall-runoff modeling." Water Resources Management, Vol. 23, No. 14, pp. 2877-2894. https://doi.org/10.1007/s11269-009-9414-5
  27. Qi, Y., Zhou, Z., Yang, L., Quan, Y., and Miao, Q. (2019). "A decomposition-ensemble learning model based on LSTM neural network for daily reservoir inflow forecasting." Water Resources Management, Vol. 33, No. 12, pp. 4123-4139. https://doi.org/10.1007/s11269-019-02345-1
  28. Song, K., Yao, T., Ling, Q., and Mei, T. (2018). "Boosting image sentiment analysis with visual attention." Neurocomputing, Vol. 312, pp. 218-228. https://doi.org/10.1016/j.neucom.2018.05.104
  29. Sutskever, I., Vinyals, O., and Le, Q.V. (2014). "Sequence to sequence learning with neural networks." Proceedings of the 27th International Conference on Neural Information Processing Systems, MIT Press, Cambridge, MA, U.S., Vol. 2, pp. 3104-3112.
  30. Tiwari, M.K., and Chatterjee, C. (2010). "Development of an accurate and reliable hourly flood forecasting model using waveletbootstrap-ANN (WBANN) hybrid approach." Journal of Hydrology, Vol. 394, No. 3-4, pp. 458-470. https://doi.org/10.1016/j.jhydrol.2010.10.001
  31. Xiang, Z., Yan, J., and Demir, I. (2020). "A rainfall-runoff model with LSTM-based sequence-to-sequence learning." Water Resources Research, Vol. 56, No. 1, e2019WR025326.
  32. Yan, L., Chen, C., Hang, T., and Hu, Y. (2021). "A stream prediction model based on attention-LSTM." Earth Science Informatics, Vol. 14, No. 2, pp. 723-733. https://doi.org/10.1007/s12145-021-00571-z
  33. Zhou, H., Zhang, Y., Yang, L., Liu, Q., Yan, K., and Du, Y. (2019). "Short-term photovoltaic power forecasting based on long short term memory neural network and attention mechanism." IEEE Access, Vol. 7, pp. 78063-78074. https://doi.org/10.1109/access.2019.2923006