Economic and Environmental Geology (자원환경지질)

The Korean Society of Economic and Environmental Geology (대한자원환경지질학회)
권호 표지
DOI QR코드

DOI QR Code

Applying Transfer Learning to Improve the Performance of Deep Learning-based Groundwater Level Prediction Model with Insufficient Training Data

딥러닝 기반 지하수위 예측 모델 개발에 있어 데이터 부족 문제 해결을 위한 전이학습의 응용

  • Jiho Jeong (Department of Geology, Kyungpook National University) ;
  • Jina Jeong (Department of Geology, Kyungpook National University)
  • 정지호 (경북대학교 지질학과) ;
  • 정진아 (경북대학교 지질학과)
  • Received : 2024.09.26
  • Accepted : 2024.10.11
  • Published : 2024.10.29
Download PDF
⟨ Previous Next ⟩

Abstract

Predicting groundwater levels with data-driven models like artificial neural networks typically requires a substantial amount of data. However, when groundwater monitoring wells are newly developed or when a significant portion of the data is invalid (for example, due to missing values or outliers), acquiring an adequate dataset for training prediction models becomes challenging, leading to diminished prediction accuracy. This study proposes a method based on transfer learning to address the issue of insufficient training data. The Gated Recurrent Unit (GRU) was used as the primary data-driven model for predictions. A GRU-based pretrained network for the transfer learning process was developed using groundwater level and corresponding rainfall data collected from 89 monitoring stations nationwide. Subsequently, this pretrained network was fine-tuned using a small amount of training data obtained from the target monitoring well to develop the final prediction model. To verify the effectiveness of the transfer learning algorithm, two different groundwater level prediction models were evaluated: 1) a GRU-based model trained with insufficient data from the target well, and 2) a GRU-based model utilizing the transfer learning algorithm. Comparative verification was conducted with groundwater level data obtained from wells at two different locations, where the model using the transfer learning algorithm demonstrated superior performance compared to the other. This study confirms that the transfer learning algorithm can significantly enhance the performance of groundwater level prediction models, irrespective of the amount of available training data.

인공신경망과 같은 데이터 기반 모델을 활용하여 지하수위를 예측하기 위해서는 일반적으로 충분한 양의 데이터가 필요하다. 그러나 지하수위 모니터링 관정이 새로 개발되거나 유효하지 않은 데이터(예를 들어, 결측치 또는 이상치)가 다수 관찰되는 경우, 예측 모델을 적절히 학습하기 위한 데이터셋을 확보하기가 어려우며, 이는 예측 정확도의 저하로 이어진다. 본 연구에서는 이러한 학습 데이터 부족 문제를 해결하기 위해 전이 학습(Transfer Learning)을 기반으로 하는 방법을 제안하였다. 게이트 순환 유닛(Gated Recurrent Unit, GRU)을 예측을 위한 기본 데이터 기반 모델로 사용하였다. 전이학습 과정을 위한 GRU 기반 사전 학습 네트워크(Pretrained network)는 국내 전역의 89개 모니터링 지점에서 수집된 지하수위 및 이에 대응하는 강우 데이터를 활용하여 개발되었다. 그 후, 타겟 모니터링 관정에서 확보된 소량의 학습 데이터를 사용하여 사전 학습된 네트워크에 대한 미세 조정(Fine-tuning) 을 통해 최종 예측 모델을 개발하였다. 전이학습 알고리즘의 효과를 검증하기 위해 두 가지 서로 다른 지하수위 예측 모델을 비교 평가하였다: 1) 타겟 관정의 불충분한 데이터를 사용하여 학습된 GRU 기반 지하수위 예측 모델 및2) 전이학습 알고리즘을 기반으로 한 GRU 기반 예측 모델. 두 개의 다른 위치에 존재하는 관정에서 획득한 지하수위 자료에 대한 비교 검증이 이루어졌으며, 전이학습 알고리즘을 활용한 모델이 다른 모델에 비해 우수한 성능을 보였다. 이를 통해 사용 가능한 학습 데이터의 양에 상관없이 전이학습 알고리즘이 지하수위 예측 모델 성능 향상에 크게 기여할 수 있음을 확인하였다.

Keywords

Acknowledgement

이 논문은 2023학년도 경북대학교 연구년 교수 연구비에 의하여 연구되었음

References

  1. Altunkaynak, A. (2007) Forecasting Surface Water Level Fluctuations of Lake Van by Artificial Neural Networks, Water Resour. Manage, v.21, p.399-408. https://doi.org/10.1007/s11269-006-9022-6.
  2. Buck-Sorlin, G. (2013) Process-based Model. In: Dubitzky, W., Wolkenhauer, O., Cho, KH., Yokota, H. (eds) Encyclopedia of Systems Biology. Springer, New York, NY. https://doi.org/10.1007/978-1-4419-9863-7_1545.
  3. Changnon Jr., S.A. 1987. An assessment of climate change, water resources, and policy research. Water Int., v.12, p.69-76. https://doi.org/10.1080/02508068708686570
  4. Cho, K., van Merrienboer, B., Bahdanau, D. and Bengio, Y. (2014) On the properties of neural machine translation: Encoder-decoder approaches. arXiv preprint arXiv:1409.1259. https://doi.org/10.3115/v1/W14-4012
  5. Clifton, C. (2010) Water and climate change: Impacts on groundwater resources and adaptation options, Water Working Notes, Note No. 25, June 2010, World Bank Group.
  6. Coulibaly, P., Anctil, F., Aravena, R. and Bobee, B. (2001) Artificial neural network modeling of water table depth fluctuations. Water Resour. Res., v.37(4), p.885-896. https://doi.org/10.1029/2000WR900368
  7. Cuthbert, M.O. (2010) An improved time series approach for estimating groundwater recharge from groundwater level fluctuations. Water Resour. Res., v.46, W09515. https://doi:10.1029/2009WR008572.
  8. Dai, W., Chen, Y., Xue, G.R., Yang, Q. and Yu, Y. (2008) Translated Learning. Proc. 21st Ann. Conf. Neural Information Processing Systems.
  9. Djurovic, N., Domazet, M., Stricevic, R., Pocuca, V., Spalevic, V., Pivic, R., Gregoric, E. and Domazet, U. (2015) Comparison of Groundwater Level Models Based on Artificial Neural Networks and ANFIS. The Scientific World Journal, vol. 2015, Article ID 742138, 13 pages. https://doi.org/10.1155/2015/742138.
  10. Emamgholizadeh, S., Moslemi, K. and Karami, G. (2014) Prediction the Groundwater Level of Bastam Plain (Iran) by Artificial Neural Network (ANN) and Adaptive Neuro-Fuzzy Inference System (ANFIS). Water Resour. Manage, v.28, p.5433-5446. https://doi.org/10.1007/s11269-014-0810-0.
  11. Gharehbaghi, A., Ghasemlounia, R., Ahmadi, F. and Albaji, M. (2022) Groundwater level prediction with meteorologically sensitive Gated Recurrent Unit (GRU) neural networks. J. Hydrol., v.612, 128262, ISSN 0022-1694, https://doi.org/10.1016/j.jhydrol.2022.128262.
  12. Gong, Y., Zhang, Y., Lan, S. and Wang, H. (2016) A Comparative Study of Artificial Neural Networks, Support Vector Machines and Adaptive Neuro Fuzzy Inference System for Forecasting Groundwater Levels near Lake Okeechobee, Florida. Water Resour. Manage, v.30, p.375-391. https://doi.org/10.1007/s11269-015-1167-8.
  13. Green, T.R. (2016) Linking Climate Change and Groundwater. In: Jakeman AJ, Barreteau O, Hunt RJ, Rinaudo JD, Ross A (eds) Integrated Groundwater Management. Springer, Cham, pp.97-141. https://doi.org/10.1007/978-3-319-23576-9_5.
  14. Hare, D.K., Helton, A.M., Johnson, Z.C., Lane, J. and Briggs, M. (2021) Continental-scale analysis of shallow and deep groundwater contributions to streams. Nat. Commun., v.12, 1450. https://doi.org/10.1038/s41467-021-21651-0.
  15. IGRAC (2020) Groundwater monitoring programmes: A global overview of quantitative groundwater monitoring networks.
  16. Jafari, M.M., Ojaghlou, H., Zare, M. and Schumann, G.J. (2021) Application of a Novel Hybrid Wavelet-ANFIS/Fuzzy C-Means Clustering Model to Predict Groundwater Fluctuations. Atmosphere, doi:10.3390/atmos12010009.
  17. Jahanara, A.A. and Khodashenas, S.R. (2019) Prediction of Ground Water Table Using NF-GMDH Based Evolutionary Algorithms. KSCE J. Civ. Eng., v.23, p.5235-5243. https://doi.org/10.1007/s12205-019-0804-9.
  18. Jeong, J. and Park, E. (2017) A shallow water table fluctuation model in response to precipitation with consideration of unsaturated gravitational flow. Water Resour. Res., v.53(4), p.3505-3512. https://doi.org/10.1002/2016WR020177
  19. Jeong, J. and Park, E. 2019. Comparative applications of data-driven models representing water table fluctuations. J. Hydrol., v.572, p.261-273. https://doi.org/10.1016/j.jhydrol.2019.02.051
  20. Kingma, D.P. and Ba, J. 2015. Adam: A method for stochastic optimization. International Conference on Learning Representations. https://doi.org/10.48550/arXiv.1412.6980
  21. Kisi, O. and Shiri, J. (2012) Wavelet and neuro-fuzzy conjunction model for predicting water table depth fluctuations. Hydrology Research, v.43, p.286-300. doi: 10.2166/nh.2012.104b.
  22. Kombo, O.H., Kumaran, S., Sheikh, Y.H., Bovim, A. and Jayavel, K. (2020) Long-Term Groundwater Level Prediction Model Based on Hybrid KNN-RF Technique. Hydrology, v.7, 59. https://doi.org/10.3390/hydrology7030059.
  23. Li, X., Ma, X., Xiao, F., Xiao, C., Wang, F. and Zhang, S. (2022) Time-series production forecasting method based on the integration of Bidirectional Gated Recurrent Unit (Bi-GRU) network and Sparrow Search Algorithm (SSA). J. Pet. Sci. Eng., v.208, Part A, 109309, ISSN 0920-4105, https://doi.org/10.1016/j.petrol.2021.109309.
  24. Mahmoud Khaki, M. Khaki, Ismail Yusoff, I. Yusoff and Nur Islami, N. Islami. (2015) Simulation of groundwater level through artificial intelligence system. Environ. Earth Sci., v.73, p.8357-8367. doi: 10.1007/s12665-014-3997-8.
  25. Maier, H.R. and Dandy, G.C. (1996) The use of artificial neural networks for the prediction of water quality parameters. Water Resour. Res., v.32, p.1013-1022. https://doi.org/10.1029/96WR03529
  26. Maiti, S. and Tiwari, R.K. (2014) A comparative study of artificial neural networks, Bayesian neural networks and adaptive neuro-fuzzy inference system in groundwater level prediction. Environ. Earth Sci., v.71, p.3147-3160. https://doi.org/10.1007/s12665-013-2702-7.
  27. Marchant, B.P. and Bloomfield, J.P. (2018) Spatio-temporal modelling of the status of groundwater droughts. J. Hydrol., v.564, p.397-413. https://doi.org/10.1016/j.jhydrol.2018.07.009
  28. Mirarabi, A., Nassery, H.R. and Nakhaei, M. (2019) Evaluation of data-driven models (SVR and ANN) for groundwater-level prediction in confined and unconfined systems. Environ. Earth Sci., v.78, 489. https://doi.org/10.1007/s12665-019-8474-y.
  29. Moosavi, V., Vafakhah, M., Shirmohammadi, B. and Ranjbar, M. (2014) Optimization of Wavelet-ANFIS and Wavelet-ANN Hybrid Models by Taguchi Method for Groundwater Level Forecasting. Arab J. Sci. Eng., v.39, p.1785-1796. https://doi.org/10.1007/s13369-013-0762-3.
  30. Nadiri, A., Naderi, K., Khatibi, R. and Gharekhani, M. (2019) Modelling groundwater level variations by learning from multiple models using fuzzy logic. Hydrol. Sci. J., v.64(2), p.210-226. doi: 10.1080/02626667.2018.1554940.
  31. Pan, M., Zhou, H., Cao, J., Liu, Y., Hao, J., Li, S. and Chen, C. (2020) Water Level Prediction Model Based on GRU and CNN. IEEE Access, v.8, p.60090-60100. doi: 10.1109/ACCESS.2020.2982433.
  32. Pan, S. and Yang, Q. (2010) A Survey on Transfer Learning. IEEE Trans. Knowl. Data Eng., v.22(10), p.1345-1359. doi: 10.1109/TKDE.2009.191.
  33. Paszke, A., Gross, S., Massa, F., Lerer, A., Bradbury, J., Chanan, G., Killeen, T., Lin, Z., Gimelshein, N., Antiga, L., Desmaison, A., Kopf, A., Yang, E., DeVito, Z., Raison, M., Tejani, A., Chilamkurthy, S., Steiner, B., Fang, L., Bai, J. and Chintala, S. (2019) PyTorch: An imperative style, high-performance deep learning library. In 33rd Annual Conference on Neural Information Processing Systems, 1. doi: 10.48550/arXiv.1912.01703
  34. Peterson, T.J. and Western, A.W. (2018) Statistical interpolation of groundwater hydrographs. Water Resour. Res., v.54, p.4663-4680. doi: 10.1029/2017WR021838
  35. Pham, Q.B., Kumar, M. and Di Nunno, F. (2022) Groundwater level prediction using machine learning algorithms in a drought-prone area. Neural Comput. Applic., v.34, p.10751-10773. https://doi.org/10.1007/s00521-022-07009-7.
  36. Roberts, M.J., Braun, N.O., Sinclair, T.R., Lobell, D.B. and Schlenker, W. (2017) Comparing and combining process-based crop models and statistical models with some implications for climate change. Environ. Res. Lett., v.12(9), Article ID 095010, 2017. http://dx.doi.org/10.1088/1748-9326/aa7f33
  37. Seeboonruang, U. (2014) An empirical decomposition of deep groundwater time series and possible link to climate variability. Glob. Nest J., v.16, p.87-103. https://doi.org/10.30955/gnj.001244
  38. Shi, X., Fan, W. and Ren, J. (2008) Actively Transfer Domain Knowledge. Proc. European Conf. Machine Learning and Knowledge Discovery in Databases (ECML/PKDD '08), 342-357. https://doi.org/10.1007/978-3-540-87481-2_23
  39. Siebert, S., Burke, J., Faures, J., Frenken, K., Hoogeveen, J., Doell, P. and Portmann, F. (2010) Groundwater use for irrigation-A global inventory. Hydrol. Earth Syst. Sci., v.14, p.1863-1880. doi: 10.5194/hess-14-1863-2010.
  40. Sun, J., Hu, L., Li, D., Sun, K. and Yang, Z. (2022) Data-driven models for accurate groundwater level prediction and their practical significance in groundwater management. J. Hydrol., v.608, 127630. https://doi.org/10.1016/j.jhydrol.2022.127630.
  41. Swain, S., Sahoo, S., Taloor, A., Mishra, S. and Pandey, A. (2022) Exploring recent groundwater level changes using Innovative Trend Analysis (ITA) technique over three districts of Jharkhand, India, Groundwater for Sustainable Development, v.18, 100783, ISSN 2352-801X, https://doi.org/10.1016/j.gsd.2022.100783.
  42. Tao, H., Hameed, M., Marhoon, H., Zounemat-Kermani, M, Heddamm S, Kim, S., Sulaiman, S., Tan, M., Sa'adi, Z., Mehr, A., Allawi, M., Abba, S.I., Zain, J., Falah, M., Jamei, M., Bokde, N., Bayatvarkeshi, M., Al-Mukhtar, M., Bhagat, S., Tiyasha, T., Khedher, K., Al-Ansari, N., Shahid, S. and Mundher Yaseen, Z. (2022) Groundwater level prediction using machine learning models: A comprehensive review. Neurocomputing, v.489, p.271-308. https://doi.org/10.1016/j.neucom.2022.03.014
  43. Van Rossum, G. and Drake, F.L. (2009) Python 3 Reference Manual, Scotts Valley, CA: CreateSpace.
  44. Walker, G.R., Crosbie, R.S., Chiew, F.H.S., Peeters, L. and Evans, R. (2021) Groundwater Impacts and Management under a Drying Climate in Southern Australia, Water, v.13, 3588. https://doi.org/10.3390/w13243588.
  45. Wen, Q., Sun, L., Yang, F., Song, X., Gao, J., Wang, X. and Xu, H. (2022) Time series data augmentation for deep learning: A survey. arXiv preprint arXiv:2002.12478. https://doi.org/10.48550/arXiv.2002.12478
  46. Wu, W.Y., Lo, M.H., Wada, Y., Famiglietti, J., Reager, J., Yeh, P., Ducharne, A. and Yang, Z. (2020) Divergent effects of climate change on future groundwater availability in key mid-latitude aquifers. Nat. Commun., v.11, 3710. https://doi.org/10.1038/s41467-020-17581-y.
  47. Wunsch, A., Liesch, T. and Broda, S. (2018) Forecasting groundwater levels using nonlinear autoregressive networks with exogenous input (NARX). J. Hydrol., v.567, p.743-758. https://doi.org/10.1016/j.jhydrol.2018.01.045.
  48. Xu, J., Wang, K., Lin, C., Xiao, L., Huang, X. and Zhang, Y. (2021) FM-GRU: A Time Series Prediction Method for Water Quality Based on seq2seq Framework. Water, v.13, 1031. https://doi.org/10.3390/w13081031.
  49. Yoon, H., Jun, S., Hyun, Y., Bae, G. and Lee, L. (2011) A comparative study of artificial neural networks and support vector machines for predicting groundwater levels in a coastal aquifer. J. Hydrol., v.396(1-2), p.128-138. https://doi.org/10.1016/j.jhydrol.2010.11.002
  50. Young, C., Liu, W. and Hsieh, W. (2015) Predicting the Water Level Fluctuation in an Alpine Lake Using Physically Based, Artificial Neural Network, and Time Series Forecasting Models. Math. Probl. Eng., doi: 10.1155/2015/708204.
  51. Yu, J., Zhang, X., Xu, L., Dong, J. and Zhangzhong, L. (2021) A hybrid CNN-GRU model for predicting soil moisture in maize root zone, Agric. Water Manag., v.245, 106649, ISSN 0378-3774, https://doi.org/10.1016/j.agwat.2020.106649.
  52. Zhang, J., Zhu, Y., Zhang, X., Ye, M. and Yang, J. (2018) Developing a Long Short-Term Memory (LSTM) based model for predicting water table depth in agricultural areas, J. Hydrol., v.561, p.918-929, ISSN 0022-1694, https://doi.org/10.1016/j.jhydrol.2018.04.065.
  53. Zhuo, H., Yang, Q., Hu, D.H. and Li, L. (2008) Transferring Knowledge from Another Domain for Learning Action Models. Proc. 10th Pacific Rim Int'l Conf. Artificial Intelligence. https://doi.org/10.1007/978-3-540-89197-0_115