Journal of The Korean Society of Agricultural Engineers (한국농공학회논문집)

The Korean Society of Agricultural Engineers (한국농공학회)
권호 표지
DOI QR코드

DOI QR Code

Comparison of Artificial Neural Network Model Capability for Runoff Estimation about Activation Functions

활성화 함수에 따른 유출량 산정 인공신경망 모형의 성능 비교

  • Kim, Maga (Department of Rural Systems Engineering, Seoul National University) ;
  • Choi, Jin-Yong (Department of Rural Systems Engineering, Research Institute of Agriculture and Life Sciences, Global Smart Farm Convergence Major, Seoul National University) ;
  • Bang, Jehong (Department of Rural Systems Engineering, Seoul National University) ;
  • Yoon, Pureun (Department of Rural Systems Engineering, Seoul National University) ;
  • Kim, Kwihoon (Department of Rural Systems Engineering, Seoul National University)
  • Received : 2020.12.22
  • Accepted : 2021.01.18
  • Published : 2021.01.31
Download PDF
⟨ Previous Next ⟩

Abstract

Analysis of runoff is substantial for effective water management in the watershed. Runoff occurs by reaction of a watershed to the rainfall and has non-linearity and uncertainty due to the complex relation of weather and watershed factors. ANN (Artificial Neural Network), which learns from the data, is one of the machine learning technique known as a proper model to interpret non-linear data. The performance of ANN is affected by the ANN's structure, the number of hidden layer nodes, learning rate, and activation function. Especially, the activation function has a role to deliver the information entered and decides the way of making output. Therefore, It is important to apply appropriate activation functions according to the problem to solve. In this paper, ANN models were constructed to estimate runoff with different activation functions and each model was compared and evaluated. Sigmoid, Hyperbolic tangent, ReLU (Rectified Linear Unit), ELU (Exponential Linear Unit) functions were applied to the hidden layer, and Identity, ReLU, Softplus functions applied to the output layer. The statistical parameters including coefficient of determination, NSE (Nash and Sutcliffe Efficiency), NSEln (modified NSE), and PBIAS (Percent BIAS) were utilized to evaluate the ANN models. From the result, applications of Hyperbolic tangent function and ELU function to the hidden layer and Identity function to the output layer show competent performance rather than other functions which demonstrated the function selection in the ANN structure can affect the performance of ANN.

Keywords

References

  1. Ahn, S. J., K. W. Jun, and K. I. Kim, 2000. Forecasting of runoff hydrograph using neural network algorithms. Journal of Korean Water Resources Association 33(4): 505-515. (in Korean)
  2. Asadi, H., H. Shahedi, B. Jarihani, and R. C. Sidle, 2019. Rainfall-runoff modelling using hydrological connectivity index and artificial neural network approach. Water 11(2): 212. doi:10.3390/w11020212.
  3. Campolo, M., P. Andreussi, and A. Soldati, 1999. River flood forecasting with a neural network model. Water Resources Research 35(4): 1191-1197. doi:10.1029/1998WR900086.
  4. Ghorbani, M. A., H. A. Zadeh, and M. Isazadeh, 2016. A comparative study of artificial neural network (MLP, RBF) and support vector machine models for river flow prediction. Environmental Earth Sciences 75: 476. doi:10.1007/s12665-015-5096-x.
  5. Glorot, X., A. Bordes, and Y. Bengio, 2011. Deep sparse rectifier neural networks. Proceedings of the 14th International Conference on Artificial Intelligence and Statistics 15: 315-323.
  6. Gomes, G. S. da S., T. B. Ludermir, and L. M. M. R. Lima, 2011. Comparison of new activation functions in neural network for forecasting financial time series. Neural Computing and Applications 20: 417-439. doi:10.1007/s00521-010-0407-3.
  7. Gunther, Frauke, and S. Fritsch, 2010. Neuralnet: training of neural networks. Journal of Hydrologic Engineering 2(1): 30-38. doi:10.1061/(ASCE)1084-0699(1999)4:2(135).
  8. Gupta, H. V., S. Sorooshian, and P. O. Yapo, 1999. Status of automatic calibration for hydrologic models: comparison with multilevel expert calibration. Journal of Hydrologic Engineering 4(2): 135-143. https://doi.org/10.1061/(ASCE)1084-0699(1999)4:2(135)
  9. Jung, S. H., D. E. Lee, and K. S. Lee, 2018. Prediction of river water level using deep-learning open library. Journal of the Korean Society of Hazard Mitigation 18(1): 1-11. doi:10.9798/KOSHAM.2018.18.1.1.
  10. Karunanithi, N., W. J. Grenney, D. Whitley, and K. Bovee, 1994. Neural networks for river flow prediction. Journal of Computing in Civil Engineering 8(2): 201-220. doi:10.1061/(ASCE)0887-3801(1994)8:2(201).
  11. Kim, S. W., 2000. A study on the forecasting of daily streamflow using the multilayer neural networks model. Journal of Korean Water Resources Association 33(5): 537-550. (in Korean)
  12. Kingma, D. P., and J. L. Ba, 2015. Adam: a method for stochastic optimization. 3rd International Conference for Learning Representations.
  13. Kisi, O., 2007. Streamflow forecasting using different artificial neural network algorithms. Journal of Hydrologic Engineering 12(5): 532-539. doi:10.1061/(ASCE)1084-0699(2007)12:5(532).
  14. Kumar, A. R. S., K. P. Sudheer, S. K. Jain, and P. K. Agarwal, 2005. Rainfall-runoff modelling using artificial neural networks: comparison of network types. Hydrological Processes 19(6): 127-1291. doi:10.1002/hyp.5581.
  15. Kurma, S., T. Roshini, and D. Himayoun, 2019. A comparison of emotional neural network (ENN) and artificial neural network (ANN) approach for rainfallrunoff modelling. Civil Engineering Journal 5(10): 2120-2130. doi:10.28991/cej-2019-03091398.
  16. Lee, K. S., S. C. Park, H. M. Lee, and Y. H. Jin, 2000. The study on the forecasting of runoff applied the B.P. Algorithm of the artificial neural network in the Young-San river. Journal of The Korean Society of Civil Engineers 20(5B): 679-688. (in Korean)
  17. Lin, Y., H. Wen, and S. Liu, 2019. Surface runoff response to climate change based on artificial neural network (ANN) models: a case study with Zagunao catchment in upper Minjiang river, southwest China. Journal of Water and Climate Change 10(1): 158-166. doi:10.2166/wcc.2018.130.
  18. Mishra, P. K., and S. Karmakar, 2019. Performance of optimum neural network in rainfall-runoff modeling over a river basin. International Journal of Environmental Science and Technology 16: 1289-1302. doi:10.1007/s13762-018-1726-7.
  19. Moriasi, D. N., M. W. Gitau, N. Pai, and P. Daggupati, 2015. Hydrologic and water quality models: performance measures and evaluation criteria. Transactions of the ASABE 50(6): 1763-1785. doi:10.13031/trans.58.10715.
  20. Nash, J. E. and V. Sutcliffe, 1970. River flow forecasting through conceptual models part I - a discussion of principles. Journal of Hydrology 10(3): 282-290. doi:10.1016/0022-1694(70)90255-6.
  21. Patel, A. B., and G. S. Joshi, 2017. Modeling of rainfallrunoff correlations using artificial neural network-a case study of Dharoi watershed of a Sabarmati river basin, India. Civil Engineering Journal 3(2): 78-87. doi:10.28991/cej-2017-00000074.
  22. Poonia, V., and H. L. Tiwari, 2020. Rainfall-runoff modeling for the Hoshangabad basin of Narmada river using artificial neural network. Arabian Journal of Geosciences 13: 944. doi:10.1007/s12517-020-05930-6.
  23. Pushpalatha, R., C. Perrin, N. L. Moine, and V. Andreassian, 2012. A review of efficiency criteria suitable for evaluating low-flow simulations. Journal of Hydrology 420: 171-182. doi:10.1016/j.jhydrol.2011.11.055.
  24. Ramachandran, P., B. Zoph, and Q. V. Le, 2017. Searching for activation functions. 6th International Conference on Learning Representations.
  25. Sarkar, A., and R. Kumar, 2012. Artificial networks for event based rainfall-runoff modeling. Journal of Water Resource and Protection 4(10): 891-897. doi:10.4236/jwarp.2012.410105.
  26. Shamseldin, A. Y., 2010. Artificial neural network model for river flow forecasting in a developing country. Journal of Hydroinformatics 12(1): 22-35. doi:10.2166/hydro.2010.027.
  27. Shamseldin, A. Y., A. E. Nasr, and K. M. O'Connor, 2002. Comparison of differeny forms of the multi-layer feed-forward neural network method used for river flow forecasting. Hydrology and Earth System Sciences 6(4): 671-684. doi:10.5194/hess-6-671-2002.
  28. Sharma, S., S. Sharma, and A. Athaiya, 2020. Activation functions in neural networks. International Journal of Engineering Applied Sciences and Technology 4(12): 310-316. https://doi.org/10.33564/IJEAST.2020.v04i12.054
  29. Sibi, P., S. A. Jones, and P. Siddarth, 2013. Analysis of different activation functions using back propagation neural networks. Journal of Theoretical and Applied Information Technology 47(3): 1344-1348.
  30. Song, C. M., and K. H. Lee, 2020. Applicability evaluation for discharge model using curve number and convolution neural network. Ecology and Resilient Infrastructure 7(2): 114-125. doi:10.17820/eri.2020.7.2.114.
  31. Sudheer, K. P., P. C. Nayak, and K. S. Ramasastri, 2003. Improving peak flow estimation in artificial neural network river flow models. Hydrological Processes 17(3): 677-686. doi:10.1002/hyp.5103.
  32. Tokar, A. S., and M. Markus, 2000. Precipitation-runoff modeling using artificial neural networks and conceptual models. Journal of Hydrologic Engineering 5(2): 156-161. doi:10.1061/(ASCE)1084-0699(1999)4:3(232).
  33. Yadav, A. K., V. K. Chandola, A. Singh, and B. P. Singh, 2020. Rainfall-runoff modelling using artificial neural networks (ANNs) model. International Journal of Current Microbiology and Applied Sciences 9(3): 127-135. doi:10.20546/ijcmas.2020.903.016.
  34. Yeo, W. K., Y. M. Seo, S. Y. Lee, and H. K. Ji, 2010. Study on water stage prediction using hybrid model of artificial neural network and genetic algorithm. Journal of Korean Water Resources Association 43(8): 721-731. (in Korean) doi:10.3741/JKWRA.2010.43.8.721.
  35. Yoon, K. H., B. C. Seo, and H. S. Shin, 2004. Dam inflow forecasting for short term flood based on neural networks in Nakdong river basin. Journal of Korean Water Resources Association 37(1): 67-75. (in Korean) https://doi.org/10.3741/JKWRA.2004.37.1.067
  36. Zadeh, M. R., S. Amin, D. Khalili, and V. P. Singh, 2010. Daily outflow prediction by multi layer perceptron with logistic sigmoid and tangent sigmoid activation functions. Water Resources Management 24: 2673-2688. doi:10.1007/s11269-009-9573-4.