Browse > Article
http://dx.doi.org/10.5389/KSAE.2019.61.1.107

Outlier Detection of Real-Time Reservoir Water Level Data Using Threshold Model and Artificial Neural Network Model  

Kim, Maga (Department of Rural Systems Engineering, Seoul National University)
Choi, Jin-Yong (Department of Rural Systems Engineering, Research Institute of Agriculture and Life Sciences, Seoul National University)
Bang, Jehong (Department of Rural Systems Engineering, Seoul National University)
Lee, Jaeju (Rural Research Institute, Korea Rural Community Corporation)
Publication Information
Journal of The Korean Society of Agricultural Engineers / v.61, no.1, 2019 , pp. 107-120 More about this Journal
Abstract
Reservoir water level data identify the current water storage of the reservoir, and they are utilized as primary data for management and research of agricultural water. For the reservoir storage management, Korea Rural Community Corporation (KRC) installed water level stations at around 1,600 agricultural reservoirs and has been collecting the water level data every 10 minutes. However, various kinds of outliers due to noise and erroneous problems are frequently appearing because of environmental and physical causes. Therefore, it is necessary to detect outlier and improve the quality of reservoir water level data to utilize the water level data in purpose. This study was conducted to detect and classify outlier and normal data using two different models including the threshold model and the artificial neural network (ANN) model. The results were compared to evaluate the performance of the models. The threshold model identifies the outlier by setting the upper/lower bound of water level data and variation data and by setting bandwidth of water level data as a threshold of regarding erroneous water level. The ANN model was trained with prepared training dataset as normal data (T) and outlier (F), and the ANN model operated for identifying the outlier. The models are evaluated with reference data which were collected reservoir water level data in daily by KRC. The outlier detection performance of the threshold model was better than the ANN model, but ANN model showed better detection performance for not classifying normal data as outlier.
Keywords
Reservoir water level; outlier detection; threshold model; artificial neural network;
Citations & Related Records
Times Cited By KSCI : 8  (Citation Analysis)
연도 인용수 순위
1 Yeo, W. K., Y. M. Seo, S. Y. Lee, and H. K. Ji, 2010. Study on water stage prediction using hybrid model of artificial neural network and genetic algorithm. Journal of Korean Water Resources Association 43(8): 721-731 (in Korean). doi:10.3741/JKWRA.2010.43.8.721.   DOI
2 Yoon, K. H., B. C. Seo, and H. S. Shin, 2004. Dam inflow forecasting for short term flood based on neural networks in Nakdong river basin. Journal of Korean Water Resources Association 37(1): 67-75 (in Korean).   DOI
3 Ahn, S. J., K. W. Jun, and K. I. Kim, 2000b. Forecasting of runoff hydrograph using neural network algorithms. Journal of Korean Water Resources Association 33(4): 505-515 (in Korean).
4 Bang, J. H., Y. H. Lee, S. Y. Jeong, and J. Y. Choi, 2017. A study of outlier detection on time series of water-level in agricultural reservoir. In Proceedings of the Korean Society of Agricultural Engimeers Conefrence 60 (in Korean).
5 Choi, J. K., and M. S. Kang, 2000. Application of neural network to water resources. Journal of Korean National Comittee on Irrigation and Drainage 7(2): 248-258 (in Korean).
6 Choi, J. Y., 2018. Development of quality control methods for water level data and irrigation water supply estimation, 4-10. Korea Rural Community Corporation.
7 Harrison, D. L., S. J. Driscoll, and M. Kitchen, 2000. Improving precipitation estimates from weather radar using quality control and correction techniques. Meteorological Applications 7(2): 135-144. doi:10.1017/S1350482700001468.   DOI
8 Feng, S., Q. Hu, and W. Qian, 2004. Quality control of daily meteorological data in China, 195-2000: a new dataset. International Journal of Climatology 24: 853-870. doi:10.1002/joc.1047.   DOI
9 Gonzalez-Rouco, J. F., J. L. Jimenez, V. Quesada, and F. Valero, 1999. Quality control and homogeneity of precipitation data in the southwest of Europe. Journal of Climate 14: 964-978. doi:10.1175/1520-0442(2001)014<0964:QCAHOP>2.0.CO;2.   DOI
10 Günther, F., and S. Fritsch, 2010. Neuralnet: training of neural networks. The R Journal 2(1): 30-38.   DOI
11 Kang, B. S., and B. K. Lee, 2008. Predicting probability of precipitation using artificial neural network and mesoscale numerical weather prediction. Journal of The Korean Society of Civil Engineers 28(5B): 485-493 (in Korean).
12 Horsburgh, J. S., D. G. Tarboton, D. R. Maidment, amd I. Zaslavsky, 2008. A relational model for environmental and water resources data. Water Resources Research 44(5). doi:10.1029/2007WR006392.
13 Jang, S. H., J. Y. Yoon, S. D. Kim, and Y. N. Yoon, 2007. An establishment of operation and management system for flood control and conservation in reservoir with date: I. establishment of real-time inflow prediction model using recorded rainfall data. Journal of The Korean Society of Civil Engineers 27(2B): 133-140 (in Korean).
14 Jeong, G. H., and T. W. Kim, 2007. Comparing water distribution model with reservoir and water system management. Water and Future 40(10): 38-43 (in Korean).
15 Kang, M. G., H. S. Jeong, and J. T. Kim, 2010. Efficient management of agricultural canal systems through quality management of water level and water quantity data. Rural Resources 52(2): 87-96 (in Korean).
16 Kim, S. J., Y. S. Kwon, K. H. Lee, and H. S. Kim, 2010. Radar rainfall adjustment by artificial neural network and runoff analysis. Journal of The Korean Society of Civil Engineers 30(2B): 159-167 (in Korean).
17 Kilonsky, B. J., and P. Caldwell, 1991. In the pursuit of high-quality sea level data. In Oceans 91 Proceedings (2): 669-675. doi:10.1109/OCEANS.1991.627921.
18 Kim, C. H., J. A. Ryu, D. G. Kim, and G. B. Kim, 2016. Analysis of the effects of drainage systems in wetlands based on changes in groundwater level, soil moisture content, and water quality. The Journal of Engineering Geology 26(2): 251-260 (in Korean). doi:/10.9720/kseg.2016.2.251.   DOI
19 Kim, H. G., M. I. Kim, M. S. Lee, Y. S. Park, and J. H. Kwak, 2017. Correlation of deep landside occurrence and variation of groundwater level. Journal of The Korea Society of Forest Engineering and technology 15(1): 1-12 (in Korean).
20 Kim, H. K., S. M. Kim, and S. W. Park, 2006. Development of hydrologic data management system based on relational database. Journal of Korean Water Resources Association 39(10): 855-866 (in Korean).   DOI
21 Kim, S. W., 2000. A study on the forecasting of daily streamflow using the multilayer neural networks model. Journal of Korean Water Resources Association 33(5): 537-550 (in Korean).
22 Kim, S. W., and J. D. Salas, 2000. The flood water stage prediction based on neural networks method in stream gauge station. Journal of Korean Water Resources Association 33(2): 247-262 (in Korean).
23 Lee, C. W., Y. S. Mang, and Y. S. Kim, 2014. Behavior of fill dam subjected to continuous water level change and overflow. Journal of the Korean Geo-Environmental Society 15(6): 41-48 (in Korean). doi:10.14481/jkges.2014.15.6.41.
24 Mounce, S. R., R. B. Mounce, and J. B. Boxall, 2011. Novelty detection for time series data analysis in water distribution systems using support vector machines. Journal of Hydroinformatics 13(4): 672-686. doi:10.2166/hydro.2010.144.   DOI
25 Park, J. H., M. S. Kang, J. H. Song, and S. M. Jun, 2015. Design and implementation of IoT-based intelligent platform for water level monitoring. Journal of the Korean Society of Rural Planning 21(4): 177-186 (in Korean). doi:10.7851/ksrp.2015.21.4.177.   DOI
26 Mourad, M., and J. L. Bertrand-Krajewski, 2002. A method for automatic validation of long time series of data in urban hydrology. Water Science & Technology 45(4-5): 263-270. doi:10.2166/wst.2002.0601.   DOI
27 Oh, C. R., S. C. Park, H. M. Lee, and Y. P. Pyo, 2002. A forecasting of water quality in the Youngsan river using neural network. Journal of The Korean Society of Civil Engineers 22(3B): 372-382 (in Korean).
28 Oh, J. W., J. H. Park, and Y. K. Kim, 2008. Missing hydrological data estimation using neural network and real time data reconciliation. Journal of Korean Water Resources Association (10): 1059-1065 (in Korean). doi:10.3741/JKWRA.2008.41.10.1059.
29 Schneider, U., A. Becker, P. Finger, A. Meyer-Christoffer, M. Ziese, and B. Rudolf, 2014. GPCC's new land surface precipitation climatology based on quality-controlled in situ data its role in quantifying the global water cycle. Theoretical and Applied Climatology 115(1-2): 15-40. doi:10.1007/s00704-013-0860-x.   DOI
30 Seo, Y. M., E. H. Choi, and W. K. Yeo, 2017. Reservoir water level forecasting using machine learning models. Journal of the Korean Society of Agricultural Engineers 59(3): 97-110 (in Korean). doi:10.5389/KSAE.2017.59.3.097.   DOI
31 Ahn, S. J., I. S. Yeon, and K. I. Kim, 2000a. Rainfall forecasting Using neural network. Journal of The Korean Society of Civil Engineers 20(5B): 711-722 (in Korean).
32 Shim, S. B., S. K. Kim, R. H. Park, and D. K. Hoh, 1997. Decision support system for real-time reservoir operation during flood period. Journal of Korean Water Resources Association 30(5): 431-439 (in Korean).
33 Steiner, M., J. A. Smith, S. J. Burges, C. V. Alonso, and R. W. Darden, 1999. Effect of bias adjustment and rain gauge data quality control on radar rainfall estimation. Water Resources Research 35(8): 2487-2503. doi:10.1029/1999WR900142.   DOI
34 Ahn, J. H., M. S. Kang, I. H. Song, K. D. Lee, J. H. Song, and J. R. Jang, 2012. Estimation of surface runoff from paddy plots using an artificial neural network. Journal of the Korean Society of Agricultural Engineers 54(4): 66-71 (in Korean). doi:10.5389/KSAE.2012.54.4.065.