Browse > Article
http://dx.doi.org/10.5389/KSAE.2022.64.3.009

Analysis of the Optimal Window Size of Hampel Filter for Calibration of Real-time Water Level in Agricultural Reservoirs  

Joo, Dong-Hyuk (Department of Rural and Bio-Systems Engineering & BK21 Education and Research Unit for Climate-smart ReclaimedTideland Agriculture, Chonnam National University)
Na, Ra (Department of Rural and Bio-Systems Engineering & BK21 Education and Research Unit for Climate-smart ReclaimedTideland Agriculture, Chonnam National University)
Kim, Ha-Young (Department of Rural and Bio-Systems Engineering & BK21 Education and Research Unit for Climate-smart ReclaimedTideland Agriculture, Chonnam National University)
Choi, Gyu-Hoon (WeDB company)
Kwon, Jae-Hwan (Agricultural Infrastructure Project Office, Korea Rural Community Corporation (KRC))
Yoo, Seung-Hwan (Department of Rural and Bio-Systems Engineering & BK21 Education and Research Unit for Climate-smart ReclaimedTideland Agriculture, Chonnam National University)
Publication Information
Journal of The Korean Society of Agricultural Engineers / v.64, no.3, 2022 , pp. 9-24 More about this Journal
Abstract
Currently, a vast amount of hydrologic data is accumulated in real-time through automatic water level measuring instruments in agricultural reservoirs. At the same time, false and missing data points are also increasing. The applicability and reliability of quality control of hydrological data must be secured for efficient agricultural water management through calculation of water supply and disaster management. Considering the characteristics of irregularities in hydrological data caused by irrigation water usage and rainfall pattern, the Korea Rural Community Corporation is currently applying the Hampel filter as a water level data quality management method. This method uses window size as a key parameter, and if window size is large, distortion of data may occur and if window size is small, many outliers are not removed which reduces the reliability of the corrected data. Thus, selection of the optimal window size for individual reservoir is required. To ensure reliability, we compared and analyzed the RMSE (Root Mean Square Error) and NSE (Nash-Sutcliffe model efficiency coefficient) of the corrected data and the daily water level of the RIMS (Rural Infrastructure Management System) data, and the automatic outlier detection standards used by the Ministry of Environment. To select the optimal window size, we used the classification performance evaluation index of the error matrix and the rainfall data of the irrigation period, showing the optimal values at 3 h. The efficient reservoir automatic calibration technique can reduce manpower and time required for manual calibration, and is expected to improve the reliability of water level data and the value of water resources.
Keywords
Hampel filter; agricultural reservoir; window size; reliability;
Citations & Related Records
Times Cited By KSCI : 2  (Citation Analysis)
연도 인용수 순위
1 Pearson, R. K., Neuvo, Y., Astola, J., and Gabbouj, M, 2016. Generalized Hampel filters. EURASIP Journal on Advances in Signal Processing 2016(1): 1-18. doi:10.1186/s13634-016-0383-6.   DOI
2 Press, W. H., Flannery, B. P., Teukolsky, S. A., and Vetterling, W. T, 1987. Numerical recipes : the art of scientific computing, Cambridge University Press. doi:10.1063/1.2820230.   DOI
3 Pukelsheim, Friedrich, 1994. "The three sigma rule." The American Statistician 48(2): 88-91. doi:10.2307/2684253.   DOI
4 Ranjan, K. G., Tripathy, D. S., Prusty, B. R., and Jena, D, 2021. An improved sliding window prediction-based outlier detection and correction for volatile time-series. International Journal of Numerical Modelling: Electronic Networks, Devices and Fields 34(1), e2816. doi:10.1002/jnm.2816.   DOI
5 Shin, H. J., and T. H. Lee, 2020. A study on the estimation of missing hydrological data using adaptive network-based fuzzy inference system (anfis). In Proceedings of the Korea Water Resources Association Conference 264. (in Korean).
6 United States Geological Survey, 2012. Computing discharge using the index velocity method. South Carolina:United States Geological Survey 35-59.
7 Yang, M. H, W. H. Nam, H. J. Kim, T. G. Kim, A. K. Shin, and M. S. Kang, 2021. Anomaly detection in reservoir water level data using the LSTM model based on deep learning. Journal of the Korean Society of Hazard Mitigation 21(1): 71-81 (in Korean). doi:10.9798/KOSHAM.2021.21.1.71.   DOI
8 Korea Rural Community Corporation (KRC). 2019. A Study on the establishment of quality control standards for hydrologic data in agricultural reservoirs and waterways. (in Korean).
9 Ministry of Agriculture, Food and Rural Affairs, 2016. Development of information analysis technology for abnormal behavior of agricultural reservoirs using big data related to weather information and water levels. (in Korean).
10 Ministry of Environment. 2018. Environment and maintenance and management of hydrologic research facilities and standards for quality management of hydrologic data. (in Korean).
11 Ministry of Land, Transport and Maritime Affairs. 2008. Establishment of a basic plan for hydrologic investigation (2010-2019). (in Korean).
12 Ministry of Land, Transport and Maritime Affairs. 2011. The 4th Comprehensive Plan for Water Resources 2nd Amendment (2011-2020). (in Korean).
13 Nash, J. E., and Sutcliffe, J. V, 1970. "River flow forecasting through conceptual models part I - A discussion of principles". Journal of Hydrology 10(3): 282-290. doi:10.1016/0022-1694(70)90255-6.   DOI
14 Oh, C. R., S. C. Park, H. M. Lee, and Y. P. Pyo, 2002. A forecasting of water quality in the Youngsan river using neural network. Journal of The Korean Society of Civil Engineers 22(3B): 372-382. (in Korean).
15 Kim, M. G., J. Y. Choi, J. H. Bang, and J. J. Lee, 2019. Outlier detection of real-time reservoir water level data using threshold model and artificial neural network model. Journal of the Korean Society of Agricultural Engineers 61(1): 107-120. (in Korean). doi:10.5389/KSAE.2019.61.1.107.   DOI
16 Choi, G. H., K. H. Kim, S. Y. Lim, S. H. Yoo, E. H. Choi, and M. S. Kang, 2020. Quality control on water-level in irrigation canal using filter techniques. In Proceedings of the Korean Society of Agricultural Engineers Conference 107 (in Korean).
17 Kim, H. K., S. W. Park, S., and M. Kim, 2006. Development of hydrologic data management system based on relational database. Korean Journal of Hydrosciences 39(10): 855-866 (in Korean). doi:10.3741/JKWRA.2006.39.10.855.   DOI
18 Bang, J. H., Y. H. Lee, S. Y. Jeong, and J. Y. Choi, 2017. A study of outlier detection on time series of water-level in agricultural reservoir. In Proceedings of the Korean Society of Agricultural Engineers Coneference 60 (in Korean). Environmental Protection Agency, Athens, GA.
19 Choi, J. Y., 2018. Development of quality control methods for water level data and irrigation water supply estimation, 4-10. Korea Rural Community Corporation (in Korean).
20 Hyndman, Rob J. Koehler, and Anne B, 2006. "Another look at measures of forecast accuracy". International Journal of Forecasting 22(4): 679-688. doi:10.1016/j.ijforecast.2006.03.001.   DOI
21 Lee, J. W, H. S. Kim, C. H. Hwang, I. S. Kang, and H. K. Jeong, 2018. Data cleansing algorithm for reducing outlier. In Proceedings of the Korean Institute of Information and Communication Sciences Conference 342-344. (in Korean).
22 Kim, H. S., and C. S. Kim, 2003. Hydrological data quality control system for monitoring and improving the data quality. Korean Society of Civil Engineers Conference 2032-2036. (in Korean).
23 Korea Rural Community Corporation (KRC). 2018. Development of technology for securing reliability of water level measurement data and estimating water supply. (in Korean)
24 K-water. 2018. K-water Data Quality Management Guidelines (Water Information). (in Korean).
25 Ministry of the Interior and Safety. 2018. Public Data Quality Management Manual ver 2.0. (in Korean).
26 Korea Rural Community Corporation (KRC). 2020. Advanced quality management study of reservoir water level measurement data. (in Korean).
27 MathWorks. https://kr.mathworks.com
28 Ministry of Land, Infrastructure and Transport, 2018. Establishment and operation of the national hydrologic data quality management system (7th). (in Korean).