• Title/Summary/Keyword: 결측 복원

Search Result 19, Processing Time 0.032 seconds

Development of Machine Learning Based Precipitation Imputation Method (머신러닝 기반의 강우추정 방법 개발)

  • Heechan Han;Changju Kim;Donghyun Kim
    • Journal of Wetlands Research
    • /
    • v.25 no.3
    • /
    • pp.167-175
    • /
    • 2023
  • Precipitation data is one of the essential input datasets used in various fields such as wetland management, hydrological simulation, and water resource management. In order to efficiently manage water resources using precipitation data, it is essential to secure as much data as possible by minimizing the missing rate of data. In addition, more efficient hydrological simulation is possible if precipitation data for ungauged areas are secured. However, missing precipitation data have been estimated mainly by statistical equations. The purpose of this study is to propose a new method to restore missing precipitation data using machine learning algorithms that can predict new data based on correlations between data. Moreover, compared to existing statistical methods, the applicability of machine learning techniques for restoring missing precipitation data is evaluated. Representative machine learning algorithms, Artificial Neural Network (ANN) and Random Forest (RF), were applied. For the performance of classifying the occurrence of precipitation, the RF algorithm has higher accuracy in classifying the occurrence of precipitation than the ANN algorithm. The F1-score and Accuracy values, which are evaluation indicators of the classification model, were calculated as 0.80 and 0.77, while the ANN was calculated as 0.76 and 0.71. In addition, the performance of estimating precipitation also showed higher accuracy in RF than in ANN algorithm. The RMSE of the RF and ANN algorithms was 2.8 mm/day and 2.9 mm/day, and the values were calculated as 0.68 and 0.73.

Evaluation of the Utility of SSG Algorithm for Image Restoration of Landsat-8 (Landsat 8호 영상 복원을 위한 SSG 기법 활용성 평가)

  • Lee, Mi Hee;Lee, Dalgeun;Yu, Jung Hum;Kim, Jinyoung
    • Korean Journal of Remote Sensing
    • /
    • v.36 no.5_4
    • /
    • pp.1231-1244
    • /
    • 2020
  • Landsat satellites are representative optical satellites that have observed the Earth's surface for a long-term, and are suitable for long-term changes such as disaster preparedness/recovery monitoring, land use change, change detection, and time series monitoring. In this paper, clouds and cloud shadows were detected using QA bands to detect and remove clouds simply and efficiently. Then, the missing area of the experimantal image is restorated through the SSG algorithm, which does not directly refer to the pixel value of the reference image, but performs restoration to the pixel value in the Experimental image. Through this study, we presented the possibility of utilizing the modified SSG algorithm by quantitatively and qualitatively evaluating information on variousl and cover conditions in the thermal wavelength band as well as the visible wavelength band observing the surface.

Missing Value Estimation and Sensor Fault Identification using Multivariate Statistical Analysis (다변량 통계 분석을 이용한 결측 데이터의 예측과 센서이상 확인)

  • Lee, Changkyu;Lee, In-Beum
    • Korean Chemical Engineering Research
    • /
    • v.45 no.1
    • /
    • pp.87-92
    • /
    • 2007
  • Recently, developments of process monitoring system in order to detect and diagnose process abnormalities has got the spotlight in process systems engineering. Normal data obtained from processes provide available information of process characteristics to be used for modeling, monitoring, and control. Since modern chemical and environmental processes have high dimensionality, strong correlation, severe dynamics and nonlinearity, it is not easy to analyze a process through model-based approach. To overcome limitations of model-based approach, lots of system engineers and academic researchers have focused on statistical approach combined with multivariable analysis such as principal component analysis (PCA), partial least squares (PLS), and so on. Several multivariate analysis methods have been modified to apply it to a chemical process with specific characteristics such as dynamics, nonlinearity, and so on.This paper discusses about missing value estimation and sensor fault identification based on process variable reconstruction using dynamic PCA and canonical variate analysis.

Restoration of Missing Data in Satellite-Observed Sea Surface Temperature using Deep Learning Techniques (딥러닝 기법을 활용한 위성 관측 해수면 온도 자료의 결측부 복원에 관한 연구)

  • Won-Been Park;Heung-Bae Choi;Myeong-Soo Han;Ho-Sik Um;Yong-Sik Song
    • Journal of the Korean Society of Marine Environment & Safety
    • /
    • v.29 no.6
    • /
    • pp.536-542
    • /
    • 2023
  • Satellites represent cutting-edge technology, of ering significant advantages in spatial and temporal observations. National agencies worldwide harness satellite data to respond to marine accidents and analyze ocean fluctuations effectively. However, challenges arise with high-resolution satellite-based sea surface temperature data (Operational Sea Surface Temperature and Sea Ice Analysis, OSTIA), where gaps or empty areas may occur due to satellite instrumentation, geographical errors, and cloud cover. These issues can take several hours to rectify. This study addressed the issue of missing OSTIA data by employing LaMa, the latest deep learning-based algorithm. We evaluated its performance by comparing it to three existing image processing techniques. The results of this evaluation, using the coefficient of determination (R2) and mean absolute error (MAE) values, demonstrated the superior performance of the LaMa algorithm. It consistently achieved R2 values of 0.9 or higher and kept MAE values under 0.5 ℃ or less. This outperformed the traditional methods, including bilinear interpolation, bicubic interpolation, and DeepFill v1 techniques. We plan to evaluate the feasibility of integrating the LaMa technique into an operational satellite data provision system.

A development of multisite hourly rainfall simulation technique based on neyman-scott rectangular pulse model (Neyman-Scott Rectangular Pulse 모형 기반의 다지점 강수모의 기법 개발)

  • Moon, Jangwon;Kim, Janggyeong;Moon, Youngil;Kwon, Hyunhan
    • Journal of Korea Water Resources Association
    • /
    • v.49 no.11
    • /
    • pp.913-922
    • /
    • 2016
  • A long-term precipitation record is typically required for establishing the reliable water resources plan in the watershed. However, the observations in the hourly precipitation data are not always consistent and there are missing values within the time series. This study aims to develop a hourly rainfall simulator for extending rainfall data, based on the well-known Neyman-Scott Rectangular Pulse Model (NSRPM). Moreover, this study further suggests a multisite hourly rainfall simulator to better reproduce areal rainfalls for the watershed. The proposed model was validated with a network of five weather stations in the Uee-stream watershed in Seoul. The proposed model appeared a reasonable result in terms of reproducing most of the statistics (i.e. mean, variance and lag-1 autocovariance) of the rainfall time series at various aggregation levels and the spatial coherence over the weather stations.

Imputation of missing precipitation data using machine learning algorithms (머신러닝 알고리즘을 이용한 결측 강우 데이터 추정에 관한 연구)

  • Heechan Han
    • Proceedings of the Korea Water Resources Association Conference
    • /
    • 2023.05a
    • /
    • pp.320-320
    • /
    • 2023
  • 강우 데이터는 수문기상, 환경, 농업, 자연재해, 그리고 수자원 시스템 분야에서 가장 필수적인 기본 요소 중 하나이다. 또한 강우 데이터는 수문학적 분석에서 활용되는 필수 입력 자료 중 하나로 관측 데이터의 품질에 따라 수문 모형을 이용한 모의 결과물의 정확도가 결정된다고 할 수 있다. 따라서, 강우 관측소별로 강우 데이터의 품질을 어떻게 관리하느냐에 따라 수문 모형의 활용 범위 및 수자원 관리의 효율성이 결정될 수 있다. 강우의 시공간적 변동성은 수 많은 인자들과 직간접적으로 연계되어 있기 때문에 미계측 강우 자료에 대해 직접 관측이 아닌 수치 모형을 이용하여 강우의 발생과 강우량을 산정하는 것은 매우 복잡한 과제 중 하나이다. 현재 국내에서 운용되고 있는 강우 관측소의 경우에도 미계측 된 강우 데이터가 존재함으로써 강우 데이터의 활용에 제한이 생기는 경우가 있다. 따라서, 이러한 미계측 데이터의 추정 및 보완은 보다 효과적인 수재해 방지, 수자원 관리를 위한 필수 과제 중 하나이다. 일반적으로, 미계측 강우를 산정하기 위해서 Kriging, Thiessen, 등우선법, 그리고 역거리 관측법 등 다양한 수문학적 방법들이 적용되고 있다. 이러한 방법들은 산악효과나 강우 관측소의 분포 상태 등을 고려하지 못하기 때문에 측정하는 지역에 따라 강우 추정 오차가 커질 수 있다는 한계가 있다. 최근에는 데이터 관측 시스템과 빅데이터 기술의 발전과 활용 가능한 데이터의 양이 증가함에 따라 머신러닝을 활용한 사례가 증가하고 있다. 머신러닝은 데이터 사이의 관계를 기반으로 분류, 회귀, 그리고 예측 문제에 주로 사용되는 기법 중 하나이다. 따라서, 본 연구에서는 광주광역시 지역에 위치한 주요 강우 관측 지점들을 대상으로 미계측 된 시강우 데이터를 추정 및 복원하고자 한다. 여기서 데이터 추정 기술이란 미계측 강우의 발생 유무 및 강우량을 추정할 수 있는 기술을 의미한다. 이를 위해 대표적인 머신러닝 알고리즘인 인공신경망(Artificial Neural Network) 및 랜덤포레스트(Random Forest)를 적용하였다.

  • PDF

A Real-time Correction of the Underestimation Noise for GK2A Daily NDVI (GK2A 일단위 NDVI의 과소추정 노이즈 실시간 보정)

  • Lee, Soo-Jin;Youn, Youjeong;Sohn, Eunha;Kim, Mija;Lee, Yangwon
    • Korean Journal of Remote Sensing
    • /
    • v.38 no.6_1
    • /
    • pp.1301-1314
    • /
    • 2022
  • Normalized Difference Vegetation Index (NDVI) is utilized as an indicator to represent the vegetation condition on the land surface in various applications such as land cover, crop yield, agricultural drought, soil moisture, and forest disaster. However, satellite optical sensors for visible and infrared rays cannot see through the clouds, so the NDVI of the cloud pixel is not a valid value for the land surface. This study proposed a real-time correction of the underestimation noise for GEO-KOMPSAT-2A (GK2A) daily NDVI and made sure its feasibility through the quantitative comparisons with Moderate Resolution Imaging Spectroradiometer (MODIS) NDVI and the qualitative interpretation of time-series changes. The underestimation noise was effectively corrected by the procedures such as the time-series correction considering vegetation phenology, the outlier removal using long-term climatology, and the gap filling using rigorous statistical methods. The correlation with MODIS NDVI was higher, and the difference was lower, showing a 32.7% improvement compared to the original NDVI product. The proposed method has an extensibility for use in other satellite products with some modification.

Evaluation of the Satellite-based Air Temperature for All Sky Conditions Using the Automated Mountain Meteorology Station (AMOS) Records: Gangwon Province Case Study (산악기상관측정보를 이용한 위성정보 기반의 전천후 기온 자료의 평가 - 강원권역을 중심으로)

  • Jang, Keunchang;Won, Myoungsoo;Yoon, Sukhee
    • Korean Journal of Agricultural and Forest Meteorology
    • /
    • v.19 no.1
    • /
    • pp.19-26
    • /
    • 2017
  • Surface air temperature ($T_{air}$) is a key variable for the meteorology and climatology, and is a fundamental factor of the terrestrial ecosystem functions. Satellite remote sensing from the Moderate Resolution Imaging Spectroradiometer (MODIS) provides an opportunity to monitor the $T_{air}$. However, the several problems such as frequent cloud cover and mountainous region can result in substantial retrieval error and signal loss in MODIS $T_{air}$. In this study, satellite-based $T_{air}$ was estimated under both clear and cloudy sky conditions in Gangwon Province using Aqua MODIS07 temperature profile product (MYD07_L2) and GCOM-W1 Advanced Microwave Scanning Radiometer 2 (AMSR2) brightness temperature ($T_b$) at 37 GHz frequency, and was compared with the measurements from the Automated Mountain Meteorology Stations (AMOS). The application of ambient temperature lapse rate was performed to improve the retrieval accuracy in mountainous region, which showed the improvement of estimation accuracy approximately 4% of RMSE. A simple pixel-wise regression method combining synergetic information from MYD07_L2 $T_{air}$ and AMSR2 $T_b$ was applied to estimate surface $T_{air}$ for all sky conditions. The $T_{air}$ retrievals showed favorable agreement in comparison with AMOS data (r=0.80, RMSE=7.9K), though the underestimation was appeared in winter season. Substantial $T_{air}$ retrievals were estimated 61.4% (n=2,657) for cloudy sky conditions. The results presented in this study indicate that the satellite remote sensing can produce the surface $T_{air}$ at the complex mountainous region for all sky conditions.

A Study on Improvement Plans for Local Safety Assessment in Korea (국내 지역안전도 평가의 개선방안 연구)

  • Kim, Yong-Moon
    • Journal of Korean Society of Disaster and Security
    • /
    • v.14 no.4
    • /
    • pp.69-80
    • /
    • 2021
  • This study tried to suggest improvement measures by discovering problems or matters requiring improvement among the annual regional safety evaluation systems. Briefly introducing the structure and contents of the study, which is the introduction, describes the regional safety evaluation method newly applied by the Ministry of Public Administration and Security in 2020. Utilization plans were also introduced according to the local safety level that was finally evaluated by the local government. In this paper, various views of previous researchers related to regional safety are summarized and described. In addition, problems were drawn in the composition of the index of local safety, the method of calculating the index, and the application of the current index. Next, the problems of specific regional safety evaluation indicators were analyzed and solutions were presented. First, "Number of semi-basement households" is replaced with "Number of households receiving basic livelihood" of 「Social Vulnerability Index」 in the field of disaster risk factors is replaced with "the number of households receiving basic livelihood". In addition, the "Vinyl House Area" is evaluated by replacing "the number of households living in a Vinyl House, the number of container households, and the number of households in Jjok-bang villages" with data. Second, in the management and evaluation of habitual drought disaster areas, local governments with a water supply rate of 95% or higher in Counties, Cities, and Districts are treated as "missing". This is because drought disasters rarely occur in the metropolitan area and local governments that have undergone urbanization. Third, the activities of safety sheriffs, safety monitor volunteers, and disaster safety silver monitoring groups along with the local autonomous prevention foundation are added to the evaluation of the evaluation index of 「Regional Autonomous Prevention Foundation Activation」 in the field of response to disaster prevention measures. However, since the name of the local autonomous disaster prevention organization may be different for each local government, if it is an autonomous disaster prevention organization organized and active for disaster prevention, it would be appropriate to evaluate the results by summing up all of its activities. Fourth, among the Scorecard evaluation items, which is a safe city evaluation tool used by the United Nations Office for Disaster Risk Reduction(UNDRR), the item "preservation of natural buffers to strengthen the protection functions provided by natural ecosystems" is borrowed, which is closely related to natural disasters. The Scorecard evaluation is an assessment index that focuses on improving the disaster resilience of local governments while carrying out the campaign "Creating cities resilient to climate crises and disasters" emphasized by UNDRR. Finally, the names of "regional safety level" and "local safety index" are similar, so the term of local safety level is changed to "natural disaster safety level" or "natural calamity safety level". This is because only the general public can distinguish the local safety level from the local safety index.