• Title/Summary/Keyword: bias correction

Search Result 266, Processing Time 0.03 seconds

Bias Correction of Satellite-Based Precipitation Using Convolutional Neural Network

  • Le, Xuan-Hien;Lee, Gi Ha
    • Proceedings of the Korea Water Resources Association Conference
    • /
    • 2020.06a
    • /
    • pp.120-120
    • /
    • 2020
  • Spatial precipitation data is one of the essential components in modeling hydrological problems. The estimation of these data has achieved significant achievements own to the recent advances in remote sensing technology. However, there are still gaps between the satellite-derived rainfall data and observed data due to the significant dependence of rainfall on spatial and temporal characteristics. An effective approach based on the Convolutional Neural Network (CNN) model to correct the satellite-derived rainfall data is proposed in this study. The Mekong River basin, one of the largest river system in the world, was selected as a case study. The two gridded precipitation data sets with a spatial resolution of 0.25 degrees used in the CNN model are APHRODITE (Asian Precipitation - Highly-Resolved Observational Data Integration Towards Evaluation) and PERSIANN-CDR (Precipitation Estimation from Remotely Sensed Information using Artificial Neural Networks). In particular, PERSIANN-CDR data is exploited as satellite-based precipitation data and APHRODITE data is considered as observed rainfall data. In addition to developing a CNN model to correct the satellite-based rain data, another statistical method based on standard deviations for precipitation bias correction was also mentioned in this study. Estimated results indicate that the CNN model illustrates better performance both in spatial and temporal correlation when compared to the standard deviation method. The finding of this study indicated that the CNN model could produce reliable estimates for the gridded precipitation bias correction problem.

  • PDF

A selection of optimal method for bias-correction in Global Seasonal Forecast System version 5 (GloSea5) (전지구 계절예측시스템 GloSea5의 최적 편의보정기법 선정)

  • Son, Chanyoung;Song, Junghyun;Kim, Sejin;Cho, Younghyun
    • Journal of Korea Water Resources Association
    • /
    • v.50 no.8
    • /
    • pp.551-562
    • /
    • 2017
  • In order to utilize 6-month precipitation forecasts (6 months at maximum) of Global Seasonal Forecast System version 5 (GloSea5), which is being provided by KMA (Korea Meteorological Administration) since 2014, for water resources management as well as other applications, it is needed to correct the forecast model's quantitative bias against observations. This study evaluated applicability of bias-correction skill in GloSea5 and selected an optimal method among 11 techniques that include probabilistic distribution type based, parametric, and non-parametric bias-correction to fix GloSea5's bias in precipitation forecasts. Non-parametric bias-correction provided the most similar results with observed data compared to other techniques in hindcast for the past events, yet relatively generated some discrepancies in forecast. On the contrary, parametric bias-correction produced the most reliable results in both hindcast and forecast periods. The results of this study are expected to be applicable to various applications using seasonal forecast model such as water resources operation and management, hydropower, agriculture, etc.

Analysis of Rainfall-Runoff Characteristics on Bias Correction Method of Climate Change Scenarios (기후변화 시나리오 편의보정 기법에 따른 강우-유출 특성 분석)

  • Kum, Donghyuk;Park, Younsik;Jung, Young Hun;Shin, Min Hwan;Ryu, Jichul;Park, Ji Hyung;Yang, Jae E;Lim, Kyoung Jae
    • Journal of Korean Society on Water Environment
    • /
    • v.31 no.3
    • /
    • pp.241-252
    • /
    • 2015
  • Runoff behaviors by five bias correction methods were analyzed, which were Change Factor methods using past observed and estimated data by the estimation scenario with average annual calibration factor (CF_Y) or with average monthly calibration factor (CF_M), Quantile Mapping methods using past observed and estimated data considering cumulative distribution function for entire estimated data period (QM_E) or for dry and rainy season (QM_P), and Integrated method of CF_M+QM_E(CQ). The peak flow by CF_M and QM_P were twice as large as the measured peak flow, it was concluded that QM_P method has large uncertainty in monthly runoff estimation since the maximum precipitation by QM_P provided much difference to the other methods. The CQ method provided the precipitation amount, distribution, and frequency of the smallest differences to the observed data, compared to the other four methods. And the CQ method provided the rainfall-runoff behavior corresponding to the carbon dioxide emission scenario of SRES A1B. Climate change scenario with bias correction still contained uncertainty in accurate climate data generation. Therefore it is required to consider the trend of observed precipitation and the characteristics of bias correction methods so that the generated precipitation can be used properly in water resource management plan establishment.

A Numerical Study on Temperature Prediction Bias using FDS in Simulated Thermal Environments of Fire (모사된 화재의 열적환경에서 FDS를 이용한 온도 예측오차에 관한 수치해석 연구)

  • Han, Ho-Sik;Kim, Bong-Jun;Hwang, Cheol-Hong
    • Journal of the Korean Society of Safety
    • /
    • v.32 no.2
    • /
    • pp.14-20
    • /
    • 2017
  • A numerical study was conducted to identify the predictive performance for the bare-bead thermocouple (TC) using FDS (Fire Dynamics Simulator) in simulated thermal environments of fire. A relative prediction bias of TC temperature calculated from reverse-radiation correction by FDS was evaluated with the comparison of previous experimental data. As a result, it was identified that the TC temperatures predicted by FDS were lower than the temperatures measured by bare-bead TC for the ranges of heat flux and gas temperature considered. The relative prediction bias of TC temperature by FDS was gradually increased with the increase in radiative heat flux and also significantly increased with the decrease in the gas temperature. Quantitatively, at the gas temperature of $20^{\circ}C$, the TC temperature predicted by FDS had the relative bias of approximately -20% with the radiative heat flux of $20kW/m^2$ corresponding to thermal radiation level of the flashover. It is predicted from the present study that more accurate validation of fire modeling will be possible with the quantitative prediction bias occurred in the process of reverse-radiation correction of temperature predicted by FDS.

Comparison of Bias Correction Methods for the Rare Event Logistic Regression (희귀 사건 로지스틱 회귀분석을 위한 편의 수정 방법 비교 연구)

  • Kim, Hyungwoo;Ko, Taeseok;Park, No-Wook;Lee, Woojoo
    • The Korean Journal of Applied Statistics
    • /
    • v.27 no.2
    • /
    • pp.277-290
    • /
    • 2014
  • We analyzed binary landslide data from the Boeun area with logistic regression. Since the number of landslide occurrences is only 9 out of 5000 observations, this can be regarded as a rare event data. The main issue of logistic regression with the rare event data is a serious bias problem in regression coefficient estimates. Two bias correction methods were proposed before and we quantitatively compared them via simulation. Firth (1993)'s approach outperformed and provided the most stable results for analyzing the rare-event binary data.

A Study on Bias Effect on Model Selection Criteria in Graphical Lasso

  • Choi, Young-Geun;Jeong, Seyoung;Yu, Donghyeon
    • Quantitative Bio-Science
    • /
    • v.37 no.2
    • /
    • pp.133-141
    • /
    • 2018
  • Graphical lasso is one of the most popular methods to estimate a sparse precision matrix, which is an inverse of a covariance matrix. The objective function of graphical lasso imposes an ${\ell}_1$-penalty on the (vectorized) precision matrix, where a tuning parameter controls the strength of the penalization. The selection of the tuning parameter is practically and theoretically important since the performance of the estimation depends on an appropriate choice of tuning parameter. While information criteria (e.g. AIC, BIC, or extended BIC) have been widely used, they require an asymptotically unbiased estimator to select optimal tuning parameter. Thus, the biasedness of the ${\ell}_1$-regularized estimate in the graphical lasso may lead to a suboptimal tuning. In this paper, we propose a two-staged bias-correction procedure for the graphical lasso, where the first stage runs the usual graphical lasso and the second stage reruns the procedure with an additional constraint that zero estimates at the first stage remain zero. Our simulation and real data example show that the proposed bias correction improved on both edge recovery and estimation error compared to the single-staged graphical lasso.

A study of the Sampling Bias Correction on Joint Data from 1D Survey Line (1D 측선에 의한 절리 자료에 대한 편향 보정 기법에 관한 연구)

  • 엄정기
    • Tunnel and Underground Space
    • /
    • v.13 no.5
    • /
    • pp.344-352
    • /
    • 2003
  • The procedures to correct sampling biases for discontinuity data obtained from 1D survey line(borehole or scanline) is addressed. The Probability of intersection between the survey line and a circular discontinuity is considered, and a correction far orientation bias is developed assuming discontinuities as equivalent circular disks. The correction incorporates the effect of the angle between the direction of survey line and each discontinuity plane belonging to the discontinuity cluster, size of each discontinuity and length of the survey line. A procedure is provided to estimate unbiased discontinuity spacing parameters using the discontinuity spacing data based on the measurements carried out on a finite length of the survey line.

A Simple Bias-Correction Rule for the Apparent Prediction Error

  • Beong-Soo So
    • Communications for Statistical Applications and Methods
    • /
    • v.2 no.2
    • /
    • pp.146-154
    • /
    • 1995
  • By using simple Taylor expansion, we derive an easy bias-correction rule for the apparent prodiction error of the predictor defined by the general M-estimators with respect to an arbitrary measure of prediction error. Our method has a considerable computational advantage over the previous methods based on the resampling thchnique such as Cross-validaton and Boothtrap. Connections with AIC, Cross-Validation and Boothtrap are discussed too.

  • PDF

Model Performance Evaluation and Bias Correction Effect Analysis for Forecasting PM2.5 Concentrations (PM2.5 예보를 위한 모델 성능평가와 편차보정 효과 분석)

  • Ghim, Young Sung;Choi, Yongjoo;Kim, Soontae;Bae, Chang Han;Park, Jinsoo;Shin, Hye Jung
    • Journal of Korean Society for Atmospheric Environment
    • /
    • v.33 no.1
    • /
    • pp.11-18
    • /
    • 2017
  • The performance of a modeling system consisting of WRF model v3.3 and CMAQ model v4.7.1 for forecasting $PM_{2.5}$ concentrations were evaluated during the period May 2012 through December 2014. Twenty-four hour averages of $PM_{2.5}$ and its major components obtained through filter sampling at the Bulgwang intensive measurement station were used for comparison. The mean predicted $PM_{2.5}$ concentration over the entire period was 68% of the mean measured value. Predicted concentrations for major components were underestimated except for $NO_3{^-}$. The model performance for $PM_{2.5}$ generally tended to degrade with increasing the concentration level. However, the mean fractional bias (MFB) for high concentration above the $80^{th}$ percentile fell within the criteria, the level of accuracy acceptable for standard model applications. Among three bias correction methods, the ratio adjustment was generally most effective in improving the performance. Albeit for limited test conditions, this analysis demonstrated that the effects of bias correction were larger when using the data with a larger bias of predicted values from measurement values.

A copula based bias correction method of climate data

  • Gyamfi Kwame Adutwum;Eun-Sung Chung
    • Proceedings of the Korea Water Resources Association Conference
    • /
    • 2023.05a
    • /
    • pp.160-160
    • /
    • 2023
  • Generally, Global Climate Models (GCM) cannot be used directly due to their inherent error arising from over or under-estimation of climate variables compared to the observed data. Several bias correction methods have been devised to solve this problem. Most of the traditional bias correction methods are one dimensional as they bias correct the climate variables separately. One such method is the Quantile Mapping method which builds a transfer function based on the statistical differences between the GCM and observed variables. Laux et al. introduced a copula-based method that bias corrects simulated climate data by employing not one but two different climate variables simultaneously and essentially extends the traditional one dimensional method into two dimensions. but it has some limitations. This study uses objective functions to address specifically, the limitations of Laux's methods on the Quantile Mapping method. The objective functions used were the observed rank correlation function, the observed moment function and the observed likelihood function. To illustrate the performance of this method, it is applied to ten GCMs for 20 stations in South Korea. The marginal distributions used were the Weibull, Gamma, Lognormal, Logistic and the Gumbel distributions. The tested copula family include most Archimedean copula families. Five performance metrics are used to evaluate the efficiency of this method, the Mean Square Error, Root Mean Square Error, Kolmogorov-Smirnov test, Percent Bias, Nash-Sutcliffe Efficiency and the Kullback Leibler Divergence. The results showed a significant improvement of Laux's method especially when maximizing the observed rank correlation function and when maximizing a combination of the observed rank correlation and observed moments functions for all GCMs in the validation period.

  • PDF