• Title/Summary/Keyword: statistical correction

Search Result 310, Processing Time 0.025 seconds

A Dynamic Correction Technique of Time-Series Data using Anomaly Detection Model based on LSTM-GAN (LSTM-GAN 기반 이상탐지 모델을 활용한 시계열 데이터의 동적 보정기법)

  • Hanseok Jeong;Han-Joon Kim
    • The Journal of the Institute of Internet, Broadcasting and Communication
    • /
    • v.23 no.2
    • /
    • pp.103-111
    • /
    • 2023
  • This paper proposes a new data correction technique that transforms anomalies in time series data into normal values. With the recent development of IT technology, a vast amount of time-series data is being collected through sensors. However, due to sensor failures and abnormal environments, most of time-series data contain a lot of anomalies. If we build a predictive model using original data containing anomalies as it is, we cannot expect highly reliable predictive performance. Therefore, we utilizes the LSTM-GAN model to detect anomalies in the original time series data, and combines DTW (Dynamic Time Warping) and GAN techniques to replace the anomaly data with normal data in partitioned window units. The basic idea is to construct a GAN model serially by applying the statistical information of the window with normal distribution data adjacent to the window containing the detected anomalies to the DTW so as to generate normal time-series data. Through experiments using open NAB data, we empirically prove that our proposed method outperforms the conventional two correction methods.

A copula based bias correction method of climate data

  • Gyamfi Kwame Adutwum;Eun-Sung Chung
    • Proceedings of the Korea Water Resources Association Conference
    • /
    • 2023.05a
    • /
    • pp.160-160
    • /
    • 2023
  • Generally, Global Climate Models (GCM) cannot be used directly due to their inherent error arising from over or under-estimation of climate variables compared to the observed data. Several bias correction methods have been devised to solve this problem. Most of the traditional bias correction methods are one dimensional as they bias correct the climate variables separately. One such method is the Quantile Mapping method which builds a transfer function based on the statistical differences between the GCM and observed variables. Laux et al. introduced a copula-based method that bias corrects simulated climate data by employing not one but two different climate variables simultaneously and essentially extends the traditional one dimensional method into two dimensions. but it has some limitations. This study uses objective functions to address specifically, the limitations of Laux's methods on the Quantile Mapping method. The objective functions used were the observed rank correlation function, the observed moment function and the observed likelihood function. To illustrate the performance of this method, it is applied to ten GCMs for 20 stations in South Korea. The marginal distributions used were the Weibull, Gamma, Lognormal, Logistic and the Gumbel distributions. The tested copula family include most Archimedean copula families. Five performance metrics are used to evaluate the efficiency of this method, the Mean Square Error, Root Mean Square Error, Kolmogorov-Smirnov test, Percent Bias, Nash-Sutcliffe Efficiency and the Kullback Leibler Divergence. The results showed a significant improvement of Laux's method especially when maximizing the observed rank correlation function and when maximizing a combination of the observed rank correlation and observed moments functions for all GCMs in the validation period.

  • PDF

Turbid water atmospheric correction for GOCI: Modification of MUMM algorithm (GOCI영상의 탁한 해역 대기보정: MUMM 알고리즘 개선)

  • Lee, Boram;Ahn, Jae Hyun;Park, Young-Je;Kim, Sang-Wan
    • Korean Journal of Remote Sensing
    • /
    • v.29 no.2
    • /
    • pp.173-182
    • /
    • 2013
  • The early Sea-viewing Wide Field-of-view Sensor(SeaWiFS) atmospheric correction algorithm which is the basis of the atmospheric correction algorithm for Geostationary Ocean Color Imager(GOCI) assumes that water-leaving radiances is negligible at near-infrared(NIR) wavelengths. For this reason, all of the satellite measured radiances at the NIR wavelengths are assigned to aerosol radiances. However that assumption would cause underestimation of water-leaving radiances if it were applied to turbid Case-2 waters. To overcome this problem, Management Unit of the North Sea Mathematical Models(MUMM) atmospheric correction algorithm has been developed for turbid waters. This MUMM algorithm introduces new parameter ${\alpha}$, representing the ratio of water-leaving reflectance at the NIR wavelengths. ${\alpha}$ is calculated by statistical method and is assumed to be constant throughout the study area. Using this algorithm, we can obtain comparatively accurate water-leaving radiances in the moderately turbid waters where the NIR water-leaving reflectance is less than approximately 0.01. However, this algorithm still underestimates the water-leaving radiances at the extremely turbid water since the ratio of water-leaving radiance at two NIR wavelengths, ${\alpha}$ is changed with concentration of suspended particles. In this study, we modified the MUMM algorithm to calculate appropriate value for ${\alpha}$ using an iterative technique. As a result, the accuracy of water-leaving reflectance has been significantly improved. Specifically, the results show that the Root Mean Square Error(RMSE) of the modified MUMM algorithm was 0.002 while that of the MUMM algorithm was 0.0048.

Evaluation of Image for Phantom according to Normalization, Well Counter Correction in PET-CT (PET-CT Normalization, Well Counter Correction에 따른 팬텀을 이용한 영상 평가)

  • Choong-Woon Lee;Yeon-Wook You;Jong-Woon Mun;Yun-Cheol Kim
    • The Korean Journal of Nuclear Medicine Technology
    • /
    • v.27 no.1
    • /
    • pp.47-54
    • /
    • 2023
  • Purpose PET-CT imaging require an appropriate quality assurance system to achieve high efficiency and reliability. Quality control is essential for improving the quality of care and patient safety. Currently, there are performance evaluation methods of UN2-1994 and UN2-2001 proposed by NEMA and IEC for PET-CT image evaluation. In this study, we compare phantom images with the same experiments before and after PET-CT 3D normalization and well counter correction and evaluate the usefulness of quality control. Materials and methods Discovery 690 (General Electric Healthcare, USA) PET-CT equiptment was used to perform 3D normalization and well counter correction as recommended by GE Healthcare. Based on the recovery coefficients for the six spheres of the NEMA IEC Body Phantom recommended by the EARL. 20kBq/㎖ of 18F was injected into the sphere of the phantom and 2kBq/㎖ of 18F was injected into the body of phantom. PET-CT scan was performed with a radioacitivity ratio of 10:1. Images were reconstructed by appliying TOF+PSF+TOF, OSEM+PSF, OSEM and Gaussian filter 4.0, 4.5, 5.0, 5.5, 6.0, 6,5 mm with matrix size 128×128, slice thickness 3.75 mm, iteration 2, subset 16 conditions. The PET image was attenuation corrected using the CT images and analyzed using software program AW 4.7 (General Electric Healthcare, USA). The ROI was set to fit 6 spheres in the CT image, RC (Recovery Coefficient) was measured after fusion of PET and CT. Statistical analysis was performed wilcoxon signed rank test using R. Results Overall, after the quality control items were performed, the recovery coefficient of the phantom image increased and measured. Recovery coefficient according to the image reconstruction increased in the order TOF+PSF, TOF, OSEM+PSF, before and after quality control, RCmax increased by OSEM 0.13, OSEM+PSF 0.16, TOF 0.16, TOF+PSF 0.15 and RCmean increased by OSEM 0.09, OSEM+PSF 0.09, TOF 0.106, TOF+PSF 0.10. Both groups showed a statistically significant difference in Wilcoxon signed rank test results (P value<0.001). Conclusion PET-CT system require quality assurance to achieve high efficiency and reliability. Standardized intervals and procedures should be followed for quality control. We hope that this study will be a good opportunity to think about the importance of quality control in PET-CT

  • PDF

Improvement of Atmospheric Dispersion Model Performance by Pretreatment of Dispersion Coefficients (분산계수의 전처리에 의한 대기분산모델 성능의 개선)

  • Park, Ok-Hyun;Kim, Gyung-Soo
    • Journal of Korean Society for Atmospheric Environment
    • /
    • v.23 no.4
    • /
    • pp.449-456
    • /
    • 2007
  • Dispersion coefficient preprocessing schemes have been examined to improve plume dispersion model performance in complex coastal areas. The performances of various schemes for constructing the sigma correction order were evaluated through estimations of statistical measures, such as bias, gross error, R, FB, NMSE, within FAC2, MG, VG, IOA, UAPC and MRE. This was undertaken for the results of dispersion modeling, which applied each scheme. Environmental factors such as sampling time, surface roughness, plume rising, plume height and terrain rolling were considered in this study. Gaussian plume dispersion model was used to calculate 1 hr $SO_2$ concentration 4 km downwind from a power plant in Boryeung coastal area. Here, measured data for January to December of 2002 were obtained so that modelling results could be compared. To compare the performances between various schemes, integrated scores of statistical measures were obtained by giving weights for each measure and then summing each score. This was done because each statistical measure has its own function and criteria; as a result, no measure can be taken as a sole index indicative of the performance level for each modeling scheme. The best preprocessing scheme was discerned using the step-wise method. The most significant factor influencing the magnitude of real dispersion coefficients appeared to be sampling time. A second significant factor appeared to be surface roughness, with the rolling terrain being the least significant for elevated sources in a gently rolling terrain. The best sequence of correcting the sigma from P-G scheme was found to be the combination of (1) sampling time, (2) surface roughness, (3) plume rising, (4) plume height, and (5) terrain rolling.

Analysis of Refractive Error according to Aging in Young Children in South Korea (한국 어린이의 연령에 따른 굴절이상 분석)

  • Kim, Douk-Hoon;Kim, Dae-Nyoun
    • Journal of Korean Clinical Health Science
    • /
    • v.2 no.4
    • /
    • pp.239-246
    • /
    • 2014
  • Purpose. to describe the prevalence of refractive error according to aging in young children in South Korea. Methods : From July 2013 to June 2014, five hundred subjects( 250 male subjects, 250 female subjects; aged between 7 and 12 years) were performed in refraction test using the Auto-Refraction. Myopia, hyperopia, astigmatism, and anisometropia were defined as spherical equivalent(SE)${\leq}-0.50$ diopters, SE ${\geq}+2.00$ D, cylinder error ${\geq}0.75$ D and SE difference${\geq}1.00$ D between binocular eyes, respectively. Results. The refractive error by spherical equivalent among all subjects was myopia 80.41%, astigmatism 44.89%, emmetropia 18.27%, anisometropia 16.92%, and hyperopia 1.32%. The prevalence of myopia increased with age. but hyperopia was decrease. Myopia and astigmatism were much more common in male than females although the difference was not statically significant. Emmetropia and hyperopia were much more common in female than males although the difference was not statically significant. The prevalence of spherical equivalent was much common from -1.00 diopter to 0.99 diopter. On the other hand, the prevalence of myopia was much more than hyperopia. There was a statistical significance between 9 year and 10 year of female in the spherical equivalent power(p>0.05). In all another group of age, there was not a statistical significance as aging in spherical equivalent power(p>0.5). However, there was a statistical significance between male and female as age in the spherical equivalent power(P>0.01). Conclusions: Myopia was the most common refractive error in Korea young children, while hyperopia was decreased after 7 years. There was a statistical significance as age between male and female at spherical equivalent power(P> 0.01). these results suggested that the analysis of the refractive error as age at young children can give the useful diagnosis data for the correction of visual function.

Proposal to Supplement the Missing Values of Air Pollution Levels in Meteorological Dataset (기상 데이터에서 대기 오염도 요소의 결측치 보완 기법 제안)

  • Jo, Dong-Chol;Hahn, Hee-Il
    • The Journal of the Institute of Internet, Broadcasting and Communication
    • /
    • v.21 no.1
    • /
    • pp.181-187
    • /
    • 2021
  • Recently, various air pollution factors have been measured and analyzed to reduce damages caused by it. In this process, many missing values occur due to various causes. To compensate for this, basically a vast amount of training data is required. This paper proposes a statistical techniques that effectively compensates for missing values generated in the process of measuring ozone, carbon dioxide, and ultra-fine dust using a small amount of learning data. The proposed algorithm first extracts a group of meteorological data that is expected to have positive effects on the correction of missing values through statistical information analysis such as the correlation between meteorological data and air pollution level factors, p-value, etc. It is a technique that efficiently and effectively compensates for missing values by analyzing them. In order to confirm the performance of the proposed algorithm, we analyze its characteristics through various experiments and compare the performance of the well-known representative algorithms with ours.

Development of Simulated HPGe Detector Spectrum for Education (교육용 모사 HPGe 검출기 스펙트럼 개발)

  • Seo, Kyung-Won;Lee, Mo-Sung
    • Journal of Radiation Protection and Research
    • /
    • v.32 no.1
    • /
    • pp.9-13
    • /
    • 2007
  • From HPGe calibration spectrum of liquid mixed source in cylindrical vial, we developed simulated spectrum for spectrum analysis education. It is the spectrum that combine peaks separated from measured spectrum. After that, spectrum removed statistical variation of channel counts. Statistical fluctuation of the spectrum is made by Box-Muller function. The spectrum contains 18 peaks. The peak's centroid and area were defined exactly. Developed spectra are calibration spectrum, sample spectrum, background spectrum and spectra for efficiency correction for geometry and cascade coincidence.

Influence of Slashpipe Exercise on Symmetrical Contraction of Trunk Muscle in Normal Adults

  • Choi, Young In;Kim, Jung Sun;Kim, Shin Young
    • The Journal of Korean Physical Therapy
    • /
    • v.31 no.5
    • /
    • pp.298-303
    • /
    • 2019
  • Purpose: This study examined the effects of slashpipe exercise on reducing the thickness of the left and right external oblique, internal oblique, transverse abdominis, erector spinae, and multifidus muscles. Methods: A total of 29 healthy adult men and women were included in the study. They performed trunk flexion in the supine position and trunk extension in the prone position with a slashpipe and weight bar. The external oblique, internal oblique, and transverse abdominis muscles were measured in the supine position, while the erector spinae and multifidus muscles were measured in the prone position. The data were analyzed using the SPSS ver 21.0 statistical program. The difference in thickness between the right and left sides of the trunk muscle was analyzed by repeated measures analysis. The statistical significance level was set to p<0.05. Results: The results showed that the slashpipe exercise reduced significantly the difference in thickness of the oblique internus and erector spinae muscles compared to the weight bar exercise. Conclusion: The chaotic fluidity of the fluid filled inside the slashpipe could be used as sensory feedback information on body mal-alignment, which would have positively affected the symmetrical contraction of the trunk muscles as a trigger for self-correction. Therefore, it will have a useful effect not only on the health of the general public, but also on low back patients and athletes with muscle asymmetry.

Development of the Expert Seasonal Prediction System: an Application for the Seasonal Outlook in Korea

  • Kim, WonMoo;Yeo, Sae-Rim;Kim, Yoojin
    • Asia-Pacific Journal of Atmospheric Sciences
    • /
    • v.54 no.4
    • /
    • pp.563-573
    • /
    • 2018
  • An Expert Seasonal Prediction System for operational Seasonal Outlook (ESPreSSO) is developed based on the APEC Climate Center (APCC) Multi-Model Ensemble (MME) dynamical prediction and expert-guided statistical downscaling techniques. Dynamical models have improved to provide meaningful seasonal prediction, and their prediction skills are further improved by various ensemble and downscaling techniques. However, experienced scientists and forecasters make subjective correction for the operational seasonal outlook due to limited prediction skills and biases of dynamical models. Here, a hybrid seasonal prediction system that grafts experts' knowledge and understanding onto dynamical MME prediction is developed to guide operational seasonal outlook in Korea. The basis dynamical prediction is based on the APCC MME, which are statistically mapped onto the station-based observations by experienced experts. Their subjective selection undergoes objective screening and quality control to generate final seasonal outlook products after physical ensemble averaging. The prediction system is constructed based on 23-year training period of 1983-2005, and its performance and stability are assessed for the independent 11-year prediction period of 2006-2016. The results show that the ESPreSSO has reliable and stable prediction skill suitable for operational use.