• Title/Summary/Keyword: 결측 복원

Search Result 19, Processing Time 0.03 seconds

Restoration, Prediction and Noise Analysis of Geomagnetic Time-series Data (시계열 지자기 측정 자료의 복원, 예측 및 잡음 분석 연구)

  • Ji, Yoon-Soo;Oh, Seok-Hoon;Suh, Baek-Soo;Lee, Duk-Kee
    • Journal of the Korean earth science society
    • /
    • v.32 no.6
    • /
    • pp.613-628
    • /
    • 2011
  • Restoration, prediction and noise analysis of geomagnetic data measured in the Korean Peninsula were performed. Restoration methods based on an optimized principal component analysis (PCA) and the geostatistical kriging approach were proposed, and its effectiveness was also interpreted. The PCA-based method seemed to be effective to restore the periodical signals and the geostatistical approach was stable to fill the gaps of measurements. To analyze the noise level for each observatory, the geomagnetic time-series was plotted by scattergram which reflects the spatial variation, using data observed during same period. The scattergram showed that the observation made at Cheongyang seemed to have better quality in spatial continuity and stability, and the restoration result was also better than that of Icheon site. For the restoration, both of the methods, geostatistical and optimizaed PCA, showed stable result when the missing of observation was within 20 points. However, in case of more missing observations than 20 points and prediction problem, the optimized PCA seemed to be closer to the real observation considering the frequency-domain characteristics. The prediction using the optimized PCA seems to be plausible for one day of period for interpretation.

A comparison of imputation methods for the consecutive missing temperature data (연속적 결측이 존재하는 기온 자료에 대한 결측복원 기법의 비교)

  • Kim, Hee-Kyung;Kang, In-Kyeong;Lee, Jae-Won;Lee, Yung-Seop
    • The Korean Journal of Applied Statistics
    • /
    • v.29 no.3
    • /
    • pp.549-557
    • /
    • 2016
  • Consecutive missing values are likely to occur in long climate data due to system error or defective equipment. Furthermore, it is difficult to impute missing values. However, these complicated problems can be overcame by imputing missing values with reference time series. Reference time series must be composed of similar time series to time series that include missing values. We performed a simulation to compare three missing imputation methods (the adjusted normal ratio method, the regression method and the IDW method) to complete the missing values of time series. A comparison of the three missing imputation methods for the daily mean temperatures at 14 climatological stations indicated that the IDW method was better thanx others at south seaside stations. We also found the regression method was better than others at most stations (except south seaside stations).

Application of DINEOF to Reconstruct the Missing Data from GOCI Chlorophyll-a (GOCI Chlorophyll-a 결측 자료의 복원을 위한 DINEOF 방법 적용)

  • Hwang, Do-Hyun;Jung, Hahn Chul;Ahn, Jae-Hyun;Choi, Jong-Kuk
    • Korean Journal of Remote Sensing
    • /
    • v.37 no.6_1
    • /
    • pp.1507-1515
    • /
    • 2021
  • If chlorophyll-a is estimated through ocean color remote sensing, it is able to understand the global distribution of phytoplankton and primary production. However, there are missing data in the ocean color observed from the satellites due to the clouds or weather conditions. In thisstudy, the missing data of the GOCI (Geostationary Ocean Color Imager) chlorophyll-a product wasreconstructed by using DINEOF (Data INterpolation Empirical Orthogonal Functions). DINEOF reconstructs the missing data based on spatio-temporal data, and the accuracy was cross-verified by removing a part of the GOCI chlorophyll-a image and comparing it with the reconstructed image. In the study area, the optimal EOF (Empirical Orthogonal Functions) mode for DINEOF wasin 10-13. The temporal and spatialreconstructed data reflected the increasing chlorophyll-a concentration in the afternoon, and the noise of outliers was filtered. Therefore, it is expected that DINEOF is useful to reconstruct the missing images, also it is considered that it is able to use as basic data for monitoring the ocean environment.

Evaluation of the DCT-PLS Method for Spatial Gap Filling of Gridded Data (격자자료 결측복원을 위한 DCT-PLS 기법의 활용성 평가)

  • Youn, Youjeong;Kim, Seoyeon;Jeong, Yemin;Cho, Subin;Lee, Yangwon
    • Korean Journal of Remote Sensing
    • /
    • v.36 no.6_1
    • /
    • pp.1407-1419
    • /
    • 2020
  • Long time-series gridded data is crucial for the analyses of Earth environmental changes. Climate reanalysis and satellite images are now used as global-scale periodical and quantitative information for the atmosphere and land surface. This paper examines the feasibility of DCT-PLS (penalized least square regression based on discrete cosine transform) for the spatial gap filling of gridded data through the experiments for multiple variables. Because gap-free data is required for an objective comparison of original with gap-filled data, we used LDAPS (Local Data Assimilation and Prediction System) daily data and MODIS (Moderate Resolution Imaging Spectroradiometer) monthly products. In the experiments for relative humidity, wind speed, LST (land surface temperature), and NDVI (normalized difference vegetation index), we made sure that randomly generated gaps were retrieved very similar to the original data. The correlation coefficients were over 0.95 for the four variables. Because the DCT-PLS method does not require ancillary data and can refer to both spatial and temporal information with a fast computation, it can be applied to operative systems for satellite data processing.

Development of gap filling technique for statistical downscaling of cimate change scenario data (기후변화 시나리오 자료의 통계적 상세화를 위한 결측자료 보정 기법 개발)

  • Cho, Jaepil;Kim, Kwang-Hyung;Park, Jihoon
    • Proceedings of the Korea Water Resources Association Conference
    • /
    • 2019.05a
    • /
    • pp.16-16
    • /
    • 2019
  • 기후변화 시나리오 및 계절예측 자료를 포함한 기후정보를 수자원 분야에 활용하기 위해서는 기후정보의 시 공간적인 상세화(donwscaling)을 필요로 한다. 상세화의 경우 역학적 상세화와 통계학적 상세화로 구분될 수 있으며, 통계학적 상세화를 위해서는 대상 지역의 기후특성을 대표할 수 있는 장기 관측 자료의 확보가 중요하다. 국내의 경우에는 자동기상관측장비(Automatic Weather System, AWS)와 종관기상관측장비(Automatic Synoptic Observation System, ASOS)로 부터 수집된 기상관측자료를 사용할 수 있으나 기후변화 시나리오의 통계적 상세화를 위해서는 30년 이상의 자료 기간을 포함하는 ASOS 자료가 적합하다. 하지만 개발도상국과 같이 기상관측기반이 열악한 지역에서는 잦은 결측 등으로 인하여 품질이 좋은 관측자료의 획득이 어려운 상황이다. 따라서 본 연구에서는 측이 포함된 장기 기상관측 자료로부터 대상 지역의 기후특성을 재현할 수 있도록 기본적인 QC(Quality Control)을 거쳐 결측 자료를 보완할 수 있는 기법 및 R 기반패키지를 개발하여 적용성을 평가하였다. 개발된 기법의 적용성 평가를 위해서 기상청에서 QC를 통해 제공하고 있는 60개 ASOS 지점의 관측자료 중 강수량과 기온 변수를 사용하였다. 최대 50%까지의 현실적인 결측 패턴을 임의로 생성하기 위해 실제 개발도상국 관측자료의 일단위 결측 패턴을 이용하였다. 자료의 QC는 관측일 누락/중복 및 문자형 관측값 등 기본적인 오류 검사, 기온의 경우 물리적 허용 범위에 대한 검사, 최고기온과 최저기온의 비교 및 계측기 오작동에 의한 동일한 값의 반복 등을 포함한 내적 일치성 검사를 우선적으로 수행한다. 이후 결측값에 대해서 인근 기상관측소와의 상관성 분석 결과를 기반으로 결측값을 채우고, 최종적으로는 다양한 위성자료 및 재분석 자료 중에서 일단위 기후특성의 재현성 평가를 통해 선정된 격자형 자료와의 상관성 분석 결과를 기반으로 결측값을 보정하였다. 기온의 경우는 결측률이 높더라도 월평균 기후특성에 큰 영향을 미치지 않았지만 강수의 경우에는 5% 이상의 결측이 발생하는 경우 월평균 강수량에 영향을 미쳐 지역의 강수량을 과소 추정하는 결과를 보였다. 개발된 QC 기법을 강수 자료에 적용한 결과 월평균 기후특성을 잘 복원하는 결과를 보였지만, 일단위 강우 사상의 재현에 있어서는 미흡한 결과를 보였다.

  • PDF

Cloud Detection and Restoration of Landsat-8 using STARFM (재난 모니터링을 위한 Landsat 8호 영상의 구름 탐지 및 복원 연구)

  • Lee, Mi Hee;Cheon, Eun Ji;Eo, Yang Dam
    • Korean Journal of Remote Sensing
    • /
    • v.35 no.5_2
    • /
    • pp.861-871
    • /
    • 2019
  • Landsat satellite images have been increasingly used for disaster damage analysis and disaster monitoring because they can be used for periodic and broad observation of disaster damage area. However, periodic disaster monitoring has limitation because of areas having missing data due to clouds as a characteristic of optical satellite images. Therefore, a study needs to be conducted for restoration of missing areas. This study detected and removed clouds and cloud shadows by using the quality assessment (QA) band provided when acquiring Landsat-8 images, and performed image restoration of removed areas through a spatial and temporal adaptive reflectance fusion (STARFM) algorithm. The restored image by the proposed method is compared with the restored image by conventional image restoration method throught MLC method. As a results, the restoration method by STARFM showed an overall accuracy of 89.40%, and it is confirmed that the restoration method is more efficient than the conventional image restoration method. Therefore, the results of this study are expected to increase the utilization of disaster analysis using Landsat satellite images.

Personalized Data Restoration Algorithm to Improve Wearable Device Service (웨어러블 디바이스 서비스 향상을 위한 개인 맞춤형 데이터 복원 알고리즘)

  • Kikun Park;Hye-Rim Bae
    • The Journal of Bigdata
    • /
    • v.6 no.2
    • /
    • pp.51-60
    • /
    • 2021
  • The market size of wearable devices is growing rapidly every year, and manufacturers around the world are introducing products that utilize their unique characteristics to keep up with the demand. Among them, smart watches are wearable devices with a very high share in sales, and they provide a variety of services to users by using information collected in real-time. The quality of service depends on the accuracy of the data collected by the smart watch, but data measurement may not be possible depending on the situation. This paper introduces a method to restore data that a smart watch could not collect. It deals with the similarity calculation method of trajectory information measured over time for data restoration and introduces a procedure for restoring missing sections according to the similarity. To prove the performance of the proposed methodology, a comparative experiment with a machine learning algorithm was conducted. Finally, the expected effects of this study and future research directions are discussed.

Comparison of the Editing Method of Missing Area in 3D Scanned Image of Men's Crotch (3차원 스캔한 인체 샅부위의 결측부위 복원 방법 비교)

  • Kim, So-Young;Hong, Kyung-Hi
    • Journal of the Korean Society of Clothing and Textiles
    • /
    • v.33 no.3
    • /
    • pp.401-409
    • /
    • 2009
  • The shape of crotch area is very important to develop functional clothing as well as other ergonomic goods such as chair or saddle etc. However, it is inevitable that 3D scanned image of crotch would have missing part due to its folded shape including overlapping legs nearby. Therefore, the objectives of this research was to compare reconstruction methods of missing parts at crotch using seven dummies of real men's replicas. Two reconstruction methods adopted were kinds of 'fill- hole' in Rapidform 2004, one was 'smooth' and the other was 'curvature'. Each restored image was compared with the original shape of the dummies. As results, the average distance was 0.66mm between original and 'smooth' treated images and 0.59mm between original and 'curvature' treated, which was not statistically different. Average area of restored crotch region was $8740.04cm^2$ by 'smooth' method and $8405.02cm^2$ by 'curvature' method which is close to the original area of $8413.76cm^2$. Statistical difference was found between images of original and 'smooth' ones$(p=0.04^*)$. However, there was no difference between original and 'curvature' treated images, which indicates that 'curvature' method is more useful to fill the hole compared with 'smooth' method.

Gap-Filling of Sentinel-2 NDVI Using Sentinel-1 Radar Vegetation Indices and AutoML (Sentinel-1 레이더 식생지수와 AutoML을 이용한 Sentinel-2 NDVI 결측화소 복원)

  • Youjeong Youn;Jonggu Kang;Seoyeon Kim;Yemin Jeong;Soyeon Choi;Yungyo Im;Youngmin Seo;Myoungsoo Won;Junghwa Chun;Kyungmin Kim;Keunchang Jang;Joongbin Lim;Yangwon Lee
    • Korean Journal of Remote Sensing
    • /
    • v.39 no.6_1
    • /
    • pp.1341-1352
    • /
    • 2023
  • The normalized difference vegetation index (NDVI) derived from satellite images is a crucial tool to monitor forests and agriculture for broad areas because the periodic acquisition of the data is ensured. However, optical sensor-based vegetation indices(VI) are not accessible in some areas covered by clouds. This paper presented a synthetic aperture radar (SAR) based approach to retrieval of the optical sensor-based NDVI using machine learning. SAR system can observe the land surface day and night in all weather conditions. Radar vegetation indices (RVI) from the Sentinel-1 vertical-vertical (VV) and vertical-horizontal (VH) polarizations, surface elevation, and air temperature are used as the input features for an automated machine learning (AutoML) model to conduct the gap-filling of the Sentinel-2 NDVI. The mean bias error (MAE) was 7.214E-05, and the correlation coefficient (CC) was 0.878, demonstrating the feasibility of the proposed method. This approach can be applied to gap-free nationwide NDVI construction using Sentinel-1 and Sentinel-2 images for environmental monitoring and resource management.

Spatial Gap-filling of GK-2A/AMI Hourly AOD Products Using Meteorological Data and Machine Learning (기상모델자료와 기계학습을 이용한 GK-2A/AMI Hourly AOD 산출물의 결측화소 복원)

  • Youn, Youjeong;Kang, Jonggu;Kim, Geunah;Park, Ganghyun;Choi, Soyeon;Lee, Yangwon
    • Korean Journal of Remote Sensing
    • /
    • v.38 no.5_3
    • /
    • pp.953-966
    • /
    • 2022
  • Since aerosols adversely affect human health, such as deteriorating air quality, quantitative observation of the distribution and characteristics of aerosols is essential. Recently, satellite-based Aerosol Optical Depth (AOD) data is used in various studies as periodic and quantitative information acquisition means on the global scale, but optical sensor-based satellite AOD images are missing in some areas with cloud conditions. In this study, we produced gap-free GeoKompsat 2A (GK-2A) Advanced Meteorological Imager (AMI) AOD hourly images after generating a Random Forest based gap-filling model using grid meteorological and geographic elements as input variables. The accuracy of the model is Mean Bias Error (MBE) of -0.002 and Root Mean Square Error (RMSE) of 0.145, which is higher than the target accuracy of the original data and considering that the target object is an atmospheric variable with Correlation Coefficient (CC) of 0.714, it is a model with sufficient explanatory power. The high temporal resolution of geostationary satellites is suitable for diurnal variation observation and is an important model for other research such as input for atmospheric correction, estimation of ground PM, analysis of small fires or pollutants.