• Title/Summary/Keyword: Variables Pre-processing Model

Search Result 20, Processing Time 0.023 seconds

Development of Pre-Processing and Bias Correction Modules for AMSU-A Satellite Data in the KIAPS Observation Processing System (KIAPS 관측자료 처리시스템에서의 AMSU-A 위성자료 초기 전처리와 편향보정 모듈 개발)

  • Lee, Sihye;Kim, Ju-Hye;Kang, Jeon-Ho;Chun, Hyoung-Wook
    • Atmosphere
    • /
    • v.23 no.4
    • /
    • pp.453-470
    • /
    • 2013
  • As a part of the KIAPS Observation Processing System (KOPS), we have developed the modules of satellite radiance data pre-processing and quality control, which include observation operators to interpolate model state variables into radiances in observation space. AMSU-A (Advanced Microwave Sounding Unit-A) level-1d radiance data have been extracted using the BUFR (Binary Universal Form for the Representation of meteorological data) decoder and a first guess has been calculated with RTTOV (Radiative Transfer for TIROS Operational Vertical Sounder) version 10.2. For initial quality checks, the pixels contaminated by large amounts of cloud liquid water, heavy precipitation, and sea ice have been removed. Channels for assimilation, rejection, or monitoring have been respectively selected for different surface types since the errors from the skin temperature are caused by inaccurate surface emissivity. Correcting the bias caused by errors in the instruments and radiative transfer model is crucial in radiance data pre-processing. We have developed bias correction modules in two steps based on 30-day innovation statistics (observed radiance minus background; O-B). The scan bias correction has been calculated individually for each channel, satellite, and scan position. Then a multiple linear regression of the scan-bias-corrected innovations with several predictors has been employed to correct the airmass bias.

A Pre-processing Process Using TadGAN-based Time-series Anomaly Detection (TadGAN 기반 시계열 이상 탐지를 활용한 전처리 프로세스 연구)

  • Lee, Seung Hoon;Kim, Yong Soo
    • Journal of Korean Society for Quality Management
    • /
    • v.50 no.3
    • /
    • pp.459-471
    • /
    • 2022
  • Purpose: The purpose of this study was to increase prediction accuracy for an anomaly interval identified using an artificial intelligence-based time series anomaly detection technique by establishing a pre-processing process. Methods: Significant variables were extracted by applying feature selection techniques, and anomalies were derived using the TadGAN time series anomaly detection algorithm. After applying machine learning and deep learning methodologies using normal section data (excluding anomaly sections), the explanatory power of the anomaly sections was demonstrated through performance comparison. Results: The results of the machine learning methodology, the performance was the best when SHAP and TadGAN were applied, and the results in the deep learning, the performance was excellent when Chi-square Test and TadGAN were applied. Comparing each performance with the papers applied with a Conventional methodology using the same data, it can be seen that the performance of the MLR was significantly improved to 15%, Random Forest to 24%, XGBoost to 30%, Lasso Regression to 73%, LSTM to 17% and GRU to 19%. Conclusion: Based on the proposed process, when detecting unsupervised learning anomalies of data that are not actually labeled in various fields such as cyber security, financial sector, behavior pattern field, SNS. It is expected to prove the accuracy and explanation of the anomaly detection section and improve the performance of the model.

Assessment of Improving SWAT Weather Input Data using Basic Spatial Interpolation Method

  • Felix, Micah Lourdes;Choi, Mikyoung;Zhang, Ning;Jung, Kwansue
    • Proceedings of the Korea Water Resources Association Conference
    • /
    • 2022.05a
    • /
    • pp.368-368
    • /
    • 2022
  • The Soil and Water Assessment Tool (SWAT) has been widely used to simulate the long-term hydrological conditions of a catchment. Two output variables, outflow and sediment yield have been widely investigated in the field of water resources management, especially in determining the conditions of ungauged subbasins. The presence of missing data in weather input data can cause poor representation of the climate conditions in a catchment especially for large or mountainous catchments. Therefore, in this study, a custom module was developed and evaluated to determine the efficiency of utilizing basic spatial interpolation methods in the estimation of weather input data. The module has been written in Python language and can be considered as a pre-processing module prior to using the SWAT model. The results of this study suggests that the utilization of the proposed pre-processing module can improve the simulation results for both outflow and sediment yield in a catchment, even in the presence of missing data.

  • PDF

A Study on Productivity Factors of Chinese Container Terminals

  • Lu, Bo;Park, Nam-Kyu
    • Journal of Navigation and Port Research
    • /
    • v.34 no.7
    • /
    • pp.559-566
    • /
    • 2010
  • The container port industry has been variously studied by many researchers, because the contemporary container transportation and container port industries play a pivotal role in globalization of the world economy. For container terminals, the productivity, affected by many factors, is an important target in measuring container terminal performance. Under this background, finding the critical factors affecting the productivity is necessary. Regression analysis can be used to identify which independent variables are related to the dependent variable, and explore the relationships of them. The aim of paper is to evaluate the factors affecting the productivity of Chinese major terminals by using a regression statistical analysis modeling approach, which is to establish the variable preprocessing model (VPM) and regression analysis model (RAM), by means of collecting the major Chinese container terminals data in the year of 2008.

A study on the process of tube end spinning by the upper bound method and the finite element method (상계해법과 유한요소법을 이용한 스피닝공정 해석에 관한 연구)

  • 김전형;홍성인;이정환;이영선
    • Transactions of Materials Processing
    • /
    • v.6 no.6
    • /
    • pp.517-526
    • /
    • 1997
  • The purpose of this study is to investigate changes in the wall thickness of tube sinking and working forces by the upper bound method and ABAQUS code. The independent variables are ; workpiece material, original wall thickness of tube, die angle, friction, and reduction of diameter. The results indicate that these five variables are factors of the increase in wall-thickness and working forces. Three variables, a inner tube wall angle and two angles of the velocity discontinuous surfaces, are optimized in this proposed velocity field by the upper bound method. In this method, we can estimate the working forces and final tube thicknesses similar to actual forming process. Optimum process variables which are obtained by upper bound method are used in ABAQUS pre-model.

  • PDF

Absolute Atmospheric Correction Procedure for the EO-1 Hyperion Data Using MODTRAN Code

  • Kim, Sun-Hwa;Kang, Sung-Jin;Chi, Jun-Hwa;Lee, Kyu-Sung
    • Korean Journal of Remote Sensing
    • /
    • v.23 no.1
    • /
    • pp.7-14
    • /
    • 2007
  • Atmospheric correction is one of critical procedures to extract quantitative information related to biophysical variables from hyperspectral imagery. Most atmospheric correction algorithms developed for hyperspectral data have been based upon atmospheric radiative transfer (RT) codes, such as MODTRAN. Because of the difficulty in acquisition of atmospheric data at the time of image capture, the complexity of RT model, and large volume of hyperspectral data, atmospheric correction can be very difficult and time-consuming processing. In this study, we attempted to develop an efficient method for the atmospheric correction of EO-1 Hyperion data. This method uses the pre-calculated look-up-table (LUT) for fast and simple processing. The pre-calculated LUT was generated by successive running of MODTRAN model with several input parameters related to solar and sensor geometry, radiometric specification of sensor, and atmospheric condition. Atmospheric water vapour contents image was generated directly from a few absorption bands of Hyperion data themselves and used one of input parameters. This new atmospheric correction method was tested on the Hyperion data acquired on June 3, 2001 over Seoul area. Reflectance spectra of several known targets corresponded with the typical pattern of spectral reflectance on the atmospherically corrected Hyperion image, although further improvement to reduce sensor noise is necessary.

Verification of the Global Numerical Weather Prediction Using SYNOP Surface Observation Data (SYNOP 지상관측자료를 활용한 수치모델 전구 예측성 검증)

  • Lee, Eun-Hee;Choi, In-Jin;Kim, Ki-Byung;Kang, Jeon-Ho;Lee, Juwon;Lee, Eunjeong;Seol, Kyung-Hee
    • Atmosphere
    • /
    • v.27 no.2
    • /
    • pp.235-249
    • /
    • 2017
  • This paper describes methodology verifying near-surface predictability of numerical weather prediction models against the surface synoptic weather station network (SYNOP) observation. As verification variables, temperature, wind, humidity-related variables, total cloud cover, and surface pressure are included in this tool. Quality controlled SYNOP observation through the pre-processing for data assimilation is used. To consider the difference of topographic height between observation and model grid points, vertical inter/extrapolation is applied for temperature, humidity, and surface pressure verification. This verification algorithm is applied for verifying medium-range forecasts by a global forecasting model developed by Korea Institute of Atmospheric Prediction Systems to measure the near-surface predictability of the model and to evaluate the capability of the developed verification tool. It is found that the verification of near-surface prediction against SYNOP observation shows consistency with verification of upper atmosphere against global radiosonde observation, suggesting reliability of those data and demonstrating importance of verification against in-situ measurement as well. Although verifying modeled total cloud cover with observation might have limitation due to the different definition between the model and observation, it is also capable to diagnose the relative bias of model predictability such as a regional reliability and diurnal evolution of the bias.

DEVELOPMENT OF ATMOSPHERIC CORRECTION ALGORITHM FOR HYPERSPECTRAL DATA USING MODTRAN MODEL

  • Kim, Sun-Hwa;Kang, Sung-Jin;Ji, Jun-Hwa;Lee, Kyu-Sung
    • Proceedings of the KSRS Conference
    • /
    • v.2
    • /
    • pp.619-622
    • /
    • 2006
  • Atmospheric correction is one of critical procedures to extract quantitative information related to biophysical variables from hyperspectral data. In this study, we attempted to generate the water vapor contents image from hyperspectral data itself and developed the atmospheric correction algorithm for EO-1 Hyperion data using pre-calculated atmospheric look-up-table (LUT) for fast processing. To apply the new atmospheric correction algorithm, Hyperion data acquired June 3, 2001 over Seoul area is used. Reflectance spectrums of various targets on atmospheric corrected Hyperion reflectance images showed the general spectral pattern although there must be further development to reduce the spectral noise.

  • PDF

Process Design on Fabrication of Large Sized Ring by Mandrel Forging of Hollow Cast Ingot (중공 잉곳을 이용한 대형 링 단조품 제조공정 설계 연구)

  • Lee, S.U.;Lee, Y.S.;Lee, M.W.;Lee, D.H.;Kim, S.S.
    • Transactions of Materials Processing
    • /
    • v.19 no.6
    • /
    • pp.329-336
    • /
    • 2010
  • Ring forging process is more appropriate for high-length and thin walled ring, because it utilizes the forging press and hence does not require heavy-duty ring rolling mill. Although ring forging process is very simple and economic for facilities, the process is not efficient because of multi-forging-step and low material utilization. An effective ring forging process is developed using a hollow ingot. When a hollow ingot is used with a workpiece, the ingot can be forged into a final ring without multi-stage pre-forging process, such as, cogging, upsetting, and piercing, etc.. Finally it has advantages of the material utilization and process improvement because a few reheating and forging process are not necessary to make workpiece for ring forging. The important design variables are the applied plastic deformation energy to eliminate cast structure and make uniform properties. In this study, the mechanical properties after forging of hollow cast ingot were investigated from the experiment using circumferential sectional model. Also, the effects of process variables were studied by FEM simulation on the basis of thermo-visco-plastic constitutive equation. Applied strain is different at each position in length direction because diameter of hollow ingot is different in length direction. The different strain distribution become into a narrow gap by additional plastic deformation during diameter extension process.

Inverter-Based Solar Power Prediction Algorithm Using Artificial Neural Network Regression Model (인공 신경망 회귀 모델을 활용한 인버터 기반 태양광 발전량 예측 알고리즘)

  • Gun-Ha Park;Su-Chang Lim;Jong-Chan Kim
    • The Journal of the Korea institute of electronic communication sciences
    • /
    • v.19 no.2
    • /
    • pp.383-388
    • /
    • 2024
  • This paper is a study to derive the predicted value of power generation based on the photovoltaic power generation data measured in Jeollanam-do, South Korea. Multivariate variables such as direct current, alternating current, and environmental data were measured in the inverter to measure the amount of power generation, and pre-processing was performed to ensure the stability and reliability of the measured values. Correlation analysis used only data with high correlation with power generation in time series data for prediction using partial autocorrelation function (PACF). Deep learning models were used to measure the amount of power generation to predict the amount of photovoltaic power generation, and the results of correlation analysis of each multivariate variable were used to increase the prediction accuracy. Learning using refined data was more stable than when existing data were used as it was, and the solar power generation prediction algorithm was improved by using only highly correlated variables among multivariate variables by reflecting the correlation analysis results.