• Title/Summary/Keyword: Data Pre-processing

Search Result 801, Processing Time 0.031 seconds

A Study on Image Processing of Tree Discharges for Insulation Destructive Prediction (절연파괴 예측을 위한 트리방전의 영상처리에 관한 연구)

  • 오무송;김태성
    • Journal of the Korean Institute of Electrical and Electronic Material Engineers
    • /
    • v.14 no.1
    • /
    • pp.26-33
    • /
    • 2001
  • The proposed system was composed of pre-processor which was executing binary/high-pass filtering and post-processor which ranged from statistic data to prediction. In post-processor work, step one was filter process of image, step two was image recognition, and step three was destruction degree/time prediction. After these processing, we could predict image of the last destruction timestamp. This research was produced variation value according to growth of tree pattern. This result showed improved correction, when this research was applied image Processing. Pre-processing step of original image had good result binary work after high pas- filter execution. In the case of using partial discharge of the image, our research could predict the last destruction timestamp. By means of experimental data, this prediction system was acquired $\pm$3.2% error range.

  • PDF

Classification of Gripping Movement in Daily Life Using EMG-based Spider Chart and Deep Learning (근전도 기반의 Spider Chart와 딥러닝을 활용한 일상생활 잡기 손동작 분류)

  • Lee, Seong Mun;Pi, Sheung Hoon;Han, Seung Ho;Jo, Yong Un;Oh, Do Chang
    • Journal of Biomedical Engineering Research
    • /
    • v.43 no.5
    • /
    • pp.299-307
    • /
    • 2022
  • In this paper, we propose a pre-processing method that converts to Spider Chart image data for classification of gripping movement using EMG (electromyography) sensors and Convolution Neural Networks (CNN) deep learning. First, raw data for six hand gestures are extracted from five test subjects using an 8-channel armband and converted into Spider Chart data of octagonal shapes, which are divided into several sliding windows and are learned. In classifying six hand gestures, the classification performance is compared with the proposed pre-processing method and the existing methods. Deep learning was performed on the dataset by dividing 70% of the total into training, 15% as testing, and 15% as validation. For system performance evaluation, five cross-validations were applied by dividing 80% of the entire dataset by training and 20% by testing. The proposed method generates 97% and 94.54% in cross-validation and general tests, respectively, using the Spider Chart preprocessing, which was better results than the conventional methods.

A Pre-processing Process Using TadGAN-based Time-series Anomaly Detection (TadGAN 기반 시계열 이상 탐지를 활용한 전처리 프로세스 연구)

  • Lee, Seung Hoon;Kim, Yong Soo
    • Journal of Korean Society for Quality Management
    • /
    • v.50 no.3
    • /
    • pp.459-471
    • /
    • 2022
  • Purpose: The purpose of this study was to increase prediction accuracy for an anomaly interval identified using an artificial intelligence-based time series anomaly detection technique by establishing a pre-processing process. Methods: Significant variables were extracted by applying feature selection techniques, and anomalies were derived using the TadGAN time series anomaly detection algorithm. After applying machine learning and deep learning methodologies using normal section data (excluding anomaly sections), the explanatory power of the anomaly sections was demonstrated through performance comparison. Results: The results of the machine learning methodology, the performance was the best when SHAP and TadGAN were applied, and the results in the deep learning, the performance was excellent when Chi-square Test and TadGAN were applied. Comparing each performance with the papers applied with a Conventional methodology using the same data, it can be seen that the performance of the MLR was significantly improved to 15%, Random Forest to 24%, XGBoost to 30%, Lasso Regression to 73%, LSTM to 17% and GRU to 19%. Conclusion: Based on the proposed process, when detecting unsupervised learning anomalies of data that are not actually labeled in various fields such as cyber security, financial sector, behavior pattern field, SNS. It is expected to prove the accuracy and explanation of the anomaly detection section and improve the performance of the model.

KorPatELECTRA : A Pre-trained Language Model for Korean Patent Literature to improve performance in the field of natural language processing(Korean Patent ELECTRA)

  • Jang, Ji-Mo;Min, Jae-Ok;Noh, Han-Sung
    • Journal of the Korea Society of Computer and Information
    • /
    • v.27 no.2
    • /
    • pp.15-23
    • /
    • 2022
  • In the field of patents, as NLP(Natural Language Processing) is a challenging task due to the linguistic specificity of patent literature, there is an urgent need to research a language model optimized for Korean patent literature. Recently, in the field of NLP, there have been continuous attempts to establish a pre-trained language model for specific domains to improve performance in various tasks of related fields. Among them, ELECTRA is a pre-trained language model by Google using a new method called RTD(Replaced Token Detection), after BERT, for increasing training efficiency. The purpose of this paper is to propose KorPatELECTRA pre-trained on a large amount of Korean patent literature data. In addition, optimal pre-training was conducted by preprocessing the training corpus according to the characteristics of the patent literature and applying patent vocabulary and tokenizer. In order to confirm the performance, KorPatELECTRA was tested for NER(Named Entity Recognition), MRC(Machine Reading Comprehension), and patent classification tasks using actual patent data, and the most excellent performance was verified in all the three tasks compared to comparative general-purpose language models.

GOES-9 Raw Data Acquisition & Image Extraction

  • Kang C. H.;Park D. J.;Koo I. H.;Ahn S. I.;Kim E. K.
    • Proceedings of the KSRS Conference
    • /
    • 2005.10a
    • /
    • pp.582-585
    • /
    • 2005
  • The Geostationary Operational Environmental Satellite (GOES) 9, which is currently located at 155°E geostationary orbits, has transmitted earth observation data acquired by imager to CDA at NOAA. After the acquisition on ground, observation data are corrected on ground and re-transmitted to GOES-9 for the dissemination to users. In this paper, the procedure and result from raw data acquisition and pre-processing for earth observation imagery retrieval from GOES-9 Raw data acquired in Korea at May 2005 are introduced.

  • PDF

Assessment of Improving SWAT Weather Input Data using Basic Spatial Interpolation Method

  • Felix, Micah Lourdes;Choi, Mikyoung;Zhang, Ning;Jung, Kwansue
    • Proceedings of the Korea Water Resources Association Conference
    • /
    • 2022.05a
    • /
    • pp.368-368
    • /
    • 2022
  • The Soil and Water Assessment Tool (SWAT) has been widely used to simulate the long-term hydrological conditions of a catchment. Two output variables, outflow and sediment yield have been widely investigated in the field of water resources management, especially in determining the conditions of ungauged subbasins. The presence of missing data in weather input data can cause poor representation of the climate conditions in a catchment especially for large or mountainous catchments. Therefore, in this study, a custom module was developed and evaluated to determine the efficiency of utilizing basic spatial interpolation methods in the estimation of weather input data. The module has been written in Python language and can be considered as a pre-processing module prior to using the SWAT model. The results of this study suggests that the utilization of the proposed pre-processing module can improve the simulation results for both outflow and sediment yield in a catchment, even in the presence of missing data.

  • PDF

Pre- and Post Processing System on Prediction Analysis of Thermal Stress in Mass Concrete Structure (매스콘크리트의 온도균열 예측해석에서의 전후처리 시스템 개발에 관한 연구)

  • 김유석;강석화;박칠림
    • Proceedings of the Korea Concrete Institute Conference
    • /
    • 1996.04a
    • /
    • pp.270-274
    • /
    • 1996
  • Until recently pre & post-processing of finite element model has been heavily relied on expensive graphic peripheral devices. But today, with the aid of inexpensive microcomputers, very effective pre & postprocessor graphics has been developed. In this study, Pre & Post processor(MASSPRE, MASSPOST) of prediction analysis of thermal stress in mass concrete structure is developed. The developed pre & post processors are raise to the efficiency in making input data for the main program and analysis of the results produced by the main program. This MASSPOST presents a stress contour graph, volume slice, time-temperature history graph, time-stress history graph, etc.

  • PDF

UAV-based Image Acquisition, Pre-processing, Transmission System Using Mobile Communication Networks (이동통신망을 활용한 무인비행장치 기반 이미지 획득, 전처리, 전송 시스템)

  • Park, Jong-Hong;Ahn, Il-Yeop
    • Proceedings of the Korean Institute of Information and Commucation Sciences Conference
    • /
    • 2022.10a
    • /
    • pp.594-596
    • /
    • 2022
  • This paper relates to a system for pre-processing high-definition images acquired through a camera mounted on an unmanned aerial vehicle(UAV) and transmitting them to a server through a mobile communication network. In the case of the existing UAV system for image acquisition service, the acquired image was stored in the external storage device of the camera mounted on the UAV, and the image was checked by directly moving the storage device after the flight was completed. In the case of this method, there is a limitation in that it is impossible to check whether image acquisition or pre-processing is properly performed before directly checking image data through an external storage device. In addition, since the data is stored only in an external storage device, there is a disadvantage that data sharing is cumbersome. In this paper, to solve the above problems, we propose a system that can remotely check images in real time. Furthermore, we propose a system and method capable of performing pre-processing such as geo-tagging and transmission through a mobile communication network in addition to image acquisition through shooting in an UAV.

  • PDF

A Hybrid Clustering Technique for Processing Large Data (대용량 데이터 처리를 위한 하이브리드형 클러스터링 기법)

  • Kim, Man-Sun;Lee, Sang-Yong
    • The KIPS Transactions:PartB
    • /
    • v.10B no.1
    • /
    • pp.33-40
    • /
    • 2003
  • Data mining plays an important role in a knowledge discovery process and various algorithms of data mining can be selected for the specific purpose. Most of traditional hierachical clustering methode are suitable for processing small data sets, so they difficulties in handling large data sets because of limited resources and insufficient efficiency. In this study we propose a hybrid neural networks clustering technique, called PPC for Pre-Post Clustering that can be applied to large data sets and find unknown patterns. PPC combinds an artificial intelligence method, SOM and a statistical method, hierarchical clustering technique, and clusters data through two processes. In pre-clustering process, PPC digests large data sets using SOM. Then in post-clustering, PPC measures Similarity values according to cohesive distances which show inner features, and adjacent distances which show external distances between clusters. At last PPC clusters large data sets using the simularity values. Experiment with UCI repository data showed that PPC had better cohensive values than the other clustering techniques.

INTRODUCTION OF COMS IDACS SYSTEM FOR METEOROLOGCIAL AND OCDAN MISSION

  • Lim, Hyun-Su;Park, Durk-Jong;Koo, In-Hoi;Kang, Chi-Ho
    • Proceedings of the KSRS Conference
    • /
    • v.1
    • /
    • pp.67-70
    • /
    • 2006
  • KARI is developing Image Data Acquisition and Control System (IDACS) for pre-processing meteorological and ocean data acquired on geostationary orbit. This paper describes the functions and architecture of IDACS and gives its operation policy including backup operation to overcome limitation of single-configured antenna system. The COMS IDACS provides the capability to receive the raw sensor data and disseminate processed MI data to users via a satellite. From the processed image data, users can produce a set of meteorological and ocean products for a wide range of applications. Most of IDACS subsystems are being developed by Korean technologies and experience acquired from previous projects. In case of COMS geometric correction software module, as it is closely dependent on the characteristics of imagers and spacecraft bus system, it is being co-developed with overseas prime contractor who develops spacecraft bus system.

  • PDF