• Title/Summary/Keyword: Data Cleaning

Search Result 422, Processing Time 0.027 seconds

Automatic Cleaning Algorithm of Asset Data for Transmission Cable (지중 송전케이블 자산데이터의 자동 정제 알고리즘 개발연구)

  • Hwang, Jae-Sang;Mun, Sung-Duk;Kim, Tae-Joon;Kim, Kang-Sik
    • KEPCO Journal on Electric Power and Energy
    • /
    • v.7 no.1
    • /
    • pp.79-84
    • /
    • 2021
  • The fundamental element to be kept for big data analysis, artificial intelligence technologies and asset management system is a data quality, which could directly affect the entire system reliability. For this reason, the momentum of data cleaning works is recently increased and data cleaning methods have been investigating around the world. In the field of electric power, however, asset data cleaning methods have not been fully determined therefore, automatic cleaning algorithm of asset data for transmission cables has been studied in this paper. Cleaning algorithm is composed of missing data treatment and outlier data one. Rule-based and expert opinion based cleaning methods are converged and utilized for these dirty data.

Automatic Algorithm for Cleaning Asset Data of Overhead Transmission Line (가공송전 전선 자산데이터의 정제 자동화 알고리즘 개발 연구)

  • Mun, Sung-Duk;Kim, Tae-Joon;Kim, Kang-Sik;Hwang, Jae-Sang
    • KEPCO Journal on Electric Power and Energy
    • /
    • v.7 no.1
    • /
    • pp.73-77
    • /
    • 2021
  • As the big data analysis technologies has been developed worldwide, the importance of asset management for electric power facilities based data analysis is increasing. It is essential to secure quality of data that will determine the performance of the RISK evaluation algorithm for asset management. To improve reliability of asset management, asset data must be preprocessed. In particular, the process of cleaning dirty data is required, and it is also urgent to develop an algorithm to reduce time and improve accuracy for data treatment. In this paper, the result of the development of an automatic cleaning algorithm specialized in overhead transmission asset data is presented. A data cleaning algorithm was developed to enable data clean by analyzing quality and overall pattern of raw data.

An Analysis Method of Superlarge Manufacturing Process Data Using Data Cleaning and Graphical Analysis (데이터 정제와 그래프 분석을 이용한 대용량 공정데이터 분석 방법)

  • 박재홍;변재현
    • Journal of Korean Society for Quality Management
    • /
    • v.30 no.2
    • /
    • pp.72-85
    • /
    • 2002
  • Advances in computer and sensor technology have made it possible to obtain superlarge manufacturing process data in real time, letting us extract meaningful information from these superlarge data sets. We propose a systematic data analysis procedure which field engineers can apply easily to manufacture quality products. The procedure consists of data cleaning and data analysis stages. Data cleaning stage is to construct a database suitable for statistical analysis from the original superlarge manufacturing process data. In the data analysis stage, we suggest a graphical easy-to-implement approach to extract practical information from the cleaned database. This study will help manufacturing companies to achieve six sigma quality.

Surface Cleaning of a Wafer Contaminated by Fingerprint Using a Laser Cleaning Technology (레이저 세정기술을 이용한 웨이퍼의 표면세정)

  • Lee, Myong-Hwa;Baek, Ji-Young;Song, Jae-Dong;Kim, Sang-Bum;Kim, Gyung-Soo
    • Journal of ILASS-Korea
    • /
    • v.12 no.4
    • /
    • pp.185-190
    • /
    • 2007
  • There is a growing interest to develop a new cleaning technology to overcome the disadvantages of wet cleaning technologies such as environmental pollution and the cleaning difficulty of contaminants on integrated circuits. Laser cleaning is a potential technology to remove various pollutants on a wafer surface. However, there is no fundamental data about cleaning efficiencies and cleaning mechanisms of contaminants on a wafer surface using a laser cleaning technology. Therefore, the cleaning characteristics of a wafer surface using an excimer laser were investigated in this study. Fingerprint consisting of inorganic and organic materials was chosen as a representative of pollutants and the effectiveness of a laser irradiation on a wafer cleaning has been investigated qualitatively and quantitatively. The results have shown that cleaning degree is proportional to the laser irradiation time and repetition rate, and quantitative analysis conducted by an image processing method also have shown the same trend. Furthermore, the cleaning efficiency of a wafer contaminated by fingerprint strongly depended on a photothermal cleaning mechanism and the species were removed in order of hydrophilic and hydrophobic contaminants by laser irradiation.

  • PDF

Field Application of an Alternative Cleaning Agent to Methylene Chloride (Methylene Chloride의 대체세정제 적용 사례 연구)

  • 배재흠;신민철;이통영;조기수
    • Journal of the Korean institute of surface engineering
    • /
    • v.32 no.2
    • /
    • pp.109-124
    • /
    • 1999
  • A study for replacing methylene chloride for an alternative cleaning agent which can be utilized for cleaning aluminium parts of hard disk drives was carried out at Suwon factory of Samsung Electro-mechanics Co. Ltd. As a result, quite a good cleaning agent was selected through data collection of alternative cleaning agents, their analysis of environment effects and safty factors and testing of their cleaning power. As compared with methylene chloride, the selected cleaning agent has better cleaning power. It is also more environmental-friendly, less toxic and more economical. And the operating cost reduction due to adoption of this alternative cleaning agent in Samsung Electro-mechanics Co. Ltd. was more than ₩l46,000,000 annually without any capital instrument in addition to improvement of environmental and safty conditions in the plant.

  • PDF

A Study on the Evaluation Methods of Residual Flux Cleaning Ability by Alternative Semi-Aqueous Cleaners Using Metal Test Tools After Soldering with Solder Paste (솔더페이스트로 솔더링 후 잔류 플럭스 오염물에 대한 준수계 세정제의 금속치구를 이용한 세정성능 평가방법 연구)

  • Lee, Dong-Kee
    • Clean Technology
    • /
    • v.14 no.2
    • /
    • pp.103-109
    • /
    • 2008
  • In this study, in order to develop evaluation method of the cleaning efficiency of residual flux which remains on the surface after soldering with solder paste, a specially designed metal tool is used to reduce spread uncertainty of flux while soldering. Using this tool, the measurement of cleaning efficiency of flux after soldering for some typical alternative semi-aqueous cleaners and 1,1,1-TCE by weighing method was conducted. As the test result of cleaning efficiency for each cleaner at several different cleaning times, the precision of the data is confirmed to within about 4% relative standard deviation (RSD) range. So, it is considered that this would be a good evaluation method for evaluating the cleaning efficiency of the residual flux which remains after solder paste soldering in the alternative cleaning. The results of this test method shows that the cleaning efficiency of ST 100SX and Neozal 750H in the cleaning of residual flux was better than other semi-aqueous cleaners, but its cleaning efficiency was clearly inferior to 1,1,1-TCE.

  • PDF

Data Cleaning and Integration of Multi-year Dietary Survey in the Korea National Health and Nutrition Examination Survey (KNHANES) using Database Normalization Theory (데이터베이스 정규화 이론을 이용한 국민건강영양조사 중 다년도 식이조사 자료 정제 및 통합)

  • Kwon, Namji;Suh, Jihye;Lee, Hunjoo
    • Journal of Environmental Health Sciences
    • /
    • v.43 no.4
    • /
    • pp.298-306
    • /
    • 2017
  • Objectives: Since 1998, the Korea National Health and Nutrition Examination Survey (KNHANES) has been conducted in order to investigate the health and nutritional status of Koreans. The food intake data of individuals in the KNHANES has also been utilized as source dataset for risk assessment of chemicals via food. To improve the reliability of intake estimation and prevent missing data for less-responded foods, the structure of integrated long-standing datasets is significant. However, it is difficult to merge multi-year survey datasets due to ineffective cleaning processes for handling extensive numbers of codes for each food item along with changes in dietary habits over time. Therefore, this study aims at 1) cleaning the process of abnormal data 2) generation of integrated long-standing raw data, and 3) contributing to the production of consistent dietary exposure factors. Methods: Codebooks, the guideline book, and raw intake data from KNHANES V and VI were used for analysis. The violation of the primary key constraint and the $1^{st}-3rd$ normal form in relational database theory were tested for the codebook and the structure of the raw data, respectively. Afterwards, the cleaning process was executed for the raw data by using these integrated codes. Results: Duplication of key records and abnormality in table structures were observed. However, after adjusting according to the suggested method above, the codes were corrected and integrated codes were newly created. Finally, we were able to clean the raw data provided by respondents to the KNHANES survey. Conclusion: The results of this study will contribute to the integration of the multi-year datasets and help improve the data production system by clarifying, testing, and verifying the primary key, integrity of the code, and primitive data structure according to the database normalization theory in the national health data.

PECVD Chamber Cleaning End Point Detection (EPD) Using Optical Emission Spectroscopy Data

  • Lee, Ho Jae;Seo, Dongsun;Hong, Sang Jeen;May, Gary S.
    • Transactions on Electrical and Electronic Materials
    • /
    • v.14 no.5
    • /
    • pp.254-257
    • /
    • 2013
  • In-situ optical emission spectroscopy (OES) is employed for PECVD chamber monitoring. OES is used as an addon sensor to monitoring and cleaning end point detection (EPD). On monitoring plasma chemistry using OES, the process gas and by-product gas are simultaneously monitored. Principal component analysis (PCA) enhances the capability of end point detection using OES data. Through chamber cleaning monitoring using OES, cleaning time is reduced by 53%, in general. Therefore, the gas usage of fluorine is also reduced, so satisfying Green Fab challenge in semiconductor manufacturing.

Monitoring of The Advanced Water Treatment Using AODV Algorithm (AODV 알고리즘을 이용한 정수시설의 모니터링)

  • Bae, Jong-Il
    • Proceedings of the KIEE Conference
    • /
    • 2011.07a
    • /
    • pp.2015-2016
    • /
    • 2011
  • Detection of data cleaning bed of because method of communication uses AODV(Ad hoc On-demand Distance Vector) Zig-Bee communication though communication method is important being separated by several part structurally data monitor ring do it way to understand be. Although data that detect in transmitter will get into several kinds, data of each senses is detected by 4 - 20 mA conclusively and remainder data is consisted of temperature data. That is measured in transmitter in mast baud via conversion relation to do monitorring norm of data being decided to PC mistake of measured value so that can do control between enforcement compose. Speak that can manage equipment of cleaning bed by number of persons who write if detect data detection in wide cleaning bed through Zig-Bee communication and is easy because also administration of data consists of real time.

  • PDF