• Title/Summary/Keyword: Data Preprocessing

Search Result 939, Processing Time 0.031 seconds

Data Management and Analysis in Foundry Industry (1) (주조공정 데이터 처리 및 분석 (1))

  • Cho, In-Sung
    • Journal of Korea Foundry Society
    • /
    • v.42 no.1
    • /
    • pp.35-41
    • /
    • 2022
  • In the present paper, the data management of casting processes has been discussed. In order to construct a smart factory in the foundry industry, understanding of the whole casting processes has to be in the first place. Casting process data can be obtained at the kiosk operated by casting engineers and data acquired by sensors in the foundry facility. However, preprocessing of the casting process data must be carried out in order to analyze the casting process by the data. Techniques and some examples for data preprocessing in the foundry was introduced.

A Study on Development Environments for Machine Learning (머신러닝 자동화를 위한 개발 환경에 관한 연구)

  • Kim, Dong Gil;Park, Yong-Soon;Park, Lae-Jeong;Chung, Tae-Yun
    • IEMEK Journal of Embedded Systems and Applications
    • /
    • v.15 no.6
    • /
    • pp.307-316
    • /
    • 2020
  • Machine learning model data is highly affected by performance. preprocessing is needed to enable analysis of various types of data, such as letters, numbers, and special characters. This paper proposes a development environment that aims to process categorical and continuous data according to the type of missing values in stage 1, implementing the function of selecting the best performing algorithm in stage 2 and automating the process of checking model performance in stage 3. Using this model, machine learning models can be created without prior knowledge of data preprocessing.

A Study on Real-time Data Preprocessing Technique for Small Millimeter Wave Radar (소형 밀리미터파 레이더를 위한 실시간 데이터 전처리 방법 연구)

  • Choi, Jinkyu;Shin, Youngcheol;Hong, Soonil;Park, Changhyun;Kim, Younjin;Kim, Hongrak;Kwon, Junbeom
    • The Journal of the Institute of Internet, Broadcasting and Communication
    • /
    • v.19 no.6
    • /
    • pp.79-85
    • /
    • 2019
  • Recently, small radar require the development of small millimeter wave radar with high distance resolution to disable the target's system with a single strike. Small millimeter wave radar with high distance resolution need to process large amounts of data in real time to acquire and track target. In this paper, we summarized the real-time data preprocessing method to process the large amount of data required for small millimeter wave radar. In addition, the digital IF(Intermediate Frequency) receiver, Window processing, and, DFT(Discrete Fourier Transform) functions presented by real-time data preprocessing are implemented using FPGA(Field Programmable Gate Array). Finally the implemented real-time data preprocessing module was applied to the signal processor for small millimeter wave radar and verified by performance test related to the real-time preprocessing function.

Multiple Model Prediction System Based on Optimal TS Fuzzy Model and Its Applications to Time Series Forecasting (최적 TS 퍼지 모델 기반 다중 모델 예측 시스템의 구현과 시계열 예측 응용)

  • Bang, Young-Keun;Lee, Chul-Heui
    • Journal of Industrial Technology
    • /
    • v.28 no.B
    • /
    • pp.101-109
    • /
    • 2008
  • In general, non-stationary or chaos time series forecasting is very difficult since there exists a drift and/or nonlinearities in them. To overcome this situation, we suggest a new prediction method based on multiple model TS fuzzy predictors combined with preprocessing of time series data, where, instead of time series data, the differences of them are applied to predictors as input. In preprocessing procedure, the candidates of optimal difference interval are determined by using con-elation analysis and corresponding difference data are generated. And then, for each of them, TS fuzzy predictor is constructed by using k-means clustering algorithm and least squares method. Finally, the best predictor which minimizes the performance index is selected and it works on hereafter for prediction. Computer simulation is performed to show the effectiveness and usefulness of our method.

  • PDF

Preprocessing Technique for Lane Detection Using Image Clustering and HSV Color Model (영상 클러스터링과 HSV 컬러 모델을 이용한 차선 검출 전처리 기법)

  • Choi, Na-Rae;Choi, Sang-Il
    • Journal of Korea Multimedia Society
    • /
    • v.20 no.2
    • /
    • pp.144-152
    • /
    • 2017
  • Among the technologies for implementing autonomous vehicles, advanced driver assistance system is a key technology to support driver's safe driving. In the technology using the vision sensor having a high utility, various preprocessing methods are used prior to feature extraction for lane detection. However, in the existing methods, the unnecessary lane candidates such as cars, lawns, and road separator in the road area are false positive. In addition, there are cases where the lane candidate itself can not be extracted in the area under the overpass, the lane within the dark shadow, the center lane of yellow, and weak lane. In this paper, we propose an efficient preprocessing method using k-means clustering for image division and the HSV color model. When the proposed preprocessing method is applied, the true positive region is maximally maintained during the lane detection and many false positive regions are removed.

Energy-Aware Data-Preprocessing Scheme for Efficient Audio Deep Learning in Solar-Powered IoT Edge Computing Environments (태양 에너지 수집형 IoT 엣지 컴퓨팅 환경에서 효율적인 오디오 딥러닝을 위한 에너지 적응형 데이터 전처리 기법)

  • Yeontae Yoo;Dong Kun Noh
    • IEMEK Journal of Embedded Systems and Applications
    • /
    • v.18 no.4
    • /
    • pp.159-164
    • /
    • 2023
  • Solar energy harvesting IoT devices prioritize maximizing the utilization of collected energy due to the periodic recharging nature of solar energy, rather than minimizing energy consumption. Meanwhile, research on edge AI, which performs machine learning near the data source instead of the cloud, is actively conducted for reasons such as data confidentiality and privacy, response time, and cost. One such research area involves performing various audio AI applications using audio data collected from multiple IoT devices in an IoT edge computing environment. However, in most studies, IoT devices only perform sensing data transmission to the edge server, and all processes, including data preprocessing, are performed on the edge server. In this case, it not only leads to overload issues on the edge server but also causes network congestion by transmitting unnecessary data for learning. On the other way, if data preprocessing is delegated to each IoT device to address this issue, it leads to another problem of increased blackout time due to energy shortages in the devices. In this paper, we aim to alleviate the problem of increased blackout time in devices while mitigating issues in server-centric edge AI environments by determining where the data preprocessed based on the energy state of each IoT device. In the proposed method, IoT devices only perform the preprocessing process, which includes sound discrimination and noise removal, and transmit to the server if there is more energy available than the energy threshold required for the basic operation of the device.

Personalized Service Based on Context Awareness through User Emotional Perception in Mobile Environment (모바일 환경에서의 상황인식 기반 사용자 감성인지를 통한 개인화 서비스)

  • Kwon, Il-Kyoung;Lee, Sang-Yong
    • Journal of Digital Convergence
    • /
    • v.10 no.2
    • /
    • pp.287-292
    • /
    • 2012
  • In this paper, user personalized services through the emotion perception required to support location-based sensing data preprocessing techniques and emotion data preprocessing techniques is studied for user's emotion data building and preprocessing in V-A emotion model. For this purpose the granular context tree and string matching based emotion pattern matching techniques are used. In addition, context-aware and personalized recommendation services technique using probabilistic reasoning is studied for personalized services based on context awareness.

NETWORK DESIGN AND PREPROCESSING FOR MULTI-SCALE SPHERICAL BASIS FUNCTION REPRESENTATION

  • Oh, Hee-Seok;Kim, Dong-Hoh
    • Journal of the Korean Statistical Society
    • /
    • v.36 no.2
    • /
    • pp.209-228
    • /
    • 2007
  • Given scattered surface air temperatures observed by a network of weather stations, it is an important problem to estimate the entire temperature field for every location on the globe. Recently, a multi-scale spherical basis function (SBF) representation was proposed by Li (1999) for representing scattered data on the sphere. However, for a successful application of Li (1999)'s method, some practical issues such as network design, bandwidth selection of SBFs and initial coefficients are to be resolved. This paper proposes automatic procedures to design network and to select bandwidths. This paper also considers a preprocessing problem to obtain a stable initial coefficients from scattered data. Experiments with real temperature data demonstrate the promising empirical properties of the proposed approaches.

Development of Classification Model Using Neural Network (신경회로망을 이용한 분류모형 개발)

  • Park, Kwang-Bak;Park, Young-Man;Hwang, Seung-Gook
    • Journal of the Korean Institute of Intelligent Systems
    • /
    • v.18 no.5
    • /
    • pp.638-641
    • /
    • 2008
  • In this paper, a model to classify the method using the fuzzy TAM with preprocessing of data was developed. The preprocessing method can be divide the problem using the characteristics in the case of category type factor. In case of continuous type factor, if there was exist factor's range which is not overlapping by class, the data belong to the range was fixed and eliminated in classification. After these preprocessing of data, classified operation of Fuzzy TAM is performed.

Ontology based Preprocessing Scheme for Mining Data Streams from Sensor Networks (센서 네트워크의 데이터 스트림 마이닝을 위한 온톨로지 기반의 전처리 기법)

  • Jung, Jason J.
    • Journal of Intelligence and Information Systems
    • /
    • v.15 no.3
    • /
    • pp.67-80
    • /
    • 2009
  • By a number of sensors and sensor networks, we can collect environmental information from a certain sensor space. To discover more useful information and knowledge, we want to employ data mining methodologies to sensor data stream from such sensor spaces. In this paper, we present a novel data preprocessing scheme to improve the performances of the data mining algorithms. Especially, ontologies are applied to represent meanings of the sensor data. For evaluating the proposed method, we have collected sensor streams for about 30 days, and simulated them to compare with other approaches.

  • PDF