• Title/Summary/Keyword: Preprocessing Methods

Search Result 506, Processing Time 0.032 seconds

Electric Load Forecasting using Data Preprocessing and Fuzzy Logic System (데이터 전처리와 퍼지 논리 시스템을 이용한 전력 부하 예측)

  • Bang, Young-Keun;Lee, Chul-Heui
    • The Transactions of The Korean Institute of Electrical Engineers
    • /
    • v.66 no.12
    • /
    • pp.1751-1758
    • /
    • 2017
  • This paper presents a fuzzy logic system with data preprocessing to make the accurate electric power load prediction system. The fuzzy logic system acceptably treats the hidden characteristic of the nonlinear data. The data preprocessing processes the original data to provide more information of its characteristics. Thus the combination of two methods can predict the given data more accurately. The former uses TSK fuzzy logic system to apply the linguistic rule base and the linear regression model while the latter uses the linear interpolation method. Finally, four regional electric power load data in taiwan are used to evaluate the performance of the proposed prediction system.

Experimental Comparison of CNN-based Steganalysis Methods with Structural Differences (구조적인 차이를 가지는 CNN 기반의 스테그아날리시스 방법의 실험적 비교)

  • Kim, Jaeyoung;Park, Hanhoon;Park, Jong-Il
    • Journal of Broadcast Engineering
    • /
    • v.24 no.2
    • /
    • pp.315-328
    • /
    • 2019
  • Image steganalysis is an algorithm that classifies input images into stego images with steganography methods and cover images without steganography methods. Previously, handcrafted feature-based steganalysis methods have been mainly studied. However, CNN-based objects recognition has achieved great successes and CNN-based steganalysis is actively studied recently. Unlike object recognition, CNN-based steganalysis requires preprocessing filters to discriminate the subtle difference between cover images from stego images. Therefore, CNN-based steganalysis studies have focused on developing effective preprocessing filters as well as network structures. In this paper, we compare previous studies in same experimental conditions, and based on the results, we analy ze the performance variation caused by the differences in preprocessing filter and network structure.

Parallax Map Preprocessing Algorithm for Performance Improvement of Hole-Filling (홀 채우기의 성능 개선을 위한 시차지도의 전처리 알고리즘)

  • Kim, Jun-Ho;Lee, Si-Woong
    • The Journal of the Korea Contents Association
    • /
    • v.13 no.10
    • /
    • pp.62-70
    • /
    • 2013
  • DIBR(Depth Image Based Rendering) is a kind of view synthesis algorithm to generate images at free view points from the reference color image and its depth map. One of the main challenges of DIBR is the occurrence of holes that correspond to uncovered backgrounds at the synthesized view. In order to cover holes efficiently, two main approaches have been actively investigated. One is to develop preprocessing algorithms for depth maps or parallax maps to reduce the size of possible holes, and the other is to develop hole filling methods to fill the generated holes using adjacent pixels in non-hole areas. Most conventional preprocessing algorithms for reducing the size of holes are based on the smoothing process of depth map. Filtering of depth map, however, attenuates the resolution of depth map and generates geometric distortions. In this paper, we proposes a novel preprocessing algorithm for parallax map to improve the performance of hole-filling by avoiding the drawbacks of conventional methods.

Prediction of the price for stock index futures using integrated artificial intelligence techniques with categorical preprocessing

  • Kim, Kyoung-jae;Han, Ingoo
    • Proceedings of the Korean Operations and Management Science Society Conference
    • /
    • 1997.10a
    • /
    • pp.105-108
    • /
    • 1997
  • Previous studies in stock market predictions using artificial intelligence techniques such as artificial neural networks and case-based reasoning, have focused mainly on spot market prediction. Korea launched trading in index futures market (KOSPI 200) on May 3, 1996, then more people became attracted to this market. Thus, this research intends to predict the daily up/down fluctuant direction of the price for KOSPI 200 index futures to meet this recent surge of interest. The forecasting methodologies employed in this research are the integration of genetic algorithm and artificial neural network (GAANN) and the integration of genetic algorithm and case-based reasoning (GACBR). Genetic algorithm was mainly used to select relevant input variables. This study adopts the categorical data preprocessing based on expert's knowledge as well as traditional data preprocessing. The experimental results of each forecasting method with each data preprocessing method are compared and statistically tested. Artificial neural network and case-based reasoning methods with best performance are integrated. Out-of-the Model Integration and In-Model Integration are presented as the integration methodology. The research outcomes are as follows; First, genetic algorithms are useful and effective method to select input variables for Al techniques. Second, the results of the experiment with categorical data preprocessing significantly outperform that with traditional data preprocessing in forecasting up/down fluctuant direction of index futures price. Third, the integration of genetic algorithm and case-based reasoning (GACBR) outperforms the integration of genetic algorithm and artificial neural network (GAANN). Forth, the integration of genetic algorithm, case-based reasoning and artificial neural network (GAANN-GACBR, GACBRNN and GANNCBR) provide worse results than GACBR.

  • PDF

Supervised learning-based DDoS attacks detection: Tuning hyperparameters

  • Kim, Meejoung
    • ETRI Journal
    • /
    • v.41 no.5
    • /
    • pp.560-573
    • /
    • 2019
  • Two supervised learning algorithms, a basic neural network and a long short-term memory recurrent neural network, are applied to traffic including DDoS attacks. The joint effects of preprocessing methods and hyperparameters for machine learning on performance are investigated. Values representing attack characteristics are extracted from datasets and preprocessed by two methods. Binary classification and two optimizers are used. Some hyperparameters are obtained exhaustively for fast and accurate detection, while others are fixed with constants to account for performance and data characteristics. An experiment is performed via TensorFlow on three traffic datasets. Three scenarios are considered to investigate the effects of learning former traffic on sequential traffic analysis and the effects of learning one dataset on application to another dataset, and determine whether the algorithms can be used for recent attack traffic. Experimental results show that the used preprocessing methods, neural network architectures and hyperparameters, and the optimizers are appropriate for DDoS attack detection. The obtained results provide a criterion for the detection accuracy of attacks.

PREPROCESSING EFFECTS ON ON-LINE SSC MEASUREMENT OF FUJI APPLE BY NIR SPECTROSCOPY

  • Ryu, D.S.;Noh, S.H.;Hwang, I.G.
    • Proceedings of the Korean Society for Agricultural Machinery Conference
    • /
    • 2000.11c
    • /
    • pp.560-568
    • /
    • 2000
  • The aims of this research were to investigate the preprocessing effect of spectrum data on prediction performance and to develop a robust model to predict SSC in intact apple. Spectrum data of 320 Fuji apples were measured with the on-line transmittance measurement system at the wavelength range of 550∼1100nm. Preprocess methods adopted for the tests were Savitzky Golay, MSC, SNV, first derivative and OSC. Several combinations of those methods were applied to the raw spectrum data set to investigate the relative effect of each method on the performance of the calibration model. PLS method was used to regress the preprocessed data set and the SSCs of samples, and the cross-validation was to select the optimal number of PLS factors. Smoothing and scattering corection were essential in increasing the prediction performance of PLS regression model and the OSC contributed to reduction of the number of PLS factors. The first derivative resulted in unfavorable effect on the prediction performance. MSC and SNV showed similar effect. A robust calibration model could be developed by the preprocessing combination of Savitzky Golay smoothing, MSC and OSC, which resulted in SEP= 0.507, bias=0.032 and R$^2$=0.8823.

  • PDF

Document Filtering Algorithm for Efficient Preprocessing of XML Information Retrieval (XML 정보검색의 효율적 전처리를 위한 문서여과 알고리즘)

  • Kong Yong-Hae;Kim Myung-Sook
    • Journal of the Korea Academia-Industrial cooperation Society
    • /
    • v.6 no.1
    • /
    • pp.1-11
    • /
    • 2005
  • The paper proposes a preprocessing method for efficient processing of XML queries in information retrieval with a large amount of XML documents. The conventional preprocessing methods filter out XML documents by parsing XML document for keyword of query or by comparing query signatures with signatures of XML document to be generated. But these methods are dependent on a query and are very in efficient for a large amount of XML documents. For this, we generate a universal DTD based on ontology of a domain. The universal DTD is applicable to the XML documents when they contain information of a same domain even when they have different structures and attributes. Then, using the universal DTD, we filter out the XML documents that are not bounded in the domain. We evaluate the performance of this method through experiments.

  • PDF

Study on Fault Diagnosis and Data Processing Techniques for Substrate Transfer Robots Using Vibration Sensor Data

  • MD Saiful Islam;Mi-Jin Kim;Kyo-Mun Ku;Hyo-Young Kim;Kihyun Kim
    • Journal of the Microelectronics and Packaging Society
    • /
    • v.31 no.2
    • /
    • pp.45-53
    • /
    • 2024
  • The maintenance of semiconductor equipment is crucial for the continuous growth of the semiconductor market. System management is imperative given the anticipated increase in the capacity and complexity of industrial equipment. Ensuring optimal operation of manufacturing processes is essential to maintaining a steady supply of numerous parts. Particularly, monitoring the status of substrate transfer robots, which play a central role in these processes, is crucial. Diagnosing failures of their major components is vital for preventive maintenance. Fault diagnosis methods can be broadly categorized into physics-based and data-driven approaches. This study focuses on data-driven fault diagnosis methods due to the limitations of physics-based approaches. We propose a methodology for data acquisition and preprocessing for robot fault diagnosis. Data is gathered from vibration sensors, and the data preprocessing method is applied to the vibration signals. Subsequently, the dataset is trained using Gradient Tree-based XGBoost machine learning classification algorithms. The effectiveness of the proposed model is validated through performance evaluation metrics, including accuracy, F1 score, and confusion matrix. The XGBoost classifiers achieve an accuracy of approximately 92.76% and an equivalent F1 score. ROC curves indicate exceptional performance in class discrimination, with 100% discrimination for the normal class and 98% discrimination for abnormal classes.

A comparative study of Depth Preprocessing Method for 3D Data Service Based on Depth Image Based Rendering over T-DMB (지상파 DMB에서의 깊이 영상 기반 렌더링 기반의 3차원 서비스를 위한 깊이 영상 전처리 기술의 비교 연구)

  • Oh, Young-Jin;Jung, Kwang-Hee;Kim, Joong-Kyu;Lee, Gwang-Soon;Lee, Hyun;Hur, Nam-Ho;Kim, Jin-Woong
    • Proceedings of the IEEK Conference
    • /
    • 2008.06a
    • /
    • pp.815-816
    • /
    • 2008
  • In this paper, we evaluate depth image preprocessing for 3D data service based on DIBR over T-DMB. We evaluate two preprocessing methods of depth images. These are gaussian smoothing and adaptive smoothing. The results show that adaptive smoothing is more suitable for images with sharp transition of depth.

  • PDF

A Facial Expression Recognition Method Using Two-Stream Convolutional Networks in Natural Scenes

  • Zhao, Lixin
    • Journal of Information Processing Systems
    • /
    • v.17 no.2
    • /
    • pp.399-410
    • /
    • 2021
  • Aiming at the problem that complex external variables in natural scenes have a greater impact on facial expression recognition results, a facial expression recognition method based on two-stream convolutional neural network is proposed. The model introduces exponentially enhanced shared input weights before each level of convolution input, and uses soft attention mechanism modules on the space-time features of the combination of static and dynamic streams. This enables the network to autonomously find areas that are more relevant to the expression category and pay more attention to these areas. Through these means, the information of irrelevant interference areas is suppressed. In order to solve the problem of poor local robustness caused by lighting and expression changes, this paper also performs lighting preprocessing with the lighting preprocessing chain algorithm to eliminate most of the lighting effects. Experimental results on AFEW6.0 and Multi-PIE datasets show that the recognition rates of this method are 95.05% and 61.40%, respectively, which are better than other comparison methods.