• Title/Summary/Keyword: Outlier

Search Result 654, Processing Time 0.025 seconds

Modeling of Strength of High Performance Concrete with Artificial Neural Network and Mahalanobis Distance Outlier Detection Method (신경망 이론과 Mahalanobis Distance 이상치 탐색방법을 이용한 고강도 콘크리트 강도 예측 모델 개발에 관한 연구)

  • Hong, Jung-Eui
    • Journal of Korean Society of Industrial and Systems Engineering
    • /
    • v.33 no.4
    • /
    • pp.122-129
    • /
    • 2010
  • High-performance concrete (HPC) is a new terminology used in concrete construction industry. Several studies have shown that concrete strength development is determined not only by the water-to-cement ratio but also influenced by the content of other concrete ingredients. HPC is a highly complex material, which makes modeling its behavior a very difficult task. This paper aimed at demonstrating the possibilities of adapting artificial neural network (ANN) to predict the comprresive strength of HPC. Mahalanobis Distance (MD) outlier detection method used for the purpose increase prediction ability of ANN. The detailed procedure of calculating Mahalanobis Distance (MD) is described. The effects of outlier compared with before and after artificial neural network training. MD outlier detection method successfully removed existence of outlier and improved the neural network training and prediction performance.

An Improved Iterative Procedure for Outlier Detection in Time Series (시계열 이상치 탐지를 위한 개선된 반복적 절차)

  • Bui, Anh Tuan;Jun, Chi-Hyuck
    • Journal of Korean Institute of Industrial Engineers
    • /
    • v.38 no.1
    • /
    • pp.17-24
    • /
    • 2012
  • We address some potential problems with the existing procedures of outlier detection in time series. Also we propose modifications in estimating model parameters and outlier effects in order to reduce the number of tests and to increase the detection accuracy. Experiments with some artificial data sets show that the proposed procedure significantly reduces the number of tests and enhances the accuracy of estimated parameters as well as the detection power.

On the Efficiency of Outlier Cleaners in Spatial Data Analysis (공간통계분석에서 이상점 수정방법의 효율성비교)

  • 이진희;신기일
    • The Korean Journal of Applied Statistics
    • /
    • v.17 no.2
    • /
    • pp.327-336
    • /
    • 2004
  • Many researchers have used the robust variogram to reduce the effect of outliers in spatial data analysis. Recently it is known that estimating the variogram after replacing outliers is more efficient. In this paper, we suggest a new data cleaner for geostatistic data analysis and compare the efficiency of outlier cleaners.

Fast Outlier Removal for Image Registration based on Modified K-means Clustering

  • Soh, Young-Sung;Qadir, Mudasar;Kim, In-Taek
    • Journal of the Institute of Convergence Signal Processing
    • /
    • v.16 no.1
    • /
    • pp.9-14
    • /
    • 2015
  • Outlier detection and removal is a crucial step needed for various image processing applications such as image registration. Random Sample Consensus (RANSAC) is known to be the best algorithm so far for the outlier detection and removal. However RANSAC requires a cosiderable computation time. To drastically reduce the computation time while preserving the comparable quality, a outlier detection and removal method based on modified K-means is proposed. The original K-means was conducted first for matching point pairs and then cluster merging and member exclusion step are performed in the modification step. We applied the methods to various images with highly repetitive patterns under several geometric distortions and obtained successful results. We compared the proposed method with RANSAC and showed that the proposed method runs 3~10 times faster than RANSAC.

Simultaneous outlier detection and variable selection via difference-based regression model and stochastic search variable selection

  • Park, Jong Suk;Park, Chun Gun;Lee, Kyeong Eun
    • Communications for Statistical Applications and Methods
    • /
    • v.26 no.2
    • /
    • pp.149-161
    • /
    • 2019
  • In this article, we suggest the following approaches to simultaneous variable selection and outlier detection. First, we determine possible candidates for outliers using properties of an intercept estimator in a difference-based regression model, and the information of outliers is reflected in the multiple regression model adding mean shift parameters. Second, we select the best model from the model including the outlier candidates as predictors using stochastic search variable selection. Finally, we evaluate our method using simulations and real data analysis to yield promising results. In addition, we need to develop our method to make robust estimates. We will also to the nonparametric regression model for simultaneous outlier detection and variable selection.

Indoor Positioning System Using Robust Outlier Extended Kalman Filter (이상 잡음에 강인한 확장 칼만 필터를 이용한 실내 위치 추정 시스템)

  • Kim, Dong-Seon;Yeom, Hak-Sun;Kim, Sun-Woo
    • The Journal of Korean Institute of Electromagnetic Engineering and Science
    • /
    • v.20 no.9
    • /
    • pp.954-960
    • /
    • 2009
  • In this paper, Indoor Positioning System based on Wi-Fi system which is one of the key technology in LBS(Location Based Service) is proposed. The proposed system estimates distance between MS(Mobile Station) and AP(Access Point) using RSSI(Received Signal Strength Indicator). RSSI is affected by outlier that originate from indoor environment complexity and obstacle. In this paper, we introduce a Robust outlier Extended Kalman Filter that can ignore, real-time outlier in the observations. To demonstrate performance of proposed indoor positioning system, we used a PDA as the MS.

Outlier Detection Using Support Vector Machines (서포트벡터 기계를 이용한 이상치 진단)

  • Seo, Han-Son;Yoon, Min
    • Communications for Statistical Applications and Methods
    • /
    • v.18 no.2
    • /
    • pp.171-177
    • /
    • 2011
  • In order to construct approximation functions for real data, it is necessary to remove the outliers from the measured raw data before constructing the model. Conventionally, visualization and maximum residual error have been used for outlier detection, but they often fail to detect outliers for nonlinear functions with multidimensional input. Although the standard support vector regression based outlier detection methods for nonlinear function with multidimensional input have achieved good performance, they have practical issues in computational cost and parameter adjustments. In this paper we propose a practical approach to outlier detection using support vector regression that reduces computational time and defines outlier threshold suitably. We apply this approach to real data examples for validity.

Fuzzy Learning Rule Using the Distance between Datum and the Centroids of Clusters (데이터와 클러스터들의 대표값들 사이의 거리를 이용한 퍼지학습법칙)

  • Kim, Yong-Soo
    • Journal of the Korean Institute of Intelligent Systems
    • /
    • v.17 no.4
    • /
    • pp.472-476
    • /
    • 2007
  • Learning rule affects importantly the performance of neural network. This paper proposes a new fuzzy learning rule that uses the learning rate considering the distance between the input vector and the prototypes of classes. When the learning rule updates the prototypes of classes, this consideration reduces the effect of outlier on the prototypes of classes. This comes from making the effect of the input vector, which locates near the decision boundary, larger than an outlier. Therefore, it can prevents an outlier from deteriorating the decision boundary. This new fuzzy learning rule is integrated into IAFC(Integrated Adaptive Fuzzy Clustering) fuzzy neural network. Iris data set is used to compare the performance of the proposed fuzzy neural network with those of other supervised neural networks. The results show that the proposed fuzzy neural network is better than other supervised neural networks.

Compound Outlier Assessment and Verification for Multiple Field Monitoring Data (다수 계측 데이터에 대한 복합 이상치 평가 및 검증)

  • Jeon, Jesung
    • Journal of the Korean GEO-environmental Society
    • /
    • v.19 no.1
    • /
    • pp.5-14
    • /
    • 2018
  • All kinds of monitoring data in construction site could have outlier created from diverse cause. In this study generation technique of synthesis value, its regression, final outlier detection and assessment are conducted to distinct outlier data included in extensive time series dataset. Synthesis value having weight factor of correlation between a number of datasets consist of many monitoring data enable to detect outlier by increasing its correlation. Standard artificial dataset in which intentional outliers are inserted has been used for assessment of synthesis value technique. These results showed increase of detection accuracy for outlier and general tendency in case of having different time series models in common. Accuracy of outlier detection increased in case of using more dataset and showing similar time series pattern.

Outlier detection in time series data (시계열 자료에서의 특이치 발견)

  • Choi, Jeong In;Um, In Ok;Choa, Hyung Jun
    • The Korean Journal of Applied Statistics
    • /
    • v.29 no.5
    • /
    • pp.907-920
    • /
    • 2016
  • This study suggests an outlier detection algorithm that uses quantile autoregressive model in time series data, eventually applying it to actual stock manipulation cases by comparing its performance to existing methods. Studies on outlier detection have traditionally been conducted mostly in general data and those in time series data are insufficient. They have also been limited to a parametric model, which is not convenient as it is complicated with an analysis that takes a long time. Thus, we suggest a new algorithm of outlier detection in time series data and through various simulations, compare it to existing algorithms. Especially, the outlier detection algorithm in time series data can be useful in finding stock manipulation. If stock price which had a certain pattern goes out of flow and generates an outlier, it can be due to intentional intervention and manipulation. We examined how fast the model can detect stock manipulations by applying it to actual stock manipulation cases.