• Title/Summary/Keyword: Outlier

Search Result 653, Processing Time 0.03 seconds

Outlier Detection in Random Effects Model Using Fractional Bayes Factor

  • Chung, Younshik
    • Communications for Statistical Applications and Methods
    • /
    • v.7 no.1
    • /
    • pp.141-150
    • /
    • 2000
  • In this paper we propose a method of computing Bayes factor to detect an outlier in a random effects model. When no information is available and hence improper noninformative priors should be used Bayes factor includes the unspecified constants and has complicated computational burden. To solve this problem we use the fractional Bayes factor (FBF) of O-Hagan(1995) and the generalized Savage0-Dickey density ratio of Verdinelli and Wasserman (1995) The proposed method is applied to outlier deterction problem We perform a simulation of the proposed approach with a simulated data set including an outlier and also analyze a real data set.

  • PDF

Dam Sensor Outlier Detection using Mixed Prediction Model and Supervised Learning

  • Park, Chang-Mok
    • International journal of advanced smart convergence
    • /
    • v.7 no.1
    • /
    • pp.24-32
    • /
    • 2018
  • An outlier detection method using mixed prediction model has been described in this paper. The mixed prediction model consists of time-series model and regression model. The parameter estimation of the prediction model was performed using supervised learning and a genetic algorithm is adopted for a learning method. The experiments were performed in artificial and real data set. The prediction performance is compared with the existing prediction methods using artificial data. Outlier detection is conducted using the real sensor measurements in a dam. The validity of the proposed method was shown in the experiments.

Outlier Detection By Clustering-Based Ensemble Model Construction (클러스터링 기반 앙상블 모델 구성을 이용한 이상치 탐지)

  • Park, Cheong Hee;Kim, Taegong;Kim, Jiil;Choi, Semok;Lee, Gyeong-Hoon
    • KIPS Transactions on Software and Data Engineering
    • /
    • v.7 no.11
    • /
    • pp.435-442
    • /
    • 2018
  • Outlier detection means to detect data samples that deviate significantly from the distribution of normal data. Most outlier detection methods calculate an outlier score that indicates the extent to which a data sample is out of normal state and determine it to be an outlier when its outlier score is above a given threshold. However, since the range of an outlier score is different for each data and the outliers exist at a smaller ratio than the normal data, it is very difficult to determine the threshold value for an outlier score. Further, in an actual situation, it is not easy to acquire data including a sufficient amount of outliers available for learning. In this paper, we propose a clustering-based outlier detection method by constructing a model representing a normal data region using only normal data and performing binary classification of outliers and normal data for new data samples. Then, by dividing the given normal data into chunks, and constructing a clustering model for each chunk, we expand it to the ensemble method combining the decision by the models and apply it to the streaming data with dynamic changes. Experimental results using real data and artificial data show high performance of the proposed method.

Outlier Detection in Time Series Monitoring Datasets using Rule Based and Correlation Analysis Method (규칙기반 및 상관분석 방법을 이용한 시계열 계측 데이터의 이상치 판정)

  • Jeon, Jesung;Koo, Jakap;Park, Changmok
    • Journal of the Korean GEO-environmental Society
    • /
    • v.16 no.5
    • /
    • pp.43-53
    • /
    • 2015
  • In this study, detection methods of outlier in various monitoring data that fit into big data category were developed and outlier detections were conducted for both artificial data and real field monitoring data. Rule-based methods applied rate of change and probability of error for monitoring data are effective to detect a large-scale short faults and constant faults having no change within a certain period. There are however, problems with misjudgement that consider the normal data with a large scale variation as outlier caused by using independent single dataset. Rule-based methods for noise faults detection have a limit to application of real monitoring data due to the problem with a choice of proper window size of data and finding of threshold for outlier judgment. A correlation analysis among different two datasets were very effective to detect localized outlier and abnormal variation for short and long-term monitoring dataset if reasonable range of training data could be selected.

A Score test for Detection of Outliers in Nonlinear Regression

  • Kahng, Myung-Wook
    • Journal of the Korean Statistical Society
    • /
    • v.22 no.2
    • /
    • pp.201-208
    • /
    • 1993
  • Given the specific mean shift outlier model, the score test for multiple outliers in nonlinear regression is discussed as an alternative to the likelihood ratio test. The geometric interpretation of the score statistic is also presented.

  • PDF

Testing Outliers in Nonlinear Regression

  • Kahng, Myung-Wook
    • Journal of the Korean Statistical Society
    • /
    • v.24 no.2
    • /
    • pp.419-437
    • /
    • 1995
  • Given the specific mean shift outlier model, several standard approaches to obtaining test statistic for outliers are discussed. Each of these is developed in detail for the nonlinear regression model, and each leads to an equivalent distribution. The geometric interpretations of the statistics and accuracy of linear approximation are also presented.

  • PDF

Asymptotic Properties of Outlier Tests in Nonlinear Regression

  • Kahng, Myung-Wook
    • Journal of the Korean Data and Information Science Society
    • /
    • v.17 no.1
    • /
    • pp.205-211
    • /
    • 2006
  • For a linear regression model, the necessary and sufficient condition for the asymptotic consistency of the outlier test statistic is known. An analogous condition for the nonlinear regression model is considered in this paper.

  • PDF

A Novel Battery State of Health Estimation Method Based on Outlier Detection Algorithm

  • Piao, Chang-hao;Hu, Zi-hao;Su, Ling;Zhao, Jian-fei
    • Journal of Electrical Engineering and Technology
    • /
    • v.11 no.6
    • /
    • pp.1802-1811
    • /
    • 2016
  • A novel battery SOH estimation algorithm based on outlier detection has been presented. The Battery state of health (SOH) is one of the most important parameters that describes the usability state of the power battery system. Firstly, a battery system model with lifetime fading characteristic was established, and the battery characteristic parameters were acquired from the lifetime fading process. Then, the outlier detection method based on angular distribution was used to identify the outliers among the battery behaviors. Lastly, the functional relationship between battery SOH and the outlier distribution was obtained by polynomial fitting method. The experimental results show that the algorithm can identify the outliers accurately, and the absolute error between the SOH estimation value and true value is less than 3%.

Fused Navigation of Unmanned Surface Vehicle and Detection of GPS Abnormality (무인 수상정의 융합 항법 및 GPS 이상 검출)

  • Ko, Nak Yong;Jeong, Seokki
    • Journal of Institute of Control, Robotics and Systems
    • /
    • v.22 no.9
    • /
    • pp.723-732
    • /
    • 2016
  • This paper proposes an approach to fused navigation of an unmanned surface vehicle(USV) and to detection of the outlier or interference of global positioning system(GPS). The method fuses available sensor measurements through extended Kalman filter(EKF) to find the location and attitude of the USV. The method uses error covariance of EKF for detection of GPS outlier or interference. When outlier or interference of the GPS is detected, the method excludes GPS data from navigation process. The measurements to be fused for the navigation are GPS, acceleration, angular rate, magnetic field, linear velocity, range and bearing to acoustic beacons. The method is tested through simulated data and measurement data produced through ground navigation. The results show that the method detects GPS outlier or interference as well as the GPS recovery, which frees navigation from the problem of GPS abnormality.