• Title/Summary/Keyword: Outliers

Search Result 655, Processing Time 0.032 seconds

A Study of Statistical Approach for Detection of Outliers in Network Traffic

  • Kim, Sahm-Yeong;Yun, Joo-Beom;Park, Eung-Ki
    • Journal of the Korean Data and Information Science Society
    • /
    • v.16 no.4
    • /
    • pp.979-987
    • /
    • 2005
  • In this research we study conventional and new statistical methods to analyse and detect outliers in network traffic and we apply the nonlinear time series model to make better performance of detecting abnormal traffic rather the linear time series model to compare the performances of the two models.

  • PDF

Efficient Estimation of the Parameters of the Pareto Distribution in the Presence of Outliers

  • Dixit, U.J.;Jabbari Nooghabi, M.
    • Communications for Statistical Applications and Methods
    • /
    • v.18 no.6
    • /
    • pp.817-835
    • /
    • 2011
  • The moment(MM) and least squares(LS) estimations of the parameters are derived for the Pareto distribution in the presence of outliers. Further, we have derived a mixture method(MIX) of estimations with MM and LS that shows that the MIX is more efficient. In the final section we have given an example of actual data from a medical insurance company.

Weighted Least Absolute Deviation Lasso Estimator

  • Jung, Kang-Mo
    • Communications for Statistical Applications and Methods
    • /
    • v.18 no.6
    • /
    • pp.733-739
    • /
    • 2011
  • The linear absolute shrinkage and selection operator(Lasso) method improves the low prediction accuracy and poor interpretation of the ordinary least squares(OLS) estimate through the use of $L_1$ regularization on the regression coefficients. However, the Lasso is not robust to outliers, because the Lasso method minimizes the sum of squared residual errors. Even though the least absolute deviation(LAD) estimator is an alternative to the OLS estimate, it is sensitive to leverage points. We propose a robust Lasso estimator that is not sensitive to outliers, heavy-tailed errors or leverage points.

A Study On Detecting Outliers In Two-Way Tables (이원배치법(二元配置法)에서의 이상치(異常置) 발견방법(發見方法)에 대한 연구(硏究))

  • Gang, Eun-Mi
    • Journal of Korean Society for Quality Management
    • /
    • v.15 no.1
    • /
    • pp.63-67
    • /
    • 1987
  • Basic problems in the study of detecting outliers from data of experimental designs are that they are difficult to detect and their presence influences the analysis of variance of the data set. This article is concerned with mainly detecting outliers in two-way tables with no replications. Various methods are reviewed and their relations to the Andrews-Pregibon's Statistic and Cook's Statistic are derived.

  • PDF

Statistical Outliers in Florida Counties at the Presidential Election 2000 (2000년 미국대선 플로리다주의 투표결과 분석)

  • 김현철
    • The Korean Journal of Applied Statistics
    • /
    • v.15 no.1
    • /
    • pp.21-32
    • /
    • 2002
  • We searched out in the votes data of the State of Florida at presidential election 2000. We used a multivariate regression analysis. We got there were several outliers including Palm Beach County. It means that we should analyze the number of disqualified ballots which were double-punched as well as the votes, to insist the " Butterfly Ballot" made Palm Beach outlier.

A Score Test for Detection of Outliers in Generalized Linear Models

  • Kahng, Myung-Wook;Kim, Min-Kyung
    • Journal of the Korean Data and Information Science Society
    • /
    • v.15 no.1
    • /
    • pp.129-139
    • /
    • 2004
  • We consider the problem of testing for outliers in generalized linear model. We proceed by first specifying a mean shift outlier model, assuming the suspect set of ourliers is known. Given this model, we discuss standard approaches to obtaining score test for outliers as an alternative to the likelihood ratio test.

  • PDF

Robust Designs to Outliers for Response Surface Experiments

  • Jeong B. Yoo;Park, Sung H.
    • Journal of the Korean Statistical Society
    • /
    • v.20 no.2
    • /
    • pp.147-155
    • /
    • 1991
  • This paper treats a robust design criterion which minimizes the effects of outliers and model inadequacy, and investigates robust designs for some response surface designs. In order to develop a robust design criterion and robust design, the integrated mean squared error of *(equation omitted) over a region is utilized, where *(equation omitted). is the estimated response by the minimum bias estimation proposed by carson, Manson and Hader (1969) . According to the number of aberrant observations and their positions, the proposed criterion and designs are studied. Also further development of the proposed criterion is treated when outliers can occur in any position of a design.

  • PDF

Unified methods for variable selection and outlier detection in a linear regression

  • Seo, Han Son
    • Communications for Statistical Applications and Methods
    • /
    • v.26 no.6
    • /
    • pp.575-582
    • /
    • 2019
  • The problem of selecting variables in the presence of outliers is considered. Variable selection and outlier detection are not separable problems because each observation affects the fitted regression equation differently and has a different influence on each variable. We suggest a simultaneous method for variable selection and outlier detection in a linear regression model. The suggested procedure uses a sequential method to detect outliers and uses all possible subset regressions for model selections. A simplified version of the procedure is also proposed to reduce the computational burden. The procedures are compared to other variable selection methods using real data sets known to contain outliers. Examples show that the proposed procedures are effective and superior to robust algorithms in selecting the best model.

Development of a Observational Settlement Analysis Method Using Outliers (이상치를 이용한 관측적 침하예측기법의 개발)

  • 우철웅;장병욱
    • Magazine of the Korean Society of Agricultural Engineers
    • /
    • v.45 no.5
    • /
    • pp.140-150
    • /
    • 2003
  • Observational methods such as the Asaoka's method and the hyperbolic method are widely applied on the settlement analysis using observed settlement. The most unreliable aspects in those methods is arose from the subjective discretion of initial non-linearity on linear regression. The initial non-linearity is inevitable due to the settlement behaviour itself. Therefore an objective method is essential to achieve more reliable results on settlement analysis. It was found that the initial non-linear data are statistical outliers. New automation algorithms of the hyperbolic and the Asaoka's method were developed based on outlier detection method. The methods are a successive detection of outliers and a searching method of suitable hyperbolic range for the Asaoka's and the hyperbolic method respectively. Applicability of the algorithms was verified through case studies.

A study of a new statistic for detection of outliers and/or influential observations in regression diagnostics (회귀진단에서 이상치와 영향관측치를 동시에 발견하는 새로운 통계량에 관한 연구)

  • 강은미
    • The Korean Journal of Applied Statistics
    • /
    • v.6 no.1
    • /
    • pp.67-78
    • /
    • 1993
  • A new diagnostic statistic for detecting outliers and influential observations in linear models is suggested and studied in this paper. The proposed statistic is a weighted sum of two measures; one is for detecting outliers and the other is for detecting influential observations. The merit of this statistic is that it is possible to distinguish outliers from influential observations. We have done some Monte-Carlo Simulation to find the probability distribution of this statistic.

  • PDF