• Title/Summary/Keyword: Outliers

Search Result 656, Processing Time 0.027 seconds

Least quantile squares method for the detection of outliers

  • Seo, Han Son;Yoon, Min
    • Communications for Statistical Applications and Methods
    • /
    • v.28 no.1
    • /
    • pp.81-88
    • /
    • 2021
  • k-least quantile of squares (k-LQS) estimates are a generalization of least median of squares (LMS) estimates. They have not been used as much as LMS because their breakdown points become small as k increases. But if the size of outliers is assumed to be fixed LQS estimates yield a good fit to the majority of data and residuals calculated from LQS estimates can be a reliable tool to detect outliers. We propose to use LQS estimates for separating a clean set from the data in the context of outlyingness of the cases. Three procedures are suggested for the identification of outliers using LQS estimates. Examples are provided to illustrate the methods. A Monte Carlo study show that proposed methods are effective.

Robust tests for heteroscedasticity using outlier detection methods (이상치 탐지법을 이용한 강건 이분산 검정)

  • Seo, Han Son;Yoon, Min
    • The Korean Journal of Applied Statistics
    • /
    • v.29 no.3
    • /
    • pp.399-408
    • /
    • 2016
  • There is a need to detect heteroscedasticity in a regression analysis; however, it invalidates the standard inference procedure. The diagnostics on heteroscedasticity may be distorted when both outliers and heteroscedasticity exist. Available heteroscedasticity detection methods in the presence of outliers usually use robust estimators or separating outliers from the data. Several approaches have been suggested to identify outliers in the heteroscedasticity problem. In this article conventional tests on heteroscedasticity are modified by using a sequential outlier detection methods to separate outliers from contaminated data. The performance of the proposed method is compared with original tests by a Monte Carlo study and examples.

Outlier Impact on the Power of Significance Test for Cronbach Alpha Reliability Coefficient

  • Yonghwan Um
    • Journal of the Korea Society of Computer and Information
    • /
    • v.28 no.5
    • /
    • pp.179-187
    • /
    • 2023
  • In this paper, we studied the impact of outliers on the power of the significance tests for Cronbach alpha reliability coefficient. Four variables were varied: sample size, the number of items, the number of outliers and population Cronbach Alpha levels. We simulated data using multivariate normal distribution and used outliers sampled from uniform distribution. To test the significance of Cronbach Alpha Reliability, parametric approach(F statistic) and permutation method were used. Consequently, we observed that the powers of permutation test are equal to or greater than those of F test under all conditions, and also both F test and permutation test lose the power as the number of outliers increases, and that these effects of outliers on the power are enhanced for increasing population alpha levels.

A Procedure for Indentifying Outliers in Multivariate Data (다변량 자료에서 다수 이상치 인식의 절차)

  • Yum, Joon-Keun;Park, Jong-Goo;Kim, Jong-Woo
    • Journal of Korean Society for Quality Management
    • /
    • v.23 no.4
    • /
    • pp.28-41
    • /
    • 1995
  • We consider the problem of identifying multiple outliers in linear model. The available regression diagnostic methods often do not succeed in detecting multiple outliers because of the masking and swamping effect. Recently, among the various robust estimator of reducing the effect of outliers, LMS(Least Meadian Square) estimator has been to be a suitable method proposed to expose outliers and leverage points. However, as you know it, the data analysis method with LMS estimator is to be taken the median of the squared residuals in the sample which is extracted the sample space. Then this model causes the trouble, for the number of the chosen sample is nCp, i.e. as the size of sample space n is increasing, the number is increasing fastly. And the covariance matrix may be the singular matrix, so that matrix is approching collinearity. Thus we propose a procedure ELMS for the resampling in LMS method and study the size of the effective elementary set in this algorithm.

  • PDF

Firework plot for evaluating the impact of outliers in statistical inference (통계적 추론에서 특이점의 영향을 평가하기 위한 탐색적 자료분석 그림도구로서의 불꽃그림)

  • Moon, Sungho
    • The Korean Journal of Applied Statistics
    • /
    • v.31 no.1
    • /
    • pp.155-165
    • /
    • 2018
  • Outliers and influential observations often distort many numerical measures for data analysis. Jang and Anderson-Cook (Quality and Reliability Engineering International, 30, 1409-1425, 2014) proposed a graphical firework plot method for exploratory analysis purpose to provide a possible visualization of the trace of the impact of the possible outlying and influential observations on the univariate/bivariate data analysis and regression. They developed 3-D plot as well as pairwise plot for the appropriate measures of interest. We use firework plots as a graphical exploratory data analysis tool to detect outliers and evaluate the impact of outliers in statistical inference.

A study on the Flood Frequency Analyzed in Consideration of Low Outliers. (Low Outliers를 고려한 홍수빈도분석에 관한 연구)

  • 이순혁;홍성표;박명근
    • Magazine of the Korean Society of Agricultural Engineers
    • /
    • v.30 no.4
    • /
    • pp.62-70
    • /
    • 1988
  • This study was conducted to solve the problems for the unsuitable parameters and the uncertainty of design flood can be appeared by low outliers were inclined to the lower part from the trend of the balance of the data. Derivation of reasonable design flood was attempted finally by modification of low outliers with analysis of flood frequency by means of Log Pearson Type Ill distribution. Three subwatersheds were selected as studying basins with the annual maximum series including low outliers along Geum River basin. The results through this study were analyzed and summarized as follows. 1. Log Pearson Type In distribution was confirmed as a reasonable one by X$^2$ goodness of fit test at Gong Ju, Gyu Am, og Cheon watershed along Geum River basin. 2. Probable flood flows for each watershed were derivated by flood frequency curve with outliers. 3. Weighted skew coefficient for each watershed was calculated for the evaluation of freq- uency factor which is needed for the modification of low outlier. 4. It was confirrned that adjusted frequency curve has a lower tendency than that of deletion of low outlier in common at all watersheds. 5. Final probable flood flows were derivated by modification with evaluation of modified basic statistics for three watersheds. 6. In comparison with a frequency curve with modification and one with outlier, The former has a higher probable flood flow within three years of return periods than that of the latter, and vice versa over three years of return periods.

  • PDF

Soft-Decision Based Quantization of the Multimedia Signal Considering the Outliers in Rate-Allocation and Distortion (이상 비트율 할당과 신호왜곡 문제점을 고려한 멀티미디어 신호의 연판정 양자화 방법)

  • Lim, Jong-Wook;Noh, Myung-Hoon;Kim, Moo-Young
    • The Journal of the Acoustical Society of Korea
    • /
    • v.29 no.4
    • /
    • pp.286-293
    • /
    • 2010
  • There are two major conventional quantization algorithms: resolution-constrained quantization (RCQ) and entropy-constrained quantization (ECQ). Although RCQ works well for fixed transmission-rate, it produces the distortion outliers since the cell sizes are different. Compared with RCQ, ECQ has the constraints on the cell size but it produces the rate outliers. We propose the cell-size constrained vector quantization (CCVQ) that improves the generalized Lloyd algorithm (GLA). The CCVQ algorithm is able to make a soft-decision between RCQ and ECQ by using the flexible penalty measure according to the cell size. Although the proposed method increases the small amount of overall mean-distortion, it can reduce the distortion outliers.

Outlier Detection in Growth Curve Model

  • Shim, Kyu-Bark
    • Journal of the Korean Data and Information Science Society
    • /
    • v.14 no.2
    • /
    • pp.313-323
    • /
    • 2003
  • For the growth curve model with arbitrary covariance structure, known as unstructured covariance matrix, the problems of detecting outliers are discussed in this paper. In order to detect outliers in the growth curve model, the test statistics using U-distribution is established. After detecting outliers in growth curve model, we test homo and/or hetero-geneous covariance matrices using PSR Quasi-Bayes Criterion. For illustration, one numerical example is discussed, which compares between before and after outlier deleting.

  • PDF

The Weight Function in BIRQ Estimator for the AR(1) Model with Additive Outliers

  • Jung Byoung Cheol;Han Sang Moon
    • Proceedings of the Korean Statistical Society Conference
    • /
    • 2004.11a
    • /
    • pp.129-134
    • /
    • 2004
  • In this study, we investigate the effects of the weight function in the bounded influence regression quantile (BIRQ) estimator for the AR(1) model with additive outliers. In order to down-weight the outliers of X-axis, the Mallows' (1973) weight function has been commonly used in the BIRQ estimator. However, in our Monte Carlo study, the BIRQ estimator using the Tukey's bisquare weight function shows less MSE and bias than that of using the Mallows' weight function or Huber's weight function.

  • PDF

Post-Processing for Reducing Corner Outliers (Corner outlier 제거를 위한 후처리 기법)

  • 홍윤표;전병우
    • Proceedings of the IEEK Conference
    • /
    • 2003.11a
    • /
    • pp.11-14
    • /
    • 2003
  • In block-based lossy video compression, severe quantization causes discontinuities along block boundaries so that annoying blocking artifacts are visible in decoded video imases. These blocking artifacts significantly decrease the subjective image quality. In order to reduce the blocking artifacts in decoded images, many algorithms have been proposed However studies on so called, corner outliers, have been very limited. Corner outliers make image edges look disconnected from those of neighboring blocks at cross block boundary. In order to solve this problem, we propose a corner outlier detection and compensation algorithm as post-processing in spatial domain The experiment results show that the proposed method provides much improved subjective image quality.

  • PDF