• Communications for Statistical Applications and Methods
• /
• v.3 no.2
• /
• pp.53-67
• /
• 1996

Numerous classical as well as robust methods have been proposed in the literature for the detection of multiple outlier in multivariate data. The effectiveness and power of each of these methods have not been thoroughly investigated. In this paper we first reduce the vast number of outlier detection methods to a small number of viable ones. This reduction is based on previous work of other researches and on some theoretical arguments. Then we design and implement a Monte Carlo experiment for comparing these methods. The main goal of our study is to determine which methods are most powerful in the detection of multiple outlier and in dealing with the masking and swamping problems. The results of the Monte Carlo study indicate that two of the methods seem to hace better performances than the others for the detection of multiple outlier in multivariate data.

• The Korean Journal of Applied Statistics
• /
• v.35 no.3
• /
• pp.395-406
• /
• 2022

This paper studies the outlier detection method for multivariate long memory time series. The existing outlier detection methods are based on a short memory VARMA model, so they are not suitable for multivariate long memory time series. It is because higher order of autoregressive model is necessary to account for long memory, however, it can also induce estimation instability as the number of parameter increases. To resolve this issue, we propose outlier detection methods based on the VHAR structure. We also adapt the robust estimation method to estimate VHAR coefficients more efficiently. Our simulation results show that our proposed method performs well in detecting outliers in multivariate long memory time series. Empirical analysis with stock index shows RVHAR model finds additional outliers that existing model does not detect.

• The Journal of Korea Institute of Information, Electronics, and Communication Technology
• /
• v.14 no.1
• /
• pp.22-28
• /
• 2021

Multivariate processes, such as chemical and mechanical process, power plants are operated in a state where several facilities are complexly connected, the fault of a particular system can also have fatal consequences for the entire process. In addition, since process data is measured in an unstable environment, outlier is likely to be include in the data. Therefore, monitoring technology is essential, which can remove outlier from measured data and detect failures in advance. In this paper, data obtained from dynamic and multivariate process models was used to detect fault in various type of processes. The dynamic process is a simulation of a process with autoregressive property, and the multivariate process is a model that describes a situation when a specific sensor fault. Mahalanobis distance was used to remove outlier contained in the data generated by dynamic process model and multivariate process model, and fault detection was performed using ICA. For comparison, we compared performance with and a conventional single ICA method. The proposed fault detection method improves performance by 0.84%p for bias data and 6.82%p for drift data in the dynamic process. In the case of the multivariate process, the performance was improves by 3.78%p, therefore, the proposed method showed better fault detection performance.

• Journal of Korean Society of Coastal and Ocean Engineers
• /
• v.33 no.6
• /
• pp.265-274
• /
• 2021

Outlier detection research in ocean data has traditionally been performed using statistical and distance-based machine learning algorithms. Recently, AI-based methods have received a lot of attention and so-called supervised learning methods that require classification information for data are mainly used. This supervised learning method requires a lot of time and costs because classification information (label) must be manually designated for all data required for learning. In this study, an autoencoder based on unsupervised learning was applied as an outlier detection to overcome this problem. For the experiment, two experiments were designed: one is univariate learning, in which only SST data was used among the observation data of Deokjeok Island and the other is multivariate learning, in which SST, air temperature, wind direction, wind speed, air pressure, and humidity were used. Period of data is 25 years from 1996 to 2020, and a pre-processing considering the characteristics of ocean data was applied to the data. An outlier detection of actual SST data was tried with a learned univariate and multivariate autoencoder. We tried to detect outliers in real SST data using trained univariate and multivariate autoencoders. To compare model performance, various outlier detection methods were applied to synthetic data with artificially inserted errors. As a result of quantitatively evaluating the performance of these methods, the multivariate/univariate accuracy was about 96%/91%, respectively, indicating that the multivariate autoencoder had better outlier detection performance. Outlier detection using an unsupervised learning-based autoencoder is expected to be used in various ways in that it can reduce subjective classification errors and cost and time required for data labeling.

• Communications for Statistical Applications and Methods
• /
• v.1 no.1
• /
• pp.33-40
• /
• 1994

The conditional expectation of a random variable in a multivariate normal random vector is a multiple linear regression on its predecessors. Using this fact, the least median of squares estimation method developed in a multiple linear regression is adapted to a multivariate data to identify influential observations. The resulting method clearly detect outliers and it avoids the masking effect.

• The Korean Journal of Applied Statistics
• /
• v.25 no.6
• /
• pp.1019-1026
• /
• 2012

This paper presents a method for the identification of "edge observations" located on a boundary area constructed by a truncation variable as well as for the identification of outliers and the after fit of multivariate skew $t$-distribution(MST) to asymmetric data. The detection of edge observation is important in data analysis because it provides information on a certain critical area in observation space. The proposed method is applied to an Australian Institute of Sport(AIS) dataset that is well known for asymmetry in data space.

• Annual Conference of KIPS
• /
• 2023.05a
• /
• pp.417-419
• /
• 2023

Hypertension is one of the leading causes of mortality worldwide. In recent years, the incidence of hypertension has increased dramatically, not only among the elderly but also among young people. In this regard, the use of machine-learning methods to diagnose the causes of hypertension has increased in recent years. In this study, we improved the prediction of hypertension detection using Mahalanobis distance-based multivariate outlier removal using the KNHANES database from the Korean national health data and the COVID-19 dataset from Kaggle. This study was divided into two modules. Initially, the data preprocessing step used merged datasets and decision-tree classifier-based feature selection. The next module applies a predictive analysis step to remove multivariate outliers using the Mahalanobis distance from the experimental dataset and makes a prediction of hypertension. In this study, we compared the accuracy of each classification model. The best results showed that the proposed MAH_RF algorithm had an accuracy of 82.66%. The proposed method can be used not only for hypertension but also for the detection of various diseases such as stroke and cardiovascular disease.

• Journal of applied mathematics & informatics
• /
• v.26 no.5_6
• /
• pp.1133-1138
• /
• 2008

A general method of detecting outliers based on a change of likelihood by using the influence function is suggested. It can be applied to all kinds of distributions that are specified by parameters. For the multivariate normal case, specific computations are made to get the corresponding conditional influence function. A numerical example is provided for illustration.

• Journal of Korean Society for Quality Management
• /
• v.29 no.1
• /
• pp.1-10
• /
• 2001

We consider the problem of detecting special variations in multivariate $T^2$-control chart when two or more multivariate outliers are present. Since a multivariate outlier may reflect slippage in mean, variance, or correlation, it can distort the sample mean vector and sample covariance matrix. Damaged sample mean vector and sample covariance matrix have difficulty in examining special variations clearly, An alternative to detection outliers or special variations is to use robust estimators of mean vector and covariance matrix that are less sensitive to extreme observations than are the standard estimators $\bar{x}$ and $\textbf{S}$. We applied popular minimum volume ellipsoid(MVE) and minimum covariance determinant(MCD) method to estimate mean vector and covariance matrix and compared its results with standard $T^2$-control chart using simulated multivariate data with outliers. We found that the modified $T^2$-control chart based on the above robust methods were more effective in detecting special variations clearly than the standard $T^2$-control chart.

• Journal of the Korean Operations Research and Management Science Society
• /
• v.36 no.1
• /
• pp.39-55
• /
• 2011

Process monitoring has been emphasized for the monitoring of complex system such as chemical processing industries to achieve the efficiency enhancement, quality management, safety improvement. Recently, ICA (Independent Component Analysis) based MSPC (Multivariate Statistical Process Control) was widely used in process monitoring approaches. Moreover, DICA (Dynamic ICA) has been introduced to consider the system dynamics. However, the existing approaches show the limitation that their performances are strongly dependent on the statistical distributions of control variables. To improve the limitation, we propose a novel approach for process monitoring by integrating DICA and LOF (Local Outlier Factor). In this paper, we aim to improve the fault detection rate with the proposed method. LOF detects local outliers by using density of surrounding space so that its performance is regardless of data distribution. Therefore, the proposed method not only can consider the system dynamics but can also assure robust performance regardless of the statistical distributions of control variables. Comparison experiments were conducted on the widely used benchmark dataset, Tennessee Eastman process (TE process), and showed the improved performance than existing approaches.