• The Korean Journal of Applied Statistics
• /
• v.35 no.3
• /
• pp.395-406
• /
• 2022

This paper studies the outlier detection method for multivariate long memory time series. The existing outlier detection methods are based on a short memory VARMA model, so they are not suitable for multivariate long memory time series. It is because higher order of autoregressive model is necessary to account for long memory, however, it can also induce estimation instability as the number of parameter increases. To resolve this issue, we propose outlier detection methods based on the VHAR structure. We also adapt the robust estimation method to estimate VHAR coefficients more efficiently. Our simulation results show that our proposed method performs well in detecting outliers in multivariate long memory time series. Empirical analysis with stock index shows RVHAR model finds additional outliers that existing model does not detect.

• The Korean Journal of Applied Statistics
• /
• v.34 no.3
• /
• pp.507-521
• /
• 2021

Detecting outliers among high-dimensional data encounters a challenging problem of screening the variables since relevant information is often contained in only a few of the variables. Otherwise, when a number of irrelevant variables are included in the data, the distances between all observations tend to become similar which leads to making the degree of outlierness of all observations alike. The subspace outlier detection method overcomes the problem by measuring the degree of outlierness of the observation based on the relevant subsets of the entire variables. In this paper, we survey recent subspace outlier detection techniques, classifying them into three major types according to the subspace selection method. And we summarize the techniques of each type based on how to select the relevant subspaces and how to measure the degree of outlierness. In addition, we introduce some computing tools for implementing the subspace outlier detection techniques and present results from the simulation study and real data analysis.

• Journal of Digital Convergence
• /
• v.14 no.7
• /
• pp.309-319
• /
• 2016

There is a need to develop a data quality management algorithm in order to improve the quality of health care data. In this study, we developed a data quality control algorithms associated diseases related to diabetes in patients with hypertension. To make a data quality algorithm, we extracted hypertension patients from 2011 and 2012 discharge damage survey data. As the result of developing Data quality management algorithm, significant factors in hypertension patients with diabetes are gender, age, Glomerular disorders in diabetes mellitus, Diabetic retinopathy, Diabetic polyneuropathy, Closed [percutaneous] [needle] biopsy of kidney. Depending on the decision tree results, we defined Outlier which was probability values associated with a patient having diabetes corporal with hypertension or more than 80%, or not more than 20%, and found six groups with extreme values for diabetes accompanying hypertension patients. Thus there is a need to check the actual data contained in the Outlier(extreme value) groups to improve the quality of the data.

• Journal of the Korean Operations Research and Management Science Society
• /
• v.36 no.1
• /
• pp.39-55
• /
• 2011

Process monitoring has been emphasized for the monitoring of complex system such as chemical processing industries to achieve the efficiency enhancement, quality management, safety improvement. Recently, ICA (Independent Component Analysis) based MSPC (Multivariate Statistical Process Control) was widely used in process monitoring approaches. Moreover, DICA (Dynamic ICA) has been introduced to consider the system dynamics. However, the existing approaches show the limitation that their performances are strongly dependent on the statistical distributions of control variables. To improve the limitation, we propose a novel approach for process monitoring by integrating DICA and LOF (Local Outlier Factor). In this paper, we aim to improve the fault detection rate with the proposed method. LOF detects local outliers by using density of surrounding space so that its performance is regardless of data distribution. Therefore, the proposed method not only can consider the system dynamics but can also assure robust performance regardless of the statistical distributions of control variables. Comparison experiments were conducted on the widely used benchmark dataset, Tennessee Eastman process (TE process), and showed the improved performance than existing approaches.

• The Journal of Society for e-Business Studies
• /
• v.15 no.4
• /
• pp.303-317
• /
• 2010

In this paper, we propose a novel approach to fault prediction for real-time business process monitoring method using extended KNN imputation based LOF prediction. Existing rule-based approaches to process monitoring has some limitations like late alarm for fault occurrence or no indicators about real-time progress, since there exist unobserved attributes according to the monitoring phase during process executions. To improve these limitations, we propose an algorithm for LOF prediction by adopting the imputation method to assume unobserved attributes. LOF of ongoing instance is calculated by assuming next probable progresses after the monitoring phase, which is conducted during entire monitoring phases so that we can predict the abnormal termination of the ongoing instance. By visualizing the real-time progress in terms of the probability on abnormal termination, we can provide more proactive operations to opportunities or risks during the real-time monitoring.

• Journal of Korean Society of Transportation
• /
• v.27 no.2
• /
• pp.47-56
• /
• 2009

Effective detection and correction of outliers of raw traffic data collected from the field is of keen interest because reliable traffic information is highly dependent on the quality of raw data. Global positioning system (GPS) based traffic surveillance systems are capable of producing individual vehicle speeds that are invaluable for various traffic management and information strategies. This study proposed a locally weighted regression (LWR) based filtering method for individual vehicle speed data. An important feature of this study was to propose a technique to generate synthetic outliers for more systematic evaluation of the proposed method. It was identified by performance evaluations that the proposed LWR-based method outperformed an exponential smoothing. The proposed method is expected to be effectively utilized for filtering out raw individual vehicle speed data.

• The Journal of Bigdata
• /
• v.7 no.1
• /
• pp.125-137
• /
• 2022

The transportation industry is one of the important industries due to the geographical requirements surrounded by the sea on three sides of Korea and the problem of resource poverty, which relies on imports for most of its resource consumption. Among them, the proportion of the shipping industry is large enough to account for most of the transportation industry, and maintenance in the shipping industry is also important in improving the operational efficiency and reducing costs of ships. However, currently, inspections are conducted every certain period of time for maintenance of ships, resulting in time and cost, and the cause is not properly identified. Therefore, in this study, the proposed methodology, LSTM-AutoEncoder, is used to detect abnormalities that may cause ship failure by considering the time of actual ship operation data. In addition, clustering is performed through clustering, and the potential causes of ship main engine failure are identified by grouping outlier by factor. This enables faster monitoring of various information on the ship and identifies the degree of abnormality. In addition, the current ship's fault monitoring system will be equipped with a concrete alarm point setting and a fault diagnosis system, and it will be able to help find the maintenance time.

• The Journal of The Korea Institute of Intelligent Transport Systems
• /
• v.22 no.2
• /
• pp.80-92
• /
• 2023

As cities are becoming densely populated, previously unexpected events such as crimes, accidents, and infectious diseases are bound to affect user demands. With a time-series prediction of demand using information with uncertainty, it is impossible to derive reliable results. In particular, the COVID-19 outbreak in early 2020 caused changes in abnormal travel patterns and made it difficult to predict demand for time series. A methodology that accurately predicts demand by detecting and reflecting these changes is, therefore, required. The current study suggests a time series modeling pipeline that automatically detects and predicts abnormal events caused by COVID-19. We expect its wide application in various situations where there is a change in demand due to irregular and abnormal events.

• Journal of Platform Technology
• /
• v.12 no.1
• /
• pp.55-66
• /
• 2024

The K-POP market has become influential not only in culture but also in society as a whole, including diplomacy and environmental movements. As a result, various papers have been conducted based on machine learning to identify the success factors of idols by utilizing traditional data such as music and recordings. However, there is a limitation that previous studies have not reflected the influence of new media platforms such as Instagram releases, YouTube shorts, TikTok, Twitter, etc. on the popularity of idols. Therefore, it is difficult to clarify the causal relationship of recent idol success factors because the existing studies do not consider the daily changing media trends. To solve these problems, this paper proposes a data collection system and analysis methodology for idol-related data. By developing a container-based real-time data collection automation system that reflects the specificity of idol data, we secure the stability and scalability of idol data collection and compare and analyze the clusters of successful idols through a K-Means clustering-based outlier detection model. As a result, we were able to identify commonalities among successful idols such as gender, time of success after album release, and association with new media. Through this, it is expected that we can finally plan optimal comeback promotions for each idol, album type, and comeback period to improve the chances of idol success.