• Title/Summary/Keyword: Outlier model

Search Result 210, Processing Time 0.024 seconds

TIME-VARIANT OUTLIER DETECTION METHOD ON GEOSENSOR NETWORKS

  • Kim, Dong-Phil;I, Gyeong-Min;Lee, Dong-Gyu;Ryu, Keun-Ho
    • Proceedings of the KSRS Conference
    • /
    • 2008.10a
    • /
    • pp.410-413
    • /
    • 2008
  • Existing Outlier detections have been widely studied in geosensor networks. Recently, machine learning and data mining have been applied the outlier detection method to build a model that distinguishes outliers based on anchored criterion. However, it is difficult for the existing methods to detect outliers against incoming time-variant data, because outlier detection needs to monitor incoming data and classify irregular attacks. Therefore, in order to solve the problem, we propose a time-variant outlier detection using 2-dimensional grid method based on unanchored criterion. In the paper, outliers using geosensor data was performed to classify efficiently. The proposed method can be utilized applications such as network intrusion detection, stock market analysis, and error data detection in bank account.

  • PDF

Outlier Detection Based on Discrete Wavelet Transform with Application to Saudi Stock Market Closed Price Series

  • RASHEDI, Khudhayr A.;ISMAIL, Mohd T.;WADI, S. Al;SERROUKH, Abdeslam
    • The Journal of Asian Finance, Economics and Business
    • /
    • v.7 no.12
    • /
    • pp.1-10
    • /
    • 2020
  • This study investigates the problem of outlier detection based on discrete wavelet transform in the context of time series data where the identification and treatment of outliers constitute an important component. An outlier is defined as a data point that deviates so much from the rest of observations within a data sample. In this work we focus on the application of the traditional method suggested by Tukey (1977) for detecting outliers in the closed price series of the Saudi Arabia stock market (Tadawul) between Oct. 2011 and Dec. 2019. The method is applied to the details obtained from the MODWT (Maximal-Overlap Discrete Wavelet Transform) of the original series. The result show that the suggested methodology was successful in detecting all of the outliers in the series. The findings of this study suggest that we can model and forecast the volatility of returns from the reconstructed series without outliers using GARCH models. The estimated GARCH volatility model was compared to other asymmetric GARCH models using standard forecast error metrics. It is found that the performance of the standard GARCH model were as good as that of the gjrGARCH model over the out-of-sample forecasts for returns among other GARCH specifications.

A Test on a Specific Set of Outlier Candidates in a Linear Model (선형모형에서 특정 이상치 후보군에 대한 검정)

  • Seo, Han Son;Yoon, Min
    • The Korean Journal of Applied Statistics
    • /
    • v.27 no.2
    • /
    • pp.307-315
    • /
    • 2014
  • An exact distribution of the test statistic to test for multiple outlier candidates does not generally exist; therefore, tests of individual outliers (or tests using simulated critical-values) are usually conducted instead of testing for groups of outliers. This article is on procedures to test outlying observations. We suggest a method that can be applied to arbitrary observations or multiple outlier candidates detected by an outlier detecting method. A Monte Carlo study performance is used to compare the proposed method with others.

Smart contract research for data outlier detection and processing of ARIMA model

  • Min, Youn-A
    • International Journal of Internet, Broadcasting and Communication
    • /
    • v.14 no.4
    • /
    • pp.140-147
    • /
    • 2022
  • In this study, in order to efficiently detect data patterns and outliers in time series data, outlier detection processing is performed for each section based on a smart contract in the data preprocessing process, and parameters for the ARIMA model are determined by generating and reflecting the significance and outlier-related parameters of the data. It was created and applied to the modified arithmetic expression to lower the data abnormality. To evaluate the performance of this study, the normality of the data was compared and evaluated when the parameters of the general ARIMA model and the ARIMA model through this study were applied, and a performance improvement of more than 6% was confirmed.

A Novel Battery State of Health Estimation Method Based on Outlier Detection Algorithm

  • Piao, Chang-hao;Hu, Zi-hao;Su, Ling;Zhao, Jian-fei
    • Journal of Electrical Engineering and Technology
    • /
    • v.11 no.6
    • /
    • pp.1802-1811
    • /
    • 2016
  • A novel battery SOH estimation algorithm based on outlier detection has been presented. The Battery state of health (SOH) is one of the most important parameters that describes the usability state of the power battery system. Firstly, a battery system model with lifetime fading characteristic was established, and the battery characteristic parameters were acquired from the lifetime fading process. Then, the outlier detection method based on angular distribution was used to identify the outliers among the battery behaviors. Lastly, the functional relationship between battery SOH and the outlier distribution was obtained by polynomial fitting method. The experimental results show that the algorithm can identify the outliers accurately, and the absolute error between the SOH estimation value and true value is less than 3%.

An Improved Iterative Procedure for Outlier Detection in Time Series (시계열 이상치 탐지를 위한 개선된 반복적 절차)

  • Bui, Anh Tuan;Jun, Chi-Hyuck
    • Journal of Korean Institute of Industrial Engineers
    • /
    • v.38 no.1
    • /
    • pp.17-24
    • /
    • 2012
  • We address some potential problems with the existing procedures of outlier detection in time series. Also we propose modifications in estimating model parameters and outlier effects in order to reduce the number of tests and to increase the detection accuracy. Experiments with some artificial data sets show that the proposed procedure significantly reduces the number of tests and enhances the accuracy of estimated parameters as well as the detection power.

Outlier Detection in Growth Curve Model

  • Shim, Kyu-Bark
    • Journal of the Korean Data and Information Science Society
    • /
    • v.14 no.2
    • /
    • pp.313-323
    • /
    • 2003
  • For the growth curve model with arbitrary covariance structure, known as unstructured covariance matrix, the problems of detecting outliers are discussed in this paper. In order to detect outliers in the growth curve model, the test statistics using U-distribution is established. After detecting outliers in growth curve model, we test homo and/or hetero-geneous covariance matrices using PSR Quasi-Bayes Criterion. For illustration, one numerical example is discussed, which compares between before and after outlier deleting.

  • PDF

A Score Test for Detection of Outliers in Generalized Linear Models

  • Kahng, Myung-Wook;Kim, Min-Kyung
    • Journal of the Korean Data and Information Science Society
    • /
    • v.15 no.1
    • /
    • pp.129-139
    • /
    • 2004
  • We consider the problem of testing for outliers in generalized linear model. We proceed by first specifying a mean shift outlier model, assuming the suspect set of ourliers is known. Given this model, we discuss standard approaches to obtaining score test for outliers as an alternative to the likelihood ratio test.

  • PDF

Machine learning application to seismic site classification prediction model using Horizontal-to-Vertical Spectral Ratio (HVSR) of strong-ground motions

  • Francis G. Phi;Bumsu Cho;Jungeun Kim;Hyungik Cho;Yun Wook Choo;Dookie Kim;Inhi Kim
    • Geomechanics and Engineering
    • /
    • v.37 no.6
    • /
    • pp.539-554
    • /
    • 2024
  • This study explores development of prediction model for seismic site classification through the integration of machine learning techniques with horizontal-to-vertical spectral ratio (HVSR) methodologies. To improve model accuracy, the research employs outlier detection methods and, synthetic minority over-sampling technique (SMOTE) for data balance, and evaluates using seven machine learning models using seismic data from KiK-net. Notably, light gradient boosting method (LGBM), gradient boosting, and decision tree models exhibit improved performance when coupled with SMOTE, while Multiple linear regression (MLR) and Support vector machine (SVM) models show reduced efficacy. Outlier detection techniques significantly enhance accuracy, particularly for LGBM, gradient boosting, and voting boosting. The ensemble of LGBM with the isolation forest and SMOTE achieves the highest accuracy of 0.91, with LGBM and local outlier factor yielding the highest F1-score of 0.79. Consistently outperforming other models, LGBM proves most efficient for seismic site classification when supported by appropriate preprocessing procedures. These findings show the significance of outlier detection and data balancing for precise seismic soil classification prediction, offering insights and highlighting the potential of machine learning in optimizing site classification accuracy.

Outlier detection for multivariate long memory processes (다변량 장기 종속 시계열에서의 이상점 탐지)

  • Kim, Kyunghee;Yu, Seungyeon;Baek, Changryong
    • The Korean Journal of Applied Statistics
    • /
    • v.35 no.3
    • /
    • pp.395-406
    • /
    • 2022
  • This paper studies the outlier detection method for multivariate long memory time series. The existing outlier detection methods are based on a short memory VARMA model, so they are not suitable for multivariate long memory time series. It is because higher order of autoregressive model is necessary to account for long memory, however, it can also induce estimation instability as the number of parameter increases. To resolve this issue, we propose outlier detection methods based on the VHAR structure. We also adapt the robust estimation method to estimate VHAR coefficients more efficiently. Our simulation results show that our proposed method performs well in detecting outliers in multivariate long memory time series. Empirical analysis with stock index shows RVHAR model finds additional outliers that existing model does not detect.