• 제목/요약/키워드: LOF(Local Outlier Factor)

검색결과 7건 처리시간 0.018초

대용량 자료 분석을 위한 밀도기반 이상치 탐지 (Density-based Outlier Detection for Very Large Data)

  • 김승;조남욱;강석호
    • 한국경영과학회지
    • /
    • 제35권2호
    • /
    • pp.71-88
    • /
    • 2010
  • A density-based outlier detection such as an LOF (Local Outlier Factor) tries to find an outlying observation by using density of its surrounding space. In spite of several advantages of a density-based outlier detection method, the computational complexity of outlier detection has been one of major barriers in its application. In this paper, we present an LOF algorithm that can reduce computation time of a density based outlier detection algorithm. A kd-tree indexing and approximated k-nearest neighbor search algorithm (ANN) are adopted in the proposed method. A set of experiments was conducted to examine performance of the proposed algorithm. The results show that the proposed method can effectively detect local outliers in reduced computation time.

LOF를 이용한 ICA 기반 통계적 공정관리의 성능 개선 방법론 (The Use of Local Outlier Factor(LOF) for Improving Performance of Independent Component Analysis(ICA) based Statistical Process Control(SPC))

  • 이재신;강복영;강석호
    • 한국경영과학회지
    • /
    • 제36권1호
    • /
    • pp.39-55
    • /
    • 2011
  • Process monitoring has been emphasized for the monitoring of complex system such as chemical processing industries to achieve the efficiency enhancement, quality management, safety improvement. Recently, ICA (Independent Component Analysis) based MSPC (Multivariate Statistical Process Control) was widely used in process monitoring approaches. Moreover, DICA (Dynamic ICA) has been introduced to consider the system dynamics. However, the existing approaches show the limitation that their performances are strongly dependent on the statistical distributions of control variables. To improve the limitation, we propose a novel approach for process monitoring by integrating DICA and LOF (Local Outlier Factor). In this paper, we aim to improve the fault detection rate with the proposed method. LOF detects local outliers by using density of surrounding space so that its performance is regardless of data distribution. Therefore, the proposed method not only can consider the system dynamics but can also assure robust performance regardless of the statistical distributions of control variables. Comparison experiments were conducted on the widely used benchmark dataset, Tennessee Eastman process (TE process), and showed the improved performance than existing approaches.

Incremental Local Outlier Factor를 이용한 Support Vector Machine의 성능 개선에 관한 연구 (A study on improvement of Support Vector Machine with Incremental Local Outlier Factor)

  • 김민규;손수일;유석인
    • 한국정보과학회:학술대회논문집
    • /
    • 한국정보과학회 2011년도 한국컴퓨터종합학술대회논문집 Vol.38 No.1(C)
    • /
    • pp.354-357
    • /
    • 2011
  • Support Vector Machine (SVM)은 주어진 데이터 중에서 각 클래스를 잘 표현하는 Support Vector (SV)를 계산함으로써 새로운 데이터를 분류하는 알고리즘이다. SVM은 전체 데이터 분포를 고려하지 않기 때문에 잘못된 데이터에 의해 분류가 잘못될 가능성이 적다. 하지만, SV가 잘못되었을 경우에는 정확도가 감소하게 되는 문제점이 있다. 본 논문에서는 SV가 잘못 주어진 데이터일 가능성을 고려, 아웃라이어 검출 알고리즘인 Local Outlier Factor (LOF) 알고리즘을 이용해 주어진 데이터 중 잘못된 데이터를 제거함으로써 SVM의 분류 정확도를 높이는 알고리즘을 제안하였다. 추가적으로, Incremental LOF를 이용해 새로운 데이터 중 판단하기 어려운 데이터를 제거함으로써 SVM의 정확도를 보다 향상시켰다. 제안된 방법은 두 개의 클래스를 가진 데이터에 대해서 실험하였다.

Plagiarism Detection among Source Codes using Adaptive Methods

  • Lee, Yun-Jung;Lim, Jin-Su;Ji, Jeong-Hoon;Cho, Hwaun-Gue;Woo, Gyun
    • KSII Transactions on Internet and Information Systems (TIIS)
    • /
    • 제6권6호
    • /
    • pp.1627-1648
    • /
    • 2012
  • We propose an adaptive method for detecting plagiarized pairs from a large set of source code. This method is adaptive in that it uses an adaptive algorithm and it provides an adaptive threshold for determining plagiarism. Conventional algorithms are based on greedy string tiling or on local alignments of two code strings. However, most of them are not adaptive; they do not consider the characteristics of the program set, thereby causing a problem for a program set in which all the programs are inherently similar. We propose adaptive local alignment-a variant of local alignment that uses an adaptive similarity matrix. Each entry of this matrix is the logarithm of the probabilities of the keywords based on their frequency in a given program set. We also propose an adaptive threshold based on the local outlier factor (LOF), which represents the likelihood of an entity being an outlier. Experimental results indicate that our method is more sensitive than JPlag, which uses greedy string tiling for detecting plagiarism-suspected code pairs. Further, the adaptive threshold based on the LOF is shown to be effective, and the detection performance shows high sensitivity with negligible loss of specificity, compared with that using a fixed threshold.

실시간 비즈니스 프로세스 모니터링 방법론을 위한 확장 KNN 대체 기반 LOF 예측 알고리즘 (Extended KNN Imputation Based LOF Prediction Algorithm for Real-time Business Process Monitoring Method)

  • 강복영;김동수;강석호
    • 한국전자거래학회지
    • /
    • 제15권4호
    • /
    • pp.303-317
    • /
    • 2010
  • 본 논문에서는 KNN 대체와 LOF 알고리즘의 결합 모델을 확장하여 실시간 비즈니스 프로세스 모니터링을 위한 비정상 종료 예측 방법론을 제안하였다. 기존의 룰 기반 모니터링 방법론은 실시간 프로세스 진행 정도에 따른 비관측 정보에 기인하여 조기 경보 및 실시간 대응이 힘들다는 한계점을 안고 있다. 이를 해결하기 위하여 비관측 정보에 대한 가정 및 진행 중인 프로세스의 향후 경로 예측을 통해 종료 시점에서 예상되는 LOF를 추정하기 위한 알고리즘을 제안하였다. 이 알고리즘을 적용하여 실시간 비즈니스 프로세스 모니터링 과정에서 각 관측 시점마다 종료 시점에서의 결과를 예측함으로써, 전 시점에 걸친 추세를 살펴종료 패턴을 예측할 수 있다. 이를 통해 비즈니스 프로세스의 실시간 진척에 대한 정보를 가시화함으로써 기회 및 위협에 사전에 대응할 수 있게 하여 프로세스 관리 수준의 향상을 기대할 수 있을 것으로 예상된다.

Preemptive Failure Detection using Contamination-Based Stacking Ensemble in Missiles

  • Seong-Mok Kim;Ye-Eun Jeong;Yong Soo Kim;Youn-Ho Lee;Seung Young Lee
    • KSII Transactions on Internet and Information Systems (TIIS)
    • /
    • 제18권5호
    • /
    • pp.1301-1316
    • /
    • 2024
  • In modern warfare, missiles play a pivotal role but typically spend the majority of their lifecycle in long-term storage or standby mode, making it difficult to detect failures. Preemptive detection of missiles that will fail is crucial to preventing severe consequences, including safety hazards and mission failures. This study proposes a contamination-based stacking ensemble model, employing the local outlier factor (LOF), to detect such missiles. The proposed model creates multiple base LOF models with different contamination values and combines their anomaly scores to achieve a robust anomaly detection. A comparative performance analysis was conducted between the proposed model and the traditional single LOF model, using production-related inspection data from missiles deployed in the military. The experimental results showed that, with the contamination parameter set to 0.1, the proposed model exhibited an increase of approximately 22 percentage points in accuracy and 71 percentage points in F1-score compared to the single LOF model. This approach enables the preemptive identification of potential failures, undetectable through traditional statistical quality control methods. Consequently, it contributes to lower missile failure rates in real battlefield scenarios, leading to significant time and cost savings in the military industry.

An Effective Anomaly Detection Approach based on Hybrid Unsupervised Learning Technologies in NIDS

  • Kangseok Kim
    • KSII Transactions on Internet and Information Systems (TIIS)
    • /
    • 제18권2호
    • /
    • pp.494-510
    • /
    • 2024
  • Internet users are exposed to sophisticated cyberattacks that intrusion detection systems have difficulty detecting. Therefore, research is increasing on intrusion detection methods that use artificial intelligence technology for detecting novel cyberattacks. Unsupervised learning-based methods are being researched that learn only from normal data and detect abnormal behaviors by finding patterns. This study developed an anomaly-detection method based on unsupervised machines and deep learning for a network intrusion detection system (NIDS). We present a hybrid anomaly detection approach based on unsupervised learning techniques using the autoencoder (AE), Isolation Forest (IF), and Local Outlier Factor (LOF) algorithms. An oversampling approach that increased the detection rate was also examined. A hybrid approach that combined deep learning algorithms and traditional machine learning algorithms was highly effective in setting the thresholds for anomalies without subjective human judgment. It achieved precision and recall rates respectively of 88.2% and 92.8% when combining two AEs, IF, and LOF while using an oversampling approach to learn more unknown normal data improved the detection accuracy. This approach achieved precision and recall rates respectively of 88.2% and 94.6%, further improving the detection accuracy compared with the hybrid method. Therefore, in NIDS the proposed approach provides high reliability for detecting cyberattacks.