• KSII Transactions on Internet and Information Systems (TIIS)
• /
• 제14권7호
• /
• pp.3093-3115
• /
• 2020

Network anomaly detection system plays an essential role in detecting network anomaly and ensuring network security. Anomaly detection system based machine learning has become an increasingly popular solution. However, due to the unbalance and high-dimension characteristics of network traffic, the existing methods unable to achieve the excellent performance of high accuracy and low false alarm rate. To address this problem, a new network anomaly detection method based on data balancing and recursive feature addition is proposed. Firstly, data balancing algorithm based on improved KNN outlier detection is designed to select part respective data on each category. Combination optimization about parameters of improved KNN outlier detection is implemented by genetic algorithm. Next, recursive feature addition algorithm based on correlation analysis is proposed to select effective features, in which a cross contingency test is utilized to analyze correlation and obtain a features subset with a strong correlation. Then, random forests model is as the classification model to detection anomaly. Finally, the proposed algorithm is evaluated on benchmark datasets KDD Cup 1999 and UNSW_NB15. The result illustrates the proposed strategies enhance accuracy and recall, and decrease the false alarm rate. Compared with other algorithms, this algorithm still achieves significant effects, especially recall in the small category.

• ETRI Journal
• /
• 제46권3호
• /
• pp.513-525
• /
• 2024

Deep neural networks trained on labeled medical data face major challenges owing to the economic costs of data acquisition through expensive medical imaging devices, expert labor for data annotation, and large datasets to achieve optimal model performance. The heterogeneity of diseases, such as Alzheimer's disease, further complicates deep learning because the test cases may substantially differ from the training data, possibly increasing the rate of false positives. We propose a reconstruction-based self-supervised anomaly detection model to overcome these challenges. It has a dual-subnetwork encoder that enhances feature encoding augmented by skip connections to the decoder for improving the gradient flow. The novel encoder captures local and global features to improve image reconstruction. In addition, we introduce an entropy-based image conversion method. Extensive evaluations show that the proposed model outperforms benchmark models in anomaly detection and classification using an encoder. The supervised and unsupervised models show improved performances when trained with data preprocessed using the proposed image conversion method.

• KSII Transactions on Internet and Information Systems (TIIS)
• /
• 제12권8호
• /
• pp.3946-3965
• /
• 2018

Network anomaly detection in Software Defined Networking, especially the detection of DDoS attack, has been given great attention in recent years. It is convenient to build the Traffic Matrix from a global view in SDN. However, the monitoring and management of high-volume feature-rich traffic in large networks brings significant challenges. In this paper, we propose a moving window Principal Components Analysis based anomaly detection and mitigation approach to map data onto a low-dimensional subspace and keep monitoring the network state in real-time. Once the anomaly is detected, the controller will install the defense flow table rules onto the corresponding data plane switches to mitigate the attack. Furthermore, we evaluate our approach with experiments. The Receiver Operating Characteristic curves show that our approach performs well in both detection probability and false alarm probability compared with the entropy-based approach. In addition, the mitigation effect is impressive that our approach can prevent most of the attacking traffic. At last, we evaluate the overhead of the system, including the detection delay and utilization of CPU, which is not excessive. Our anomaly detection approach is lightweight and effective.

• Nuclear Engineering and Technology
• /
• 제55권2호
• /
• pp.603-622
• /
• 2023

Human error (HE) is an important concern in safety-critical systems such as nuclear power plants (NPPs). HE has played a role in many accidents and outage incidents in NPPs. Despite the increased automation in NPPs, HE remains unavoidable. Hence, the need for HE detection is as important as HE prevention efforts. In NPPs, HE is rather rare. Hence, anomaly detection, a widely used machine learning technique for detecting rare anomalous instances, can be repurposed to detect potential HE. In this study, we develop an unsupervised anomaly detection technique based on generative adversarial networks (GANs) to detect anomalies in manually collected surveillance data in NPPs. More specifically, our GAN is trained to detect mismatches between automatically recorded sensor data and manually collected surveillance data, and hence, identify anomalous instances that can be attributed to HE. We test our GAN on both a real-world dataset and an external dataset obtained from a testbed, and we benchmark our results against state-of-the-art unsupervised anomaly detection algorithms, including one-class support vector machine and isolation forest. Our results show that the proposed GAN provides improved anomaly detection performance. Our study is promising for the future development of artificial intelligence based HE detection systems.

• 산업경영시스템학회지
• /
• 제45권4호
• /
• pp.86-98
• /
• 2022

Recently, many studies have been conducted to improve quality by applying machine learning models to semiconductor manufacturing process data. However, in the semiconductor manufacturing process, the ratio of good products is much higher than that of defective products, so the problem of data imbalance is serious in terms of machine learning. In addition, since the number of features of data used in machine learning is very large, it is very important to perform machine learning by extracting only important features from among them to increase accuracy and utilization. This study proposes an anomaly detection methodology that can learn excellently despite data imbalance and high-dimensional characteristics of semiconductor process data. The anomaly detection methodology applies the LIME algorithm after applying the SMOTE method and the RFECV method. The proposed methodology analyzes the classification result of the anomaly classification model, detects the cause of the anomaly, and derives a semiconductor process requiring action. The proposed methodology confirmed applicability and feasibility through application of cases.

• 한국정보통신학회:학술대회논문집
• /
• 한국정보통신학회 2022년도 춘계학술대회
• /
• pp.104-107
• /
• 2022

오토인코더 딥러닝 모델은 이상 데이터도 정상 데이터로 복원하는 능력이 우수하여 이상탐지에 적절하지 못한 경우가 발생한다. 그리고 데이터의 일부를 가린(마스킹) 후 가린 데이터를 복원하는 방식인 Inpainting 방식은 잡음이 많은 이미지에 대해서는 복원능력이 떨어지는 문제점을 가지고 있다. 본 논문에서는 MLP-Mixer 모델을 수정·개선하여 이미지를 일정 비율로 마스킹하고 마스킹된 이미지의 압축된 정보를 모델에 전달해 이미지를 재구성하는 방식을 사용하였다. MVTec AD 데이터 셋의 정상 데이터로 학습한 모델을 구축한 뒤, 정상과 이상 이미지를 각각 입력하여 재구성 오류를 구하고 이를 통해 이상탐지를 수행하였다. 성능 평가 결과 제안된 방식이 기존의 방식에 비해 이상탐지 성능이 우수한 것으로 나타났다.

• KSII Transactions on Internet and Information Systems (TIIS)
• /
• 제10권8호
• /
• pp.3865-3883
• /
• 2016

Infrastructure as a Service (IaaS) encapsulates computer hardware into a large amount of virtual and manageable instances mainly in the form of virtual machine (VM), and provides rental service for users. Currently, VM anomaly incidents occasionally occur, which leads to performance issues and even downtime. This paper aims at detecting anomalous VMs based on performance metrics data of VMs. Due to the dynamic nature and increasing scale of IaaS, detecting anomalous VMs from voluminous correlated and non-Gaussian monitored performance data is a challenging task. This paper designs an anomaly detection framework to solve this challenge. First, it collects 53 performance metrics to reflect the running state of each VM. The collected performance metrics are testified not to follow the Gaussian distribution. Then, it employs independent components analysis (ICA) instead of principal component analysis (PCA) to extract independent components from collected non-Gaussian performance metric data. For anomaly detection, it employs multi-class Bayesian classification to determine the current state of each VM. To evaluate the performance of the designed detection framework, four types of anomalies are separately or jointly injected into randomly selected VMs in a campus-wide testbed. The experimental results show that ICA-based detection mechanism outperforms PCA-based and LDA-based detection mechanisms in terms of sensitivity and specificity.

• 정보보호학회논문지
• /
• 제32권4호
• /
• pp.647-660
• /
• 2022

최근 정상 데이터와 일부 비정상 데이터를 보유한 환경에서 딥러닝 기반 준 지도 학습 이상 탐지 기법이 매우 효과적으로 동작함이 알려져 있다. 하지만 사이버 보안 분야와 같이 실제 시스템에 대한 알려지지 않은 공격 등 비정상 데이터 확보가 어려운 환경에서는 비정상 데이터 부족이 발생할 가능성이 있다. 본 논문은 비정상 데이터가 정상 데이터보다 극히 작은 환경에서 준 지도 이상 탐지 기법에 적용 가능한 섭동을 활용한 초구 기반 비정상 데이터 증강 기법인 ADA-PH(Abnormal Data Augmentation Method using Perturbation based on Hypersphere)를 제안한다. ADA-PH는 정상 데이터를 잘 표현할 수 있는 초구의 중심으로부터 상대적으로 먼 거리에 위치한 샘플에 대해 적대적 섭동을 추가함으로써 비정상 데이터를 생성한다. 제안하는 기법은 비정상 데이터가 극소수로 존재하는 네트워크 침입 탐지 데이터셋에 대하여 데이터 증강을 수행하지 않았을 경우보다 평균적으로 23.63% 향상된 AUC가 도출되었고, 다른 증강 기법들과 비교했을 때 가장 높은 AUC가 또한 도출되었다. 또한, 실제 비정상 데이터에 유사한지에 대한 정량적 및 정성적 분석을 수행하였다.

• Smart Structures and Systems
• /
• 제29권1호
• /
• pp.63-75
• /
• 2022

The raw data collected by structural health monitoring (SHM) systems may suffer multiple patterns of anomalies, which pose a significant barrier for an automatic and accurate structural condition assessment. Therefore, the detection and classification of these anomalies is an essential pre-processing step for SHM systems. However, the heterogeneous data patterns, scarce anomalous samples and severe class imbalance make data anomaly detection difficult. In this regard, this study proposes a convolutional neural network-based data anomaly detection method. The time and frequency domains data are transferred as images and used as the input of the neural network for training. ResNet18 is adopted as the feature extractor to avoid training with massive labelled data. In addition, the focal loss function is adopted to soften the class imbalance-induced classification bias. The effectiveness of the proposed method is validated using acceleration data collected in a long-span cable-stayed bridge. The proposed approach detects and classifies data anomalies with high accuracy.

• 한국전자거래학회지
• /
• 제20권4호
• /
• pp.41-59
• /
• 2015

시스템이 다양화 되면서 동시에 저장된 로그도 다양하게 분석할 필요가 생겼다. 이러한 로그 데이터 분석에 관한 필요성이 강해지는 환경이 시간 순으로 발생하는 이벤트 단위의 로그로부터 프로세스 모델을 도출하고, 시스템을 개선시키는 활동에 이바지하도록 요구하고 있다. 기존에는 개별 이벤트 단위의 로그를 분석하면서 속성들의 관계를 파악하는 연구가 활발했다. 본 논문에서는 로그 데이터를 활용한 예외적인 형태의 프로세스 인스턴스를 판별하는 방법으로 LAPID(Local Anomaly Process Instance Detection)를 제안한다. LAPID는 액티비티-릴레이션 매트릭스(Activity relation matrix)를 사용해서 계산된 거리 값을 활용하여, API(Anomaly Process Instance)를 탐색한다. 제시한 방법의 유용성을 검증하기 위하여 항만 물류에서 발생하는 컨테이너 이동에 대한 트레이스(Trace)를 포함하는 로그 데이터에서 예외적인 상황의 프로세스 실행이 가지는 특징을 도출하였다. 이를 위하여 본 논문에서는 국내의 실제 항만에서 발생한 이벤트 로그를 이용하여 사례연구를 수행하였다.