• Title/Summary/Keyword: performance anomaly

Search Result 277, Processing Time 0.03 seconds

A Novel Network Anomaly Detection Method based on Data Balancing and Recursive Feature Addition

  • Liu, Xinqian;Ren, Jiadong;He, Haitao;Wang, Qian;Sun, Shengting
    • KSII Transactions on Internet and Information Systems (TIIS)
    • /
    • v.14 no.7
    • /
    • pp.3093-3115
    • /
    • 2020
  • Network anomaly detection system plays an essential role in detecting network anomaly and ensuring network security. Anomaly detection system based machine learning has become an increasingly popular solution. However, due to the unbalance and high-dimension characteristics of network traffic, the existing methods unable to achieve the excellent performance of high accuracy and low false alarm rate. To address this problem, a new network anomaly detection method based on data balancing and recursive feature addition is proposed. Firstly, data balancing algorithm based on improved KNN outlier detection is designed to select part respective data on each category. Combination optimization about parameters of improved KNN outlier detection is implemented by genetic algorithm. Next, recursive feature addition algorithm based on correlation analysis is proposed to select effective features, in which a cross contingency test is utilized to analyze correlation and obtain a features subset with a strong correlation. Then, random forests model is as the classification model to detection anomaly. Finally, the proposed algorithm is evaluated on benchmark datasets KDD Cup 1999 and UNSW_NB15. The result illustrates the proposed strategies enhance accuracy and recall, and decrease the false alarm rate. Compared with other algorithms, this algorithm still achieves significant effects, especially recall in the small category.

Anomaly Detection using Geometric Transformation of Normal Sample Images (정상 샘플 이미지의 기하학적 변환을 사용한 이상 징후 검출)

  • Kwon, Yong-Wan;Kang, Dong-Joong
    • The Journal of the Institute of Internet, Broadcasting and Communication
    • /
    • v.22 no.4
    • /
    • pp.157-163
    • /
    • 2022
  • Recently, with the development of automation in the industrial field, research on anomaly detection is being actively conducted. An application for anomaly detection used in factory automation is camera-based defect inspection. Vision camera inspection shows high performance and efficiency in factory automation, but it is difficult to overcome the instability of lighting and environmental conditions. Although camera inspection using deep learning can solve the problem of vision camera inspection with much higher performance, it is difficult to apply to actual industrial fields because it requires a huge amount of normal and abnormal data for learning. Therefore, in this study, we propose a network that overcomes the problem of collecting abnormal data with 72 geometric transformation deep learning methods using only normal data and adds an outlier exposure method for performance improvement. By applying and verifying this to the MVTec data set, which is a database for auto-mobile parts data and outlier detection, it is shown that it can be applied in actual industrial sites.

Resolving data imbalance through differentiated anomaly data processing based on verification data (검증데이터 기반의 차별화된 이상데이터 처리를 통한 데이터 불균형 해소 방법)

  • Hwang, Chulhyun
    • Journal of Intelligence and Information Systems
    • /
    • v.28 no.4
    • /
    • pp.179-190
    • /
    • 2022
  • Data imbalance refers to a phenomenon in which the number of data in one category is too large or too small compared to another category. Due to this, it has been raised as a major factor that deteriorates performance in machine learning that utilizes classification algorithms. In order to solve the data imbalance problem, various ovrsampling methods for amplifying prime number distribution data have been proposed. Among them, SMOTE is the most representative method. In order to maximize the amplification effect of minority distribution data, various methods have emerged that remove noise included in data (SMOTE-IPF) or enhance only border lines (Borderline SMOTE). This paper proposes a method to ultimately improve classification performance by improving the processing method for anomaly data in the traditional SMOTE method that amplifies minority classification data. The proposed method consistently presented relatively high classification performance compared to the existing methods through experiments.

Analysis of detected anomalies in VOC reduction facilities using deep learning

  • Min-Ji Son;Myung Ho Kim
    • Journal of the Korea Society of Computer and Information
    • /
    • v.28 no.4
    • /
    • pp.13-20
    • /
    • 2023
  • In this paper, the actual data of VOC reduction facilities was analyzed through a model that detects and predicts data anomalies. Using the USAD model, which shows stable performance in the field of anomaly detection, anomalies in real-time data are detected and sensors that cause anomalies are searched. In addition, we propose a method of predicting and warning, when abnormalities that time will occur by predicting future outliers with an auto-regressive model. The experiment was conducted with the actual data of the VOC reduction facility, and the anomaly detection test results showed high detection rates with precision, recall, and F1-score of 98.54%, 89.08%, and 93.57%, respectively. As a result, averaging of the precision, recall, and F1-score for 8 sensors of detection rates were 99.64%, 99.37%, and 99.63%. In addition, the Hamming loss obtained to confirm the validity of the detection experiment for each sensor was 0.0058, showing stable performance. And the abnormal prediction test result showed stable performance with an average absolute error of 0.0902.

Comparison of System Call Sequence Embedding Approaches for Anomaly Detection (이상 탐지를 위한 시스템콜 시퀀스 임베딩 접근 방식 비교)

  • Lee, Keun-Seop;Park, Kyungseon;Kim, Kangseok
    • Journal of Convergence for Information Technology
    • /
    • v.12 no.2
    • /
    • pp.47-53
    • /
    • 2022
  • Recently, with the change of the intelligent security paradigm, study to apply various information generated from various information security systems to AI-based anomaly detection is increasing. Therefore, in this study, in order to convert log-like time series data into a vector, which is a numerical feature, the CBOW and Skip-gram inference methods of deep learning-based Word2Vec model and statistical method based on the coincidence frequency were used to transform the published ADFA system call data. In relation to this, an experiment was carried out through conversion into various embedding vectors considering the dimension of vector, the length of sequence, and the window size. In addition, the performance of the embedding methods used as well as the detection performance were compared and evaluated through GRU-based anomaly detection model using vectors generated by the embedding model as an input. Compared to the statistical model, it was confirmed that the Skip-gram maintains more stable performance without biasing a specific window size or sequence length, and is more effective in making each event of sequence data into an embedding vector.

Channel Heterogeneity Aware Channel Assignment for IEEE 802.11 Multi-Radio Multi-Rate Wireless Networks (IEEE 802.11 다중 라디오 다중 전송률 무선 네트워크를 위한 채널 이질성 인지 채널 할당)

  • Kim, Sok-Hyong;Kim, Dong-Wook;Suh, Young-Joo
    • The Journal of Korean Institute of Communications and Information Sciences
    • /
    • v.36 no.11A
    • /
    • pp.870-877
    • /
    • 2011
  • IEEE 802.11 devices are widely used, and terminals can be equipped with multiple IEEE 802.11 interfaces as low-cost IEEE 802.11 devices are deployed. The off-the-shelf IEEE 802.11 devices provide multiple channels and multiple data rates. In practical multi-channel networks, since there is channel heterogeneity which indicates that channels have different signal characteristics for the same node, channels should be efficiently assigned to improve network capacity. In addition, in multi-rate networks, low-rate links severely degrade the performance of high-rate links on the same channel, which is known as performance anomaly. Therefore, in this paper, we propose a heterogeneity aware channel assignment (HACA) algorithm that improves network performance by reflecting channel heterogeneity and performance anomaly. Through NS-2 simulations, we validate that the HACA algorithm shows improved performance compared with existing channel assignment algorithms that do not reflect channel heterogeneity.

Technical Trends of GNSS Clock Anomaly Detection and Resolution (항법위성시계 노후에 따른 이상 현상 감지 및 극복 기술현황)

  • Heo, Youn-Jeong;Cho, Jeong-Ho;Heo, Moon-Beom;Sim, Eun-Sup
    • Current Industrial and Technological Trends in Aerospace
    • /
    • v.8 no.1
    • /
    • pp.77-85
    • /
    • 2010
  • The current GPS constellation consists of 32 Block IIA/IIR/IIR-M satellites including 12 Block IIA satellites on service over 15 years. The satellites in poor space conditions may suffer from anomalies, especially influenced by aging atomic clocks which are of importance positioning and timing. Recently, the IGS Ultra-rapid predicted products have not shown acceptably high quality prediction performance because the Block IIA cesium clocks may be easily affected by various factors such as temperature and environment. The anomalies of aging clocks involve lower performance of positioning in the GPS applications. We, thus, describe satellite clock behaviors and anomalies induced by aging clocks and their detection technologies to avoid such anomalies.

  • PDF

The Design and Implementation of Anomaly Traffic Analysis System using Data Mining

  • Lee, Se-Yul;Cho, Sang-Yeop;Kim, Yong-Soo
    • International Journal of Fuzzy Logic and Intelligent Systems
    • /
    • v.8 no.4
    • /
    • pp.316-321
    • /
    • 2008
  • Advanced computer network technology enables computers to be connected in an open network environment. Despite the growing numbers of security threats to networks, most intrusion detection identifies security attacks mainly by detecting misuse using a set of rules based on past hacking patterns. This pattern matching has a high rate of false positives and can not detect new hacking patterns, which makes it vulnerable to previously unidentified attack patterns and variations in attack and increases false negatives. Intrusion detection and analysis technologies are thus required. This paper investigates the asymmetric costs of false errors to enhance the performances the detection systems. The proposed method utilizes the network model to consider the cost ratio of false errors. By comparing false positive errors with false negative errors, this scheme achieved better performance on the view point of both security and system performance objectives. The results of our empirical experiment show that the network model provides high accuracy in detection. In addition, the simulation results show that effectiveness of anomaly traffic detection is enhanced by considering the costs of false errors.

Anomaly Detection System of IoT Platform using Machine Learning (기계학습을 활용한 IoT 플랫폼의 이상감지 시스템)

  • Im, SeonYeol;Choi, HyoKeun;Yi, KyuYull;Lee, TeaHun;Yu, HeonChang
    • Proceedings of the Korea Information Processing Society Conference
    • /
    • 2018.10a
    • /
    • pp.1001-1004
    • /
    • 2018
  • As the industry generates a lot of data, it is increasingly dependent on the IoT platform. For this reason, the performance and anomaly detection of IoT platform is becoming an important factor. In this paper, we propose a system model of IoT platform that detects device anomaly without performance issue. The proposed system uses Micro Batch which calculates the data transmission cycle to provide Soft Real-time service. In the industry, it was difficult to collect abnormal data, so the Hotelling's $T^2$ model was applied to the data analysis experiment. And the Hotelling's $T^2$ model successfully detected anomalies.

Anomaly Detection and Performance Analysis using Deep Learning (딥러닝을 활용한 설비 이상 탐지 및 성능 분석)

  • Hwang, Ju-hyo;Jin, Kyo-hong
    • Proceedings of the Korean Institute of Information and Commucation Sciences Conference
    • /
    • 2021.10a
    • /
    • pp.78-81
    • /
    • 2021
  • Through the smart factory construction project, sensors can be installed in manufacturing production facilities and various process data can be collected in real time. Through this, research on real-time facility anomaly detection is being actively conducted to reduce production interruption due to facility abnormality in the manufacturing process. In this paper, to detect abnormalities in production facilities, the manufacturing data was applied to deep learning models Autoencoder(AE), VAE(Variational Autoencoder), and AAE(Adversarial Autoencoder) to derive the results. Manufacturing data was used as input data through a simple moving average technique and preprocessing process, and performance analysis was conducted according to the window size of the simple movement average technique and the feature vector size of the AE model.

  • PDF