• Title/Summary/Keyword: anomaly detection

Search Result 640, Processing Time 0.029 seconds

gMLP-based Self-Supervised Learning Anomaly Detection using a Simple Synthetic Data Generation Method (단순한 합성데이터 생성 방식을 활용한 gMLP 기반 자기 지도 학습 이상탐지 기법)

  • Ju-Hyo, Hwang;Kyo-Hong, Jin
    • Journal of the Korea Institute of Information and Communication Engineering
    • /
    • v.27 no.1
    • /
    • pp.8-14
    • /
    • 2023
  • The existing self-supervised learning-based CutPaste generated synthetic data by cutting and attaching specific patches from normal images and then performed anomaly detection. However, this method has a problem in that there is a clear difference in the boundary of the patch. NSA for solving these problems have achieved higher anomaly detection performance by generating natural synthetic data through Poisson Blending. However, NSA has the disadvantage of having many hyperparameters that need to be adjusted for each class. In this paper, synthetic data similar to normal were generated by a simple method of making the size of the synthetic patch very small. At this time, since the patches are so locally synthesized, models that learn local features can easily overfit synthetic data. Therefore, we performed anomaly detection using gMLP, which learns global features, and even with simple synthesis methods, we were able to achieve higher performance than conventional self-supervised learning techniques.

Development of Security Anomaly Detection Algorithms using Machine Learning (기계 학습을 활용한 보안 이상징후 식별 알고리즘 개발)

  • Hwangbo, Hyunwoo;Kim, Jae Kyung
    • The Journal of Society for e-Business Studies
    • /
    • v.27 no.1
    • /
    • pp.1-13
    • /
    • 2022
  • With the development of network technologies, the security to protect organizational resources from internal and external intrusions and threats becomes more important. Therefore in recent years, the anomaly detection algorithm that detects and prevents security threats with respect to various security log events has been actively studied. Security anomaly detection algorithms that have been developed based on rule-based or statistical learning in the past are gradually evolving into modeling based on machine learning and deep learning. In this study, we propose a deep-autoencoder model that transforms LSTM-autoencoder as an optimal algorithm to detect insider threats in advance using various machine learning analysis methodologies. This study has academic significance in that it improved the possibility of adaptive security through the development of an anomaly detection algorithm based on unsupervised learning, and reduced the false positive rate compared to the existing algorithm through supervised true positive labeling.

MLOps workflow language and platform for time series data anomaly detection

  • Sohn, Jung-Mo;Kim, Su-Min
    • Journal of the Korea Society of Computer and Information
    • /
    • v.27 no.11
    • /
    • pp.19-27
    • /
    • 2022
  • In this study, we propose a language and platform to describe and manage the MLOps(Machine Learning Operations) workflow for time series data anomaly detection. Time series data is collected in many fields, such as IoT sensors, system performance indicators, and user access. In addition, it is used in many applications such as system monitoring and anomaly detection. In order to perform prediction and anomaly detection of time series data, the MLOps platform that can quickly and flexibly apply the analyzed model to the production environment is required. Thus, we developed Python-based AI/ML Modeling Language (AMML) to easily configure and execute MLOps workflows. Python is widely used in data analysis. The proposed MLOps platform can extract and preprocess time series data from various data sources (R-DB, NoSql DB, Log File, etc.) using AMML and predict it through a deep learning model. To verify the applicability of AMML, the workflow for generating a transformer oil temperature prediction deep learning model was configured with AMML and it was confirmed that the training was performed normally.

A Method for Region-Specific Anomaly Detection on Patch-wise Segmented PA Chest Radiograph (PA 흉부 X-선 영상 패치 분할에 의한 지역 특수성 이상 탐지 방법)

  • Hyun-bin Kim;Jun-Chul Chun
    • Journal of Internet Computing and Services
    • /
    • v.24 no.1
    • /
    • pp.49-59
    • /
    • 2023
  • Recently, attention to the pandemic situation represented by COVID-19 emerged problems caused by unexpected shortage of medical personnel. In this paper, we present a method for diagnosing the presence or absence of lesional sign on PA chest X-ray images as computer vision solution to support diagnosis tasks. Method for visual anomaly detection based on feature modeling can be also applied to X-ray images. With extracting feature vectors from PA chest X-ray images and divide to patch unit, region-specific abnormality can be detected. As preliminary experiment, we created simulation data set containing multiple objects and present results of the comparative experiments in this paper. We present method to improve both efficiency and performance of the process through hard masking of patch features to aligned images. By summing up regional specificity and global anomaly detection results, it shows improved performance by 0.069 AUROC compared to previous studies. By aggregating region-specific and global anomaly detection results, it shows improved performance by 0.069 AUROC compared to our last study.

Online anomaly detection algorithm based on deep support vector data description using incremental centroid update (점진적 중심 갱신을 이용한 deep support vector data description 기반의 온라인 비정상 탐지 알고리즘)

  • Lee, Kibae;Ko, Guhn Hyeok;Lee, Chong Hyun
    • The Journal of the Acoustical Society of Korea
    • /
    • v.41 no.2
    • /
    • pp.199-209
    • /
    • 2022
  • Typical anomaly detection algorithms are trained by using prior data. Thus the batch learning based algorithms cause inevitable performance degradation when characteristics of newly incoming normal data change over time. We propose an online anomaly detection algorithm which can consider the gradual characteristic changes of incoming normal data. The proposed algorithm based on one-class classification model includes both offline and online learning procedures. In offline learning procedure, the algorithm learns the prior data to be close to centroid of the latent space and then updates the centroid of the latent space incrementally by new incoming data. In the online learning, the algorithm continues learning by using the updated centroid. Through experiments using public underwater acoustic data, the proposed online anomaly detection algorithm takes only approximately 2 % additional learning time for the incremental centroid update and learning. Nevertheless, the proposed algorithm shows 19.10 % improvement in Area Under the receiver operating characteristic Curve (AUC) performance compared to the offline learning model when new incoming normal data comes.

Cable anomaly detection driven by spatiotemporal correlation dissimilarity measurements of bridge grouped cable forces

  • Dong-Hui, Yang;Hai-Lun, Gu;Ting-Hua, Yi;Zhan-Jun, Wu
    • Smart Structures and Systems
    • /
    • v.30 no.6
    • /
    • pp.661-671
    • /
    • 2022
  • Stayed cables are the key components for transmitting loads in cable-stayed bridges. Therefore, it is very important to evaluate the cable force condition to ensure bridge safety. An online condition assessment and anomaly localization method is proposed for cables based on the spatiotemporal correlation of grouped cable forces. First, an anomaly sensitive feature index is obtained based on the distribution characteristics of grouped cable forces. Second, an adaptive anomaly detection method based on the k-nearest neighbor rule is used to perform dissimilarity measurements on the extracted feature index, and such a method can effectively remove the interference of environment factors and vehicle loads on online condition assessment of the grouped cable forces. Furthermore, an online anomaly isolation and localization method for stay cables is established, and the complete decomposition contributions method is used to decompose the feature matrix of the grouped cable forces and build an anomaly isolation index. Finally, case studies were carried out to validate the proposed method using an in-service cable-stayed bridge equipped with a structural health monitoring system. The results show that the proposed approach is sensitive to the abnormal distribution of grouped cable forces and is robust to the influence of interference factors. In addition, the proposed approach can also localize the cables with abnormal cable forces online, which can be successfully applied to the field monitoring of cables for cable-stayed bridges.

A Predictive Bearing Anomaly Detection Model Using the SWT-SVD Preprocessing Algorithm (SWT-SVD 전처리 알고리즘을 적용한 예측적 베어링 이상탐지 모델)

  • So-hyang Bak;Kwanghoon Pio Kim
    • Journal of Internet Computing and Services
    • /
    • v.25 no.1
    • /
    • pp.109-121
    • /
    • 2024
  • In various manufacturing processes such as textiles and automobiles, when equipment breaks down or stops, the machines do not work, which leads to time and financial losses for the company. Therefore, it is important to detect equipment abnormalities in advance so that equipment failures can be predicted and repaired before they occur. Most equipment failures are caused by bearing failures, which are essential parts of equipment, and detection bearing anomaly is the essence of PHM(Prognostics and Health Management) research. In this paper, we propose a preprocessing algorithm called SWT-SVD, which analyzes vibration signals from bearings and apply it to an anomaly transformer, one of the time series anomaly detection model networks, to implement bearing anomaly detection model. Vibration signals from the bearing manufacturing process contain noise due to the real-time generation of sensor values. To reduce noise in vibration signals, we use the Stationary Wavelet Transform to extract frequency components and perform preprocessing to extract meaningful features through the Singular Value Decomposition algorithm. For experimental validation of the proposed SWT-SVD preprocessing method in the bearing anomaly detection model, we utilize the PHM-2012-Challenge dataset provided by the IEEE PHM Conference. The experimental results demonstrate significant performance with an accuracy of 0.98 and an F1-Score of 0.97. Additionally, to substantiate performance improvement, we conduct a comparative analysis with previous studies, confirming that the proposed preprocessing method outperforms previous preprocessing methods in terms of performance.

Hierarchical Flow-Based Anomaly Detection Model for Motor Gearbox Defect Detection

  • Younghwa Lee;Il-Sik Chang;Suseong Oh;Youngjin Nam;Youngteuk Chae;Geonyoung Choi;Gooman Park
    • KSII Transactions on Internet and Information Systems (TIIS)
    • /
    • v.17 no.6
    • /
    • pp.1516-1529
    • /
    • 2023
  • In this paper, a motor gearbox fault-detection system based on a hierarchical flow-based model is proposed. The proposed system is used for the anomaly detection of a motion sound-based actuator module. The proposed flow-based model, which is a generative model, learns by directly modeling a data distribution function. As the objective function is the maximum likelihood value of the input data, the training is stable and simple to use for anomaly detection. The operation sound of a car's side-view mirror motor is converted into a Mel-spectrogram image, consisting of a folding signal and an unfolding signal, and used as training data in this experiment. The proposed system is composed of an encoder and a decoder. The data extracted from the layer of the pretrained feature extractor are used as the decoder input data in the encoder. This information is used in the decoder by performing an interlayer cross-scale convolution operation. The experimental results indicate that the context information of various dimensions extracted from the interlayer hierarchical data improves the defect detection accuracy. This paper is notable because it uses acoustic data and a normalizing flow model to detect outliers based on the features of experimental data.

The Design and Implementation of Anomaly Traffic Analysis System using Data Mining

  • Lee, Se-Yul;Cho, Sang-Yeop;Kim, Yong-Soo
    • International Journal of Fuzzy Logic and Intelligent Systems
    • /
    • v.8 no.4
    • /
    • pp.316-321
    • /
    • 2008
  • Advanced computer network technology enables computers to be connected in an open network environment. Despite the growing numbers of security threats to networks, most intrusion detection identifies security attacks mainly by detecting misuse using a set of rules based on past hacking patterns. This pattern matching has a high rate of false positives and can not detect new hacking patterns, which makes it vulnerable to previously unidentified attack patterns and variations in attack and increases false negatives. Intrusion detection and analysis technologies are thus required. This paper investigates the asymmetric costs of false errors to enhance the performances the detection systems. The proposed method utilizes the network model to consider the cost ratio of false errors. By comparing false positive errors with false negative errors, this scheme achieved better performance on the view point of both security and system performance objectives. The results of our empirical experiment show that the network model provides high accuracy in detection. In addition, the simulation results show that effectiveness of anomaly traffic detection is enhanced by considering the costs of false errors.

Probabilistic Soft Error Detection Based on Anomaly Speculation

  • Yoo, Joon-Hyuk
    • Journal of Information Processing Systems
    • /
    • v.7 no.3
    • /
    • pp.435-446
    • /
    • 2011
  • Microprocessors are becoming increasingly vulnerable to soft errors due to the current trends of semiconductor technology scaling. Traditional redundant multi-threading architectures provide perfect fault tolerance by re-executing all the computations. However, such a full re-execution technique significantly increases the verification workload on the processor resources, resulting in severe performance degradation. This paper presents a pro-active verification management approach to mitigate the verification workload to increase its performance with a minimal effect on overall reliability. An anomaly-speculation-based filter checker is proposed to guide a verification priority before the re-execution process starts. This technique is accomplished by exploiting a value similarity property, which is defined by a frequent occurrence of partially identical values. Based on the biased distribution of similarity distance measure, this paper investigates further application to exploit similar values for soft error tolerance with anomaly speculation. Extensive measurements prove that the majority of instructions produce values, which are different from the previous result value, only in a few bits. Experimental results show that the proposed scheme accelerates the processor to be 180% faster than traditional fully-fault-tolerant processor with a minimal impact on overall soft error rate.