• Title/Summary/Keyword: anomaly detection

Search Result 659, Processing Time 0.027 seconds

Detection of Defect Patterns on Wafer Bin Map Using Fully Convolutional Data Description (FCDD) (FCDD 기반 웨이퍼 빈 맵 상의 결함패턴 탐지)

  • Seung-Jun Jang;Suk Joo Bae
    • Journal of Korean Society of Industrial and Systems Engineering
    • /
    • v.46 no.2
    • /
    • pp.1-12
    • /
    • 2023
  • To make semiconductor chips, a number of complex semiconductor manufacturing processes are required. Semiconductor chips that have undergone complex processes are subjected to EDS(Electrical Die Sorting) tests to check product quality, and a wafer bin map reflecting the information about the normal and defective chips is created. Defective chips found in the wafer bin map form various patterns, which are called defective patterns, and the defective patterns are a very important clue in determining the cause of defects in the process and design of semiconductors. Therefore, it is desired to automatically and quickly detect defective patterns in the field, and various methods have been proposed to detect defective patterns. Existing methods have considered simple, complex, and new defect patterns, but they had the disadvantage of being unable to provide field engineers the evidence of classification results through deep learning. It is necessary to supplement this and provide detailed information on the size, location, and patterns of the defects. In this paper, we propose an anomaly detection framework that can be explained through FCDD(Fully Convolutional Data Description) trained only with normal data to provide field engineers with details such as detection results of abnormal defect patterns, defect size, and location of defect patterns on wafer bin map. The results are analyzed using open dataset, providing prominent results of the proposed anomaly detection framework.

An Architecture for Efficient Intrusion Detection System of Abnormal Traffic (비정상 트래픽 상황에서 효율적 침입 탐지 시스템(EIDS) 구조 연구)

  • Kwon, Young-Jae;Lee, Du-Man;Yim, Hong-Bin;Jung, Jae-Il
    • Proceedings of the IEEK Conference
    • /
    • 2006.06a
    • /
    • pp.207-208
    • /
    • 2006
  • Intrusion detection technology is highlighted in order to establish a safe information-oriented environment. Intrusion detection system can be categorized into anomaly detection and misuse detection according to intrusion detection pattern. In this paper, we propose an architecture to make up for the defect of conventional anomaly intrusion detection. This architecture reduces additional resource consumption and cost by placing the agent in the strategic location in Internet.

  • PDF

LSTM-based Anomaly Detection on Big Data for Smart Factory Monitoring (스마트 팩토리 모니터링을 위한 빅 데이터의 LSTM 기반 이상 탐지)

  • Nguyen, Van Quan;Van Ma, Linh;Kim, Jinsul
    • Journal of Digital Contents Society
    • /
    • v.19 no.4
    • /
    • pp.789-799
    • /
    • 2018
  • This article presents machine learning based approach on Big data to analyzing time series data for anomaly detection in such industrial complex system. Long Short-Term Memory (LSTM) network have been demonstrated to be improved version of RNN and have become a useful aid for many tasks. This LSTM based model learn the higher level temporal features as well as temporal pattern, then such predictor is used to prediction stage to estimate future data. The prediction error is the difference between predicted output made by predictor and actual in-coming values. An error-distribution estimation model is built using a Gaussian distribution to calculate the anomaly in the score of the observation. In this manner, we move from the concept of a single anomaly to the idea of the collective anomaly. This work can assist the monitoring and management of Smart Factory in minimizing failure and improving manufacturing quality.

Anomaly Detection In Real Power Plant Vibration Data by MSCRED Base Model Improved By Subset Sampling Validation (Subset 샘플링 검증 기법을 활용한 MSCRED 모델 기반 발전소 진동 데이터의 이상 진단)

  • Hong, Su-Woong;Kwon, Jang-Woo
    • Journal of Convergence for Information Technology
    • /
    • v.12 no.1
    • /
    • pp.31-38
    • /
    • 2022
  • This paper applies an expert independent unsupervised neural network learning-based multivariate time series data analysis model, MSCRED(Multi-Scale Convolutional Recurrent Encoder-Decoder), and to overcome the limitation, because the MCRED is based on Auto-encoder model, that train data must not to be contaminated, by using learning data sampling technique, called Subset Sampling Validation. By using the vibration data of power plant equipment that has been labeled, the classification performance of MSCRED is evaluated with the Anomaly Score in many cases, 1) the abnormal data is mixed with the training data 2) when the abnormal data is removed from the training data in case 1. Through this, this paper presents an expert-independent anomaly diagnosis framework that is strong against error data, and presents a concise and accurate solution in various fields of multivariate time series data.

SHM data anomaly classification using machine learning strategies: A comparative study

  • Chou, Jau-Yu;Fu, Yuguang;Huang, Shieh-Kung;Chang, Chia-Ming
    • Smart Structures and Systems
    • /
    • v.29 no.1
    • /
    • pp.77-91
    • /
    • 2022
  • Various monitoring systems have been implemented in civil infrastructure to ensure structural safety and integrity. In long-term monitoring, these systems generate a large amount of data, where anomalies are not unusual and can pose unique challenges for structural health monitoring applications, such as system identification and damage detection. Therefore, developing efficient techniques is quite essential to recognize the anomalies in monitoring data. In this study, several machine learning techniques are explored and implemented to detect and classify various types of data anomalies. A field dataset, which consists of one month long acceleration data obtained from a long-span cable-stayed bridge in China, is employed to examine the machine learning techniques for automated data anomaly detection. These techniques include the statistic-based pattern recognition network, spectrogram-based convolutional neural network, image-based time history convolutional neural network, image-based time-frequency hybrid convolution neural network (GoogLeNet), and proposed ensemble neural network model. The ensemble model deliberately combines different machine learning models to enhance anomaly classification performance. The results show that all these techniques can successfully detect and classify six types of data anomalies (i.e., missing, minor, outlier, square, trend, drift). Moreover, both image-based time history convolutional neural network and GoogLeNet are further investigated for the capability of autonomous online anomaly classification and found to effectively classify anomalies with decent performance. As seen in comparison with accuracy, the proposed ensemble neural network model outperforms the other three machine learning techniques. This study also evaluates the proposed ensemble neural network model to a blind test dataset. As found in the results, this ensemble model is effective for data anomaly detection and applicable for the signal characteristics changing over time.

Leision Detection in Chest X-ray Images based on Coreset of Patch Feature (패치 특징 코어세트 기반의 흉부 X-Ray 영상에서의 병변 유무 감지)

  • Kim, Hyun-bin;Chun, Jun-Chul
    • Journal of Internet Computing and Services
    • /
    • v.23 no.3
    • /
    • pp.35-45
    • /
    • 2022
  • Even in recent years, treatment of first-aid patients is still often delayed due to a shortage of medical resources in marginalized areas. Research on automating the analysis of medical data to solve the problems of inaccessibility for medical services and shortage of medical personnel is ongoing. Computer vision-based medical inspection automation requires a lot of cost in data collection and labeling for training purposes. These problems stand out in the works of classifying lesion that are rare, or pathological features and pathogenesis that are difficult to clearly define visually. Anomaly detection is attracting as a method that can significantly reduce the cost of data collection by adopting an unsupervised learning strategy. In this paper, we propose methods for detecting abnormal images on chest X-RAY images as follows based on existing anomaly detection techniques. (1) Normalize the brightness range of medical images resampled as optimal resolution. (2) Some feature vectors with high representative power are selected in set of patch features extracted as intermediate-level from lesion-free images. (3) Measure the difference from the feature vectors of lesion-free data selected based on the nearest neighbor search algorithm. The proposed system can simultaneously perform anomaly classification and localization for each image. In this paper, the anomaly detection performance of the proposed system for chest X-RAY images of PA projection is measured and presented by detailed conditions. We demonstrate effect of anomaly detection for medical images by showing 0.705 classification AUROC for random subset extracted from the PadChest dataset. The proposed system can be usefully used to improve the clinical diagnosis workflow of medical institutions, and can effectively support early diagnosis in medically poor area.

Anomaly Detection of Generative Adversarial Networks considering Quality and Distortion of Images (이미지의 질과 왜곡을 고려한 적대적 생성 신경망과 이를 이용한 비정상 검출)

  • Seo, Tae-Moon;Kang, Min-Guk;Kang, Dong-Joong
    • The Journal of the Institute of Internet, Broadcasting and Communication
    • /
    • v.20 no.3
    • /
    • pp.171-179
    • /
    • 2020
  • Recently, studies have shown that convolution neural networks are achieving the best performance in image classification, object detection, and image generation. Vision based defect inspection which is more economical than other defect inspection, is a very important for a factory automation. Although supervised anomaly detection algorithm has far exceeded the performance of traditional machine learning based method, it is inefficient for real industrial field due to its tedious annotation work, In this paper, we propose ADGAN, a unsupervised anomaly detection architecture using the variational autoencoder and the generative adversarial network which give great results in image generation task, and demonstrate whether the proposed network architecture identifies anomalous images well on MNIST benchmark dataset as well as our own welding defect dataset.

Anomaly Detection of Hadoop Log Data Using Moving Average and 3-Sigma (이동 평균과 3-시그마를 이용한 하둡 로그 데이터의 이상 탐지)

  • Son, Siwoon;Gil, Myeong-Seon;Moon, Yang-Sae;Won, Hee-Sun
    • KIPS Transactions on Software and Data Engineering
    • /
    • v.5 no.6
    • /
    • pp.283-288
    • /
    • 2016
  • In recent years, there have been many research efforts on Big Data, and many companies developed a variety of relevant products. Accordingly, we are able to store and analyze a large volume of log data, which have been difficult to be handled in the traditional computing environment. To handle a large volume of log data, which rapidly occur in multiple servers, in this paper we design a new data storage architecture to efficiently analyze those big log data through Apache Hive. We then design and implement anomaly detection methods, which identify abnormal status of servers from log data, based on moving average and 3-sigma techniques. We also show effectiveness of the proposed detection methods by demonstrating that our methods identifies anomalies correctly. These results show that our anomaly detection is an excellent approach for properly detecting anomalies from Hadoop log data.

Deep Learning-based Vehicle Anomaly Detection using Road CCTV Data (도로 CCTV 데이터를 활용한 딥러닝 기반 차량 이상 감지)

  • Shin, Dong-Hoon;Baek, Ji-Won;Park, Roy C.;Chung, Kyungyong
    • Journal of the Korea Convergence Society
    • /
    • v.12 no.2
    • /
    • pp.1-6
    • /
    • 2021
  • In the modern society, traffic problems are occurring as vehicle ownership increases. In particular, the incidence of highway traffic accidents is low, but the fatality rate is high. Therefore, a technology for detecting an abnormality in a vehicle is being studied. Among them, there is a vehicle anomaly detection technology using deep learning. This detects vehicle abnormalities such as a stopped vehicle due to an accident or engine failure. However, if an abnormality occurs on the road, it is possible to quickly respond to the driver's location. In this study, we propose a deep learning-based vehicle anomaly detection using road CCTV data. The proposed method preprocesses the road CCTV data. The pre-processing uses the background extraction algorithm MOG2 to separate the background and the foreground. The foreground refers to a vehicle with displacement, and a vehicle with an abnormality on the road is judged as a background because there is no displacement. The image that the background is extracted detects an object using YOLOv4. It is determined that the vehicle is abnormal.

Anomaly Detection using Geometric Transformation of Normal Sample Images (정상 샘플 이미지의 기하학적 변환을 사용한 이상 징후 검출)

  • Kwon, Yong-Wan;Kang, Dong-Joong
    • The Journal of the Institute of Internet, Broadcasting and Communication
    • /
    • v.22 no.4
    • /
    • pp.157-163
    • /
    • 2022
  • Recently, with the development of automation in the industrial field, research on anomaly detection is being actively conducted. An application for anomaly detection used in factory automation is camera-based defect inspection. Vision camera inspection shows high performance and efficiency in factory automation, but it is difficult to overcome the instability of lighting and environmental conditions. Although camera inspection using deep learning can solve the problem of vision camera inspection with much higher performance, it is difficult to apply to actual industrial fields because it requires a huge amount of normal and abnormal data for learning. Therefore, in this study, we propose a network that overcomes the problem of collecting abnormal data with 72 geometric transformation deep learning methods using only normal data and adds an outlier exposure method for performance improvement. By applying and verifying this to the MVTec data set, which is a database for auto-mobile parts data and outlier detection, it is shown that it can be applied in actual industrial sites.