• Journal of Korean Society of Industrial and Systems Engineering
• /
• v.46 no.2
• /
• pp.1-12
• /
• 2023

To make semiconductor chips, a number of complex semiconductor manufacturing processes are required. Semiconductor chips that have undergone complex processes are subjected to EDS(Electrical Die Sorting) tests to check product quality, and a wafer bin map reflecting the information about the normal and defective chips is created. Defective chips found in the wafer bin map form various patterns, which are called defective patterns, and the defective patterns are a very important clue in determining the cause of defects in the process and design of semiconductors. Therefore, it is desired to automatically and quickly detect defective patterns in the field, and various methods have been proposed to detect defective patterns. Existing methods have considered simple, complex, and new defect patterns, but they had the disadvantage of being unable to provide field engineers the evidence of classification results through deep learning. It is necessary to supplement this and provide detailed information on the size, location, and patterns of the defects. In this paper, we propose an anomaly detection framework that can be explained through FCDD(Fully Convolutional Data Description) trained only with normal data to provide field engineers with details such as detection results of abnormal defect patterns, defect size, and location of defect patterns on wafer bin map. The results are analyzed using open dataset, providing prominent results of the proposed anomaly detection framework.

• Journal of Intelligence and Information Systems
• /
• v.27 no.3
• /
• pp.57-73
• /
• 2021

Maintenance and prevention of failure through anomaly detection of ICT infrastructure is becoming important. System monitoring data is multidimensional time series data. When we deal with multidimensional time series data, we have difficulty in considering both characteristics of multidimensional data and characteristics of time series data. When dealing with multidimensional data, correlation between variables should be considered. Existing methods such as probability and linear base, distance base, etc. are degraded due to limitations called the curse of dimensions. In addition, time series data is preprocessed by applying sliding window technique and time series decomposition for self-correlation analysis. These techniques are the cause of increasing the dimension of data, so it is necessary to supplement them. The anomaly detection field is an old research field, and statistical methods and regression analysis were used in the early days. Currently, there are active studies to apply machine learning and artificial neural network technology to this field. Statistically based methods are difficult to apply when data is non-homogeneous, and do not detect local outliers well. The regression analysis method compares the predictive value and the actual value after learning the regression formula based on the parametric statistics and it detects abnormality. Anomaly detection using regression analysis has the disadvantage that the performance is lowered when the model is not solid and the noise or outliers of the data are included. There is a restriction that learning data with noise or outliers should be used. The autoencoder using artificial neural networks is learned to output as similar as possible to input data. It has many advantages compared to existing probability and linear model, cluster analysis, and map learning. It can be applied to data that does not satisfy probability distribution or linear assumption. In addition, it is possible to learn non-mapping without label data for teaching. However, there is a limitation of local outlier identification of multidimensional data in anomaly detection, and there is a problem that the dimension of data is greatly increased due to the characteristics of time series data. In this study, we propose a CMAE (Conditional Multimodal Autoencoder) that enhances the performance of anomaly detection by considering local outliers and time series characteristics. First, we applied Multimodal Autoencoder (MAE) to improve the limitations of local outlier identification of multidimensional data. Multimodals are commonly used to learn different types of inputs, such as voice and image. The different modal shares the bottleneck effect of Autoencoder and it learns correlation. In addition, CAE (Conditional Autoencoder) was used to learn the characteristics of time series data effectively without increasing the dimension of data. In general, conditional input mainly uses category variables, but in this study, time was used as a condition to learn periodicity. The CMAE model proposed in this paper was verified by comparing with the Unimodal Autoencoder (UAE) and Multi-modal Autoencoder (MAE). The restoration performance of Autoencoder for 41 variables was confirmed in the proposed model and the comparison model. The restoration performance is different by variables, and the restoration is normally well operated because the loss value is small for Memory, Disk, and Network modals in all three Autoencoder models. The process modal did not show a significant difference in all three models, and the CPU modal showed excellent performance in CMAE. ROC curve was prepared for the evaluation of anomaly detection performance in the proposed model and the comparison model, and AUC, accuracy, precision, recall, and F1-score were compared. In all indicators, the performance was shown in the order of CMAE, MAE, and AE. Especially, the reproduction rate was 0.9828 for CMAE, which can be confirmed to detect almost most of the abnormalities. The accuracy of the model was also improved and 87.12%, and the F1-score was 0.8883, which is considered to be suitable for anomaly detection. In practical aspect, the proposed model has an additional advantage in addition to performance improvement. The use of techniques such as time series decomposition and sliding windows has the disadvantage of managing unnecessary procedures; and their dimensional increase can cause a decrease in the computational speed in inference.The proposed model has characteristics that are easy to apply to practical tasks such as inference speed and model management.

• KSII Transactions on Internet and Information Systems (TIIS)
• /
• v.10 no.8
• /
• pp.3884-3910
• /
• 2016

Intrusion Detection System (IDS) in general considers a big amount of data that are highly redundant and irrelevant. This trait causes slow instruction, assessment procedures, high resource consumption and poor detection rate. Due to their expensive computational requirements during both training and detection, IDSs are mostly ineffective for real-time anomaly detection. This paper proposes a dimensionality reduction technique that is able to enhance the performance of IDSs up to constant time O(1) based on the Principle Component Analysis (PCA). Furthermore, the present study offers a feature selection approach for identifying major components in real time. The PCA algorithm transforms high-dimensional feature vectors into a low-dimensional feature space, which is used to determine the optimum volume of factors. The proposed approach was assessed using HTTP packet payload of ISCX 2012 IDS and DARPA 1999 dataset. The experimental outcome demonstrated that our proposed anomaly detection achieved promising results with 97% detection rate with 1.2% false positive rate for ISCX 2012 dataset and 100% detection rate with 0.06% false positive rate for DARPA 1999 dataset. Our proposed anomaly detection also achieved comparable performance in terms of computational complexity when compared to three state-of-the-art anomaly detection systems.

• Journal of Information Processing Systems
• /
• v.1 no.1 s.1
• /
• pp.14-21
• /
• 2005

Even though mainly statistical methods have been used in anomaly network intrusion detection, to detect various attack types, machine learning based anomaly detection was introduced. Machine learning based anomaly detection started from research applying traditional learning algorithms of artificial intelligence to intrusion detection. However, detection rates of these methods are not satisfactory. Especially, high false positive and repeated alarms about the same attack are problems. The main reason for this is that one packet is used as a basic learning unit. Most attacks consist of more than one packet. In addition, an attack does not lead to a consecutive packet stream. Therefore, with grouping of related packets, a new approach of group-based learning and detection is needed. This type of approach is similar to that of multiple-instance problems in the artificial intelligence community, which cannot clearly classify one instance, but classification of a group is possible. We suggest group generation algorithm grouping related packets, and a learning algorithm based on a unit of such group. To verify the usefulness of the suggested algorithm, 1998 DARPA data was used and the results show that our approach is quite useful.

• The Journal of Korean Institute of Communications and Information Sciences
• /
• v.34 no.3B
• /
• pp.311-317
• /
• 2009

The traditional network anomaly detection systems execute the threshold-based detection without considering dynamic network environments, which causes false positive and limits an effective resource utilization. To overcome the drawbacks, we present the adaptive network anomaly detection model based on artificial immune system (AIS) in centralized network. AIS is inspired from human immune system that has learning, adaptation and memory. In our proposed model, the interaction between dendritic cell and T-cell of human immune system is adopted. We design the main components, such as central node and router node, and define functions of them. The central node analyzes the anomaly information received from the related router nodes, decides response policy and sends the policy to corresponding nodes. The router node consists of detector module and responder module. The detector module perceives the anomaly depending on learning data and the responder module settles the anomaly according to the policy received from central node. Finally we evaluate the possibility of the proposed detection model through simulation.

• Journal of Internet Computing and Services
• /
• v.6 no.5
• /
• pp.45-56
• /
• 2005

The weak foundation of the computing environment caused information leakage and hacking to be uncontrollable, Therefore, dynamic control of security threats and real-time reaction to identical or similar types of accidents after intrusion are considered to be important, h one of the solutions to solve the problem, studies on intrusion detection systems are actively being conducted. To improve the anomaly IDS using system calls, this study focuses on neural networks learning using the soundex algorithm which is designed to change feature selection and variable length data into a fixed length learning pattern, That Is, by changing variable length sequential system call data into a fixed iength behavior pattern using the soundex algorithm, this study conducted neural networks learning by using a backpropagation algorithm. The backpropagation neural networks technique is applied for anomaly detection of system calls using Sendmail Data of UNM to demonstrate its performance.

• Economic and Environmental Geology
• /
• v.32 no.2
• /
• pp.177-187
• /
• 1999

Gravity, magnetic and VLF surveys were conducted to investigat the structural stability and hazards associated with the Ubong landfill in Pohang City, which has been built to dump industrial wastes. In 1994, the collapse of a bank happened in the 6th landfill site due to sudden heavy rain, and a large quantity of waste materials flowed out to the nearby landfill sites, factories and roads. We used $10{\times}10m$ resolution DEM data for gravity reductions. The maximum variation of the terrain effect in the survey area is about 0.5 mgal and the terrain effect is large in the vicinity of bank boundary. The Bouguer gravity anomaly map shows the effect due to the variatino of thickness and type of waste materials. The small negative gravity anomaly increases from the 9th site to the 6th site. The small negative gravity anomaly of the 9th site reflects the relatively shallow dumping depth of average 14.5 m in this site and increased density of waste materials by the repeated stabilization process of soil overlaying. The 6th site is located at the center of the former valley and rainfall and groundwater are expected to flow from south-east to north-west. Therefore, considering the previous accident of mixing waste and bank materials at the north-west boundary of the landfill, there may be some environmental problems of leakage of contaminated water and bank stability. The complex inversion technique using Simulated annealing and Marquardt-Levenberg methods was applied to calculate three-dimensional density distribution from gravity data. In the case of 6th site, it is apparent that the landfill had been dumped in four sectors. However, most part of the 9th site and showed that high magnetic industrial wastes were concentrated in the 6th site. The result of magnetic survey showing low magnetic anomalies along the boundaries of two sites is similar to that of gravity data. The VLF data also reveals four divided sectors in the 6th site, and overall anomaly trend indicates the directio of former valley.

• Journal of the Korea Society of Computer and Information
• /
• v.28 no.4
• /
• pp.13-20
• /
• 2023

In this paper, the actual data of VOC reduction facilities was analyzed through a model that detects and predicts data anomalies. Using the USAD model, which shows stable performance in the field of anomaly detection, anomalies in real-time data are detected and sensors that cause anomalies are searched. In addition, we propose a method of predicting and warning, when abnormalities that time will occur by predicting future outliers with an auto-regressive model. The experiment was conducted with the actual data of the VOC reduction facility, and the anomaly detection test results showed high detection rates with precision, recall, and F1-score of 98.54%, 89.08%, and 93.57%, respectively. As a result, averaging of the precision, recall, and F1-score for 8 sensors of detection rates were 99.64%, 99.37%, and 99.63%. In addition, the Hamming loss obtained to confirm the validity of the detection experiment for each sensor was 0.0058, showing stable performance. And the abnormal prediction test result showed stable performance with an average absolute error of 0.0902.

• Journal of Internet Computing and Services
• /
• v.23 no.3
• /
• pp.35-45
• /
• 2022

Even in recent years, treatment of first-aid patients is still often delayed due to a shortage of medical resources in marginalized areas. Research on automating the analysis of medical data to solve the problems of inaccessibility for medical services and shortage of medical personnel is ongoing. Computer vision-based medical inspection automation requires a lot of cost in data collection and labeling for training purposes. These problems stand out in the works of classifying lesion that are rare, or pathological features and pathogenesis that are difficult to clearly define visually. Anomaly detection is attracting as a method that can significantly reduce the cost of data collection by adopting an unsupervised learning strategy. In this paper, we propose methods for detecting abnormal images on chest X-RAY images as follows based on existing anomaly detection techniques. (1) Normalize the brightness range of medical images resampled as optimal resolution. (2) Some feature vectors with high representative power are selected in set of patch features extracted as intermediate-level from lesion-free images. (3) Measure the difference from the feature vectors of lesion-free data selected based on the nearest neighbor search algorithm. The proposed system can simultaneously perform anomaly classification and localization for each image. In this paper, the anomaly detection performance of the proposed system for chest X-RAY images of PA projection is measured and presented by detailed conditions. We demonstrate effect of anomaly detection for medical images by showing 0.705 classification AUROC for random subset extracted from the PadChest dataset. The proposed system can be usefully used to improve the clinical diagnosis workflow of medical institutions, and can effectively support early diagnosis in medically poor area.

• The Journal of the Institute of Internet, Broadcasting and Communication
• /
• v.23 no.2
• /
• pp.103-111
• /
• 2023

This paper proposes a new data correction technique that transforms anomalies in time series data into normal values. With the recent development of IT technology, a vast amount of time-series data is being collected through sensors. However, due to sensor failures and abnormal environments, most of time-series data contain a lot of anomalies. If we build a predictive model using original data containing anomalies as it is, we cannot expect highly reliable predictive performance. Therefore, we utilizes the LSTM-GAN model to detect anomalies in the original time series data, and combines DTW (Dynamic Time Warping) and GAN techniques to replace the anomaly data with normal data in partitioned window units. The basic idea is to construct a GAN model serially by applying the statistical information of the window with normal distribution data adjacent to the window containing the detected anomalies to the DTW so as to generate normal time-series data. Through experiments using open NAB data, we empirically prove that our proposed method outperforms the conventional two correction methods.