• Journal of Information Processing Systems
• /
• v.1 no.1 s.1
• /
• pp.14-21
• /
• 2005

Even though mainly statistical methods have been used in anomaly network intrusion detection, to detect various attack types, machine learning based anomaly detection was introduced. Machine learning based anomaly detection started from research applying traditional learning algorithms of artificial intelligence to intrusion detection. However, detection rates of these methods are not satisfactory. Especially, high false positive and repeated alarms about the same attack are problems. The main reason for this is that one packet is used as a basic learning unit. Most attacks consist of more than one packet. In addition, an attack does not lead to a consecutive packet stream. Therefore, with grouping of related packets, a new approach of group-based learning and detection is needed. This type of approach is similar to that of multiple-instance problems in the artificial intelligence community, which cannot clearly classify one instance, but classification of a group is possible. We suggest group generation algorithm grouping related packets, and a learning algorithm based on a unit of such group. To verify the usefulness of the suggested algorithm, 1998 DARPA data was used and the results show that our approach is quite useful.

• Journal of Korean Society of Industrial and Systems Engineering
• /
• v.43 no.2
• /
• pp.33-38
• /
• 2020

Anomaly detection of Machine Learning such as PCA anomaly detection and CNN image classification has been focused on cross-sectional data. In this paper, two approaches has been suggested to apply ML techniques for identifying the failure time of big time series data. PCA anomaly detection to identify time rows as normal or abnormal was suggested by converting subjects identification problem to time domain. CNN image classification was suggested to identify the failure time by re-structuring of time series data, which computed the correlation matrix of one minute data and converted to tiff image format. Also, LASSO, one of feature selection methods, was applied to select the most affecting variables which could identify the failure status. For the empirical study, time series data was collected in seconds from a power generator of 214 components for 25 minutes including 20 minutes before the failure time. The failure time was predicted and detected 9 minutes 17 seconds before the failure time by PCA anomaly detection, but was not detected by the combination of LASSO and PCA because the target variable was binary variable which was assigned on the base of the failure time. CNN image classification with the train data of 10 normal status image and 5 failure status images detected just one minute before.

• Journal of Korea Society of Industrial Information Systems
• /
• v.14 no.3
• /
• pp.22-29
• /
• 2009

In previous work, anomaly-based intrusion detection techniques have been widely used to effectively detect various intrusions into a computer. This is because the anomaly-based detection techniques can effectively handle previously unknown intrusion methods. However, most of the previous work assumed that the normal network connections are fixed. For this reason, a new network connection may be regarded as an anomalous event. This paper proposes a new anomaly detection method based on an association-mining algorithm. The proposed method is composed of two phases: intra-packet association mining and inter-packet association mining. The performances of the proposed method are comparatively verified with JAM, which is a conventional representative intrusion detection method.

• Journal of KIBIM
• /
• v.13 no.3
• /
• pp.12-20
• /
• 2023

Deep learning-based anomaly detection technology is used in various fields such as computer vision, speech recognition, and natural language processing. In particular, this technology is applied in various fields such as monitoring manufacturing equipment abnormalities, detecting financial fraud, detecting network hacking, and detecting anomalies in medical images. However, in the field of construction and architecture, research on deep learning-based data anomaly detection technology is difficult due to the lack of digitization of domain knowledge due to late digital conversion, lack of learning data, and difficulties in collecting and processing field data in real time. This study acquires necessary data through IoT (Internet of Things) from the viewpoint of monitoring for environmental management of architectural spaces, converts them into a database, learns deep learning, and then supports anomaly patterns using AI (Artificial Infelligence) deep learning-based anomaly detection. We propose an implementation process. The results of this study suggest an effective environmental anomaly pattern detection solution architecture for environmental management of architectural spaces, proving its feasibility. The proposed method enables quick response through real-time data processing and analysis collected from IoT. In order to confirm the effectiveness of the proposed method, performance analysis is performed through prototype implementation to derive the results.

• Journal of Digital Contents Society
• /
• v.19 no.2
• /
• pp.377-383
• /
• 2018

Internet of Things (IoT) is producing various data as the smart environment comes. The IoT data collection is used as important data to judge systems's status. Therefore, it is important to monitor the anomaly state of the sensor in real-time and to detect anomaly data. However, it is necessary to convert the IoT data into a normalized data structure for anomaly detection because of the variety of data structures and protocols. Thus, we can expect a good quality effect such as accurate analysis data quality and service quality. In this paper, we propose an anomaly detection system based on big data from collected sensor data. The proposed system is applied to ensure anomaly detection and keep data quality. In addition, we applied the machine learning model of support vector machine using anomaly detection based on time-series data. As a result, machine learning using preprocessed data was able to accurately detect and predict anomaly.

• Smart Structures and Systems
• /
• v.29 no.6
• /
• pp.757-766
• /
• 2022

To train deep learning algorithms, a sufficient number of data are required. However, in most engineering systems, the acquisition of fault data is difficult or sometimes not feasible, while normal data are secured. The dearth of data is one of the major challenges to developing deep learning models, and fault diagnosis in particular cannot be made in the absence of fault data. With this context, this paper proposes an anomaly detection methodology for rotating machines using only normal data with self-labeling. Since only normal data are used for anomaly detection, a self-labeling method is used to generate a new labeled dataset. The overall procedure includes the following three steps: (1) transformation of normal data to self-labeled data based on a pretext task, (2) training the convolutional neural networks (CNN), and (3) anomaly detection using defined anomaly score based on the softmax output of the trained CNN. The softmax value of the abnormal sample shows different behavior from the normal softmax values. To verify the proposed method, four case studies were conducted, on the Case Western Reserve University (CWRU) bearing dataset, IEEE PHM 2012 data challenge dataset, PHMAP 2021 data challenge dataset, and laboratory bearing testbed; and the results were compared to those of existing machine learning and deep learning methods. The results showed that the proposed algorithm could detect faults in the bearing testbed and compressor with over 99.7% accuracy. In particular, it was possible to detect not only bearing faults but also structural faults such as unbalance and belt looseness with very high accuracy. Compared with the existing GAN, the autoencoder-based anomaly detection algorithm, the proposed method showed high anomaly detection performance.

• Nuclear Engineering and Technology
• /
• v.55 no.2
• /
• pp.603-622
• /
• 2023

Human error (HE) is an important concern in safety-critical systems such as nuclear power plants (NPPs). HE has played a role in many accidents and outage incidents in NPPs. Despite the increased automation in NPPs, HE remains unavoidable. Hence, the need for HE detection is as important as HE prevention efforts. In NPPs, HE is rather rare. Hence, anomaly detection, a widely used machine learning technique for detecting rare anomalous instances, can be repurposed to detect potential HE. In this study, we develop an unsupervised anomaly detection technique based on generative adversarial networks (GANs) to detect anomalies in manually collected surveillance data in NPPs. More specifically, our GAN is trained to detect mismatches between automatically recorded sensor data and manually collected surveillance data, and hence, identify anomalous instances that can be attributed to HE. We test our GAN on both a real-world dataset and an external dataset obtained from a testbed, and we benchmark our results against state-of-the-art unsupervised anomaly detection algorithms, including one-class support vector machine and isolation forest. Our results show that the proposed GAN provides improved anomaly detection performance. Our study is promising for the future development of artificial intelligence based HE detection systems.

• Journal of KIISE
• /
• v.45 no.3
• /
• pp.294-302
• /
• 2018

With the emergence of the new service industry due to the development of information and communication technology, cyber space risks such as personal information infringement and industrial confidentiality leakage have diversified, and the security problem has emerged as a critical issue. In this paper, we propose a behavior-based anomaly detection method that is suitable for real-time and large-volume data analysis technology. We show that the proposed detection method is superior to existing signature security countermeasures that are based on large-capacity user log data according to in-company personal information abuse and internal information leakage. As the proposed behavior-based anomaly detection method requires a technique for processing large amounts of data, a real-time search engine is used, called Elasticsearch, which is based on an inverted index. In addition, statistical based frequency analysis and preprocessing were performed for data analysis, and the DBSCAN algorithm, which is a density based clustering method, was applied to classify abnormal data with an example for easy analysis through visualization. Unlike the existing anomaly detection system, the proposed behavior-based anomaly detection technique is promising as it enables anomaly detection analysis without the need to set the threshold value separately, and was proposed from a statistical perspective.

• KSII Transactions on Internet and Information Systems (TIIS)
• /
• v.14 no.7
• /
• pp.3093-3115
• /
• 2020

Network anomaly detection system plays an essential role in detecting network anomaly and ensuring network security. Anomaly detection system based machine learning has become an increasingly popular solution. However, due to the unbalance and high-dimension characteristics of network traffic, the existing methods unable to achieve the excellent performance of high accuracy and low false alarm rate. To address this problem, a new network anomaly detection method based on data balancing and recursive feature addition is proposed. Firstly, data balancing algorithm based on improved KNN outlier detection is designed to select part respective data on each category. Combination optimization about parameters of improved KNN outlier detection is implemented by genetic algorithm. Next, recursive feature addition algorithm based on correlation analysis is proposed to select effective features, in which a cross contingency test is utilized to analyze correlation and obtain a features subset with a strong correlation. Then, random forests model is as the classification model to detection anomaly. Finally, the proposed algorithm is evaluated on benchmark datasets KDD Cup 1999 and UNSW_NB15. The result illustrates the proposed strategies enhance accuracy and recall, and decrease the false alarm rate. Compared with other algorithms, this algorithm still achieves significant effects, especially recall in the small category.

• ETRI Journal
• /
• v.34 no.2
• /
• pp.288-291
• /
• 2012

In this letter, a new online anomaly detection approach for software systems is proposed. The novelty of the proposed approach is to apply a new semantic kernel function for a support vector machine (SVM) classifier to detect fault-suspicious execution paths at runtime in a reasonable amount of time. The kernel uses a new sequence matching algorithm to measure similarities among program execution paths in a customized feature space whose dimensions represent the largest common subpaths among the execution paths. To increase the precision of the SVM classifier, each common subpath is given weights according to its ability to discern executions as correct or anomalous. Experiment results show that compared with the known kernels, the proposed SVM kernel will improve the time overhead of online anomaly detection by up to 170%, while improving the precision of anomaly alerts by up to 140%.