• 제목/요약/키워드: data anomaly detection

검색결과 400건 처리시간 0.028초

FADA: A fuzzy anomaly detection algorithm for MANETs (모바일 애드-혹 망을 위한 퍼지 비정상 행위 탐지 알고리즘)

  • Bae, Ihn-Han
    • Journal of the Korean Data and Information Science Society
    • /
    • 제21권6호
    • /
    • pp.1125-1136
    • /
    • 2010
  • Lately there exist increasing demands for online abnormality monitoring over trajectory stream, which are obtained from moving object tracking devices. This problem is challenging due to the requirement of high speed data processing within limited space cost. In this paper, we present a FADA (Fuzzy Anomaly Detection Algorithm) which constructs normal profile by computing mobility feature information from the GPS (Global Positioning System) logs of mobile devices in MANETs (Mobile Ad-hoc Networks), computes a fuzzy dissimilarity between the current mobility feature information of the mobile device and the mobility feature information in the normal profile, and detects effectively the anomaly behaviors of mobile devices on the basis of the computed fuzzy dissimilarity. The performance of proposed FADA is evaluated through simulation.

Host Anomaly Detection of Neural Networks and Neural-fuzzy Techniques with Soundex Algorithm (사운덱스 알고리즘을 적용한 신경망라 뉴로-처지 기법의 호스트 이상 탐지)

  • Cha, Byung-Rae;Kim, Hyung-Jong;Park, Bong-Gu;Cho, Hyug-Hyun
    • Journal of the Korea Institute of Information Security & Cryptology
    • /
    • 제15권2호
    • /
    • pp.13-22
    • /
    • 2005
  • To improve the anomaly IDS using system calls, this study focuses on Neural Networks Learning using the Soundex algorithm which is designed to change feature selection and variable length data into a fixed length learning pattern. That is, by changing variable length sequential system call data into a fixed length behavior pattern using the Soundex algorithm, this study conducted neural networks learning by using a backpropagation algorithm with fuzzy membership function. The back-propagation neural networks and Neuro-Fuzzy technique are applied for anomaly intrusion detection of system calls using Sendmail Data of UNM to demonstrate its aspect of he complexity of time, space and MDL performance.

Detection of Defect Patterns on Wafer Bin Map Using Fully Convolutional Data Description (FCDD) (FCDD 기반 웨이퍼 빈 맵 상의 결함패턴 탐지)

  • Seung-Jun Jang;Suk Joo Bae
    • Journal of Korean Society of Industrial and Systems Engineering
    • /
    • 제46권2호
    • /
    • pp.1-12
    • /
    • 2023
  • To make semiconductor chips, a number of complex semiconductor manufacturing processes are required. Semiconductor chips that have undergone complex processes are subjected to EDS(Electrical Die Sorting) tests to check product quality, and a wafer bin map reflecting the information about the normal and defective chips is created. Defective chips found in the wafer bin map form various patterns, which are called defective patterns, and the defective patterns are a very important clue in determining the cause of defects in the process and design of semiconductors. Therefore, it is desired to automatically and quickly detect defective patterns in the field, and various methods have been proposed to detect defective patterns. Existing methods have considered simple, complex, and new defect patterns, but they had the disadvantage of being unable to provide field engineers the evidence of classification results through deep learning. It is necessary to supplement this and provide detailed information on the size, location, and patterns of the defects. In this paper, we propose an anomaly detection framework that can be explained through FCDD(Fully Convolutional Data Description) trained only with normal data to provide field engineers with details such as detection results of abnormal defect patterns, defect size, and location of defect patterns on wafer bin map. The results are analyzed using open dataset, providing prominent results of the proposed anomaly detection framework.

Anomaly Intrusion Detection using Fuzzy Membership Function and Neural Networks (퍼지 멤버쉽 함수와 신경망을 이용한 이상 침입 탐지)

  • Cha, Byung-Rae
    • The KIPS Transactions:PartC
    • /
    • 제11C권5호
    • /
    • pp.595-604
    • /
    • 2004
  • By the help of expansion of computer network and rapid growth of Internet, the information infrastructure is now able to provide a wide range of services. Especially open architecture - the inherent nature of Internet - has not only got in the way of offering QoS service, managing networks, but also made the users vulnerable to both the threat of backing and the issue of information leak. Thus, people recognized the importance of both taking active, prompt and real-time action against intrusion threat, and at the same time, analyzing the similar patterns of in-trusion already known. There are now many researches underway on Intrusion Detection System(IDS). The paper carries research on the in-trusion detection system which hired supervised learning algorithm and Fuzzy membership function especially with Neuro-Fuzzy model in order to improve its performance. It modifies tansigmoid transfer function of Neural Networks into fuzzy membership function, so that it can reduce the uncertainty of anomaly intrusion detection. Finally, the fuzzy logic suggested here has been applied to a network-based anomaly intrusion detection system, tested against intrusion data offered by DARPA 2000 Intrusion Data Sets, and proven that it overcomes the shortcomings that Anomaly Intrusion Detection usually has.

Data Fusion Algorithm based on Inference for Anomaly Detection in the Next-Generation Intrusion Detection (차세대 침입탐지에서 이상탐지를 위한 추론 기반 데이터 융합 알고리즘)

  • Kim, Dong-Wook;Han, Myung-Mook
    • Journal of the Korean Institute of Intelligent Systems
    • /
    • 제26권3호
    • /
    • pp.233-238
    • /
    • 2016
  • In this paper, we propose the algorithms of processing the uncertainty data using data fusion for the next generation intrusion detection. In the next generation intrusion detection, a lot of data are collected by many of network sensors to discover knowledge from generating information in cyber space. It is necessary the data fusion process to extract knowledge from collected sensors data. In this paper, we have proposed method to represent the uncertainty data, by classifying where is a confidence interval in interval of uncertainty data through feature analysis of different data using inference method with Dempster-Shafer Evidence Theory. In this paper, we have implemented a detection experiment that is classified by the confidence interval using IRIS plant Data Set for anomaly detection of uncertainty data. As a result, we found that it is possible to classify data by confidence interval.

Anomaly Detection System in Mechanical Facility Equipment: Using Long Short-Term Memory Variational Autoencoder (LSTM-VAE를 활용한 기계시설물 장치의 이상 탐지 시스템)

  • Seo, Jaehong;Park, Junsung;Yoo, Joonwoo;Park, Heejun
    • Journal of Korean Society for Quality Management
    • /
    • 제49권4호
    • /
    • pp.581-594
    • /
    • 2021
  • Purpose: The purpose of this study is to compare machine learning models for anomaly detection of mechanical facility equipment and suggest an anomaly detection system for mechanical facility equipment in subway stations. It helps to predict failures and plan the maintenance of facility. Ultimately it aims to improve the quality of facility equipment. Methods: The data collected from Daejeon Metropolitan Rapid Transit Corporation was used in this experiment. The experiment was performed using Python, Scikit-learn, tensorflow 2.0 for preprocessing and machine learning. Also it was conducted in two failure states of the equipment. We compared and analyzed five unsupervised machine learning models focused on model Long Short-Term Memory Variational Autoencoder(LSTM-VAE). Results: In both experiments, change in vibration and current data was observed when there is a defect. When the rotating body failure was happened, the magnitude of vibration has increased but current has decreased. In situation of axis alignment failure, both of vibration and current have increased. In addition, model LSTM-VAE showed superior accuracy than the other four base-line models. Conclusion: According to the results, model LSTM-VAE showed outstanding performance with more than 97% of accuracy in the experiments. Thus, the quality of mechanical facility equipment will be improved if the proposed anomaly detection system is established with this model used.

Detection algorithm of ionospheric delay anomaly based on multi-reference stations for ionospheric scintillation

  • Yoo, Yun-Ja;Cho, Deuk-Jae;Park, Sang-Hyun;Shin, Mi-Young
    • Journal of Navigation and Port Research
    • /
    • 제35권9호
    • /
    • pp.701-706
    • /
    • 2011
  • Radio waves including GPS signals, various TV communications, and radio broadcasting can be disturbed by a strong solar storm, which may occur due to solar flares and produce an ionospheric delay anomaly in the ionosphere according to the change of total electron content. Electron density irregularities can cause deep signal fading, frequently known as ionospheric scintillation, which can result in the positioning error using GPS signal. This paper proposes a detection algorithm for the ionosphere delay anomaly during a solar storm by using multi-reference stations. Different TEC grid which has irregular electron density was applied above one reference station. Then the ionospheric delay in zenith direction applied different TEC will show comparatively large ionospheric zenith delay due to the electron irregularity. The ionospheric slant delay applied an elevation angle at reference station was analyzed to detect the ionospheric delay anomaly that can result in positioning error. A simulation test was implemented and a proposed detection algorithm using data logged by four reference stations was applied to detect the ionospheric delay anomaly compared to a criterion.

A hybrid deep neural network compression approach enabling edge intelligence for data anomaly detection in smart structural health monitoring systems

  • Tarutal Ghosh Mondal;Jau-Yu Chou;Yuguang Fu;Jianxiao Mao
    • Smart Structures and Systems
    • /
    • 제32권3호
    • /
    • pp.179-193
    • /
    • 2023
  • This study explores an alternative to the existing centralized process for data anomaly detection in modern Internet of Things (IoT)-based structural health monitoring (SHM) systems. An edge intelligence framework is proposed for the early detection and classification of various data anomalies facilitating quality enhancement of acquired data before transmitting to a central system. State-of-the-art deep neural network pruning techniques are investigated and compared aiming to significantly reduce the network size so that it can run efficiently on resource-constrained edge devices such as wireless smart sensors. Further, depthwise separable convolution (DSC) is invoked, the integration of which with advanced structural pruning methods exhibited superior compression capability. Last but not least, quantization-aware training (QAT) is adopted for faster processing and lower memory and power consumption. The proposed edge intelligence framework will eventually lead to reduced network overload and latency. This will enable intelligent self-adaptation strategies to be employed to timely deal with a faulty sensor, minimizing the wasteful use of power, memory, and other resources in wireless smart sensors, increasing efficiency, and reducing maintenance costs for modern smart SHM systems. This study presents a theoretical foundation for the proposed framework, the validation of which through actual field trials is a scope for future work.

Anomaly detection of smart metering system for power management with battery storage system/electric vehicle

  • Sangkeum Lee;Sarvar Hussain Nengroo;Hojun Jin;Yoonmee Doh;Chungho Lee;Taewook Heo;Dongsoo Har
    • ETRI Journal
    • /
    • 제45권4호
    • /
    • pp.650-665
    • /
    • 2023
  • A novel smart metering technique capable of anomaly detection was proposed for real-time home power management system. Smart meter data generated in real-time were obtained from 900 households of single apartments. To detect outliers and missing values in smart meter data, a deep learning model, the autoencoder, consisting of a graph convolutional network and bidirectional long short-term memory network, was applied to the smart metering technique. Power management based on the smart metering technique was executed by multi-objective optimization in the presence of a battery storage system and an electric vehicle. The results of the power management employing the proposed smart metering technique indicate a reduction in electricity cost and amount of power supplied by the grid compared to the results of power management without anomaly detection.

An Adaptive Anomaly Detection Model Design based on Artificial Immune System in Central Network (중앙 집중형 망에서 인공면역체계 기반의 적응적 망 이상 상태 탐지 모델 설계)

  • Yoo, Kyoung-Min;Yang, Won-Hyuk;Lee, Sang-Yeol;Jeong, Hye-Ryun;So, Won-Ho;Kim, Young-Chon
    • The Journal of Korean Institute of Communications and Information Sciences
    • /
    • 제34권3B호
    • /
    • pp.311-317
    • /
    • 2009
  • The traditional network anomaly detection systems execute the threshold-based detection without considering dynamic network environments, which causes false positive and limits an effective resource utilization. To overcome the drawbacks, we present the adaptive network anomaly detection model based on artificial immune system (AIS) in centralized network. AIS is inspired from human immune system that has learning, adaptation and memory. In our proposed model, the interaction between dendritic cell and T-cell of human immune system is adopted. We design the main components, such as central node and router node, and define functions of them. The central node analyzes the anomaly information received from the related router nodes, decides response policy and sends the policy to corresponding nodes. The router node consists of detector module and responder module. The detector module perceives the anomaly depending on learning data and the responder module settles the anomaly according to the policy received from central node. Finally we evaluate the possibility of the proposed detection model through simulation.