• Title/Summary/Keyword: Anomaly detection system

Search Result 286, Processing Time 0.025 seconds

Performance Comparison of Anomaly Detection Algorithms: in terms of Anomaly Type and Data Properties (이상탐지 알고리즘 성능 비교: 이상치 유형과 데이터 속성 관점에서)

  • Jaeung Kim;Seung Ryul Jeong;Namgyu Kim
    • Journal of Intelligence and Information Systems
    • /
    • v.29 no.3
    • /
    • pp.229-247
    • /
    • 2023
  • With the increasing emphasis on anomaly detection across various fields, diverse anomaly detection algorithms have been developed for various data types and anomaly patterns. However, the performance of anomaly detection algorithms is generally evaluated on publicly available datasets, and the specific performance of each algorithm on anomalies of particular types remains unexplored. Consequently, selecting an appropriate anomaly detection algorithm for specific analytical contexts poses challenges. Therefore, in this paper, we aim to investigate the types of anomalies and various attributes of data. Subsequently, we intend to propose approaches that can assist in the selection of appropriate anomaly detection algorithms based on this understanding. Specifically, this study compares the performance of anomaly detection algorithms for four types of anomalies: local, global, contextual, and clustered anomalies. Through further analysis, the impact of label availability, data quantity, and dimensionality on algorithm performance is examined. Experimental results demonstrate that the most effective algorithm varies depending on the type of anomaly, and certain algorithms exhibit stable performance even in the absence of anomaly-specific information. Furthermore, in some types of anomalies, the performance of unsupervised anomaly detection algorithms was observed to be lower than that of supervised and semi-supervised learning algorithms. Lastly, we found that the performance of most algorithms is more strongly influenced by the type of anomalies when the data quantity is relatively scarce or abundant. Additionally, in cases of higher dimensionality, it was noted that excellent performance was exhibited in detecting local and global anomalies, while lower performance was observed for clustered anomaly types.

Real-time Intrusion-Detection Parallel System for the Prevention of Anomalous Computer Behaviours (비정상적인 컴퓨터 행위 방지를 위한 실시간 침입 탐지 병렬 시스템에 관한 연구)

  • 유은진;전문석
    • Review of KIISC
    • /
    • v.5 no.2
    • /
    • pp.32-48
    • /
    • 1995
  • Our paper describes an Intrusion Detection Parallel System(IDPS) which detects an anomaly activity corresponding to the actions that interaction between near detection events. IDES uses parallel inductive approaches regarding the problem of real-time anomaly behavior detection on rule-based system. This approach uses sequential rule that describes user's behavior and characteristics dependent on time. and that audits user's activities by using rule base as data base to store user's behavior pattern. When user's activity deviates significantly from expected behavior described in rule base. anomaly behaviors are recorded. Observed behavior is flagged as a potential intrusion if it deviates significantly from the expected behavior or if it triggers a rule in the parallel inductive system.

  • PDF

Keyed learning: An adversarial learning framework-formalization, challenges, and anomaly detection applications

  • Bergadano, Francesco
    • ETRI Journal
    • /
    • v.41 no.5
    • /
    • pp.608-618
    • /
    • 2019
  • We propose a general framework for keyed learning, where a secret key is used as an additional input of an adversarial learning system. We also define models and formal challenges for an adversary who knows the learning algorithm and its input data but has no access to the key value. This adversarial learning framework is subsequently applied to a more specific context of anomaly detection, where the secret key finds additional practical uses and guides the entire learning and alarm-generating procedure.

Anomaly Detection in Smart Homes Using Bayesian Networks

  • Saqaeeyan, Sasan;javadi, Hamid Haj Seyyed;Amirkhani, Hossein
    • KSII Transactions on Internet and Information Systems (TIIS)
    • /
    • v.14 no.4
    • /
    • pp.1796-1816
    • /
    • 2020
  • The health and safety of elderly and disabled patients who cannot live alone is an important issue. Timely detection of sudden events is necessary to protect these people, and anomaly detection in smart homes is an efficient approach to extracting such information. In the real world, there is a causal relationship between an occupant's behaviour and the order in which appliances are used in the home. Bayesian networks are appropriate tools for assessing the probability of an effect due to the occurrence of its causes, and vice versa. This paper defines different subsets of random variables on the basis of sensory data from a smart home, and it presents an anomaly detection system based on various models of Bayesian networks and drawing upon these variables. We examine different models to obtain the best network, one that has higher assessment scores and a smaller size. Experimental evaluations of real datasets show the effectiveness of the proposed method.

Anomaly detection of smart metering system for power management with battery storage system/electric vehicle

  • Sangkeum Lee;Sarvar Hussain Nengroo;Hojun Jin;Yoonmee Doh;Chungho Lee;Taewook Heo;Dongsoo Har
    • ETRI Journal
    • /
    • v.45 no.4
    • /
    • pp.650-665
    • /
    • 2023
  • A novel smart metering technique capable of anomaly detection was proposed for real-time home power management system. Smart meter data generated in real-time were obtained from 900 households of single apartments. To detect outliers and missing values in smart meter data, a deep learning model, the autoencoder, consisting of a graph convolutional network and bidirectional long short-term memory network, was applied to the smart metering technique. Power management based on the smart metering technique was executed by multi-objective optimization in the presence of a battery storage system and an electric vehicle. The results of the power management employing the proposed smart metering technique indicate a reduction in electricity cost and amount of power supplied by the grid compared to the results of power management without anomaly detection.

Anomaly Intrusion Detection using Fuzzy Membership Function and Neural Networks (퍼지 멤버쉽 함수와 신경망을 이용한 이상 침입 탐지)

  • Cha, Byung-Rae
    • The KIPS Transactions:PartC
    • /
    • v.11C no.5
    • /
    • pp.595-604
    • /
    • 2004
  • By the help of expansion of computer network and rapid growth of Internet, the information infrastructure is now able to provide a wide range of services. Especially open architecture - the inherent nature of Internet - has not only got in the way of offering QoS service, managing networks, but also made the users vulnerable to both the threat of backing and the issue of information leak. Thus, people recognized the importance of both taking active, prompt and real-time action against intrusion threat, and at the same time, analyzing the similar patterns of in-trusion already known. There are now many researches underway on Intrusion Detection System(IDS). The paper carries research on the in-trusion detection system which hired supervised learning algorithm and Fuzzy membership function especially with Neuro-Fuzzy model in order to improve its performance. It modifies tansigmoid transfer function of Neural Networks into fuzzy membership function, so that it can reduce the uncertainty of anomaly intrusion detection. Finally, the fuzzy logic suggested here has been applied to a network-based anomaly intrusion detection system, tested against intrusion data offered by DARPA 2000 Intrusion Data Sets, and proven that it overcomes the shortcomings that Anomaly Intrusion Detection usually has.

Anomaly Detection Analysis using Repository based on Inverted Index (역방향 인덱스 기반의 저장소를 이용한 이상 탐지 분석)

  • Park, Jumi;Cho, Weduke;Kim, Kangseok
    • Journal of KIISE
    • /
    • v.45 no.3
    • /
    • pp.294-302
    • /
    • 2018
  • With the emergence of the new service industry due to the development of information and communication technology, cyber space risks such as personal information infringement and industrial confidentiality leakage have diversified, and the security problem has emerged as a critical issue. In this paper, we propose a behavior-based anomaly detection method that is suitable for real-time and large-volume data analysis technology. We show that the proposed detection method is superior to existing signature security countermeasures that are based on large-capacity user log data according to in-company personal information abuse and internal information leakage. As the proposed behavior-based anomaly detection method requires a technique for processing large amounts of data, a real-time search engine is used, called Elasticsearch, which is based on an inverted index. In addition, statistical based frequency analysis and preprocessing were performed for data analysis, and the DBSCAN algorithm, which is a density based clustering method, was applied to classify abnormal data with an example for easy analysis through visualization. Unlike the existing anomaly detection system, the proposed behavior-based anomaly detection technique is promising as it enables anomaly detection analysis without the need to set the threshold value separately, and was proposed from a statistical perspective.

A Study on Traffic Anomaly Detection Scheme Based Time Series Model (시계열 모델 기반 트래픽 이상 징후 탐지 기법에 관한 연구)

  • Cho, Kang-Hong;Lee, Do-Hoon
    • The Journal of Korean Institute of Communications and Information Sciences
    • /
    • v.33 no.5B
    • /
    • pp.304-309
    • /
    • 2008
  • This paper propose the traffic anomaly detection scheme based time series model. We apply ARIMA prediction model to this scheme and transform the value of the abnormal symptom into the probability value to maximize the traffic anomaly symptom detection. For this, we have evaluated the abnormal detection performance for the proposed model using total traffic and web traffic included the attack traffic. We will expect to have an great effect if this scheme is included in some network based intrusion detection system.

An Effective Anomaly Detection Approach based on Hybrid Unsupervised Learning Technologies in NIDS

  • Kangseok Kim
    • KSII Transactions on Internet and Information Systems (TIIS)
    • /
    • v.18 no.2
    • /
    • pp.494-510
    • /
    • 2024
  • Internet users are exposed to sophisticated cyberattacks that intrusion detection systems have difficulty detecting. Therefore, research is increasing on intrusion detection methods that use artificial intelligence technology for detecting novel cyberattacks. Unsupervised learning-based methods are being researched that learn only from normal data and detect abnormal behaviors by finding patterns. This study developed an anomaly-detection method based on unsupervised machines and deep learning for a network intrusion detection system (NIDS). We present a hybrid anomaly detection approach based on unsupervised learning techniques using the autoencoder (AE), Isolation Forest (IF), and Local Outlier Factor (LOF) algorithms. An oversampling approach that increased the detection rate was also examined. A hybrid approach that combined deep learning algorithms and traditional machine learning algorithms was highly effective in setting the thresholds for anomalies without subjective human judgment. It achieved precision and recall rates respectively of 88.2% and 92.8% when combining two AEs, IF, and LOF while using an oversampling approach to learn more unknown normal data improved the detection accuracy. This approach achieved precision and recall rates respectively of 88.2% and 94.6%, further improving the detection accuracy compared with the hybrid method. Therefore, in NIDS the proposed approach provides high reliability for detecting cyberattacks.

Anomaly behavior detection using Negative Selection algorithm based anomaly detector (Negative Selection 알고리즘 기반 이상탐지기를 이용한 이상행 위 탐지)

  • 김미선;서재현
    • Proceedings of the Korean Institute of Information and Commucation Sciences Conference
    • /
    • 2004.05b
    • /
    • pp.391-394
    • /
    • 2004
  • Change of paradigm of network attack technique was begun by fast extension of the latest Internet and new attack form is appearing. But, Most intrusion detection systems detect informed attack type because is doing based on misuse detection, and active correspondence is difficult in new attack. Therefore, to heighten detection rate for new attack pattern, visibilitys to apply human immunity mechanism are appearing. In this paper, we create self-file from normal behavior profile about network packet and embody self recognition algorithm to use self-nonself discrimination in the human immune system to detect anomaly behavior. Sense change because monitors self-file creating anomaly detector based on Negative Selection Algorithm that is self recognition algorithm's one and detects anomaly behavior. And we achieve simulation to use DARPA Network Dataset and verify effectiveness of algorithm through the anomaly detection rate.

  • PDF