• Title/Summary/Keyword: Anomaly data detection

Search Result 402, Processing Time 0.022 seconds

Enhanced Network Intrusion Detection using Deep Convolutional Neural Networks

  • Naseer, Sheraz;Saleem, Yasir
    • KSII Transactions on Internet and Information Systems (TIIS)
    • /
    • v.12 no.10
    • /
    • pp.5159-5178
    • /
    • 2018
  • Network Intrusion detection is a rapidly growing field of information security due to its importance for modern IT infrastructure. Many supervised and unsupervised learning techniques have been devised by researchers from discipline of machine learning and data mining to achieve reliable detection of anomalies. In this paper, a deep convolutional neural network (DCNN) based intrusion detection system (IDS) is proposed, implemented and analyzed. Deep CNN core of proposed IDS is fine-tuned using Randomized search over configuration space. Proposed system is trained and tested on NSLKDD training and testing datasets using GPU. Performance comparisons of proposed DCNN model are provided with other classifiers using well-known metrics including Receiver operating characteristics (RoC) curve, Area under RoC curve (AuC), accuracy, precision-recall curve and mean average precision (mAP). The experimental results of proposed DCNN based IDS shows promising results for real world application in anomaly detection systems.

Development of facility safety diagnosis system for offshore wind power using semi-supervised machine learning (준지도 학습 머신러닝을 이용한 해상 풍력용 설비안전 진단 시스템의 개발)

  • Woo-Jin Choi
    • Journal of Wind Energy
    • /
    • v.13 no.3
    • /
    • pp.33-42
    • /
    • 2022
  • In this paper, a semi-supervised machine learning technique applied to actual field vibration data acquired from Jeju-do wind turbines for predictive diagnosis of abnormal conditions of offshore wind turbines is introduced. Semi-supervised machine learning, which combines un-supervised learning with supervised learning, can be used to perform anomaly detection in situations where sufficient fault data cannot be obtained. The signal processing results using the spectrogram of the original signal were shown, and external data were used to overcome the problem that disturbance reactions easily occurred due to the imbalance between the number of normal and abnormal data. Out of distribution (OOD), which uses external data, is a technology that is regarded as abnormal data that is unlikely to occur in reality, but we were able to use it by expanding it. By rearranging the distribution of data in this way, classification can be performed more robustly. Specifically, by observing the trends of the abnormal score and the change in the feature of the representation layer, continuous learning was performed through a mixture of existing and new data.

Application of Discrete Wavelet Transforms to Identify Unknown Attacks in Anomaly Detection Analysis (이상 탐지 분석에서 알려지지 않는 공격을 식별하기 위한 이산 웨이블릿 변환 적용 연구)

  • Kim, Dong-Wook;Shin, Gun-Yoon;Yun, Ji-Young;Kim, Sang-Soo;Han, Myung-Mook
    • Journal of Internet Computing and Services
    • /
    • v.22 no.3
    • /
    • pp.45-52
    • /
    • 2021
  • Although many studies have been conducted to identify unknown attacks in cyber security intrusion detection systems, studies based on outliers are attracting attention. Accordingly, we identify outliers by defining categories for unknown attacks. The unknown attacks were investigated in two categories: first, there are factors that generate variant attacks, and second, studies that classify them into new types. We have conducted outlier studies that can identify similar data, such as variants, in the category of studies that generate variant attacks. The big problem of identifying anomalies in the intrusion detection system is that normal and aggressive behavior share the same space. For this, we applied a technique that can be divided into clear types for normal and attack by discrete wavelet transformation and detected anomalies. As a result, we confirmed that the outliers can be identified through One-Class SVM in the data reconstructed by discrete wavelet transform.

On the Hybrid Intrusion Detection System based Biometric Efficiency (생체 면역 기반의 하이브리드 침입 탐지 시스템에 관하여)

  • 양은목;이상용;서창호;김석우
    • Convergence Security Journal
    • /
    • v.1 no.1
    • /
    • pp.57-68
    • /
    • 2001
  • Computer security is considered important because of the side effect generated from the expansion of computer network and rapid increase of the use of computer. Intrusion Detection System(IDS) has been an active research area to reduce the risk from intruders. In this paper, the Hybrid Intrusion Detection System(HIDS) based biometric immuntiy collects and filters audit data by misuse detection is innate immune, and anomaly detection is acquirement immune in multi-hosts. Since, collect and detect audit data from one the system in molt-hosts, it is design and implement of the intrusion detection system which has the immuntiy the detection intrusion in one host possibly can detect in multi-hosts and in the method of misuses detection subsequently.

  • PDF

Design of Multi-Level Abnormal Detection System Suitable for Time-Series Data (시계열 데이터에 적합한 다단계 비정상 탐지 시스템 설계)

  • Chae, Moon-Chang;Lim, Hyeok;Kang, Namhi
    • The Journal of the Institute of Internet, Broadcasting and Communication
    • /
    • v.16 no.6
    • /
    • pp.1-7
    • /
    • 2016
  • As new information and communication technologies evolve, security threats are also becoming increasingly intelligent and advanced. In this paper, we analyze the time series data continuously entered through a series of periods from the network device or lightweight IoT (Internet of Things) devices by using the statistical technique and propose a system to detect abnormal behaviors of the device or abnormality based on the analysis results. The proposed system performs the first level abnormal detection by using previously entered data set, thereafter performs the second level anomaly detection according to the trust bound configured by using stored time series data based on time attribute or group attribute. Multi-level analysis is able to improve reliability and to reduce false positives as well through a variety of decision data set.

Design and Implementation of a Web Application Firewall with Multi-layered Web Filter (다중 계층 웹 필터를 사용하는 웹 애플리케이션 방화벽의 설계 및 구현)

  • Jang, Sung-Min;Won, Yoo-Hun
    • Journal of the Korea Society of Computer and Information
    • /
    • v.14 no.12
    • /
    • pp.157-167
    • /
    • 2009
  • Recently, the leakage of confidential information and personal information is taking place on the Internet more frequently than ever before. Most of such online security incidents are caused by attacks on vulnerabilities in web applications developed carelessly. It is impossible to detect an attack on a web application with existing firewalls and intrusion detection systems. Besides, the signature-based detection has a limited capability in detecting new threats. Therefore, many researches concerning the method to detect attacks on web applications are employing anomaly-based detection methods that use the web traffic analysis. Much research about anomaly-based detection through the normal web traffic analysis focus on three problems - the method to accurately analyze given web traffic, system performance needed for inspecting application payload of the packet required to detect attack on application layer and the maintenance and costs of lots of network security devices newly installed. The UTM(Unified Threat Management) system, a suggested solution for the problem, had a goal of resolving all of security problems at a time, but is not being widely used due to its low efficiency and high costs. Besides, the web filter that performs one of the functions of the UTM system, can not adequately detect a variety of recent sophisticated attacks on web applications. In order to resolve such problems, studies are being carried out on the web application firewall to introduce a new network security system. As such studies focus on speeding up packet processing by depending on high-priced hardware, the costs to deploy a web application firewall are rising. In addition, the current anomaly-based detection technologies that do not take into account the characteristics of the web application is causing lots of false positives and false negatives. In order to reduce false positives and false negatives, this study suggested a realtime anomaly detection method based on the analysis of the length of parameter value contained in the web client's request. In addition, it designed and suggested a WAF(Web Application Firewall) that can be applied to a low-priced system or legacy system to process application data without the help of an exclusive hardware. Furthermore, it suggested a method to resolve sluggish performance attributed to copying packets into application area for application data processing, Consequently, this study provide to deploy an effective web application firewall at a low cost at the moment when the deployment of an additional security system was considered burdened due to lots of network security systems currently used.

Abnormality Detection Control System using Charging Data (충전데이터를 이용한 이상감지 제어시스템)

  • Moon, Sang-Ho
    • Journal of the Korea Institute of Information and Communication Engineering
    • /
    • v.26 no.2
    • /
    • pp.313-316
    • /
    • 2022
  • In this paper, we implement a system that detects abnormalities in the charging data transmitted from the charger during the charging process of electric vehicles and controls them remotely. Using classification algorithms such as logistic regression, KNN, SVM, and decision trees, to do this, an analysis model is created that judges the data received from the charger as normal and abnormal. In addition, a model is created to determine the cause of the abnormality using the existing charging data based on the analysis of the type of charger abnormality. Finally, it is solved using unsupervised learning method to find new patterns of abnormal data.

Study on Lifelog Anomaly Detection using VAE-based Machine Learning Model (VAE(Variational AutoEncoder) 기반 머신러닝 모델을 활용한 체중 라이프로그 이상탐지에 관한 연구)

  • Kim, Jiyong;Park, Minseo
    • The Journal of the Convergence on Culture Technology
    • /
    • v.8 no.4
    • /
    • pp.91-98
    • /
    • 2022
  • Lifelog data continuously collected through a wearable device may contain many outliers, so in order to improve data quality, it is necessary to find and remove outliers. In general, since the number of outliers is less than the number of normal data, a class imbalance problem occurs. To solve this imbalance problem, we propose a method that applies Variational AutoEncoder to outliers. After preprocessing the outlier data with proposed method, it is verified through a number of machine learning models(classification). As a result of verification using body weight data, it was confirmed that the performance was improved in all classification models. Based on the experimental results, when analyzing lifelog body weight data, we propose to apply the LightGBM model with the best performance after preprocessing the data using the outlier processing method proposed in this study.

Anomaly Detections Model of Aviation System by CNN (합성곱 신경망(CNN)을 활용한 항공 시스템의 이상 탐지 모델 연구)

  • Hyun-Jae Im;Tae-Rim Kim;Jong-Gyu Song;Bum-Su Kim
    • Journal of Aerospace System Engineering
    • /
    • v.17 no.4
    • /
    • pp.67-74
    • /
    • 2023
  • Recently, Urban Aircraft Mobility (UAM) has been attracting attention as a transportation system of the future, and small drones also play a role in various industries. The failure of various types of aviation systems can lead to crashes, which can result in significant property damage or loss of life. In the defense industry, where aviation systems are widely used, the failure of aviation systems can lead to mission failure. Therefore, this study proposes an anomaly detection model using deep learning technology to detect anomalies in aviation systems to improve the reliability of development and production, and prevent accidents during operation. As training and evaluating data sets, current data from aviation systems in an extremely low-temperature environment was utilized, and a deep learning network was implemented using the convolutional neural network, which is a deep learning technique that is commonly used for image recognition. In an extremely low-temperature environment, various types of failure occurred in the system's internal sensors and components, and singular points in current data were observed. As a result of training and evaluating the model using current data in the case of system failure and normal, it was confirmed that the abnormality was detected with a recall of 98 % or more.

Sound Source Level Error on Element Spacing and Depth of Hydrophone Array (수중청음기 배열의 간격 및 깊이 변화에 따른 측정 소음준위 오차)

  • 윤종락
    • Proceedings of the Acoustical Society of Korea Conference
    • /
    • 1997.06a
    • /
    • pp.68-74
    • /
    • 1997
  • Ship radiated noise is an infortant parameter which dtermines Anti Submarine Warfare(ASW) countermeansure or passive Sonar detection and classification performance. Its measurement should be performed under controlled ocean acoustic environment. In data reduction of the measured data from hydrophone array, theeffect fo ambient noise, surface reflection and bottom reflection etc. should be compensated to obtain the source level of the ship radiated noise. This study describes the measurement hydrophone array design criteria based on the analysis of transimission anomaly due to the surface reflection.

  • PDF