• 제목/요약/키워드: Anomaly detection

검색결과 670건 처리시간 0.049초

An Outlier Cluster Detection Technique for Real-time Network Intrusion Detection Systems (실시간 네트워크 침입탐지 시스템을 위한 아웃라이어 클러스터 검출 기법)

  • Chang, Jae-Young;Park, Jong-Myoung;Kim, Han-Joon
    • Journal of Internet Computing and Services
    • /
    • 제8권6호
    • /
    • pp.43-53
    • /
    • 2007
  • Intrusion detection system(IDS) has recently evolved while combining signature-based detection approach with anomaly detection approach. Although signature-based IDS tools have been commonly used by utilizing machine learning algorithms, they only detect network intrusions with already known patterns, Ideal IDS tools should always keep the signature database of your detection system up-to-date. The system needs to generate the signatures to detect new possible attacks while monitoring and analyzing incoming network data. In this paper, we propose a new outlier cluster detection algorithm with density (or influence) function, Our method assumes that an outlier is a kind of cluster with similar instances instead of a single object in the context of network intrusion, Through extensive experiments using KDD 1999 Cup Intrusion Detection dataset. we show that the proposed method outperform the conventional outlier detection method using Euclidean distance function, specially when attacks occurs frequently.

  • PDF

Design of Multi-Level Abnormal Detection System Suitable for Time-Series Data (시계열 데이터에 적합한 다단계 비정상 탐지 시스템 설계)

  • Chae, Moon-Chang;Lim, Hyeok;Kang, Namhi
    • The Journal of the Institute of Internet, Broadcasting and Communication
    • /
    • 제16권6호
    • /
    • pp.1-7
    • /
    • 2016
  • As new information and communication technologies evolve, security threats are also becoming increasingly intelligent and advanced. In this paper, we analyze the time series data continuously entered through a series of periods from the network device or lightweight IoT (Internet of Things) devices by using the statistical technique and propose a system to detect abnormal behaviors of the device or abnormality based on the analysis results. The proposed system performs the first level abnormal detection by using previously entered data set, thereafter performs the second level anomaly detection according to the trust bound configured by using stored time series data based on time attribute or group attribute. Multi-level analysis is able to improve reliability and to reduce false positives as well through a variety of decision data set.

Analyzing Effective of Activation Functions on Recurrent Neural Networks for Intrusion Detection

  • Le, Thi-Thu-Huong;Kim, Jihyun;Kim, Howon
    • Journal of Multimedia Information System
    • /
    • 제3권3호
    • /
    • pp.91-96
    • /
    • 2016
  • Network security is an interesting area in Information Technology. It has an important role for the manager monitor and control operating of the network. There are many techniques to help us prevent anomaly or malicious activities such as firewall configuration etc. Intrusion Detection System (IDS) is one of effective method help us reduce the cost to build. The more attacks occur, the more necessary intrusion detection needs. IDS is a software or hardware systems, even though is a combination of them. Its major role is detecting malicious activity. In recently, there are many researchers proposed techniques or algorithms to build a tool in this field. In this paper, we improve the performance of IDS. We explore and analyze the impact of activation functions applying to recurrent neural network model. We use to KDD cup dataset for our experiment. By our experimental results, we verify that our new tool of IDS is really significant in this field.

Outlier detection of main engine data of a ship using ensemble method (앙상블 기법을 이용한 선박 메인엔진 빅데이터의 이상치 탐지)

  • KIM, Dong-Hyun;LEE, Ji-Hwan;LEE, Sang-Bong;JUNG, Bong-Kyu
    • Journal of the Korean Society of Fisheries and Ocean Technology
    • /
    • 제56권4호
    • /
    • pp.384-394
    • /
    • 2020
  • This paper proposes an outlier detection model based on machine learning that can diagnose the presence or absence of major engine parts through unsupervised learning analysis of main engine big data of a ship. Engine big data of the ship was collected for more than seven months, and expert knowledge and correlation analysis were performed to select features that are closely related to the operation of the main engine. For unsupervised learning analysis, ensemble model wherein many predictive models are strategically combined to increase the model performance, is used for anomaly detection. As a result, the proposed model successfully detected the anomalous engine status from the normal status. To validate our approach, clustering analysis was conducted to find out the different patterns of anomalies the anomalous point. By examining distribution of each cluster, we could successfully find the patterns of anomalies.

An Abnormal Worker Movement Detection System Based on Data Stream Processing and Hierarchical Clustering

  • Duong, Dat Van Anh;Lan, Doi Thi;Yoon, Seokhoon
    • International Journal of Internet, Broadcasting and Communication
    • /
    • 제14권4호
    • /
    • pp.88-95
    • /
    • 2022
  • Detecting anomalies in human movement is an important task in industrial applications, such as monitoring industrial disasters or accidents and recognizing unauthorized factory intruders. In this paper, we propose an abnormal worker movement detection system based on data stream processing and hierarchical clustering. In the proposed system, Apache Spark is used for streaming the location data of people. A hierarchical clustering-based anomalous trajectory detection algorithm is designed for detecting anomalies in human movement. The algorithm is integrated into Apache Spark for detecting anomalies from location data. Specifically, the location information is streamed to Apache Spark using the message queuing telemetry transport protocol. Then, Apache Spark processes and stores location data in a data frame. When there is a request from a client, the processed data in the data frame is taken and put into the proposed algorithm for detecting anomalies. A real mobility trace of people is used to evaluate the proposed system. The obtained results show that the system has high performance and can be used for a wide range of industrial applications.

A Network Intrusion Security Detection Method Using BiLSTM-CNN in Big Data Environment

  • Hong Wang
    • Journal of Information Processing Systems
    • /
    • 제19권5호
    • /
    • pp.688-701
    • /
    • 2023
  • The conventional methods of network intrusion detection system (NIDS) cannot measure the trend of intrusiondetection targets effectively, which lead to low detection accuracy. In this study, a NIDS method which based on a deep neural network in a big-data environment is proposed. Firstly, the entire framework of the NIDS model is constructed in two stages. Feature reduction and anomaly probability output are used at the core of the two stages. Subsequently, a convolutional neural network, which encompasses a down sampling layer and a characteristic extractor consist of a convolution layer, the correlation of inputs is realized by introducing bidirectional long short-term memory. Finally, after the convolution layer, a pooling layer is added to sample the required features according to different sampling rules, which promotes the overall performance of the NIDS model. The proposed NIDS method and three other methods are compared, and it is broken down under the conditions of the two databases through simulation experiments. The results demonstrate that the proposed model is superior to the other three methods of NIDS in two databases, in terms of precision, accuracy, F1- score, and recall, which are 91.64%, 93.35%, 92.25%, and 91.87%, respectively. The proposed algorithm is significant for improving the accuracy of NIDS.

Reinforcement Mining Method for Anomaly Detection and Misuse Detection using Post-processing and Training Method (이상탐지(Anomaly Detection) 및 오용탐지(Misuse Detection) 분석의 정확도 향상을 위한 개선된 데이터마이닝 방법 연구)

  • Choi Yun-Jeong;Park Seung-Soo
    • Proceedings of the Korean Information Science Society Conference
    • /
    • 한국정보과학회 2006년도 한국컴퓨터종합학술대회 논문집 Vol.33 No.1 (B)
    • /
    • pp.238-240
    • /
    • 2006
  • 네트워크상에서 발생하는 다양한 형태의 대량의 데이터를 정확하고 효율적으로 분석하기 위해 설계되고 있는 마이닝 시스템들은 목표지향적으로 훈련데이터들을 어떻게 구축하여 다룰 것인지에 대한 문제보다는 대부분 얼마나 많은 데이터 마이닝 기법을 지원하고 이를 적용할 수 있는지 등의 기법에 초점을 두고 있다. 따라서, 점점 더 에이전트화, 분산화, 자동화 및 은닉화 되는 최근의 보안공격기법을 정확하게 탐지하기 위한 방법은 미흡한 실정이다. 본 연구에서는 유비쿼터스 환경 내에서 발생 가능한 문제 중 복잡하고 지능화된 침입패턴의 탐지를 위해 데이터 마이닝 기법과 결함허용방법을 이용하는 개선된 학습알고리즘과 후처리 방법에 의한 RTPID(Refinement Training and Post-processing for Intrusion Detection)시스템을 제안한다. 본 논문에서의 RTPID 시스템은 active learning과 post-processing을 이용하여, 네트워크 내에서 발생 가능한 침입형태들을 정확하고 효율적으로 다루어 분석하고 있다. 이는 기법에만 초점을 맞춘 기존의 데이터마이닝 분석을 개선하고 있으며, 특히 제안된 분석 프로세스를 진행하는 동안 능동학습방법의 장점을 수용하여 학습효과는 높이며 비용을 감소시킬 수 있는 자가학습방법(self learning)방법의 효과를 기대할 수 있다. 이는 관리자의 개입을 최소화하는 학습방법이면서 동시에 False Positive와 False Negative 의 오류를 매우 효율적으로 개선하는 방법으로 기대된다. 본 논문의 제안방법은 분석도구나 시스템에 의존하지 않기 때문에, 유사한 문제를 안고 있는 여러 분야의 네트웍 환경에 적용될 수 있다.더욱 높은성능을 가짐을 알 수 있다.의 각 노드의 전력이 위험할 때 에러 패킷을 발생하는 기법을 추가하였다. NS-2 시뮬레이터를 이용하여 실험을 한 결과, 제안한 기법이 AOMDV에 비해 경로 탐색 횟수가 최대 36.57% 까지 감소되었음을 알 수 있었다.의 작용보다 더 강력함을 시사하고 있다.TEX>로 최고값을 나타내었으며 그 후 감소하여 담금 10일에는 $1.61{\sim}2.34%$였다. 시험구간에는 KKR, SKR이 비교적 높은 값을 나타내었다. 무기질 함량은 발효기간이 경과할수록 증하였고 Ca는 $2.95{\sim}36.76$, Cu는 $0.01{\sim}0.14$, Fe는 $0.71{\sim}3.23$, K는 $110.89{\sim}517.33$, Mg는 $34.78{\sim}122.40$, Mn은 $0.56{\sim}5.98$, Na는 $0.19{\sim}14.36$, Zn은 $0.90{\sim}5.71ppm$을 나타내었으며, 시험구별로 보면 WNR, BNR구가 Na만 제외한 다른 무기성분 함량이 가장 높았다.O to reduce I/O cost by reusing data already present in the memory of other nodes. Finally, chunking and on-line compression mechanisms are included in both models. We demonstrate that we can obtain significantly high-performanc

  • PDF

An Intrusion Detection System based on the Artificial Neural Network for Real Time Detection (실시간 탐지를 위한 인공신경망 기반의 네트워크 침입탐지 시스템)

  • Kim, Tae Hee;Kang, Seung Ho
    • Convergence Security Journal
    • /
    • 제17권1호
    • /
    • pp.31-38
    • /
    • 2017
  • As the cyber-attacks through the networks advance, it is difficult for the intrusion detection system based on the simple rules to detect the novel type of attacks such as Advanced Persistent Threat(APT) attack. At present, many types of research have been focused on the application of machine learning techniques to the intrusion detection system in order to detect previously unknown attacks. In the case of using the machine learning techniques, the performance of the intrusion detection system largely depends on the feature set which is used as an input to the system. Generally, more features increase the accuracy of the intrusion detection system whereas they cause a problem when fast responses are required owing to their large elapsed time. In this paper, we present a network intrusion detection system based on artificial neural network, which adopts a multi-objective genetic algorithm to satisfy the both requirements: accuracy, and fast response. The comparison between the proposing approach and previously proposed other approaches is conducted against NSL_KDD data set for the evaluation of the performance of the proposing approach.

Intrusion Detection System Based on Multi-Class SVM (다중 클래스 SVM기반의 침입탐지 시스템)

  • Lee Hansung;Song Jiyoung;Kim Eunyoung;Lee Chulho;Park Daihee
    • Journal of the Korean Institute of Intelligent Systems
    • /
    • 제15권3호
    • /
    • pp.282-288
    • /
    • 2005
  • In this paper, we propose a new intrusion detection model, which keeps advantages of existing misuse detection model and anomaly detection model and resolves their problems. This new intrusion detection system, named to MMIDS, was designed to satisfy all the following requirements : 1) Fast detection of new types of attack unknown to the system; 2) Provision of detail information about the detected types of attack; 3) cost-effective maintenance due to fast and efficient learning and update; 4) incrementality and scalability of system. The fast and efficient training and updating faculties of proposed novel multi-class SVM which is a core component of MMIDS provide cost-effective maintenance of intrusion detection system. According to the experimental results, our method can provide superior performance in separating similar patterns and detailed separation capability of MMIDS is relatively good.

Sequence based Intrusion Detection using Similarity Matching of the Multiple Sequence Alignments (다중서열정렬의 유사도 매칭을 이용한 순서기반 침입탐지)

  • Kim Yong-Min
    • Journal of the Korea Institute of Information Security & Cryptology
    • /
    • 제16권1호
    • /
    • pp.115-122
    • /
    • 2006
  • The most methods for intrusion detection are based on the misuse detection which accumulates hewn intrusion information and makes a decision of an attack against any behavior data. However it is very difficult to detect a new or modified aoack with only the collected patterns of attack behaviors. Therefore, if considering that the method of anomaly behavior detection actually has a high false detection rate, a new approach is required for very huge intrusion patterns based on sequence. The approach can improve a possibility for intrusion detection of known attacks as well as modified and unknown attacks in addition to the similarity measurement of intrusion patterns. This paper proposes a method which applies the multiple sequence alignments technique to the similarity matching of the sequence based intrusion patterns. It enables the statistical analysis of sequence patterns and can be implemented easily. Also, the method reduces the number of detection alerts and false detection for attacks according to the changes of a sequence size.