• 제목/요약/키워드: Anomaly data detection

Search Result 402, Processing Time 0.021 seconds

Reinforcement Mining Method for Anomaly Detection and Misuse Detection using Post-processing and Training Method (이상탐지(Anomaly Detection) 및 오용탐지(Misuse Detection) 분석의 정확도 향상을 위한 개선된 데이터마이닝 방법 연구)

  • Choi Yun-Jeong;Park Seung-Soo
    • Proceedings of the Korean Information Science Society Conference
    • /
    • 2006.06b
    • /
    • pp.238-240
    • /
    • 2006
  • 네트워크상에서 발생하는 다양한 형태의 대량의 데이터를 정확하고 효율적으로 분석하기 위해 설계되고 있는 마이닝 시스템들은 목표지향적으로 훈련데이터들을 어떻게 구축하여 다룰 것인지에 대한 문제보다는 대부분 얼마나 많은 데이터 마이닝 기법을 지원하고 이를 적용할 수 있는지 등의 기법에 초점을 두고 있다. 따라서, 점점 더 에이전트화, 분산화, 자동화 및 은닉화 되는 최근의 보안공격기법을 정확하게 탐지하기 위한 방법은 미흡한 실정이다. 본 연구에서는 유비쿼터스 환경 내에서 발생 가능한 문제 중 복잡하고 지능화된 침입패턴의 탐지를 위해 데이터 마이닝 기법과 결함허용방법을 이용하는 개선된 학습알고리즘과 후처리 방법에 의한 RTPID(Refinement Training and Post-processing for Intrusion Detection)시스템을 제안한다. 본 논문에서의 RTPID 시스템은 active learning과 post-processing을 이용하여, 네트워크 내에서 발생 가능한 침입형태들을 정확하고 효율적으로 다루어 분석하고 있다. 이는 기법에만 초점을 맞춘 기존의 데이터마이닝 분석을 개선하고 있으며, 특히 제안된 분석 프로세스를 진행하는 동안 능동학습방법의 장점을 수용하여 학습효과는 높이며 비용을 감소시킬 수 있는 자가학습방법(self learning)방법의 효과를 기대할 수 있다. 이는 관리자의 개입을 최소화하는 학습방법이면서 동시에 False Positive와 False Negative 의 오류를 매우 효율적으로 개선하는 방법으로 기대된다. 본 논문의 제안방법은 분석도구나 시스템에 의존하지 않기 때문에, 유사한 문제를 안고 있는 여러 분야의 네트웍 환경에 적용될 수 있다.더욱 높은성능을 가짐을 알 수 있다.의 각 노드의 전력이 위험할 때 에러 패킷을 발생하는 기법을 추가하였다. NS-2 시뮬레이터를 이용하여 실험을 한 결과, 제안한 기법이 AOMDV에 비해 경로 탐색 횟수가 최대 36.57% 까지 감소되었음을 알 수 있었다.의 작용보다 더 강력함을 시사하고 있다.TEX>로 최고값을 나타내었으며 그 후 감소하여 담금 10일에는 $1.61{\sim}2.34%$였다. 시험구간에는 KKR, SKR이 비교적 높은 값을 나타내었다. 무기질 함량은 발효기간이 경과할수록 증하였고 Ca는 $2.95{\sim}36.76$, Cu는 $0.01{\sim}0.14$, Fe는 $0.71{\sim}3.23$, K는 $110.89{\sim}517.33$, Mg는 $34.78{\sim}122.40$, Mn은 $0.56{\sim}5.98$, Na는 $0.19{\sim}14.36$, Zn은 $0.90{\sim}5.71ppm$을 나타내었으며, 시험구별로 보면 WNR, BNR구가 Na만 제외한 다른 무기성분 함량이 가장 높았다.O to reduce I/O cost by reusing data already present in the memory of other nodes. Finally, chunking and on-line compression mechanisms are included in both models. We demonstrate that we can obtain significantly high-performanc

  • PDF

Data Mining Approaches for DDoS Attack Detection (분산 서비스거부 공격 탐지를 위한 데이터 마이닝 기법)

  • Kim, Mi-Hui;Na, Hyun-Jung;Chae, Ki-Joon;Bang, Hyo-Chan;Na, Jung-Chan
    • Journal of KIISE:Information Networking
    • /
    • v.32 no.3
    • /
    • pp.279-290
    • /
    • 2005
  • Recently, as the serious damage caused by DDoS attacks increases, the rapid detection and the proper response mechanisms are urgent. However, existing security mechanisms do not effectively defend against these attacks, or the defense capability of some mechanisms is only limited to specific DDoS attacks. In this paper, we propose a detection architecture against DDoS attack using data mining technology that can classify the latest types of DDoS attack, and can detect the modification of existing attacks as well as the novel attacks. This architecture consists of a Misuse Detection Module modeling to classify the existing attacks, and an Anomaly Detection Module modeling to detect the novel attacks. And it utilizes the off-line generated models in order to detect the DDoS attack using the real-time traffic. We gathered the NetFlow data generated at an access router of our network in order to model the real network traffic and test it. The NetFlow provides the useful flow-based statistical information without tremendous preprocessing. Also, we mounted the well-known DDoS attack tools to gather the attack traffic. And then, our experimental results show that our approach can provide the outstanding performance against existing attacks, and provide the possibility of detection against the novel attack.

Efficient Masquerade Detection Based on SVM (SVM 기반의 효율적인 신분위장기법 탐지)

  • 김한성;권영희;차성덕
    • Journal of the Korea Institute of Information Security & Cryptology
    • /
    • v.13 no.5
    • /
    • pp.91-104
    • /
    • 2003
  • A masquerader is someone who pretends to be another user while invading the target user's accounts, directories, or files. The masquerade attack is the most serious computer misuse. Because, in most cases, after securing the other's password, the masquerader enters the computer system. The system such as IDS could not detect or response to the masquerader. The masquerade detection is the effort to find the masquerader automatically. This system will detect the activities of a masquerader by determining that user's activities violate a profile developed for that user with his audit data. From 1988, there are many efforts on this topic, but the success of the offers was limited and the performance was unsatisfactory. In this report we propose efficient masquerade detection system using SVM which create the user profile.

Fault Detection in Diecasting Process Based on Deep-Learning (다단계 딥러닝 기반 다이캐스팅 공정 불량 검출)

  • Jeongsu Lee;Youngsim, Choi
    • Journal of Korea Foundry Society
    • /
    • v.42 no.6
    • /
    • pp.369-376
    • /
    • 2022
  • The die-casting process is an important process for various industries, but there are limitations in the profitability and productivity of related companies due to the high defect rate. In order to overcome this, this study has developed die-casting fault detection modules based on industrial AI technologies. The developed module is constructed from three-stage models depending on the characteristics of the dataset. The first-stage model conducts fault detection based on supervised learning from the dataset without labels. The second-stage model realizes one-class classification based on semi-supervised learning, where the dataset only has production success labels. The third-stage model corresponds to fault detection based on supervised learning, where the dataset includes a small amount of production failure cases. The developed fault detection module exhibited outstanding performance with roughly 96% accuracy for actual process data.

Automatic False-Alarm Labeling for Sensor Data

  • Adi, Taufik Nur;Bae, Hyerim;Wahid, Nur Ahmad
    • Journal of the Korea Society of Computer and Information
    • /
    • v.24 no.2
    • /
    • pp.139-147
    • /
    • 2019
  • A false alarm, which is an incorrect report of an emergency, could trigger an unnecessary action. The predictive maintenance framework developed in our previous work has a feature whereby a machine alarm is triggered based on sensor data evaluation. The sensor data evaluator performs three essential evaluation steps. First, it evaluates each sensor data value based on its threshold (lower and upper bound) and labels the data value as "alarm" when the threshold is exceeded. Second, it calculates the duration of the occurrence of the alarm. Finally, in the third step, a domain expert is required to assess the results from the previous two steps and to determine, thereby, whether the alarm is true or false. There are drawbacks of the current evaluation method. It suffers from a high false-alarm ratio, and moreover, given the vast amount of sensor data to be assessed by the domain expert, the process of evaluation is prolonged and inefficient. In this paper, we propose a method for automatic false-alarm labeling that mimics how the domain expert determines false alarms. The domain expert determines false alarms by evaluating two critical factors, specifically the duration of alarm occurrence and identification of anomalies before or while the alarm occurs. In our proposed method, Hierarchical Temporal Memory (HTM) is utilized to detect anomalies. It is an unsupervised approach that is suitable to our main data characteristic, which is the lack of an example of the normal form of sensor data. The result shows that the technique is effective for automatic labeling of false alarms in sensor data.

Performance Comparison of Machine Learning Algorithms for Network Traffic Security in Medical Equipment (의료기기 네트워크 트래픽 보안 관련 머신러닝 알고리즘 성능 비교)

  • Seung Hyoung Ko;Joon Ho Park;Da Woon Wang;Eun Seok Kang;Hyun Wook Han
    • Journal of Information Technology Services
    • /
    • v.22 no.5
    • /
    • pp.99-108
    • /
    • 2023
  • As the computerization of hospitals becomes more advanced, security issues regarding data generated from various medical devices within hospitals are gradually increasing. For example, because hospital data contains a variety of personal information, attempts to attack it have been continuously made. In order to safely protect data from external attacks, each hospital has formed an internal team to continuously monitor whether the computer network is safely protected. However, there are limits to how humans can monitor attacks that occur on networks within hospitals in real time. Recently, artificial intelligence models have shown excellent performance in detecting outliers. In this paper, an experiment was conducted to verify how well an artificial intelligence model classifies normal and abnormal data in network traffic data generated from medical devices. There are several models used for outlier detection, but among them, Random Forest and Tabnet were used. Tabnet is a deep learning algorithm related to receive and classify structured data. Two algorithms were trained using open traffic network data, and the classification accuracy of the model was measured using test data. As a result, the random forest algorithm showed a classification accuracy of 93%, and Tapnet showed a classification accuracy of 99%. Therefore, it is expected that most outliers that may occur in a hospital network can be detected using an excellent algorithm such as Tabnet.

Intrusion Detection based on Clustering a Data Stream (데이터 스트림 클러스터링을 이용한 침임탐지)

  • Oh Sang-Hyun;Kang Jin-Suk;Byun Yung-Cheol
    • Proceedings of the Korea Contents Association Conference
    • /
    • 2005.11a
    • /
    • pp.529-532
    • /
    • 2005
  • In anomaly intrusion detection, how to model the normal behavior of activities performed by a user is an important issue. To extract the normal behavior as a profile, conventional data mining techniques are widely applied to a finite audit data set. However, these approaches can only model the static behavior of a user in the audit data set This drawback can be overcome by viewing the continuous activities of a user as an audit data stream. This paper proposes a new clustering algorithm which continuously models a data stream. A set of features is used to represent the characteristics of an activity. For each feature, the clusters of feature values corresponding to activities observed so far in an audit data stream are identified by the proposed clustering algorithm for data streams. As a result, without maintaining any historical activity of a user physically, new activities of the user can be continuously reflected to the on-going result of clustering.

  • PDF

Detection of unexploded ordnance (UXO) using marine magnetic gradiometer data (해양 자력구배 탐사자료를 이용한 UXO 탐지)

  • Salem Ahmed;Hamada Toshio;Asahina Joseph Kiyoshi;Ushijima Keisuke
    • Geophysics and Geophysical Exploration
    • /
    • v.8 no.1
    • /
    • pp.97-103
    • /
    • 2005
  • Recent development of marine magnetic gradient systems, using arrays of sensors, has made it possible to survey large contaminated areas very quickly. However, underwater Unexploded Ordnances (UXO) can be moved by water currents. Because of this mobility, the cleanup process in such situations becomes dynamic rather than static. This implies that detection should occur in near real-time for successful remediation. Therefore, there is a need for a fast interpretation method to rapidly detect signatures of underwater objects in marine magnetic data. In this paper, we present a fast method for location and characterization of underwater UXOs. The approach utilises gradient interpretation techniques (analytic signal and Euler methods) to locate the objects precisely. Then, using an iterative linear least-squares technique, we obtain the magnetization characteristics of the sources. The approach was applied to a theoretical marine magnetic anomaly, with random errors, over a known source. We demonstrate the practical utility of the method using marine magnetic gradient data from Japan.

An Anomalous Sequence Detection Method Based on An Extended LSTM Autoencoder (확장된 LSTM 오토인코더 기반 이상 시퀀스 탐지 기법)

  • Lee, Jooyeon;Lee, Ki Yong
    • The Journal of Society for e-Business Studies
    • /
    • v.26 no.1
    • /
    • pp.127-140
    • /
    • 2021
  • Recently, sequence data containing time information, such as sensor measurement data and purchase history, has been generated in various applications. So far, many methods for finding sequences that are significantly different from other sequences among given sequences have been proposed. However, most of them have a limitation that they consider only the order of elements in the sequences. Therefore, in this paper, we propose a new anomalous sequence detection method that considers both the order of elements and the time interval between elements. The proposed method uses an extended LSTM autoencoder model, which has an additional layer that converts a sequence into a form that can help effectively learn both the order of elements and the time interval between elements. The proposed method learns the features of the given sequences with the extended LSTM autoencoder model, and then detects sequences that the model does not reconstruct well as anomalous sequences. Using experiments on synthetic data that contains both normal and anomalous sequences, we show that the proposed method achieves an accuracy close to 100% compared to the method that uses only the traditional LSTM autoencoder.

Electrical resistivity survey and interpretation considering excavation effects for the detection of loose ground in urban area

  • Seo Young Song;Bitnarae Kim;Ahyun Cho;Juyeon Jeong;Dongkweon Lee;Myung Jin Nam
    • Geomechanics and Engineering
    • /
    • v.35 no.2
    • /
    • pp.109-119
    • /
    • 2023
  • Ground subsidence in urban areas due to excessive development and degraded underground facilities is a serious problem. Geophysical surveys have been conducted to estimate the distribution and scale of cavities and subsidence. In this study, electrical resistivity tomography (ERT) was performed near an area of road subsidence in an urban area. The subsidence arose due to groundwater leakage that carried soil into a neighboring excavation site. The ERT survey line was located between the main subsidence area and an excavation site. Because ERT data are affected by rapid topographic changes and surrounding structures, the influence of the excavation site on the data was analyzed through field-scale numerical modeling. The effect of an excavation should be considered when interpreting ERT data because it can lead to wrong anomalous results. A method for performing 2D inversion after correcting resistivity data for the effect of the excavation site was proposed. This method was initially tested using a field-scale numerical model that included the excavation site and subsurface anomaly, which was a loosened zone, and was then applied to field data. In addition, ERT data were interpreted using an existing in-house 3D algorithm, which considered the effect of excavation sites. The inversion results demonstrated that conductive anomalies in the loosened zone were greater compared to the inversion that did not consider the effects of excavation.