• Title/Summary/Keyword: Abnormal Data Detection

Search Result 362, Processing Time 0.024 seconds

Abnormal Data Augmentation Method Using Perturbation Based on Hypersphere for Semi-Supervised Anomaly Detection (준 지도 이상 탐지 기법의 성능 향상을 위한 섭동을 활용한 초구 기반 비정상 데이터 증강 기법)

  • Jung, Byeonggil;Kwon, Junhyung;Min, Dongjun;Lee, Sangkyun
    • Journal of the Korea Institute of Information Security & Cryptology
    • /
    • v.32 no.4
    • /
    • pp.647-660
    • /
    • 2022
  • Recent works demonstrate that the semi-supervised anomaly detection method functions quite well in the environment with normal data and some anomalous data. However, abnormal data shortages can occur in an environment where it is difficult to reserve anomalous data, such as an unknown attack in the cyber security fields. In this paper, we propose ADA-PH(Abnormal Data Augmentation Method using Perturbation based on Hypersphere), a novel anomalous data augmentation method that is applicable in an environment where abnormal data is insufficient to secure the performance of the semi-supervised anomaly detection method. ADA-PH generates abnormal data by perturbing samples located relatively far from the center of the hypersphere. With the network intrusion detection datasets where abnormal data is rare, ADA-PH shows 23.63% higher AUC performance than anomaly detection without data augmentation and even performs better than the other augmentation methods. Also, we further conduct quantitative and qualitative analysis on whether generated abnormal data is anomalous.

Abnormal signal detection based on parallel autoencoders (병렬 오토인코더 기반의 비정상 신호 탐지)

  • Lee, Kibae;Lee, Chong Hyun
    • The Journal of the Acoustical Society of Korea
    • /
    • v.40 no.4
    • /
    • pp.337-346
    • /
    • 2021
  • Detection of abnormal signal generally can be done by using features of normal signals as main information because of data imbalance. This paper propose an efficient method for abnormal signal detection using parallel AutoEncoder (AE) which can use features of abnormal signals as well. The proposed Parallel AE (PAE) is composed of a normal and an abnormal reconstructors having identical AE structure and train features of normal and abnormal signals, respectively. The PAE can effectively solve the imbalanced data problem by sequentially training normal and abnormal data. For further detection performance improvement, additional binary classifier can be added to the PAE. Through experiments using public acoustic data, we obtain that the proposed PAE shows Area Under Curve (AUC) improvement of minimum 22 % at the expenses of training time increased by 1.31 ~ 1.61 times to the single AE. Furthermore, the PAE shows 93 % AUC improvement in detecting abnormal underwater acoustic signal when pre-trained PAE is transferred to train open underwater acoustic data.

A Study on the Improvement of Bayesian networks in e-Trade (전자무역의 베이지안 네트워크 개선방안에 관한 연구)

  • Jeong, Boon-Do
    • International Commerce and Information Review
    • /
    • v.9 no.3
    • /
    • pp.305-320
    • /
    • 2007
  • With expanded use of B2B(between enterprises), B2G(between enterprises and government) and EDI(Electronic Data Interchange), and increased amount of available network information and information protection threat, as it was judged that security can not be perfectly assured only with security technology such as electronic signature/authorization and access control, Bayesian networks have been developed for protection of information. Therefore, this study speculates Bayesian networks system, centering on ERP(Enterprise Resource Planning). The Bayesian networks system is one of the methods to resolve uncertainty in electronic data interchange and is applied to overcome uncertainty of abnormal invasion detection in ERP. Bayesian networks are applied to construct profiling for system call and network data, and simulate against abnormal invasion detection. The host-based abnormal invasion detection system in electronic trade analyses system call, applies Bayesian probability values, and constructs normal behavior profile to detect abnormal behaviors. This study assumes before and after of delivery behavior of the electronic document through Bayesian probability value and expresses before and after of the delivery behavior or events based on Bayesian networks. Therefore, profiling process using Bayesian networks can be applied for abnormal invasion detection based on host and network. In respect to transmission and reception of electronic documents, we need further studies on standards that classify abnormal invasion of various patterns in ERP and evaluate them by Bayesian probability values, and on classification of B2B invasion pattern genealogy to effectively detect deformed abnormal invasion patterns.

  • PDF

Optimizing of Intrusion Detection Algorithm Performance and The development of Evaluation Methodology (침입탐지 알고리즘 성능 최적화 및 평가 방법론 개발)

  • Shin, Dae Cheol;Kim, Hong Yoon
    • Journal of Korea Society of Digital Industry and Information Management
    • /
    • v.8 no.1
    • /
    • pp.125-137
    • /
    • 2012
  • As the Internet use explodes recently, the malicious attacks and hacking for a system connected to network occur frequently. For such reason, lots of intrusion detection system has been developed. Intrusion detection system has abilities to detect abnormal behavior and unknown intrusions also it can detect intrusions by using patterns studied from various penetration methods. Various algorithms are studying now such as the statistical method for detecting abnormal behavior, extracting abnormal behavior, and developing patterns that can be expected. Etc. This study using clustering of data mining and association rule analyzes detecting areas based on two models and helps design detection system which detecting abnormal behavior, unknown attack, misuse attack in a large network.

Detection of System Abnormal State by Cyber Attack (사이버 공격에 의한 시스템 이상상태 탐지 기법)

  • Yoon, Yeo-jeong;Jung, You-jin
    • Journal of the Korea Institute of Information Security & Cryptology
    • /
    • v.29 no.5
    • /
    • pp.1027-1037
    • /
    • 2019
  • Conventional cyber-attack detection solutions are generally based on signature-based or malicious behavior analysis so that have had difficulty in detecting unknown method-based attacks. Since the various information occurring all the time reflects the state of the system, by modeling it in a steady state and detecting an abnormal state, an unknown attack can be detected. Since a variety of system information occurs in a string form, word embedding, ie, techniques for converting strings into vectors preserving their order and semantics, can be used for modeling and detection. Novelty Detection, which is a technique for detecting a small number of abnormal data in a plurality of normal data, can be performed in order to detect an abnormal condition. This paper proposes a method to detect system anomaly by cyber attack using embedding and novelty detection.

Dynamic data validation and reconciliation for improving the detection of sodium leakage in a sodium-cooled fast reactor

  • Sangjun Park;Jongin Yang;Jewhan Lee;Gyunyoung Heo
    • Nuclear Engineering and Technology
    • /
    • v.55 no.4
    • /
    • pp.1528-1539
    • /
    • 2023
  • Since the leakage of sodium in an SFR (sodium-cooled fast reactor) causes an explosion upon reaction with air and water, sodium leakages represent an important safety issue. In this study, a novel technique for improving the reliability of sodium leakage detection applying DDVR (dynamic data validation and reconciliation) is proposed and verified to resolve this technical issue. DDVR is an approach that aims to improve the accuracy of a target system in a dynamic state by minimizing random errors, such as from the uncertainty of instruments and the surrounding environment, and by eliminating gross errors, such as instrument failure, miscalibration, or aging, using the spatial redundancy of measurements in a physical model and the reliability information of the instruments. DDVR also makes it possible to estimate the state of unmeasured points. To validate this approach for supporting sodium leakage detection, this study applies experimental data from a sodium leakage detection experiment performed by the Korea Atomic Energy Research Institute. The validation results show that the reliability of sodium leakage detection is improved by cooperation between DDVR and hardware measurements. Based on these findings, technology integrating software and hardware approaches is suggested to improve the reliability of sodium leakage detection by presenting the expected true state of the system.

Design of Multi-Level Abnormal Detection System Suitable for Time-Series Data (시계열 데이터에 적합한 다단계 비정상 탐지 시스템 설계)

  • Chae, Moon-Chang;Lim, Hyeok;Kang, Namhi
    • The Journal of the Institute of Internet, Broadcasting and Communication
    • /
    • v.16 no.6
    • /
    • pp.1-7
    • /
    • 2016
  • As new information and communication technologies evolve, security threats are also becoming increasingly intelligent and advanced. In this paper, we analyze the time series data continuously entered through a series of periods from the network device or lightweight IoT (Internet of Things) devices by using the statistical technique and propose a system to detect abnormal behaviors of the device or abnormality based on the analysis results. The proposed system performs the first level abnormal detection by using previously entered data set, thereafter performs the second level anomaly detection according to the trust bound configured by using stored time series data based on time attribute or group attribute. Multi-level analysis is able to improve reliability and to reduce false positives as well through a variety of decision data set.

Implementation and Evaluation of Abnormal ECG Detection Algorithm Using DTW Minimum Accumulation Distance (DTW 최소누적거리를 이용한 심전도 이상 검출 알고리즘 구현 및 평가)

  • Noh, Yun-Hong;Lee, Young-Dong;Jeong, Do-Un
    • Journal of Sensor Science and Technology
    • /
    • v.21 no.1
    • /
    • pp.39-45
    • /
    • 2012
  • Recently the convergence of healthcare technology is used for daily life healthcare monitoring. Cardiac arrhythmia is presented by the state of the heart irregularity. Abnormal heart's electrical signal pathway or heart's tissue disorder could be the cause of cardiac arrhythmia. Fatal arrhythmia could put patient's life at risk. Therefore arrhythmia detection is very important. Previous studies on the detection of arrhythmia in various ECG analysis and classification methods had been carried out. In this paper, an ECG signal processing techniques to detect abnormal ECG based on DTW minimum accumulation distance through the template matching for normalized data and variable threshold method for ECG R-peak detection. Signal processing techniques able to determine the occurrence of normal ECG and abnormal ECG. Abnormal ECG detection algorithm using DTW minimum accumulation distance method is performed using MITBIH database for performance evaluation. Experiment result shows the average percentage accuracy of using the propose method for Rpeak detection is 99.63 % and abnormal detection is 99.60 %.

The Fault Diagnosis Method of Diesel Engines Using a Statistical Analysis Method (통계적 분석기법을 이용한 디젤기관의 고장진단 방법에 관한 연구)

  • Kim, Young-Il;Oh, Hyun-Kyung;Yu, Yung-Ho
    • Journal of Advanced Marine Engineering and Technology
    • /
    • v.30 no.2
    • /
    • pp.247-252
    • /
    • 2006
  • Almost ship monitoring systems are event driven alarm system which warn only when the measurement value is over or under set point. These kinds of system cannot warn until signal is growing to abnormal state that the signal is over or under the set point. therefore cannot play a role for preventive maintenance system. This paper proposes fault diagnosis method which is able to diagnose and forecast the fault from present operating condition by analyzing monitored signals with present ship monitoring system without any additional sensors. By analyzing the data with high correlation coefficient(CC), correlation level of interactive data can be defined. Knowledge base of abnormal detection can be built by referring level of CC(Fault Detection CC. FDCC) to detect abnormal data among monitored data from monitoring system and knowledge base of diagnosis built by referring CC among interactive data for related machine each other to diagnose fault part.

The Fault Diagnosis Method of Diesel Engines Using a Statistical Analysis Method (통계적분석기법을 이용한 디젤기관의 고장진단 방법에 관한 연구)

  • Kim, Young-Il;Oh, Hyun-Gyeong;Cheon, Hang-Chun;Yu, Yung-Ho
    • Proceedings of the Korean Society of Marine Engineers Conference
    • /
    • 2005.06a
    • /
    • pp.281-286
    • /
    • 2005
  • Almost ship monitoring systems are event driven alarm system which warn only when the measurement value is over or under set point. These kinds of system cannot warn while signal is growing to abnormal state until the signal is over or under the set point and cannot play a role for preventive maintenance system. This paper proposes fault diagnosis method which is able to diagnose and forecast the fault from present operating condition by analyzing monitored signals with present ship monitoring system without additional sensors. By analyzing this data having high correlation coefficient(CC), correlation level of interactive data can be understood. Knowledge base of abnormal detection can be built by referring level of CC(Fault Detection CC, FDCC) to detect abnormal data among monitored data from monitoring system and knowledge base of diagnosis built by referring CC among interactive data for related machine each other to diagnose fault part.

  • PDF