• Title/Summary/Keyword: Anomaly Data

Search Result 783, Processing Time 0.037 seconds

Robust transformer-based anomaly detection for nuclear power data using maximum correntropy criterion

  • Shuang Yi;Sheng Zheng;Senquan Yang;Guangrong Zhou;Junjie He
    • Nuclear Engineering and Technology
    • /
    • v.56 no.4
    • /
    • pp.1284-1295
    • /
    • 2024
  • Due to increasing operational security demands, digital and intelligent condition monitoring of nuclear power plants is becoming more significant. However, establishing an accurate and effective anomaly detection model is still challenging. This is mainly because of data characteristics of nuclear power data, including the lack of clear class labels combined with frequent interference from outliers and anomalies. In this paper, we introduce a Transformer-based unsupervised model for anomaly detection of nuclear power data, a modified loss function based on the maximum correntropy criterion (MCC) is applied in the model training to improve the robustness. Experimental results on simulation datasets demonstrate that the proposed Trans-MCC model achieves equivalent or superior detection performance to the baseline models, and the use of the MCC loss function is proven can obviously alleviate the negative effect of outliers and anomalies in the training procedure, the F1 score is improved by up to 0.31 compared to Trans-MSE on a specific dataset. Further studies on genuine nuclear power data have verified the model's capability to detect anomalies at an earlier stage, which is significant to condition monitoring.

Decision Tree Techniques with Feature Reduction for Network Anomaly Detection (네트워크 비정상 탐지를 위한 속성 축소를 반영한 의사결정나무 기술)

  • Kang, Koohong
    • Journal of the Korea Institute of Information Security & Cryptology
    • /
    • v.29 no.4
    • /
    • pp.795-805
    • /
    • 2019
  • Recently, there is a growing interest in network anomaly detection technology to tackle unknown attacks. For this purpose, diverse studies using data mining, machine learning, and deep learning have been applied to detect network anomalies. In this paper, we evaluate the decision tree to see its feasibility for network anomaly detection on NSL-KDD data set, which is one of the most popular data mining techniques for classification. In order to handle the over-fitting problem of decision tree, we select 13 features from the original 41 features of the data set using chi-square test, and then model the decision tree using TensorFlow and Scik-Learn, yielding 84% and 70% of binary classification accuracies on the KDDTest+ and KDDTest-21 of NSL-KDD test data set. This result shows 3% and 6% improvements compared to the previous 81% and 64% of binary classification accuracies by decision tree technologies, respectively.

Anomaly Detection Methodology Based on Multimodal Deep Learning (멀티모달 딥 러닝 기반 이상 상황 탐지 방법론)

  • Lee, DongHoon;Kim, Namgyu
    • Journal of Intelligence and Information Systems
    • /
    • v.28 no.2
    • /
    • pp.101-125
    • /
    • 2022
  • Recently, with the development of computing technology and the improvement of the cloud environment, deep learning technology has developed, and attempts to apply deep learning to various fields are increasing. A typical example is anomaly detection, which is a technique for identifying values or patterns that deviate from normal data. Among the representative types of anomaly detection, it is very difficult to detect a contextual anomaly that requires understanding of the overall situation. In general, detection of anomalies in image data is performed using a pre-trained model trained on large data. However, since this pre-trained model was created by focusing on object classification of images, there is a limit to be applied to anomaly detection that needs to understand complex situations created by various objects. Therefore, in this study, we newly propose a two-step pre-trained model for detecting abnormal situation. Our methodology performs additional learning from image captioning to understand not only mere objects but also the complicated situation created by them. Specifically, the proposed methodology transfers knowledge of the pre-trained model that has learned object classification with ImageNet data to the image captioning model, and uses the caption that describes the situation represented by the image. Afterwards, the weight obtained by learning the situational characteristics through images and captions is extracted and fine-tuning is performed to generate an anomaly detection model. To evaluate the performance of the proposed methodology, an anomaly detection experiment was performed on 400 situational images and the experimental results showed that the proposed methodology was superior in terms of anomaly detection accuracy and F1-score compared to the existing traditional pre-trained model.

Wavenumber correlation analysis of satellite magnetometer observations

  • Kim, Jeong-Woo;Kim, Won-Kyun;Kim, Hye-Yun
    • Proceedings of the KSEEG Conference
    • /
    • 2000.04b
    • /
    • pp.311-313
    • /
    • 2000
  • Identifying anomaly correlations between data sets is the basis for rationalizing geopotenial interpretation and theory. A procedure between the two or more geopotential data sets. Anomaly features that show direct, inverse, or no correlationsbetween the data may be separated by applying filters in the frequency domains of the data sets. The correlation filter passes or rejects wavenumbers between co-registered data sets based on the correlation coefficient between common wavenumbers as given by the cosine of their phase difference. This study includes as example of Magsat magnetic anomaly profile that illustrates the usefulness of the procedure for extracting correlative features between the sets.

  • PDF

Multi-sensor data-based anomaly detection and diagnosis of a pumped storage hydropower plant

  • Sojin Shin;Cheolgyu Hyun;Seongpil Cho;Phill-Seung Lee
    • Structural Engineering and Mechanics
    • /
    • v.88 no.6
    • /
    • pp.569-581
    • /
    • 2023
  • This paper introduces a system to detect and diagnose anomalies in pumped storage hydropower plants. We collect data from various types of sensors, including those monitoring temperature, vibration, and power. The data are classified according to the operation modes (pump and turbine operation modes) and normalized to remove the influence of the external environment. To detect anomalies and diagnose their types, we adopt a multivariate normal distribution analysis by learning the distribution of the normal data. The feasibility of the proposed system is evaluated using actual monitoring data of a pumped storage hydropower plant. The proposed system can be used to implement condition monitoring systems for other plants through modifications.

gMLP-based Self-Supervised Learning Anomaly Detection using a Simple Synthetic Data Generation Method (단순한 합성데이터 생성 방식을 활용한 gMLP 기반 자기 지도 학습 이상탐지 기법)

  • Ju-Hyo, Hwang;Kyo-Hong, Jin
    • Journal of the Korea Institute of Information and Communication Engineering
    • /
    • v.27 no.1
    • /
    • pp.8-14
    • /
    • 2023
  • The existing self-supervised learning-based CutPaste generated synthetic data by cutting and attaching specific patches from normal images and then performed anomaly detection. However, this method has a problem in that there is a clear difference in the boundary of the patch. NSA for solving these problems have achieved higher anomaly detection performance by generating natural synthetic data through Poisson Blending. However, NSA has the disadvantage of having many hyperparameters that need to be adjusted for each class. In this paper, synthetic data similar to normal were generated by a simple method of making the size of the synthetic patch very small. At this time, since the patches are so locally synthesized, models that learn local features can easily overfit synthetic data. Therefore, we performed anomaly detection using gMLP, which learns global features, and even with simple synthesis methods, we were able to achieve higher performance than conventional self-supervised learning techniques.

Anomaly Detection in Smart Homes Using Bayesian Networks

  • Saqaeeyan, Sasan;javadi, Hamid Haj Seyyed;Amirkhani, Hossein
    • KSII Transactions on Internet and Information Systems (TIIS)
    • /
    • v.14 no.4
    • /
    • pp.1796-1816
    • /
    • 2020
  • The health and safety of elderly and disabled patients who cannot live alone is an important issue. Timely detection of sudden events is necessary to protect these people, and anomaly detection in smart homes is an efficient approach to extracting such information. In the real world, there is a causal relationship between an occupant's behaviour and the order in which appliances are used in the home. Bayesian networks are appropriate tools for assessing the probability of an effect due to the occurrence of its causes, and vice versa. This paper defines different subsets of random variables on the basis of sensory data from a smart home, and it presents an anomaly detection system based on various models of Bayesian networks and drawing upon these variables. We examine different models to obtain the best network, one that has higher assessment scores and a smaller size. Experimental evaluations of real datasets show the effectiveness of the proposed method.

AN ANOMALY DETECTION METHOD BY ASSOCIATIVE CLASSIFICATION

  • Lee, Bum-Ju;Lee, Heon-Gyu;Ryu, Keun-Ho
    • Proceedings of the KSRS Conference
    • /
    • 2005.10a
    • /
    • pp.301-304
    • /
    • 2005
  • For detecting an intrusion based on the anomaly of a user's activities, previous works are concentrated on statistical techniques or frequent episode mining in order to analyze an audit data. But, since they mainly analyze the average behaviour of user's activities, some anomalies can be detected inaccurately. Therefore, we propose an anomaly detection method that utilizes an associative classification for modelling intrusion detection. Finally, we proof that a prediction model built from associative classification method yields better accuracy than a prediction model built from a traditional methods by experimental results.

  • PDF

Detection of 2002-2003 El Ni${\tilde{n}}$o Using EOS and OSMI Data

  • Lee, S.H.;Lim, H.S.;Kim, J.G.;Jun, J.N.
    • Proceedings of the KSRS Conference
    • /
    • 2003.11a
    • /
    • pp.1413-1414
    • /
    • 2003
  • Interannual variability in the patterns of satellitederived pigment concentrations, sea-level height anomaly, sea surface temperature anomaly, and zonal wind anomaly are observed during the 2002-2003 El Ni${\tilde{n}}$o. The largest spatial extent of the phytoplankton bloom was recovery from El Ni${\tilde{n}}$o over the equatorial Pacific. The evolution towards a warm episode (El Ni${\tilde{n}}$o) started from spring of 2002 and continued during January 2003, while equatorial Sea Surface Temperature Anomaly (SSTA) remained greater than +1$^{\circ}$C in the central equatorial Pacific. The EOS (Earth Observing System) and OSMI (Ocean Scanning Multispectral Imager) data are used for detection of dramatic changes in the patterns of pigment concentration during El Ni${\tilde{n}}$o.

  • PDF

FADA: A fuzzy anomaly detection algorithm for MANETs (모바일 애드-혹 망을 위한 퍼지 비정상 행위 탐지 알고리즘)

  • Bae, Ihn-Han
    • Journal of the Korean Data and Information Science Society
    • /
    • v.21 no.6
    • /
    • pp.1125-1136
    • /
    • 2010
  • Lately there exist increasing demands for online abnormality monitoring over trajectory stream, which are obtained from moving object tracking devices. This problem is challenging due to the requirement of high speed data processing within limited space cost. In this paper, we present a FADA (Fuzzy Anomaly Detection Algorithm) which constructs normal profile by computing mobility feature information from the GPS (Global Positioning System) logs of mobile devices in MANETs (Mobile Ad-hoc Networks), computes a fuzzy dissimilarity between the current mobility feature information of the mobile device and the mobility feature information in the normal profile, and detects effectively the anomaly behaviors of mobile devices on the basis of the computed fuzzy dissimilarity. The performance of proposed FADA is evaluated through simulation.