• Title/Summary/Keyword: Anomaly detection

Search Result 651, Processing Time 0.031 seconds

SAD : Web Session Anomaly Detection based on Bayesian Estimation (베이지언 추정을 이용한 웹 서비스 공격 탐지)

  • 조상현;김한성;이병희;차성덕
    • Journal of the Korea Institute of Information Security & Cryptology
    • /
    • v.13 no.2
    • /
    • pp.115-125
    • /
    • 2003
  • As Web services are generally open for external uses and not filtered by Firewall, these result in attacker's target. Web attacks which exploit vulnerable web-applications and malicious users' requests cause economical and social problems. In this paper, we are modelling general web service usages based on user-web-session and detect anomal usages with Bayesian estimation method. Finally we propose SAD(Session Anomaly Detection) for detection unknown web attacks. To evaluate SAD, we made an experiment on attack simulation with web vulnerability scanner, whisker. The results show that the detection rate of SAD is over 90%, which is influenced by several features such as size of window or training set, detection filter method and web topology.

Anomaly Detection from Hyperspectral Imagery using Transform-based Feature Selection and Local Spatial Auto-correlation Index (자료 변환 기반 특징 선택과 국소적 자기상관 지수를 이용한 초분광 영상의 이상값 탐지)

  • Park, No-Wook;Yoo, Hee-Young;Shin, Jung-Il;Lee, Kyu-Sung
    • Korean Journal of Remote Sensing
    • /
    • v.28 no.4
    • /
    • pp.357-367
    • /
    • 2012
  • This paper presents a two-stage methodology for anomaly detection from hyperspectral imagery that consists of transform-based feature extraction and selection, and computation of a local spatial auto-correlation statistic. First, principal component transform and 3D wavelet transform are applied to reduce redundant spectral information from hyperspectral imagery. Then feature selection based on global skewness and the portion of highly skewed sub-areas is followed to find optimal features for anomaly detection. Finally, a local indicator of spatial association (LISA) statistic is computed to account for both spectral and spatial information unlike traditional anomaly detection methodology based only on spectral information. An experiment using airborne CASI imagery is carried out to illustrate the applicability of the proposed anomaly detection methodology. From the experiments, anomaly detection based on the LISA statistic linked with the selection of optimal features outperformed both the traditional RX detector which uses only spectral information, and the case using major principal components with large eigen-values. The combination of low- and high-frequency components by 3D wavelet transform showed the best detection capability, compared with the case using optimal features selected from principal components.

Development of an Anomaly Detection Algorithm for Verification of Radionuclide Analysis Based on Artificial Intelligence in Radioactive Wastes (방사성폐기물 핵종분석 검증용 이상 탐지를 위한 인공지능 기반 알고리즘 개발)

  • Seungsoo Jang;Jang Hee Lee;Young-su Kim;Jiseok Kim;Jeen-hyeng Kwon;Song Hyun Kim
    • Journal of Radiation Industry
    • /
    • v.17 no.1
    • /
    • pp.19-32
    • /
    • 2023
  • The amount of radioactive waste is expected to dramatically increase with decommissioning of nuclear power plants such as Kori-1, the first nuclear power plant in South Korea. Accurate nuclide analysis is necessary to manage the radioactive wastes safely, but research on verification of radionuclide analysis has yet to be well established. This study aimed to develop the technology that can verify the results of radionuclide analysis based on artificial intelligence. In this study, we propose an anomaly detection algorithm for inspecting the analysis error of radionuclide. We used the data from 'Updated Scaling Factors in Low-Level Radwaste' (NP-5077) published by EPRI (Electric Power Research Institute), and resampling was performed using SMOTE (Synthetic Minority Oversampling Technique) algorithm to augment data. 149,676 augmented data with SMOTE algorithm was used to train the artificial neural networks (classification and anomaly detection networks). 324 NP-5077 report data verified the performance of networks. The anomaly detection algorithm of radionuclide analysis was divided into two modules that detect a case where radioactive waste was incorrectly classified or discriminate an abnormal data such as loss of data or incorrectly written data. The classification network was constructed using the fully connected layer, and the anomaly detection network was composed of the encoder and decoder. The latter was operated by loading the latent vector from the end layer of the classification network. This study conducted exploratory data analysis (i.e., statistics, histogram, correlation, covariance, PCA, k-mean clustering, DBSCAN). As a result of analyzing the data, it is complicated to distinguish the type of radioactive waste because data distribution overlapped each other. In spite of these complexities, our algorithm based on deep learning can distinguish abnormal data from normal data. Radionuclide analysis was verified using our anomaly detection algorithm, and meaningful results were obtained.

Drone based Magnetic Anomaly Detection to detect Ferromagnetic Target (강자성 표적 탐지를 위한 드론 기반 자기 이상 탐지)

  • Sin Hyuk Yim;Dongkyu Kim;Ji Hun Yoon;Bona Kim;Eun Seok Bang;Kyu Min Shim;Sangkyung Lee;Jong-shick Oh
    • Journal of the Korea Institute of Military Science and Technology
    • /
    • v.26 no.4
    • /
    • pp.335-343
    • /
    • 2023
  • Drone based Magnetic Anomaly Detection measure a magnetic anomaly signal from the ferromagnetic target on the ground. We conduct a magnetic anomaly detection with 9 ferromagnetic targets on the ground. By removing the magnetic field measured in the absence of ferromagnetic targets from the experimental value, the magnetic anomaly signal is clearly measured at an altitude of 100 m. We analyze the signal characteristics by the ferromagnetic target through simulation using COMSOL multiphysics. The simulation results are within the GPS error range of the experimental results.

An Anomaly Detection Method for the Security of VANETs (VANETs의 보안을 위한 비정상 행위 탐지 방법)

  • Oh, Sun-Jin
    • The Journal of the Institute of Internet, Broadcasting and Communication
    • /
    • v.10 no.2
    • /
    • pp.77-83
    • /
    • 2010
  • Vehicular Ad Hoc Networks are self-organizing Peer-to-Peer networks that typically have highly mobile vehicle nodes, moving at high speeds, very short-lasting and unstable communication links. VANETs are formed without fixed infrastructure, central administration, and dedicated routing equipment, and network nodes are mobile, joining and leaving the network over time. So, VANET-security is very vulnerable for the intrusion of malicious and misbehaving nodes in the network, since VANETs are mostly open networks, allowing everyone connect, without centralized control. In this paper, we propose a rough set based anomaly detection method that efficiently identify malicious behavior of vehicle node activities in these VANETs, and the performance of a proposed scheme is evaluated by a simulation in terms of anomaly detection rate and false alarm rate for the threshold ${\epsilon}$.

Imbalanced SVM-Based Anomaly Detection Algorithm for Imbalanced Training Datasets

  • Wang, GuiPing;Yang, JianXi;Li, Ren
    • ETRI Journal
    • /
    • v.39 no.5
    • /
    • pp.621-631
    • /
    • 2017
  • Abnormal samples are usually difficult to obtain in production systems, resulting in imbalanced training sample sets. Namely, the number of positive samples is far less than the number of negative samples. Traditional Support Vector Machine (SVM)-based anomaly detection algorithms perform poorly for highly imbalanced datasets: the learned classification hyperplane skews toward the positive samples, resulting in a high false-negative rate. This article proposes a new imbalanced SVM (termed ImSVM)-based anomaly detection algorithm, which assigns a different weight for each positive support vector in the decision function. ImSVM adjusts the learned classification hyperplane to make the decision function achieve a maximum GMean measure value on the dataset. The above problem is converted into an unconstrained optimization problem to search the optimal weight vector. Experiments are carried out on both Cloud datasets and Knowledge Discovery and Data Mining datasets to evaluate ImSVM. Highly imbalanced training sample sets are constructed. The experimental results show that ImSVM outperforms over-sampling techniques and several existing imbalanced SVM-based techniques.

Network Anomaly Detection based on Association among Packets (패킷간 연관 관계를 이용한 네트워크 비정상행위 탐지)

  • 오상현;이원석
    • Journal of the Korea Institute of Information Security & Cryptology
    • /
    • v.12 no.5
    • /
    • pp.63-73
    • /
    • 2002
  • Recently, intrusions into a computer have been increased rapidly and also various intrusion methods have been developed. As a result. many researches have been performed to detect the activities of intruders effectively In this paper, a new association mining algorithm for anomaly network intrusion detection is proposed. For this purpose, the proposed algorithm is composed of two different phases: intra-packet association and inter-packet association. The performance of the proposed anomaly detection system is evaluated based on several experiment according to various system parameters in order to identify their practical ranges for maximizing its detection rate. As a result, an anomaly can be detected effectively.

Using machine learning for anomaly detection on a system-on-chip under gamma radiation

  • Eduardo Weber Wachter ;Server Kasap ;Sefki Kolozali ;Xiaojun Zhai ;Shoaib Ehsan;Klaus D. McDonald-Maier
    • Nuclear Engineering and Technology
    • /
    • v.54 no.11
    • /
    • pp.3985-3995
    • /
    • 2022
  • The emergence of new nanoscale technologies has imposed significant challenges to designing reliable electronic systems in radiation environments. A few types of radiation like Total Ionizing Dose (TID) can cause permanent damages on such nanoscale electronic devices, and current state-of-the-art technologies to tackle TID make use of expensive radiation-hardened devices. This paper focuses on a novel and different approach: using machine learning algorithms on consumer electronic level Field Programmable Gate Arrays (FPGAs) to tackle TID effects and monitor them to replace before they stop working. This condition has a research challenge to anticipate when the board results in a total failure due to TID effects. We observed internal measurements of FPGA boards under gamma radiation and used three different anomaly detection machine learning (ML) algorithms to detect anomalies in the sensor measurements in a gamma-radiated environment. The statistical results show a highly significant relationship between the gamma radiation exposure levels and the board measurements. Moreover, our anomaly detection results have shown that a One-Class SVM with Radial Basis Function Kernel has an average recall score of 0.95. Also, all anomalies can be detected before the boards are entirely inoperative, i.e. voltages drop to zero and confirmed with a sanity check.

Anomaly detection of smart metering system for power management with battery storage system/electric vehicle

  • Sangkeum Lee;Sarvar Hussain Nengroo;Hojun Jin;Yoonmee Doh;Chungho Lee;Taewook Heo;Dongsoo Har
    • ETRI Journal
    • /
    • v.45 no.4
    • /
    • pp.650-665
    • /
    • 2023
  • A novel smart metering technique capable of anomaly detection was proposed for real-time home power management system. Smart meter data generated in real-time were obtained from 900 households of single apartments. To detect outliers and missing values in smart meter data, a deep learning model, the autoencoder, consisting of a graph convolutional network and bidirectional long short-term memory network, was applied to the smart metering technique. Power management based on the smart metering technique was executed by multi-objective optimization in the presence of a battery storage system and an electric vehicle. The results of the power management employing the proposed smart metering technique indicate a reduction in electricity cost and amount of power supplied by the grid compared to the results of power management without anomaly detection.

Anomalous Pattern Analysis of Large-Scale Logs with Spark Cluster Environment

  • Sion Min;Youyang Kim;Byungchul Tak
    • Journal of the Korea Society of Computer and Information
    • /
    • v.29 no.3
    • /
    • pp.127-136
    • /
    • 2024
  • This study explores the correlation between system anomalies and large-scale logs within the Spark cluster environment. While research on anomaly detection using logs is growing, there remains a limitation in adequately leveraging logs from various components of the cluster and considering the relationship between anomalies and the system. Therefore, this paper analyzes the distribution of normal and abnormal logs and explores the potential for anomaly detection based on the occurrence of log templates. By employing Hadoop and Spark, normal and abnormal log data are generated, and through t-SNE and K-means clustering, templates of abnormal logs in anomalous situations are identified to comprehend anomalies. Ultimately, unique log templates occurring only during abnormal situations are identified, thereby presenting the potential for anomaly detection.