• Title/Summary/Keyword: 이상치 탐지

Search Result 148, Processing Time 0.035 seconds

A Design of SMS DDoS Detection and Defense Method using Counting Bloom Filter (Counting Bloom Filter를 이용한 SMS DDoS 탐지 및 방어 기법 설계)

  • Shin, Kwang-Kyoon;Park, Ui-Chung;Jun, Moon-Seog
    • Proceedings of the KAIS Fall Conference
    • /
    • 2011.05a
    • /
    • pp.53-56
    • /
    • 2011
  • 지난 7.7 DDoS(Distributed Denial of Service), 3.3 DDoS 대란을 통해서 보여주듯 DDoS 공격이 네트워크 주요 위협요소로 매우 부각되고 있으나, 공격에 대해서 실시간으로 감지하고 대응하기에 어렵다. 그리고 현재 여러 분야에서 매우 많은 용도로 사용되는 SMS(Short Message Service)도 DDoS 공격 수단으로 사용되어 이동전화 시스템에 큰 혼란을 야기할 수 있다. 기존의 Bloom Filter 탐지 기법은 구조가 간단하고 실시간 탐지가 가능한 장점을 갖지만 오탐지율에 대한 문제점을 가진다. 본 논문에서는 목적지 기반의 다중의 해시함수를 사용한 Counting Bloom Filter 기법을 이용하여 임계치 이상 카운트된 동일한 목적지로 발송되는 SMS에 대하여 공격으로 탐지하고 SMSC에 통보하여 차단시키는 시스템을 제안한다.

  • PDF

Detection of Anomaly VMS Messages Using Bi-Directional GPT Networks (양방향 GPT 네트워크를 이용한 VMS 메시지 이상 탐지)

  • Choi, Hyo Rim;Park, Seungyoung
    • The Journal of The Korea Institute of Intelligent Transport Systems
    • /
    • v.21 no.4
    • /
    • pp.125-144
    • /
    • 2022
  • When a variable message signs (VMS) system displays false information related to traffic safety caused by malicious attacks, it could pose a serious risk to drivers. If the normal message patterns displayed on the VMS system are learned, it would be possible to detect and respond to the anomalous messages quickly. This paper proposes a method for detecting anomalous messages by learning the normal patterns of messages using a bi-directional generative pre-trained transformer (GPT) network. In particular, the proposed method was trained using the normal messages and their system parameters to minimize the corresponding negative log-likelihood (NLL) values. After adequate training, the proposed method could detect an anomalous message when its NLL value was larger than a pre-specified threshold value. The experiment results showed that the proposed method could detect malicious messages and cases when the system error occurs.

Deep Learning-based Time Series Data Prediction Research for Performance Enhancement in Cloud Monitoring Systems (클라우드 모니터링 시스템의 성능 향상을 위한 딥러닝을 이용한 시계열 데이터 예측 연구)

  • 김동완;홍두표;신용태
    • Proceedings of the Korea Information Processing Society Conference
    • /
    • 2023.05a
    • /
    • pp.342-344
    • /
    • 2023
  • 클라우드 시장의 성장과 마이크로 서비스 접근식이 제기됨에 따라 IT인프라를 관리하기 위한 연구가 최근 활발히 이루어지고 있다. 하지만 고도화 및 분산된 환경에서 관찰 가능성 응용을 확보하기 어렵다는 문제점을 가지고 있다. 따라서 본 연구에서는 모니터링 시스템을 통한 데이터 분석 중 수집한 데이터의 분석이 난해하다는 문제를 해결하기 위한 방법을 제안한다. 제안된 방법은 NAB 데이터셋을 대상으로 STUMPY를 이용하여 데이터를 시각화하고, CNN을 이용하여 분류 작업을 수행한다. 분류를 수행한 데이터셋은 이상치 데이터와 이상 전조 데이터, 정상 데이터셋으로 분류하여 데이터셋을 구성한다. 구성한 학습 데이터셋에 대해 훈련을 마친 딥러닝 모델은 부하 테스트 환경에서 수집한 데이터에 대한 그래프 패턴을 분석하여 이상치 데이터와 이상 전조 데이터를 탐지한다.

An Anomalous Sequence Detection Method Based on An Extended LSTM Autoencoder (확장된 LSTM 오토인코더 기반 이상 시퀀스 탐지 기법)

  • Lee, Jooyeon;Lee, Ki Yong
    • The Journal of Society for e-Business Studies
    • /
    • v.26 no.1
    • /
    • pp.127-140
    • /
    • 2021
  • Recently, sequence data containing time information, such as sensor measurement data and purchase history, has been generated in various applications. So far, many methods for finding sequences that are significantly different from other sequences among given sequences have been proposed. However, most of them have a limitation that they consider only the order of elements in the sequences. Therefore, in this paper, we propose a new anomalous sequence detection method that considers both the order of elements and the time interval between elements. The proposed method uses an extended LSTM autoencoder model, which has an additional layer that converts a sequence into a form that can help effectively learn both the order of elements and the time interval between elements. The proposed method learns the features of the given sequences with the extended LSTM autoencoder model, and then detects sequences that the model does not reconstruct well as anomalous sequences. Using experiments on synthetic data that contains both normal and anomalous sequences, we show that the proposed method achieves an accuracy close to 100% compared to the method that uses only the traditional LSTM autoencoder.

A Study on the Fraud Detection for Electronic Prepayment using Machine Learning (머신러닝을 이용한 선불전자지급수단의 이상금융거래 탐지 연구)

  • Choi, Byung-Ho;Cho, Nam-Wook
    • The Journal of Society for e-Business Studies
    • /
    • v.27 no.2
    • /
    • pp.65-77
    • /
    • 2022
  • Due to the recent development in electronic financial services, transactions of electronic prepayment are rapidly growing, leading to growing fraud attempts. This paper proposes a methodology that can effectively detect fraud transactions in electronic prepayment by machine learning algorithms, including support vector machines, decision trees, and artificial neural networks. Actual transaction data of electronic prepayment services were collected and preprocessed to extract the most relevant variables from raw data. Two different approaches were explored in the paper. One is a transaction-based approach, and the other is a user ID-based approach. For the transaction-based approach, the first model is primarily based on raw data features, while the second model uses extra features in addition to the first model. The user ID-based approach also used feature engineering to extract and transform the most relevant features. Overall, the user ID-based approach showed a better performance than the transaction-based approach, where the artificial neural networks showed the best performance. The proposed method could be used to reduce the damage caused by financial accidents by detecting and blocking fraud attempts.

A Study on Effective Interpretation of AI Model based on Reference (Reference 기반 AI 모델의 효과적인 해석에 관한 연구)

  • Hyun-woo Lee;Tae-hyun Han;Yeong-ji Park;Tae-jin Lee
    • Journal of the Korea Institute of Information Security & Cryptology
    • /
    • v.33 no.3
    • /
    • pp.411-425
    • /
    • 2023
  • Today, AI (Artificial Intelligence) technology is widely used in various fields, performing classification and regression tasks according to the purpose of use, and research is also actively progressing. Especially in the field of security, unexpected threats need to be detected, and unsupervised learning-based anomaly detection techniques that can detect threats without adding known threat information to the model training process are promising methods. However, most of the preceding studies that provide interpretability for AI judgments are designed for supervised learning, so it is difficult to apply them to unsupervised learning models with fundamentally different learning methods. In addition, previously researched vision-centered AI mechanism interpretation studies are not suitable for application to the security field that is not expressed in images. Therefore, In this paper, we use a technique that provides interpretability for detected anomalies by searching for and comparing optimization references, which are the source of intrusion attacks. In this paper, based on reference, we propose additional logic to search for data closest to real data. Based on real data, it aims to provide a more intuitive interpretation of anomalies and to promote effective use of an anomaly detection model in the security field.

A Study on the Application of Outlier Analysis for Fraud Detection: Focused on Transactions of Auction Exception Agricultural Products (부정 탐지를 위한 이상치 분석 활용방안 연구 : 농수산 상장예외품목 거래를 대상으로)

  • Kim, Dongsung;Kim, Kitae;Kim, Jongwoo;Park, Steve
    • Journal of Intelligence and Information Systems
    • /
    • v.20 no.3
    • /
    • pp.93-108
    • /
    • 2014
  • To support business decision making, interests and efforts to analyze and use transaction data in different perspectives are increasing. Such efforts are not only limited to customer management or marketing, but also used for monitoring and detecting fraud transactions. Fraud transactions are evolving into various patterns by taking advantage of information technology. To reflect the evolution of fraud transactions, there are many efforts on fraud detection methods and advanced application systems in order to improve the accuracy and ease of fraud detection. As a case of fraud detection, this study aims to provide effective fraud detection methods for auction exception agricultural products in the largest Korean agricultural wholesale market. Auction exception products policy exists to complement auction-based trades in agricultural wholesale market. That is, most trades on agricultural products are performed by auction; however, specific products are assigned as auction exception products when total volumes of products are relatively small, the number of wholesalers is small, or there are difficulties for wholesalers to purchase the products. However, auction exception products policy makes several problems on fairness and transparency of transaction, which requires help of fraud detection. In this study, to generate fraud detection rules, real huge agricultural products trade transaction data from 2008 to 2010 in the market are analyzed, which increase more than 1 million transactions and 1 billion US dollar in transaction volume. Agricultural transaction data has unique characteristics such as frequent changes in supply volumes and turbulent time-dependent changes in price. Since this was the first trial to identify fraud transactions in this domain, there was no training data set for supervised learning. So, fraud detection rules are generated using outlier detection approach. We assume that outlier transactions have more possibility of fraud transactions than normal transactions. The outlier transactions are identified to compare daily average unit price, weekly average unit price, and quarterly average unit price of product items. Also quarterly averages unit price of product items of the specific wholesalers are used to identify outlier transactions. The reliability of generated fraud detection rules are confirmed by domain experts. To determine whether a transaction is fraudulent or not, normal distribution and normalized Z-value concept are applied. That is, a unit price of a transaction is transformed to Z-value to calculate the occurrence probability when we approximate the distribution of unit prices to normal distribution. The modified Z-value of the unit price in the transaction is used rather than using the original Z-value of it. The reason is that in the case of auction exception agricultural products, Z-values are influenced by outlier fraud transactions themselves because the number of wholesalers is small. The modified Z-values are called Self-Eliminated Z-scores because they are calculated excluding the unit price of the specific transaction which is subject to check whether it is fraud transaction or not. To show the usefulness of the proposed approach, a prototype of fraud transaction detection system is developed using Delphi. The system consists of five main menus and related submenus. First functionalities of the system is to import transaction databases. Next important functions are to set up fraud detection parameters. By changing fraud detection parameters, system users can control the number of potential fraud transactions. Execution functions provide fraud detection results which are found based on fraud detection parameters. The potential fraud transactions can be viewed on screen or exported as files. The study is an initial trial to identify fraud transactions in Auction Exception Agricultural Products. There are still many remained research topics of the issue. First, the scope of analysis data was limited due to the availability of data. It is necessary to include more data on transactions, wholesalers, and producers to detect fraud transactions more accurately. Next, we need to extend the scope of fraud transaction detection to fishery products. Also there are many possibilities to apply different data mining techniques for fraud detection. For example, time series approach is a potential technique to apply the problem. Even though outlier transactions are detected based on unit prices of transactions, however it is possible to derive fraud detection rules based on transaction volumes.

Outlier Detection Techniques for Biased Opinion Discovery (편향된 의견 문서 검출을 위한 이상치 탐지 기법)

  • Yeon, Jongheum;Shim, Junho;Lee, Sanggoo
    • The Journal of Society for e-Business Studies
    • /
    • v.18 no.4
    • /
    • pp.315-326
    • /
    • 2013
  • Users in social media post various types of opinions such as product reviews and movie reviews. It is a common trend that customers get assistance from the opinions in making their decisions. However, as opinion usage grows, distorted feedbacks also have increased. For example, exaggerated positive opinions are posted for promoting target products. So are negative opinions which are far from common evaluations. Finding these biased opinions becomes important to keep social media reliable. Techniques of opinion mining (or sentiment analysis) have been developed to determine sentiment polarity of opinionated documents. These techniques can be utilized for finding the biased opinions. However, the previous techniques have some drawback. They categorize the text into only positive and negative, and they also need a large amount of training data to build the classifier. In this paper, we propose methods for discovering the biased opinions which are skewed from the overall common opinions. The methods are based on angle based outlier detection and personalized PageRank, which can be applied without training data. We analyze the performance of the proposed techniques by presenting experimental results on a movie review dataset.

Development of a Stock Data Monitoring System Using the Isolation Forest Algorithm (Isolation Forest 알고리즘을 활용한 증권 데이터 모니터링 시스템 개발)

  • Woo-Yong An;Hong-Jip Kim;Jung-Yeon Kim;Seung-Hyun Seo
    • Proceedings of the Korea Information Processing Society Conference
    • /
    • 2024.05a
    • /
    • pp.488-489
    • /
    • 2024
  • 변동성이 심한 증권 데이터의 특성 상 데이터의 다양한 요소에서 장애 상황이 발생한다. 따라서 실시간 대용량 데이터 처리 과정에서 발생할 수 있는 다양한 서비스 장애 요인들을 식별하고, 이를 신속하게 대응하기 위한 효율적인 실시간 모니터링 시스템 구축이 필요하다. 본 연구는 국내 증권사로 송신되는 해외 선물옵션 및 주식 데이터를 이상치 탐지 알고리즘인 Isolation Forest 를 통해 데이터의 이상치를 판단하고 알림 신호를 발생시키는 시스템을 제안한다.

A Non-annotated Recurrent Neural Network Ensemble-based Model for Near-real Time Detection of Erroneous Sea Level Anomaly in Coastal Tide Gauge Observation (비주석 재귀신경망 앙상블 모델을 기반으로 한 조위관측소 해수위의 준실시간 이상값 탐지)

  • LEE, EUN-JOO;KIM, YOUNG-TAEG;KIM, SONG-HAK;JU, HO-JEONG;PARK, JAE-HUN
    • The Sea:JOURNAL OF THE KOREAN SOCIETY OF OCEANOGRAPHY
    • /
    • v.26 no.4
    • /
    • pp.307-326
    • /
    • 2021
  • Real-time sea level observations from tide gauges include missing and erroneous values. Classification as abnormal values can be done for the latter by the quality control procedure. Although the 3𝜎 (three standard deviations) rule has been applied in general to eliminate them, it is difficult to apply it to the sea-level data where extreme values can exist due to weather events, etc., or where erroneous values can exist even within the 3𝜎 range. An artificial intelligence model set designed in this study consists of non-annotated recurrent neural networks and ensemble techniques that do not require pre-labeling of the abnormal values. The developed model can identify an erroneous value less than 20 minutes of tide gauge recording an abnormal sea level. The validated model well separates normal and abnormal values during normal times and weather events. It was also confirmed that abnormal values can be detected even in the period of years when the sea level data have not been used for training. The artificial neural network algorithm utilized in this study is not limited to the coastal sea level, and hence it can be extended to the detection model of erroneous values in various oceanic and atmospheric data.