• 제목/요약/키워드: Anomaly data detection

검색결과 380건 처리시간 0.019초

건축공간 환경관리 지원을 위한 AI·IoT 기반 이상패턴 검출에 관한 연구 (A Study on Detection of Abnormal Patterns Based on AI·IoT to Support Environmental Management of Architectural Spaces)

  • 강태욱
    • 한국BIM학회 논문집
    • /
    • 제13권3호
    • /
    • pp.12-20
    • /
    • 2023
  • Deep learning-based anomaly detection technology is used in various fields such as computer vision, speech recognition, and natural language processing. In particular, this technology is applied in various fields such as monitoring manufacturing equipment abnormalities, detecting financial fraud, detecting network hacking, and detecting anomalies in medical images. However, in the field of construction and architecture, research on deep learning-based data anomaly detection technology is difficult due to the lack of digitization of domain knowledge due to late digital conversion, lack of learning data, and difficulties in collecting and processing field data in real time. This study acquires necessary data through IoT (Internet of Things) from the viewpoint of monitoring for environmental management of architectural spaces, converts them into a database, learns deep learning, and then supports anomaly patterns using AI (Artificial Infelligence) deep learning-based anomaly detection. We propose an implementation process. The results of this study suggest an effective environmental anomaly pattern detection solution architecture for environmental management of architectural spaces, proving its feasibility. The proposed method enables quick response through real-time data processing and analysis collected from IoT. In order to confirm the effectiveness of the proposed method, performance analysis is performed through prototype implementation to derive the results.

Tropospheric Anomaly Detection in Multi-Reference Stations Environment during Localized Atmospheric Conditions-(2) : Analytic Results of Anomaly Detection Algorithm

  • Yoo, Yun-Ja
    • 한국항해항만학회지
    • /
    • 제40권5호
    • /
    • pp.271-278
    • /
    • 2016
  • Localized atmospheric conditions between multi-reference stations can bring the tropospheric delay irregularity that becomes an error terms affecting positioning accuracy in network RTK environment. Imbalanced network error can affect the network solutions and it can corrupt the entire network solution and degrade the correction accuracy. If an anomaly could be detected before the correction message was generated, it is possible to eliminate the anomalous satellite that can cause degradation of the network solution during the tropospheric delay anomaly. An atmospheric grid that consists of four meteorological stations was used to detect an inhomogeneous weather conditions and tropospheric anomaly applied AWSs (automatic weather stations) meteorological data. The threshold of anomaly detection algorithm was determined based on the statistical weather data of AWSs for 5 years in an atmospheric grid. From the analytic results of anomaly detection algorithm it showed that the proposed algorithm can detect an anomalous satellite with an anomaly flag generation caused tropospheric delay anomaly during localized atmospheric conditions between stations. It was shown that the different precipitation condition between stations is the main factor affecting tropospheric anomalies.

Detection of multi-type data anomaly for structural health monitoring using pattern recognition neural network

  • Gao, Ke;Chen, Zhi-Dan;Weng, Shun;Zhu, Hong-Ping;Wu, Li-Ying
    • Smart Structures and Systems
    • /
    • 제29권1호
    • /
    • pp.129-140
    • /
    • 2022
  • The effectiveness of system identification, damage detection, condition assessment and other structural analyses relies heavily on the accuracy and reliability of the measured data in structural health monitoring (SHM) systems. However, data anomalies often occur in SHM systems, leading to inaccurate and untrustworthy analysis results. Therefore, anomalies in the raw data should be detected and cleansed before further analysis. Previous studies on data anomaly detection mainly focused on just single type of data anomaly for denoising or removing outliers, meanwhile, the existing methods of detecting multiple data anomalies are usually time consuming. For these reasons, recognising multiple anomaly patterns for real-time alarm and analysis in field monitoring remains a challenge. Aiming to achieve an efficient and accurate detection for multi-type data anomalies for field SHM, this study proposes a pattern-recognition-based data anomaly detection method that mainly consists of three steps: the feature extraction from the long time-series data samples, the training of a pattern recognition neural network (PRNN) using the features and finally the detection of data anomalies. The feature extraction step remarkably reduces the time cost of the network training, making the detection process very fast. The performance of the proposed method is verified on the basis of the SHM data of two practical long-span bridges. Results indicate that the proposed method recognises multiple data anomalies with very high accuracy and low calculation cost, demonstrating its applicability in field monitoring.

TadGAN 기반 시계열 이상 탐지를 활용한 전처리 프로세스 연구 (A Pre-processing Process Using TadGAN-based Time-series Anomaly Detection)

  • 이승훈;김용수
    • 품질경영학회지
    • /
    • 제50권3호
    • /
    • pp.459-471
    • /
    • 2022
  • Purpose: The purpose of this study was to increase prediction accuracy for an anomaly interval identified using an artificial intelligence-based time series anomaly detection technique by establishing a pre-processing process. Methods: Significant variables were extracted by applying feature selection techniques, and anomalies were derived using the TadGAN time series anomaly detection algorithm. After applying machine learning and deep learning methodologies using normal section data (excluding anomaly sections), the explanatory power of the anomaly sections was demonstrated through performance comparison. Results: The results of the machine learning methodology, the performance was the best when SHAP and TadGAN were applied, and the results in the deep learning, the performance was excellent when Chi-square Test and TadGAN were applied. Comparing each performance with the papers applied with a Conventional methodology using the same data, it can be seen that the performance of the MLR was significantly improved to 15%, Random Forest to 24%, XGBoost to 30%, Lasso Regression to 73%, LSTM to 17% and GRU to 19%. Conclusion: Based on the proposed process, when detecting unsupervised learning anomalies of data that are not actually labeled in various fields such as cyber security, financial sector, behavior pattern field, SNS. It is expected to prove the accuracy and explanation of the anomaly detection section and improve the performance of the model.

데이터마이닝 기법을 이용한 비정상행위 탐지 방법 연구 (Anomaly Detection Scheme Using Data Mining Methods)

  • 박광진;유황빈
    • 정보보호학회논문지
    • /
    • 제13권2호
    • /
    • pp.99-106
    • /
    • 2003
  • 네트워크 환경에서의 다양한 침입은 심각한 위험을 초래 할 수 있기 때문에 침입을 효과적으로 탐지하기 위해 데이터마이닝 기법을 발전시켜 왔다. 비정상행위 탐지 기술은 순수 데이터로 학습한 후, 비정상행위를 탐지하기 때문에 정교한 정상행위 패턴 생성이 필수적이다. 순수한 학습 데이터의 생성은 시간과 비용이 많이 드는 단점이 있다. 따라서 네트워크 상의 데이터에 대한 특징을 파악하는 것이 중요하다. 본 논문에서는 데이터마이닝의 연관규칙 및 클러스터링기법을 비정상행위 탐지에 적용하였고, 패킷내의 판정 요소에 정보이론 척도를 적용하여 불필요한 데이터를 필터링하는 방법을 제시하였다. 또한 가변길이 트랜잭션을 네트워크상의 분석 단위를 정의하는 기준으로 제시하여 행위 패턴 생성에 보다 묘사성이 높음을 보였다.

SVDD를 활용한 상업용 건물에너지 소비패턴의 이상현상 감지 (Anomaly Detection and Diagnostics (ADD) Based on Support Vector Data Description (SVDD) for Energy Consumption in Commercial Building)

  • 채영태
    • 한국건축친환경설비학회 논문집
    • /
    • 제12권6호
    • /
    • pp.579-590
    • /
    • 2018
  • Anomaly detection on building energy consumption has been regarded as an effective tool to reduce energy saving on building operation and maintenance. However, it requires energy model and FDD expert for quantitative model approach or large amount of training data for qualitative/history data approach. Both method needs additional time and labors. This study propose a machine learning and data science approach to define faulty conditions on hourly building energy consumption with reducing data amount and input requirement. It suggests an application of Support Vector Data Description (SVDD) method on training normal condition of hourly building energy consumption incorporated with hourly outdoor air temperature and time integer in a week, 168 data points and identifying hourly abnormal condition in the next day. The result shows the developed model has a better performance when the ${\nu}$ (probability of error in the training set) is 0.05 and ${\gamma}$ (radius of hyper plane) 0.2. The model accuracy to identify anomaly operation ranges from 70% (10% increase anomaly) to 95% (20% decrease anomaly) for daily total (24 hours) and from 80% (10% decrease anomaly) to 10%(15% increase anomaly) for occupied hours, respectively.

Anomaly Intrusion Detection Based on Hyper-ellipsoid in the Kernel Feature Space

  • Lee, Hansung;Moon, Daesung;Kim, Ikkyun;Jung, Hoseok;Park, Daihee
    • KSII Transactions on Internet and Information Systems (TIIS)
    • /
    • 제9권3호
    • /
    • pp.1173-1192
    • /
    • 2015
  • The Support Vector Data Description (SVDD) has achieved great success in anomaly detection, directly finding the optimal ball with a minimal radius and center, which contains most of the target data. The SVDD has some limited classification capability, because the hyper-sphere, even in feature space, can express only a limited region of the target class. This paper presents an anomaly detection algorithm for mitigating the limitations of the conventional SVDD by finding the minimum volume enclosing ellipsoid in the feature space. To evaluate the performance of the proposed approach, we tested it with intrusion detection applications. Experimental results show the prominence of the proposed approach for anomaly detection compared with the standard SVDD.

Tropospheric Anomaly Detection in Multi-reference Stations Environment during Localized Atmosphere Conditions-(1) : Basic Concept of Anomaly Detection Algorithm

  • Yoo, Yun-Ja
    • 한국항해항만학회지
    • /
    • 제40권5호
    • /
    • pp.265-270
    • /
    • 2016
  • Extreme tropospheric anomalies such as typhoons or regional torrential rain can degrade positioning accuracy of the GPS signal. It becomes one of the main error terms affecting high-precision positioning solutions in network RTK. This paper proposed a detection algorithm to be used during atmospheric anomalies in order to detect the tropospheric irregularities that can degrade the quality of correction data due to network errors caused by inhomogeneous atmospheric conditions between multi-reference stations. It uses an atmospheric grid that consists of four meteorological stations and estimates the troposphere zenith total delay difference at a low performance point in an atmospheric grid. AWS (automatic weather station) meteorological data can be applied to the proposed tropospheric anomaly detection algorithm when there are different atmospheric conditions between the stations. The concept of probability density distribution of the delta troposphere slant delay was proposed for the threshold determination.

나이브 베이지안과 데이터 마이닝을 이용한 FHIDS(Fuzzy Logic based Hybrid Intrusion Detection System) 설계 (A Design of FHIDS(Fuzzy logic based Hybrid Intrusion Detection System) using Naive Bayesian and Data Mining)

  • 이병관;정은희
    • 한국정보전자통신기술학회논문지
    • /
    • 제5권3호
    • /
    • pp.158-163
    • /
    • 2012
  • 본 논문에서 나이브 베이지안 알고리즘, 데이터 마이닝, Fuzzy logic을 이용하여 이상 공격과 오용 공격을 탐지하는 하이브리드 침입탐지시스템인 FHIDS(Fuzzy logic based Hybrid Intrusion Detection System)을 설계하였다. 본 논문에서 설계한 FHIDS의 NB-AAD(Naive Bayesian based Anomaly Attack Detection)기법은 나이브 베이지안 알고리즘을 이용해 이상 공격을 탐지하고, DM-MAD(Data Mining based Misuse Attack Detection)기법은 데이터 마이닝 알고리즘을 이용하여 패킷들의 연관 규칙을 분석하여 새로운 규칙기반 패턴을 생성하거나 변형된 규칙 기반 패턴을 추출함으로써, 새로운 공격이나 변형된 공격을 탐지한다. 그리고 FLD(Fuzzy Logic based Decision)은 NB-AAD과 DM-MAD의 결과를 이용하여 정상인지 공격인지를 판별한다. 즉, FHIDS는 이상과 오용공격을 탐지 가능하며 False Positive 비율을 감소시키고, 변형 공격 탐지율을 개선한 하이브리드 공격탐지시스템이다.

역방향 인덱스 기반의 저장소를 이용한 이상 탐지 분석 (Anomaly Detection Analysis using Repository based on Inverted Index)

  • 박주미;조위덕;김강석
    • 정보과학회 논문지
    • /
    • 제45권3호
    • /
    • pp.294-302
    • /
    • 2018
  • 정보통신 기술의 발전에 따른 새로운 서비스 산업의 출현으로 개인 정보 침해, 산업 기밀 유출 등 사이버 공간의 위험이 다양화 되어, 그에 따른 보안 문제가 중요한 이슈로 떠오르게 되었다. 본 연구에서는 기업 내 개인 정보 오남용 및 내부 정보 유출에 따른, 대용량 사용자 로그 데이터를 기반으로 기존의 시그니처(Signature) 보안 대응 방식에 비해, 실시간 및 대용량 데이터 분석기술에 적합한 행위 기반 이상 탐지방식을 제안하였다. 행위 기반 이상 탐지방식이 대용량 데이터를 처리하는 기술을 필요로 함에 따라, 역방향 인덱스(Inverted Index) 기반의 실시간 검색 엔진인 엘라스틱서치(Elasticsearch)를 사용하였다. 또한 데이터 분석을 위해 통계 기반의 빈도 분석과 전 처리 과정을 수행하였으며, 밀도 기반의 군집화 방법인 DBSCAN 알고리즘을 적용하여 이상 데이터를 분류하는 방법과 시각화를 통해 분석을 간편하게 하기위한 한 사례를 보였다. 이는 기존의 이상 탐지 시스템과 달리 임계값을 별도로 설정하지 않고 이상 탐지 분석을 시도하였다는 것과 통계적인 측면에서 이상 탐지 방식을 제안하였다는 것에 의의가 있다.