• 제목/요약/키워드: structural anomaly detection

검색결과 27건 처리시간 0.023초

Data abnormal detection using bidirectional long-short neural network combined with artificial experience

  • Yang, Kang;Jiang, Huachen;Ding, Youliang;Wang, Manya;Wan, Chunfeng
    • Smart Structures and Systems
    • /
    • 제29권1호
    • /
    • pp.117-127
    • /
    • 2022
  • Data anomalies seriously threaten the reliability of the bridge structural health monitoring system and may trigger system misjudgment. To overcome the above problem, an efficient and accurate data anomaly detection method is desiderated. Traditional anomaly detection methods extract various abnormal features as the key indicators to identify data anomalies. Then set thresholds artificially for various features to identify specific anomalies, which is the artificial experience method. However, limited by the poor generalization ability among sensors, this method often leads to high labor costs. Another approach to anomaly detection is a data-driven approach based on machine learning methods. Among these, the bidirectional long-short memory neural network (BiLSTM), as an effective classification method, excels at finding complex relationships in multivariate time series data. However, training unprocessed original signals often leads to low computation efficiency and poor convergence, for lacking appropriate feature selection. Therefore, this article combines the advantages of the two methods by proposing a deep learning method with manual experience statistical features fed into it. Experimental comparative studies illustrate that the BiLSTM model with appropriate feature input has an accuracy rate of over 87-94%. Meanwhile, this paper provides basic principles of data cleaning and discusses the typical features of various anomalies. Furthermore, the optimization strategies of the feature space selection based on artificial experience are also highlighted.

CNN based data anomaly detection using multi-channel imagery for structural health monitoring

  • Shajihan, Shaik Althaf V.;Wang, Shuo;Zhai, Guanghao;Spencer, Billie F. Jr.
    • Smart Structures and Systems
    • /
    • 제29권1호
    • /
    • pp.181-193
    • /
    • 2022
  • Data-driven structural health monitoring (SHM) of civil infrastructure can be used to continuously assess the state of a structure, allowing preemptive safety measures to be carried out. Long-term monitoring of large-scale civil infrastructure often involves data-collection using a network of numerous sensors of various types. Malfunctioning sensors in the network are common, which can disrupt the condition assessment and even lead to false-negative indications of damage. The overwhelming size of the data collected renders manual approaches to ensure data quality intractable. The task of detecting and classifying an anomaly in the raw data is non-trivial. We propose an approach to automate this task, improving upon the previously developed technique of image-based pre-processing on one-dimensional (1D) data by enriching the features of the neural network input data with multiple channels. In particular, feature engineering is employed to convert the measured time histories into a 3-channel image comprised of (i) the time history, (ii) the spectrogram, and (iii) the probability density function representation of the signal. To demonstrate this approach, a CNN model is designed and trained on a dataset consisting of acceleration records of sensors installed on a long-span bridge, with the goal of fault detection and classification. The effect of imbalance in anomaly patterns observed is studied to better account for unseen test cases. The proposed framework achieves high overall accuracy and recall even when tested on an unseen dataset that is much larger than the samples used for training, offering a viable solution for implementation on full-scale structures where limited labeled-training data is available.

Multi-sensor data-based anomaly detection and diagnosis of a pumped storage hydropower plant

  • Sojin Shin;Cheolgyu Hyun;Seongpil Cho;Phill-Seung Lee
    • Structural Engineering and Mechanics
    • /
    • 제88권6호
    • /
    • pp.569-581
    • /
    • 2023
  • This paper introduces a system to detect and diagnose anomalies in pumped storage hydropower plants. We collect data from various types of sensors, including those monitoring temperature, vibration, and power. The data are classified according to the operation modes (pump and turbine operation modes) and normalized to remove the influence of the external environment. To detect anomalies and diagnose their types, we adopt a multivariate normal distribution analysis by learning the distribution of the normal data. The feasibility of the proposed system is evaluated using actual monitoring data of a pumped storage hydropower plant. The proposed system can be used to implement condition monitoring systems for other plants through modifications.

로그 이상 탐지를 위한 도메인별 사전 훈련 언어 모델 중요성 연구 (On the Significance of Domain-Specific Pretrained Language Models for Log Anomaly Detection)

  • 레리사 아데바 질차;김득훈;곽진
    • 한국정보처리학회:학술대회논문집
    • /
    • 한국정보처리학회 2024년도 춘계학술발표대회
    • /
    • pp.337-340
    • /
    • 2024
  • Pretrained language models (PLMs) are extensively utilized to enhance the performance of log anomaly detection systems. Their effectiveness lies in their capacity to extract valuable semantic information from logs, thereby strengthening the detection performance. Nonetheless, challenges arise due to discrepancies in the distribution of log messages, hindering the development of robust and generalizable detection systems. This study investigates the structural and distributional variation across various log message datasets, underscoring the crucial role of domain-specific PLMs in overcoming the said challenge and devising robust and generalizable solutions.

Structural novelty detection based on sparse autoencoders and control charts

  • Finotti, Rafaelle P.;Gentile, Carmelo;Barbosa, Flavio;Cury, Alexandre
    • Structural Engineering and Mechanics
    • /
    • 제81권5호
    • /
    • pp.647-664
    • /
    • 2022
  • The powerful data mapping capability of computational deep learning methods has been recently explored in academic works to develop strategies for structural health monitoring through appropriate characterization of dynamic responses. In many cases, these studies concern laboratory prototypes and finite element models to validate the proposed methodologies. Therefore, the present work aims to investigate the capability of a deep learning algorithm called Sparse Autoencoder (SAE) specifically focused on detecting structural alterations in real-case studies. The idea is to characterize the dynamic responses via SAE models and, subsequently, to detect the onset of abnormal behavior through the Shewhart T control chart, calculated with SAE extracted features. The anomaly detection approach is exemplified using data from the Z24 bridge, a classical benchmark, and data from the continuous monitoring of the San Vittore bell-tower, Italy. In both cases, the influence of temperature is also evaluated. The proposed approach achieved good performance, detecting structural changes even under temperature variations.

A semi-supervised interpretable machine learning framework for sensor fault detection

  • Martakis, Panagiotis;Movsessian, Artur;Reuland, Yves;Pai, Sai G.S.;Quqa, Said;Cava, David Garcia;Tcherniak, Dmitri;Chatzi, Eleni
    • Smart Structures and Systems
    • /
    • 제29권1호
    • /
    • pp.251-266
    • /
    • 2022
  • Structural Health Monitoring (SHM) of critical infrastructure comprises a major pillar of maintenance management, shielding public safety and economic sustainability. Although SHM is usually associated with data-driven metrics and thresholds, expert judgement is essential, especially in cases where erroneous predictions can bear casualties or substantial economic loss. Considering that visual inspections are time consuming and potentially subjective, artificial-intelligence tools may be leveraged in order to minimize the inspection effort and provide objective outcomes. In this context, timely detection of sensor malfunctioning is crucial in preventing inaccurate assessment and false alarms. The present work introduces a sensor-fault detection and interpretation framework, based on the well-established support-vector machine scheme for anomaly detection, combined with a coalitional game-theory approach. The proposed framework is implemented in two datasets, provided along the 1st International Project Competition for Structural Health Monitoring (IPC-SHM 2020), comprising acceleration and cable-load measurements from two real cable-stayed bridges. The results demonstrate good predictive performance and highlight the potential for seamless adaption of the algorithm to intrinsically different data domains. For the first time, the term "decision trajectories", originating from the field of cognitive sciences, is introduced and applied in the context of SHM. This provides an intuitive and comprehensive illustration of the impact of individual features, along with an elaboration on feature dependencies that drive individual model predictions. Overall, the proposed framework provides an easy-to-train, application-agnostic and interpretable anomaly detector, which can be integrated into the preprocessing part of various SHM and condition-monitoring applications, offering a first screening of the sensor health prior to further analysis.

Mobile Ad - hoc Network에서 CP - SVM을 이용한 침입탐지 (Intrusion Detection Algorithm in Mobile Ad-hoc Network using CP-SVM)

  • 양환석
    • 디지털산업정보학회논문지
    • /
    • 제8권2호
    • /
    • pp.41-47
    • /
    • 2012
  • MANET has vulnerable structure on security owing to structural characteristics as follows. MANET consisted of moving nodes is that every nodes have to perform function of router. Every node has to provide reliable routing service in cooperation each other. These properties are caused by expose to various attacks. But, it is difficult that position of environment intrusion detection system is established, information is collected, and particularly attack is detected because of moving of nodes in MANET environment. It is not easy that important profile is constructed also. In this paper, conformal predictor - support vector machine(CP-SVM) based intrusion detection technique was proposed in order to do more accurate and efficient intrusion detection. In this study, IDS-agents calculate p value from collected packet and transmit to cluster head, and then other all cluster head have same value and detect abnormal behavior using the value. Cluster form of hierarchical structure was used to reduce consumption of nodes also. Effectiveness of proposed method was confirmed through experiment.

Deep learning-based anomaly detection in acceleration data of long-span cable-stayed bridges

  • Seungjun Lee;Jaebeom Lee;Minsun Kim;Sangmok Lee;Young-Joo Lee
    • Smart Structures and Systems
    • /
    • 제33권2호
    • /
    • pp.93-103
    • /
    • 2024
  • Despite the rapid development of sensors, structural health monitoring (SHM) still faces challenges in monitoring due to the degradation of devices and harsh environmental loads. These challenges can lead to measurement errors, missing data, or outliers, which can affect the accuracy and reliability of SHM systems. To address this problem, this study proposes a classification method that detects anomaly patterns in sensor data. The proposed classification method involves several steps. First, data scaling is conducted to adjust the scale of the raw data, which may have different magnitudes and ranges. This step ensures that the data is on the same scale, facilitating the comparison of data across different sensors. Next, informative features in the time and frequency domains are extracted and used as input for a deep neural network model. The model can effectively detect the most probable anomaly pattern, allowing for the timely identification of potential issues. To demonstrate the effectiveness of the proposed method, it was applied to actual data obtained from a long-span cable-stayed bridge in China. The results of the study have successfully verified the proposed method's applicability to practical SHM systems for civil infrastructures. The method has the potential to significantly enhance the safety and reliability of civil infrastructures by detecting potential issues and anomalies at an early stage.

MS 오피스 문서 파일 내 비정상 요소 탐지 기법 연구 (A Research of Anomaly Detection Method in MS Office Document)

  • 조성혜;이상진
    • 정보처리학회논문지:컴퓨터 및 통신 시스템
    • /
    • 제6권2호
    • /
    • pp.87-94
    • /
    • 2017
  • 최근 각종 공문서와 증빙 서류를 비롯하여 대부분의 문서가 디지털 데이터의 형태로 사용되고 있다. 특히 MS 오피스는 전 세계적으로 공공기관, 기업, 학교, 가정 등 다양한 곳에서 가장 많이 사용하고 있는 문서 편집 소프트웨어로써 악의적인 목적을 가진 사용자들이 해당 문서 프로그램의 범용성을 이용하여 MS 오피스 문서 파일을 악성 행위를 위한 매개체로 사용하고 있으며, 최근에는 단순한 사용자뿐만 아니라 국내외 정부 기관과 주요기업을 비롯하여 기반시설에서도 MS 오피스 문서 파일 형태의 악성코드가 유입되고 있다. MS 오피스 문서에 악성 코드를 삽입하는 방법은 단순히 미할당 영역에 은닉하는 방법을 사용할 뿐만 아니라 매크로 기능을 이용하는 등 다양한 방법을 통해 점점 정교한 형태로 진화되고 있다. 이러한 악성 코드들을 탐지하기 위해서 시그니처를 이용하거나 샌드박스를 이용한 탐지방법이 존재하지만, 유동적이고 복잡해지는 악성 코드들을 탐지하기에는 한계가 있다. 따라서 본 논문에서는 디지털 포렌식 관점에서 MS 오피스 문서 분석에 필요한 주요 메타데이터와 파일 포맷 구조 분석을 통해 매크로 영역과 그 외 악성 코드가 삽입될 가능성이 존재하는 영역들을 확인함으로써 MS 오피스 문서 파일 내 비정상 요소를 탐지하는 기법을 제안한다.

금산 함우라늄광상 분포지역의 항공자력/방사능 탐사자료 해석 (Interpretation of Airborne Magnetic and Radioactive Data for the Uranium Deposit in Geumsan Area)

  • 신은주;고광범;안동국;한경수
    • 지구물리와물리탐사
    • /
    • 제16권1호
    • /
    • pp.36-44
    • /
    • 2013
  • 금산지역의 흑색셰일형 우라늄광상에 대하여 항공 자력 및 방사능 탐사를 수행하였다. 각 자료의 암상분석과 선구조 분석에 의한 전반적인 지질 및 구조지질적 특성을 살펴보고 이에 기반한 우라늄 광화대의 특성화를 시도하였다. 자극화변환과 하향연속 이상도에서 우라늄광상을 배태하고 있는 흑색 및 암회색 점판암대의 뚜렷한 양의 이상을 인지함으로써 자력탐사의 적용성을 확인하였다. 이차미분 및 곡률을 이용한 선구조 분석을 통해 회색 혼펠스대와 흑색 점판암대를 대표하는 선구조를 도출하고 우라늄 광화대의 추가 부존 가능영역을 추정하였다. 이에 대한 우라늄광 배태여부는 방사능 총이상 및 우라늄 이상도에서 최종 확인하였다. 결론적으로 열변성기원의 우라늄광화대는 국부적인 반면, 흑색셰일형 광화대는 조사지역 전체에 북동-남서방향으로 연속되어 있음을 확인하였다. 또한 우라늄 광화대는 방사능 총이상의 선구조 분석을 통해 단층과 교차하는 곳은 단절되는 전형적인 구조지질적 특징을 보여주었다. 이상의 고찰로부터 항공 자력 및 방사능 탐사는 상호 보완적이며 따라서 병행 수행하는 것이 자료분석 및 해석에 매우 효과적임을 확인하였다.