• Title/Summary/Keyword: Anomaly data detection

Search Result 384, Processing Time 0.026 seconds

Automatic False-Alarm Labeling for Sensor Data

  • Adi, Taufik Nur;Bae, Hyerim;Wahid, Nur Ahmad
    • Journal of the Korea Society of Computer and Information
    • /
    • v.24 no.2
    • /
    • pp.139-147
    • /
    • 2019
  • A false alarm, which is an incorrect report of an emergency, could trigger an unnecessary action. The predictive maintenance framework developed in our previous work has a feature whereby a machine alarm is triggered based on sensor data evaluation. The sensor data evaluator performs three essential evaluation steps. First, it evaluates each sensor data value based on its threshold (lower and upper bound) and labels the data value as "alarm" when the threshold is exceeded. Second, it calculates the duration of the occurrence of the alarm. Finally, in the third step, a domain expert is required to assess the results from the previous two steps and to determine, thereby, whether the alarm is true or false. There are drawbacks of the current evaluation method. It suffers from a high false-alarm ratio, and moreover, given the vast amount of sensor data to be assessed by the domain expert, the process of evaluation is prolonged and inefficient. In this paper, we propose a method for automatic false-alarm labeling that mimics how the domain expert determines false alarms. The domain expert determines false alarms by evaluating two critical factors, specifically the duration of alarm occurrence and identification of anomalies before or while the alarm occurs. In our proposed method, Hierarchical Temporal Memory (HTM) is utilized to detect anomalies. It is an unsupervised approach that is suitable to our main data characteristic, which is the lack of an example of the normal form of sensor data. The result shows that the technique is effective for automatic labeling of false alarms in sensor data.

Performance Comparison of Machine Learning Algorithms for Network Traffic Security in Medical Equipment (의료기기 네트워크 트래픽 보안 관련 머신러닝 알고리즘 성능 비교)

  • Seung Hyoung Ko;Joon Ho Park;Da Woon Wang;Eun Seok Kang;Hyun Wook Han
    • Journal of Information Technology Services
    • /
    • v.22 no.5
    • /
    • pp.99-108
    • /
    • 2023
  • As the computerization of hospitals becomes more advanced, security issues regarding data generated from various medical devices within hospitals are gradually increasing. For example, because hospital data contains a variety of personal information, attempts to attack it have been continuously made. In order to safely protect data from external attacks, each hospital has formed an internal team to continuously monitor whether the computer network is safely protected. However, there are limits to how humans can monitor attacks that occur on networks within hospitals in real time. Recently, artificial intelligence models have shown excellent performance in detecting outliers. In this paper, an experiment was conducted to verify how well an artificial intelligence model classifies normal and abnormal data in network traffic data generated from medical devices. There are several models used for outlier detection, but among them, Random Forest and Tabnet were used. Tabnet is a deep learning algorithm related to receive and classify structured data. Two algorithms were trained using open traffic network data, and the classification accuracy of the model was measured using test data. As a result, the random forest algorithm showed a classification accuracy of 93%, and Tapnet showed a classification accuracy of 99%. Therefore, it is expected that most outliers that may occur in a hospital network can be detected using an excellent algorithm such as Tabnet.

Intrusion Detection based on Clustering a Data Stream (데이터 스트림 클러스터링을 이용한 침임탐지)

  • Oh Sang-Hyun;Kang Jin-Suk;Byun Yung-Cheol
    • Proceedings of the Korea Contents Association Conference
    • /
    • 2005.11a
    • /
    • pp.529-532
    • /
    • 2005
  • In anomaly intrusion detection, how to model the normal behavior of activities performed by a user is an important issue. To extract the normal behavior as a profile, conventional data mining techniques are widely applied to a finite audit data set. However, these approaches can only model the static behavior of a user in the audit data set This drawback can be overcome by viewing the continuous activities of a user as an audit data stream. This paper proposes a new clustering algorithm which continuously models a data stream. A set of features is used to represent the characteristics of an activity. For each feature, the clusters of feature values corresponding to activities observed so far in an audit data stream are identified by the proposed clustering algorithm for data streams. As a result, without maintaining any historical activity of a user physically, new activities of the user can be continuously reflected to the on-going result of clustering.

  • PDF

Detection of unexploded ordnance (UXO) using marine magnetic gradiometer data (해양 자력구배 탐사자료를 이용한 UXO 탐지)

  • Salem Ahmed;Hamada Toshio;Asahina Joseph Kiyoshi;Ushijima Keisuke
    • Geophysics and Geophysical Exploration
    • /
    • v.8 no.1
    • /
    • pp.97-103
    • /
    • 2005
  • Recent development of marine magnetic gradient systems, using arrays of sensors, has made it possible to survey large contaminated areas very quickly. However, underwater Unexploded Ordnances (UXO) can be moved by water currents. Because of this mobility, the cleanup process in such situations becomes dynamic rather than static. This implies that detection should occur in near real-time for successful remediation. Therefore, there is a need for a fast interpretation method to rapidly detect signatures of underwater objects in marine magnetic data. In this paper, we present a fast method for location and characterization of underwater UXOs. The approach utilises gradient interpretation techniques (analytic signal and Euler methods) to locate the objects precisely. Then, using an iterative linear least-squares technique, we obtain the magnetization characteristics of the sources. The approach was applied to a theoretical marine magnetic anomaly, with random errors, over a known source. We demonstrate the practical utility of the method using marine magnetic gradient data from Japan.

An Anomalous Sequence Detection Method Based on An Extended LSTM Autoencoder (확장된 LSTM 오토인코더 기반 이상 시퀀스 탐지 기법)

  • Lee, Jooyeon;Lee, Ki Yong
    • The Journal of Society for e-Business Studies
    • /
    • v.26 no.1
    • /
    • pp.127-140
    • /
    • 2021
  • Recently, sequence data containing time information, such as sensor measurement data and purchase history, has been generated in various applications. So far, many methods for finding sequences that are significantly different from other sequences among given sequences have been proposed. However, most of them have a limitation that they consider only the order of elements in the sequences. Therefore, in this paper, we propose a new anomalous sequence detection method that considers both the order of elements and the time interval between elements. The proposed method uses an extended LSTM autoencoder model, which has an additional layer that converts a sequence into a form that can help effectively learn both the order of elements and the time interval between elements. The proposed method learns the features of the given sequences with the extended LSTM autoencoder model, and then detects sequences that the model does not reconstruct well as anomalous sequences. Using experiments on synthetic data that contains both normal and anomalous sequences, we show that the proposed method achieves an accuracy close to 100% compared to the method that uses only the traditional LSTM autoencoder.

Electrical resistivity survey and interpretation considering excavation effects for the detection of loose ground in urban area

  • Seo Young Song;Bitnarae Kim;Ahyun Cho;Juyeon Jeong;Dongkweon Lee;Myung Jin Nam
    • Geomechanics and Engineering
    • /
    • v.35 no.2
    • /
    • pp.109-119
    • /
    • 2023
  • Ground subsidence in urban areas due to excessive development and degraded underground facilities is a serious problem. Geophysical surveys have been conducted to estimate the distribution and scale of cavities and subsidence. In this study, electrical resistivity tomography (ERT) was performed near an area of road subsidence in an urban area. The subsidence arose due to groundwater leakage that carried soil into a neighboring excavation site. The ERT survey line was located between the main subsidence area and an excavation site. Because ERT data are affected by rapid topographic changes and surrounding structures, the influence of the excavation site on the data was analyzed through field-scale numerical modeling. The effect of an excavation should be considered when interpreting ERT data because it can lead to wrong anomalous results. A method for performing 2D inversion after correcting resistivity data for the effect of the excavation site was proposed. This method was initially tested using a field-scale numerical model that included the excavation site and subsurface anomaly, which was a loosened zone, and was then applied to field data. In addition, ERT data were interpreted using an existing in-house 3D algorithm, which considered the effect of excavation sites. The inversion results demonstrated that conductive anomalies in the loosened zone were greater compared to the inversion that did not consider the effects of excavation.

Comparison of Prediction Accuracy Between Classification and Convolution Algorithm in Fault Diagnosis of Rotatory Machines at Varying Speed (회전수가 변하는 기기의 고장진단에 있어서 특성 기반 분류와 합성곱 기반 알고리즘의 예측 정확도 비교)

  • Moon, Ki-Yeong;Kim, Hyung-Jin;Hwang, Se-Yun;Lee, Jang Hyun
    • Journal of Navigation and Port Research
    • /
    • v.46 no.3
    • /
    • pp.280-288
    • /
    • 2022
  • This study examined the diagnostics of abnormalities and faults of equipment, whose rotational speed changes even during regular operation. The purpose of this study was to suggest a procedure that can properly apply machine learning to the time series data, comprising non-stationary characteristics as the rotational speed changes. Anomaly and fault diagnosis was performed using machine learning: k-Nearest Neighbor (k-NN), Support Vector Machine (SVM), and Random Forest. To compare the diagnostic accuracy, an autoencoder was used for anomaly detection and a convolution based Conv1D was additionally used for fault diagnosis. Feature vectors comprising statistical and frequency attributes were extracted, and normalization & dimensional reduction were applied to the extracted feature vectors. Changes in the diagnostic accuracy of machine learning according to feature selection, normalization, and dimensional reduction are explained. The hyperparameter optimization process and the layered structure are also described for each algorithm. Finally, results show that machine learning can accurately diagnose the failure of a variable-rotation machine under the appropriate feature treatment, although the convolution algorithms have been widely applied to the considered problem.

Applying Novelty Detection for Checking the Integrity of BIM Entity to IFC Class Associations (Novelty detection을 이용한 BIM객체와 IFC 클래스 간 매핑의 무결성 검토에 관한 연구)

  • Koo, Bonsang;Shin, Byungjin
    • Korean Journal of Construction Engineering and Management
    • /
    • v.18 no.6
    • /
    • pp.78-88
    • /
    • 2017
  • With the growing use of BIM in the AEC industry, various new applications are being developed to meet these specific needs. Such developments have increased the importance of Industry Foundation Classes, which is the international standard for sharing BIM data and thus ensuring interoperability. However, mapping individual BIM objects to IFC entities is still a manual task, and is a main cause for errors or omissions during data transfers. This research focused on addressing this issue by applying novelty detection, which is a technique for detecting anomalies in data. By training the algorithm to learn the geometry of IFC entities, misclassifications (i.e., outliers) can be detected automatically. Two IFC classes (ifcWall, ifcDoor) were trained using objects from three BIM models. The results showed that the algorithm was able to correctly identify 141 of 160 outliers. Novelty detection is thus suggested as a competent solution to resolve the mapping issue, mainly due to its ability to create multiple inlier boundaries and ex ante training of element geometry.

Data-driven event detection method for efficient management and recovery of water distribution system man-made disasters (상수도관망 재난관리 및 복구를 위한 데이터기반 이상탐지 방법론 개발)

  • Jung, Donghwi;Ahn, Jaehyun
    • Journal of Korea Water Resources Association
    • /
    • v.51 no.8
    • /
    • pp.703-711
    • /
    • 2018
  • Water distribution system (WDS) pipe bursts are caused from excessive pressure, pipe aging, and ground shift from temperature change and earthquake. Prompt detection of and response to the failure event help prevent large-scale service interruption and catastrophic sinkhole generation. To that end, this study proposes a improved Western Electric Company (WECO) method to improve the detection effectiveness and efficiency of the original WECO method. The original WECO method is an univariate Statistical Process Control (SPC) technique used for identifying any non-random patterns in system output data. The improved WECO method multiples a threshold modifier (w) to each threshold of WECO sub-rules in order to control the sensitivity of anomaly detection in a water distribution network of interest. The Austin network was used to demonstrated the proposed method in which normal random and abnormal pipe flow data were generated. The best w value was identified from a sensitivity analysis, and the impact of measurement frequency (dt = 5, 10, 15 min etc.) was also investigated. The proposed method was compared to the original WECO method with respect to detection probability, false alarm rate, and averaged detection time. Finally, this study provides a set of guidelines on the use of the WECO method for real-life WDS pipe burst detection.

Enhancing Internet of Things Security with Random Forest-Based Anomaly Detection

  • Ahmed Al Shihimi;Muhammad R Ahmed;Thirein Myo;Badar Al Baroomi
    • International Journal of Computer Science & Network Security
    • /
    • v.24 no.6
    • /
    • pp.67-76
    • /
    • 2024
  • The Internet of Things (IoT) has revolutionized communication and device operation, but it has also brought significant security challenges. IoT networks are structured into four levels: devices, networks, applications, and services, each with specific security considerations. Personal Area Networks (PANs), Local Area Networks (LANs), and Wide Area Networks (WANs) are the three types of IoT networks, each with unique security requirements. Communication protocols such as Wi-Fi and Bluetooth, commonly used in IoT networks, are susceptible to vulnerabilities and require additional security measures. Apart from physical security, authentication, encryption, software vulnerabilities, DoS attacks, data privacy, and supply chain security pose significant challenges. Ensuring the security of IoT devices and the data they exchange is crucial. This paper utilizes the Random Forest Algorithm from machine learning to detect anomalous data in IoT devices. The dataset consists of environmental data (temperature and humidity) collected from IoT sensors in Oman. The Random Forest Algorithm is implemented and trained using Python, and the accuracy and results of the model are discussed, demonstrating the effectiveness of Random Forest for detecting IoT device data anomalies.