• Title/Summary/Keyword: data anomaly detection

Search Result 400, Processing Time 0.038 seconds

Data-driven event detection method for efficient management and recovery of water distribution system man-made disasters (상수도관망 재난관리 및 복구를 위한 데이터기반 이상탐지 방법론 개발)

  • Jung, Donghwi;Ahn, Jaehyun
    • Journal of Korea Water Resources Association
    • /
    • v.51 no.8
    • /
    • pp.703-711
    • /
    • 2018
  • Water distribution system (WDS) pipe bursts are caused from excessive pressure, pipe aging, and ground shift from temperature change and earthquake. Prompt detection of and response to the failure event help prevent large-scale service interruption and catastrophic sinkhole generation. To that end, this study proposes a improved Western Electric Company (WECO) method to improve the detection effectiveness and efficiency of the original WECO method. The original WECO method is an univariate Statistical Process Control (SPC) technique used for identifying any non-random patterns in system output data. The improved WECO method multiples a threshold modifier (w) to each threshold of WECO sub-rules in order to control the sensitivity of anomaly detection in a water distribution network of interest. The Austin network was used to demonstrated the proposed method in which normal random and abnormal pipe flow data were generated. The best w value was identified from a sensitivity analysis, and the impact of measurement frequency (dt = 5, 10, 15 min etc.) was also investigated. The proposed method was compared to the original WECO method with respect to detection probability, false alarm rate, and averaged detection time. Finally, this study provides a set of guidelines on the use of the WECO method for real-life WDS pipe burst detection.

Enhancing Internet of Things Security with Random Forest-Based Anomaly Detection

  • Ahmed Al Shihimi;Muhammad R Ahmed;Thirein Myo;Badar Al Baroomi
    • International Journal of Computer Science & Network Security
    • /
    • v.24 no.6
    • /
    • pp.67-76
    • /
    • 2024
  • The Internet of Things (IoT) has revolutionized communication and device operation, but it has also brought significant security challenges. IoT networks are structured into four levels: devices, networks, applications, and services, each with specific security considerations. Personal Area Networks (PANs), Local Area Networks (LANs), and Wide Area Networks (WANs) are the three types of IoT networks, each with unique security requirements. Communication protocols such as Wi-Fi and Bluetooth, commonly used in IoT networks, are susceptible to vulnerabilities and require additional security measures. Apart from physical security, authentication, encryption, software vulnerabilities, DoS attacks, data privacy, and supply chain security pose significant challenges. Ensuring the security of IoT devices and the data they exchange is crucial. This paper utilizes the Random Forest Algorithm from machine learning to detect anomalous data in IoT devices. The dataset consists of environmental data (temperature and humidity) collected from IoT sensors in Oman. The Random Forest Algorithm is implemented and trained using Python, and the accuracy and results of the model are discussed, demonstrating the effectiveness of Random Forest for detecting IoT device data anomalies.

Experimental Analysis of Physical Signal Jamming Attacks on Automotive LiDAR Sensors and Proposal of Countermeasures (차량용 LiDAR 센서 물리적 신호교란 공격 중심의 실험적 분석과 대응방안 제안)

  • Ji-ung Hwang;Yo-seob Yoon;In-su Oh;Kang-bin Yim
    • Journal of the Korea Institute of Information Security & Cryptology
    • /
    • v.34 no.2
    • /
    • pp.217-228
    • /
    • 2024
  • LiDAR(Light Detection And Ranging) sensors, which play a pivotal role among cameras, RADAR(RAdio Detection And Ranging), and ultrasonic sensors for the safe operation of autonomous vehicles, can recognize and detect objects in 360 degrees. However, since LiDAR sensors use lasers to measure distance, they are vulnerable to attackers and face various security threats. In this paper, we examine several security threats against LiDAR sensors: relay, spoofing, and replay attacks, analyze the possibility and impact of physical jamming attacks, and analyze the risk these attacks pose to the reliability of autonomous driving systems. Through experiments, we show that jamming attacks can cause errors in the ranging ability of LiDAR sensors. With vehicle-to-vehicle (V2V) communication, multi-sensor fusion under development and LiDAR anomaly data detection, this work aims to provide a basic direction for countermeasures against these threats enhancing the security of autonomous vehicles, and verify the practical applicability and effectiveness of the proposed countermeasures in future research.

Power Quality Warning of High-Speed Rail Based on Multi-Features Similarity

  • Bai, Jingjing;Gu, Wei;Yuan, Xiaodong;Li, Qun;Chen, Bing;Wang, Xuchong
    • Journal of Electrical Engineering and Technology
    • /
    • v.10 no.1
    • /
    • pp.92-101
    • /
    • 2015
  • As one type of power quality (PQ) disturbance sources, high-speed rail (HSR) can have major impacts on the power supply grid. Providing timely and accurate warning information for PQ problems of HSR is important for the safe and stable operation of traction power supply systems and the power supply grid. This study proposes a novel warning approach to identify PQ problems and provide warning prompts based on the monitored data of HSR. To embody the displacement and status change of monitored data, multi-features of different sliding windows are computed. To reflect the relative importance degree of these features in the overall evaluation, an analytic hierarchy process (AHP) is used to analyse the weights of multi-features. Finally, a multi-features similarity algorithm is applied to analyse the difference between monitored data and the reference data of HSR, and PQ warning results based on dynamic thresholds can be analysed to quantify its severity. Cases studies demonstrate that the proposed approach is effective and feasible, and it has now been applied to an actual PQ monitoring platform.

Trends in AI Technology for Smart Manufacturing in the Future (미래 스마트 제조를 위한 인공지능 기술동향)

  • Lee, E.S.;Bae, H.C.;Kim, H.J.;Han, H.N.;Lee, Y.K.;Son, J.Y.
    • Electronics and Telecommunications Trends
    • /
    • v.35 no.1
    • /
    • pp.60-70
    • /
    • 2020
  • Artificial intelligence (AI) is expected to bring about a wide range of changes in the industry, based on the assessment that it is the most innovative technology in the last three decades. The manufacturing field is an area in which various artificial intelligence technologies are being applied, and through accumulated data analysis, an optimal operation method can be presented to improve the productivity of manufacturing processes. In addition, AI technologies are being used throughout all areas of manufacturing, including product design, engineering, improvement of working environments, detection of anomalies in facilities, and quality control. This makes it possible to easily design and engineer products with a fast pace and provides an efficient working and training environment for workers. Also, abnormal situations related to quality deterioration can be identified, and autonomous operation of facilities without human intervention is made possible. In this paper, AI technologies used in smart factories, such as the trends in generative product design, smart workbench and real-sense interaction guide technology for work and training, anomaly detection technology for quality control, and intelligent manufacturing facility technology for autonomous production, are analyzed.

GLOBAL MONITORING OF PLANKTON BLOOMS USING MERIS MCI

  • Gower, Jim;King, Stephanie;Goncalves, Pedro
    • Proceedings of the KSRS Conference
    • /
    • v.1
    • /
    • pp.441-444
    • /
    • 2006
  • The MERIS MCI (Maximum Chlorophyll Index), measuring the radiance peak at 709 nm in water-leaving radiance, indicates the presence of a high surface concentration of chlorophyll ${\underline{a}}$ against a scattering background. The index is high in 'red tide' conditions (intense, visible, surface, plankton blooms), and is also raised when aquatic vegetation is present. A bloom search based on MCI has resulted in detection of a variety of events in Canadian, Antarctic and other waters round the world, as well as detection of extensive areas of pelagic vegetation (Sargassum spp.), previously unreported in the scientific literature. Since June 1 2006, global MCI composite images, at a spatial resolution of 5 km, are being produced daily from all MERIS (daylight) passes of Reduced Resolution (RR) data. The global composites significantly increase the area now being searched for events, though the reduced spatial resolution may cause smaller events to be missed. This paper describes the composites and gives examples of plankton bloom events that they have detected. It also shows how the composites show the effect of the South Atlantic Anomaly, where cosmic rays affect the MERIS instrument.

  • PDF

Determining the Time of Least Water Use for the Major Water Usage Types in District Metered Areas (상수관망 블록의 대표적인 용수사용 유형에 대한 최소 용수사용 시간의 결정)

  • Park, Suwan;Jung, So-Yeon;Sahleh, Vahideh
    • Journal of Korean Society of Water and Wastewater
    • /
    • v.29 no.3
    • /
    • pp.415-425
    • /
    • 2015
  • Aging water pipe networks hinder efficient management of important water service indices such as revenue water and leakage ratio due to pipe breakage and malfunctioning of pipe appurtenance. In order to control leakage in water pipe networks, various methods such as the minimum night flow analysis and sound waves method have been used. However, the accuracy and efficiency of detecting water leak by these methods need to be improved due to the increase of water consumption at night. In this study the Principal Component Analysis (PCA) technique was applied to the night water flow data of 426 days collected from a water distribution system in the interval of one hour. Based on the PCA technique, computational algorithms were developed to narrow the time windows for efficient execution of leak detection job. The algorithms were programmed on computer using the MATLAB. The presented techniques are expected to contribute to the efficient management of water pipe networks by providing more effective time windows for the detection of the anomaly of pipe network such as leak or abnormal demand.

Feature Selection with PCA based on DNS Query for Malicious Domain Classification (비정상도메인 분류를 위한 DNS 쿼리 기반의 주성분 분석을 이용한 성분추출)

  • Lim, Sun-Hee;Cho, Jaeik;Kim, Jong-Hyun;Lee, Byung Gil
    • KIPS Transactions on Computer and Communication Systems
    • /
    • v.1 no.1
    • /
    • pp.55-60
    • /
    • 2012
  • Recent botnets are widely using the DNS services at the connection of C&C server in order to evade botnet's detection. It is necessary to study on DNS analysis in order to counteract anomaly-based technique using the DNS. This paper studies collection of DNS traffic for experimental data and supervised learning for DNS traffic-based malicious domain classification such as query of domain name corresponding to C&C server from zombies. Especially, this paper would aim to determine significant features of DNS-based classification system for malicious domain extraction by the Principal Component Analysis(PCA).

A Study on Detecting Changes in Injection Molding Process through Similarity Analysis of Mold Vibration Signal Patterns (금형 기반 진동 신호 패턴의 유사도 분석을 통한 사출성형공정 변화 감지에 대한 연구)

  • Jong-Sun Kim
    • Design & Manufacturing
    • /
    • v.17 no.3
    • /
    • pp.34-40
    • /
    • 2023
  • In this study, real-time collection of mold vibration signals during injection molding processes was achieved through IoT devices installed on the mold surface. To analyze changes in the collected vibration signals, injection molding was performed under six different process conditions. Analysis of the mold vibration signals according to process conditions revealed distinct trends and patterns. Based on this result, cosine similarity was applied to compare pattern changes in the mold vibration signals. The similarity in time and acceleration vector space between the collected data was analyzed. The results showed that under identical conditions for all six process settings, the cosine similarity remained around 0.92±0.07. However, when different process conditions were applied, the cosine similarity decreased to the range of 0.47±0.07. Based on these results, a cosine similarity threshold of 0.60~0.70 was established. When applied to the analysis of mold vibration signals, it was possible to determine whether the molding process was stable or whether variations had occurred due to changes in process conditions. This establishes the potential use of cosine similarity based on mold vibration signals in future applications for real-time monitoring of molding process changes and anomaly detection.

Feature Selection for Anomaly Detection Based on Genetic Algorithm (유전 알고리즘 기반의 비정상 행위 탐지를 위한 특징선택)

  • Seo, Jae-Hyun
    • Journal of the Korea Convergence Society
    • /
    • v.9 no.7
    • /
    • pp.1-7
    • /
    • 2018
  • Feature selection, one of data preprocessing techniques, is one of major research areas in many applications dealing with large dataset. It has been used in pattern recognition, machine learning and data mining, and is now widely applied in a variety of fields such as text classification, image retrieval, intrusion detection and genome analysis. The proposed method is based on a genetic algorithm which is one of meta-heuristic algorithms. There are two methods of finding feature subsets: a filter method and a wrapper method. In this study, we use a wrapper method, which evaluates feature subsets using a real classifier, to find an optimal feature subset. The training dataset used in the experiment has a severe class imbalance and it is difficult to improve classification performance for rare classes. After preprocessing the training dataset with SMOTE, we select features and evaluate them with various machine learning algorithms.