• Title/Summary/Keyword: Anomaly Data

Search Result 790, Processing Time 0.028 seconds

A hybrid deep neural network compression approach enabling edge intelligence for data anomaly detection in smart structural health monitoring systems

  • Tarutal Ghosh Mondal;Jau-Yu Chou;Yuguang Fu;Jianxiao Mao
    • Smart Structures and Systems
    • /
    • v.32 no.3
    • /
    • pp.179-193
    • /
    • 2023
  • This study explores an alternative to the existing centralized process for data anomaly detection in modern Internet of Things (IoT)-based structural health monitoring (SHM) systems. An edge intelligence framework is proposed for the early detection and classification of various data anomalies facilitating quality enhancement of acquired data before transmitting to a central system. State-of-the-art deep neural network pruning techniques are investigated and compared aiming to significantly reduce the network size so that it can run efficiently on resource-constrained edge devices such as wireless smart sensors. Further, depthwise separable convolution (DSC) is invoked, the integration of which with advanced structural pruning methods exhibited superior compression capability. Last but not least, quantization-aware training (QAT) is adopted for faster processing and lower memory and power consumption. The proposed edge intelligence framework will eventually lead to reduced network overload and latency. This will enable intelligent self-adaptation strategies to be employed to timely deal with a faulty sensor, minimizing the wasteful use of power, memory, and other resources in wireless smart sensors, increasing efficiency, and reducing maintenance costs for modern smart SHM systems. This study presents a theoretical foundation for the proposed framework, the validation of which through actual field trials is a scope for future work.

A Real-Time Network Traffic Anomaly Detection Scheme Using NetFlow Data (NetFlow 데이터를 이용한 실시간 네트워크 트래픽 어노멀리 검출 기법)

  • Kang Koo-Hong;Jang Jong-Soo;Kim Ki-Young
    • The KIPS Transactions:PartC
    • /
    • v.12C no.1 s.97
    • /
    • pp.19-28
    • /
    • 2005
  • Recently, it has been sharply increased the interests to detect the network traffic anomalies to help protect the computer network from unknown attacks. In this paper, we propose a new anomaly detection scheme using the simple linear regression analysis for the exported LetFlow data, such as bits per second and flows per second, from a border router at a campus network. In order to verify the proposed scheme, we apply it to a real campus network and compare the results with the Holt-Winters seasonal algorithm. In particular, we integrate it into the RRDtooi for detecting the anomalies in real time.

Big Data Analysis of Software Performance Trend using SPC with Flexible Moving Window and Fuzzy Theory (가변 윈도우 기법을 적용한 통계적 공정 제어와 퍼지추론 기법을 이용한 소프트웨어 성능 변화의 빅 데이터 분석)

  • Lee, Dong-Hun;Park, Jong-Jin
    • Journal of Institute of Control, Robotics and Systems
    • /
    • v.18 no.11
    • /
    • pp.997-1004
    • /
    • 2012
  • In enterprise software projects, performance issues have become more critical during recent decades. While developing software products, many performance tests are executed in the earlier development phase against the newly added code pieces to detect possible performance regressions. In our previous research, we introduced the framework to enable automated performance anomaly detection and reduce the analysis overhead for identifying the root causes, and showed Statistical Process Control (SPC) can be successfully applied to anomaly detection. In this paper, we explain the special performance trend in which the existing anomaly detection system can hardly detect the noticeable performance change especially when a performance regression is introduced and recovered again a while later. Within the fixed number of sampling period, the fluctuation gets aggravated and the lower and upper control limit get relaxed so that sometimes the existing system hardly detect the noticeable performance change. To resolve the issue, we apply dynamically tuned sampling window size based on the performance trend, and Fuzzy theory to find an appropriate size of the moving window.

Determination of Marine Gravity anomaly Around the Korean Peninsula from GEOSAT Satellite Altimeter Measurements (GEOSAT 인공위성 해면고도 관측자료를 이용한 한반도 주변해역에서 의 중력이상의 결정)

  • 양철수;최광선
    • 한국해양학회지
    • /
    • v.29 no.4
    • /
    • pp.383-391
    • /
    • 1994
  • Sea surface height geoidal undulation, and gravity anomaly derived from satellite altimeter measurements are described. Assuming mean sea surface height (MSSH) as geoidal undulation, MSSH was converted to gravity anomaly. the result shows that the gravity anomaly derived from satellite altimeter data can be mapped to an accuracy of the surface ship gravity measurements. The data used for the conversion is the two-year mean sea surface height obtained from GEOSAT Exact Repeat Mission. The conversion was carried out using fast Fourier transform with plane approximation. In this process, the so called remove-restore method was employed.

  • PDF

Using Geometry based Anomaly Detection to check the Integrity of IFC classifications in BIM Models (기하정보 기반 이상탐지분석을 이용한 BIM 개별 부재 IFC 분류 무결성 검토에 관한 연구)

  • Koo, Bonsang;Shin, Byungjin
    • Journal of KIBIM
    • /
    • v.7 no.1
    • /
    • pp.18-27
    • /
    • 2017
  • Although Industry Foundation Classes (IFC) provide standards for exchanging Building Information Modeling (BIM) data, authoring tools still require manual mapping between BIM entities and IFC classes. This leads to errors and omissions, which results in corrupted data exchanges that are unreliable and thus compromise the validity of IFC. This research explored precedent work by Krijnen and Tamke, who suggested ways to automate the mapping of IFC classes using a machine learning technique, namely anomaly detection. The technique incorporates geometric features of individual components to find outliers among entities in identical IFC classes. This research primarily focused on applying this approach on two architectural BIM models and determining its feasibility as well as limitations. Results indicated that the approach, while effective, misclassified outliers when an IFC class had several dissimilar entities. Another issue was the lack of entities for some specific IFC classes that prohibited the anomaly detection from comparing differences. Future research to improve these issues include the addition of geometric features, using novelty detection and the inclusion of a probabilistic graph model, to improve classification accuracy.

Applying 3D U-statistic method for modeling the iron mineralization in Baghak mine, central section of Sangan iron mines

  • Ghannadpour, Seyyed Saeed;Hezarkhani, Ardeshir;Golmohammadi, Abbas
    • Geosystem Engineering
    • /
    • v.21 no.5
    • /
    • pp.262-272
    • /
    • 2018
  • The U-statistic method is one of the most important structural methods to separate the anomaly from background. It considers the location of samples and carries out the statistical analysis of the data without judging from a geochemical point of view and tries to separate subpopulations and determine anomalous areas. In the present study, 3D U-statistic method has been applied for the first time through the three-dimensional (3D) modeling of an ore deposit. In order to achieve this purpose, 3D U-statistic is applied on the data (Fe grade) resulted from the drilling network in Baghak mine, central part of the Sangan iron mines (in Khorassan Razavi Province, Iran). Afterward, results from applying 3D U-statistic method are used for 3D modeling of the iron mineralization. Results show that the anomalous values are well separated from background so that the determined samples as anomalous are not dispersed and according to their positioning, denser areas of anomalous samples could be considered as anomaly areas. And also, final results (3D model of iron mineralization) show that output model using this method is compatible with designed model for mining operation. Moreover, seen that U-statistic method in addition for separating anomaly from background, could be very efficient for the 3D modeling of different ore type.

Anomaly detection of smart metering system for power management with battery storage system/electric vehicle

  • Sangkeum Lee;Sarvar Hussain Nengroo;Hojun Jin;Yoonmee Doh;Chungho Lee;Taewook Heo;Dongsoo Har
    • ETRI Journal
    • /
    • v.45 no.4
    • /
    • pp.650-665
    • /
    • 2023
  • A novel smart metering technique capable of anomaly detection was proposed for real-time home power management system. Smart meter data generated in real-time were obtained from 900 households of single apartments. To detect outliers and missing values in smart meter data, a deep learning model, the autoencoder, consisting of a graph convolutional network and bidirectional long short-term memory network, was applied to the smart metering technique. Power management based on the smart metering technique was executed by multi-objective optimization in the presence of a battery storage system and an electric vehicle. The results of the power management employing the proposed smart metering technique indicate a reduction in electricity cost and amount of power supplied by the grid compared to the results of power management without anomaly detection.

Design and Evaluation of a Rough Set Based Anomaly Detection Scheme Considering the Age of User Profiles

  • Bae, Ihn-Han
    • Journal of Korea Multimedia Society
    • /
    • v.10 no.12
    • /
    • pp.1726-1732
    • /
    • 2007
  • The rapid proliferation of wireless networks and mobile computing applications has changed the landscape of network security. Anomaly detection is a pattern recognition task whose goal is to report the occurrence of abnormal or unknown behavior in a given system being monitored. This paper presents an efficient rough set based anomaly detection method that can effectively identify a group of especially harmful internal attackers - masqueraders in cellular mobile networks. Our scheme uses the trace data of wireless application layer by a user as feature value. Based on this, the used pattern of a mobile's user can be captured by rough sets, and the abnormal behavior of the mobile can be also detected effectively by applying a roughness membership function with the age of the user profile. The performance of the proposed scheme is evaluated by using a simulation. Simulation results demonstrate that the anomalies are well detected by the proposed scheme that considers the age of user profiles.

  • PDF

Mutual Information Applied to Anomaly Detection

  • Kopylova, Yuliya;Buell, Duncan A.;Huang, Chin-Tser;Janies, Jeff
    • Journal of Communications and Networks
    • /
    • v.10 no.1
    • /
    • pp.89-97
    • /
    • 2008
  • Anomaly detection systems playa significant role in protection mechanism against attacks launched on a network. The greatest challenge in designing systems detecting anomalous exploits is defining what to measure. Effective yet simple, Shannon entropy metrics have been successfully used to detect specific types of malicious traffic in a number of commercially available IDS's. We believe that Renyi entropy measures can also adequately describe the characteristics of a network as a whole as well as detect abnormal traces in the observed traffic. In addition, Renyi entropy metrics might boost sensitivity of the methods when disambiguating certain anomalous patterns. In this paper we describe our efforts to understand how Renyi mutual information can be applied to anomaly detection as an offline computation. An initial analysis has been performed to determine how well fast spreading worms (Slammer, Code Red, and Welchia) can be detected using our technique. We use both synthetic and real data audits to illustrate the potentials of our method and provide a tentative explanation of the results.

Hierarchical Correlation-based Anomaly Detection for Vision-based Mask Filter Inspection in Mask Production Lines (마스크 생산 라인에서 영상 기반 마스크 필터 검사를 위한 계층적 상관관계 기반 이상 현상 탐지)

  • Oh, Gunhee;Lee, Hyojin;Lee, Heoncheol
    • IEMEK Journal of Embedded Systems and Applications
    • /
    • v.16 no.6
    • /
    • pp.277-283
    • /
    • 2021
  • This paper addresses the problem of vision-based mask filter inspection for mask production systems. Machine learning-based approaches can be considered to solve the problem, but they may not be applicable to mask filter inspection if normal and anomaly mask filter data are not sufficient. In such cases, handcrafted image processing methods have to be considered to solve the problem. In this paper, we propose a hierarchical correlation-based approach that combines handcrafted image processing methods to detect anomaly mask filters. The proposed approach combines image rotation, cropping and resizing, edge detection of mask filter parts, average blurring, and correlation-based decision. The proposed approach was tested and analyzed with real mask filters. The results showed that the proposed approach was able to successfully detect anomalies in mask filters.