• Title/Summary/Keyword: unsupervised detection

Search Result 170, Processing Time 0.026 seconds

Outlier detection of main engine data of a ship using ensemble method (앙상블 기법을 이용한 선박 메인엔진 빅데이터의 이상치 탐지)

  • KIM, Dong-Hyun;LEE, Ji-Hwan;LEE, Sang-Bong;JUNG, Bong-Kyu
    • Journal of the Korean Society of Fisheries and Ocean Technology
    • /
    • v.56 no.4
    • /
    • pp.384-394
    • /
    • 2020
  • This paper proposes an outlier detection model based on machine learning that can diagnose the presence or absence of major engine parts through unsupervised learning analysis of main engine big data of a ship. Engine big data of the ship was collected for more than seven months, and expert knowledge and correlation analysis were performed to select features that are closely related to the operation of the main engine. For unsupervised learning analysis, ensemble model wherein many predictive models are strategically combined to increase the model performance, is used for anomaly detection. As a result, the proposed model successfully detected the anomalous engine status from the normal status. To validate our approach, clustering analysis was conducted to find out the different patterns of anomalies the anomalous point. By examining distribution of each cluster, we could successfully find the patterns of anomalies.

A Study on Atmospheric Data Anomaly Detection Algorithm based on Unsupervised Learning Using Adversarial Generative Neural Network (적대적 생성 신경망을 활용한 비지도 학습 기반의 대기 자료 이상 탐지 알고리즘 연구)

  • Yang, Ho-Jun;Lee, Seon-Woo;Lee, Mun-Hyung;Kim, Jong-Gu;Choi, Jung-Mu;Shin, Yu-mi;Lee, Seok-Chae;Kwon, Jang-Woo;Park, Ji-Hoon;Jung, Dong-Hee;Shin, Hye-Jung
    • Journal of Convergence for Information Technology
    • /
    • v.12 no.4
    • /
    • pp.260-269
    • /
    • 2022
  • In this paper, We propose an anomaly detection model using deep neural network to automate the identification of outliers of the national air pollution measurement network data that is previously performed by experts. We generated training data by analyzing missing values and outliers of weather data provided by the Institute of Environmental Research and based on the BeatGAN model of the unsupervised learning method, we propose a new model by changing the kernel structure, adding the convolutional filter layer and the transposed convolutional filter layer to improve anomaly detection performance. In addition, by utilizing the generative features of the proposed model to implement and apply a retraining algorithm that generates new data and uses it for training, it was confirmed that the proposed model had the highest performance compared to the original BeatGAN models and other unsupervised learning model like Iforest and One Class SVM. Through this study, it was possible to suggest a method to improve the anomaly detection performance of proposed model while avoiding overfitting without additional cost in situations where training data are insufficient due to various factors such as sensor abnormalities and inspections in actual industrial sites.

Study of an algorithm for intelligent digital protective relaying (지능형 디지탈 보호계전 알고리즘 연구)

  • 신현익;이성환;강신준;김정한;김상철
    • 제어로봇시스템학회:학술대회논문집
    • /
    • 1996.10b
    • /
    • pp.343-346
    • /
    • 1996
  • A new method for on-line induction motor fault detection is presented in this paper. This system utilizes unsupervised-learning clustering algorithm, the Dignet, proposed by Thomopoulos etc., to learn the spectral characteristics of a good motor operating on-line. After a sufficient training period, the Dignet signals one-phase ground fault, or a potential failure condition when a new cluster is formed and persists for some time. Since a fault condition is found by comparison to a prior condition of the machine, on-line failure prediction is possible with this system without requiring information on the motor of load characteristics.

  • PDF

ART1 Neural Network for the Detection of Tool Breakage (공구파단 검출을 위한 ART2 신경회로망)

  • 고태조;김희술;조동우
    • Proceedings of the Korean Society of Precision Engineering Conference
    • /
    • 1995.04b
    • /
    • pp.451-456
    • /
    • 1995
  • This study investigates the feasibility of the real time detection of tool breadage in face milling operation. The proposed methodology using an ART2 neural network overcomes a cumbersome task in terms of the learning or determining a threshold value. The features taken in the researchare the AR parameters modelled from a RLS, and those are proven to be good features for tool breakage from experiments. From the results of the off line application, we can conclude that an ART2 neural network can be well applied to the clustering of tool states in real time regardless of the unsupervised learning.

  • PDF

Earthquake Damage Monitoring for Underground Structures Based Damage Detection Techniques

  • Kim, Jin Ho;Kim, Na Eun
    • International Journal of Railway
    • /
    • v.7 no.4
    • /
    • pp.94-99
    • /
    • 2014
  • Urban railway systems are located under populated areas and are mostly constructed for underground structures which demand high standards of structural safety. However, the damage progression of underground structures is hard to evaluate and damaged underground structures may not effectively stand against successive earthquakes. This study attempts to examine initial damage-stage and to access structural damage condition of the ground structures using Earthquake Damage Monitoring (EDM) system. For actual underground structure, vulnerable damaged member of Ulchiro-3ga station is chosen by finite element analysis using applied artificial earthquake load, and then damage pattern and history of damaged members is obtained from measured acceleration data introduced unsupervised learning recognition. The result showed damage index obtained by damage scenario establishment using acceleration response of selected vulnerable members is useful. Initial damage state is detected for selected vulnerable member according to established damage scenario. Stiffness degrading ratio is increasing whereas the value of reliability interval is decreasing.

Detection of Political Manipulation through Unsupervised Learning

  • Lee, Sihyung
    • KSII Transactions on Internet and Information Systems (TIIS)
    • /
    • v.13 no.4
    • /
    • pp.1825-1844
    • /
    • 2019
  • Political campaigns circulate manipulative opinions in online communities to implant false beliefs and eventually win elections. Not only is this type of manipulation unfair, it also has long-lasting negative impacts on people's lives. Existing tools detect political manipulation based on a supervised classifier, which is accurate when trained with large labeled data. However, preparing this data becomes an excessive burden and must be repeated often to reflect changing manipulation tactics. We propose a practical detection system that requires moderate groundwork to achieve a sufficient level of accuracy. The proposed system groups opinions with similar properties into clusters, and then labels a few opinions from each cluster to build a classifier. It also models each opinion with features deduced from raw data with no additional processing. To validate the system, we collected over a million opinions during three nation-wide campaigns in South Korea. The system reduced groundwork from 200K to nearly 200 labeling tasks, and correctly identified over 90% of manipulative opinions. The system also effectively identified transitions in manipulative tactics over time. We suggest that online communities perform periodic audits using the proposed system to highlight manipulative opinions and emerging tactics.

Anomaly Detection in Sensor Data

  • Kim, Jong-Min;Baik, Jaiwook
    • Journal of Applied Reliability
    • /
    • v.18 no.1
    • /
    • pp.20-32
    • /
    • 2018
  • Purpose: The purpose of this study is to set up an anomaly detection criteria for sensor data coming from a motorcycle. Methods: Five sensor values for accelerator pedal, engine rpm, transmission rpm, gear and speed are obtained every 0.02 second from a motorcycle. Exploratory data analysis is used to find any pattern in the data. Traditional process control methods such as X control chart and time series models are fitted to find any anomaly behavior in the data. Finally unsupervised learning algorithm such as k-means clustering is used to find any anomaly spot in the sensor data. Results: According to exploratory data analysis, the distribution of accelerator pedal sensor values is very much skewed to the left. The motorcycle seemed to have been driven in a city at speed less than 45 kilometers per hour. Traditional process control charts such as X control chart fail due to severe autocorrelation in each sensor data. However, ARIMA model found three abnormal points where they are beyond 2 sigma limits in the control chart. We applied a copula based Markov chain to perform statistical process control for correlated observations. Copula based Markov model found anomaly behavior in the similar places as ARIMA model. In an unsupervised learning algorithm, large sensor values get subdivided into two, three, and four disjoint regions. So extreme sensor values are the ones that need to be tracked down for any sign of anomaly behavior in the sensor values. Conclusion: Exploratory data analysis is useful to find any pattern in the sensor data. Process control chart using ARIMA and Joe's copula based Markov model also give warnings near similar places in the data. Unsupervised learning algorithm shows us that the extreme sensor values are the ones that need to be tracked down for any sign of anomaly behavior.

Accuracy Assessment of Unsupervised Change Detection Using Automated Threshold Selection Algorithms and KOMPSAT-3A (자동 임계값 추출 알고리즘과 KOMPSAT-3A를 활용한 무감독 변화탐지의 정확도 평가)

  • Lee, Seung-Min;Jeong, Jong-Chul
    • Korean Journal of Remote Sensing
    • /
    • v.36 no.5_2
    • /
    • pp.975-988
    • /
    • 2020
  • Change detection is the process of identifying changes by observing the multi-temporal images at different times, and it is an important technique in remote sensing using satellite images. Among the change detection methods, the unsupervised change detection technique has the advantage of extracting rapidly the change area as a binary image. However, it is difficult to understand the changing pattern of land cover in binary images. This study used grid points generated from seamless digital map to evaluate the satellite image change detection results. The land cover change results were extracted using multi-temporal KOMPSAT-3A (K3A) data taken by Gimje Free Trade Zone and change detection algorithm used Spectral Angle Mapper (SAM). Change detection results were presented as binary images using the methods Otsu, Kittler, Kapur, and Tsai among the automated threshold selection algorithms. To consider the seasonal change of vegetation in the change detection process, we used the threshold of Differenced Normalized Difference Vegetation Index (dNDVI) through the probability density function. The experimental results showed the accuracy of the Otsu and Kapur was the highest at 58.16%, and the accuracy improved to 85.47% when the seasonal effects were removed through dNDVI. The algorithm generated based on this research is considered to be an effective method for accuracy assessment and identifying changes pattern when applied to unsupervised change detection.

Proposal of a new method for learning of diesel generator sounds and detecting abnormal sounds using an unsupervised deep learning algorithm

  • Hweon-Ki Jo;Song-Hyun Kim;Chang-Lak Kim
    • Nuclear Engineering and Technology
    • /
    • v.55 no.2
    • /
    • pp.506-515
    • /
    • 2023
  • This study is to find a method to learn engine sound after the start-up of a diesel generator installed in nuclear power plant with an unsupervised deep learning algorithm (CNN autoencoder) and a new method to predict the failure of a diesel generator using it. In order to learn the sound of a diesel generator with a deep learning algorithm, sound data recorded before and after the start-up of two diesel generators was used. The sound data of 20 min and 2 h were cut into 7 s, and the split sound was converted into a spectrogram image. 1200 and 7200 spectrogram images were created from sound data of 20 min and 2 h, respectively. Using two different deep learning algorithms (CNN autoencoder and binary classification), it was investigated whether the diesel generator post-start sounds were learned as normal. It was possible to accurately determine the post-start sounds as normal and the pre-start sounds as abnormal. It was also confirmed that the deep learning algorithm could detect the virtual abnormal sounds created by mixing the unusual sounds with the post-start sounds. This study showed that the unsupervised anomaly detection algorithm has a good accuracy increased about 3% with comparing to the binary classification algorithm.

Extraction of Changed Pixels for Hyperion Hyperspectral Images Using Range Average Based Buffer Zone Concept (구간평균 그래프 기반의 버퍼존 개념을 적용한 Hyperion 초분광영상의 변화화소 추출)

  • Kim, Dae-Sung;Pyen, Mu-Wook
    • Journal of the Korean Society of Surveying, Geodesy, Photogrammetry and Cartography
    • /
    • v.29 no.5
    • /
    • pp.487-496
    • /
    • 2011
  • This study is aimed to perform more reliable unsupervised change detection through the re-extraction of the changed pixels which were extracted with global thresholding by applying buffer zone concept. First, three buffer zone was divided on the basis of the thresholding value which was determined using range average and the maximum distance point from a straight line. We re-extracted the changed pixels by performing unsupervised classification for buffer zone II which consists of changed pixels and unchanged pixels. The proposed method was implemented in Hyperion hyperspectral images and evaluated comparing to the existing global thresholding method. The experimental results demonstrated that the proposed method performed more accuracy change detection for vegetation area even if extracted slightly more changed pixels.