• 제목/요약/키워드: Anomaly Data

Search Result 818, Processing Time 0.03 seconds

Gravity Potential Comparative Analysis around Korean Peninsula by EGM96 and EIGEN-CG01C Models (EGM96와 EIGEN-CG01C 모델에 의한 한반도 주변의 중력포텐셜 비교분석)

  • Yu, Sang-Hoon;Kim, Chang-Hwan;Min, Kyung-Duck
    • 한국지구물리탐사학회:학술대회논문집
    • /
    • 2005.05a
    • /
    • pp.261-266
    • /
    • 2005
  • According to development of satellite geodesy, gravity potential models which have high accuracy and resolution were released. Using the EIGEN-CG01C model based on low orbit satellite data such as CHAMP and GRACE and the EGM96 model, geoid and gravity anomaly were calculated and compared. The study area is located at $123^{\circ}{\sim}132^{\circ}$ E, $33^{\circ}{\sim}43^{\circ}$ including Korea. Comparing two models, very high correlation more than 0.90 in geoid and gravity anomaly was observed, but in amplitude analysis the EIGEN-CG01C model have higher amplitude in high frequency area. Gravity anomaly calculated with both models shows a little difference in North Korea and some coast area of the Yellow sea. Through power spectrum analysis, residual anomaly that can be used in large scale structure or underground resources survey was calculated.

  • PDF

A Method for Region-Specific Anomaly Detection on Patch-wise Segmented PA Chest Radiograph (PA 흉부 X-선 영상 패치 분할에 의한 지역 특수성 이상 탐지 방법)

  • Hyun-bin Kim;Jun-Chul Chun
    • Journal of Internet Computing and Services
    • /
    • v.24 no.1
    • /
    • pp.49-59
    • /
    • 2023
  • Recently, attention to the pandemic situation represented by COVID-19 emerged problems caused by unexpected shortage of medical personnel. In this paper, we present a method for diagnosing the presence or absence of lesional sign on PA chest X-ray images as computer vision solution to support diagnosis tasks. Method for visual anomaly detection based on feature modeling can be also applied to X-ray images. With extracting feature vectors from PA chest X-ray images and divide to patch unit, region-specific abnormality can be detected. As preliminary experiment, we created simulation data set containing multiple objects and present results of the comparative experiments in this paper. We present method to improve both efficiency and performance of the process through hard masking of patch features to aligned images. By summing up regional specificity and global anomaly detection results, it shows improved performance by 0.069 AUROC compared to previous studies. By aggregating region-specific and global anomaly detection results, it shows improved performance by 0.069 AUROC compared to our last study.

Comparative Analysis of Machine Learning Techniques for IoT Anomaly Detection Using the NSL-KDD Dataset

  • Zaryn, Good;Waleed, Farag;Xin-Wen, Wu;Soundararajan, Ezekiel;Maria, Balega;Franklin, May;Alicia, Deak
    • International Journal of Computer Science & Network Security
    • /
    • v.23 no.1
    • /
    • pp.46-52
    • /
    • 2023
  • With billions of IoT (Internet of Things) devices populating various emerging applications across the world, detecting anomalies on these devices has become incredibly important. Advanced Intrusion Detection Systems (IDS) are trained to detect abnormal network traffic, and Machine Learning (ML) algorithms are used to create detection models. In this paper, the NSL-KDD dataset was adopted to comparatively study the performance and efficiency of IoT anomaly detection models. The dataset was developed for various research purposes and is especially useful for anomaly detection. This data was used with typical machine learning algorithms including eXtreme Gradient Boosting (XGBoost), Support Vector Machines (SVM), and Deep Convolutional Neural Networks (DCNN) to identify and classify any anomalies present within the IoT applications. Our research results show that the XGBoost algorithm outperformed both the SVM and DCNN algorithms achieving the highest accuracy. In our research, each algorithm was assessed based on accuracy, precision, recall, and F1 score. Furthermore, we obtained interesting results on the execution time taken for each algorithm when running the anomaly detection. Precisely, the XGBoost algorithm was 425.53% faster when compared to the SVM algorithm and 2,075.49% faster than the DCNN algorithm. According to our experimental testing, XGBoost is the most accurate and efficient method.

Network Anomaly Detection Technologies Using Unsupervised Learning AutoEncoders (비지도학습 오토 엔코더를 활용한 네트워크 이상 검출 기술)

  • Kang, Koohong
    • Journal of the Korea Institute of Information Security & Cryptology
    • /
    • v.30 no.4
    • /
    • pp.617-629
    • /
    • 2020
  • In order to overcome the limitations of the rule-based intrusion detection system due to changes in Internet computing environments, the emergence of new services, and creativity of attackers, network anomaly detection (NAD) using machine learning and deep learning technologies has received much attention. Most of these existing machine learning and deep learning technologies for NAD use supervised learning methods to learn a set of training data set labeled 'normal' and 'attack'. This paper presents the feasibility of the unsupervised learning AutoEncoder(AE) to NAD from data sets collecting of secured network traffic without labeled responses. To verify the performance of the proposed AE mode, we present the experimental results in terms of accuracy, precision, recall, f1-score, and ROC AUC value on the NSL-KDD training and test data sets. In particular, we model a reference AE through the deep analysis of diverse AEs varying hyper-parameters such as the number of layers as well as considering the regularization and denoising effects. The reference model shows the f1-scores 90.4% and 89% of binary classification on the KDDTest+ and KDDTest-21 test data sets based on the threshold of the 82-th percentile of the AE reconstruction error of the training data set.

A Study on Efficient AI Model Drift Detection Methods for MLOps (MLOps를 위한 효율적인 AI 모델 드리프트 탐지방안 연구)

  • Ye-eun Lee;Tae-jin Lee
    • Journal of Internet Computing and Services
    • /
    • v.24 no.5
    • /
    • pp.17-27
    • /
    • 2023
  • Today, as AI (Artificial Intelligence) technology develops and its practicality increases, it is widely used in various application fields in real life. At this time, the AI model is basically learned based on various statistical properties of the learning data and then distributed to the system, but unexpected changes in the data in a rapidly changing data situation cause a decrease in the model's performance. In particular, as it becomes important to find drift signals of deployed models in order to respond to new and unknown attacks that are constantly created in the security field, the need for lifecycle management of the entire model is gradually emerging. In general, it can be detected through performance changes in the model's accuracy and error rate (loss), but there are limitations in the usage environment in that an actual label for the model prediction result is required, and the detection of the point where the actual drift occurs is uncertain. there is. This is because the model's error rate is greatly influenced by various external environmental factors, model selection and parameter settings, and new input data, so it is necessary to precisely determine when actual drift in the data occurs based only on the corresponding value. There are limits to this. Therefore, this paper proposes a method to detect when actual drift occurs through an Anomaly analysis technique based on XAI (eXplainable Artificial Intelligence). As a result of testing a classification model that detects DGA (Domain Generation Algorithm), anomaly scores were extracted through the SHAP(Shapley Additive exPlanations) Value of the data after distribution, and as a result, it was confirmed that efficient drift point detection was possible.

A method for concrete crack detection using U-Net based image inpainting technique

  • Kim, Su-Min;Sohn, Jung-Mo;Kim, Do-Soo
    • Journal of the Korea Society of Computer and Information
    • /
    • v.25 no.10
    • /
    • pp.35-42
    • /
    • 2020
  • In this study, we propose a crack detection method using limited data with a U-Net based image inpainting technique that is a modified unsupervised anomaly detection method. Concrete cracking occurs due to a variety of causes and is a factor that can cause serious damage to the structure in the long term. In general, crack investigation uses an inspector's visual inspection on the concrete surfaces, which is less objective in judgment and has a high possibility of human error. Therefore, a method with objective and accurate image analysis processing is required. In recent years, the methods using deep learning have been studied to detect cracks quickly and accurately. However, when the amount of crack data on the building or infrastructure to be inspected is small, existing crack detection models using it often show a limited performance. Therefore, in this study, an unsupervised anomaly detection method was used to augment the data on the object to be inspected, and as a result of learning using the data, we confirmed the performance of 98.78% of accuracy and 82.67% of harmonic average (F1_Score).

Clustering Normal User Behavior for Anomaly Intrusion Detection (비정상행위 탐지를 위한 사용자 정상행위 클러스터링 기법)

  • Oh, Sang-Hyun;Lee, Won-Suk
    • The KIPS Transactions:PartC
    • /
    • v.10C no.7
    • /
    • pp.857-866
    • /
    • 2003
  • For detecting an intrusion based on the anomaly of a user's activities, previous works are concentrated on statistical techniques in order to analyze an audit data set. However. since they mainly analyze the average behavior of a user's activities, some anomalies can be detected inaccurately. In this paper, a new clustering algorithm for modeling the normal pattern of a user's activities is proposed. Since clustering can identify an arbitrary number of dense ranges in an analysis domain, it can eliminate the inaccuracy caused by statistical analysis. Also, clustering can be used to model common knowledge occurring frequently in a set of transactions. Consequently, the common activities of a user can be found more accurately. The common knowledge is represented by the occurrence frequency of similar data objects by the unit of a transaction as veil as the common repetitive ratio of similar data objects in each transaction. Furthermore, the proposed method also addresses how to maintain identified common knowledge as a concise profile. As a result, the profile can be used to detect any anomalous behavior In an online transaction.

Improving the Performance of Machine Learning Models for Anomaly Detection based on Vibration Analog Signals (진동 아날로그 신호 기반의 이상상황 탐지를 위한 기계학습 모형의 성능지표 향상)

  • Jaehun Kim;Sangcheon Eom;Chulsoon Park
    • Journal of Korean Society of Industrial and Systems Engineering
    • /
    • v.47 no.2
    • /
    • pp.1-9
    • /
    • 2024
  • New motor development requires high-speed load testing using dynamo equipment to calculate the efficiency of the motor. Abnormal noise and vibration may occur in the test equipment rotating at high speed due to misalignment of the connecting shaft or looseness of the fixation, which may lead to safety accidents. In this study, three single-axis vibration sensors for X, Y, and Z axes were attached on the surface of the test motor to measure the vibration value of vibration. Analog data collected from these sensors was used in classification models for anomaly detection. Since the classification accuracy was around only 93%, commonly used hyperparameter optimization techniques such as Grid search, Random search, and Bayesian Optimization were applied to increase accuracy. In addition, Response Surface Method based on Design of Experiment was also used for hyperparameter optimization. However, it was found that there were limits to improving accuracy with these methods. The reason is that the sampling data from an analog signal does not reflect the patterns hidden in the signal. Therefore, in order to find pattern information of the sampling data, we obtained descriptive statistics such as mean, variance, skewness, kurtosis, and percentiles of the analog data, and applied them to the classification models. Classification models using descriptive statistics showed excellent performance improvement. The developed model can be used as a monitoring system that detects abnormal conditions of the motor test.

Anomaly detection in blade pitch systems of floating wind turbines using LSTM-Autoencoder (LSTM-Autoencoder를 이용한 부유식 풍력터빈 블레이드 피치 시스템의 이상징후 감지)

  • Seongpil Cho
    • Journal of Aerospace System Engineering
    • /
    • v.18 no.4
    • /
    • pp.43-52
    • /
    • 2024
  • This paper presents an anomaly detection system that uses an LSTM-Autoencoder model to identify early-stage anomalies in the blade pitch system of floating wind turbines. The sensor data used in power plant monitoring systems is primarily composed of multivariate time-series data for each component. Comprising two unidirectional LSTM networks, the system skillfully uncovers long-term dependencies hidden within sequential time-series data. The autoencoder mechanism, learning solely from normal state data, effectively classifies abnormal states. Thus, by integrating these two networks, the system can proficiently detect anomalies. To confirm the effectiveness of the proposed framework, a real multivariate time-series dataset collected from a wind turbine model was employed. The LSTM-autoencoder model showed robust performance, achieving high classification accuracy.

Autoencoder-based MCT Anomaly Detection Algorithm (오토인코더를 활용한 MCT 이상탐지 알고리즘 개발)

  • Kim, Min-hee;Jin, Kyo-hong
    • Proceedings of the Korean Institute of Information and Commucation Sciences Conference
    • /
    • 2021.10a
    • /
    • pp.89-92
    • /
    • 2021
  • In a manufacturing fields, an abnormality or breakdown of equipment is a factor that causes product defects. Recently, with the spread of smart factory services, a lot of research to predict and prevent machine's failures is actively ongoing. However, there is a big difficulty in developing a classification model because the number of abnormal or failure data of the machine is severely smaller than normal data. In this paper, we present an algorithm for detecting abnormalities in an MCT at manufacturing work site depending on the differences between inputs and outputs of Autoencoder model and analyze its performance. The algorithm detects abnormalities using only features of normal data from manufacturing data of the MCT in which abnormal data does not exist.

  • PDF