• Title/Summary/Keyword: unsupervised learning

Search Result 477, Processing Time 0.025 seconds

The Use of Unsupervised Machine Learning for the Attenuation of Seismic Noise (탄성파 자료 잡음 제거를 위한 비지도 학습 연구)

  • Kim, Sujeong;Jun, Hyunggu
    • Geophysics and Geophysical Exploration
    • /
    • v.25 no.2
    • /
    • pp.71-84
    • /
    • 2022
  • When acquiring seismic data, various types of simultaneously recorded seismic noise hinder accurate interpretation. Therefore, it is essential to attenuate this noise during the processing of seismic data and research on seismic noise attenuation. For this purpose, machine learning is extensively used. This study attempts to attenuate noise in prestack seismic data using unsupervised machine learning. Three unsupervised machine learning models, N2NUNET, PATCHUNET, and DDUL, are trained and applied to synthetic and field prestack seismic data to attenuate the noise and leave clean seismic data. The results are qualitatively and quantitatively analyzed and demonstrated that all three unsupervised learning models succeeded in removing seismic noise from both synthetic and field data. Of the three, the N2NUNET model performed the worst, and the PATCHUNET and DDUL models produced almost identical results, although the DDUL model performed slightly better.

Evaluating Unsupervised Deep Learning Models for Network Intrusion Detection Using Real Security Event Data

  • Jang, Jiho;Lim, Dongjun;Seong, Changmin;Lee, JongHun;Park, Jong-Geun;Cheong, Yun-Gyung
    • International journal of advanced smart convergence
    • /
    • v.11 no.4
    • /
    • pp.10-19
    • /
    • 2022
  • AI-based Network Intrusion Detection Systems (AI-NIDS) detect network attacks using machine learning and deep learning models. Recently, unsupervised AI-NIDS methods are getting more attention since there is no need for labeling, which is crucial for building practical NIDS systems. This paper aims to test the impact of designing autoencoder models that can be applied to unsupervised an AI-NIDS in real network systems. We collected security events of legacy network security system and carried out an experiment. We report the results and discuss the findings.

Bagging deep convolutional autoencoders trained with a mixture of real data and GAN-generated data

  • Hu, Cong;Wu, Xiao-Jun;Shu, Zhen-Qiu
    • KSII Transactions on Internet and Information Systems (TIIS)
    • /
    • v.13 no.11
    • /
    • pp.5427-5445
    • /
    • 2019
  • While deep neural networks have achieved remarkable performance in representation learning, a huge amount of labeled training data are usually required by supervised deep models such as convolutional neural networks. In this paper, we propose a new representation learning method, namely generative adversarial networks (GAN) based bagging deep convolutional autoencoders (GAN-BDCAE), which can map data to diverse hierarchical representations in an unsupervised fashion. To boost the size of training data, to train deep model and to aggregate diverse learning machines are the three principal avenues towards increasing the capabilities of representation learning of neural networks. We focus on combining those three techniques. To this aim, we adopt GAN for realistic unlabeled sample generation and bagging deep convolutional autoencoders (BDCAE) for robust feature learning. The proposed method improves the discriminative ability of learned feature embedding for solving subsequent pattern recognition problems. We evaluate our approach on three standard benchmarks and demonstrate the superiority of the proposed method compared to traditional unsupervised learning methods.

Detection of Car Hacking Using One Class Classifier (단일 클래스 분류기를 사용한 차량 해킹 탐지)

  • Seo, Jae-Hyun
    • Journal of the Korea Convergence Society
    • /
    • v.9 no.6
    • /
    • pp.33-38
    • /
    • 2018
  • In this study, we try to detect new attacks for vehicle by learning only one class. We use Car-Hacking dataset, an intrusion detection dataset, which is used to evaluate classification performance. The dataset are created by logging CAN (Controller Area Network) traffic through OBD-II port from a real vehicle. The dataset have four attack types. One class classification is one of unsupervised learning methods that classifies attack class by learning only normal class. When using unsupervised learning, it difficult to achieve high efficiency because it does not use negative instances for learning. However, unsupervised learning has the advantage for classifying unlabeled data, which are new attacks. In this study, we use one class classifier to detect new attacks that are difficult to detect using signature-based rules on network intrusion detection system. The proposed method suggests a combination of parameters that detect all new attacks and show efficient classification performance for normal dataset.

Performance Evaluation of One Class Classification to detect anomalies of NIDS (NIDS의 비정상 행위 탐지를 위한 단일 클래스 분류성능 평가)

  • Seo, Jae-Hyun
    • Journal of the Korea Convergence Society
    • /
    • v.9 no.11
    • /
    • pp.15-21
    • /
    • 2018
  • In this study, we try to detect anomalies on the network intrusion detection system by learning only one class. We use KDD CUP 1999 dataset, an intrusion detection dataset, which is used to evaluate classification performance. One class classification is one of unsupervised learning methods that classifies attack class by learning only normal class. When using unsupervised learning, it difficult to achieve relatively high classification efficiency because it does not use negative instances for learning. However, unsupervised learning has the advantage for classifying unlabeled data. In this study, we use one class classifiers based on support vector machines and density estimation to detect new unknown attacks. The test using the classifier based on density estimation has shown relatively better performance and has a detection rate of about 96% while maintaining a low FPR for the new attacks.

Classification of Traffic Flows into QoS Classes by Unsupervised Learning and KNN Clustering

  • Zeng, Yi;Chen, Thomas M.
    • KSII Transactions on Internet and Information Systems (TIIS)
    • /
    • v.3 no.2
    • /
    • pp.134-146
    • /
    • 2009
  • Traffic classification seeks to assign packet flows to an appropriate quality of service(QoS) class based on flow statistics without the need to examine packet payloads. Classification proceeds in two steps. Classification rules are first built by analyzing traffic traces, and then the classification rules are evaluated using test data. In this paper, we use self-organizing map and K-means clustering as unsupervised machine learning methods to identify the inherent classes in traffic traces. Three clusters were discovered, corresponding to transactional, bulk data transfer, and interactive applications. The K-nearest neighbor classifier was found to be highly accurate for the traffic data and significantly better compared to a minimum mean distance classifier.

Detecting Anomalies in Time-Series Data using Unsupervised Learning and Analysis on Infrequent Signatures

  • Bian, Xingchao
    • Journal of IKEEE
    • /
    • v.24 no.4
    • /
    • pp.1011-1016
    • /
    • 2020
  • We propose a framework called Stacked Gated Recurrent Unit - Infrequent Residual Analysis (SG-IRA) that detects anomalies in time-series data that can be trained on streams of raw sensor data without any pre-labeled dataset. To enable such unsupervised learning, SG-IRA includes an estimation model that uses a stacked Gated Recurrent Unit (GRU) structure and an analysis method that detects anomalies based on the difference between the estimated value and the actual measurement (residual). SG-IRA's residual analysis method dynamically adapts the detection threshold from the population using frequency analysis, unlike the baseline model that relies on a constant threshold. In this paper, SG-IRA is evaluated using the industrial control systems (ICS) datasets. SG-IRA improves the detection performance (F1 score) by 5.9% compared to the baseline model.

Deep Video Stabilization via Optical Flow in Unstable Scenes (동영상 안정화를 위한 옵티컬 플로우의 비지도 학습 방법)

  • Bohee Lee;Kwangsu Kim
    • Journal of Intelligence and Information Systems
    • /
    • v.29 no.2
    • /
    • pp.115-127
    • /
    • 2023
  • Video stabilization is one of the camera technologies that the importance is gradually increasing as the personal media market has recently become huge. For deep learning-based video stabilization, existing methods collect pairs of video datas before and after stabilization, but it takes a lot of time and effort to create synchronized datas. Recently, to solve this problem, unsupervised learning method using only unstable video data has been proposed. In this paper, we propose a network structure that learns the stabilized trajectory only with the unstable video image without the pair of unstable and stable video pair using the Convolutional Auto Encoder structure, one of the unsupervised learning methods. Optical flow data is used as network input and output, and optical flow data was mapped into grid units to simplify the network and minimize noise. In addition, to generate a stabilized trajectory with an unsupervised learning method, we define the loss function that smoothing the input optical flow data. And through comparison of the results, we confirmed that the network is learned as intended by the loss function.

Ensemble Model for Urine Spectrum Analysis Based on Hybrid Machine Learning (혼합 기계 학습 기반 소변 스펙트럼 분석 앙상블 모델)

  • Choi, Jaehyeok;Chung, Mokdong
    • Journal of Korea Multimedia Society
    • /
    • v.23 no.8
    • /
    • pp.1059-1065
    • /
    • 2020
  • In hospitals, nurses are subjectively determining the urine status to check the kidneys and circulatory system of patients whose statuses are related to patients with kidney disease, critically ill patients, and nursing homes before and after surgery. To improve this problem, this paper proposes a urine spectrum analysis system which clusters urine test results based on a hybrid machine learning model consists of unsupervised learning and supervised learning. The proposed system clusters the spectral data using unsupervised learning in the first part, and classifies them using supervised learning in the second part. The results of the proposed urine spectrum analysis system using a mixed model are evaluated with the results of pure supervised learning. This paper is expected to provide better services than existing medical services to patients by solving the shortage of nurses, shortening of examination time, and subjective evaluation in hospitals.

Deep Learning Based Monocular Depth Estimation: Survey

  • Lee, Chungkeun;Shim, Dongseok;Kim, H. Jin
    • Journal of Positioning, Navigation, and Timing
    • /
    • v.10 no.4
    • /
    • pp.297-305
    • /
    • 2021
  • Monocular depth estimation helps the robot to understand the surrounding environments in 3D. Especially, deep-learning-based monocular depth estimation has been widely researched, because it may overcome the scale ambiguity problem, which is a main issue in classical methods. Those learning based methods can be mainly divided into three parts: supervised learning, unsupervised learning, and semi-supervised learning. Supervised learning trains the network from dense ground-truth depth information, unsupervised one trains it from images sequences and semi-supervised one trains it from stereo images and sparse ground-truth depth. We describe the basics of each method, and then explain the recent research efforts to enhance the depth estimation performance.