• Title/Summary/Keyword: Dataset Generation

Search Result 196, Processing Time 0.032 seconds

A Data Sampling Technique for Secure Dataset Using Weight VAE Oversampling(W-VAE) (가중치 VAE 오버샘플링(W-VAE)을 이용한 보안데이터셋 샘플링 기법 연구)

  • Kang, Hanbada;Lee, Jaewoo
    • Journal of the Korea Institute of Information and Communication Engineering
    • /
    • v.26 no.12
    • /
    • pp.1872-1879
    • /
    • 2022
  • Recently, with the development of artificial intelligence technology, research to use artificial intelligence to detect hacking attacks is being actively conducted. However, the fact that security data is a representative imbalanced data is recognized as a major obstacle in composing the learning data, which is the key to the development of artificial intelligence models. Therefore, in this paper, we propose a W-VAE oversampling technique that applies VAE, a deep learning generation model, to data extraction for oversampling, and sets the number of oversampling for each class through weight calculation using K-NN for sampling. In this paper, a total of five oversampling techniques such as ROS, SMOTE, and ADASYN were applied through NSL-KDD, an open network security dataset. The oversampling method proposed in this paper proved to be the most effective sampling method compared to the existing oversampling method through the F1-Score evaluation index.

Hard Example Generation by Novel View Synthesis for 3-D Pose Estimation (3차원 자세 추정 기법의 성능 향상을 위한 임의 시점 합성 기반의 고난도 예제 생성)

  • Minji Kim;Sungchan Kim
    • IEMEK Journal of Embedded Systems and Applications
    • /
    • v.19 no.1
    • /
    • pp.9-17
    • /
    • 2024
  • It is widely recognized that for 3D human pose estimation (HPE), dataset acquisition is expensive and the effectiveness of augmentation techniques of conventional visual recognition tasks is limited. We address these difficulties by presenting a simple but effective method that augments input images in terms of viewpoints when training a 3D human pose estimation (HPE) model. Our intuition is that meaningful variants of the input images for HPE could be obtained by viewing a human instance in the images from an arbitrary viewpoint different from that in the original images. The core idea is to synthesize new images that have self-occlusion and thus are difficult to predict at different viewpoints even with the same pose of the original example. We incorporate this idea into the training procedure of the 3D HPE model as an augmentation stage of the input samples. We show that a strategy for augmenting the synthesized example should be carefully designed in terms of the frequency of performing the augmentation and the selection of viewpoints for synthesizing the samples. To this end, we propose a new metric to measure the prediction difficulty of input images for 3D HPE in terms of the distance between corresponding keypoints on both sides of a human body. Extensive exploration of the space of augmentation probability choices and example selection according to the proposed distance metric leads to a performance gain of up to 6.2% on Human3.6M, the well-known pose estimation dataset.

Development of Deep Learning Structure for Defective Pixel Detection of Next-Generation Smart LED Display Board using Imaging Device (영상장치를 이용한 차세대 스마트 LED 전광판의 불량픽셀 검출을 위한 딥러닝 구조 개발)

  • Sun-Gu Lee;Tae-Yoon Lee;Seung-Ho Lee
    • Journal of IKEEE
    • /
    • v.27 no.3
    • /
    • pp.345-349
    • /
    • 2023
  • In this paper, we propose a study on the development of deep learning structure for defective pixel detection of next-generation smart LED display board using imaging device. In this research, a technique utilizing imaging devices and deep learning is introduced to automatically detect defects in outdoor LED billboards. Through this approach, the effective management of LED billboards and the resolution of various errors and issues are aimed. The research process consists of three stages. Firstly, the planarized image data of the billboard is processed through calibration to completely remove the background and undergo necessary preprocessing to generate a training dataset. Secondly, the generated dataset is employed to train an object recognition network. This network is composed of a Backbone and a Head. The Backbone employs CSP-Darknet to extract feature maps, while the Head utilizes extracted feature maps as the basis for object detection. Throughout this process, the network is adjusted to align the Confidence score and Intersection over Union (IoU) error, sustaining continuous learning. In the third stage, the created model is employed to automatically detect defective pixels on actual outdoor LED billboards. The proposed method, applied in this paper, yielded results from accredited measurement experiments that achieved 100% detection of defective pixels on real LED billboards. This confirms the improved efficiency in managing and maintaining LED billboards. Such research findings are anticipated to bring about a revolutionary advancement in the management of LED billboards.

Intrusion Detection Method Using Unsupervised Learning-Based Embedding and Autoencoder (비지도 학습 기반의 임베딩과 오토인코더를 사용한 침입 탐지 방법)

  • Junwoo Lee;Kangseok Kim
    • KIPS Transactions on Software and Data Engineering
    • /
    • v.12 no.8
    • /
    • pp.355-364
    • /
    • 2023
  • As advanced cyber threats continue to increase in recent years, it is difficult to detect new types of cyber attacks with existing pattern or signature-based intrusion detection method. Therefore, research on anomaly detection methods using data learning-based artificial intelligence technology is increasing. In addition, supervised learning-based anomaly detection methods are difficult to use in real environments because they require sufficient labeled data for learning. Research on an unsupervised learning-based method that learns from normal data and detects an anomaly by finding a pattern in the data itself has been actively conducted. Therefore, this study aims to extract a latent vector that preserves useful sequence information from sequence log data and develop an anomaly detection learning model using the extracted latent vector. Word2Vec was used to create a dense vector representation corresponding to the characteristics of each sequence, and an unsupervised autoencoder was developed to extract latent vectors from sequence data expressed as dense vectors. The developed autoencoder model is a recurrent neural network GRU (Gated Recurrent Unit) based denoising autoencoder suitable for sequence data, a one-dimensional convolutional neural network-based autoencoder to solve the limited short-term memory problem that GRU can have, and an autoencoder combining GRU and one-dimensional convolution was used. The data used in the experiment is time-series-based NGIDS (Next Generation IDS Dataset) data, and as a result of the experiment, an autoencoder that combines GRU and one-dimensional convolution is better than a model using a GRU-based autoencoder or a one-dimensional convolution-based autoencoder. It was efficient in terms of learning time for extracting useful latent patterns from training data, and showed stable performance with smaller fluctuations in anomaly detection performance.

Updating Building Data in Digital Topographic Map Based on Matching and Generation of Update History Record (수치지도 건물데이터의 매칭 기반 갱신 및 이력 데이터 생성)

  • Park, Seul A;Yu, Ki Yun;Park, Woo Jin
    • Journal of the Korean Society of Surveying, Geodesy, Photogrammetry and Cartography
    • /
    • v.32 no.4_1
    • /
    • pp.311-318
    • /
    • 2014
  • The data of buildings and structures take over large portions of the mapping database with large numbers. Furthermore, those shapes and attributes of building data continuously change over time. Due to those factors, the efficient methodology of updating database for following the most recent data become necessarily. This study has purposed on extracting needed data, which has been changed, by using overlaying analysis of new and old dataset, during updating processes. Following to procedures, we firstly searched for matching pairs of objects from each dataset, and defined the classification algorithm for building updating cases by comparing; those of shape updating cases are divided into 8 cases, while those of attribute updating cases are divided into 4 cases. Also, two updated dataset are set to be automatically saved. For the study, we selected few guidelines; the layer of digital topographic map 1:5000 for the targeted updating data, the building layer of Korea Address Information System map for the reference data, as well as build-up areas in Gwanak-gu, Seoul for the test area. The result of study updated 82.1% in shape and 34.5% in attribute building objects among all.

Predicting Power Generation Patterns Using the Wind Power Data (풍력 데이터를 이용한 발전 패턴 예측)

  • Suh, Dong-Hyok;Kim, Kyu-Ik;Kim, Kwang-Deuk;Ryu, Keun-Ho
    • Journal of the Korea Society of Computer and Information
    • /
    • v.16 no.11
    • /
    • pp.245-253
    • /
    • 2011
  • Due to the imprudent spending of the fossil fuels, the environment was contaminated seriously and the exhaustion problems of the fossil fuels loomed large. Therefore people become taking a great interest in alternative energy resources which can solve problems of fossil fuels. The wind power energy is one of the most interested energy in the new and renewable energy. However, the plants of wind power energy and the traditional power plants should be balanced between the power generation and the power consumption. Therefore, we need analysis and prediction to generate power efficiently using wind energy. In this paper, we have performed a research to predict power generation patterns using the wind power data. Prediction approaches of datamining area can be used for building a prediction model. The research steps are as follows: 1) we performed preprocessing to handle the missing values and anomalous data. And we extracted the characteristic vector data. 2) The representative patterns were found by the MIA(Mean Index Adequacy) measure and the SOM(Self-Organizing Feature Map) clustering approach using the normalized dataset. We assigned the class labels to each data. 3) We built a new predicting model about the wind power generation with classification approach. In this experiment, we built a forecasting model to predict wind power generation patterns using the decision tree.

Development of SNP marker set for marker-assisted backcrossing (MABC) in cultivating tomato varieties

  • Park, GiRim;Jang, Hyun A;Jo, Sung-Hwan;Park, Younghoon;Oh, Sang-Keun;Nam, Moon
    • Korean Journal of Agricultural Science
    • /
    • v.45 no.3
    • /
    • pp.385-400
    • /
    • 2018
  • Marker-assisted backcrossing (MABC) is useful for selecting offspring with a highly recovered genetic background for a recurrent parent at early generation unlike rice and other field crops. Molecular marker sets applicable to practical MABC are scarce in vegetable crops including tomatoes. In this study, we used the National Center for Biotechnology Information- short read archive (NCBI-SRA) database that provided the whole genome sequences of 234 tomato accessions and selected 27,680 tag-single nucleotide polymorphisms (tag-SNPs) that can identify haplotypes in the tomato genome. From this SNP dataset, a total of 143 tag-SNPs that have a high polymorphism information content (PIC) value (> 0.3) and are physically evenly distributed on each chromosome were selected as a MABC marker set. This marker set was tested for its polymorphism in each pairwise cross combination constructed with 124 of the 234 tomato accessions, and a relatively high number of SNP markers polymorphic for the cross combination was observed. The reliability of the MABC SNP set was assessed by converting 18 SNPs into Luna probe-based high-resolution melting (HRM) markers and genotyping nine tomato accessions. The results show that the SNP information and HRM marker genotype matched in 98.6% of the experiment data points, indicating that our sequence analysis pipeline for SNP mining worked successfully. The tag-SNP set for the MABC developed in this study can be useful for not only a practical backcrossing program but also for cultivar identification and F1 seed purity test in tomatoes.

A Basic Study on Enhancement of Input data Quality for the CFD Model Using Airborne LiDAR data (항공 LiDAR 데이터를 활용한 CFD 모델 입력자료 품질 향상에 대한 기초연구)

  • Park, Myeong-Ha;An, Seung-Man;Choi, Yun-Soo;Jeong, In-Hun;Jeon, Byeong-Kuk
    • Spatial Information Research
    • /
    • v.20 no.1
    • /
    • pp.27-38
    • /
    • 2012
  • The recent development of CFD techniques are being involved w ith Environmental Impact Assessment and Environmental DesignroThey are being applied to the Site Planning and Engineering Design works as a new trendroHowever, CFD laboratory works are not extended to the field works in Industrial Project due to inaccuracy of the data input process that is cause by absence of regional height informationsroHence, in this study, we promote to build a new initial input data processing steps and algorithms for CFD Model generation. ENVI-met model is very popular, efficient, and freely downloadable CFD model. Light Detection And Ranging (LiDAR) are well known state of art technology and dataset proving a reliable accuracy for CFD. We use LiDAR data as a input source for CFD input producing process and algorithm development and evaluation. CFD initial input data generation process and results derived from am development and set is very useful and efficient for rapid CFD input data producing and maklomore reliable CFD Model forec st for atmospheric and climatic analysis for planning and design engineering industry.

Automated Generation of Multi-Scale Map Database for Web Map Services (웹 지도서비스를 위한 다축척 지도 데이터셋 자동생성 기법 연구)

  • Park, Woo Jin;Bang, Yoon Sik;Yu, Ki Yun
    • Journal of the Korean Society of Surveying, Geodesy, Photogrammetry and Cartography
    • /
    • v.30 no.5
    • /
    • pp.435-444
    • /
    • 2012
  • Although the multi-scale map database should be constructed for the web map services and location-based services, much part of generation process is based on the manual editing. In this study, the map generalization methodology for automatic construction of multi-scale database from the primary data is proposed. Moreover, the generalization methodology is applied to the real map data and the prototype of multi-scale map dataset is generated. Among the generalization operators, selection/elimination, simplification and amalgamation/aggregation is applied in organized manner. The algorithm and parameters for generalization is determined experimentally considering T$\ddot{o}$pfer's radical law, minimum drawable object of map and visual aspect. The target scale level is five(1:1,000, 1:5,000, 1:25,000, 1:100,000, 1:500,000) and for the target data, new address data and digital topographic map is used.

Crowd Behavior Detection using Convolutional Neural Network (컨볼루션 뉴럴 네트워크를 이용한 군중 행동 감지)

  • Ullah, Waseem;Ullah, Fath U Min;Baik, Sung Wook;Lee, Mi Young
    • The Journal of Korean Institute of Next Generation Computing
    • /
    • v.15 no.6
    • /
    • pp.7-14
    • /
    • 2019
  • The automatic monitoring and detection of crowd behavior in the surveillance videos has obtained significant attention in the field of computer vision due to its vast applications such as security, safety and protection of assets etc. Also, the field of crowd analysis is growing upwards in the research community. For this purpose, it is very necessary to detect and analyze the crowd behavior. In this paper, we proposed a deep learning-based method which detects abnormal activities in surveillance cameras installed in a smart city. A fine-tuned VGG-16 model is trained on publicly available benchmark crowd dataset and is tested on real-time streaming. The CCTV camera captures the video stream, when abnormal activity is detected, an alert is generated and is sent to the nearest police station to take immediate action before further loss. We experimentally have proven that the proposed method outperforms over the existing state-of-the-art techniques.