• 제목/요약/키워드: Dataset Generation

검색결과 194건 처리시간 0.025초

A Brief Survey into the Field of Automatic Image Dataset Generation through Web Scraping and Query Expansion

  • Bart Dikmans;Dongwann Kang
    • Journal of Information Processing Systems
    • /
    • 제19권5호
    • /
    • pp.602-613
    • /
    • 2023
  • High-quality image datasets are in high demand for various applications. With many online sources providing manually collected datasets, a persisting challenge is to fully automate the dataset collection process. In this study, we surveyed an automatic image dataset generation field through analyzing a collection of existing studies. Moreover, we examined fields that are closely related to automated dataset generation, such as query expansion, web scraping, and dataset quality. We assess how both noise and regional search engine differences can be addressed using an automated search query expansion focused on hypernyms, allowing for user-specific manual query expansion. Combining these aspects provides an outline of how a modern web scraping application can produce large-scale image datasets.

Default Prediction for Real Estate Companies with Imbalanced Dataset

  • Dong, Yuan-Xiang;Xiao, Zhi;Xiao, Xue
    • Journal of Information Processing Systems
    • /
    • 제10권2호
    • /
    • pp.314-333
    • /
    • 2014
  • When analyzing default predictions in real estate companies, the number of non-defaulted cases always greatly exceeds the defaulted ones, which creates the two-class imbalance problem. This lowers the ability of prediction models to distinguish the default sample. In order to avoid this sample selection bias and to improve the prediction model, this paper applies a minority sample generation approach to create new minority samples. The logistic regression, support vector machine (SVM) classification, and neural network (NN) classification use an imbalanced dataset. They were used as benchmarks with a single prediction model that used a balanced dataset corrected by the minority samples generation approach. Instead of using prediction-oriented tests and the overall accuracy, the true positive rate (TPR), the true negative rate (TNR), G-mean, and F-score are used to measure the performance of default prediction models for imbalanced dataset. In this paper, we describe an empirical experiment that used a sampling of 14 default and 315 non-default listed real estate companies in China and report that most results using single prediction models with a balanced dataset generated better results than an imbalanced dataset.

딥러닝 기반의 한글 폰트 연구를 위한 한글 폰트 데이터셋 (Hangul Font Dataset for Korean Font Research Based on Deep Learning)

  • 고홍희;이현수;석정재;;최재영
    • 정보처리학회논문지:소프트웨어 및 데이터공학
    • /
    • 제10권2호
    • /
    • pp.73-78
    • /
    • 2021
  • 최근 딥러닝에 대한 관심이 증가하면서 이를 이용한 다양한 분야에서 연구가 진행되고 있다. 그러나 딥러닝 기반의 생성 모델을 이용하는 폰트의 자동 생성 연구들은 로마자 및 한자와 같은 몇 언어들에 국한되어 연구되고 있다. 한글 폰트 디자인은 매우 큰 시간과 비용이 들어가는 작업으로, 딥러닝을 이용하면 손쉽게 생성할 수 있다. 한글 폰트를 생성하는 연구는 딥러닝 기반의 생성 모델들과 발맞추기 위해 프로세스 자동화 관점에서 한글 폰트 데이터셋을 준비하는 것이 중요하다. 이를 위하여 본 논문에서는 딥러닝 기반의 한글 폰트 연구를 위한 한글 폰트 데이터셋을 제안하고. 그 데이터셋을 구성하는 방법을 기술한다. 본 논문에서 제안하는 한글 폰트 데이터셋을 기반으로 딥러닝 한글 폰트 생성 어플리케이션에 적용하는 과정을 통해 제안하는 데이터셋 구성의 유용성을 보인다.

Construction of a Video Dataset for Face Tracking Benchmarking Using a Ground Truth Generation Tool

  • Do, Luu Ngoc;Yang, Hyung Jeong;Kim, Soo Hyung;Lee, Guee Sang;Na, In Seop;Kim, Sun Hee
    • International Journal of Contents
    • /
    • 제10권1호
    • /
    • pp.1-11
    • /
    • 2014
  • In the current generation of smart mobile devices, object tracking is one of the most important research topics for computer vision. Because human face tracking can be widely used for many applications, collecting a dataset of face videos is necessary for evaluating the performance of a tracker and for comparing different approaches. Unfortunately, the well-known benchmark datasets of face videos are not sufficiently diverse. As a result, it is difficult to compare the accuracy between different tracking algorithms in various conditions, namely illumination, background complexity, and subject movement. In this paper, we propose a new dataset that includes 91 face video clips that were recorded in different conditions. We also provide a semi-automatic ground-truth generation tool that can easily be used to evaluate the performance of face tracking systems. This tool helps to maintain the consistency of the definitions for the ground-truth in each frame. The resulting video data set is used to evaluate well-known approaches and test their efficiency.

Mask R-CNN을 이용한 물체인식 및 개체분할의 학습 데이터셋 자동 생성 (Automatic Dataset Generation of Object Detection and Instance Segmentation using Mask R-CNN)

  • 조현준;김다윗;송재복
    • 로봇학회논문지
    • /
    • 제14권1호
    • /
    • pp.31-39
    • /
    • 2019
  • A robot usually adopts ANN (artificial neural network)-based object detection and instance segmentation algorithms to recognize objects but creating datasets for these algorithms requires high labeling costs because the dataset should be manually labeled. In order to lower the labeling cost, a new scheme is proposed that can automatically generate a training images and label them for specific objects. This scheme uses an instance segmentation algorithm trained to give the masks of unknown objects, so that they can be obtained in a simple environment. The RGB images of objects can be obtained by using these masks, and it is necessary to label the classes of objects through a human supervision. After obtaining object images, they are synthesized with various background images to create new images. Labeling the synthesized images is performed automatically using the masks and previously input object classes. In addition, human intervention is further reduced by using the robot arm to collect object images. The experiments show that the performance of instance segmentation trained through the proposed method is equivalent to that of the real dataset and that the time required to generate the dataset can be significantly reduced.

Geometric and Semantic Improvement for Unbiased Scene Graph Generation

  • Ruhui Zhang;Pengcheng Xu;Kang Kang;You Yang
    • KSII Transactions on Internet and Information Systems (TIIS)
    • /
    • 제17권10호
    • /
    • pp.2643-2657
    • /
    • 2023
  • Scene graphs are structured representations that can clearly convey objects and the relationships between them, but are often heavily biased due to the highly skewed, long-tailed relational labeling in the dataset. Indeed, the visual world itself and its descriptions are biased. Therefore, Unbiased Scene Graph Generation (USGG) prefers to train models to eliminate long-tail effects as much as possible, rather than altering the dataset directly. To this end, we propose Geometric and Semantic Improvement (GSI) for USGG to mitigate this issue. First, to fully exploit the feature information in the images, geometric dimension and semantic dimension enhancement modules are designed. The geometric module is designed from the perspective that the position information between neighboring object pairs will affect each other, which can improve the recall rate of the overall relationship in the dataset. The semantic module further processes the embedded word vector, which can enhance the acquisition of semantic information. Then, to improve the recall rate of the tail data, the Class Balanced Seesaw Loss (CBSLoss) is designed for the tail data. The recall rate of the prediction is improved by penalizing the body or tail relations that are judged incorrectly in the dataset. The experimental findings demonstrate that the GSI method performs better than mainstream models in terms of the mean Recall@K (mR@K) metric in three tasks. The long-tailed imbalance in the Visual Genome 150 (VG150) dataset is addressed better using the GSI method than by most of the existing methods.

A Virtual Battlefield Situation Dataset Generation for Battlefield Analysis based on Artificial Intelligence

  • Cho, Eunji;Jin, Soyeon;Shin, Yukyung;Lee, Woosin
    • 한국컴퓨터정보학회논문지
    • /
    • 제27권6호
    • /
    • pp.33-42
    • /
    • 2022
  • 기존의 지능형 지휘통제체계 연구에서는 지휘관의 전장 상황 질문에 대한 분석 결과를 지식베이스 기반 상황 데이터에서 정보를 추출하여 제공해주고 있다. 하지만, 다양한 표현의 자연어가 사용된 정·첩보를 문맥에 맞게 분석하는 것이 상황 분석에 있어 중요해지면서 인공지능을 사용한 전장 상황 분석 연구가 진행되고 있다. 본 논문에서는 전장 상황 분석용 인공지능 개발에 필요한 데이터 셋을 제공하기 위해 전장 상황 모의 시나리오 기반 가설 데이터 셋 생성 방법을 제안한다. 가설 데이터 셋은 실제 전장 환경이 고려된 모의 시나리오에서 전장 지식요소를 식별하여 생성한다. 먼저 후보가설을 생성하면 자동으로 단위가설이 생성된다. 단위가설을 조합하여 유사 식별 가설 조합을 만들고, 연관된 후보가설을 그룹화하여 집합가설을 생성한다. 제안하는 방법으로 데이터 셋을 생성할 수 있음을 확인하기 위해 생성기 SW를 구현하였고, 생성기 SW로 가설 데이터 셋을 생성할 수 있음을 확인하였다.

A Study on the Generation of Datasets for Applied AI to OLED Life Prediction

  • CHUNG, Myung-Ae;HAN, Dong Hun;AHN, Seongdeok;KANG, Min Soo
    • 한국인공지능학회지
    • /
    • 제10권2호
    • /
    • pp.7-11
    • /
    • 2022
  • OLED displays cannot be used permanently due to burn-in or generation of dark spots due to degradation. Therefore, the time when the display can operate normally is very important. It is close to impossible to physically measure the time when the display operates normally. Therefore, the time that works normally should be predicted in a way other than a physical way. Therefore, if you do computer simulations based on artificial intelligence, you can increase the accuracy of prediction by saving time and continuous learning. Therefore, if we do computer simulations based on artificial intelligence, we can increase the accuracy of prediction by saving time and continuous learning. In this paper, a dataset in the form of development from generation to diffusion of dark spots, which is one of the causes related to the life of OLED, was generated by applying the finite element method. The dark spots were generated in nine conditions, such as 0.1 to 2.0 ㎛ with the size of pinholes, the number was 10 to 100, and 50% with water content. The learning data created in this way may be a criterion for generating an artificial intelligence-based dataset.

대용량 유동해석 데이터에서의 중요도 기반 스트림라인 생성 방법 (Method for Importance based Streamline Generation on the Massive Fluid Dynamics Dataset)

  • 이중연;김민아;이세훈
    • 한국콘텐츠학회논문지
    • /
    • 제18권6호
    • /
    • pp.27-37
    • /
    • 2018
  • 스트림라인 생성은 유동해석 데이터에서 유동의 흐름을 해석하기 위한 대표적인 가시화 기법이다. 그러나 효과적인 스트림라인 배치를 위한 씨드 포인트의 위치를 결정하는 것은 매우 어려운 문제이다. 한편, 대용량의 유동해석 데이터에서 씨드 포인트 결정과 스트림라인 생성 계산은 매우 오랜 시간을 필요로 한다. 본 논문에서는 효과적인 스트림라인 배치를 위해 유동해석 데이터의 중요도를 기반으로 한 씨드 포인트 결정 방법과 분산병렬 가시화 시스템 환경에서의 병렬 처리 기법을 제안한다. 또한, GLOVE 가시화 시스템에서 실제 유동해석 데이터를 이용한 구현 결과를 소개하고 이를 통해 본 논문의 제안 방법을 검증하고자 한다.

국방용 합성이미지 데이터셋 생성을 위한 대립훈련신경망 기술 적용 연구 (Synthetic Image Dataset Generation for Defense using Generative Adversarial Networks)

  • 양훈민
    • 한국군사과학기술학회지
    • /
    • 제22권1호
    • /
    • pp.49-59
    • /
    • 2019
  • Generative adversarial networks(GANs) have received great attention in the machine learning field for their capacity to model high-dimensional and complex data distribution implicitly and generate new data samples from the model distribution. This paper investigates the model training methodology, architecture, and various applications of generative adversarial networks. Experimental evaluation is also conducted for generating synthetic image dataset for defense using two types of GANs. The first one is for military image generation utilizing the deep convolutional generative adversarial networks(DCGAN). The other is for visible-to-infrared image translation utilizing the cycle-consistent generative adversarial networks(CycleGAN). Each model can yield a great diversity of high-fidelity synthetic images compared to training ones. This result opens up the possibility of using inexpensive synthetic images for training neural networks while avoiding the enormous expense of collecting large amounts of hand-annotated real dataset.