• Title/Summary/Keyword: dataset

Search Result 3,881, Processing Time 0.032 seconds

Detection of Car Hacking Using One Class Classifier (단일 클래스 분류기를 사용한 차량 해킹 탐지)

  • Seo, Jae-Hyun
    • Journal of the Korea Convergence Society
    • /
    • v.9 no.6
    • /
    • pp.33-38
    • /
    • 2018
  • In this study, we try to detect new attacks for vehicle by learning only one class. We use Car-Hacking dataset, an intrusion detection dataset, which is used to evaluate classification performance. The dataset are created by logging CAN (Controller Area Network) traffic through OBD-II port from a real vehicle. The dataset have four attack types. One class classification is one of unsupervised learning methods that classifies attack class by learning only normal class. When using unsupervised learning, it difficult to achieve high efficiency because it does not use negative instances for learning. However, unsupervised learning has the advantage for classifying unlabeled data, which are new attacks. In this study, we use one class classifier to detect new attacks that are difficult to detect using signature-based rules on network intrusion detection system. The proposed method suggests a combination of parameters that detect all new attacks and show efficient classification performance for normal dataset.

Building a Korean Text Summarization Dataset Using News Articles of Social Media (신문기사와 소셜 미디어를 활용한 한국어 문서요약 데이터 구축)

  • Lee, Gyoung Ho;Park, Yo-Han;Lee, Kong Joo
    • KIPS Transactions on Software and Data Engineering
    • /
    • v.9 no.8
    • /
    • pp.251-258
    • /
    • 2020
  • A training dataset for text summarization consists of pairs of a document and its summary. As conventional approaches to building text summarization dataset are human labor intensive, it is not easy to construct large datasets for text summarization. A collection of news articles is one of the most popular resources for text summarization because it is easily accessible, large-scale and high-quality text. From social media news services, we can collect not only headlines and subheads of news articles but also summary descriptions that human editors write about the news articles. Approximately 425,000 pairs of news articles and their summaries are collected from social media. We implemented an automatic extractive summarizer and trained it on the dataset. The performance of the summarizer is compared with unsupervised models. The summarizer achieved better results than unsupervised models in terms of ROUGE score.

Correlation Analysis of Dataset Size and Accuracy of the CNN-based Malware Detection Algorithm (CNN Mobile Net 기반 악성코드 탐지 모델에서의 학습 데이터 크기와 검출 정확도의 상관관계 분석)

  • Choi, Dong Jun;Lee, Jae Woo
    • Convergence Security Journal
    • /
    • v.20 no.3
    • /
    • pp.53-60
    • /
    • 2020
  • At the present stage of the fourth industrial revolution, machine learning and artificial intelligence technologies are rapidly developing, and there is a movement to apply machine learning technology in the security field. Malicious code, including new and transformed, generates an average of 390,000 a day worldwide. Statistics show that security companies ignore or miss 31 percent of alarms. As many malicious codes are generated, it is becoming difficult for humans to detect all malicious codes. As a result, research on the detection of malware and network intrusion events through machine learning is being actively conducted in academia and industry. In international conferences and journals, research on security data analysis using deep learning, a field of machine learning, is presented. have. However, these papers focus on detection accuracy and modify several parameters to improve detection accuracy but do not consider the ratio of dataset. Therefore, this paper aims to reduce the cost and resources of many machine learning research by finding the ratio of dataset that can derive the highest detection accuracy in CNN Mobile net-based malware detection model.

Selection of the Optimum Global Natural Vegetation Mapping System for Estimating Potential Forest Area (지구상(地球上)의 잠재삼림면적(潜在森林面積)을 추정(推定)하기 위한 적정(適定) 식생도제작(植生圖製作) 시스템의 선발(選拔))

  • Cha, Gyung Soo
    • Journal of Korean Society of Forest Science
    • /
    • v.86 no.1
    • /
    • pp.25-34
    • /
    • 1997
  • The optimum global natural vegetation mapping(GNVM) system was selected as a series of the study to estimate potential forest area of the globe. To select the system, three types of GNVM systems which are simple system with Light Climatic Dataset(LCD), altitude-allowed system with LCD and altitude-allowed system with Heavy Climatic Dataset(HCD) were established and compared. The three GNVM systems spherically interpolate such spotty climate data as those observed at weather stations the world over onto $1^{\circ}{\times}1^{\circ}$ grid points, product vegetation type classification, and produce a potential natural vegetation(PNV) map and a PNV area. As a result of comparison with three GNVM systems, altitude-allowed LCD system represented natural vegetation distribution better than other versions. The difference between the simple system versus the one with altitude allowance indicated that the simple version tends to over-represent the warmer climate areas and under-represent cold and hostile climate areas. In the difference between altitude-allowed versions of LCD and HCD, HCD version tended to overestimate moist climate areas and to underestimate dry climate areas.

  • PDF

An Efficient One Class Classifier Using Gaussian-based Hyper-Rectangle Generation (가우시안 기반 Hyper-Rectangle 생성을 이용한 효율적 단일 분류기)

  • Kim, Do Gyun;Choi, Jin Young;Ko, Jeonghan
    • Journal of Korean Society of Industrial and Systems Engineering
    • /
    • v.41 no.2
    • /
    • pp.56-64
    • /
    • 2018
  • In recent years, imbalanced data is one of the most important and frequent issue for quality control in industrial field. As an example, defect rate has been drastically reduced thanks to highly developed technology and quality management, so that only few defective data can be obtained from production process. Therefore, quality classification should be performed under the condition that one class (defective dataset) is even smaller than the other class (good dataset). However, traditional multi-class classification methods are not appropriate to deal with such an imbalanced dataset, since they classify data from the difference between one class and the others that can hardly be found in imbalanced datasets. Thus, one-class classification that thoroughly learns patterns of target class is more suitable for imbalanced dataset since it only focuses on data in a target class. So far, several one-class classification methods such as one-class support vector machine, neural network and decision tree there have been suggested. One-class support vector machine and neural network can guarantee good classification rate, and decision tree can provide a set of rules that can be clearly interpreted. However, the classifiers obtained from the former two methods consist of complex mathematical functions and cannot be easily understood by users. In case of decision tree, the criterion for rule generation is ambiguous. Therefore, as an alternative, a new one-class classifier using hyper-rectangles was proposed, which performs precise classification compared to other methods and generates rules clearly understood by users as well. In this paper, we suggest an approach for improving the limitations of those previous one-class classification algorithms. Specifically, the suggested approach produces more improved one-class classifier using hyper-rectangles generated by using Gaussian function. The performance of the suggested algorithm is verified by a numerical experiment, which uses several datasets in UCI machine learning repository.

Design of Particle Swarm Optimization-based Polynomial Neural Networks (입자 군집 최적화 알고리즘 기반 다항식 신경회로망의 설계)

  • Park, Ho-Sung;Kim, Ki-Sang;Oh, Sung-Kwun
    • The Transactions of The Korean Institute of Electrical Engineers
    • /
    • v.60 no.2
    • /
    • pp.398-406
    • /
    • 2011
  • In this paper, we introduce a new architecture of PSO-based Polynomial Neural Networks (PNN) and discuss its comprehensive design methodology. The conventional PNN is based on a extended Group Method of Data Handling (GMDH) method, and utilized the polynomial order (viz. linear, quadratic, and modified quadratic) as well as the number of node inputs fixed (selected in advance by designer) at Polynomial Neurons located in each layer through a growth process of the network. Moreover it does not guarantee that the conventional PNN generated through learning results in the optimal network architecture. The PSO-based PNN results in a structurally optimized structure and comes with a higher level of flexibility that the one encountered in the conventional PNN. The PSO-based design procedure being applied at each layer of PNN leads to the selection of preferred PNs with specific local characteristics (such as the number of input variables, input variables, and the order of the polynomial) available within the PNN. In the sequel, two general optimization mechanisms of the PSO-based PNN are explored: the structural optimization is realized via PSO whereas in case of the parametric optimization we proceed with a standard least square method-based learning. To evaluate the performance of the PSO-based PNN, the model is experimented with using Gas furnace process data, and pH neutralization process data. For the characteristic analysis of the given entire data with non-linearity and the construction of efficient model, the given entire system data is partitioned into two type such as Division I(Training dataset and Testing dataset) and Division II(Training dataset, Validation dataset, and Testing dataset). A comparative analysis shows that the proposed PSO-based PNN is model with higher accuracy as well as more superb predictive capability than other intelligent models presented previously.

Using Data Mining Techniques to Predict Win-Loss in Korean Professional Baseball Games (데이터마이닝을 활용한 한국프로야구 승패예측모형 수립에 관한 연구)

  • Oh, Younhak;Kim, Han;Yun, Jaesub;Lee, Jong-Seok
    • Journal of Korean Institute of Industrial Engineers
    • /
    • v.40 no.1
    • /
    • pp.8-17
    • /
    • 2014
  • In this research, we employed various data mining techniques to build predictive models for win-loss prediction in Korean professional baseball games. The historical data containing information about players and teams was obtained from the official materials that are provided by the KBO website. Using the collected raw data, we additionally prepared two more types of dataset, which are in ratio and binary format respectively. Dividing away-team's records by the records of the corresponding home-team generated the ratio dataset, while the binary dataset was obtained by comparing the record values. We applied seven classification techniques to three (raw, ratio, and binary) datasets. The employed data mining techniques are decision tree, random forest, logistic regression, neural network, support vector machine, linear discriminant analysis, and quadratic discriminant analysis. Among 21(= 3 datasets${\times}$7 techniques) prediction scenarios, the most accurate model was obtained from the random forest technique based on the binary dataset, which prediction accuracy was 84.14%. It was also observed that using the ratio and the binary dataset helped to build better prediction models than using the raw data. From the capability of variable selection in decision tree, random forest, and stepwise logistic regression, we found that annual salary, earned run, strikeout, pitcher's winning percentage, and four balls are important winning factors of a game. This research is distinct from existing studies in that we used three different types of data and various data mining techniques for win-loss prediction in Korean professional baseball games.

A Development of Façade Dataset Construction Technology Using Deep Learning-based Automatic Image Labeling (딥러닝 기반 이미지 자동 레이블링을 활용한 건축물 파사드 데이터세트 구축 기술 개발)

  • Gu, Hyeong-Mo;Seo, Ji-Hyo;Choo, Seung-Yeon
    • Journal of the Architectural Institute of Korea Planning & Design
    • /
    • v.35 no.12
    • /
    • pp.43-53
    • /
    • 2019
  • The construction industry has made great strides in the past decades by utilizing computer programs including CAD. However, compared to other manufacturing sectors, labor productivity is low due to the high proportion of workers' knowledge-based task in addition to simple repetitive task. Therefore, the knowledge-based task efficiency of workers should be improved by recognizing the visual information of computers. A computer needs a lot of training data, such as the ImageNet project, to recognize visual information. This study, aim at proposing building facade datasets that is efficiently constructed by quickly collecting building facade data through portal site road view and automatically labeling using deep learning as part of construction of image dataset for visual recognition construction by the computer. As a method proposed in this study, we constructed a dataset for a part of Dongseong-ro, Daegu Metropolitan City and analyzed the utility and reliability of the dataset. Through this, it was confirmed that the computer could extract the significant facade information of the portal site road view by recognizing the visual information of the building facade image. Additionally, In contribution to verifying the feasibility of building construction image datasets. this study suggests the possibility of securing quantitative and qualitative facade design knowledge by extracting the facade design knowledge from any facade all over the world.

Remote Sensing Image Classification for Land Cover Mapping in Developing Countries: A Novel Deep Learning Approach

  • Lynda, Nzurumike Obianuju;Nnanna, Nwojo Agwu;Boukar, Moussa Mahamat
    • International Journal of Computer Science & Network Security
    • /
    • v.22 no.2
    • /
    • pp.214-222
    • /
    • 2022
  • Convolutional Neural networks (CNNs) are a category of deep learning networks that have proven very effective in computer vision tasks such as image classification. Notwithstanding, not much has been seen in its use for remote sensing image classification in developing countries. This is majorly due to the scarcity of training data. Recently, transfer learning technique has successfully been used to develop state-of-the art models for remote sensing (RS) image classification tasks using training and testing data from well-known RS data repositories. However, the ability of such model to classify RS test data from a different dataset has not been sufficiently investigated. In this paper, we propose a deep CNN model that can classify RS test data from a dataset different from the training dataset. To achieve our objective, we first, re-trained a ResNet-50 model using EuroSAT, a large-scale RS dataset to develop a base model then we integrated Augmentation and Ensemble learning to improve its generalization ability. We further experimented on the ability of this model to classify a novel dataset (Nig_Images). The final classification results shows that our model achieves a 96% and 80% accuracy on EuroSAT and Nig_Images test data respectively. Adequate knowledge and usage of this framework is expected to encourage research and the usage of deep CNNs for land cover mapping in cases of lack of training data as obtainable in developing countries.

Developing and Pre-Processing a Dataset using a Rhetorical Relation to Build a Question-Answering System based on an Unsupervised Learning Approach

  • Dutta, Ashit Kumar;Wahab sait, Abdul Rahaman;Keshta, Ismail Mohamed;Elhalles, Abheer
    • International Journal of Computer Science & Network Security
    • /
    • v.21 no.11
    • /
    • pp.199-206
    • /
    • 2021
  • Rhetorical relations between two text fragments are essential information and support natural language processing applications such as Question - Answering (QA) system and automatic text summarization to produce an effective outcome. Question - Answering (QA) system facilitates users to retrieve a meaningful response. There is a demand for rhetorical relation based datasets to develop such a system to interpret and respond to user requests. There are a limited number of datasets for developing an Arabic QA system. Thus, there is a lack of an effective QA system in the Arabic language. Recent research works reveal that unsupervised learning can support the QA system to reply to users queries. In this study, researchers intend to develop a rhetorical relation based dataset for implementing unsupervised learning applications. A web crawler is developed to crawl Arabic content from the web. A discourse-annotated corpus is generated using the rhetorical structural theory. A Naïve Bayes based QA system is developed to evaluate the performance of datasets. The outcome shows that the performance of the QA system is improved with proposed dataset and able to answer user queries with an appropriate response. In addition, the results on fine-grained and coarse-grained relations reveal that the dataset is highly reliable.