• 제목/요약/키워드: Dataset Training

검색결과 640건 처리시간 0.022초

Removing Out - Of - Distribution Samples on Classification Task

  • Dang, Thanh-Vu;Vo, Hoang-Trong;Yu, Gwang-Hyun;Lee, Ju-Hwan;Nguyen, Huy-Toan;Kim, Jin-Young
    • 스마트미디어저널
    • /
    • 제9권3호
    • /
    • pp.80-89
    • /
    • 2020
  • Out - of - distribution (OOD) samples are frequently encountered when deploying a classification model in plenty of real-world machine learning-based applications. Those samples are normally sampling far away from the training distribution, but many classifiers still assign them high reliability to belong to one of the training categories. In this study, we address the problem of removing OOD examples by estimating marginal density estimation using variational autoencoder (VAE). We also investigate other proper methods, such as temperature scaling, Gaussian discrimination analysis, and label smoothing. We use Chonnam National University (CNU) weeds dataset as the in - distribution dataset and CIFAR-10, CalTeach as the OOD datasets. Quantitative results show that the proposed framework can reject the OOD test samples with a suitable threshold.

Classification-Based Approach for Hybridizing Statistical and Rule-Based Machine Translation

  • Park, Eun-Jin;Kwon, Oh-Woog;Kim, Kangil;Kim, Young-Kil
    • ETRI Journal
    • /
    • 제37권3호
    • /
    • pp.541-550
    • /
    • 2015
  • In this paper, we propose a classification-based approach for hybridizing statistical machine translation and rulebased machine translation. Both the training dataset used in the learning of our proposed classifier and our feature extraction method affect the hybridization quality. To create one such training dataset, a previous approach used auto-evaluation metrics to determine from a set of component machine translation (MT) systems which gave the more accurate translation (by a comparative method). Once this had been determined, the most accurate translation was then labelled in such a way so as to indicate the MT system from which it came. In this previous approach, when the metric evaluation scores were low, there existed a high level of uncertainty as to which of the component MT systems was actually producing the better translation. To relax such uncertainty or error in classification, we propose an alternative approach to such labeling; that is, a cut-off method. In our experiments, using the aforementioned cut-off method in our proposed classifier, we managed to achieve a translation accuracy of 81.5% - a 5.0% improvement over existing methods.

Tissue Level Based Deep Learning Framework for Early Detection of Dysplasia in Oral Squamous Epithelium

  • Gupta, Rachit Kumar;Kaur, Mandeep;Manhas, Jatinder
    • Journal of Multimedia Information System
    • /
    • 제6권2호
    • /
    • pp.81-86
    • /
    • 2019
  • Deep learning is emerging as one of the best tool in processing data related to medical imaging. In our research work, we have proposed a deep learning based framework CNN (Convolutional Neural Network) for the classification of dysplastic tissue images. The CNN has classified the given images into 4 different classes namely normal tissue, mild dysplastic tissue, moderate dysplastic tissue and severe dysplastic tissue. The dataset under taken for the study consists of 672 tissue images of epithelial squamous layer of oral cavity captured out of the biopsy samples of 52 patients. After applying the data pre-processing and augmentation on the given dataset, 2688 images were created. Further, these 2688 images were classified into 4 categories with the help of expert Oral Pathologist. The classified data was supplied to the convolutional neural network for training and testing of the proposed framework. It has been observed that training data shows 91.65% accuracy whereas the testing data achieves 89.3% accuracy. The results produced by our proposed framework are also tested and validated by comparing the manual results produced by the medical experts working in this area.

An Extended Work Architecture for Online Threat Prediction in Tweeter Dataset

  • Sheoran, Savita Kumari;Yadav, Partibha
    • International Journal of Computer Science & Network Security
    • /
    • 제21권1호
    • /
    • pp.97-106
    • /
    • 2021
  • Social networking platforms have become a smart way for people to interact and meet on internet. It provides a way to keep in touch with friends, families, colleagues, business partners, and many more. Among the various social networking sites, Twitter is one of the fastest-growing sites where users can read the news, share ideas, discuss issues etc. Due to its vast popularity, the accounts of legitimate users are vulnerable to the large number of threats. Spam and Malware are some of the most affecting threats found on Twitter. Therefore, in order to enjoy seamless services it is required to secure Twitter against malicious users by fixing them in advance. Various researches have used many Machine Learning (ML) based approaches to detect spammers on Twitter. This research aims to devise a secure system based on Hybrid Similarity Cosine and Soft Cosine measured in combination with Genetic Algorithm (GA) and Artificial Neural Network (ANN) to secure Twitter network against spammers. The similarity among tweets is determined using Cosine with Soft Cosine which has been applied on the Twitter dataset. GA has been utilized to enhance training with minimum training error by selecting the best suitable features according to the designed fitness function. The tweets have been classified as spammer and non-spammer based on ANN structure along with the voting rule. The True Positive Rate (TPR), False Positive Rate (FPR) and Classification Accuracy are considered as the evaluation parameter to evaluate the performance of system designed in this research. The simulation results reveals that our proposed model outperform the existing state-of-arts.

A Novel Transfer Learning-Based Algorithm for Detecting Violence Images

  • Meng, Yuyan;Yuan, Deyu;Su, Shaofan;Ming, Yang
    • KSII Transactions on Internet and Information Systems (TIIS)
    • /
    • 제16권6호
    • /
    • pp.1818-1832
    • /
    • 2022
  • Violence in the Internet era poses a new challenge to the current counter-riot work, and according to research and analysis, most of the violent incidents occurring are related to the dissemination of violence images. The use of the popular deep learning neural network to automatically analyze the massive amount of images on the Internet has become one of the important tools in the current counter-violence work. This paper focuses on the use of transfer learning techniques and the introduction of an attention mechanism to the residual network (ResNet) model for the classification and identification of violence images. Firstly, the feature elements of the violence images are identified and a targeted dataset is constructed; secondly, due to the small number of positive samples of violence images, pre-training and attention mechanisms are introduced to suggest improvements to the traditional residual network; finally, the improved model is trained and tested on the constructed dedicated dataset. The research results show that the improved network model can quickly and accurately identify violence images with an average accuracy rate of 92.20%, thus effectively reducing the cost of manual identification and providing decision support for combating rebel organization activities.

근전도 기반의 Spider Chart와 딥러닝을 활용한 일상생활 잡기 손동작 분류 (Classification of Gripping Movement in Daily Life Using EMG-based Spider Chart and Deep Learning)

  • 이성문;피승훈;한승호;조용운;오도창
    • 대한의용생체공학회:의공학회지
    • /
    • 제43권5호
    • /
    • pp.299-307
    • /
    • 2022
  • In this paper, we propose a pre-processing method that converts to Spider Chart image data for classification of gripping movement using EMG (electromyography) sensors and Convolution Neural Networks (CNN) deep learning. First, raw data for six hand gestures are extracted from five test subjects using an 8-channel armband and converted into Spider Chart data of octagonal shapes, which are divided into several sliding windows and are learned. In classifying six hand gestures, the classification performance is compared with the proposed pre-processing method and the existing methods. Deep learning was performed on the dataset by dividing 70% of the total into training, 15% as testing, and 15% as validation. For system performance evaluation, five cross-validations were applied by dividing 80% of the entire dataset by training and 20% by testing. The proposed method generates 97% and 94.54% in cross-validation and general tests, respectively, using the Spider Chart preprocessing, which was better results than the conventional methods.

가상 환경에서의 딥러닝 기반 폐색영역 검출을 위한 데이터베이스 구축 (Construction of Database for Deep Learning-based Occlusion Area Detection in the Virtual Environment)

  • 김경수;이재인;곽석우;강원율;신대영;황성호
    • 드라이브 ㆍ 컨트롤
    • /
    • 제19권3호
    • /
    • pp.9-15
    • /
    • 2022
  • This paper proposes a method for constructing and verifying datasets used in deep learning technology, to prevent safety accidents in automated construction machinery or autonomous vehicles. Although open datasets for developing image recognition technologies are challenging to meet requirements desired by users, this study proposes the interface of virtual simulators to facilitate the creation of training datasets desired by users. The pixel-level training image dataset was verified by creating scenarios, including various road types and objects in a virtual environment. Detecting an object from an image may interfere with the accurate path determination due to occlusion areas covered by another object. Thus, we construct a database, for developing an occlusion area detection algorithm in a virtual environment. Additionally, we present the possibility of its use as a deep learning dataset to calculate a grid map, that enables path search considering occlusion areas. Custom datasets are built using the RDBMS system.

Improve the Performance of Semi-Supervised Side-channel Analysis Using HWFilter Method

  • Hong Zhang;Lang Li;Di Li
    • KSII Transactions on Internet and Information Systems (TIIS)
    • /
    • 제18권3호
    • /
    • pp.738-754
    • /
    • 2024
  • Side-channel analysis (SCA) is a cryptanalytic technique that exploits physical leakages, such as power consumption or electromagnetic emanations, from cryptographic devices to extract secret keys used in cryptographic algorithms. Recent studies have shown that training SCA models with semi-supervised learning can effectively overcome the problem of few labeled power traces. However, the process of training SCA models using semi-supervised learning generates many pseudo-labels. The performance of the SCA model can be reduced by some of these pseudo-labels. To solve this issue, we propose the HWFilter method to improve semi-supervised SCA. This method uses a Hamming Weight Pseudo-label Filter (HWPF) to filter the pseudo-labels generated by the semi-supervised SCA model, which enhances the model's performance. Furthermore, we introduce a normal distribution method for constructing the HWPF. In the normal distribution method, the Hamming weights (HWs) of power traces can be obtained from the normal distribution of power points. These HWs are filtered and combined into a HWPF. The HWFilter was tested using the ASCADv1 database and the AES_HD dataset. The experimental results demonstrate that the HWFilter method can significantly enhance the performance of semi-supervised SCA models. In the ASCADv1 database, the model with HWFilter requires only 33 power traces to recover the key. In the AES_HD dataset, the model with HWFilter outperforms the current best semi-supervised SCA model by 12%.

Clustering-Based Federated Learning for Enhancing Data Privacy in Internet of Vehicles

  • Zilong Jin;Jin Wang;Lejun Zhang
    • KSII Transactions on Internet and Information Systems (TIIS)
    • /
    • 제18권6호
    • /
    • pp.1462-1477
    • /
    • 2024
  • With the evolving complexity of connected vehicle features, the volume and diversity of data generated during driving continue to escalate. Enabling data sharing among interconnected vehicles holds promise for improving users' driving experiences and alleviating traffic congestion. Yet, the unintentional disclosure of users' private information through data sharing poses a risk, potentially compromising the interests of vehicle users and, in certain cases, endangering driving safety. Federated learning (FL) is a newly emerged distributed machine learning paradigm, which is expected to play a prominent role for privacy-preserving learning in autonomous vehicles. While FL holds significant potential to enhance the architecture of the Internet of Vehicles (IoV), the dynamic mobility of vehicles poses a considerable challenge to integrating FL with vehicular networks. In this paper, a novel clustered FL framework is proposed which is efficient for reducing communication and protecting data privacy. By assessing the similarity among feature vectors, vehicles are categorized into distinct clusters. An optimal vehicle is elected as the cluster head, which enhances the efficiency of personalized data processing and model training while reducing communication overhead. Simultaneously, the Local Differential Privacy (LDP) mechanism is incorporated during local training to safeguard vehicle privacy. The simulation results obtained from the 20newsgroups dataset and the MNIST dataset validate the effectiveness of the proposed scheme, indicating that the proposed scheme can ensure data privacy effectively while reducing communication overhead.

유전 알고리즘 기반의 비정상 행위 탐지를 위한 특징선택 (Feature Selection for Anomaly Detection Based on Genetic Algorithm)

  • 서재현
    • 한국융합학회논문지
    • /
    • 제9권7호
    • /
    • pp.1-7
    • /
    • 2018
  • 데이터 전처리 기법 중 하나인 특징 선택은 대규모 데이터셋을 다루는 다양한 응용분야에서 주요 연구 분야 중 하나로 각광받고 있다. 특징 선택은 패턴 인식, 기계학습 및 데이터 마이닝에서 사용됐고, 최근에는 텍스트 분류, 이미지 검색, 침입 탐지 및 게놈 분석과 같은 다양한 분야에 널리 적용되고 있다. 제안 방법은 메타 휴리스틱 알고리즘 중의 하나인 유전 알고리즘을 기반으로 한다. 특징 부분 집합을 찾는 방법은 크게 필터(filter) 방법과 래퍼(wrapper) 방법이 있는데, 본 연구에서는 최적의 특징 부분 집합을 찾기 위해 실제 분류기를 사용한 평가를 하는 래퍼 방법을 사용한다. 실험에 사용한 훈련 데이터셋은 클래스 불균형이 심하여 희소클래스에 대한 분류 성능을 높이기 어렵다. SMOTE 기법을 적용한 훈련 데이터셋을 사용하여 특징 선택을 하고 다양한 기계학습 알고리즘을 사용하여 선택한 특징들의 성능을 평가한다.