• 제목/요약/키워드: Classification Accuracy Test

검색결과 393건 처리시간 0.031초

ROC와 CAP 곡선에서의 최적 분류점 (Optimal Threshold from ROC and CAP Curves)

  • 홍종선;최진수
    • 응용통계연구
    • /
    • 제22권5호
    • /
    • pp.911-921
    • /
    • 2009
  • 신용평가 연구에서 부도와 정상차주에 대한 판별력을 평가하는 방법으로 Receiver Operating Characteristic(ROC)와 Cumulative Accuracy Profile(CAP) 곡선을 사용한다. ROC 곡선에서 최적의 분류정확도를 갖는 분류점과 CAP 곡선에서 최대의 이익을 나타내는 분류점은 일반적인 정확도의 개념으로 정의된 동일한 성과를 가진 접선을 사용하여 구한다. 본 연구에서는 정확도의 대안적인 측도로 진실율을 제안하고, 이 진실율을 이용하여 ROC와 CAP 곡선에서 대안적인 최적의 분류점을 구한다. 대부분 실제 차주의 모집단에서 부도차주는 정상차주보다 훨씬 수가 적다. 이러한 경우에 진실율은 정확도보다 비용함수의 측면에서 더욱 효율적일 수 있다. 진실율을 이용하여 최적의 분류정확도를 나타내는 분류점과 최대의 이익을 의미하는 분류점에 대응하는 스코어는 동일하다는 것을 보였으며, 이 스코어는 부도와 정상 차주의 분포함수의 동일성을 검정하는 Kolmogorov-Smirnov 통계량에 대응하는 스코어와도 일치하는 것을 발견하였다.

편광현미경 이미지 기반 염기성 화산암 분류를 위한 인공지능 모델의 효용성 평가 (Evaluating the Effectiveness of an Artificial Intelligence Model for Classification of Basic Volcanic Rocks Based on Polarized Microscope Image)

  • 심호;정원우;홍성식;서재원;박창윤;송윤구
    • 자원환경지질
    • /
    • 제55권3호
    • /
    • pp.309-316
    • /
    • 2022
  • 암석 분류에 필요한 인적, 시간적 소모를 최소화하기 위해 최근 인공지능을 활용한 암석 분류 연구가 대두되었다. 이에 본 연구에서는 편광현미경 박편 이미지를 활용하여 염기성 화산암을 세분류하고자 하였다. 분류에 사용된 인공지능 모델은 Tensorflow, Keras 라이브러리를 기반으로 합성곱 신경망 모델을 자체 제작하였다. Olivine basalt, basaltic andesite, olivine tholeiite, trachytic olivine basalt 기준시료 박편을 개방 니콜, 직교 니콜, 그리고 gypsum plate를 장착하고 촬영한 이미지 총 720장을 인공지능 모델에 training : test = 7 : 3 비율로 학습시켰다. 학습결과, 80~90%이상의 분류 정확도를 보였다. 각각의 인공지능 모델의 분류 정확도를 확인하였을 때, 본 모델의 암석분류 방식이 지질학자의 암석 분류 프로세스와 크게 다르지 않을 것으로 예상된다. 나아가 본 모델 뿐 아니라 보다 다양한 암석종을 세분시키는 모델을 제작하여 통합한다면, 데이터 분류의 신속성과 비전문가의 접근성 모두를 만족시키는 인공지능 모델을 개발할 수 있으며, 이를 통해 암석학 기초연구의 새로운 틀을 마련할 수 있을 것으로 생각된다.

초분광 반사광 영상을 이용한 무(Raphanus sativus L) 종자의 발아와 불발아 비파괴 판별 (Nondestructive Classification of Viable and Non-viable Radish (Raphanus sativus L) Seeds using Hyperspectral Reflectance Imaging)

  • 안치국;모창연;강점순;조병관
    • Journal of Biosystems Engineering
    • /
    • 제37권6호
    • /
    • pp.411-419
    • /
    • 2012
  • Purpose: Nondestructive evaluation of seed viability is a highly demanded technique in the seed industry. In this study, hyperspectral imaging system was used for discrimination of viable and non-viable radish seeds. Method: The spectral data with the range from 400 to 1000 nm measured by hyperspectral reflectance imaging system were used. A calibration and a test models were developed by partial least square discrimination analysis (PLS-DA) for classification of viable and non-viable radish seeds. Either each data set of visible (400~750 nm) and NIR (750~1000 nm) spectra and the spectra of the combined spectral ranges were used for developing models. Results: The discrimination accuracy of calibration was 84% for visible range and 76.3% for NIR range. The discrimination accuracy of test was 84.2% for visible range and 75.8% for NIR range. The discrimination accuracies of calibration and test with full range were 92.2% and 92.5%, respectively. The resultant images based on the optimal PLS-DA model showed high performance for the discrimination of the nonviable seeds from the viable seeds with the accuracy of 95%. Conclusions: The results showed that hyperspectral reflectance imaging has good potential for discriminating nonviable radish seeds from massive amounts of viable seeds.

인공신경망 기반 가스 분류기의 설계 (Design of Gas Classifier Based On Artificial Neural Network)

  • 정우재;김민우;조재찬;정윤호
    • 전기전자학회논문지
    • /
    • 제22권3호
    • /
    • pp.700-705
    • /
    • 2018
  • 본 논문에서는 restricted coulomb energy(RCE) 신경망 기반 가스 분류기를 제안하고, 이의 실시간 학습 및 분류를 위한 하드웨어 구현 결과를 제시한다. RCE 신경망은 네트워크 구조가 학습에 따라 유동적이며, 실시간 학습 및 분류가 가능하므로, 가스 분류 응용에 적합한 특징을 갖는다. 설계된 가스 분류기는 UCI gas dataset에 대해 99.2%의 분류 정확도를 보였으며, Intel-Altera cyclone IV FPGA 기반 구현 결과, 26,702개의 logic elements로 구현 가능함을 확인하였다. 또한, FPGA test system을 구성하여 63MHz의 동작 주파수로 실시간 검증을 수행하였다.

Remote Sensing Image Classification for Land Cover Mapping in Developing Countries: A Novel Deep Learning Approach

  • Lynda, Nzurumike Obianuju;Nnanna, Nwojo Agwu;Boukar, Moussa Mahamat
    • International Journal of Computer Science & Network Security
    • /
    • 제22권2호
    • /
    • pp.214-222
    • /
    • 2022
  • Convolutional Neural networks (CNNs) are a category of deep learning networks that have proven very effective in computer vision tasks such as image classification. Notwithstanding, not much has been seen in its use for remote sensing image classification in developing countries. This is majorly due to the scarcity of training data. Recently, transfer learning technique has successfully been used to develop state-of-the art models for remote sensing (RS) image classification tasks using training and testing data from well-known RS data repositories. However, the ability of such model to classify RS test data from a different dataset has not been sufficiently investigated. In this paper, we propose a deep CNN model that can classify RS test data from a dataset different from the training dataset. To achieve our objective, we first, re-trained a ResNet-50 model using EuroSAT, a large-scale RS dataset to develop a base model then we integrated Augmentation and Ensemble learning to improve its generalization ability. We further experimented on the ability of this model to classify a novel dataset (Nig_Images). The final classification results shows that our model achieves a 96% and 80% accuracy on EuroSAT and Nig_Images test data respectively. Adequate knowledge and usage of this framework is expected to encourage research and the usage of deep CNNs for land cover mapping in cases of lack of training data as obtainable in developing countries.

베이지안 분류기를 이용한 소프트웨어 품질 분류 (Software Quality Classification using Bayesian Classifier)

  • 홍의석
    • 한국IT서비스학회지
    • /
    • 제11권1호
    • /
    • pp.211-221
    • /
    • 2012
  • Many metric-based classification models have been proposed to predict fault-proneness of software module. This paper presents two prediction models using Bayesian classifier which is one of the most popular modern classification algorithms. Bayesian model based on Bayesian probability theory can be a promising technique for software quality prediction. This is due to the ability to represent uncertainty using probabilities and the ability to partly incorporate expert's knowledge into training data. The two models, Na$\ddot{i}$veBayes(NB) and Bayesian Belief Network(BBN), are constructed and dimensionality reduction of training data and test data are performed before model evaluation. Prediction accuracy of the model is evaluated using two prediction error measures, Type I error and Type II error, and compared with well-known prediction models, backpropagation neural network model and support vector machine model. The results show that the prediction performance of BBN model is slightly better than that of NB. For the data set with ambiguity, although the BBN model's prediction accuracy is not as good as the compared models, it achieves better performance than the compared models for the data set without ambiguity.

Transfer Learning Based Real-Time Crack Detection Using Unmanned Aerial System

  • Yuvaraj, N.;Kim, Bubryur;Preethaa, K. R. Sri
    • 국제초고층학회논문집
    • /
    • 제9권4호
    • /
    • pp.351-360
    • /
    • 2020
  • Monitoring civil structures periodically is necessary for ensuring the fitness of the structures. Cracks on inner and outer surfaces of the building plays a vital role in indicating the health of the building. Conventionally, human visual inspection techniques were carried up to human reachable altitudes. Monitoring of high rise infrastructures cannot be done using this primitive method. Also, there is a necessity for more accurate prediction of cracks on building surfaces for ensuring the health and safety of the building. The proposed research focused on developing an efficient crack classification model using Transfer Learning enabled EfficientNet (TL-EN) architecture. Though many other pre-trained models were available for crack classification, they rely on more number of training parameters for better accuracy. The TL-EN model attained an accuracy of 0.99 with less number of parameters on large dataset. A bench marked METU dataset with 40000 images were used to test and validate the proposed model. The surfaces of high rise buildings were investigated using vision enabled Unmanned Arial Vehicles (UAV). These UAV is fabricated with TL-EN model schema for capturing and analyzing the real time streaming video of building surfaces.

협업필터링에서 고객의 평가치를 이용한 선호도 예측의 사전평가에 관한 연구 (Pre-Evaluation for Prediction Accuracy by Using the Customer's Ratings in Collaborative Filtering)

  • 이석준;김선옥
    • Asia pacific journal of information systems
    • /
    • 제17권4호
    • /
    • pp.187-206
    • /
    • 2007
  • The development of computer and information technology has been combined with the information superhighway internet infrastructure, so information widely spreads not only in special fields but also in the daily lives of people. Information ubiquity influences the traditional way of transaction, and leads a new E-commerce which distinguishes from the existing E-commerce. Not only goods as physical but also service as non-physical come into E-commerce. As the scale of E-Commerce is being enlarged as well. It keeps people from finding information they want. Recommender systems are now becoming the main tools for E-Commerce to mitigate the information overload. Recommender systems can be defined as systems for suggesting some Items(goods or service) considering customers' interests or tastes. They are being used by E-commerce web sites to suggest products to their customers who want to find something for them and to provide them with information to help them decide which to purchase. There are several approaches of recommending goods to customer in recommender system but in this study, the main subject is focused on collaborative filtering technique. This study presents a possibility of pre-evaluation for the prediction performance of customer's preference in collaborative filtering before the process of customer's preference prediction. Pre-evaluation for the prediction performance of each customer having low performance is classified by using the statistical features of ratings rated by each customer is conducted before the prediction process. In this study, MovieLens 100K dataset is used to analyze the accuracy of classification. The classification criteria are set by using the training sets divided 80% from the 100K dataset. In the process of classification, the customers are divided into two groups, classified group and non classified group. To compare the prediction performance of classified group and non classified group, the prediction process runs the 20% test set through the Neighborhood Based Collaborative Filtering Algorithm and Correspondence Mean Algorithm. The prediction errors from those prediction algorithm are allocated to each customer and compared with each user's error. Research hypothesis : Two research hypotheses are formulated in this study to test the accuracy of the classification criterion as follows. Hypothesis 1: The estimation accuracy of groups classified according to the standard deviation of each user's ratings has significant difference. To test the Hypothesis 1, the standard deviation is calculated for each user in training set which is divided 80% from MovieLens 100K dataset. Four groups are classified according to the quartile of the each user's standard deviations. It is compared to test the estimation errors of each group which results from test set are significantly different. Hypothesis 2: The estimation accuracy of groups that are classified according to the distribution of each user's ratings have significant differences. To test the Hypothesis 2, the distributions of each user's ratings are compared with the distribution of ratings of all customers in training set which is divided 80% from MovieLens 100K dataset. It assumes that the customers whose ratings' distribution are different from that of all customers would have low performance, so six types of different distributions are set to be compared. The test groups are classified into fit group or non-fit group according to the each type of different distribution assumed. The degrees in accordance with each type of distribution and each customer's distributions are tested by the test of ${\chi}^2$ goodness-of-fit and classified two groups for testing the difference of the mean of errors. Also, the degree of goodness-of-fit with the distribution of each user's ratings and the average distribution of the ratings in the training set are closely related to the prediction errors from those prediction algorithms. Through this study, the customers who have lower performance of prediction than the rest in the system are classified by those two criteria, which are set by statistical features of customers ratings in the training set, before the prediction process.

마이크로어레이 데이터를 이용한 암 분류 표지 유전자 선별 시스템 (An Intelligent System of Marker Gene Selection for Classification of Cancers using Microarray Data)

  • 박수영;정채영
    • 한국정보통신학회논문지
    • /
    • 제14권10호
    • /
    • pp.2365-2370
    • /
    • 2010
  • 마이크로어레이를 기반으로 하는 암 분류 방법은 암 종류에 따라 다르게 발현되는 유전자 양상을 통계적으로 발견함으로써 정확한 암 분류에 기여할 수 있다. 따라서 현재의 마이크로어레이 기술을 이용해서 효과적으로 암을 분류하기 위해서는 특정 암과 밀접하게 관련이 있는 정보력 있는 유전자를 선택하는 과정이 필수적이다. 본 논문에서는 난소 암 마이크로어레이 데이터를 이용하여 암에 영향을 미치는 가장 다르게 발현할 가능성이 있는 표지 유전자를 추출할 수 있는 시스템을 고안하고, 다층퍼셉트론 분류기를 이용하여 기존의 마이크로어레이 시스템과 분류 성능을 비교분석하였다. 그 결과 ANOVA를 이용하여 선택된 표지 유전자를 포함하는 마이크로어레이 데이터 셋에서 98.61%의 향상된 분류 성능을 보였다.

텍스트 분류 기반 기계학습의 정신과 진단 예측 적용 (Application of Text-Classification Based Machine Learning in Predicting Psychiatric Diagnosis)

  • 백두현;황민규;이민지;우성일;한상우;이연정;황재욱
    • 생물정신의학
    • /
    • 제27권1호
    • /
    • pp.18-26
    • /
    • 2020
  • Objectives The aim was to find effective vectorization and classification models to predict a psychiatric diagnosis from text-based medical records. Methods Electronic medical records (n = 494) of present illness were collected retrospectively in inpatient admission notes with three diagnoses of major depressive disorder, type 1 bipolar disorder, and schizophrenia. Data were split into 400 training data and 94 independent validation data. Data were vectorized by two different models such as term frequency-inverse document frequency (TF-IDF) and Doc2vec. Machine learning models for classification including stochastic gradient descent, logistic regression, support vector classification, and deep learning (DL) were applied to predict three psychiatric diagnoses. Five-fold cross-validation was used to find an effective model. Metrics such as accuracy, precision, recall, and F1-score were measured for comparison between the models. Results Five-fold cross-validation in training data showed DL model with Doc2vec was the most effective model to predict the diagnosis (accuracy = 0.87, F1-score = 0.87). However, these metrics have been reduced in independent test data set with final working DL models (accuracy = 0.79, F1-score = 0.79), while the model of logistic regression and support vector machine with Doc2vec showed slightly better performance (accuracy = 0.80, F1-score = 0.80) than the DL models with Doc2vec and others with TF-IDF. Conclusions The current results suggest that the vectorization may have more impact on the performance of classification than the machine learning model. However, data set had a number of limitations including small sample size, imbalance among the category, and its generalizability. With this regard, the need for research with multi-sites and large samples is suggested to improve the machine learning models.