• Title/Summary/Keyword: classifier evaluation

Search Result 147, Processing Time 0.023 seconds

A Comparative Study of Phishing Websites Classification Based on Classifier Ensembles

  • Tama, Bayu Adhi;Rhee, Kyung-Hyune
    • Journal of Multimedia Information System
    • /
    • v.5 no.2
    • /
    • pp.99-104
    • /
    • 2018
  • Phishing website has become a crucial concern in cyber security applications. It is performed by fraudulently deceiving users with the aim of obtaining their sensitive information such as bank account information, credit card, username, and password. The threat has led to huge losses to online retailers, e-business platform, financial institutions, and to name but a few. One way to build anti-phishing detection mechanism is to construct classification algorithm based on machine learning techniques. The objective of this paper is to compare different classifier ensemble approaches, i.e. random forest, rotation forest, gradient boosted machine, and extreme gradient boosting against single classifiers, i.e. decision tree, classification and regression tree, and credal decision tree in the case of website phishing. Area under ROC curve (AUC) is employed as a performance metric, whilst statistical tests are used as baseline indicator of significance evaluation among classifiers. The paper contributes the existing literature on making a benchmark of classifier ensembles for web phishing detection.

A Study on Modulation Classification of PSK Signals Based on Statistical Moments (통계적 모먼트에 의한 PSK 신호의 변조분류에 관한 연구)

  • 이원철;한영열
    • The Journal of Korean Institute of Communications and Information Sciences
    • /
    • v.19 no.6
    • /
    • pp.1004-1015
    • /
    • 1994
  • Modulation type classifier based on statistical moments has been successfully employed to classify PSK signals. Previously, the classifier developed utilizes the statistical moment of samples of the received signal phase, which may be difficult to extract from received signal. In this paper we propose a new moments-based classifier to classify PSK signals by using the moments of the demodulated signal for PSK. THe demodulated signal can be easily extracted from the conventional demodulation of PSK. The evaluation of the performance of the proposed classifier for PSK signals has been investigated in additive white Gaussian noise environment using the exact distribution of the demodulated signal. The performances of classifier in terms of probability of misclassification were evaluated. We found that the coherent system classifier gave 4dB improvement for BPSK and 3dB for QPSK over noncoherent system classifier, when the probability of misclassification is 10 and m equals to 4.

  • PDF

Cognitive Impairment Prediction Model Using AutoML and Lifelog

  • Hyunchul Choi;Chiho Yoon;Sae Bom Lee
    • Journal of the Korea Society of Computer and Information
    • /
    • v.28 no.11
    • /
    • pp.53-63
    • /
    • 2023
  • This study developed a cognitive impairment predictive model as one of the screening tests for preventing dementia in the elderly by using Automated Machine Learning(AutoML). We used 'Wearable lifelog data for high-risk dementia patients' of National Information Society Agency, then conducted using PyCaret 3.0.0 in the Google Colaboratory environment. This study analysis steps are as follows; first, selecting five models demonstrating excellent classification performance for the model development and lifelog data analysis. Next, using ensemble learning to integrate these models and assess their performance. It was found that Voting Classifier, Gradient Boosting Classifier, Extreme Gradient Boosting, Light Gradient Boosting Machine, Extra Trees Classifier, and Random Forest Classifier model showed high predictive performance in that order. This study findings, furthermore, emphasized on the the crucial importance of 'Average respiration per minute during sleep' and 'Average heart rate per minute during sleep' as the most critical feature variables for accurate predictions. Finally, these study results suggest that consideration of the possibility of using machine learning and lifelog as a means to more effectively manage and prevent cognitive impairment in the elderly.

A Genetic Algorithm-based Classifier Ensemble Optimization for Activity Recognition in Smart Homes

  • Fatima, Iram;Fahim, Muhammad;Lee, Young-Koo;Lee, Sungyoung
    • KSII Transactions on Internet and Information Systems (TIIS)
    • /
    • v.7 no.11
    • /
    • pp.2853-2873
    • /
    • 2013
  • Over the last few years, one of the most common purposes of smart homes is to provide human centric services in the domain of u-healthcare by analyzing inhabitants' daily living. Currently, the major challenges in activity recognition include the reliability of prediction of each classifier as they differ according to smart homes characteristics. Smart homes indicate variation in terms of performed activities, deployed sensors, environment settings, and inhabitants' characteristics. It is not possible that one classifier always performs better than all the other classifiers for every possible situation. This observation has motivated towards combining multiple classifiers to take advantage of their complementary performance for high accuracy. Therefore, in this paper, a method for activity recognition is proposed by optimizing the output of multiple classifiers with Genetic Algorithm (GA). Our proposed method combines the measurement level output of different classifiers for each activity class to make up the ensemble. For the evaluation of the proposed method, experiments are performed on three real datasets from CASAS smart home. The results show that our method systematically outperforms single classifier and traditional multiclass models. The significant improvement is achieved from 0.82 to 0.90 in the F-measures of recognized activities as compare to existing methods.

Development of Adaptive Signal Pattern Recognition Program and Application to Classification of Defects in Weld Zone by AE Method (적응형 신호 형상 인식 프로그램 개발과 AE법에 의한 용접부 결함 분류에 관한 적용 연구)

  • Lee, K.Y.;Lim, J.M.;Kim, J.S.
    • Journal of the Korean Society for Nondestructive Testing
    • /
    • v.16 no.1
    • /
    • pp.34-45
    • /
    • 1996
  • The signal pattern recognition program which can perform signal acquisition and processing, the extraction and selection of features, the classifier design and the evaluation, is developed and applied to the classification of artificial defects in the weld zone of Austenitic STS304. The neural network classifier is compared with the linear discriminant function classifier and the empirical Bayesian classifier. The signal through a broadband sensor is compared with that through a resonance type sensor. In recognition rate, the neural network classifier is best, and the signal through a broadband sensor is better.

  • PDF

Evaluation of Robust Classifier Algorithm for Tissue Classification under Various Noise Levels

  • Youn, Su Hyun;Shin, Ki Young;Choi, Ahnryul;Mun, Joung Hwan
    • ETRI Journal
    • /
    • v.39 no.1
    • /
    • pp.87-96
    • /
    • 2017
  • Ultrasonic surgical devices are routinely used for surgical procedures. The incision and coagulation of tissue generate a temperature of $40^{\circ}C-150^{\circ}C$ and depend on the controllable output power level of the surgical device. Recently, research on the classification of grasped tissues to automatically control the power level was published. However, this research did not consider the specific characteristics of the surgical device, tissue denaturalization, and so on. Therefore, this research proposes a robust algorithm that simulates noise to resemble real situations and classifies tissue using conventional classifier algorithms. In this research, the bioimpedance spectrum for six tissues (liver, large intestine, kidney, lung, muscle, and fat) is measured, and five classifier algorithms are used. A signal-to-noise ratio of additive white Gaussian noise diversifies the testing sets, and as a result, each classifier's performance exhibits a difference. The k-nearest neighbors algorithm shows the highest classification rate of 92.09% (p < 0.01) and a standard deviation of 1.92%, which confirms high reproducibility.

Performance evaluation of sleep stage classifier for the sleep-inducing portable neurofeedback system (포터블 수면유도 뉴로피드백 시스템 구현을 위한 수면뇌파 상태 분류기 성능 평가)

  • Lee, Taek
    • Journal of the Korea Convergence Society
    • /
    • v.9 no.11
    • /
    • pp.83-90
    • /
    • 2018
  • Recently, many people have suffered from insomnia, labor loss, cognitive decline, and mental illness. The solution to this problem is almost entirely cognitive therapy or medication, but it is not recommended in the long term due to side effects and dependency problems. Therefore, in this paper, we propose a neuro feedback system based on portable EEG that helps induce sleeping. We design and evaluate the EEG classifier, which is the most important function to implement the system, and propose an optimized classifier modeling method for various factors that can affect performance. When using the proposed classifier, we could distinguish 97.9% of awakening and sleep phase in portable EEG.

Machine Learning-based model for predicting changes in user evaluation reflecting the period of the product (제품 사용 기간을 반영한 기계학습 기반 사용자 평가 변화 예측 모델)

  • Boo Hyunkyung;Kim Namgyu
    • Journal of Korea Society of Digital Industry and Information Management
    • /
    • v.19 no.1
    • /
    • pp.91-107
    • /
    • 2023
  • With the recent expansion of the commerce ecosystem, a large number of user evaluations have been produced. Accordingly, attempts to create business insights using user evaluation data have been actively made. However, since user evaluation can change after the user experiences the product, it is difficult to say that the analysis based only on reviews immediately after purchase fully reflects the user's evaluation of the product. Moreover, studies conducted so far on user evaluation have overlooked the fact that the length of time a user has used a product can affect the user's product evaluation. Therefore, in this study, we build a model that predicts the direction of change in the user's rating after use from the user's rating and reviews immediately after purchase. In particular, the proposed model reflects the product's period of use in predicting the change direction of the star rating. However, since the posterior information on the duration of product use cannot be used as input in the inference process, we propose a structure that utilizes information about the product's period of use using an auxiliary classifier. As a result of an experiment using 599,889 user evaluation data collected from the shopping platform 'N' company, we confirmed that the proposed model performed better than the existing model in terms of accuracy.

Rejection Scheme of Nearest Neighbor Classifier for Diagnosis of Rotating Machine Fault (회전 기계 고장 진단을 위한 최근접 이웃 분류기의 기각 전략)

  • Choe, Yeong-Il;Park, Gwang-Ho;Gi, Chang-Du
    • Journal of the Korean Society for Precision Engineering
    • /
    • v.19 no.3
    • /
    • pp.52-58
    • /
    • 2002
  • The purpose of condition monitoring and fault diagnosis is to detect faults occurring in machinery in order to improve the level of safety in plants and reduce operational and maintenance costs. The recognition performance is important not only to gain a high recognition rate bur a1so to minimize the diagnosis failures error rate by using off effective rejection module. We examined the problem of performance evaluation for the rejection scheme considering the accuracy of individual c1asses in order to increase the recognition performance. We use the Smith's method among the previous studies related to rejection method. Nearest neighbor classifier is used for classifying the machine conditions from the vibration signals. The experiment results for the performance evaluation of rejection show the modified optimum rejection method is superior to others.

Credit Risk Evaluations of Online Retail Enterprises Using Support Vector Machines Ensemble: An Empirical Study from China

  • LI, Xin;XIA, Han
    • The Journal of Asian Finance, Economics and Business
    • /
    • v.9 no.8
    • /
    • pp.89-97
    • /
    • 2022
  • The e-commerce market faces significant credit risks due to the complexity of the industry and information asymmetries. Therefore, credit risk has started to stymie the growth of e-commerce. However, there is no reliable system for evaluating the creditworthiness of e-commerce companies. Therefore, this paper constructs a credit risk evaluation index system that comprehensively considers the online and offline behavior of online retail enterprises, including 15 indicators that reflect online credit risk and 15 indicators that reflect offline credit risk. This paper establishes an integration method based on a fuzzy integral support vector machine, which takes the factor analysis results of the credit risk evaluation index system of online retail enterprises as the input and the credit risk evaluation results of online retail enterprises as the output. The classification results of each sub-classifier and the importance of each sub-classifier decision to the final decision have been taken into account in this method. Select the sample data of 1500 online retail loan customers from a bank to test the model. The empirical results demonstrate that the proposed method outperforms a single SVM and traditional SVMs aggregation technique via majority voting in terms of classification accuracy, which provides a basis for banks to establish a reliable evaluation system.