• Journal of KIISE:Software and Applications
• /
• v.30 no.3_4
• /
• pp.251-258
• /
• 2003

A support vector machine (SVM) is supposed to provide a good generalization performance, but the actual performance of a actually implemented SVM is often far from the theoretically expected level. This is largely because the implementation is based on an approximated algorithm, due to the high complexity of time and space. To improve this limitation, we propose ensemble of SVMs by using Bagging (bootstrap aggregating) and Boosting. By a Bagging stage each individual SVM is trained independently using randomly chosen training samples via a bootstrap technique. By a Boosting stage an individual SVM is trained by choosing training samples according to their probability distribution. The probability distribution is updated by the error of independent classifiers, and the process is iterated. After the training stage, they are aggregated to make a collective decision in several ways, such ai majority voting, the LSE(least squares estimation) -based weighting, and double layer hierarchical combining. The simulation results for IRIS data classification, the hand-written digit recognition and Face detection show that the proposed SVM ensembles greatly outperforms a single SVM in terms of classification accuracy.

• Journal of the Korean Data and Information Science Society
• /
• v.24 no.6
• /
• pp.1113-1125
• /
• 2013

Corporate bankruptcy prediction has been an important topic in the accounting and finance field for a long time. Several data mining techniques have been used for bankruptcy prediction. However, there are many limits for application to real classification problem with a single model. This study proposes ensemble SVM (support vector machine) model which assembles different SVM models with each different kernel functions. Our ensemble model is made and evaluated by v-fold cross-validation approach. The k top performing models are recruited into the ensemble. The classification is then carried out using the majority voting opinion of the ensemble. In this paper, we investigate the performance of ensemble SVM classifier in terms of accuracy, error rate, sensitivity, specificity, ROC curve, and AUC to compare with single SVM classifiers based on financial ratios dataset and simulation dataset. The results confirmed the advantages of our method: It is robust while providing good performance.

• Proceedings of the Korean Information Science Society Conference
• /
• 2006.06a
• /
• pp.76-78
• /
• 2006

현재 심혈관 질환은 암 다음으로 높은 사망 원인으로 기록되고 있어 심혈관 질환에 대한 초기 진단은 질환의 치료에 매우 중요한 문제로 대두되고 있다. 본 논문에서는 SVM을 이용하여 심혈관질환 환자의 질환 단계를 예측하였다. 일반적으로 이진분류에 사용되는 SVM을 이용하여 정상 및 질환 $1{\sim}3$기의 총 4가지 분류가 필요한 다분류 분류문제를 처리하기 위해서 논문에서는 독립적 학습된 단일 SVM 분류기들을 결합하여 분류를 수행하는 SVM 앙상블 방법을 사용하였다. 단일 분류기의 결합은 Majority voting, 최소자승에러기반 가중치 부여, 2단계층 결합 등의 방법으로 수행하여 심혈관 질환 분류에 적합한 앙상블의 구성을 시도하였다. 실험 데이터는 (주)제노프라의 압타머 칩 데이터를 사용하였다. 서로 다른 데이터를 이용하여 학습된 이종의 SVM들을 결합한 결과 질환단계 예측에 있어서 단일 SVM을 이용하여 질환 단계를 예측하는 경우 보다 향상된 질환단계 예측 성능을 관찰할 수 있었으며, 심혈관 질환의 예측에 대해서는 단일 SVM 분류기의 2단 계층 결합법이 가장 좋은 성능을 보임을 확인하였다.

• Journal of the Korea Society of Computer and Information
• /
• v.19 no.6
• /
• pp.19-26
• /
• 2014

An ensemble classifier system is a widely-used multi-classifier system, which combines the results from each classifier and, as a result, achieves better classification result than any single classifier used. Several methods have been used to build an ensemble classifier including boosting, which is a cascade method where misclassified examples in previous stage are used to boost the performance in current stage. Boosting is, however, a serial method which does not form a complete feedback loop. In this paper, proposed is context sensitive SVM ensemble (CASE) which adopts SVM, one of the best classifiers in term of classification rate, as a basic classifier and clustering method to divide feature space into contexts. As CASE divides feature space and trains SVMs simultaneously, the result from one component can be applied to the other and CASE achieves better result than boosting. Experimental results prove the usefulness of the proposed method.

• Proceedings of the Korean Information Science Society Conference
• /
• 2004.10b
• /
• pp.706-708
• /
• 2004

대부분의 기계학습 알고리즘은 학습 데이터에서 각각의 범주간의 비율이 동일하거나 비슷하다는 가정 하에 문제를 풀게 된다. 그러나 실제 문제에서는 그 비율이 동일하지 않으며 매우 큰 차이를 보이기도 하는데, 이는 분류 성능을 저하시키는 요인이기도 하다 따라서 본 논문에서는 이러한 데이터의 불균형 문제를 해소하는 방안으로 SVM 앙상블 기법을 적용한 샘플링을 제안하고 이를 실제 불균형 데이터에 적용함으로써 제안된 방법이 기존의 방법들에 비해 향상된 성능을 나타내는 것을 보였다.

• Journal of Korea Society of Industrial Information Systems
• /
• v.17 no.7
• /
• pp.139-148
• /
• 2012

An ensemble of classifiers is to employ a set of individually trained classifiers and combine their predictions. It has been found that in most cases the ensembles produce more accurate predictions than the base classifiers. Combining outputs from multiple classifiers, known as ensemble learning, is one of the standard and most important techniques for improving classification accuracy in machine learning. An ensemble of classifiers is efficient only if the individual classifiers make decisions as diverse as possible. Bagging is the most popular method of ensemble learning to generate a diverse set of classifiers. Diversity in bagging is obtained by using different training sets. The different training data subsets are randomly drawn with replacement from the entire training dataset. The random subspace method is an ensemble construction technique using different attribute subsets. In the random subspace, the training dataset is also modified as in bagging. However, this modification is performed in the feature space. Bagging and random subspace are quite well known and popular ensemble algorithms. However, few studies have dealt with the integration of bagging and random subspace using SVM Classifiers, though there is a great potential for useful applications in this area. The focus of this paper is to propose methods for improving SVM performance using hybrid ensemble strategy for bankruptcy prediction. This paper applies the proposed ensemble model to the bankruptcy prediction problem using a real data set from Korean companies.

• Proceedings of the Korea Information Processing Society Conference
• /
• 2003.11a
• /
• pp.301-304
• /
• 2003

SVM은 일반화된 높은 분류 정확률을 보인다. 그러나, SVM은 데이터의 양이 커질수록 높은 시간 공간적 복잡성 때문에 근사화 알고리즘(Approximation Algorithm)을 이용한다. 이러한 접근 방법은 실제구현에서 높은 시간 공간적 복잡성을 요구하여 분류 정확률과 효율성을 낮아지게 한다. 따라서, 본 논문은 SVM을 앙상블 구조로 구성하여 분류 정확률을 높이고, 분류자의 최적 하이퍼플랜(Optimal Hyperplane)결정을 위해 마진 벡터만을 이용하여 시간 공간적 문제를 해결하였다. 실험결과, 본 논문에서 제시한 방법이 단일 SVM을 이용한 방법보다 높은 분류 정확률과 높은 효율성을 가짐을 보여준다...

• Proceedings of the Korean Information Science Society Conference
• /
• 2003.10a
• /
• pp.190-192
• /
• 2003

소비자의 신용 대출 규모가 점차 증가하면서 기업에서 고객의 신용 등급에 의한 정확한 고객 분류를 필요로 하고 있다 이를 위해 판별 분석과 신경망의 역전파(BP: Back Propagation)를 이용한 고객 분류 시스템이 연구되었다. 그러나, 판별 분석을 사용한 방법은 불규칙한 신용 거래의 성향을 보이는 비정규 분포의 고객 데이터의 영향으로 여러 개의 판별 함수와 판별점이 존재하여 분류 정확도가 떨어지는 단점이 있다. 신경망을 이용한 방법은 불규칙한 신용 거래의 성향을 보이는 고객 데이터에 의해서, 지역 최소점(Local Minima)에 빠져 최대의 분류 정확률을 보이는 분류자를 얻지 못하는 경우가 발생할 수 있다. 본 논문에서는 이러한 기존 연구의 분류 정확률을 저하시키는 단점을 해결하기 위해 SVM(Support Vector Machine)을 사용하여 고객의 신용 등급을 분류하는 방법을 제안한다. SVM은 SV(Support Vector)의 수에 의해서 학습 성능이 좌우되므로, 불규칙한 거래 성향을 보이는 고객에 대해서도 높은 차원으로의 매핑을 통하여, 효과적으로 학습시킬 수 있어 분류의 정확도를 높일 수 있다 하지만, SVM은 근사화 알고리즘(Approximation Algorithms)을 이용하므로 분류 정확도가 이론적인 성능에 미치지 못한다. 따라서, 본 논문은 점진적 앙상블 SVM을 사용하여, 기존의 고객 분류 시스템의 문제점을 해결하고 실제적으로 SVM의 분류 정확률을 높인다. 실험 결과는 점진적 앙상블 SVM을 이용한 방법의 정확성이 기존의 방법보다 높다는 것을 보여준다.

• The KIPS Transactions:PartB
• /
• v.14B no.4
• /
• pp.303-310
• /
• 2007

In machine learning, the ensemble classifier that is a set of classifiers have been introduced for higher accuracy than individual classifiers. We propose a new ensemble learning method that employs a set of region based classifiers. To show the performance of the proposed method. we compared its performance with that of bagging and boosting, which ard existing ensemble methods. Since the distribution of data can be different in different regions in the feature space, we split the data and generate classifiers based on each region and apply a weighted voting among the classifiers. We used 11 data sets from the UCI Machine Learning Repository to compare the performance of our new ensemble method with that of individual classifiers as well as existing ensemble methods such as bagging and boosting. As a result, we found that our method produced improved performance, particularly when the base learner is Naive Bayes or SVM.

• Journal of the Korean Chemical Society
• /
• v.53 no.6
• /
• pp.653-662
• /
• 2009

Developing effective tools for predicting absorption, distribution, metabolism, excretion properties and toxicity (ADME/T) of new chemical entities in the early stage of drug design is one of the most important tasks in drug discovery and development today. As one of these attempts, support vector machines (SVM) has recently been exploited for the prediction of ADME/T related properties. However, two problems in SVM modeling, i.e. feature selection and parameters setting, are still far from solved. The two problems have been shown to be crucial to the efficiency and accuracy of SVM classification. In particular, the feature selection and optimal SVM parameters setting influence each other, which indicates that they should be dealt with simultaneously. In this account, we present an integrated practical solution, in which genetic-based algorithm (GA) is used for feature selection and grid search (GS) method for parameters optimization. hERG ion-channel inhibitor classification models of ADME/T related properties has been built for assessing and testing the proposed GA-GS-SVM. We generated 6 different models that are 3 different single models and 3 different ensemble models using training set - 1891 compounds and validated with external test set - 175 compounds. We compared single model with ensemble model to solve data imbalance problems. It was able to improve accuracy of prediction to use ensemble model.