• Title/Summary/Keyword: boosting_based classifier

Search Result 43, Processing Time 0.028 seconds

Diversity based Ensemble Genetic Programming for Improving Classification Performance (분류 성능 향상을 위한 다양성 기반 앙상블 유전자 프로그래밍)

  • Hong Jin-Hyuk;Cho Sung-Bae
    • Journal of KIISE:Software and Applications
    • /
    • v.32 no.12
    • /
    • pp.1229-1237
    • /
    • 2005
  • Combining multiple classifiers has been actively exploited to improve classification performance. It is required to construct a pool of accurate and diverse base classifier for obtaining a good ensemble classifier. Conventionally ensemble learning techniques such as bagging and boosting have been used and the diversify of base classifiers for the training set has been estimated, but there are some limitations in classifying gene expression profiles since only a few training samples are available. This paper proposes an ensemble technique that analyzes the diversity of classification rules obtained by genetic programming. Genetic programming generates interpretable rules, and a sample is classified by combining the most diverse set of rules. We have applied the proposed method to cancer classification with gene expression profiles. Experiments on lymphoma cancer dataset, prostate cancer dataset and ovarian cancer dataset have illustrated the usefulness of the proposed method. h higher classification accuracy has been obtained with the proposed method than without considering diversity. It has been also confirmed that the diversity increases classification performance.

Classifying Instantaneous Cognitive States from fMRI using Discriminant based Feature Selection and Adaboost

  • Vu, Tien Duong;Yang, Hyung-Jeong;Do, Luu Ngoc;Thieu, Thao Nguyen
    • Smart Media Journal
    • /
    • v.5 no.1
    • /
    • pp.30-37
    • /
    • 2016
  • In recent decades, the study of human brain function has dramatically increased thanks to the advent of Functional Magnetic Resonance Imaging. This is a powerful tool which provides a deep view of the activities of the brain. From fMRI data, the neuroscientists analyze which parts of the brain have responsibility for a particular action and finding the common pattern representing each state involved in these tasks. This is one of the most challenges in neuroscience area because of noisy, sparsity of data as well as the differences of anatomical brain structure of each person. In this paper, we propose the use of appropriate discriminant methods, such as Fisher Discriminant Ratio and hypothesis testing, together with strong boosting ability of Adaboost classifier. We prove that discriminant methods are effective in classifying cognitive states. The experiment results show significant better accuracy than previous works. We also show that it is possible to train a successful classifier without prior anatomical knowledge and use only a small number of features.

An Active Learning-based Method for Composing Training Document Set in Bayesian Text Classification Systems (베이지언 문서분류시스템을 위한 능동적 학습 기반의 학습문서집합 구성방법)

  • 김제욱;김한준;이상구
    • Journal of KIISE:Software and Applications
    • /
    • v.29 no.12
    • /
    • pp.966-978
    • /
    • 2002
  • There are two important problems in improving text classification systems based on machine learning approach. The first one, called "selection problem", is how to select a minimum number of informative documents from a given document collection. The second one, called "composition problem", is how to reorganize selected training documents so that they can fit an adopted learning method. The former problem is addressed in "active learning" algorithms, and the latter is discussed in "boosting" algorithms. This paper proposes a new learning method, called AdaBUS, which proactively solves the above problems in the context of Naive Bayes classification systems. The proposed method constructs more accurate classification hypothesis by increasing the valiance in "weak" hypotheses that determine the final classification hypothesis. Consequently, the proposed algorithm yields perturbation effect makes the boosting algorithm work properly. Through the empirical experiment using the Routers-21578 document collection, we show that the AdaBUS algorithm more significantly improves the Naive Bayes-based classification system than other conventional learning methodson system than other conventional learning methods

Analysis of Important Indicators of TCB Using GBM (일반화가속모형을 이용한 기술신용평가 주요 지표 분석)

  • Jeon, Woo-Jeong(Michael);Seo, Young-Wook
    • The Journal of Society for e-Business Studies
    • /
    • v.22 no.4
    • /
    • pp.159-173
    • /
    • 2017
  • In order to provide technical financial support to small and medium-sized venture companies based on technology, the government implemented the TCB evaluation, which is a kind of technology rating evaluation, from the Kibo and a qualified private TCB. In this paper, we briefly review the current state of TCB evaluation and available indicators related to technology evaluation accumulated in the Korea Credit Information Services (TDB), and then use indicators that have a significant effect on the technology rating score. Multiple regression techniques will be explored. And the relative importance and classification accuracy of the indicators were calculated by applying the key indicators as independent features applied to the generalized boosting model, which is a representative machine learning classifier, as the class influence and the fitness of each model. As a result of the analysis, it was analyzed that the relative importance between the two models was not significantly different. However, GBM model had more weight on the InnoBiz certification, R&D department, patent registration and venture confirmation indicators than regression model.

Ensemble Learning with Support Vector Machines for Bond Rating (회사채 신용등급 예측을 위한 SVM 앙상블학습)

  • Kim, Myoung-Jong
    • Journal of Intelligence and Information Systems
    • /
    • v.18 no.2
    • /
    • pp.29-45
    • /
    • 2012
  • Bond rating is regarded as an important event for measuring financial risk of companies and for determining the investment returns of investors. As a result, it has been a popular research topic for researchers to predict companies' credit ratings by applying statistical and machine learning techniques. The statistical techniques, including multiple regression, multiple discriminant analysis (MDA), logistic models (LOGIT), and probit analysis, have been traditionally used in bond rating. However, one major drawback is that it should be based on strict assumptions. Such strict assumptions include linearity, normality, independence among predictor variables and pre-existing functional forms relating the criterion variablesand the predictor variables. Those strict assumptions of traditional statistics have limited their application to the real world. Machine learning techniques also used in bond rating prediction models include decision trees (DT), neural networks (NN), and Support Vector Machine (SVM). Especially, SVM is recognized as a new and promising classification and regression analysis method. SVM learns a separating hyperplane that can maximize the margin between two categories. SVM is simple enough to be analyzed mathematical, and leads to high performance in practical applications. SVM implements the structuralrisk minimization principle and searches to minimize an upper bound of the generalization error. In addition, the solution of SVM may be a global optimum and thus, overfitting is unlikely to occur with SVM. In addition, SVM does not require too many data sample for training since it builds prediction models by only using some representative sample near the boundaries called support vectors. A number of experimental researches have indicated that SVM has been successfully applied in a variety of pattern recognition fields. However, there are three major drawbacks that can be potential causes for degrading SVM's performance. First, SVM is originally proposed for solving binary-class classification problems. Methods for combining SVMs for multi-class classification such as One-Against-One, One-Against-All have been proposed, but they do not improve the performance in multi-class classification problem as much as SVM for binary-class classification. Second, approximation algorithms (e.g. decomposition methods, sequential minimal optimization algorithm) could be used for effective multi-class computation to reduce computation time, but it could deteriorate classification performance. Third, the difficulty in multi-class prediction problems is in data imbalance problem that can occur when the number of instances in one class greatly outnumbers the number of instances in the other class. Such data sets often cause a default classifier to be built due to skewed boundary and thus the reduction in the classification accuracy of such a classifier. SVM ensemble learning is one of machine learning methods to cope with the above drawbacks. Ensemble learning is a method for improving the performance of classification and prediction algorithms. AdaBoost is one of the widely used ensemble learning techniques. It constructs a composite classifier by sequentially training classifiers while increasing weight on the misclassified observations through iterations. The observations that are incorrectly predicted by previous classifiers are chosen more often than examples that are correctly predicted. Thus Boosting attempts to produce new classifiers that are better able to predict examples for which the current ensemble's performance is poor. In this way, it can reinforce the training of the misclassified observations of the minority class. This paper proposes a multiclass Geometric Mean-based Boosting (MGM-Boost) to resolve multiclass prediction problem. Since MGM-Boost introduces the notion of geometric mean into AdaBoost, it can perform learning process considering the geometric mean-based accuracy and errors of multiclass. This study applies MGM-Boost to the real-world bond rating case for Korean companies to examine the feasibility of MGM-Boost. 10-fold cross validations for threetimes with different random seeds are performed in order to ensure that the comparison among three different classifiers does not happen by chance. For each of 10-fold cross validation, the entire data set is first partitioned into tenequal-sized sets, and then each set is in turn used as the test set while the classifier trains on the other nine sets. That is, cross-validated folds have been tested independently of each algorithm. Through these steps, we have obtained the results for classifiers on each of the 30 experiments. In the comparison of arithmetic mean-based prediction accuracy between individual classifiers, MGM-Boost (52.95%) shows higher prediction accuracy than both AdaBoost (51.69%) and SVM (49.47%). MGM-Boost (28.12%) also shows the higher prediction accuracy than AdaBoost (24.65%) and SVM (15.42%)in terms of geometric mean-based prediction accuracy. T-test is used to examine whether the performance of each classifiers for 30 folds is significantly different. The results indicate that performance of MGM-Boost is significantly different from AdaBoost and SVM classifiers at 1% level. These results mean that MGM-Boost can provide robust and stable solutions to multi-classproblems such as bond rating.

The I-MCTBoost Classifier for Real-time Face Detection in Depth Image (깊이영상에서 실시간 얼굴 검출을 위한 I-MCTBoost)

  • Joo, Sung-Il;Weon, Sun-Hee;Choi, Hyung-Il
    • Journal of the Korea Society of Computer and Information
    • /
    • v.19 no.3
    • /
    • pp.25-35
    • /
    • 2014
  • This paper proposes a method of boosting-based classification for the purpose of real-time face detection. The proposed method uses depth images to ensure strong performance of face detection in response to changes in lighting and face size, and uses the depth difference feature to conduct learning and recognition through the I-MCTBoost classifier. I-MCTBoost performs recognition by connecting the strong classifiers that are constituted from weak classifiers. The learning process for the weak classifiers is as follows: first, depth difference features are generated, and eight of these features are combined to form the weak classifier, and each feature is expressed as a binary bit. Strong classifiers undergo learning through the process of repeatedly selecting a specified number of weak classifiers, and become capable of strong classification through a learning process in which the weight of the learning samples are renewed and learning data is added. This paper explains depth difference features and proposes a learning method for the weak classifiers and strong classifiers of I-MCTBoost. Lastly, the paper presents comparisons of the proposed classifiers and the classifiers using conventional MCT through qualitative and quantitative analyses to establish the feasibility and efficiency of the proposed classifiers.

Rotated face detection based on sharing features (특징들의 공유에 의한 기울어진 얼굴 검출)

  • Song, Young-Mo;Ko, Yun-Ho
    • Proceedings of the IEEK Conference
    • /
    • 2009.05a
    • /
    • pp.31-33
    • /
    • 2009
  • Face detection using AdaBoost algorithm is capable of processing images rapidly while having high detection rates. It seemed to be the fastest and the most robust and it is still today. Many improvements or extensions of this method have been proposed. However, previous approaches only deal with upright faces. They suffer from limited discriminant capability for rotated faces as these methods apply the same features for both upright and rotated faces. To solve this problem, it is necessary that we rotate input images or make independently trained detectors. However, this can be slow and can require a lot of training data, since each classifier requires the computation of many different image features. This paper proposes a robust algorithm for finding rotated faces within an image. It reduces the computational and sample complexity, by finding common features that can be shared across the classes. And it will be able to apply with multi-class object detection.

  • PDF

Investigation of Polarimetric SAR Remote Sensing for Landslide Detection Using PALSAR-2 Quad-pol Data

  • Cho, KeunHoo;Park, Sang-Eun;Cho, Jae-Hyoung;Moon, Hyoi;Han, Seung-hoon
    • Korean Journal of Remote Sensing
    • /
    • v.34 no.4
    • /
    • pp.591-600
    • /
    • 2018
  • Recent SAR systems provide fully polarimetric SAR data, which is known to be useful in a variety of applications such as disaster monitoring, target recognition, and land cover classification. The objective of this study is to evaluate the performance of polarization SAR data for landslide detection. The detectability of different SAR parameters was investigated based on the supervised classification approach. The classifier used in this study is the Adaptive Boosting algorithms. A fully polarimetric L-band PALSAR-2 data was used to examine landslides caused by the 2016 Kumamoto earthquake in Kyushu, Japan. Experimental results show that fully polarimetric features from the target decomposition technique can provide improved detectability of landslide site with significant reduction of false alarms as compared with the single polarimetric observables.

Real-time Hand Region Detection based on Cascade using Depth Information (깊이정보를 이용한 케스케이드 방식의 실시간 손 영역 검출)

  • Joo, Sung Il;Weon, Sun Hee;Choi, Hyung Il
    • KIPS Transactions on Software and Data Engineering
    • /
    • v.2 no.10
    • /
    • pp.713-722
    • /
    • 2013
  • This paper proposes a method of using depth information to detect the hand region in real-time based on the cascade method. In order to ensure stable and speedy detection of the hand region even under conditions of lighting changes in the test environment, this study uses only features based on depth information, and proposes a method of detecting the hand region by means of a classifier that uses boosting and cascading methods. First, in order to extract features using only depth information, we calculate the difference between the depth value at the center of the input image and the average of depth value within the segmented block, and to ensure that hand regions of all sizes will be detected, we use the central depth value and the second order linear model to predict the size of the hand region. The cascade method is applied to implement training and recognition by extracting features from the hand region. The classifier proposed in this paper maintains accuracy and enhances speed by composing each stage into a single weak classifier and obtaining the threshold value that satisfies the detection rate while exhibiting the lowest error rate to perform over-fitting training. The trained classifier is used to classify the hand region, and detects the final hand region in the final merger stage. Lastly, to verify performance, we perform quantitative and qualitative comparative analyses with various conventional AdaBoost algorithms to confirm the efficiency of the hand region detection algorithm proposed in this paper.

Anomalous Trajectory Detection in Surveillance Systems Using Pedestrian and Surrounding Information

  • Doan, Trung Nghia;Kim, Sunwoong;Vo, Le Cuong;Lee, Hyuk-Jae
    • IEIE Transactions on Smart Processing and Computing
    • /
    • v.5 no.4
    • /
    • pp.256-266
    • /
    • 2016
  • Concurrently detected and annotated abnormal events can have a significant impact on surveillance systems. By considering the specific domain of pedestrian trajectories, this paper presents two main contributions. First, as introduced in much of the work on trajectory-based anomaly detection in the literature, only information about pedestrian paths, such as direction and speed, is considered. Differing from previous work, this paper proposes a framework that deals with additional types of trajectory-based anomalies. These abnormal events take places when a person enters prohibited areas. Those restricted regions are constructed by an online learning algorithm that uses surrounding information, including detected pedestrians and background scenes. Second, a simple data-boosting technique is introduced to overcome a lack of training data; such a problem particularly challenges all previous work, owing to the significantly low frequency of abnormal events. This technique only requires normal trajectories and fundamental information about scenes to increase the amount of training data for both normal and abnormal trajectories. With the increased amount of training data, the conventional abnormal trajectory classifier is able to achieve better prediction accuracy without falling into the over-fitting problem caused by complex learning models. Finally, the proposed framework (which annotates tracks that enter prohibited areas) and a conventional abnormal trajectory detector (using the data-boosting technique) are integrated to form a united detector. Such a detector deals with different types of anomalous trajectories in a hierarchical order. The experimental results show that all proposed detectors can effectively detect anomalous trajectories in the test phase.