Browse > Article

A Branch-and-Bound Algorithm for Finding an Optimal Solution of Transductive Support Vector Machines  

Park Chan-Kyoo (동국대학교 경영학과)
Publication Information
Journal of the Korean Operations Research and Management Science Society / v.31, no.2, 2006 , pp. 69-85 More about this Journal
Abstract
Transductive Support Vector Machine(TSVM) is one of semi-supervised learning algorithms which exploit the domain structure of the whole data by considering labeled and unlabeled data together. Although it was proposed several years ago, there has been no efficient algorithm which can handle problems with more than hundreds of training examples. In this paper, we propose an efficient branch-and-bound algorithm which can solve large-scale TSVM problems with thousands of training examples. The proposed algorithm uses two bounding techniques: min-cut bound and reduced SVM bound. The min-cut bound is derived from a capacitated graph whose cuts represent a lower bound to the optimal objective function value of the dual problem. The reduced SVM bound is obtained by constructing the SVM problem with only labeled data. Experimental results show that the accuracy rate of TSVM can be significantly improved by learning from the optimal solution of TSVM, rather than an approximated solution.
Keywords
Transductive Support Vector Machine; Semi-supervised Learning; Branch-and-bound Algorithm; Optimal Solution;
Citations & Related Records
연도 인용수 순위
  • Reference
1 Bie, T. De and N. Christianini, 'Convex Methods for Transduction,' Advances in Neural Information Processing Systems, Vol. 16(2004), MIT Press
2 Cristianini, J. Shawe-Taylor, An Introduction to Support Vector Machines and other Kernel-based Learning Methods, Cambridge University Press, 2000
3 Schokopf, B. and A.J. Smola, Learning with Kernels：Support Vector Machines, Regularization, Optimization, and Beyond, MIT Press, 2002
4 Yu, S.X. and J. Shi, 'Grouping with Bias,' Advances in Neural Information Processing Systems, Vol.14(2001), MIT Press
5 Zhu, X., Z. Ghahramani, and J. Lafferty, 'Semi-supervised Learning using Gaussian Fields and Harmonic Functions,' Proceedings of the 20th International Conference on Machine Learning(ICML), 2003
6 Joachims, T., 'Transductive Inference for Text Classification using Support Vector Machines,' Proceedings of the 20th International Conference on Machine Learning (ICML), 1999
7 Shi, J. and J. Malik, 'Normalized Cuts and Image Segmentation,' IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol.22(2000), pp.888-905   DOI   ScienceOn
8 Bennett K.P. and A. Demiriz, 'Semi-supervised support vector machines,' Advances in Neural Information Processing Systems, Vol.10(1998), MIT Press
9 Joachims, T., 'Transductive Learning via Spectral Graph Partitioning,' Proceedings of the 20th International Conference on Machine Learning(ICML), 2003
10 Szummer, M. and T. Jaakkola, 'Partially Labeled Classification with Markov Random Walks,' Advances in Neural Information Processing Systems, Vol.14(2002), MIT Press
11 Chapelle, O., J. Weston, and B. Sch${kopf, 'Cluster Kernels for Semi-supervised Learning,' Advances in Neural Information Processing Systems, Vol.15(2003), MIT Press
12 Chang, C.-C. and C.-J. Lin, LIBSVM：A Library for Support Vector Machines, http: //www.csie.ntu.edu.tw/-cjlin/libsvm, 2001
13 Platt, J., 'Fast Training of Support Vector Machines using Sequential Minimal Optimization,' In B. Schokopf, C.J.C. Burges, and A.J. Smola, editors, Advances in Kernel Methods-Support Vector Learning, pp. 185-208
14 Blum, A. and S. Chawla, 'Learning from Labeled and Unlabeled Data using Graph Mincuts,' Proceedings of the 18th International Conference on Machine Learning (ICML), 2001
15 Seeger, M., Learning with Labeled and Unlabeled Data, Technical Report, Institute for Adaptive and Neural Computation, University of Edinburgh, 2001
16 Vapnik, V., Statistical Learning Theory, Wiley, 1998
17 Nilsson, N.J., Introduction to Machine Learning, http://robotics.stanford.edu/people/ nilsson/mlbook.html