Management Science and Financial Engineering

The Korean Operations Research and Management Science Society (한국경영과학회)
권호 표지

Implementing a Branch-and-bound Algorithm for Transductive Support Vector Machines

  • Received : 2009.12.21
  • Accepted : 2010.04.08
  • Published : 2010.05.31
Download PDF
⟨ Previous Next ⟩

Abstract

Semi-supervised learning incorporates unlabeled examples, whose labels are unknown, as well as labeled examples into learning process. Although transductive support vector machine (TSVM), one of semi-supervised learning models, was proposed about a decade ago, its application to large-scaled data has still been limited due to its high computational complexity. Our previous research addressed this limitation by introducing a branch-and-bound algorithm for finding an optimal solution to TSVM. In this paper, we propose three new techniques to enhance the performance of the branch-and-bound algorithm. The first one tightens min-cut bound, one of two bounding strategies. Another technique exploits a graph-based approximation to a support vector machine problem to avoid the most time-consuming step. The last one tries to fix the labels of unlabeled examples whose labels can be obviously predicted based on labeled examples. Experimental results are presented which demonstrate that the proposed techniques can reduce drastically the number of subproblems and eventually computational time.

Keywords

References

  1. Asuncion, A. and D. J. Newman, UCI Machine Learning Repository (http://www. ics.uci.edu/: mlearn/MLRepository.html), Irvine, CA: University of California, School of Information and Computer Science, 2007.
  2. Bennett, K. P. and A. Demiriz, "Semi-supervised support vector machines," Advance in Neural Information Processing Systems(NIPS) 10 (1998), MIT Press.
  3. Bie, T. a nd De, N. Christianini, "Convex methods for transduction," Advances in Neural Information Processing Systems(NIPS) 16 (2004), MIT Press.
  4. Blum, A. and T. Mitchell, "Combining labeled and unlabeled data with co-training," COLT: Proceedings of the Workshop on Computational Learning Theory 1998.
  5. Blum, A. and S. Chawla, "Learning from labeled and unlabeled data using graph mincuts," Proceedings of the 18th International Conference on Machine Learning(ICML) ACM Press, 2001.
  6. Chang, C-C. and C-J. Lin, UBSVM: a library for support vector machines, http://www.csie.ntu.edu.tw/cjlin/libsvm. 2001.
  7. Chapelle, O., J. Weston, and B. Scholkopf, "Cluster kernels for semi-supervised learning," Advances in Neural Information Processing Systems (NIPS) 15 (2003), MIT Press.
  8. Cristianini, J. Shawe-Taylor, An introduction to support vector machines and other kernel-based learning methods, Cambridge University Press, 2000.
  9. Culp, M. and G. Michailidis, "An iterative algorithm for extending learners to a semi supervised setting," The 2007 Joint Statistical Meeting(JSM) 2007.
  10. Hull, J. J., "A database for handwritten text recognition research," IEEE Transactions on Pattern Analysis and Machine Intelligence 16 (1994), 550-554. https://doi.org/10.1109/34.291440
  11. Joachims, T., "Transductive inference for text classification using support vector machines," Proceedings of the 20th International Conference on Machine Learning(ICML) ACM Press, 1999.
  12. Joachims, T., "Transductive learning via spectral graph partitioning," Proceedings of the 20th International Conference on Machine Learning(ICML), ACM Press, 2003.
  13. Olivier, C., V. Sindhwani, and S. S. Keerthi, "Branch and bound for semisupervised support vector machines," Advances in Neural Information Processing Systems(NIPS) 18 (2006), MIT Press.
  14. Park, C-K., "A branch-and-bound algorithm for finding an optimal solution of transductive support vector machines," Journal of the Korean Operations Research and Management Science Society 31,2 (2006), 69-85.
  15. Platt, J., "Fast training of support vector machines using sequential minimal optimization," In: B. Scholkopf, C. J. C. Burges, and A. J. Smola, eds., Advances in Kernel Methods-Support Vector Learning, MIT Press, (1999), 85-208.
  16. Ratsaby, J. and S. Venkatesh, "Learning from a mixture of labeled and unlabeled examples with parametric side information," Proceedings of the Eighth Annual Conference on Computational Learning Theory (1995), 412-417.
  17. Scholkopf, B. and A. J. Smola, Learning with Kernels: Support Vector Machines, Regularization, Optimization, and Beyond, MIT Press, 2002.
  18. Shi, J. and J. Malik, "Normalized cuts and image segmentation," IEEE Transactions on Pattern Analysis and Machine Intelligence 22 (2000), 888-905. https://doi.org/10.1109/34.868688
  19. Szummer, M., T. Jaakkola, "Partially labeled classification with Markov random walks," Advances in Neural Information Processing Systems(NIPS) 14 (2002), MIT Press.
  20. Vapnik, V., Statistical Learning Theory, Wiley, 1998.
  21. Virginia, R., "Learning classification with unlabeled data," Advances in Neural Information Processing Systems(NIPS) 5 (1993), MIT Press.
  22. Yu, S. X. and J. Shi, "Grouping with bias," Advances in Neural Information Processing Systems(NIPS) 14 (2001), MIT Press
  23. Zhu, X., Z. Ghahramani, and J. Lafferty, "Semi-supervised learning using Gaussian fields and harmonic functions," Proceedings of the 20th International Conference on Machine Learning (ICML), ACM Press, 2003.
  24. Zhu, X. and A. B. Goldberg, Introduction to Semi-Supervised Learning, Morgan and Claypool Publishers, 2009.