Journal of the Korean Data and Information Science Society

The Korean Data and Information Science Society (한국데이터정보과학회)
권호 표지
DOI QR코드

DOI QR Code

Support vector machines for big data analysis

빅 데이터 분석을 위한 지지벡터기계

  • Choi, Hosik (Department of Informational Statistics, Hoseo University) ;
  • Park, Hye Won (Department of Statistics, University of Seoul) ;
  • Park, Changyi (Department of Statistics, University of Seoul)
  • 최호식 (호서대학교 정보통계학과) ;
  • 박혜원 (서울시립대학교 통계학과) ;
  • 박창이 (서울시립대학교 통계학과)
  • Received : 2013.06.19
  • Accepted : 2013.08.05
  • Published : 2013.09.30
Download PDF
⟨ Previous Next ⟩

Abstract

We cannot analyze big data, which attracts recent attentions in industry and academy, by batch processing algorithms developed in data mining because big data, by definition, cannot be uploaded and processed in the memory of a single system. So an imminent issue is to develop various leaning algorithms so that they can be applied to big data. In this paper, we review various algorithms for support vector machines in the literature. Particularly, we introduce online type and parallel processing algorithms that are expected to be useful in big data classifications and compare the strengths, the weaknesses and the performances of those algorithms through simulations for linear classification.

최근 산/학계에서 주목받고 있는 빅 데이터는 정의상 한꺼번에 자료를 메모리에 올려 분석할 수 없기 때문에 기존의 데이터마이닝 시대에 개발된 일괄처리 (batch processing) 방식의 알고리즘을 적용할 수 없게 된다. 따라서 가장 시급히 해결해야 하는 문제는 기존의 여러 가지 기계학습방법을 빅 데이터에 적용할 수 있도록 분산처리 (distributed processing)를 수행하는 적절한 알고리즘을 개발하는 것이라 볼 수 있다. 본 논문에서는 분류문제에서 각광받는 지지벡터기계 (support vector machines)의 여러 알고리즘을 살펴보고자 한다. 특히 빅 데이터 분류문제에 유용할 것으로 예상되는 온라인 타입 알고리즘과 병렬처리 알고리즘에 대하여 소개하고, 이러한 알고리즘들의 성능 및 장단점을 선형분류에 대한 모의실험을 통해서 살펴본다.

Keywords

References

  1. Bordes, A., Bottou, L. and Gallinari, P. (2008). SGD-QN: Careful quasi-Newton stochastic gradient descent. Journal of Machine Learning Research, 10, 1737-1754.
  2. Bottou, L. and Bousquet, O. (2008). The tradeoffs of large scale learning. In Advances in Neural Information Processing Systems, 20, 161-168.
  3. Boyd, S., Parikh, N., Chu, E., Peleato, B. and Eckstein, J. (2010). Distributed optimization and statistical learning via the alternating direction method of multipliers. Foundations and Trendsr in Machine Learning, 3, 1-122. https://doi.org/10.1561/2200000016
  4. Cortes, C. and Vapnik, V. (1995). Support vector networks. Machine Learning, 20, 273-297.
  5. Duchi, J. and Singer, Y. (2009). Efficient online and batch learning using forward-backward splitting. Journal of Machine Learning Research, 10, 2873-2898.
  6. Fan, R.-E., Chen, P.-H. and Lin, C.-J. (2005). Working set selection using second order information for training SVM. Journal of Machine Learning Research, 6, 1889-1918.
  7. Forero, P. A., Cano, A. and Giannakis, G. B. (2010). Consensus-based distributed support vector machines. Journal of Machine Learning Research, 11, 1663-1707.
  8. Franc, V. and Sonnenburg, S. (2008). Optimized cutting plane algorithm for support vector machines. In Proceedings of the 25th International Conference on Machine Learning, ACM, 320-327.
  9. Hsieh, C.-J., and Chang, K.-W., Lin, C.-J., Keerthi, S. S and Sundararajan, S. (2008). A dual coordinate descent method for large-scale linear SVM. In Proceedings of the 25th International Conference on Machine Learning, ACM, 408-415.
  10. Park, C., Kim, Y., Kim, J., Song, J. and Choi, H. (2013). Data mining using R, 2nd Edition, Kyowoo Publisher, Seoul.
  11. Park, D.-J., Yun, Y.-B. and Yoon, M. (2012). Prediction of bankruptcy data using machine learning techniques. Journal of the Korean Data & Information Science Society, 23, 569-577. https://doi.org/10.7465/jkdi.2012.23.3.569
  12. Park, H.-J. (2011). Online abnormal events detection with online support vector machine. Journal of the Korean Data & Information Science Society, 22, 197-206.
  13. Pi, S.-Y., Park, H.-J. and Ryu, K.-H. (2011) An analysis of satisfaction index on computer education of university using kernel machine. Journal of the Korean Data & Information Science Society, 22, 921-929.
  14. Platt, J. C. (1999). Fast training of support vector machines using sequential minimal optimization. In Advances in Kernal Methods - Support Vector Learning, MIT Press, 185-208.
  15. Shalev-Shwartz, S., Singer, Y., Srebro, N. and Cotter, A. (2011). Pegasos: Primal estimated sub-gradient solver for SVM. Mathematical Programming B, 127, 3-30. https://doi.org/10.1007/s10107-010-0420-4
  16. Smola, A. J., Vishwanathan, S. V. N. and Le, Q. V. (2007). Bundle methods for machine learning. In Advances in Neural Information Processing Systems, 20, MIT Press, 1377-1384.
  17. Zou, H. and Hastie, T. (2005). Regularization and variable selection via the elastic net. Journal of the Royal Statistical Society B, 67, 301-320. https://doi.org/10.1111/j.1467-9868.2005.00503.x

Cited by

  1. Study on the social issue sentiment classification using text mining vol.26, pp.5, 2015, https://doi.org/10.7465/jkdi.2015.26.5.1167
  2. Structuring of unstructured big data and visual interpretation vol.25, pp.6, 2014, https://doi.org/10.7465/jkdi.2014.25.6.1431
  3. Big data mining for natural disaster analysis vol.26, pp.5, 2015, https://doi.org/10.7465/jkdi.2015.26.5.1105
  4. Development of Heavy Rain Damage Prediction Functions in the Seoul Capital Area Using Machine Learning Techniques vol.18, pp.7, 2018, https://doi.org/10.9798/KOSHAM.2018.18.7.435
  5. 통계적 기계학습에서의 ADMM 알고리즘의 활용 vol.28, pp.6, 2017, https://doi.org/10.7465/jkdi.2017.28.6.1229