KIISE Transactions on Computing Practices (정보과학회 컴퓨팅의 실제 논문지)

Korean Institute of Information Scientists and Engineers (한국정보과학회)
권호 표지
DOI QR코드

DOI QR Code

Oversampling-Based Ensemble Learning Methods for Imbalanced Data

불균형 데이터 처리를 위한 과표본화 기반 앙상블 학습 기법

  • Received : 2014.01.23
  • Accepted : 2014.09.09
  • Published : 2014.10.15
⟨ Previous Next ⟩

Abstract

Handwritten character recognition data is usually imbalanced because it is collected from the natural language sentences written by different writers. The imbalanced data can cause seriously negative effect on the performance of most of machine learning algorithms. But this problem is typically ignored in handwritten character recognition, because it is considered that most of difficulties in handwritten character recognition is caused by the high variance in data set and similar shapes between characters. We propose the oversampling-based ensemble learning methods to solve imbalanced data problem in handwritten character recognition and to improve the recognition accuracy. Also we show that proposed method achieved improvements in recognition accuracy of minor classes as well as overall recognition accuracy empirically.

필기체 낱글자 인식을 위해서 사용되는 데이터는 일반적으로 다수의 사용자들로부터 수집된 자연언어 문장들을 이용하기 때문에 해당 언어의 언어적 특성에 따라서 낱글자의 종류별 개수 차이가 매우 큰 특징이 있다. 일반적인 기계학습 문제에서 학습데이터의 불균형 문제는 성능을 저하시키는 중요한 요인으로 작용하지만, 필기체 인식에서는 데이터 자체의 높은 분산과 비슷한 모양의 낱글자 등이 성능 저하의 주요인이라 생각하기 때문에 이를 크게 고려하지 않고 있다. 본 논문에서는 이러한 데이터의 불균형 문제를 고려하여 필기체 인식기의 성능을 향상시킬 수 있는 과표본화 기반의 앙상블 학습 기법을 제안한다. 제안한 방법은 데이터의 불균형 문제를 고려하지 않은 방법보다 전체적으로 향상된 성능을 보일 뿐만 아니라 데이터의 개수가 부족한 낱글자들의 분류성능에 있어서도 향상된 결과를 보여준다.

Keywords

Acknowledgement

Supported by : 한국연구재단

References

  1. S. Ertekin, J. Huang, L. Bottou, L. Giles, "Learning on the border: active learning in imbalanced data classification," Proc. of ACM conference on Conference on Information and Knowledge Management, pp. 127-136, 2007.
  2. A. Estabrooks, T. Jo, and N. Japkowicz, "A multiple resampling method for learning from imbalanced data sets," Computational Intelligence, Vol. 20, No. 1, pp. 18-36, 2004. https://doi.org/10.1111/j.0824-7935.2004.t01-1-00228.x
  3. H. He, and E. A. Garcia, "Learning from Imbalanced Data," IEEE Transactions on knowledge and data engineering, Vol. 21, No. 9, pp. 1236-1284, 2009.
  4. N. V. Chawla, L. O. Hall, K. W. Bowyer, and W. P. Kegelmeyer, "Smote: Synthetic minority oversampling technique," Journal of Artificial Intelligence Research, Vol. 16, pp. 321-357, 2002.
  5. H. Han, W. Wang, B. Mao, "Borderlinesmote: A new over-sampling method in imbalanced data sets learning," Proc. of International Conference on Intelligent Computing, pp. 878-887, 2005.
  6. H. He, Y. Bai, E.A. Garcia, S. Li, "ADASYN: Adaptive Synthetic Sampling Approach for Imbalanced Learning," Proc. of International Joint Conference on Neural Networks, pp. 1322-1328, 2008.
  7. J. V. Hulse, T. M. Khoshgoftaar, A. Napolitano, "Experimental perspectives on learning from imbalanced data," Proc. of International Conference on Machine Learning, pp. 935-942, 2007.
  8. X. Y. Liu, J. Wu, Z. H. Zhou, "Exploratory Under Sampling for Class Imbalance Learning," Proc. of International Conference on Data Mining, pp. 965- 969, 2006.
  9. M. Kubat, S. Matwin, "Addressing the Curse of Imbalanced Training Sets: One-Sided Selection," Proc. of International Conference on Machine Learning, pp. 179-186, 1997.
  10. L. Breiman, "Bagging predictors," Machine Learning, Vol. 24, No. 2, pp. 123-140, 1996.
  11. Y. Freund and R. E. Schapire, "A decision-theoretic generalization of on-line learning and an application to boosting," Journal of Computer and System Sciences, Vol. 55, No. 1. pp. 119-139, 1997. https://doi.org/10.1006/jcss.1997.1504
  12. T. J. Kim, H. Y. Jang, J. W. Park, S. T. Hwang, B. T. Zhang, "Ensemble Methods with increasing data for online handwriting recognition," Proc. of the KIISE Korea Computer Congress 2013, pp. 1396- 1398, 2013.
  13. I. Guyon, L. Schomaker, R. Plamondon, M. Liberman, S. Janet, "UNIPEN project of on-line data exchange and recognizer benchmarks," Proc. of International Conferences on Pattern Recognition, pp. 29-33, 1994.