Journal of the Institute of Electronics Engineers of Korea CI (전자공학회논문지CI)

The Institute of Electronics and Information Engineers (대한전자공학회)
권호 표지

On Optimizing LDA-extentions Using a Pre-Clustering

사전 클러스터링을 이용한 LDA-확장법들의 최적화

  • 김상운 (명지대학교 컴퓨터공학과) ;
  • 구범용 (명지대학교 전자공학과) ;
  • 최우영 (명지대학교 전자공학과)
  • Published : 2007.05.25
Download PDF
⟨ Previous Next ⟩

Abstract

For high-dimensional pattern recognition, such as face classification, the small number of training samples leads to the Small Sample Size problem when the number of pattern samples is smaller than the number of dimensionality. Recently, various LDA-extensions have been developed, including LDA, PCA+LDA, and Direct-LDA, to address the problem. This paper proposes a method of improving the classification efficiency by increasing the number of (sub)-classes through pre-clustering a training set prior to the execution of Direct-LDA. In LDA (or Direct-LDA), since the number of classes of the training set puts a limit to the dimensionality to be reduced, it is increased to the number of sub-classes that is obtained through clustering so that the classification performance of LDA-extensions can be improved. In other words, the eigen space of the training set consists of the range space and the null space, and the dimensionality of the range space increases as the number of classes increases. Therefore, when constructing the transformation matrix, through minimizing the null space, the loss of discriminatve information resulted from this space can be minimized. Experimental results for the artificial data of X-OR samples as well as the bench mark face databases of AT&T and Yale demonstrate that the classification efficiency of the proposed method could be improved.

얼굴인식 등과 같은 고차원 패턴인식에서 학습패턴의 수가 패턴 차원에 비해 매우 적을 경우 희소성 문제(the Small Sample Size problem)가 발생한다. 최근 이 문제를 해결하기 위하여 LDA, PCA+LDA, Direct-LDA 등을 비롯한 다양한 LDA-확장 법이 제안되었다. 본 논문에서는 LDA-확장 법으로 차원을 축소하기 전에 학습 패턴을 사전 클러스터링하여 서브 클래스 수를 증가시키는 방법으로 LDA-확장에 기반을 둔 식별기의 성능을 향상시키는 방법을 제안한다. LDA (또는 Direct-LDA)에서 축소된 특징공간의 차원은 학습패턴의 클래스 수로 제한되기 때문에 LDA의 식별 성능을 향상시킬 수 있도록 학습패턴을 사전에 클러스터링하여 서브 클래스의 수를 증가시키는 방법이다. 즉, 학습패턴의 특성공간(the eigen space)은 레인지 공간(the range space)과 널 공간(the null space)으로 구성되며, 레인지 공간의 차원은 클래스 수의 증가에 따라 증가한다. 따라서 변환 행렬을 구성할 때 클래스의 수를 늘려 널 공간을 최소화하게 되면 이 공간에 기인한 정보의 손실을 최소화 할 수 있다. 제안 방법을 X-OR 형태의 인공데이터와 AT&T와 Yale 벤취마크 얼굴영상 데이터베이스를 대상으로 실험한 결과 본 방법의 효용성을 확인하였다.

Keywords

References

  1. A. K. Jain, R. P. W. Duin, and J. Mao, 'Statical pattern recognition: a review', IEEE Trans. Pattern Anal. Intell. vol. 22, no. 1 pp. 4-37, March 2000 https://doi.org/10.1109/34.824819
  2. K. Fukunaga, Introduction to Statistical Pattern Recognition, Second Edition, Academic Press, San Diego, pp. 452-459, 1990
  3. R. Duda, P. Hart, and D. Stork, Pattern Classification, Wiley, New York, 2001
  4. 김상운, MATLAB으로 배우는 패턴인식 및 학습, 홍릉과학출판사, 서울, 2005. 1
  5. A. K. Jain and B. Chandrasekaran, 'Dimensionality and sample size consideration in pattern recognition', Handbook of Statistics, P. R. Krishnaiah and L. N. Kanal, Eds., vol. 2, pp. 835-855, 1982
  6. M. Turk and A. P. Pentland, 'Face recognition using eigenfaces,' Proc. of IEEE Conf. on Computer Vision and Pattern Recognition. pp. 586-591, 1991 https://doi.org/10.1109/CVPR.1991.139758
  7. P. N. Belhumeur, J. P. Hespanha, and D. J. Kriegman, 'Eigenfaces vs. Fisherfaces: recognition using class specific linear projection,'IEEE Trans. Pattern Anal. Machine Intell., vol. 19, no. 7, pp. 711-720, May 1997 https://doi.org/10.1109/34.598228
  8. J. Ye, R. Janardan, C. H Park, and H. Park, 'An optimization criterion for generalized discriminant analysis on undersampled problems', IEEE Trans. Pattern Anal. and Machine Intell., vol. 26, no. 8, pp. 982 -994, Aug. 2004 https://doi.org/10.1109/TPAMI.2004.37
  9. L. Chen, H. Liao, M. Ko, J. Lin, and G. Yu, 'A new LDA-based face recognition system which can solve the small sample size problem', Pattern Recognition, vol. 33, pp. 1713-1726, 2000 https://doi.org/10.1016/S0031-3203(99)00139-9
  10. P. Yu and J. Yang, 'A direct LDA algorithm for high-dimensional data with application to face recognition', Pattern Recognition, vol. 34, pp. 2067-2070, 2001 https://doi.org/10.1016/S0031-3203(00)00162-X
  11. H. Gao and J. W. Davis, 'Why direct LDA is not equivalent to LDA', Pattern Recognition vol. 39, pp. 1002-1006, 2006 https://doi.org/10.1016/j.patcog.2005.11.016
  12. H. Cevikalp, M. Neamtu, M. Wilkes, and A. Barkana, 'Discriminative common vectors for face recognition', IEEE Trans. Pattern Anal. and Machine Intell., vol. 27, no. 1, pp. 4-13, Jan 2005 https://doi.org/10.1109/TPAMI.2005.9
  13. 김상운, 'Prototype Reduction Schemes와 Mahala-nobis거리를 이용한 Relational Discriminant Analysis,' 대한전자공학회논문지-CI, vol. 43, no. 1, pp. 9 - 16, 2006. 1
  14. S. -W. Kim, 'On solving the small sample size problem using a dissimilarity representation for face recognition', Lecture Note in Computer Science, vol. LNCS-4179, pp. 1174-1185, 2006 https://doi.org/10.1007/11864349_107
  15. G. Baudat and F. Anouar, 'Generalized Discriminant Analysis Using a Kernel Approach' Neural Comput., vol. 12, pp. 2385 - 2404, 2000 https://doi.org/10.1162/089976600300014980
  16. S. -W. Kim and B. J. Oommen, 'On optimizing kernel-based Fisher discriminant analysis using prototype reduction schemes', Lecture Note in Computer Science, vol. LNCS-4109, pp. 826-834, 2006 https://doi.org/10.1007/11815921_91
  17. M. Zhu and A. M. Martinez, 'Subclass discriminant analysis', IEEE Trans. Pattern Anal. and Machine Intell., vol. 28, no. 8, pp. 1274-1286, Aug. 2006 https://doi.org/10.1109/TPAMI.2006.172