Browse > Article

On Optimizing LDA-extentions Using a Pre-Clustering  

Kim, Sang-Woon (Dept of Computer Engineering, Myongji University)
Koo, Byum-Yong (Dept of Electronic Engineering, Myongji University)
Choi, Woo-Young (Dept of Electronic Engineering, Myongji University)
Publication Information
Journal of the Institute of Electronics Engineers of Korea CI / v.44, no.3, 2007 , pp. 98-107 More about this Journal
Abstract
For high-dimensional pattern recognition, such as face classification, the small number of training samples leads to the Small Sample Size problem when the number of pattern samples is smaller than the number of dimensionality. Recently, various LDA-extensions have been developed, including LDA, PCA+LDA, and Direct-LDA, to address the problem. This paper proposes a method of improving the classification efficiency by increasing the number of (sub)-classes through pre-clustering a training set prior to the execution of Direct-LDA. In LDA (or Direct-LDA), since the number of classes of the training set puts a limit to the dimensionality to be reduced, it is increased to the number of sub-classes that is obtained through clustering so that the classification performance of LDA-extensions can be improved. In other words, the eigen space of the training set consists of the range space and the null space, and the dimensionality of the range space increases as the number of classes increases. Therefore, when constructing the transformation matrix, through minimizing the null space, the loss of discriminatve information resulted from this space can be minimized. Experimental results for the artificial data of X-OR samples as well as the bench mark face databases of AT&T and Yale demonstrate that the classification efficiency of the proposed method could be improved.
Keywords
PCA; LDA; Direct-LDA;
Citations & Related Records
Times Cited By KSCI : 1  (Citation Analysis)
연도 인용수 순위
1 S. -W. Kim and B. J. Oommen, 'On optimizing kernel-based Fisher discriminant analysis using prototype reduction schemes', Lecture Note in Computer Science, vol. LNCS-4109, pp. 826-834, 2006   DOI   ScienceOn
2 M. Zhu and A. M. Martinez, 'Subclass discriminant analysis', IEEE Trans. Pattern Anal. and Machine Intell., vol. 28, no. 8, pp. 1274-1286, Aug. 2006   DOI   ScienceOn
3 S. -W. Kim, 'On solving the small sample size problem using a dissimilarity representation for face recognition', Lecture Note in Computer Science, vol. LNCS-4179, pp. 1174-1185, 2006   DOI   ScienceOn
4 G. Baudat and F. Anouar, 'Generalized Discriminant Analysis Using a Kernel Approach' Neural Comput., vol. 12, pp. 2385 - 2404, 2000   DOI
5 H. Cevikalp, M. Neamtu, M. Wilkes, and A. Barkana, 'Discriminative common vectors for face recognition', IEEE Trans. Pattern Anal. and Machine Intell., vol. 27, no. 1, pp. 4-13, Jan 2005   DOI   ScienceOn
6 L. Chen, H. Liao, M. Ko, J. Lin, and G. Yu, 'A new LDA-based face recognition system which can solve the small sample size problem', Pattern Recognition, vol. 33, pp. 1713-1726, 2000   DOI   ScienceOn
7 김상운, 'Prototype Reduction Schemes와 Mahala-nobis거리를 이용한 Relational Discriminant Analysis,' 대한전자공학회논문지-CI, vol. 43, no. 1, pp. 9 - 16, 2006. 1   과학기술학회마을
8 H. Gao and J. W. Davis, 'Why direct LDA is not equivalent to LDA', Pattern Recognition vol. 39, pp. 1002-1006, 2006   DOI   ScienceOn
9 J. Ye, R. Janardan, C. H Park, and H. Park, 'An optimization criterion for generalized discriminant analysis on undersampled problems', IEEE Trans. Pattern Anal. and Machine Intell., vol. 26, no. 8, pp. 982 -994, Aug. 2004   DOI   ScienceOn
10 P. Yu and J. Yang, 'A direct LDA algorithm for high-dimensional data with application to face recognition', Pattern Recognition, vol. 34, pp. 2067-2070, 2001   DOI   ScienceOn
11 A. K. Jain, R. P. W. Duin, and J. Mao, 'Statical pattern recognition: a review', IEEE Trans. Pattern Anal. Intell. vol. 22, no. 1 pp. 4-37, March 2000   DOI   ScienceOn
12 M. Turk and A. P. Pentland, 'Face recognition using eigenfaces,' Proc. of IEEE Conf. on Computer Vision and Pattern Recognition. pp. 586-591, 1991   DOI
13 P. N. Belhumeur, J. P. Hespanha, and D. J. Kriegman, 'Eigenfaces vs. Fisherfaces: recognition using class specific linear projection,'IEEE Trans. Pattern Anal. Machine Intell., vol. 19, no. 7, pp. 711-720, May 1997   DOI   ScienceOn
14 A. K. Jain and B. Chandrasekaran, 'Dimensionality and sample size consideration in pattern recognition', Handbook of Statistics, P. R. Krishnaiah and L. N. Kanal, Eds., vol. 2, pp. 835-855, 1982
15 K. Fukunaga, Introduction to Statistical Pattern Recognition, Second Edition, Academic Press, San Diego, pp. 452-459, 1990
16 R. Duda, P. Hart, and D. Stork, Pattern Classification, Wiley, New York, 2001
17 김상운, MATLAB으로 배우는 패턴인식 및 학습, 홍릉과학출판사, 서울, 2005. 1