KSII Transactions on Internet and Information Systems (TIIS)

Korean Society for Internet Information (한국인터넷정보학회)
권호 표지
DOI QR코드

DOI QR Code

An Improved method of Two Stage Linear Discriminant Analysis

  • Chen, Yarui (Computer Science and Information Engineering Institute, Tianjin University of Science and Technology) ;
  • Tao, Xin (Computer Science and Information Engineering Institute, Tianjin University of Science and Technology) ;
  • Xiong, Congcong (Computer Science and Information Engineering Institute, Tianjin University of Science and Technology) ;
  • Yang, Jucheng (Computer Science and Information Engineering Institute, Tianjin University of Science and Technology)
  • Received : 2017.06.22
  • Accepted : 2017.11.19
  • Published : 2018.03.31
Download PDF
⟨ Previous Next ⟩

Abstract

The two-stage linear discrimination analysis (TSLDA) is a feature extraction technique to solve the small size sample problem in the field of image recognition. The TSLDA has retained all subspace information of the between-class scatter and within-class scatter. However, the feature information in the four subspaces may not be entirely beneficial for classification, and the regularization procedure for eliminating singular metrics in TSLDA has higher time complexity. In order to address these drawbacks, this paper proposes an improved two-stage linear discriminant analysis (Improved TSLDA). The Improved TSLDA proposes a selection and compression method to extract superior feature information from the four subspaces to constitute optimal projection space, where it defines a single Fisher criterion to measure the importance of single feature vector. Meanwhile, Improved TSLDA also applies an approximation matrix method to eliminate the singular matrices and reduce its time complexity. This paper presents comparative experiments on five face databases and one handwritten digit database to validate the effectiveness of the Improved TSLDA.

Keywords

References

  1. Christopher Bishop, "Pattern Recognition and Machine Learning," Springer, Germany, 2008.
  2. M Imani and H Ghassemian, "Two Dimensional Linear Discriminant Analyses for Hyperspectral Data," Photogrammetric Engineering & Remote Sensing, vol.81, no.10, pp.777-786, 2015. https://doi.org/10.14358/PERS.81.10.777
  3. A Sharma and KK Paliwal, "Linear discriminant analysis for the small sample size problem: an overview," International Journal of Machine Learning & Cybernetics, vol.6, no.3, pp. 443-454, 2015. https://doi.org/10.1007/s13042-013-0226-9
  4. P.N. Belhumeur, J.P. Hespanha and D.J. Kriegman, "Eigenfaces vs. Fisherfaces: Recognition using class specific linear projection," IEEE Transactions on Pattern Analysis & Machine Intelligence, vol.19, no.7, pp. 711-720, 1997. https://doi.org/10.1109/34.598228
  5. JW Lu, KN Plataniotis and AN Venetsanopoulos, "Regularization studies of linear discriminant analysis in small sample size scenarios with application to face recognition," Pattern Recognition Letter, vol.26, no.2, pp. 181-191, 2005. https://doi.org/10.1016/j.patrec.2004.09.014
  6. A Sharma and KK Paliwal, "A deterministic approach to regularized linear discriminant analysis," Neurocomputing, vol.151, no.11, pp. 207-214, 2015. https://doi.org/10.1016/j.neucom.2014.09.051
  7. A Sharma and KK Paliwal, "Approximate LDA Technique for Dimensionality Reduction in the Small Sample Size Case," Journal of Pattern Recognition Research, pp.298-306, 2011.
  8. S Wang, "Linear Discriminant Analysis Based on Clustering Regularization," Master's thesis of Tianjin University, 2013.
  9. A Sharma, KK Paliwal, Seiya Imoto and Satoru Miyano, "A feature selection method using improved regularized linear discriminant analysis," Machine Vision and Applications, vol.25, no.25, pp. 775-786, 2014. https://doi.org/10.1007/s00138-013-0577-y
  10. Y Liu and SZ liao, "Granularity selection for cross-validation of SVM," Information Sciences, vol. 378, pp. 475-483,2017 https://doi.org/10.1016/j.ins.2016.06.051
  11. WY Yang and HY Wu, "Regularized complete linear discriminant analysis," Neurocomputing, vol. 137, no.11, pp. 185-191, 2014. https://doi.org/10.1016/j.neucom.2013.08.048
  12. LF Chen, HYM Liao, MT Ko, JC Lin and GJ Yu, "A new LDA-based face recognition system which can solve the small sample size problem," Pattern Recognition, vol.33, no.10, pp. 1713-1726, 2000. https://doi.org/10.1016/S0031-3203(99)00139-9
  13. H Yu and H Yang, "A direct LDA algorithm for high-dimensional data-with application to face recognition," Pattern Recognition, vol.34, no.10, pp. 2067-2070, 2001. https://doi.org/10.1016/S0031-3203(00)00162-X
  14. A Sharma and Kuldip K Paliwal, "A two-stage linear discriminant analysis for face-recognition," Pattern Recognition Letters, vol.33, no.9, pp. 1157-1162, 2012. https://doi.org/10.1016/j.patrec.2012.02.001
  15. KK Paliwal and A Sharma, "A Improved direct LDA and its application to DNA microarray gene expression data," Pattern Recognition Letters, vol.31, no.16, pp. 2489-2492, 2010. https://doi.org/10.1016/j.patrec.2010.08.003
  16. XS Zhuang, DQ Dai, "Inverse Fisher discriminate criteria for small sample size problem and its application to face recognition," Pattern Recognition, vol.38, no.11, pp.2192-2194, 2005. https://doi.org/10.1016/j.patcog.2005.02.011
  17. Pi-Fuei Hsieh, "Impact and realization of increased class separability on the small sample size problem in hyperspectral classification," Canadian Journal of Remote Sensing, vol.25, no.3, pp: 248-261, 2009.
  18. Fadi Dornaika and ALireza Bosagzadeh, "On Solving the Small Sample Size Problem for Marginal Fisher Analysis," Image Analysis and Recognition, pp.116-123, 2013.
  19. JH Zhao, L Shi and J Zhu, "Two-Stage Regularized Linear Discriminant Analysis for 2-D Data," IEEE Transactions on Neural Networks And Learning Systems, vol.26, no.8, pp.1669-1681, 2015. https://doi.org/10.1109/TNNLS.2014.2350993
  20. MW Jian and KM Lam, "Simultaneous Hallucination and Recognition of Low-Resolution Faces Based on Singular Value Decomposition," IEEE Transactions on Circuits and Systems for Video Technology, vol.25, no.11 pp. 1761-1772, 2015. https://doi.org/10.1109/TCSVT.2015.2400772
  21. MWJian,KM Lam and JY Dong. "A Novel Face-Hallucination Scheme Based on Singular Value Decomposition," Pattern Recognition, vol. 46, no. 11, pp. 3091-3102, November 2013. https://doi.org/10.1016/j.patcog.2013.03.020
  22. MW Jian, KM Lam and JY Dong. "Facial-Feature Detection and Localization Based on a Hierarchical Scheme," Information Sciences, vol. 262, pp. 1-14, 2014. https://doi.org/10.1016/j.ins.2013.12.001
  23. MW Jian and KM Lam, "Face-Image Retrieval Based on Singular Values and Potential-Field Representation," Signal Processing, vol. 100, pp. 9-15, 2014. https://doi.org/10.1016/j.sigpro.2014.01.004
  24. YD Wen, KP Zhang, ZF Li and Y Qiao, "A Discriminative Feature Learning Approach for Deep Face Recognition," in Proc. of the 14th European Conference of Computer Vision (ECCV 2016), Amsterdam, Netherlands, 499-515, October 2016.
  25. ZC Li, J Liu, JH Tang and HQ Lu, "Robust Structured Subspace Learning for Data Representation," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol.37, no.10, pp.2085-2098. https://doi.org/10.1109/TPAMI.2015.2400461
  26. ZC Li, J Liu, Y Yang, XF Zhou and HQ Lu, "Clustering-Guided Sparse Structural Learning for Unsupervised Feature Selection," IEEE Transactions On Knowledge And Data Engineering, vol.26, no.9, pp. 2138-2150, 2014. https://doi.org/10.1109/TKDE.2013.65
  27. J Gui, ZN Sun, J Cheng and SW Ji, "How to Estimate the Regularization Parameter for Spectral Regression Discriminant Analysis and its Kernel Version?" IEEE Transactions On Circuits And Systems For Video Technology, vol.24, no.2, pp.211-223, 2014. https://doi.org/10.1109/TCSVT.2013.2273652
  28. Andrew R. Webb, Statistical Pattern Recognition, 3nd Edition, Wiley, America, 2011.
  29. JP Ye and Q Li, "A two-stage linear discriminant analysis via QR-decomposition," IEEE Transactions on Pattern Analysis & Machine Intelligence, vol.27, no.6, pp.929-941, 2005. https://doi.org/10.1109/TPAMI.2005.110
  30. ORL Database of Faces.
  31. Yale Face Database.
  32. AR Face Database.
  33. FERET Database.
  34. CMU-PIE.
  35. MNIST Database.