Browse > Article
http://dx.doi.org/10.3837/tiis.2018.03.015

An Improved method of Two Stage Linear Discriminant Analysis  

Chen, Yarui (Computer Science and Information Engineering Institute, Tianjin University of Science and Technology)
Tao, Xin (Computer Science and Information Engineering Institute, Tianjin University of Science and Technology)
Xiong, Congcong (Computer Science and Information Engineering Institute, Tianjin University of Science and Technology)
Yang, Jucheng (Computer Science and Information Engineering Institute, Tianjin University of Science and Technology)
Publication Information
KSII Transactions on Internet and Information Systems (TIIS) / v.12, no.3, 2018 , pp. 1243-1263 More about this Journal
Abstract
The two-stage linear discrimination analysis (TSLDA) is a feature extraction technique to solve the small size sample problem in the field of image recognition. The TSLDA has retained all subspace information of the between-class scatter and within-class scatter. However, the feature information in the four subspaces may not be entirely beneficial for classification, and the regularization procedure for eliminating singular metrics in TSLDA has higher time complexity. In order to address these drawbacks, this paper proposes an improved two-stage linear discriminant analysis (Improved TSLDA). The Improved TSLDA proposes a selection and compression method to extract superior feature information from the four subspaces to constitute optimal projection space, where it defines a single Fisher criterion to measure the importance of single feature vector. Meanwhile, Improved TSLDA also applies an approximation matrix method to eliminate the singular matrices and reduce its time complexity. This paper presents comparative experiments on five face databases and one handwritten digit database to validate the effectiveness of the Improved TSLDA.
Keywords
Two-stage linear discrimination analysis; Small size sample problem; Selection and compression method; Approximation matrix method; Regularization;
Citations & Related Records
연도 인용수 순위
  • Reference
1 ZC Li, J Liu, Y Yang, XF Zhou and HQ Lu, "Clustering-Guided Sparse Structural Learning for Unsupervised Feature Selection," IEEE Transactions On Knowledge And Data Engineering, vol.26, no.9, pp. 2138-2150, 2014.   DOI
2 J Gui, ZN Sun, J Cheng and SW Ji, "How to Estimate the Regularization Parameter for Spectral Regression Discriminant Analysis and its Kernel Version?" IEEE Transactions On Circuits And Systems For Video Technology, vol.24, no.2, pp.211-223, 2014.   DOI
3 Andrew R. Webb, Statistical Pattern Recognition, 3nd Edition, Wiley, America, 2011.
4 JP Ye and Q Li, "A two-stage linear discriminant analysis via QR-decomposition," IEEE Transactions on Pattern Analysis & Machine Intelligence, vol.27, no.6, pp.929-941, 2005.   DOI
5 ORL Database of Faces.
6 Yale Face Database.
7 AR Face Database.
8 FERET Database.
9 CMU-PIE.
10 MNIST Database.
11 P.N. Belhumeur, J.P. Hespanha and D.J. Kriegman, "Eigenfaces vs. Fisherfaces: Recognition using class specific linear projection," IEEE Transactions on Pattern Analysis & Machine Intelligence, vol.19, no.7, pp. 711-720, 1997.   DOI
12 Christopher Bishop, "Pattern Recognition and Machine Learning," Springer, Germany, 2008.
13 M Imani and H Ghassemian, "Two Dimensional Linear Discriminant Analyses for Hyperspectral Data," Photogrammetric Engineering & Remote Sensing, vol.81, no.10, pp.777-786, 2015.   DOI
14 A Sharma and KK Paliwal, "Linear discriminant analysis for the small sample size problem: an overview," International Journal of Machine Learning & Cybernetics, vol.6, no.3, pp. 443-454, 2015.   DOI
15 WY Yang and HY Wu, "Regularized complete linear discriminant analysis," Neurocomputing, vol. 137, no.11, pp. 185-191, 2014.   DOI
16 JW Lu, KN Plataniotis and AN Venetsanopoulos, "Regularization studies of linear discriminant analysis in small sample size scenarios with application to face recognition," Pattern Recognition Letter, vol.26, no.2, pp. 181-191, 2005.   DOI
17 A Sharma and KK Paliwal, "A deterministic approach to regularized linear discriminant analysis," Neurocomputing, vol.151, no.11, pp. 207-214, 2015.   DOI
18 S Wang, "Linear Discriminant Analysis Based on Clustering Regularization," Master's thesis of Tianjin University, 2013.
19 A Sharma, KK Paliwal, Seiya Imoto and Satoru Miyano, "A feature selection method using improved regularized linear discriminant analysis," Machine Vision and Applications, vol.25, no.25, pp. 775-786, 2014.   DOI
20 Y Liu and SZ liao, "Granularity selection for cross-validation of SVM," Information Sciences, vol. 378, pp. 475-483,2017   DOI
21 LF Chen, HYM Liao, MT Ko, JC Lin and GJ Yu, "A new LDA-based face recognition system which can solve the small sample size problem," Pattern Recognition, vol.33, no.10, pp. 1713-1726, 2000.   DOI
22 H Yu and H Yang, "A direct LDA algorithm for high-dimensional data-with application to face recognition," Pattern Recognition, vol.34, no.10, pp. 2067-2070, 2001.   DOI
23 A Sharma and Kuldip K Paliwal, "A two-stage linear discriminant analysis for face-recognition," Pattern Recognition Letters, vol.33, no.9, pp. 1157-1162, 2012.   DOI
24 JH Zhao, L Shi and J Zhu, "Two-Stage Regularized Linear Discriminant Analysis for 2-D Data," IEEE Transactions on Neural Networks And Learning Systems, vol.26, no.8, pp.1669-1681, 2015.   DOI
25 KK Paliwal and A Sharma, "A Improved direct LDA and its application to DNA microarray gene expression data," Pattern Recognition Letters, vol.31, no.16, pp. 2489-2492, 2010.   DOI
26 XS Zhuang, DQ Dai, "Inverse Fisher discriminate criteria for small sample size problem and its application to face recognition," Pattern Recognition, vol.38, no.11, pp.2192-2194, 2005.   DOI
27 A Sharma and KK Paliwal, "Approximate LDA Technique for Dimensionality Reduction in the Small Sample Size Case," Journal of Pattern Recognition Research, pp.298-306, 2011.
28 Pi-Fuei Hsieh, "Impact and realization of increased class separability on the small sample size problem in hyperspectral classification," Canadian Journal of Remote Sensing, vol.25, no.3, pp: 248-261, 2009.
29 Fadi Dornaika and ALireza Bosagzadeh, "On Solving the Small Sample Size Problem for Marginal Fisher Analysis," Image Analysis and Recognition, pp.116-123, 2013.
30 MW Jian and KM Lam, "Simultaneous Hallucination and Recognition of Low-Resolution Faces Based on Singular Value Decomposition," IEEE Transactions on Circuits and Systems for Video Technology, vol.25, no.11 pp. 1761-1772, 2015.   DOI
31 YD Wen, KP Zhang, ZF Li and Y Qiao, "A Discriminative Feature Learning Approach for Deep Face Recognition," in Proc. of the 14th European Conference of Computer Vision (ECCV 2016), Amsterdam, Netherlands, 499-515, October 2016.
32 MWJian,KM Lam and JY Dong. "A Novel Face-Hallucination Scheme Based on Singular Value Decomposition," Pattern Recognition, vol. 46, no. 11, pp. 3091-3102, November 2013.   DOI
33 MW Jian, KM Lam and JY Dong. "Facial-Feature Detection and Localization Based on a Hierarchical Scheme," Information Sciences, vol. 262, pp. 1-14, 2014.   DOI
34 MW Jian and KM Lam, "Face-Image Retrieval Based on Singular Values and Potential-Field Representation," Signal Processing, vol. 100, pp. 9-15, 2014.   DOI
35 ZC Li, J Liu, JH Tang and HQ Lu, "Robust Structured Subspace Learning for Data Representation," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol.37, no.10, pp.2085-2098.   DOI