• International Journal of Fuzzy Logic and Intelligent Systems
• /
• v.10 no.2
• /
• pp.95-100
• /
• 2010

This paper proposes a new spatial regularization of Fisher linear discriminant analysis (LDA) to reduce the overfitting due to small size sample (SSS) problem in face recognition. Many regularized LDAs have been proposed to alleviate the overfitting by regularizing an estimate of the within-class scatter matrix. Spatial regularization methods have been suggested that make the discriminant vectors spatially smooth, leading to mitigation of the overfitting. As a generalized version of the spatially regularized LDA, the proposed regularized LDA utilizes the non-uniformity of spatial correlation structures in face images in adding a spatial smoothness constraint into an LDA framework. The region-dependent spatial regularization is advantageous for capturing the non-flat spatial correlation structure within face image as well as obtaining a spatially smooth projection of LDA. Experimental results on public face databases such as ORL and CMU PIE show that the proposed regularized LDA performs well especially when the number of training images per individual is quite small, compared with other regularized LDAs.

• Management Science and Financial Engineering
• /
• v.12 no.2
• /
• pp.71-85
• /
• 2006

Collaborative filtering, among other recommender systems, has been known as the most successful recommendation technique. However, it requires the user-item rating data, which may not be easily available. As an alternative, some collaborative filtering algorithms have been developed recently by utilizing the market basket data in the form of the binary user-item matrix. Viewing the recommendation scheme as a two-class classification problem, we proposed a new collaborative filtering scheme using a regularized discriminant analysis applied to the binary user-item data. The proposed discriminant model was built in terms of the major principal components and was used for predicting the probability of purchasing a particular item by an active user. The proposed scheme was illustrated with two modified real data sets and its performance was compared with the existing user-based approach in terms of the recommendation precision.

• Communications for Statistical Applications and Methods
• /
• v.6 no.3
• /
• pp.831-842
• /
• 1999

We propose a wavelet decomposition feature extraction method for the hand-written character recognition. Comparing the recognition rates of which methods with original image features and with selected features by the wavelet decomposition we study the characteristics of the proposed method. LDA(Linear Discriminant Analysis) QDA(Quadratic Discriminant Analysis) RDA(Regularized Discriminant Analysis) and NN(Neural network) are used for the calculation of recognition rates. 6000 hand-written numerals from CENPARMI at Concordia University are used for the experiment. We found that the set of significantly selected wavelet decomposed features generates higher recognition rate than the original image features.

• Proceedings of the Korean Society of Near Infrared Spectroscopy Conference
• /
• 2001.06a
• /
• pp.1244-1244
• /
• 2001

In linear discriminant analysis there are two important properties concerning the effectiveness of discriminant function modeling. The first is the separability of the discriminant function for different classes. The separability reaches its optimum by maximizing the ratio of between-class to within-class variance. The second is the stability of the discriminant function against noises present in the measurement variables. One can optimize the stability by exploring the discriminant variates in a principal variation subspace, i. e., the directions that account for a majority of the total variation of the data. An unstable discriminant function will exhibit inflated variance in the prediction of future unclassified objects, exposed to a significantly increased risk of erroneous prediction. Therefore, an ideal discriminant function should not only separate different classes with a minimum misclassification rate for the training set, but also possess a good stability such that the prediction variance for unclassified objects can be as small as possible. In other words, an optimal classifier should find a balance between the separability and the stability. This is of special significance for multivariate spectroscopy-based classification where multicollinearity always leads to discriminant directions located in low-spread subspaces. A new regularized discriminant analysis technique, the principal discriminant variate (PDV) method, has been developed for handling effectively multicollinear data commonly encountered in multivariate spectroscopy-based classification. The motivation behind this method is to seek a sequence of discriminant directions that not only optimize the separability between different classes, but also account for a maximized variation present in the data. Three different formulations for the PDV methods are suggested, and an effective computing procedure is proposed for a PDV method. Near-infrared (NIR) spectra of blood plasma samples from mastitic and healthy cows have been used to evaluate the behavior of the PDV method in comparison with principal component analysis (PCA), discriminant partial least squares (DPLS), soft independent modeling of class analogies (SIMCA) and Fisher linear discriminant analysis (FLDA). Results obtained demonstrate that the PDV method exhibits improved stability in prediction without significant loss of separability. The NIR spectra of blood plasma samples from mastitic and healthy cows are clearly discriminated between by the PDV method. Moreover, the proposed method provides superior performance to PCA, DPLS, SIMCA and FLDA, indicating that PDV is a promising tool in discriminant analysis of spectra-characterized samples with only small compositional difference, thereby providing a useful means for spectroscopy-based clinic applications.

• Proceedings of the Korean Society of Near Infrared Spectroscopy Conference
• /
• 2001.06a
• /
• pp.1042-1042
• /
• 2001

In linear discriminant analysis there are two important properties concerning the effectiveness of discriminant function modeling. The first is the separability of the discriminant function for different classes. The separability reaches its optimum by maximizing the ratio of between-class to within-class variance. The second is the stability of the discriminant function against noises present in the measurement variables. One can optimize the stability by exploring the discriminant variates in a principal variation subspace, i. e., the directions that account for a majority of the total variation of the data. An unstable discriminant function will exhibit inflated variance in the prediction of future unclassified objects, exposed to a significantly increased risk of erroneous prediction. Therefore, an ideal discriminant function should not only separate different classes with a minimum misclassification rate for the training set, but also possess a good stability such that the prediction variance for unclassified objects can be as small as possible. In other words, an optimal classifier should find a balance between the separability and the stability. This is of special significance for multivariate spectroscopy-based classification where multicollinearity always leads to discriminant directions located in low-spread subspaces. A new regularized discriminant analysis technique, the principal discriminant variate (PDV) method, has been developed for handling effectively multicollinear data commonly encountered in multivariate spectroscopy-based classification. The motivation behind this method is to seek a sequence of discriminant directions that not only optimize the separability between different classes, but also account for a maximized variation present in the data. Three different formulations for the PDV methods are suggested, and an effective computing procedure is proposed for a PDV method. Near-infrared (NIR) spectra of blood plasma samples from daily monitoring of two Japanese cows have been used to evaluate the behavior of the PDV method in comparison with principal component analysis (PCA), discriminant partial least squares (DPLS), soft independent modeling of class analogies (SIMCA) and Fisher linear discriminant analysis (FLDA). Results obtained demonstrate that the PDV method exhibits improved stability in prediction without significant loss of separability. The NIR spectra of blood plasma samples from two cows are clearly discriminated between by the PDV method. Moreover, the proposed method provides superior performance to PCA, DPLS, SIMCA md FLDA, indicating that PDV is a promising tool in discriminant analysis of spectra-characterized samples with only small compositional difference.

• Journal of the Korean Data and Information Science Society
• /
• v.21 no.4
• /
• pp.689-697
• /
• 2010

In this study, we propose a new procedure for improving the character recognition of text area extracted from video images. The recognition of strings extracted from video, which are mixed with Hangul, English, numbers and special characters, etc., is more difficult than general character recognition because of various fonts and size, graphic forms of letters tilted image, disconnection, miscellaneous videos, tangency, characters of low definition, etc. We improved the recognition rate by taking commonly used letters and leaving out the barely used ones instead of recognizing all of the letters, and then using SVM and RDA character recognition methods. Our numerical results indicate that combining SVM and RDA performs better than other methods.

• Communications for Statistical Applications and Methods
• /
• v.22 no.6
• /
• pp.575-587
• /
• 2015

The development in data collection techniques results in high dimensional data sets, where discrimination is an important and commonly encountered problem that are crucial to resolve when high dimensional data is heterogeneous (non-common variance covariance structure for classes). An example of this is to classify microbial habitat preferences based on codon/bi-codon usage. Habitat preference is important to study for evolutionary genetic relationships and may help industry produce specific enzymes. Most classification procedures assume homogeneity (common variance covariance structure for all classes), which is not guaranteed in most high dimensional data sets. We have introduced regularized elimination in partial least square coupled with QDA (rePLS-QDA) for the parsimonious variable selection and classification of high dimensional heterogeneous data sets based on recently introduced regularized elimination for variable selection in partial least square (rePLS) and heterogeneous classification procedure quadratic discriminant analysis (QDA). A comparison of proposed and existing methods is conducted over the simulated data set; in addition, the proposed procedure is implemented to classify microbial habitat preferences by their codon/bi-codon usage. Five bacterial habitats (Aquatic, Host Associated, Multiple, Specialized and Terrestrial) are modeled. The classification accuracy of each habitat is satisfactory and ranges from 89.1% to 100% on test data. Interesting codon/bi-codons usage, their mutual interactions influential for respective habitat preference are identified. The proposed method also produced results that concurred with known biological characteristics that will help researchers better understand divergence of species.