Browse > Article

On-line Nonlinear Principal Component Analysis for Nonlinear Feature Extraction  

김병주 (영산대학교 네트워크정보공학부)
심주용 (대구카톨릭대학교 정보통계학)
황창하 (대구카톨릭대학교 정보통계학)
김일곤 (경북대학교 컴퓨터과학과)
Publication Information
Journal of KIISE:Software and Applications / v.31, no.3, 2004 , pp. 361-368 More about this Journal
Abstract
The purpose of this study is to propose a new on-line nonlinear PCA(OL-NPCA) method for a nonlinear feature extraction from the incremental data. Kernel PCA(KPCA) is widely used for nonlinear feature extraction, however, it has been pointed out that KPCA has the following problems. First, applying KPCA to N patterns requires storing and finding the eigenvectors of a N${\times}$N kernel matrix, which is infeasible for a large number of data N. Second problem is that in order to update the eigenvectors with an another data, the whole eigenspace should be recomputed. OL-NPCA overcomes these problems by incremental eigenspace update method with a feature mapping function. According to the experimental results, which comes from applying OL-NPCA to a toy and a large data problem, OL-NPCA shows following advantages. First, OL-NPCA is more efficient in memory requirement than KPCA. Second advantage is that OL-NPCA is comparable in performance to KPCA. Furthermore, performance of OL-NPCA can be easily improved by re-learning the data.
Keywords
On-line nonlinear Principal Component Analysis; Kernel Principal Component Analysis; Nonlinear feature extractor; Eigenvalue; Eigenvector;
Citations & Related Records
연도 인용수 순위
  • Reference
1 I.T. Jolliffe, 'Principal Component Analysis,' New York Springer-Verlag, 1986
2 M.E. Tipping and C.M. Bishop., 'Mixtures of probabilistic principal component analyser's,' Neural Computation 11(2), pp.443-482, 1998   DOI   ScienceOn
3 H. Murakami, B.V.K.V Kumar., 'Efficient calculation of primary images from a set of images,' IEEE PAMI, 4(5), pp.511-515, 1982   DOI
4 J. Winkeler, B.S. Manjunath and S. Chandrasekaran., 'Subset selection for active object recognition,' In CVPR, volume 2, pp.511-516, IEEE Computer Society Press, June 1999   DOI
5 P. Hall, D. Marshall, and R. Martin., 'Incremental eigenalysis for classification,' In British Machine Vision Conference, volume 1, pp. 286-295, September 1998
6 M.A. Krarner., 'Nonlinear principal component analysis using autoassociative neural networks,' AICHE Journal 37(2), pp.233-243, 1991   DOI
7 K.I. Diamantaras and S.Y. Kung., Principal Component Neural Networks: Theory and Applications, New York John Wiley & Sons, Inc. 1996
8 B. Scholkopf, A. Smola and K.R. Muller., 'Nonlinear component analysis as a kernel eigenvalue problem,' Neural Computation 10(5), pp.1299-1319, 1998   DOI   ScienceOn
9 R. Rosipal and M. Girolami., 'An Expectation Maximization approach to nonlinear component analysis,' Neural Computation, 13(3) pp.505-510, 2001   DOI   ScienceOn
10 P. Moerland, 'An on-line EM algorithm applied to kernel PCA,' IDlAP Research Report. 2000
11 A.J. Smola, O.L. Mangasarian, and B. Scholkopf., 'Sparse kernel feature analysis,' Technical Report 99-03, University of Wisconsin, Data Mining Institute, Madison, 1999
12 P. Moerland, 'Mixture models for unsupervised and supervised learning,' PhD. thesis, IDIAP research report IDIAP-PR 00-18, 2000
13 Cover, 'Geometrical and statistical properties of system of linear inequalities with applications in pattern recognition,' IEEE Transactions on Electronic Computers, Vol. EC-14, 326-334. 1965
14 B. Scholkopf, S. Mika, C. Burges, P. Knirsch, K.R. Miller, G. Ratsch and A.J. Smola., 'Input Space versus Feature Space in Kernel-based Methods:' IEEE Transactions on Neural Networks, vol. 10, pp.1000-1017, September 1999   DOI   ScienceOn
15 J. Mercer., 'Functions of positive and negative type and their connecrion with the theory of integral equations,' Philos. Trans. Roy. Soc. London, Vol. 209, pp.415-446, 1909   DOI
16 V. N. Vapnik., Statistical learning theory. John Wiley & Sons, New York, 1998