Browse > Article
http://dx.doi.org/10.5391/JKIIS.2007.17.2.220

Input Variables Selection by Principal Component Analysis and Mutual Information Estimation  

Cho, Yong-Hyun (대구가톨릭대학교 컴퓨터정보통신공학부)
Hong, Seong-Jun (대구가톨릭대학교 컴퓨터정보통신공학부)
Publication Information
Journal of the Korean Institute of Intelligent Systems / v.17, no.2, 2007 , pp. 220-225 More about this Journal
Abstract
This paper presents an efficient input variable selection method using both principal component analysis(PCA) and adaptive partition mutual information(AP-MI) estimation. PCA which is based on 2nd order statistics, is applied to prevent a overestimation by quickly removing the dependence between input variables. AP-MI estimation is also applied to estimate an accurate dependence information by equally partitioning the samples of input variable for calculating the probability density function. The proposed method has been applied to 2 problems for selecting the input variables, which are the 7 artificial signals of 500 samples and the 24 environmental pollution signals of 55 samples, respectively. The experimental results show that the proposed methods has a fast and accurate selection performance. The proposed method has also respectively better performance than AP-MI estimation without the PCA and regular partition MI estimation.
Keywords
Principal Component Analysis; Mutual Information Estimation; Input Variable Selection Adaptive Partition; Regular Partition;
Citations & Related Records
연도 인용수 순위
  • Reference
1 A. Cichock and R. Unbehaun, Neural Networks for Optimization and Signal Processing. John Wiley & Sons., New York, 1993
2 T. Trappenberg, J Ouyang, and A. Back, 'Input Variable Selection : Mutual Information and Linear Mixing Measures', IEEE Transactions on Knowledge and Data Engineering, Vol. 1, No.8, pp. 37-46, Jan. 2006
3 A. Back and T. Trappenberg, 'Selecting Inputs for Modelling Using Normalized Higher Order Statistics and Independent Component Analysis,' IEEE Transactions on Neural Networks, Vol. 12, No.3, pp. 612-617, March. 2001   DOI   ScienceOn
4 K. I. Diamantaras and S. Y. Kung, Principal Component Neural Networks : Theory and Applications, Adaptive and learning Systems for Signal Processing, Communications, and Control, John Wiley & Sons, Inc., 1996
5 S. Haykin, Neural Networks : A Comprehensive Foundation, Prentice-Hall, 2ed, London, 1999
6 A. Back and A. Cichocki, 'Input Variable Selection Using Independent Component Analysis and Higher Order Statistics', Proc. of ICA99, Jan. 1999
7 A. Back and T. Trappenberg. 'Input Variable Selection Using Independent Component Analysis,' International joint Conference on Neural Networks, pp. 1-5, Washington, 1999
8 P. Foldiak, 'Adaptive Network for Optimal Linear Feature Extraction,' International Joint Conference on Neural Networks, Washington D.C., Vol. 1, pp. 401-406, June 1989