Browse > Article
http://dx.doi.org/10.7465/jkdi.2014.25.5.1079

Efficient variable selection method using conditional mutual information  

Ahn, Chi Kyung (Department of Statistics, Sunkyunkwan University)
Kim, Donguk (Department of Statistics, Sunkyunkwan University)
Publication Information
Journal of the Korean Data and Information Science Society / v.25, no.5, 2014 , pp. 1079-1094 More about this Journal
Abstract
In this paper, we study efficient gene selection methods by using conditional mutual information. We suggest gene selection methods using conditional mutual information based on semiparametric methods utilizing multivariate normal distribution and Edgeworth approximation. We compare our suggested methods with other methods such as mutual information filter, SVM-RFE, Cai et al. (2009)'s gene selection (MIGS-original) in SVM classification. By these experiments, we show that gene selection methods using conditional mutual information based on semiparametric methods have better performance than mutual information filter. Furthermore, we show that they take far less computing time than Cai et al. (2009)'s gene selection but have similar performance.
Keywords
Classification; conditional mutual information; Edgeworth approximation; entropy; forward selection; high-dimensional data; support vector machines; variable selection;
Citations & Related Records
연도 인용수 순위
  • Reference
1 Withers, C. S. (2000). A simple expression for the multivariate Hermite polynomials. Statistics & Probability Letters, 47, 165-169.   DOI   ScienceOn
2 Efron, B. and Tibshirani, R. (1997). Improvements on cross-validation; The bootstrap method. Journal of the American Statistical Association, 92, 548-560.
3 Fleuret, F. (2004). Fast binary feature selection with conditional mutual information. Journal of Machine Learning Research, 5, 1531-1555.
4 Golub, T. R., Slonim, T. K., Tamayo, P., Huard, C., Gaasenbeek, M., Mesirov, J. P., Coller, H., Loh, M. H., Downing, J. R., Caligiuri, M. A., Bloomeld, C. D. and Lander, E. S. (1999). Molecular classification of cancer: Class discovery and class prediction by gene expression monitoring. Science, 286, 531-537.   DOI   ScienceOn
5 Grad, H. (1949). Note on n-dimensional Hermite polynomials. Communications on Pure and Applied Mathematics, 3, 325-330.
6 Guyon, I. and Elisseeff, A. (2003). An introduction to variable and feature selection. Journal of Machine Learning Research, 3, 1157-1182.
7 Guyon, I., Weston, J., Barnhill, S. and Vapnik, V. (2002). Gene selection for cancer classification using support vector machines. Machine Learning, 46, 389-422.   DOI
8 Hulle, M. M. A. (2005). Edgeworth approximation of multivariate differential entropy. Neural Computation, 17, 1903-1910.   DOI   ScienceOn
9 Kwak, N. and Choi, C. H. (2002). Input feature selection by mutual information based on parzen window. Pattern Analysis and Machine Intelligence, 24, 1667-1671.   DOI   ScienceOn
10 Maz'ya, V. and Schmidt, G. (1996). On approximate approximations using Gaussian kernels. IMA Journal of Numerical Analysis, 16, 13-29.   DOI   ScienceOn
11 Silverman, B. W. (1992). Density estimation for statistics and data analysis, Chapman & Hall, London.
12 Tusher, V. G., Tibshirani, R. and Chu, G. (2001). Significance analysis of microarrays applied to ionizing radiation response. PNAS, 98, 5116-5121.   DOI   ScienceOn
13 McCullagh, P. (1987). Tensor methods in statistics, Chapman & Hall, London.
14 Cai, R., Hao, Z., Yang, X. and Wen, W. (2009). An efficient gene selection algorithm based on mutual information. Neurocomputing, 72, 991-999.   DOI   ScienceOn
15 Chang, C. F., Wai, K. M. and Patterton, H. G. (2004). Calculating the statistical significance of physical clusters of co-regulated genes in the genome: The role of chromatin in domain-wide gene regulation. Nucleic Acids, 32, 1798-1807.   DOI   ScienceOn
16 Cover, T. M. and Thomas, J. A. (2006). Elements of information theory, 2nd Ed., Wiley, New York.