Browse > Article
http://dx.doi.org/10.9708/jksci.2021.26.07.001

Sparse and low-rank feature selection for multi-label learning  

Lim, Hyunki (Div. of AI Computer Science and Engineering, Kyonggi University)
Publication Information
Journal of the Korea Society of Computer and Information / v.26, no.7, 2021 , pp. 1-7 More about this Journal
Abstract
In this paper, we propose a feature selection technique for multi-label classification. Many existing feature selection techniques have selected features by calculating the relation between features and labels such as a mutual information scale. However, since the mutual information measure requires a joint probability, it is difficult to calculate the joint probability from an actual premise feature set. Therefore, it has the disadvantage that only a few features can be calculated and only local optimization is possible. Away from this regional optimization problem, we propose a feature selection technique that constructs a low-rank space in the entire given feature space and selects features with sparsity. To this end, we designed a regression-based objective function using Nuclear norm, and proposed an algorithm of gradient descent method to solve the optimization problem of this objective function. Based on the results of multi-label classification experiments on four data and three multi-label classification performance, the proposed methodology showed better performance than the existing feature selection technique. In addition, it was showed by experimental results that the performance change is insensitive even to the parameter value change of the proposed objective function.
Keywords
Multi-label learning; feature selection; low-rank approximation; sparsity; optimization method;
Citations & Related Records
연도 인용수 순위
  • Reference
1 S. Jungjit, M. Michaelis, A. A. Freitas, J. Cinatl, "Two extensions to multi-lable correlation-based feature selection: A case study in bioinformatics," IEEE Internaional Conference on Systems, Man, and Cybernetics, pp. 1519-1524, 2013.
2 H. Lim, J. Lee, and D.-W. Kim, "Optimization approach for fetaure selection in multi-label classificaiton," Pattern Recognition Letters, vol. 89, pp. 25-30, April 2017. DOI: 10.1016/j.patrec.2017.02.004   DOI
3 Z. Zhang, Y. Xu, J. Yang, X. Li, and D. Zhang, "A survey of sparse representation: algorithms and applications," IEEE access, Vol. 3, pp. 490-530, May 2015. 10.1109/ACCESS.2015.2430359   DOI
4 H. Zhang, J. Qian, B. Zhang, J. Yang, C. Gong, and Y. Wei, "Low-Rank matrix recovery via modified Schatten-p norm minimization with convergence guarantees," IEEE Transactions on Image Processing, Vol. 29, pp. 3132-3142, December 2019. DOI: 10.1109/TIP.2019.2957925   DOI
5 D. P. Bertsekas, "Constrained optimization and Lagrange multiplier methods," Academic press, 2014.
6 M.-L. Zhang and Z.-H. Zhou, "A review on multi-label learning algorithms,", IEEE Transations on Knowledge and Data Engineering, vol. 26, pp. 1819-1837, August 2014. DOI: 10.1109/TKDE.2013.39   DOI
7 M.-L. Zhang and Z.-H. Zhou, "ML-KNN: A lazy learning approach to multi-label learning," Pattern Recognition, vol. 40, pp. 2038-2048, July 2007. DOI: 10.1016/j.patcog.2006.12.019   DOI
8 O. Reyes, C. Morell, and S. Ventura, "Scalable extensions of the relieff algorithm for weighting and selecting features on the multi-label learning context," Neurocompuing, vol. 161, pp. 168-182, November 2015. DOI: 10.1016/j.neucom.2015.06.010   DOI
9 X. Zhu, S. Zhang, Y. Li, J. Zhang, L. Yang, and Y. Fang, "Low-rank sparse subspace for spectral clustering," IEEE Transactions on Knowledge and Data Engineering, Vol. 31, pp. 1532-1543, July 2018. DOI: 10.1109/TKDE.2018.2858782   DOI
10 R. Shang, Y. Meng, W. Wang, F. Shang, and L. Jiao, "Local discriminative based sparse subspace learning for feature selection," Pattern Recognition, Vol. 92, pp. 219-230, August 2019. DOI: 10.1016/j.patcog.2019.03.026   DOI
11 Y. Zhang, Q. Wang, D.-W. Gong, and X.-F. Song, "Nonnegative laplacian embedding guided subspace learning for unsupervised feature selection," Pattern Recognition, Vol. 93, pp. 337-352, September 2019. DOI: 10.1016/j.patcog.2019.04.020   DOI
12 Y. Lin, Q. Hu, J. Liu, and J. Duan, "Multi-label feature selection based on max-dependency and min-redundancy," Neurocomputing, vol. 168 pp. 92-103, November 2015. DOI: 10.1016/j.neucom.2015.06.010   DOI
13 S. Sharmin, M. Shoyaib, A. A. Ali, M. A. H. Khan, and O. Chae, "Simultaneous feature selection and discretization based on mutual information," Pattern Recognition, vol. 91, pp. 162-174, July 2019. DOI: 10.1016/j.patcog.2019.02.016   DOI
14 R. B. Pereira, A. Plastino, B. Zadrozny, and L. H. C. Merschmann, "Categorizing feature selection methods for multi-label classification," Artificial Intelligence Review, Vol. 49, pp. 57-78, 2018. DOI: 10.1007/s10462-016-9516-4   DOI
15 J. Lee, H. Lim, and D-.W. Kim, "Approximating mutual information for multi-label feature selection," Electronics Letters, vol. 54, pp. 76-77, July 2012. DOI: 10.1049/el.2012.1600   DOI
16 X. Zhen, M. Yu, X. He, and S. Li, "Multi-target regression via robust low-rank learning," IEEE Transactions on Pattern Analyasis and Machine Intelligence, Vol. 40, pp. 497-504, March 2017. DOI: 10.1109/TPAMI.2017.2688363   DOI
17 M. R. Boutell, J. Luo, X. Shen, and C. M. Xipeng, "Learning multi-label scene classification," Pattern Recognition, vol. 37, pp. 1757-1771, September 2004. DOI: 10.1016/j.patcog.2004.03.009   DOI
18 M.-L. Zhang, J. M. Pena, and V. Robles, "Feature selection for multi-label naive Bayes classification," Information Sciences, vol. 179, pp. 3218-3229, September 2009. DOI: 10.1016/j.ins.2009.06.010   DOI
19 K. Trohidis, G. Tsoumakas, G. Kalliris, and I. P. Vlahavas, "Multi-label classification of music into emotions," ISMIR, pp. 325-330, 2008.
20 F. Briggs, H. Yonghong, R. Raich, and others, "New methods for acoustic classification of multiple simutaneous bird species in a noisy environment," IEEE International Workshop on Machine Learning for Signal Processing, pp. 1-8, 2013.
21 J. Lee and D.-W. Kim, "Scls: Multi-label feature selection based on scalable criterion for large label set," Pattern Recognition, vol. 66, pp. 342-352, June 2017. DOI: 10.1016/j.patcog.2017.01.014   DOI
22 A. Elisseeff, J. Weston, and others, "A kernel method for multi-labelled classification," NIPS, pp. 681-687, 2001.
23 A. Cano, J. M. Luna, E. L. Gibaja, and S. Ventura, "LAIM discretization for multi-label data," Information Sciences, Vol. 330, pp. 370-384, 2016. DOI: 10.1016/j.ins.2015.10.032   DOI