Browse > Article
http://dx.doi.org/10.5391/IJFIS.2013.13.4.254

An Overview of Unsupervised and Semi-Supervised Fuzzy Kernel Clustering  

Frigui, Hichem (Multimedia Research Lab, University of Louisville)
Bchir, Ouiem (Computer Science Department, College of Computer and Information Systems (CCIS), King Saud University)
Baili, Naouel (Quintiles)
Publication Information
International Journal of Fuzzy Logic and Intelligent Systems / v.13, no.4, 2013 , pp. 254-268 More about this Journal
Abstract
For real-world clustering tasks, the input data is typically not easily separable due to the highly complex data structure or when clusters vary in size, density and shape. Kernel-based clustering has proven to be an effective approach to partition such data. In this paper, we provide an overview of several fuzzy kernel clustering algorithms. We focus on methods that optimize an fuzzy C-mean-type objective function. We highlight the advantages and disadvantages of each method. In addition to the completely unsupervised algorithms, we also provide an overview of some semi-supervised fuzzy kernel clustering algorithms. These algorithms use partial supervision information to guide the optimization process and avoid local minima. We also provide an overview of the different approaches that have been used to extend kernel clustering to handle very large data sets.
Keywords
Fuzzy clustering; Kernel-based clustering; Relational Kernel clustering; Multiple Kernel clustering; Semi-supervised clustering;
Citations & Related Records
연도 인용수 순위
  • Reference
1 M. Girolami, "Mercer kernel-based clustering in feature space," IEEE Transactions on Neural Networks, vol. 13, no. 3, pp. 780-784, May 2002. http://dx.doi.org/10.1109/TNN.2002.1000150   DOI   ScienceOn
2 R. Inokuchi and S. Miyamoto, "LVQ clustering and SOM using a kernel function," in Proceedings of the IEEE International Conference on Fuzzy Systems, Budapest, Hungary, July 25-29, 2004, pp. 1497-1500. http://dx.doi.org/10.1109/FUZZY.2004.1375395   DOI
3 A. K. Qin and P. N. Suganthan, "Kernel neural gas algorithms with application to cluster analysis," in Proceedings of the 17th International Conference on Pattern Recognition, Cambridge, United Kingdom, August 23-26, 2004, pp. 617-620. http://dx.doi.org/10.1109/ICPR.2004.1333848   DOI
4 U. Luxburg, "A tutorial on spectral clustering," Statistics and Computing, vol. 17, no. 4, pp. 395-416, Dec. 2007. http://dx.doi.org/10.1007/s11222-007-9033-z   DOI   ScienceOn
5 X. Peng, Y. Chai, L. Xu, and X. Man, "Research on fault diagnosis of marine diesel engine based on grey relational analysis and kernel fuzzy C-means clustering," in The Fifth International Conference on Intelligent Computation Technology and Automation, Zhangjiajie, China, January 12-14, 2012, pp. 283-286. http://dx.doi.org/10.1109/ICICTA.2012.78   DOI
6 J. Fan and Y. Wu, "Watermarking algorithm based on kernel fuzzy clustering and singular value decomposition in the complex wavelet transform domain," in International Conference on Information Technology, Computer Engineering and Management Sciences, Nanjing, China, September 24-25, 2011, pp. 42-46. http://dx.doi.org/10.1109/ICM.2011.121   DOI
7 L. Y. Qi and K. G. Wang, "Kernel fuzzy clustering based classification of Ancient-Ceramic fragments," in The 2nd IEEE International Conference on Information Management and Engineering, Chengdu, China, April 16-18, 2010, pp. 348-350. http://dx.doi.org/10.1109/ICIME.2010.5477818   DOI
8 D. M. Tsai and C. C. Lin, "Fuzzy C-means based clustering for linearly and nonlinearly separable data," Pattern Recognition, vol. 44, no. 8, pp. 1750-1760, Aug. 2011. http://dx.doi.org/10.1016/j.patcog.2011.02.009   DOI   ScienceOn
9 D. H. Liu, J. P. Bian, and X. Y. Sun, "The study of fault diagnosis model of DGA for oil-immersed transformer based on fuzzy means Kernel clustering and SVM multi-class object simplified structure," in Proceedings of the 7th International Conference on Machine Learning and Cybernetics, Kunming, China, July 12-15, 2008, pp. 1505-1509. http://dx.doi.org/10.1109/ICMLC.2008.4620644   DOI
10 B. Qu and H. Wang, "Dynamic fuzzy kernel clustering analysis of enterprises independent: innovation capability based on artificial immunity," in International Workshop on Modelling, Simulation and Optimization, Hong Kong, December 27-28, 2009, pp. 216-220. http://dx.doi.org/10.1109/WMSO.2008.102   DOI
11 X. Li and S. Bian, "A kernel fuzzy clustering algorithm with spatial constraint based on improved expectation maximization for image segmentation," in International Conference on Measuring Technology and Mechatronics Automation, Zhangjiajie, China, April 11-12, 2009, pp. 529-533. http://dx.doi.org/10.1109/ICMTMA.2009.59   DOI
12 L. Liao and T. S. Lin, "A fast spatial constrained fuzzy kernel clustering algorithm for MRI brain image segmentation," in International Conference on Wavelet Analysis and Pattern Recognition, Beijing, China, November 2-4, 2008, pp. 82-87. http://dx.doi.org/10.1109/ICWAPR.2007.4420641   DOI
13 R. J. G. B. Campello, "A fuzzy extension of the Rand index and other related indexes for clustering and classification assessment," Pattern Recognition Letters, vol. 28, no. 7, pp. 833-841, May 2007. http://dx.doi.org/10.1016/j.patrec.2006.11.010   DOI   ScienceOn
14 N. Aronszajn, "Theory of reproducing kernels," Transactions of the American Mathematical Society, vol. 68, no. 3, pp. 337-404, May 1950.   DOI   ScienceOn
15 M. Sato-Ilic, S. Ito, and S. Takahashi, "Generalized kernel fuzzy clustering model," in IEEE International Conference on Fuzzy Systems, Jeju, Korea, August 20-24, 2009, pp. 421-426. http://dx.doi.org/10.1109/FUZZY.2009.5276876   DOI
16 J. Ye, S. Ji, and J. Chen, "Learning the kernel matrix in discriminant analysis via quadratically constrained quadratic programming," in Proceedings of the 13th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, San Jose, CA, August 12-15, 2007, pp. 854-863. http://dx.doi.org/10.1145/1281192.1281283   DOI
17 S. Shalev-Shwartz, Y. Singer, and A. Y. Ng, "Online and batch learning of pseudo-metrics," in Proceedings of the 21st International Conference on Machine Learning, Banff, Canada, July 4-8, 2004, p. 94.
18 O. Bchir, H. Frigui, and M. M. Ben Ismail, "Semi-supervised clustering and local scale learning algorithm," in World Congress on Computer and Information Technology, Sousse, Tunisia, June 22-24, 2013, article number 6618774. http://dx.doi.org/10.1109/WCCIT.2013.6618774   DOI
19 D. Q. Zhang and S. C. Chen, "Kernel-based fuzzy and possibilistic c-means clustering," in Proceedings of the International Conference on Artificial Neural Network, Istanbul, Turkey, June 26-29, 2003, pp. 122-125.
20 D. Zhang and S. Chen, "Fuzzy clustering using kernel method," in International Conference on Control and Automation, Xiamen, China, June 19, 2002, pp. 162-163. http://dx.doi.org/10.1109/ICCA.2002.1229535   DOI
21 R. J. Hathaway, J. M. Huband, and J. C. Bezdek, "A kernelized non-euclidean relational fuzzy c-means algorithm," in IEEE International Conference on Fuzzy Systems, Reno, NV, May 22-25, 2005, pp. 414-419. http://dx.doi.org/10.1109/FUZZY.2005.1452429   DOI
22 O. Bchir and H. Frigui, "Fuzzy relational kernel clustering with local scaling parameter learning," in Proceedings of the 20th IEEE International Workshop on Machine Learning for Signal Processing, Kittila, Finland, August 29-September 1, 2010, pp. 289-294. http://dx.doi.org/10.1109/MLSP.2010.5589234   DOI
23 O. Bchir and H. Frigui, "Fuzzy clustering with learnable cluster dependent kernels," in IEEE International Conference on Fuzzy Systems, Taipei, Taiwan, June 27-30, 2011, pp. 2521-2527. http://dx.doi.org/10.1109/FUZZY.2011.6007411   DOI
24 G. R. G. Lanckriet, N. Cristianini, P. Bartlett, L. El Ghaoui, and M. I. Jordan, "Learning the kernel matrix with semidefinite programming," Journal of Machine Learning Research, vol. 5, pp. 27-72, Jan. 2004.
25 F. R. Bach, G. R. G. Lanckriet, and M. I. Jordan, "Multiple kernel learning, conic duality, and the SMO algorithm," in Proceedings of the 21st International Conference on Machine Learning, Banff, Canada, July 4-8, 2004, pp. 41-48.
26 A. Rakotomamonjy, F. Bach, S. Canu, and Y. Grandvalet, "More efficiency in multiple kernel learning," in Proceedings of the 24th International Conference on Machine Learning, Corvalis, OR, June 20-24, 2007, pp. 775-782. http://dx.doi.org/10.1145/1273496.1273594   DOI
27 N. Baili and H. Frigui, "Fuzzy clustering with multiple kernels in feature space," in IEEE International Conference on Fuzzy Systems, Brisbane, Australia, June 10-15, 2012. http://dx.doi.org/10.1109/FUZZ-IEEE.2012.6251146   DOI
28 M. Gonen and E. Alpaydin, "Localized multiple kernel learning," in Proceedings of the 25th International Conference on Machine Learning, Helsinki, Finland, July 5-9, 2008, pp. 352-359.
29 B. Zhao, J. T. Kwok, and C. Zhang, "Multiple kernel clustering," in Proceedings of the 9th SIAM International Conference on Data Mining, Sparks, NV, April 30-May 2, 2009, pp. 638-649.
30 H. C. Huang, Y. Y. Chuang, and C. S. Chen, "Multiple kernel fuzzy clustering," IEEE Transactions on Fuzzy Systems, vol. 20, no. 1, pp. 120-134, Feb. 2012. http://dx.doi.org/10.1109/TFUZZ.2011.2170175   DOI   ScienceOn
31 N. Baili and H. Frigui, "Relational fuzzy clustering with multiple kernels," in Proceedings of the 11th IEEE International Conference on Data Mining, Vancouver, Canada, December 11, 2011, pp. 488-495. http://dx.doi.org/10.1109/ICDMW.2011.145   DOI
32 R. J. Hathaway, J. W. Davenport, and J. C. Bezdek, "Relational duals of the c-means clustering algorithms," Pattern Recognition, vol. 22, no. 2, pp. 205-212, 1989.   DOI   ScienceOn
33 E. P. Xing, A. Y. Ng, M. I. Jordan, and S. Russell, "Distance metric learning, with application to clustering with side-information," in The 17th Annual Conference on Neural Information Processing Systems, Vancouver & Whistler, Canada, December 8-13, 2003.
34 K. Weinberger, J. Blitzer, and L. Saul, "Distance metric learning for large margin nearest neighbor classification," in The 19th Annual Conference on Neural Information Processing Systems, Vancouver & Whistler, Canada, December 5-10, 2005.
35 H. Zhang and J. Lu, "Semi-supervised fuzzy clustering: a kernel-based approach," Knowledge-Based Systems, vol. 22, no. 6, pp. 477-481, Aug. 2009. http://dx.doi.org/10.1016/j.knosys.2009.06.009   DOI   ScienceOn
36 A. Globerson and S. Roweis, "Metric learning by collapsing classes," in The 19th Annual Conference on Neural Information Processing Systems, Vancouver & Whistler, Canada, December 5-10, 2005.
37 J. V. Davis, B. Kulis, P. Jain, S. Sra, and I. S. Dhillon, "Information-theoretic metric learning," in Proceedings of the 24th International Conference on Machine Learning, Corvallis, OR, June 20-24, 2007, pp. 209-216.
38 R. Chatpatanasiri, T. Korsrilabutr, P. Tangchanachaianan, and B. Kijsirikul, "On kernelization of supervised Mahalanobis distance learners," arXiv:0804.1441. http://arxiv.org/abs/0804.1441
39 O. Bchir, H. Frigui, and M. M. B. Ismail, "Semi-supervised fuzzy clustering with learnable cluster dependent kernels," International Journal on Artificial Intelligence Tools, vol. 22, no. 3, article number 1350013, Jun. 2013. http://dx.doi.org/10.1142/S0218213013500139   DOI
40 N. Baili and H. Frigui, "Semi-supervised clustering with cluster-dependent multiple kernels," in The 4th International Conference on Information, Intelligence, Systems and Applications Piraeus, Greece, July 10-12, 2013.
41 J. H. Chiang and P. Y. Hao, "A new kernel-based fuzzy clustering approach: support vector clustering with cell growing," IEEE Transactions on Fuzzy Systems, vol. 11, no. 4, pp. 518-527, Aug. 2003. http://dx.doi.org/10.1109/TFUZZ.2003.814839   DOI   ScienceOn
42 B. Scholkopf, J. C. Platt, J. Shawe-Taylor, A. J. Smola, and R. C. Williamson, "Estimating the support of a high-dimensional distribution," Neural Computation, vol. 13, no. 7, pp. 1443-1471, Jul. 2001. http://dx.doi.org/10.1162/089976601750264965   DOI   ScienceOn
43 H. Frigui and O. Nasraoui, "Unsupervised learning of prototypes and attribute weights," Pattern Recognition, vol. 37, no. 3, pp. 567-581, Mar. 2004. http://dx.doi.org/10.1016/j.patcog.2003.08.002   DOI   ScienceOn
44 F. C. H. Rhee, K. S. Choi, and B. I. Choi, "Kernel approach to possibilistic C-means clustering," International Journal of Intelligent Systems, vol. 24, no. 3, pp. 272-292, Mar. 2009. http://dx.doi.org/10.1002/int.20336   DOI   ScienceOn
45 R. Krishnapuram and J. M. Keller, "A possibilistic approach to clustering," IEEE Transactions on Fuzzy Systems, vol. 1, no. 2, pp. 98-110, May 1993. http://dx.doi.org/10.1109/91.227387   DOI   ScienceOn
46 H. Shen, J. Yang, S. Wang, and X. Liu, "Attribute weighted mercer kernel based fuzzy clustering algorithm for general non-spherical datasets," Soft Computing, vol. 10, no. 11, pp. 1061-1073, Sep. 2006. http://dx.doi.org/10.1007/s00500-005-0043-5   DOI
47 M. Sato and Y. Sato, "On a general fuzzy additive clustering model," Intelligent Automation & Soft Computing, vol. 1, no. 4, pp. 439-448, Jan. 1995. http://dx.doi.org/10.1080/10798587.1995.10750648   DOI
48 M. R. P. Ferreira and F. D. A. T. de Carvalho, "Kernel fuzzy clustering methods based on local adaptive distances," in IEEE International Conference on Fuzzy Systems, Brisbane, Australia, June 10-15, 2012, pp. 1-8. http://dx.doi.org/10.1109/FUZZ-IEEE.2012.6251352   DOI
49 D. D. Nguyen, L. T. Ngo, and L. T. Pham, "GMKIT2-FCM: a genetic-based improved multiple kernel interval type-2 fuzzy C-means clustering," in IEEE International Conference on Cybernetics, Lausanne, Switzerland, June 13-15, 2013, pp. 104-109. http://dx.doi.org/10.1109/CYBConf.2013.6617457   DOI
50 J. A. Abhishek and F. C. H. Rhee, "Interval type-2 fuzzy C-means using multiple kernels," in IEEE International Conference on Fuzzy Systems, Hyderabad, India, July 7-10, 2013, pp. 1-8. http://dx.doi.org/10.1109/FUZZ-IEEE.2013.6622306   DOI
51 T. C. Havens, J. C. Bezdek, C. Leckie, L. O. Hall, and M. Palaniswami, "Fuzzy c-Means algorithms for very large data," IEEE Transactions on Fuzzy Systems, vol. 20, no. 6, pp. 1130-1146, Dec. 2012. http://dx.doi.org/10.1109/TFUZZ.2012.2201485   DOI   ScienceOn
52 M. A. Raza and F. C. H. Rhee, "Interval type-2 approach to kernel possibilistic C-means clustering," in IEEE International Conference on Fuzzy Systems, Brisbane, Australia, June 10-15, 2012, pp. 1-7. http://dx.doi.org/10.1109/FUZZ-IEEE.2012.6251233   DOI
53 K. Lin, "A novel evolutionary kernel intuitionistic fuzzy C-means clustering algorithm," IEEE Transactions on Fuzzy Systems, vol. PP, no. 99, article number TFS-2013-0121.R2, Aug. 2013. http://dx.doi.org/10.1109/TFUZZ.2013.2280141   DOI   ScienceOn
54 R. J. Hathaway and J. C. Bezdek, "Extending fuzzy and probabilistic clustering to very large data sets," Computational Statistics & Data Analysis, vol. 51, no. 1, pp. 215-234, Nov. 2006. http://dx.doi.org/10.1016/j.csda.2006.02.008   DOI   ScienceOn
55 N. Baili and H. Frigui, "Incremental fuzzy clustering with multiple kernels," ATSP under review, 2014.
56 H. Frigui and C. Hwang, "Fuzzy clustering and aggregation of relational data with instance-level constraints," IEEE Transactions on Fuzzy Systems, vol. 16, no. 6, pp. 1565-1581, Dec. 2008. http://dx.doi.org/10.1109/TFUZZ.2008.2005692   DOI   ScienceOn
57 K. T. Atanassov, "Intuitionistic fuzzy sets," Fuzzy Sets and Systems, vol. 20, no. 1, pp. 87-96, Aug. 1986. http://dx.doi.org/10.1016/S0165-0114(86)80034-3   DOI   ScienceOn
58 T. C. Havens, J. C. Bezdek, and M. Palaniswami, "Incremental kernel fuzzy c-means," in Computational Intelligence, Studies in Computational Intelligence vol 399, K. Madani, A. D. Correia, A. Rosa, and J. Filipe, Eds. Heidelberg: Springer Berlin, 2012, pp. 3-18. http://dx.doi.org/10.1007/978-3-642-27534-0_1   DOI
59 W. Pedrycz, A. Amato, V. Di Lecce, and V. Piuri, "Fuzzy clustering with partial supervision in organization and classification of digital images," IEEE Transactions on Fuzzy Systems, vol. 16, no. 4, pp. 1008-1026, Aug. 2008. http://dx.doi.org/10.1109/TFUZZ.2008.917287   DOI   ScienceOn
60 V. Loia, W. Pedrycz, and S. Senatore, "Semantic web content analysis: a study in proximity-based collaborative clustering," IEEE Transactions on Fuzzy Systems, vol. 15, no. 6, pp. 1294-1312, Dec. 2007. http://dx.doi.org/10.1109/TFUZZ.2006.889970   DOI   ScienceOn
61 Y. J. Horng, S. M. Chen, Y. C. Chang, and C. H. Lee, "A new method for fuzzy information retrieval based on fuzzy hierarchical clustering and fuzzy inference techniques," IEEE Transactions on Fuzzy Systems, vol. 13, no. 2, pp. 216-228, Apr. 2005. http://dx.doi.org/10.1109/TFUZZ.2004.840134   DOI   ScienceOn
62 S. P. Chatzis and T. A. Varvarigou, "A fuzzy clustering approach toward Hidden Markov random field models for enhanced spatially constrained image segmentation," IEEE Transactions on Fuzzy Systems, vol. 16, no. 5, pp. 1351-1361, Oct. 2008. http://dx.doi.org/10.1109/TFUZZ.2008.2005008   DOI   ScienceOn
63 J. C. Bezdek, Pattern Recognition With Fuzzy Objective Function Algorithms, New York, NY: Plenum Press, 1981.
64 B. Scholkopf, A. Smola, and K. R. Mller, "Nonlinear component analysis as a kernel eigenvalue problem," Neural Computation, vol. 10, no. 5, pp. 1299-1319, Jul. 1998. http://dx.doi.org/10.1162/089976698300017467   DOI   ScienceOn
65 P. X. Liu and M. Q. H. Meng, "Online data-driven fuzzy clustering with applications to real-time robotic tracking," IEEE Transactions on Fuzzy Systems, vol. 12, no. 4, pp. 516-523, Aug. 2004. http://dx.doi.org/10.1109/TFUZZ.2004.832521   DOI   ScienceOn
66 H. Liu, C. Wu, H. Wan, and H. Wang, "Parameter identification based on stabilization diagram with kernel fuzzy clustering method," in International Conference on Transportation, Mechanical, and Electrical Engineering, Changchun, China, December 16-18, 2011, pp. 1185-1188. http://dx.doi.org/10.1109/TMEE.2011.6199417   DOI
67 C. Mei, H. Xu, and J. Liu, "A novel NN-based soft sensor based on modified fuzzy kernel clustering for fermentation process," in International Conference on Intelligent Human-Machine Systems and Cybernetics, Hangzhou, China, August 26-27, 2009, pp. 113-117. http://dx.doi.org/10.1109/IHMSC.2009.153   DOI
68 R. J. Hathaway and J. C. Bezdek, "Nerf c-means: non-Euclidean relational fuzzy clustering," Pattern Recognition, vol. 27, no. 3, pp. 429-437, Mar. 1994. http://dx.doi.org/10.1016/0031-3203(94)90119-8   DOI   ScienceOn