International Journal of Fuzzy Logic and Intelligent Systems

Korean Institute of Intelligent Systems (한국지능시스템학회)
권호 표지
DOI QR코드

DOI QR Code

An Overview of Unsupervised and Semi-Supervised Fuzzy Kernel Clustering

  • Frigui, Hichem (Multimedia Research Lab, University of Louisville) ;
  • Bchir, Ouiem (Computer Science Department, College of Computer and Information Systems (CCIS), King Saud University) ;
  • Baili, Naouel (Quintiles)
  • Received : 2013.12.12
  • Accepted : 2013.12.24
  • Published : 2013.12.25
Download PDF
⟨ Previous Next ⟩

Abstract

For real-world clustering tasks, the input data is typically not easily separable due to the highly complex data structure or when clusters vary in size, density and shape. Kernel-based clustering has proven to be an effective approach to partition such data. In this paper, we provide an overview of several fuzzy kernel clustering algorithms. We focus on methods that optimize an fuzzy C-mean-type objective function. We highlight the advantages and disadvantages of each method. In addition to the completely unsupervised algorithms, we also provide an overview of some semi-supervised fuzzy kernel clustering algorithms. These algorithms use partial supervision information to guide the optimization process and avoid local minima. We also provide an overview of the different approaches that have been used to extend kernel clustering to handle very large data sets.

Keywords

References

  1. W. Pedrycz, A. Amato, V. Di Lecce, and V. Piuri, "Fuzzy clustering with partial supervision in organization and classification of digital images," IEEE Transactions on Fuzzy Systems, vol. 16, no. 4, pp. 1008-1026, Aug. 2008. http://dx.doi.org/10.1109/TFUZZ.2008.917287
  2. V. Loia, W. Pedrycz, and S. Senatore, "Semantic web content analysis: a study in proximity-based collaborative clustering," IEEE Transactions on Fuzzy Systems, vol. 15, no. 6, pp. 1294-1312, Dec. 2007. http://dx.doi.org/10.1109/TFUZZ.2006.889970
  3. H. Frigui and C. Hwang, "Fuzzy clustering and aggregation of relational data with instance-level constraints," IEEE Transactions on Fuzzy Systems, vol. 16, no. 6, pp. 1565-1581, Dec. 2008. http://dx.doi.org/10.1109/TFUZZ.2008.2005692
  4. Y. J. Horng, S. M. Chen, Y. C. Chang, and C. H. Lee, "A new method for fuzzy information retrieval based on fuzzy hierarchical clustering and fuzzy inference techniques," IEEE Transactions on Fuzzy Systems, vol. 13, no. 2, pp. 216-228, Apr. 2005. http://dx.doi.org/10.1109/TFUZZ.2004.840134
  5. S. P. Chatzis and T. A. Varvarigou, "A fuzzy clustering approach toward Hidden Markov random field models for enhanced spatially constrained image segmentation," IEEE Transactions on Fuzzy Systems, vol. 16, no. 5, pp. 1351-1361, Oct. 2008. http://dx.doi.org/10.1109/TFUZZ.2008.2005008
  6. P. X. Liu and M. Q. H. Meng, "Online data-driven fuzzy clustering with applications to real-time robotic tracking," IEEE Transactions on Fuzzy Systems, vol. 12, no. 4, pp. 516-523, Aug. 2004. http://dx.doi.org/10.1109/TFUZZ.2004.832521
  7. J. C. Bezdek, Pattern Recognition With Fuzzy Objective Function Algorithms, New York, NY: Plenum Press, 1981.
  8. M. Girolami, "Mercer kernel-based clustering in feature space," IEEE Transactions on Neural Networks, vol. 13, no. 3, pp. 780-784, May 2002. http://dx.doi.org/10.1109/TNN.2002.1000150
  9. R. Inokuchi and S. Miyamoto, "LVQ clustering and SOM using a kernel function," in Proceedings of the IEEE International Conference on Fuzzy Systems, Budapest, Hungary, July 25-29, 2004, pp. 1497-1500. http://dx.doi.org/10.1109/FUZZY.2004.1375395
  10. A. K. Qin and P. N. Suganthan, "Kernel neural gas algorithms with application to cluster analysis," in Proceedings of the 17th International Conference on Pattern Recognition, Cambridge, United Kingdom, August 23-26, 2004, pp. 617-620. http://dx.doi.org/10.1109/ICPR.2004.1333848
  11. U. Luxburg, "A tutorial on spectral clustering," Statistics and Computing, vol. 17, no. 4, pp. 395-416, Dec. 2007. http://dx.doi.org/10.1007/s11222-007-9033-z
  12. X. Peng, Y. Chai, L. Xu, and X. Man, "Research on fault diagnosis of marine diesel engine based on grey relational analysis and kernel fuzzy C-means clustering," in The Fifth International Conference on Intelligent Computation Technology and Automation, Zhangjiajie, China, January 12-14, 2012, pp. 283-286. http://dx.doi.org/10.1109/ICICTA.2012.78
  13. H. Liu, C. Wu, H. Wan, and H. Wang, "Parameter identification based on stabilization diagram with kernel fuzzy clustering method," in International Conference on Transportation, Mechanical, and Electrical Engineering, Changchun, China, December 16-18, 2011, pp. 1185-1188. http://dx.doi.org/10.1109/TMEE.2011.6199417
  14. J. Fan and Y. Wu, "Watermarking algorithm based on kernel fuzzy clustering and singular value decomposition in the complex wavelet transform domain," in International Conference on Information Technology, Computer Engineering and Management Sciences, Nanjing, China, September 24-25, 2011, pp. 42-46. http://dx.doi.org/10.1109/ICM.2011.121
  15. L. Y. Qi and K. G. Wang, "Kernel fuzzy clustering based classification of Ancient-Ceramic fragments," in The 2nd IEEE International Conference on Information Management and Engineering, Chengdu, China, April 16-18, 2010, pp. 348-350. http://dx.doi.org/10.1109/ICIME.2010.5477818
  16. D. M. Tsai and C. C. Lin, "Fuzzy C-means based clustering for linearly and nonlinearly separable data," Pattern Recognition, vol. 44, no. 8, pp. 1750-1760, Aug. 2011. http://dx.doi.org/10.1016/j.patcog.2011.02.009
  17. C. Mei, H. Xu, and J. Liu, "A novel NN-based soft sensor based on modified fuzzy kernel clustering for fermentation process," in International Conference on Intelligent Human-Machine Systems and Cybernetics, Hangzhou, China, August 26-27, 2009, pp. 113-117. http://dx.doi.org/10.1109/IHMSC.2009.153
  18. D. H. Liu, J. P. Bian, and X. Y. Sun, "The study of fault diagnosis model of DGA for oil-immersed transformer based on fuzzy means Kernel clustering and SVM multi-class object simplified structure," in Proceedings of the 7th International Conference on Machine Learning and Cybernetics, Kunming, China, July 12-15, 2008, pp. 1505-1509. http://dx.doi.org/10.1109/ICMLC.2008.4620644
  19. B. Qu and H. Wang, "Dynamic fuzzy kernel clustering analysis of enterprises independent: innovation capability based on artificial immunity," in International Workshop on Modelling, Simulation and Optimization, Hong Kong, December 27-28, 2009, pp. 216-220. http://dx.doi.org/10.1109/WMSO.2008.102
  20. X. Li and S. Bian, "A kernel fuzzy clustering algorithm with spatial constraint based on improved expectation maximization for image segmentation," in International Conference on Measuring Technology and Mechatronics Automation, Zhangjiajie, China, April 11-12, 2009, pp. 529-533. http://dx.doi.org/10.1109/ICMTMA.2009.59
  21. L. Liao and T. S. Lin, "A fast spatial constrained fuzzy kernel clustering algorithm for MRI brain image segmentation," in International Conference on Wavelet Analysis and Pattern Recognition, Beijing, China, November 2-4, 2008, pp. 82-87. http://dx.doi.org/10.1109/ICWAPR.2007.4420641
  22. R. J. G. B. Campello, "A fuzzy extension of the Rand index and other related indexes for clustering and classification assessment," Pattern Recognition Letters, vol. 28, no. 7, pp. 833-841, May 2007. http://dx.doi.org/10.1016/j.patrec.2006.11.010
  23. B. Scholkopf, A. Smola, and K. R. Mller, "Nonlinear component analysis as a kernel eigenvalue problem," Neural Computation, vol. 10, no. 5, pp. 1299-1319, Jul. 1998. http://dx.doi.org/10.1162/089976698300017467
  24. N. Aronszajn, "Theory of reproducing kernels," Transactions of the American Mathematical Society, vol. 68, no. 3, pp. 337-404, May 1950. https://doi.org/10.1090/S0002-9947-1950-0051437-7
  25. D. Zhang and S. Chen, "Fuzzy clustering using kernel method," in International Conference on Control and Automation, Xiamen, China, June 19, 2002, pp. 162-163. http://dx.doi.org/10.1109/ICCA.2002.1229535
  26. D. Q. Zhang and S. C. Chen, "Kernel-based fuzzy and possibilistic c-means clustering," in Proceedings of the International Conference on Artificial Neural Network, Istanbul, Turkey, June 26-29, 2003, pp. 122-125.
  27. R. J. Hathaway, J. M. Huband, and J. C. Bezdek, "A kernelized non-euclidean relational fuzzy c-means algorithm," in IEEE International Conference on Fuzzy Systems, Reno, NV, May 22-25, 2005, pp. 414-419. http://dx.doi.org/10.1109/FUZZY.2005.1452429
  28. R. J. Hathaway and J. C. Bezdek, "Nerf c-means: non-Euclidean relational fuzzy clustering," Pattern Recognition, vol. 27, no. 3, pp. 429-437, Mar. 1994. http://dx.doi.org/10.1016/0031-3203(94)90119-8
  29. O. Bchir and H. Frigui, "Fuzzy relational kernel clustering with local scaling parameter learning," in Proceedings of the 20th IEEE International Workshop on Machine Learning for Signal Processing, Kittila, Finland, August 29-September 1, 2010, pp. 289-294. http://dx.doi.org/10.1109/MLSP.2010.5589234
  30. O. Bchir and H. Frigui, "Fuzzy clustering with learnable cluster dependent kernels," in IEEE International Conference on Fuzzy Systems, Taipei, Taiwan, June 27-30, 2011, pp. 2521-2527. http://dx.doi.org/10.1109/FUZZY.2011.6007411
  31. G. R. G. Lanckriet, N. Cristianini, P. Bartlett, L. El Ghaoui, and M. I. Jordan, "Learning the kernel matrix with semidefinite programming," Journal of Machine Learning Research, vol. 5, pp. 27-72, Jan. 2004.
  32. F. R. Bach, G. R. G. Lanckriet, and M. I. Jordan, "Multiple kernel learning, conic duality, and the SMO algorithm," in Proceedings of the 21st International Conference on Machine Learning, Banff, Canada, July 4-8, 2004, pp. 41-48.
  33. A. Rakotomamonjy, F. Bach, S. Canu, and Y. Grandvalet, "More efficiency in multiple kernel learning," in Proceedings of the 24th International Conference on Machine Learning, Corvalis, OR, June 20-24, 2007, pp. 775-782. http://dx.doi.org/10.1145/1273496.1273594
  34. J. Ye, S. Ji, and J. Chen, "Learning the kernel matrix in discriminant analysis via quadratically constrained quadratic programming," in Proceedings of the 13th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, San Jose, CA, August 12-15, 2007, pp. 854-863. http://dx.doi.org/10.1145/1281192.1281283
  35. M. Gonen and E. Alpaydin, "Localized multiple kernel learning," in Proceedings of the 25th International Conference on Machine Learning, Helsinki, Finland, July 5-9, 2008, pp. 352-359.
  36. B. Zhao, J. T. Kwok, and C. Zhang, "Multiple kernel clustering," in Proceedings of the 9th SIAM International Conference on Data Mining, Sparks, NV, April 30-May 2, 2009, pp. 638-649.
  37. H. C. Huang, Y. Y. Chuang, and C. S. Chen, "Multiple kernel fuzzy clustering," IEEE Transactions on Fuzzy Systems, vol. 20, no. 1, pp. 120-134, Feb. 2012. http://dx.doi.org/10.1109/TFUZZ.2011.2170175
  38. N. Baili and H. Frigui, "Fuzzy clustering with multiple kernels in feature space," in IEEE International Conference on Fuzzy Systems, Brisbane, Australia, June 10-15, 2012. http://dx.doi.org/10.1109/FUZZ-IEEE.2012.6251146
  39. N. Baili and H. Frigui, "Relational fuzzy clustering with multiple kernels," in Proceedings of the 11th IEEE International Conference on Data Mining, Vancouver, Canada, December 11, 2011, pp. 488-495. http://dx.doi.org/10.1109/ICDMW.2011.145
  40. R. J. Hathaway, J. W. Davenport, and J. C. Bezdek, "Relational duals of the c-means clustering algorithms," Pattern Recognition, vol. 22, no. 2, pp. 205-212, 1989. https://doi.org/10.1016/0031-3203(89)90066-6
  41. E. P. Xing, A. Y. Ng, M. I. Jordan, and S. Russell, "Distance metric learning, with application to clustering with side-information," in The 17th Annual Conference on Neural Information Processing Systems, Vancouver & Whistler, Canada, December 8-13, 2003.
  42. K. Weinberger, J. Blitzer, and L. Saul, "Distance metric learning for large margin nearest neighbor classification," in The 19th Annual Conference on Neural Information Processing Systems, Vancouver & Whistler, Canada, December 5-10, 2005.
  43. A. Globerson and S. Roweis, "Metric learning by collapsing classes," in The 19th Annual Conference on Neural Information Processing Systems, Vancouver & Whistler, Canada, December 5-10, 2005.
  44. S. Shalev-Shwartz, Y. Singer, and A. Y. Ng, "Online and batch learning of pseudo-metrics," in Proceedings of the 21st International Conference on Machine Learning, Banff, Canada, July 4-8, 2004, p. 94.
  45. J. V. Davis, B. Kulis, P. Jain, S. Sra, and I. S. Dhillon, "Information-theoretic metric learning," in Proceedings of the 24th International Conference on Machine Learning, Corvallis, OR, June 20-24, 2007, pp. 209-216.
  46. R. Chatpatanasiri, T. Korsrilabutr, P. Tangchanachaianan, and B. Kijsirikul, "On kernelization of supervised Mahalanobis distance learners," arXiv:0804.1441. http://arxiv.org/abs/0804.1441
  47. H. Zhang and J. Lu, "Semi-supervised fuzzy clustering: a kernel-based approach," Knowledge-Based Systems, vol. 22, no. 6, pp. 477-481, Aug. 2009. http://dx.doi.org/10.1016/j.knosys.2009.06.009
  48. O. Bchir, H. Frigui, and M. M. Ben Ismail, "Semi-supervised clustering and local scale learning algorithm," in World Congress on Computer and Information Technology, Sousse, Tunisia, June 22-24, 2013, article number 6618774. http://dx.doi.org/10.1109/WCCIT.2013.6618774
  49. O. Bchir, H. Frigui, and M. M. B. Ismail, "Semi-supervised fuzzy clustering with learnable cluster dependent kernels," International Journal on Artificial Intelligence Tools, vol. 22, no. 3, article number 1350013, Jun. 2013. http://dx.doi.org/10.1142/S0218213013500139
  50. N. Baili and H. Frigui, "Semi-supervised clustering with cluster-dependent multiple kernels," in The 4th International Conference on Information, Intelligence, Systems and Applications Piraeus, Greece, July 10-12, 2013.
  51. J. H. Chiang and P. Y. Hao, "A new kernel-based fuzzy clustering approach: support vector clustering with cell growing," IEEE Transactions on Fuzzy Systems, vol. 11, no. 4, pp. 518-527, Aug. 2003. http://dx.doi.org/10.1109/TFUZZ.2003.814839
  52. B. Scholkopf, J. C. Platt, J. Shawe-Taylor, A. J. Smola, and R. C. Williamson, "Estimating the support of a high-dimensional distribution," Neural Computation, vol. 13, no. 7, pp. 1443-1471, Jul. 2001. http://dx.doi.org/10.1162/089976601750264965
  53. F. C. H. Rhee, K. S. Choi, and B. I. Choi, "Kernel approach to possibilistic C-means clustering," International Journal of Intelligent Systems, vol. 24, no. 3, pp. 272-292, Mar. 2009. http://dx.doi.org/10.1002/int.20336
  54. R. Krishnapuram and J. M. Keller, "A possibilistic approach to clustering," IEEE Transactions on Fuzzy Systems, vol. 1, no. 2, pp. 98-110, May 1993. http://dx.doi.org/10.1109/91.227387
  55. H. Shen, J. Yang, S. Wang, and X. Liu, "Attribute weighted mercer kernel based fuzzy clustering algorithm for general non-spherical datasets," Soft Computing, vol. 10, no. 11, pp. 1061-1073, Sep. 2006. http://dx.doi.org/10.1007/s00500-005-0043-5
  56. H. Frigui and O. Nasraoui, "Unsupervised learning of prototypes and attribute weights," Pattern Recognition, vol. 37, no. 3, pp. 567-581, Mar. 2004. http://dx.doi.org/10.1016/j.patcog.2003.08.002
  57. M. Sato-Ilic, S. Ito, and S. Takahashi, "Generalized kernel fuzzy clustering model," in IEEE International Conference on Fuzzy Systems, Jeju, Korea, August 20-24, 2009, pp. 421-426. http://dx.doi.org/10.1109/FUZZY.2009.5276876
  58. M. Sato and Y. Sato, "On a general fuzzy additive clustering model," Intelligent Automation & Soft Computing, vol. 1, no. 4, pp. 439-448, Jan. 1995. http://dx.doi.org/10.1080/10798587.1995.10750648
  59. M. R. P. Ferreira and F. D. A. T. de Carvalho, "Kernel fuzzy clustering methods based on local adaptive distances," in IEEE International Conference on Fuzzy Systems, Brisbane, Australia, June 10-15, 2012, pp. 1-8. http://dx.doi.org/10.1109/FUZZ-IEEE.2012.6251352
  60. D. D. Nguyen, L. T. Ngo, and L. T. Pham, "GMKIT2-FCM: a genetic-based improved multiple kernel interval type-2 fuzzy C-means clustering," in IEEE International Conference on Cybernetics, Lausanne, Switzerland, June 13-15, 2013, pp. 104-109. http://dx.doi.org/10.1109/CYBConf.2013.6617457
  61. J. A. Abhishek and F. C. H. Rhee, "Interval type-2 fuzzy C-means using multiple kernels," in IEEE International Conference on Fuzzy Systems, Hyderabad, India, July 7-10, 2013, pp. 1-8. http://dx.doi.org/10.1109/FUZZ-IEEE.2013.6622306
  62. M. A. Raza and F. C. H. Rhee, "Interval type-2 approach to kernel possibilistic C-means clustering," in IEEE International Conference on Fuzzy Systems, Brisbane, Australia, June 10-15, 2012, pp. 1-7. http://dx.doi.org/10.1109/FUZZ-IEEE.2012.6251233
  63. K. Lin, "A novel evolutionary kernel intuitionistic fuzzy C-means clustering algorithm," IEEE Transactions on Fuzzy Systems, vol. PP, no. 99, article number TFS-2013-0121.R2, Aug. 2013. http://dx.doi.org/10.1109/TFUZZ.2013.2280141
  64. K. T. Atanassov, "Intuitionistic fuzzy sets," Fuzzy Sets and Systems, vol. 20, no. 1, pp. 87-96, Aug. 1986. http://dx.doi.org/10.1016/S0165-0114(86)80034-3
  65. R. J. Hathaway and J. C. Bezdek, "Extending fuzzy and probabilistic clustering to very large data sets," Computational Statistics & Data Analysis, vol. 51, no. 1, pp. 215-234, Nov. 2006. http://dx.doi.org/10.1016/j.csda.2006.02.008
  66. T. C. Havens, J. C. Bezdek, C. Leckie, L. O. Hall, and M. Palaniswami, "Fuzzy c-Means algorithms for very large data," IEEE Transactions on Fuzzy Systems, vol. 20, no. 6, pp. 1130-1146, Dec. 2012. http://dx.doi.org/10.1109/TFUZZ.2012.2201485
  67. T. C. Havens, J. C. Bezdek, and M. Palaniswami, "Incremental kernel fuzzy c-means," in Computational Intelligence, Studies in Computational Intelligence vol 399, K. Madani, A. D. Correia, A. Rosa, and J. Filipe, Eds. Heidelberg: Springer Berlin, 2012, pp. 3-18. http://dx.doi.org/10.1007/978-3-642-27534-0_1
  68. N. Baili and H. Frigui, "Incremental fuzzy clustering with multiple kernels," ATSP under review, 2014.

Cited by

  1. Polynomial Fuzzy Radial Basis Function Neural Network Classifiers Realized with the Aid of Boundary Area Decision vol.9, pp.6, 2014, https://doi.org/10.5370/JEET.2014.9.6.2098