Browse > Article
http://dx.doi.org/10.5351/KJAS.2021.34.6.905

Probabilistic reduced K-means cluster analysis  

Lee, Seunghoon (Department of Statistics, Korea University)
Song, Juwon (Department of Statistics, Korea University)
Publication Information
The Korean Journal of Applied Statistics / v.34, no.6, 2021 , pp. 905-922 More about this Journal
Abstract
Cluster analysis is one of unsupervised learning techniques used for discovering clusters when there is no prior knowledge of group membership. K-means, one of the commonly used cluster analysis techniques, may fail when the number of variables becomes large. In such high-dimensional cases, it is common to perform tandem analysis, K-means cluster analysis after reducing the number of variables using dimension reduction methods. However, there is no guarantee that the reduced dimension reveals the cluster structure properly. Principal component analysis may mask the structure of clusters, especially when there are large variances for variables that are not related to cluster structure. To overcome this, techniques that perform dimension reduction and cluster analysis simultaneously have been suggested. This study proposes probabilistic reduced K-means, the transition of reduced K-means (De Soete and Caroll, 1994) into a probabilistic framework. Simulation shows that the proposed method performs better than tandem clustering or clustering without any dimension reduction. When the number of the variables is larger than the number of samples in each cluster, probabilistic reduced K-means show better formation of clusters than non-probabilistic reduced K-means. In the application to a real data set, it revealed similar or better cluster structure compared to other methods.
Keywords
cluster analysis; dimension reduction; unsupervised learning; EM-algorithm; high-dimension;
Citations & Related Records
연도 인용수 순위
  • Reference
1 De Soete G and Carroll JD (1994). K-means clustering in a low-dimensional euclidean space, New Approaches in Classification and Data Analysis, 212-219, Heidelberg: Springer.
2 Ding C, He X, Zha H, and Simon HD (2002). Adaptive dimension reduction for Clustering High Dimensional Data, Proc 2nd IEEE Intl Conf Data Mining, 147-154.
3 Gilley OW and Pace RK (1996). On the Harrison and Rubinfeld data, Journal of Environmental Economics and Management, 31, 403-405.   DOI
4 Harrison D and Rubinfeld DL (1978). Hedonic prices and the demand for clean air, Journal of Environmental Economics and Management, 5, 81-102.   DOI
5 Milligan GW and Cooper MC (1985). An examination of procedures for determining the number of clusters in a data set, Psychometrika, 50, 159-179.   DOI
6 Arabie P and Hubert L (1994). Cluster analysis in marketing research, Advanced Methods in Marketing Research, 160-189, Oxford: Blackwell.
7 Choi Y and Baek I (2017). Implication and lesson for city reconstruction experience in the United States, Monthly Housing Finance Report(in korean version), 157, 22-35.
8 Dempster A, Laird N, and Rubin D (1977). Maximum likelihood from incomplete data via the EM algorithm, Journal of the Royal Statistical Society, Series B (Methodological), 39, 1-38.   DOI
9 Timmerman ME, Ceulemans E, De Roover K, and Van Leeuwen K (2013). Subspace K-means clustering, Behavior Research Methods, 45, 1011-1023.   DOI
10 Tipping ME and Bishop CM (1999b). Probabilistic principal component analysis, Journal of the Royal Statistical Society, Series B (Statistical Methodology), 61, 611-622.   DOI
11 Roweis S (1998). EM algorithms for PCA and SPCA. Advances in Neural Information Processing Systems, Advances in Neural Information Processing Systems 10 - Proceedings of the 1997 Conference, 626-632, Elsevier.
12 Lee S (2021). Probabilistic reduced K-means cluster analysis, Master's thesis, Department of Statistics, Korea University, Korea(in korean version).
13 Ghahramani Z and Hinton GE (1996). The EM algorithm for mixtures of factor analyzers, Technical Report CRG-TR-96-1, University of Toronto, Canada.
14 Rocci R, Gattone SA, and Vichi M (2011). A new dimension reduction method: factor discriminant K-means, Journal of Classification, 28, 210-226.   DOI
15 Vichi M and Kiers HAL (2001). Factorial K-means analysis for two-way data, Computational Statistics and Data Analysis, 37, 49-64.   DOI
16 Timmerman ME, Ceulemans E, Kiers HAL, and Vichi M (2010). Factorial and reduced K-means reconsidered, Computational Statistics and Data Analysis, 54, 1858-1871.   DOI
17 Tipping ME and Bishop CM (1999a). Mixtures of probabilistic principal component analyzers, Neural Computation, 11, 443-482.   DOI