The Korean Journal of Applied Statistics (응용통계연구)

The Korean Statistical Society (한국통계학회)
권호 표지
DOI QR코드

DOI QR Code

Probabilistic reduced K-means cluster analysis

확률적 reduced K-means 군집분석

  • Received : 2021.07.28
  • Accepted : 2021.09.25
  • Published : 2021.12.31
Download PDF
⟨ Previous Next ⟩

Abstract

Cluster analysis is one of unsupervised learning techniques used for discovering clusters when there is no prior knowledge of group membership. K-means, one of the commonly used cluster analysis techniques, may fail when the number of variables becomes large. In such high-dimensional cases, it is common to perform tandem analysis, K-means cluster analysis after reducing the number of variables using dimension reduction methods. However, there is no guarantee that the reduced dimension reveals the cluster structure properly. Principal component analysis may mask the structure of clusters, especially when there are large variances for variables that are not related to cluster structure. To overcome this, techniques that perform dimension reduction and cluster analysis simultaneously have been suggested. This study proposes probabilistic reduced K-means, the transition of reduced K-means (De Soete and Caroll, 1994) into a probabilistic framework. Simulation shows that the proposed method performs better than tandem clustering or clustering without any dimension reduction. When the number of the variables is larger than the number of samples in each cluster, probabilistic reduced K-means show better formation of clusters than non-probabilistic reduced K-means. In the application to a real data set, it revealed similar or better cluster structure compared to other methods.

라벨 없이 진행되는 비지도 학습 중 하나인 군집분석은 자료에 어떤 그룹이 내포되어 있는지 사전 지식이 없을 경우에 군집을 발굴하고, 군집 간의 특성 차이와 군집 안에서의 유사성을 분석하고자 할 때 유용한 방법이다. 기본적인 군집분석 중 하나인 K-means 방법은 변수의 개수가 많아질 때 잘 동작하지 않을 수 있으며, 군집에 대한 해석도 쉽지 않은 문제가 있다. 따라서 고차원 자료의 경우 주성분 분석과 같은 차원 축소 방법을 사용하여 변수의 개수를 줄인 후에 K-means 군집분석을 행하는 Tandem 군집분석이 제안되었다. 하지만 차원 축소 방법을 이용해서 찾아낸 축소 차원이 반드시 군집에 대한 구조를 잘 반영할 것이라는 보장은 없다. 특히 군집의 구조와는 상관없는 변수들의 분산 또는 공분산이 클 때, 주성분 분석을 통한 차원 축소는 오히려 군집의 구조를 가릴 수 있다. 이에 따라 군집분석과 차원 축소를 동시에 진행하는 방법들이 제안되어 왔다. 그 중에서도 본 연구에서는 De Soete와 Carroll (1994)이 제안한 방법론을 확률적인 모형으로 바꿔 군집분석을 진행하는 확률적 reduced K-means를 제안한다. 모의실험 결과 차원 축소를 배제한 군집분석과 Tandem 군집분석보다 더 좋은 군집을 형성함을 알 수 있었고 군집 당 표본 크기에 비해 변수의 개수가 많은 자료에서 기존의 비 확률적 reduced K-means 군집분석에 비해 우수한 성능을 확인했다. 보스턴 자료에서는 다른 군집분석 방법론보다 명확한 군집이 형성됨을 확인했다.

Keywords

References

  1. Arabie P and Hubert L (1994). Cluster analysis in marketing research, Advanced Methods in Marketing Research, 160-189, Oxford: Blackwell.
  2. Choi Y and Baek I (2017). Implication and lesson for city reconstruction experience in the United States, Monthly Housing Finance Report(in korean version), 157, 22-35.
  3. Dempster A, Laird N, and Rubin D (1977). Maximum likelihood from incomplete data via the EM algorithm, Journal of the Royal Statistical Society, Series B (Methodological), 39, 1-38. https://doi.org/10.1111/j.2517-6161.1977.tb01600.x
  4. De Soete G and Carroll JD (1994). K-means clustering in a low-dimensional euclidean space, New Approaches in Classification and Data Analysis, 212-219, Heidelberg: Springer.
  5. Ding C, He X, Zha H, and Simon HD (2002). Adaptive dimension reduction for Clustering High Dimensional Data, Proc 2nd IEEE Intl Conf Data Mining, 147-154.
  6. Ghahramani Z and Hinton GE (1996). The EM algorithm for mixtures of factor analyzers, Technical Report CRG-TR-96-1, University of Toronto, Canada.
  7. Gilley OW and Pace RK (1996). On the Harrison and Rubinfeld data, Journal of Environmental Economics and Management, 31, 403-405. https://doi.org/10.1006/jeem.1996.0052
  8. Harrison D and Rubinfeld DL (1978). Hedonic prices and the demand for clean air, Journal of Environmental Economics and Management, 5, 81-102. https://doi.org/10.1016/0095-0696(78)90006-2
  9. Milligan GW and Cooper MC (1985). An examination of procedures for determining the number of clusters in a data set, Psychometrika, 50, 159-179. https://doi.org/10.1007/BF02294245
  10. Rocci R, Gattone SA, and Vichi M (2011). A new dimension reduction method: factor discriminant K-means, Journal of Classification, 28, 210-226. https://doi.org/10.1007/s00357-011-9085-9
  11. Roweis S (1998). EM algorithms for PCA and SPCA. Advances in Neural Information Processing Systems, Advances in Neural Information Processing Systems 10 - Proceedings of the 1997 Conference, 626-632, Elsevier.
  12. Lee S (2021). Probabilistic reduced K-means cluster analysis, Master's thesis, Department of Statistics, Korea University, Korea(in korean version).
  13. Timmerman ME, Ceulemans E, De Roover K, and Van Leeuwen K (2013). Subspace K-means clustering, Behavior Research Methods, 45, 1011-1023. https://doi.org/10.3758/s13428-013-0329-y
  14. Timmerman ME, Ceulemans E, Kiers HAL, and Vichi M (2010). Factorial and reduced K-means reconsidered, Computational Statistics and Data Analysis, 54, 1858-1871. https://doi.org/10.1016/j.csda.2010.02.009
  15. Tipping ME and Bishop CM (1999a). Mixtures of probabilistic principal component analyzers, Neural Computation, 11, 443-482. https://doi.org/10.1162/089976699300016728
  16. Tipping ME and Bishop CM (1999b). Probabilistic principal component analysis, Journal of the Royal Statistical Society, Series B (Statistical Methodology), 61, 611-622. https://doi.org/10.1111/1467-9868.00196
  17. Vichi M and Kiers HAL (2001). Factorial K-means analysis for two-way data, Computational Statistics and Data Analysis, 37, 49-64. https://doi.org/10.1016/S0167-9473(00)00064-5