• Title/Summary/Keyword: data clustering

Search Result 2,729, Processing Time 0.036 seconds

Development of Similarity-Based Document Clustering System (유사성 계수에 의한 문서 클러스터링 시스템 개발)

  • Woo Hoon-Shik;Yim Dong-Soon
    • Proceedings of the Society of Korea Industrial and System Engineering Conference
    • /
    • 2002.05a
    • /
    • pp.119-124
    • /
    • 2002
  • Clustering of data is of a great interest in many data mining applications. In the field of document clustering, a document is represented as a data in a high dimensional space. Therefore, the document clustering can be accomplished with a general data clustering techniques. In this paper, we introduce a document clustering system based on similarity among documents. The developed system consists of three functions: 1) gatherings documents utilizing a search agent; 2) determining similarity coefficients between any two documents from term frequencies; 3) clustering documents with similarity coefficients. Especially, the document clustering is accomplished by a hybrid algorithm utilizing genetic and K-Means methods.

  • PDF

Geodesic Clustering for Covariance Matrices

  • Lee, Haesung;Ahn, Hyun-Jung;Kim, Kwang-Rae;Kim, Peter T.;Koo, Ja-Yong
    • Communications for Statistical Applications and Methods
    • /
    • v.22 no.4
    • /
    • pp.321-331
    • /
    • 2015
  • The K-means clustering algorithm is a popular and widely used method for clustering. For covariance matrices, we consider a geodesic clustering algorithm based on the K-means clustering framework in consideration of symmetric positive definite matrices as a Riemannian (non-Euclidean) manifold. This paper considers a geodesic clustering algorithm for data consisting of symmetric positive definite (SPD) matrices, utilizing the Riemannian geometric structure for SPD matrices and the idea of a K-means clustering algorithm. A K-means clustering algorithm is divided into two main steps for which we need a dissimilarity measure between two matrix data points and a way of computing centroids for observations in clusters. In order to use the Riemannian structure, we adopt the geodesic distance and the intrinsic mean for symmetric positive definite matrices. We demonstrate our proposed method through simulations as well as application to real financial data.

Clustering non-stationary advanced metering infrastructure data

  • Kang, Donghyun;Lim, Yaeji
    • Communications for Statistical Applications and Methods
    • /
    • v.29 no.2
    • /
    • pp.225-238
    • /
    • 2022
  • In this paper, we propose a clustering method for advanced metering infrastructure (AMI) data in Korea. As AMI data presents non-stationarity, we consider time-dependent frequency domain principal components analysis, which is a proper method for locally stationary time series data. We develop a new clustering method based on time-varying eigenvectors, and our method provides a meaningful result that is different from the clustering results obtained by employing conventional methods, such as K-means and K-centres functional clustering. Simulation study demonstrates the superiority of the proposed approach. We further apply the clustering results to the evaluation of the electricity price system in South Korea, and validate the reform of the progressive electricity tariff system.

Empirical Comparisons of Clustering Algorithms using Silhouette Information

  • Jun, Sung-Hae;Lee, Seung-Joo
    • International Journal of Fuzzy Logic and Intelligent Systems
    • /
    • v.10 no.1
    • /
    • pp.31-36
    • /
    • 2010
  • Many clustering algorithms have been used in diverse fields. When we need to group given data set into clusters, many clustering algorithms based on similarity or distance measures are considered. Most clustering works have been based on hierarchical and non-hierarchical clustering algorithms. Generally, for the clustering works, researchers have used clustering algorithms case by case from these algorithms. Also they have to determine proper clustering methods subjectively by their prior knowledge. In this paper, to solve the subjective problem of clustering we make empirical comparisons of popular clustering algorithms which are hierarchical and non hierarchical techniques using Silhouette measure. We use silhouette information to evaluate the clustering results such as the number of clusters and cluster variance. We verify our comparison study by experimental results using data sets from UCI machine learning repository. Therefore we are able to use efficient and objective clustering algorithms.

Curve Clustering in Microarray

  • Lee, Kyeong-Eun
    • Journal of the Korean Data and Information Science Society
    • /
    • v.15 no.3
    • /
    • pp.575-584
    • /
    • 2004
  • We propose a Bayesian model-based approach using a mixture of Dirichlet processes model with discrete wavelet transform, for curve clustering in the microarray data with time-course gene expressions.

  • PDF

A Constraint-based Semi-supervised Clustering Through Initial Prediction of Unlabeled Data (비분류표시 데이터의 초기예측을 통한 제약기반 부분-지도 군집분석)

  • Kim, Eung-Gu;Jeon, Chi-Hyeok
    • Proceedings of the Korean Operations and Management Science Society Conference
    • /
    • 2007.11a
    • /
    • pp.383-387
    • /
    • 2007
  • Traditional clustering is regarded as an unsupervised teaming to analyze unlabeled data. Semi-supervised clustering uses a small amount of labeled data to predict labels of unlabeled data as well as to improve clustering performance. Previous methods use constraints generated from available labeled data in clustering process. We propose a new constraint-based semi-supervised clustering method by reflecting initial predicted labels of unlabeled data. We evaluate and compare the performance of the proposed method in terms of classification errors through numerical experiments with blinded labeled data.

  • PDF

Augmented Rotation-Based Transformation for Privacy-Preserving Data Clustering

  • Hong, Do-Won;Mohaisen, Abedelaziz
    • ETRI Journal
    • /
    • v.32 no.3
    • /
    • pp.351-361
    • /
    • 2010
  • Multiple rotation-based transformation (MRBT) was introduced recently for mitigating the apriori-knowledge independent component analysis (AK-ICA) attack on rotation-based transformation (RBT), which is used for privacy-preserving data clustering. MRBT is shown to mitigate the AK-ICA attack but at the expense of data utility by not enabling conventional clustering. In this paper, we extend the MRBT scheme and introduce an augmented rotation-based transformation (ARBT) scheme that utilizes linearity of transformation and that both mitigates the AK-ICA attack and enables conventional clustering on data subsets transformed using the MRBT. In order to demonstrate the computational feasibility aspect of ARBT along with RBT and MRBT, we develop a toolkit and use it to empirically compare the different schemes of privacy-preserving data clustering based on data transformation in terms of their overhead and privacy.

Cluster ing for Analysis of Raman Hyper spectral Dental Data

  • Jung, Sung-Hwan
    • Journal of Korea Multimedia Society
    • /
    • v.16 no.1
    • /
    • pp.19-28
    • /
    • 2013
  • In this research, we presented an effective clustering method based on ICA for the analysis of huge Raman hyperspectral dental data. The hyperspectral dataset captured by HR800 micro Raman spectrometer at UMKC-CRISP(University of Missouri-Kansas City Center for Research on Interfacial Structure and Properties), has 569 local points. Each point has 1,005 hyperspectal dentin data. We compared the clustering effectiveness and the clustering time for the case of using all dataset directly and the cases of using the scores after PCA and ICA. As the result of experiment, the cases of using the scores after PCA and ICA showed, not only more detailed internal dentin information in the aspect of medical analysis, but also about 7~19 times much shorter processing times for clustering. ICA based approach also presented better performance than that of PCA, in terms of the detailed internal information of dentin and the clustering time. Therefore, we could confirm the effectiveness of ICA for the analysis of Raman hyperspectral dental data.

Mining Clusters of Sequence Data using Sequence Element-based Similarity Measure (시퀀스 요소 기반의 유사도를 이용한 시퀀스 데이터 클러스터링)

  • 오승준;김재련
    • Proceedings of the Korea Inteligent Information System Society Conference
    • /
    • 2004.11a
    • /
    • pp.221-229
    • /
    • 2004
  • Recently, there has been enormous growth in the amount of commercial and scientific data, such as protein sequences, retail transactions, and web-logs. Such datasets consist of sequence data that have an inherent sequential nature. However, only a few of the existing clustering algorithms consider sequentiality. This study presents a method for clustering such sequence datasets. The similarity between sequences must be decided before clustering the sequences. This study proposes a new similarity measure to compute the similarity between two sequences using a sequence element. Two clustering algorithms using the proposed similarity measure are proposed: a hierarchical clustering algorithm and a scalable clustering algorithm that uses sampling and a k-nearest neighbor method. Using a splice dataset and synthetic datasets, we show that the quality of clusters generated by our proposed clustering algorithms is better than that of clusters produced by traditional clustering algorithms.

  • PDF

The Study on Improvement of Cohesion of Clustering in Incremental Concept Learning (점진적 개념학습의 클러스터 응집도 개선)

  • Baek, Hey-Jung;Park, Young-Tack
    • The KIPS Transactions:PartB
    • /
    • v.10B no.3
    • /
    • pp.297-304
    • /
    • 2003
  • Nowdays, with the explosive growth of the web information, web users Increase requests of systems which collect and analyze web pages that are relevant. The systems which were develop to solve the request were used clustering methods to improve the duality of information. Clustering is defining inter relationship of unordered data and grouping data systematically. The systems using clustering provide the grouped information to the users. So, they understand the information efficiently. We proposed a hybrid clustering method to cluster a large quantity of data efficiently. By that method, We generate initial clusters using COBWEB Algorithm and refine them using Ezioni Algorithm. This paper adds two ideas in prior hybrid clustering method to increment accuracy and efficiency of clusters. Firstly, we propose the clustering method considering weight of attributes of data. Second, we redefine evaluation functions which generate initial clusters to increase efficiency in clustering. Clustering method proposed in this paper processes a large quantity of data and diminish of dependancy on sequence of input of data. So the clusters are useful to make user profiles in high quality. Ultimately, we will show that the proposed clustering method outperforms the pervious clustering method in the aspect of precision and execution speed.