• 제목/요약/키워드: Variable Clustering

검색결과 155건 처리시간 0.032초

On the clustering of huge categorical data

  • Kim, Dae-Hak
    • Journal of the Korean Data and Information Science Society
    • /
    • 제21권6호
    • /
    • pp.1353-1359
    • /
    • 2010
  • Basic objective in cluster analysis is to discover natural groupings of items. In general, clustering is conducted based on some similarity (or dissimilarity) matrix or the original input data. Various measures of similarities between objects are developed. In this paper, we consider a clustering of huge categorical real data set which shows the aspects of time-location-activity of Korean people. Some useful similarity measure for the data set, are developed and adopted for the categorical variables. Hierarchical and nonhierarchical clustering method are applied for the considered data set which is huge and consists of many categorical variables.

정규분포기반 두각 혼합모형의 순환적 적합을 이용한 군집분석에서의 변수선택 (Variable Selection in Clustering by Recursive Fit of Normal Distribution-based Salient Mixture Model)

  • 김승구
    • 응용통계연구
    • /
    • 제26권5호
    • /
    • pp.821-834
    • /
    • 2013
  • Law 등 (2004)은 군집분석에서 변수선택을 위해 정규분포기반 "두각 혼합모형(salient mixture model)"의 사용을 제안하였다. 본 논문에서는 이 모형의 적합 상의 문제점과 변수선택의 결함을 지적하고 그 대안을 제시한다. 모의자료와 실자료를 바탕으로 제안된 방법이 기존의 방법보다 유용함을 보였다.

다구찌 디자인을 이용한 데이터 퓨전 및 군집분석 분류 성능 비교 (Comparison Study for Data Fusion and Clustering Classification Performances)

  • 신형원;손소영
    • 한국경영과학회:학술대회논문집
    • /
    • 대한산업공학회/한국경영과학회 2000년도 춘계공동학술대회 논문집
    • /
    • pp.601-604
    • /
    • 2000
  • In this paper, we compare the classification performance of both data fusion and clustering algorithms (Data Bagging, Variable Selection Bagging, Parameter Combining, Clustering) to logistic regression in consideration of various characteristics of input data. Four factors used to simulate the logistic model are (1) correlation among input variables (2) variance of observation (3) training data size and (4) input-output function. Since the relationship between input & output is not typically known, we use Taguchi design to improve the practicality of our study results by letting it as a noise factor. Experimental study results indicate the following: Clustering based logistic regression turns out to provide the highest classification accuracy when input variables are weakly correlated and the variance of data is high. When there is high correlation among input variables, variable bagging performs better than logistic regression. When there is strong correlation among input variables and high variance between observations, bagging appears to be marginally better than logistic regression but was not significant.

  • PDF

DETECTING VARIABILITY IN ASTRONOMICAL TIME SERIES DATA: APPLICATIONS OF CLUSTERING METHODS IN CLOUD COMPUTING ENVIRONMENTS

  • 신민수;변용익;장서원;김대원;김명진;이동욱;함재균;정용환;윤준연;곽재혁;김주현
    • 천문학회보
    • /
    • 제36권2호
    • /
    • pp.131.1-131.1
    • /
    • 2011
  • We present applications of clustering methods to detect variability in massive astronomical time series data. Focusing on variability of bright stars, we use clustering methods to separate possible variable sources from other time series data, which include intrinsically non-variable sources and data with common systematic patterns. We already finished the analysis of the Northern Sky Variability Survey data, which include about 16 million light curves, and present candidate variable sources with their association to other data at different wavelengths. We also apply our clustering method to the light curves of bright objects in the SuperWASP Data Release 1. For the analysis of the SuperWASP data, we exploit a elastically configurable Cloud computing environments that the KISTI Supercomputing Center is deploying. Two quite different configurations are incorporated in our Cloud computing test bed. One system uses the Hadoop distributed processing with its distributed file system, using distributed processing with data locality condition. Another one adopts the Condor and the Lustre network file system. We present test results, considering performance of processing a large number of light curves, and finding clusters of variable and non-variable objects.

  • PDF

분산 모바일 서비스의 다중 스트리밍을 위한 가변 클러스터링 관리 (Variable Clustering Management for Multiple Streaming of Distributed Mobile Service)

  • 정택원;이종득
    • 한국지능시스템학회논문지
    • /
    • 제19권4호
    • /
    • pp.485-492
    • /
    • 2009
  • 모바일 서비스 환경에서 시간 동기화에 의해 생성된 패턴들은 데이터 스트리밍으로 인하여 인스턴스 값들이 다르게 스트리밍 된다. 본 논문에서는 유연한 클러스터링을 지원하기 위해 가변클러스터링 관리 기법을 제안하며, 이 구조는 다중 데이터 스트리밍을 동적으로 관리하도록 지원한다. 제안되는 기법은 일반적인 스트리밍기법과 달리 데이터 스트림 환경에서 동기화를 효율적으로 지원하는 기능을 수행하며, 구조적 표현단계와 적합성 표현단계를 거쳐 클러스터링 스트리밍이 관리된다. 구조적 표현 단계는 레벨정합과 누적정합을 수행하여 스트림 구조가 표현되며, 동적 세그먼트와 정적세그먼트 관리를 통해서 클러스터링 관리가 가변적으로 수행되도록 하였다. 제안된 기법의 성능 평가를 위해서 k-means 기법, C/S 서버기법 그리고 CDN 기법과 시뮬레이션평가를 수행하였으며 그 결과 제안된 기법의 성능이 효율적임을 알 수 있었다.

가변적 클러스터 개수에 대한 문서군집화 평가방법 (The Evaluation Measure of Text Clustering for the Variable Number of Clusters)

  • 조태호
    • 한국정보과학회:학술대회논문집
    • /
    • 한국정보과학회 2006년도 가을 학술발표논문집 Vol.33 No.2 (B)
    • /
    • pp.233-237
    • /
    • 2006
  • This study proposes an innovative measure for evaluating the performance of text clustering. In using K-means algorithm and Kohonen Networks for text clustering, the number clusters is fixed initially by configuring it as their parameter, while in using single pass algorithm for text clustering, the number of clusters is not predictable. Using labeled documents, the result of text clustering using K-means algorithm or Kohonen Network is able to be evaluated by setting the number of clusters as the number of the given target categories, mapping each cluster to a target category, and using the evaluation measures of text. But in using single pass algorithm, if the number of clusters is different from the number of target categories, such measures are useless for evaluating the result of text clustering. This study proposes an evaluation measure of text clustering based on intra-cluster similarity and inter-cluster similarity, what is called CI (Clustering Index) in this article.

  • PDF

Reduction of Fuzzy Rules and Membership Functions and Its Application to Fuzzy PI and PD Type Controllers

  • Chopra Seema;Mitra Ranajit;Kumar Vijay
    • International Journal of Control, Automation, and Systems
    • /
    • 제4권4호
    • /
    • pp.438-447
    • /
    • 2006
  • Fuzzy controller's design depends mainly on the rule base and membership functions over the controller's input and output ranges. This paper presents two different approaches to deal with these design issues. A simple and efficient approach; namely, Fuzzy Subtractive Clustering is used to identify the rule base needed to realize Fuzzy PI and PD type controllers. This technique provides a mechanism to obtain the reduced rule set covering the whole input/output space as well as membership functions for each input variable. But it is found that some membership functions projected from different clusters have high degree of similarity. The number of membership functions of each input variable is then reduced using a similarity measure. In this paper, the fuzzy subtractive clustering approach is shown to reduce 49 rules to 8 rules and number of membership functions to 4 and 6 for input variables (error and change in error) maintaining almost the same level of performance. Simulation on a wide range of linear and nonlinear processes is carried out and results are compared with fuzzy PI and PD type controllers without clustering in terms of several performance measures such as peak overshoot, settling time, rise time, integral absolute error (IAE) and integral-of-time multiplied absolute error (ITAE) and in each case the proposed schemes shows an identical performance.

비음수 행렬 분해와 군집의 응집도를 이용한 문서군집 (Document Clustering Method using Coherence of Cluster and Non-negative Matrix Factorization)

  • 김철원;박선
    • 한국정보통신학회논문지
    • /
    • 제13권12호
    • /
    • pp.2603-2608
    • /
    • 2009
  • 문서군집은 정보검색의 많은 응용분야에 사용되는 중요한 문서 분석 방법이다. 본 논문은 비음수 행렬 분해 (NMF, non-negative matrix factorization)를 군집방법과 군집의 응집도(coherence of cluster)를 이용한 군집 내 문서들의 정제를 이용한 새로운 문서군집방법을 제안한다. 제안된 방법은 문서집합의 내부구조를 나타내는 의미특징행렬과 의미변수행렬 이용하여 문서군집의 성능을 높일 수 있고, 문장들 간의 유사도에 기반 한 군집의 응집도를 이용하여 군집내의 문서들을 정제하여서 재 할당함으로써 군집의 효율을 향상시킬 수 있다. 실험결과 제안방법을 적용한 문서군집방법이 다른 문서군집 방법에 비하여 좋은 성능을 보인다.

STATISTICAL NOISE BAND REMOVAL FOR SURFACE CLUSTERING OF HYPERSPECTRAL DATA

  • Huan, Nguyen Van;Kim, Hak-Il
    • 대한원격탐사학회:학술대회논문집
    • /
    • 대한원격탐사학회 2008년도 International Symposium on Remote Sensing
    • /
    • pp.111-114
    • /
    • 2008
  • The existence of noise bands may deform the typical shape of the spectrum, making the accuracy of clustering degraded. This paper proposes a statistical approach to remove noise bands in hyperspectral data using the correlation coefficient of bands as an indicator. Considering each band as a random variable, two adjacent signal bands in hyperspectral data are highly correlative. On the contrary, existence of a noise band will produce a low correlation. For clustering, the unsupervised ${\kappa}$-nearest neighbor clustering method is implemented in accordance with three well-accepted spectral matching measures, namely ED, SAM and SID. Furthermore, this paper proposes a hierarchical scheme of combining those measures. Finally, a separability assessment based on the between-class and the within-class scatter matrices is followed to evaluate the applicability of the proposed noise band removal method. Also, the paper brings out a comparison for spectral matching measures.

  • PDF

이중 K-평균 군집화 (Double K-Means Clustering)

  • 허명회
    • 응용통계연구
    • /
    • 제13권2호
    • /
    • pp.343-352
    • /
    • 2000
  • K-평균 군집화(K-means clustering)는 비계층적 군집화 방법이 하나로서 큰 자료에서 개체 군집화에 효율적인 것으로 알려져 있다. 그러나 종종 비교적 균일한 대군집의 일부를 소군집에 떼어주는 오류를 범하기도 한다. 이 연구에서는 그러한 현상을 정확히 인지하고 이에 대한 대책으로서 ‘이중 K-평균 군집화(double K-means clustering)’방법을 제시한다. 또한 실증적 사례에 새 방법론을 적용해보고 토의한다.

  • PDF