• 제목/요약/키워드: Data Clustering

검색결과 2,731건 처리시간 0.03초

Comparison of the Performance of Clustering Analysis using Data Reduction Techniques to Identify Energy Use Patterns

  • Song, Kwonsik;Park, Moonseo;Lee, Hyun-Soo;Ahn, Joseph
    • 국제학술발표논문집
    • /
    • The 6th International Conference on Construction Engineering and Project Management
    • /
    • pp.559-563
    • /
    • 2015
  • Identification of energy use patterns in buildings has a great opportunity for energy saving. To find what energy use patterns exist, clustering analysis has been commonly used such as K-means and hierarchical clustering method. In case of high dimensional data such as energy use time-series, data reduction should be considered to avoid the curse of dimensionality. Principle Component Analysis, Autocorrelation Function, Discrete Fourier Transform and Discrete Wavelet Transform have been widely used to map the original data into the lower dimensional spaces. However, there still remains an ongoing issue since the performance of clustering analysis is dependent on data type, purpose and application. Therefore, we need to understand which data reduction techniques are suitable for energy use management. This research aims find the best clustering method using energy use data obtained from Seoul National University campus. The results of this research show that most experiments with data reduction techniques have a better performance. Also, the results obtained helps facility managers optimally control energy systems such as HVAC to reduce energy use in buildings.

  • PDF

시간단위 전력사용량 시계열 패턴의 군집 및 분류분석 (Clustering and classification to characterize daily electricity demand)

  • 박다인;윤상후
    • Journal of the Korean Data and Information Science Society
    • /
    • 제28권2호
    • /
    • pp.395-406
    • /
    • 2017
  • 전력 공급 시스템의 효율적인 운영을 위해 전력수요예측은 필수적이다. 본 연구에서는 군집분석과 분류분석을 이용하여 일 단위 시간별 전력수요량 시계열 패턴의 유형을 살펴보고자 한다. 전력거래소에서 수집된 2008년 1월 1일부터 2012년 12월 31일까지의 일 단위 시간별 전력수요량 데이터를 추세성분, 계절성분, 오차 성분으로 구성된 시계열 자료로 변환하여 사용하였다. 추세성분을 제거한 시계열 자료의 패턴을 구분하기 위한 군집 분석방법은 k-평균 군집분석 (k-means), 가우시안혼합모델 혼합 모델 군집분석 (Gaussian mixture model), 함수적 군집분석 (functional clustering)을 고려하였다. 주성분분석을 통해 24시간 자료를 2개의 요인로 축소한 후 k-평균 군집분석과 가우시안 혼합 모델, 함수적 군집분석을 수행하였다. 군집분석 결과를 토대로 2008년부터 2011년까지 총 4년간 데이터를 4가지 분류분석방법인 의사결정나무, RF (random forest), Naive bayes, SVM (support vector machine)을 통해 훈련시켜 2012년 군집을 예측하였다. 분석 결과 가우시안 혼합 분포기반 군집분석과 RF를 이용한 군집예측 결과의 성능이 가장 우수하였다.

다차원 데이터에 대한 심층 군집 네트워크의 성능향상 방법 (Performance Improvement of Deep Clustering Networks for Multi Dimensional Data)

  • 이현진
    • 한국멀티미디어학회논문지
    • /
    • 제21권8호
    • /
    • pp.952-959
    • /
    • 2018
  • Clustering is one of the most fundamental algorithms in machine learning. The performance of clustering is affected by the distribution of data, and when there are more data or more dimensions, the performance is degraded. For this reason, we use a stacked auto encoder, one of the deep learning algorithms, to reduce the dimension of data which generate a feature vector that best represents the input data. We use k-means, which is a famous algorithm, as a clustering. Sine the feature vector which reduced dimensions are also multi dimensional, we use the Euclidean distance as well as the cosine similarity to increase the performance which calculating the similarity between the center of the cluster and the data as a vector. A deep clustering networks combining a stacked auto encoder and k-means re-trains the networks when the k-means result changes. When re-training the networks, the loss function of the stacked auto encoder and the loss function of the k-means are combined to improve the performance and the stability of the network. Experiments of benchmark image ad document dataset empirically validated the power of the proposed algorithm.

K-평균 군집화의 재현성 평가 및 응용 (Reproducibility Assessment of K-Means Clustering and Applications)

  • 허명회;이용구
    • 응용통계연구
    • /
    • 제17권1호
    • /
    • pp.135-144
    • /
    • 2004
  • K-평균 군집화(K-means clustering)는 고객 세분화(customer segmentation) 등 데이터 마이닝에서 중요한 한 몫을 하는 비지도 학습방법 (unsupervised learning method)이다. K-평균 군집화가 재현성(reproducibility)이 있는가를 보기 위하여, 다수의 기존 연구에서는 관측 자료를 2개 셋으로 나눈 자료 분할(data partitioning) 방법이 활용되고 있다. 본 교신에서 우리는 이보다 개념적으로 명확한 새로운 자료 분할 방법을 제안한다. 이 방법은 관측 자료를 3개 셋으로 나누어 그 중 2개 자료 셋을 독립적인 군집화 규칙을 생성하는 데 사용하고 나머지 1개의 자료 셋을 규칙간 일치성을 테스트하는데 사용한다. 또한 2개의 군집화 규칙간 일치성 평가를 위한 지표로서 엔트로피 기준의 환용 방법을 제시한다.

다자간 환경에서 프라이버시를 보호하는 효율적인 DBSCAN 군집화 기법 (Practical Privacy-Preserving DBSCAN Clustering Over Horizontally Partitioned Data)

  • 김기성;정익래
    • 정보보호학회논문지
    • /
    • 제20권3호
    • /
    • pp.105-111
    • /
    • 2010
  • 본 논문은 다자간 환경에서 프라이버시를 보호하는 효율적인 DBSCAN 군집화 기법을 제안한다. 기존 DBSCAN 군집화 기법에 가짜 데이터 삽입을 통한 프라이버시 보호 기법을 적용해 다자간 환경에서 프라이버시를 보호하는 기법으로 확장했다. 기존의 프라이버시를 보호하는 다자간 환경의 군집화 기법들은 비효율적이어서 실제 환경에 적용하기 힘들지만 제안한 기법은 이러한 문제를 해결한 매우 효율적인 기법이다. 본 기법은 다자간 환경뿐만 아니라 비 다자간 환경에도 적용 가능한 효율적인 기법이다.

속성유사도에 따른 사회연결망 서브그룹의 군집유효성 (Clustering Validity of Social Network Subgroup Using Attribute Similarity)

  • 윤한성
    • 디지털산업정보학회논문지
    • /
    • 제17권1호
    • /
    • pp.75-84
    • /
    • 2021
  • For analyzing big data, the social network is increasingly being utilized through relational data, which means the connection characteristics between entities such as people and objects. When the relational data does not exist directly, a social network can be configured by calculating relational data such as attribute similarity from attribute data of entities and using it as links. In this paper, the composition method of the social network using the attribute similarity between entities as a connection relationship, and the clustering method using subgroups for the configured social network are suggested, and the clustering effectiveness of the clustering results is evaluated. The analysis results can vary depending on the type and characteristics of the data to be analyzed, the type of attribute similarity selected, and the criterion value. In addition, the clustering effectiveness may not be consistent depending on the its evaluation method. Therefore, selections and experiments are necessary for better analysis results. Since the analysis results may be different depending on the type and characteristics of the analysis target, options for clustering, etc., there is a limitation. In addition, for performance evaluation of clustering, a study is needed to compare the method of this paper with the conventional method such as k-means.

연속형 자료에 대한 나무형 군집화 (Tree-structured Clustering for Continuous Data)

  • 허명회;양경숙
    • 응용통계연구
    • /
    • 제18권3호
    • /
    • pp.661-671
    • /
    • 2005
  • 본 연구는 반복분할(recursive partitioning)에 의한 군집화 방법을 개발하고 활용 예를 보인다. 노드 분리 기준으로는 Overall R-Square를 채택하였고 실용적인 노드 분리 결정 방법을 제안하였다. 이 방법은 연속형 자료에 대하여 나무 형태의 해석하기 쉬운 단순한 규칙을 제공하면서 동시에 변수선택기능을 제공한다. 환용 예로서 Fisher의 붓꽃데이터와 Telecom 사례에 적용해 보았다. K-평균 군집화와 다른 몇 가지 사항이 관측되었다.

A Mixed Co-clustering Algorithm Based on Information Bottleneck

  • Liu, Yongli;Duan, Tianyi;Wan, Xing;Chao, Hao
    • Journal of Information Processing Systems
    • /
    • 제13권6호
    • /
    • pp.1467-1486
    • /
    • 2017
  • Fuzzy co-clustering is sensitive to noise data. To overcome this noise sensitivity defect, possibilistic clustering relaxes the constraints in FCM-type fuzzy (co-)clustering. In this paper, we introduce a new possibilistic fuzzy co-clustering algorithm based on information bottleneck (ibPFCC). This algorithm combines fuzzy co-clustering and possibilistic clustering, and formulates an objective function which includes a distance function that employs information bottleneck theory to measure the distance between feature data point and feature cluster centroid. Many experiments were conducted on three datasets and one artificial dataset. Experimental results show that ibPFCC is better than such prominent fuzzy (co-)clustering algorithms as FCM, FCCM, RFCC and FCCI, in terms of accuracy and robustness.

QCanvas: An Advanced Tool for Data Clustering and Visualization of Genomics Data

  • Kim, Nayoung;Park, Herin;He, Ningning;Lee, Hyeon Young;Yoon, Sukjoon
    • Genomics & Informatics
    • /
    • 제10권4호
    • /
    • pp.263-265
    • /
    • 2012
  • We developed a user-friendly, interactive program to simultaneously cluster and visualize omics data, such as DNA and protein array profiles. This program provides diverse algorithms for the hierarchical clustering of two-dimensional data. The clustering results can be interactively visualized and optimized on a heatmap. The present tool does not require any prior knowledge of scripting languages to carry out the data clustering and visualization. Furthermore, the heatmaps allow the selective display of data points satisfying user-defined criteria. For example, a clustered heatmap of experimental values can be differentially visualized based on statistical values, such as p-values. Including diverse menu-based display options, QCanvas provides a convenient graphical user interface for pattern analysis and visualization with high-quality graphics.

Volatility clustering in data breach counts

  • Shim, Hyunoo;Kim, Changki;Choi, Yang Ho
    • Communications for Statistical Applications and Methods
    • /
    • 제27권4호
    • /
    • pp.487-500
    • /
    • 2020
  • Insurers face increasing demands for cyber liability; entailed in part by a variety of new forms of risk of data breaches. As data breach occurrences develop, our understanding of the volatility in data breach counts has also become important as well as its expected occurrences. Volatility clustering, the tendency of large changes in a random variable to cluster together in time, are frequently observed in many financial asset prices, asset returns, and it is questioned whether the volatility of data breach occurrences are also clustered in time. We now present volatility analysis based on INGARCH models, i.e., integer-valued generalized autoregressive conditional heteroskedasticity time series model for frequency counts due to data breaches. Using the INGARCH(1, 1) model with data breach samples, we show evidence of temporal volatility clustering for data breaches. In addition, we present that the firms' volatilities are correlated between some they belong to and that such a clustering effect remains even after excluding the effect of financial covariates such as the VIX and the stock return of S&P500 that have their own volatility clustering.