• Journal of the Korean Data and Information Science Society
• /
• v.26 no.1
• /
• pp.89-99
• /
• 2015

The World Economic Forum and the Korean Ministry of Knowledge Economy have selected big data as one of the top 10 in core information technology. The key of big data is to analyze effectively the properties that do have data. Clustering analysis method of big data techniques is a method of assigning a set of objects into the clusters so that the objects in the same cluster are more similar to each other clusters. Similarity measures being used in the cluster analysis may be classified into various types depending on the nature of the data. In this paper, we studied upper and lower bounds for binary similarity measures with negative matches such as Russel and Rao measure, simple matching measure by Sokal and Michener, Rogers and Tanimoto measure, Sokal and Sneath measure, Hamann measure, and Baroni-Urbani and Buser mesures I, II. And the comparative studies with these measures were shown by real data and simulated experiment.

• Journal of the Korean Data and Information Science Society
• /
• v.26 no.2
• /
• pp.367-376
• /
• 2015

Today, big data has become a hot keyword in that big data may be defined as collection of data sets so huge and complex that it becomes difficult to process by traditional methods. Clustering method is to identify the information in a big database by assigning a set of objects into the clusters so that the objects in the same cluster are more similar to each other clusters. The similarity measures being used in the cluster analysis may be classified into various types depending on the nature of the data. In this paper, we computed upper and lower limits for probability interestingness measure based similarity measures without marginal probability such as Yule I and II, Michael, Digby, Baulieu, and Dispersion measure. And we compared these measures by real data and simulated experiment. By Warrens (2008), Coefficients with the same quantities in the numerator and denominator, that are bounded, and are close to each other in the ordering, are likely to be more similar. Thus, results on bounds provide means of classifying various measures. Also, knowing which coefficients are similar provides insight into the stability of a given algorithm.

• Journal of the Korean Data and Information Science Society
• /
• v.22 no.3
• /
• pp.495-503
• /
• 2011

The most widely used data mining technique is to find association rules. Association rule mining is the method to quantify the relationship between each set of items in very huge database based on the association thresholds. There are some basic association thresholds to explore meaningful association rules ; support, confidence, lift, etc. Among them, confidence is the most frequently used, but it has the drawback that it can not determine the direction of the association. The net confidence and the attributably pure confidence were developed to compensate for this drawback, but they have other drawbacks.In this paper we consider some predictive similarity measures for binary data in cluster analysis and multi-dimensional analysis as association threshold to compensate for these drawbacks. The comparative studies with net confidence, attributably pure confidence, and some predictive similarity measures are shown by numerical example.

• Journal of the Korean Data and Information Science Society
• /
• v.22 no.6
• /
• pp.1113-1121
• /
• 2011

Recently, a variety of data mining techniques has been applied in various fields like healthcare, insurance, and internet shopping mall. Association rule mining is a popular and well researched method for discovering interesting relations among large set of data items. Association rule mining is the method to quantify the relationship between each set of items in very huge database based on the association thresholds. There are three primary quality measures for association rules; support and confidence and lift. In this paper we consider some similarity measures with negative co-occurrence frequencies which is widely used in cluster analysis or multi-dimensional analysis as association thresholds. The comparative studies with support, confidence and some similarity measures are shown by numerical example.

• Journal of the Korean Institute of Intelligent Systems
• /
• v.15 no.5
• /
• pp.521-526
• /
• 2005

The fuzzy entropy is proposed for measuring of uncertainty with the help of relation between distance measure and similarity measure. The proposed fuzzy entropy is constructed through a distance measure. In this study, Hamming distance measure is employed for a distance measure. Also a similarity measure is constructed through a distance measure for the measure of similarity between fuzzy sets or crisp sets and the proposed fuzzy entropies and similarity measures are proved.

• Journal of the Korean Data and Information Science Society
• /
• v.26 no.4
• /
• pp.857-864
• /
• 2015

By Wikipedia, data mining is the computational process of discovering patterns in huge data sets involving methods at the intersection of association rule, decision tree, clustering, artificial intelligence, machine learning. Clustering or cluster analysis is the task of grouping a set of objects in such a way that objects in the same group are more similar to each other than to those in other groups. The similarity measures being used in the clustering may be classified into various types depending on the characteristics of data. In this paper, we computed bounds for similarity measures based on the probabilistic interestingness measure with partially marginal probability such as Peirce I, Peirce II, Cole I, Cole II, Loevinger, Park I, and Park II measure. We confirmed the absolute value of Loevinger measure wasthe upper limit of the absolute value of any other existing measures. Ordering of other measures is determined by the size of concurrence proportion, non-simultaneous occurrence proportion, and mismatch proportion.

• Journal of the Korean Data and Information Science Society
• /
• v.23 no.6
• /
• pp.1127-1135
• /
• 2012

Association rule mining is the method to quantify the relationship between each set of items in a large database. One of the well-studied problems in data mining is exploration for association rules. There are three primary quality measures for association rule, support and confidence and lift. We generate some association rules using confidence. Confidence is the most important measure of these measures, but it is an asymmetric measure and has only positive value. Thus we can face with difficult problems in generation of association rules. In this paper we apply the similarity measures by probabilistic interestingness measure to find a solution to this problem. The comparative studies with support, two confidences, lift, and some similarity measures by probabilistic interestingness measure are shown by numerical example. As the result, we knew that the similarity measures by probabilistic interestingness measure could be seen the degree of association same as confidence. And we could confirm the direction of association because they had the sign of their values.

• The Journal of Korean Institute of Communications and Information Sciences
• /
• v.30 no.9C
• /
• pp.854-859
• /
• 2005

We construct the fuzzy entropy for measuring of uncertainty with the help of relation between distance measure and similarity measure. Proposed fuzzy entropy is constructed through distance measure. In this study, the distance measure is used Hamming distance measure. Also for the measure of similarity between fuzzy sets or crisp sets, we construct similarity measure through distance measure, and the proposed 려zzy entropies and similarity measures are proved.

• Journal of the Korean Data and Information Science Society
• /
• v.24 no.1
• /
• pp.117-124
• /
• 2013

Association rule of data mining techniques is the method to quantify the relationship between a set of items in a huge database, andhas been applied in various fields like internet shopping mall, healthcare, insurance, and education. There are three primary interestingness measures for association rule, support and confidence and lift. Confidence is the most important measure of these measures, and we generate some association rules using confidence. But it is an asymmetric measure and has only positive value. So we can face with difficult problems in generation of association rules. In this paper we apply the similarity measures by probabilistic interestingness measure (PIM) with all marginal proportions (AMP) to solve this problem. The comparative studies with support, confidences, lift, chi-square statistics, and some similarity measures by PIM with AMPare shown by numerical example. As the result, we knew that the similarity measures by PIM with AMP could be seen the degree of association same as confidence. And we could confirm the direction of association because they had the sign of their values, and select the best similarity measure by PIM with AMP.

• Journal of KIISE:Databases
• /
• v.37 no.2
• /
• pp.71-81
• /
• 2010

The problem of finding clusters is widely used in numerous applications, such as pattern recognition, image analysis, market analysis. The important factors that decide cluster quality are the similarity measure and the number of attributes. Similarity measures should be defined with respect to the data types. Existing similarity measures are well applicable to numerical attribute values. However, those measures do not work well when the data is described by categorical attributes, that is, when no inherent similarity measure between values. In high dimensional spaces, conventional clustering algorithms tend to break down because of sparsity of data points. To overcome this difficulty, a subspace clustering approach has been proposed. It is based on the observation that different clusters may exist in different subspaces. In this paper, we propose a new similarity measure for clustering of high dimensional categorical data. The measure is defined based on the fact that a good clustering is one where each cluster should have certain information that can distinguish it with other clusters. We also try to capture on the attribute dependencies. This study is meaningful because there has been no method to use both of them. Experimental results on real datasets show clusters obtained by our proposed similarity measure are good enough with respect to clustering accuracy.