• 한국정보처리학회:학술대회논문집
• /
• 한국정보처리학회 2008년도 추계학술발표대회
• /
• pp.275-278
• /
• 2008

데이터마이닝은 현재 매우 각광 받고 있는 분야다. 연관규칙탐사는 트랜잭션 데이터베이스에서 일정빈도 이상의 패턴을 찾아내는 작업을 말한다. 그중 빈발서브그래프패턴 마이닝은 최근 관심이 늘어나고 있으며, 그 활용도 또한 매우 높다. 그래프마이닝은 아이템셋마이닝보다 훨씬 더 많은 계산을 필요로 한다. 중복을 최소화 하는 방법이 필요하며, 그중 가장 좋은 성능을 보이는 GASTON 알고리즘을 분석한다.

• 디지털콘텐츠학회 논문지
• /
• 제18권8호
• /
• pp.1593-1601
• /
• 2017

빅데이터 플랫폼에서, 연관 룰 마이닝 응용프로그램은 여러 가치를 창출할 수 있다. 예를 들어, 농업 빅데이터 플랫폼에서 농가 소득을 높일 수 있는 농작물들을 농업인들에게 추천할 수 있다. 이 연관 룰 마이닝의 주요 절차는 빈발 아이템셋 마이닝으로, 이는 동시에 나타나는 아이템의 셋을 찾는 작업이다. Apriori를 비롯한 이전 연구에서는 대규모의 가능한 아이템 셋에 의한 메모리 오버로드의 이유로 만족할 만한 성능을 보일 수 없었다. 이를 개선하고자, 아이템 셋을 작은 크기로 분할하여 순차적으로 계산하도록 하는 SON 알고리즘이 제안되었다. 하지만, 단일 머신에서 SON 알고리즘을 돌릴 경우 많은 시간이 소요된다. 이 논문에서는 하둡기반의 빅데이터 플랫폼에서 SON 알고리즘 병렬처리 방식을 이용한 연관룰 탐색 기법을 소개한다. 연관 룰 마이닝을 위한 전처리, SON 알고리즘 기반 빈발 아이템셋 마이닝, 그리고 연관룰 검출 절차를 Hadoop기반의 빅데이터 플랫폼에 구현하였다. 실제 데이터를 활용한 실험을 통해 제안된 연관 룰 마이닝 기법은 Brute Force 기법의 성능을 압도하는 것을 확인하였다.

• 정보처리학회논문지D
• /
• 제13D권3호
• /
• pp.293-298
• /
• 2006

데이터베이스에는 많은 데이터들이 저장되어 있다. 무수히 많은 데이터들로부터 어떠한 정보를 얻기 위해서는 질의문을 사용하면 된다. 질의문을 통해 얻는 정보들은 기본적이고 단순한 정보들이다. 데이터 마이닝은 데이터베이스를 통해서 얻을 수 없는 정보를 얻게 해주는 기법이다. 데이터 마이닝 기법에는 여러 가지가 있지만 본 논문에서는 클러스터링과 연관규칙을 찾아내는 기법을 다룬다. 기존의 연관규칙 기법에서의 문제점을 보완하고 더 나은 규칙들을 찾아내기 위한 방법을 제시한다. 여기에 클러스터링 방법을 적용하게 되는데 기존의 거리기반이나 범주 기반 등의 클러스터링이 아닌 연관규칙에 적합한 클러스터링 기법을 제안하여 적용하게 된다. 각 클러스터의 연관규칙들을 찾게 되면 기존의 전체 데이터베이스에서 찾아진 연관규칙 뿐만 아니라 클러스터들의 특징이 될 규칙들도 찾을 수 있게 된다. 본 연구를 통해 대용량 데이터베이스의 많은 트랜잭션 접근을 줄이고 소집단의 연관성도 찾을 수 있다.

• Journal of Computing Science and Engineering
• /
• 제1권2호
• /
• pp.177-194
• /
• 2007

Index selection is one of the most important decisions to take in the physical design of relational data warehouses. Indices reduce significantly the cost of processing complex OLAP queries, but require storage cost and induce maintenance overhead. Two main types of indices are available: mono-attribute indices (e.g., B-tree, bitmap, hash, etc.) and multi-attribute indices (join indices, bitmap join indices). To optimize star join queries characterized by joins between a large fact table and multiple dimension tables and selections on dimension tables, bitmap join indices are well adapted. They require less storage cost due to their binary representation. However, selecting these indices is a difficult task due to the exponential number of candidate attributes to be indexed. Most of approaches for index selection follow two main steps: (1) pruning the search space (i.e., reducing the number of candidate attributes) and (2) selecting indices using the pruned search space. In this paper, we first propose a data mining driven approach to prune the search space of bitmap join index selection problem. As opposed to an existing our technique that only uses frequency of attributes in queries as a pruning metric, our technique uses not only frequencies, but also other parameters such as the size of dimension tables involved in the indexing process, size of each dimension tuple, and page size on disk. We then define a greedy algorithm to select bitmap join indices that minimize processing cost and verify storage constraint. Finally, in order to evaluate the efficiency of our approach, we compare it with some existing techniques.

• Asia pacific journal of information systems
• /
• 제18권2호
• /
• pp.95-114
• /
• 2008

Recent interests in data mining result from the expansion of the amount of business data and the growing business needs for extracting valuable knowledge from the data and then utilizing it for decision making process. In particular, recent advances in association rule mining techniques enable us to acquire knowledge concerning sales patterns among individual items from the voluminous transactional data. Certainly, one of the major purposes of association rule mining is to utilize acquired knowledge in providing marketing strategies such as cross-selling, sales promotion, and shelf-space allocation. In spite of the potential applicability of association rule mining, unfortunately, it is not often the case that the marketing mix acquired from data mining leads to the realized profit. The main difficulty of mining-based profit realization can be found in the fact that tremendous numbers of patterns are discovered by the association rule mining. Due to the many patterns, data mining experts should perform additional mining of the results of initial mining in order to extract only actionable and profitable knowledge, which exhausts much time and costs. In the literature, a number of interestingness measures have been devised for estimating discovered patterns. Most of the measures can be directly calculated from what is known as a contingency table, which summarizes the sales frequencies of exclusive items or itemsets. A contingency table can provide brief insights into the relationship between two or more itemsets of concern. However, it is important to note that some useful information concerning sales transactions may be lost when a contingency table is constructed. For instance, information regarding the size of each market basket(i.e., the number of items in each transaction) cannot be described in a contingency table. It is natural that a larger basket has a tendency to consist of more sales patterns. Therefore, if two itemsets are sold together in a very large basket, it can be expected that the basket contains two or more patterns and that the two itemsets belong to mutually different patterns. Therefore, we should classify frequent itemset into two categories, inter-pattern co-occurrence and intra-pattern co-occurrence, and investigate the effect of the market basket size on the two categories. This notion implies that any interestingness measures for association rules should consider not only the total frequency of target itemsets but also the size of each basket. There have been many attempts on analyzing various interestingness measures in the literature. Most of them have conducted qualitative comparison among various measures. The studies proposed desirable properties of interestingness measures and then surveyed how many properties are obeyed by each measure. However, relatively few attentions have been made on evaluating how well the patterns discovered by each measure are regarded to be valuable in the real world. In this paper, attempts are made to propose two notions regarding association rule measures. First, a quantitative criterion for estimating accuracy of association rule measures is presented. According to this criterion, a measure can be considered to be accurate if it assigns high scores to meaningful patterns that actually exist and low scores to arbitrary patterns that co-occur by coincidence. Next, complementary measures are presented to improve the accuracy of traditional association rule measures. By adopting the factor of market basket size, the devised measures attempt to discriminate the co-occurrence of itemsets in a small basket from another co-occurrence in a large basket. Intensive computer simulations under various workloads were performed in order to analyze the accuracy of various interestingness measures including traditional measures and the proposed measures.