• Proceedings of the IEEK Conference
• /
• 2003.07c
• /
• pp.2561-2564
• /
• 2003

Data mining is an effective method of the discovery of useful information such as rules and previously unknown patterns existing in large databases. The discovery of association rules is an important data mining problem. We have developed a new parallel mining called Distributed Frequent Pattern Tree (abbreviated by DFPT) algorithm on a distributed shared nothing parallel system to detect association rules. DFPT algorithm is devised for parallel execution of the FP-growth algorithm. It needs only two full disk data scanning of the database by eliminating the need for generating the candidate items. We have achieved good workload balancing throughout the mining process by distributing the work equally to all processors. We implemented the algorithm on a PC cluster system, and observed that the algorithm outperformed the Improved Count Distribution scheme.

• Journal of Navigation and Port Research
• /
• v.44 no.5
• /
• pp.391-399
• /
• 2020

Every port is competing for attracting loyal customers from other ports to achieve more profits stably. This paper proposes a data-mining scheme to facilitate this process. For resolving the problem, the OD (Origination-Destination) data are gathered from the AIS (Automatic Identification System) data. The OD data are clustered according to the arrival dates and ports. The FP-growth algorithm is applied to mine the frequent patterns of ships arriving at ports. Maintaining a loyal customer list for port updates and accuracy is critical in establishing its usefulness. These lists are critical as they can be used to provide suggestions for new products and services to loyal customers. Finally, based on the frequent patterns of the ships and the mode of arrival times, a formula proposed in this paper to derive shipping companies' loyalty to ports was applied. The case of Kaohsiung port was shown as an example of our algorithm, and the OD data of ships in 2017-2018 were processed. Using the results of our algorithm, other rival ports, such as Shanghai or Busan, may attract customers no longer loyal to Kaohsiung ports in the last two years and attract them as new loyal customers.

• The KIPS Transactions:PartD
• /
• v.18D no.3
• /
• pp.169-178
• /
• 2011

Recently, data sharing between enterprises or organizations is required matter for task cooperation. In this process, when the enterprise opens its database to the affiliates, it can be occurred to problem leaked sensitive information. To resolve this problem it is needed to hide sensitive information from the database. Previous research hiding sensitive information applied different heuristic algorithms to maintain quality of the database. But there have been few studies analyzing the effects on the items modified during the hiding process and trying to minimize the hided items. This paper suggests eFP-Tree(Extended Frequent Pattern Tree) based FP-Tree(Frequent Pattern Tree) to hide sensitive frequent itemsets. Node formation of eFP-Tree uses border to minimize impacts of non sensitive frequent itemsets in hiding process, by organizing all transaction, sensitive and border information differently to before. As a result to apply eFP-Tree to the example transaction database, the lost items were less than 10%, proving it is more effective than the existing algorithm and maintain the quality of database to the optimal.

• Proceedings of the Korea Information Processing Society Conference
• /
• 2007.05a
• /
• pp.31-33
• /
• 2007

Devising an efficient one-pass frequent pattern mining algorithm has been an issue in data mining research in recent past. Pattern growth algorithms like FP-Growth which are found more efficient than candidate generation and test algorithms still require two database scans. Moreover, FP-growth approach requires rebuilding the base-tree while mining with different support counts. In this paper we propose an item-based tree, called I-Tree that not only efficiently mines frequent patterns with single database scan but also provides multiple mining scopes with multiple support thresholds. The 'build-once-mine-many' property of I-Tree allows it to construct the tree only once and perform mining operation several times with the variation of support count values.

• KIPS Transactions on Software and Data Engineering
• /
• v.5 no.12
• /
• pp.623-634
• /
• 2016

Most of existing frequent pattern mining methods address time efficiency and greatly rely on the primary memory. However, in the era of big data, the size of real-world databases to mined is exponentially increasing, and hence the primary memory is not sufficient enough to mine for frequent patterns from large real-world data sets. To solve this problem, there are some researches for frequent pattern mining method based on disk, but the processing time compared to the memory based methods took very time consuming. There are some researches to improve scalability of frequent pattern mining, but their processes are very time consuming compare to the memory based methods. In this paper, we present PPFP as a novel disk-based approach for mining frequent itemset from big data; and hence we reduced the main memory size bottleneck. PPFP algorithm is based on FP-growth method which is one of the most popular and efficient frequent pattern mining approaches. The mining with PPFP consists of two setps. (1) Constructing an IFP-tree: After construct FP-tree, we assign index number for each node in FP-tree with novel index numbering method, and then insert the indexed FP-tree (IFP-tree) into disk as IFP-table. (2) Mining frequent patterns with PPFP: Mine frequent patterns by expending patterns using stack based PUSH-POP method (PPFP method). Through this new approach, by using a very small amount of memory for recursive and time consuming operation in mining process, we improved the scalability and time efficiency of the frequent pattern mining. And the reported test results demonstrate them.

• The Journal of the Institute of Internet, Broadcasting and Communication
• /
• v.19 no.2
• /
• pp.127-133
• /
• 2019

Since frequent item extraction so far requires searching for patterns and traversal for the FP-Tree, it is more likely to store the mining data in a tree and thus CPU time is required for its searching. In order to overcome these drawbacks, in this paper, we provide each item with its location identification of transaction data without relying on conditional FP-Tree and convert transaction data into 2-dimensional position information look-up table, resulting in the facilitation of time and spatial accessibility. We propose an algorithm that considers the mapping scheme between the location of items and items that guarantees the linear time complexity. Experimental results show that the proposed method can reduce many execution time and memory usage based on the data set obtained from the FIMI repository website.

• Proceedings of the Korean Information Science Society Conference
• /
• 2002.10c
• /
• pp.259-261
• /
• 2002

데이터마이닝 분야에서 빈발항목집합 탐사에 관한 연구는 활발히 진행되어 왔지만 여전히 많은 메모리 공간과 시간을 필요로 한다. 특히 apriori 알고리즘에 기반한 방법들은 긴 패턴이 생성될수록 지수적으로 시간과 공간이 증가한다. 최근에 발표된 fp-growth는 일반적인 데이터 집합에서 우수한 성능을 보이나 희소 데이터 집합에서 효율적인 성능을 보여주지 못한다. 본 논문에서는 길이가 2인 빈발항목집합 L2에 기반한 L2-tree 구조를 제안한다. 또한 L2-tree에서 빈발항목집합을 탐사하는 L2-traverse 알고리즘을 제안한다. L2-tree는 L2를 기반으로 하기 때문에 L2가 상대적으로 적은 희소 데이터 집합 환경에서 적은 메모리 공간을 사용하게 된다. L2-traverse 알고리즘은 별도의 추출 데이터베이스를 생성하는 FP-growth와 달리 단순히 L2-tree를 오직 한번의 깊이 우선 탐사를 통해 빈발항목집합을 찾는다. 최적화 기법으로써 길이가 3인 빈발항목집합 L3가 되지 않는 L2 패턴들을 미리 제거하는 방법으로 C3-traverse 알고리즘을 제안하며 실험을 통해 기존 알고리즘과 비교 검증한다.

• Journal of Information Processing Systems
• /
• v.17 no.4
• /
• pp.675-689
• /
• 2021

Nowadays, cloud computing is being adopted for more organizations. However, since cloud computing has a virtualized, volatile, scalable and multi-tenancy distributed nature, it is challenging task to perform attack detection in the cloud following conventional processes. This work proposes a solution which aims to collect web server logs by using Flume and filter them through Spark Streaming in order to only consider suspicious data or data related to denial-of-service attacks and reduce the data that will be stored in Hadoop Distributed File System for posterior analysis with the frequent pattern (FP)-Growth algorithm. With the proposed system, we can address some of the difficulties in security for cloud environment, facilitating the data collection, reducing detection time and consequently enabling an almost real-time attack detection.

• The KIPS Transactions:PartD
• /
• v.14D no.1 s.111
• /
• pp.9-20
• /
• 2007

Groups of genes control the functioning of a cell by complex interactions. Such interactions of gene groups are tailed Gene Regulatory Networks(GRNs). Two previous data mining approaches, clustering and classification, have been used to analyze gene expression data. Though these mining tools are useful for determining membership of genes by homology, they don't identify the regulatory relationships among genes found in the same class of molecular actions. Furthermore, we need to understand the mechanism of how genes relate and how they regulate one another. In order to detect regulatory relationships among genes from time-series Microarray data, we propose a novel approach using frequent pattern mining and chain rules. In this approach, we propose a method for transforming gene expression data to make suitable for frequent pattern mining, and gene expression patterns we detected by applying the FP-growth algorithm. Next, we construct a gene regulatory network from frequent gene patterns using chain rules. Finally, we validate our proposed method through our experimental results, which are consistent with published results.

• Journal of KIISE:Databases
• /
• v.36 no.5
• /
• pp.341-351
• /
• 2009

Recently high utility pattern (HUP) mining is one of the most important research issuer in data mining since it can consider the different weight Haloes of items. However, existing mining algorithms suffer from the performance degradation because it cannot easily apply Apriori-principle for pattern mining. In this paper, we introduce new high utility pattern mining approach by using a prefix-tree as in FP-Growth algorithm. Our approach stores the weight value of each item into a node and utilizes them for pruning unnecessary patterns. We compare the performance characteristics of three different prefix-tree structures. By thorough experimentation, we also prove that our approach can give performance improvement to a degree.