• International Journal of Computer Science & Network Security
• /
• v.23 no.5
• /
• pp.207-211
• /
• 2023

Traditional methods for datamining typically assume that the data is small, centralized, memory resident and static. But this assumption is no longer acceptable, because datasets are growing very fast hence becoming huge from time to time. There is fast growing need to manage data with efficient mining algorithms. In such a scenario it is inevitable to carry out data mining in a distributed environment and Frequent Itemset Mining (FIM) is no exception. Thus, the need of an efficient incremental mining algorithm arises. We propose the Distributed Incremental Approximate Frequent Itemset Mining (DIAFIM) which is an incremental FIM algorithm and works on the distributed parallel MapReduce environment. The key contribution of this research is devising an incremental mining algorithm that works on the distributed parallel MapReduce environment.

• Proceedings of the Korean Society of Computer Information Conference
• /
• 2012.07a
• /
• pp.135-137
• /
• 2012

A existing recommedation system using association rules has the problem, which is suffered from inefficiency by reprocessing of the data which have already been processed in the incremental data environment in which new data are added persistently. We propose the recommendation technique using incremental frequent pattern mining based on RFM in e-commerce. The proposed can extract frequent items and create association rules using frequent patterns mining rapidly when new data are added persistently.

• Journal of Internet Computing and Services
• /
• v.21 no.2
• /
• pp.39-48
• /
• 2020

Data mining techniques have been suggested to find efficiently meaningful and useful information. Especially, in the big data environments, as data becomes accumulated in several applications, related pattern mining methods have been proposed. Recently, instead of analyzing not only static data stored already in files or databases, mining dynamic data incrementally generated in a real time is considered as more interesting research areas because these dynamic data can be only one time read. With this reason, researches of how these dynamic data are mined efficiently have been studied. Moreover, approaches of mining representative patterns such as maximal pattern mining have been proposed since a huge number of result patterns as mining results are generated. As another issue, to discover more meaningful patterns in real world, weights of items in weighted pattern mining have been used, In real situation, profits, costs, and so on of items can be utilized as weights. In this paper, we analyzed weighted maximal pattern mining approaches for data generated incrementally. Maximal representative pattern mining techniques, and incremental pattern mining methods. And then, the application scenarios for analyzing the required commodity patterns in infants are presented by applying weighting representative pattern mining. Furthermore, the performance of state-of-the-art algorithms have been evaluated. As a result, we show that incremental weighted maximal pattern mining technique has better performance than incremental weighted pattern mining and weighted maximal pattern mining.

• Proceedings of the KSRS Conference
• /
• 2004.10a
• /
• pp.716-718
• /
• 2004

The false alarm data in intrusion detection systems are divided into false positive and false negative. The false positive makes bad effects on the performance of intrusion detection system. And the false negative makes bad effects on the efficiency of intrusion detection system. Recently, the most of works have been studied the data mining technique for analysis of alert data. However, the false alarm data not only increase data volume but also change patterns of alert data along the time line. Therefore, we need a tool that can analyze patterns that change characteristics when we look for new patterns. In this paper, we focus on the false positives and present a framework for analysis of false alarm pattern from the alert data. In this work, we also apply incremental data mining techniques to analyze patterns of false alarms among alert data that are incremental over the time. Finally, we achieved flexibility by using dynamic support threshold, because the volume of alert data as well as included false alarms increases irregular.

• Spatial Information Research
• /
• v.19 no.2
• /
• pp.71-76
• /
• 2011

In this paper, for using emerging patterns to define and analyze the significant difference of safe and non-safe power load lines, and identify which line is potentially non-safe, we proposed an incremental TFP-tree algorithm for mining emerging patterns that can search efficiently within limitation of memory. Especially, the concept of pre-infrequent patterns pruning and use of two different minimum supports, made the algorithm possible to mine most emerging patterns and handle the problem of mining from incrementally increased, large size of data sets such as power consumption data.

• International Journal of Contents
• /
• v.15 no.3
• /
• pp.32-38
• /
• 2019

We usually suffer from difficulties in treating or managing Big Data generated from various digital media and/or sensors using traditional mining techniques. Additionally, there are many problems relative to the lack of memory and the burden of the learning curve, etc. in an increasing capacity of large volumes of text when new data are continuously accumulated because we ineffectively analyze total data including data previously analyzed and collected. In this paper, we propose a general-purpose classifier and its structure to solve these problems. We depart from the current feature-reduction methods and introduce a new scheme that only adopts changed elements when new features are partially accumulated in this free-style learning environment. The incremental learning module built from a gradually progressive formation learns only changed parts of data without any re-processing of current accumulations while traditional methods re-learn total data for every adding or changing of data. Additionally, users can freely merge new data with previous data throughout the resource management procedure whenever re-learning is needed. At the end of this paper, we confirm a good performance of this method in data processing based on the Big Data environment throughout an analysis because of its learning efficiency. Also, comparing this algorithm with those of NB and SVM, we can achieve an accuracy of approximately 95% in all three models. We expect that our method will be a viable substitute for high performance and accuracy relative to large computing systems for Big Data analysis using a PC cluster environment.

• Proceedings of the KSRS Conference
• /
• v.2
• /
• pp.635-638
• /
• 2006

Regression of conventional prediction techniques in data mining uses the model which is generated from the training step. This model is applied to new input data without any change. If this model is applied directly to time series, the rate of prediction accuracy will be decreased. This paper proposes an incremental regression for time series prediction like typhoon track prediction. This technique considers the characteristic of time series which may be changed over time. It is composed of two steps. The first step executes a fractional process for applying input data to the regression model. The second step updates the model by using its information as new data. Additionally, the model is maintained by only recent data in a queue. This approach has the following two advantages. It maintains the minimum information of the model by using a matrix, so space complexity is reduced. Moreover, it prevents the increment of error rate by updating the model over time. Accuracy rate of the proposed method is measured by RME(Relative Mean Error) and RMSE(Root Mean Square Error). The results of typhoon track prediction experiment are performed by the proposed technique IMLR(Incremental Multiple Linear Regression) is more efficient than those of MLR(Multiple Linear Regression) and SVR(Support Vector Regression).

• The KIPS Transactions:PartC
• /
• v.13C no.3 s.106
• /
• pp.295-302
• /
• 2006

An intrusion detection system writes a lot of alarms against attack behaviors in real time. These alarms contain not only actual attack alarms, but also false alarms that are mistakes made by the intrusion detection system. False alarms are the main reason that reduces the efficiency of the intrusion detection system, and we propose framework for false alarms analysis in the paper. Also, we apply an incremental data mining method for pattern analysis of false alarms increasing continuously. The framework consists of GUI, DB Manager, Alert Preprocessor, and False Alarm Analyzer. We analyze the false alarms increasingly through the experiment of the proposed framework and show that false alarms are reduced by applying the analyzed false alarm rules in the intrusion detection system.

• Proceedings of the KSRS Conference
• /
• 2003.11a
• /
• pp.597-599
• /
• 2003

Moving pattern is as a kind of sequential pattern, which can be extracted from the large volume of location history data. This sort of knowledge is very useful in supporting intelligence to the LBS or GIS. In this paper, we proposed the continuous moving pattern mining approach in LBS platform and LBS Miner. The location updates of moving objects affect the set of the rules maintained. In our approach, we use the validity thresholds that indicate the next time to invoke the incremental pattern mining. The mining system will play a major role in supporting the various LBS solutions.

• Journal of KIISE:Databases
• /
• v.37 no.5
• /
• pp.221-227
• /
• 2010

Web access sequence mining can discover the frequently accessed web pages pursued by users. Utility-based web access sequence mining handles non-binary occurrences of web pages and extracts more useful knowledge from web logs. However, the existing utility-based web access sequence mining approach considers web access sequences from the very beginning of web logs and therefore it is not suitable for mining data streams where the volume of data is huge and unbounded. At the same time, it cannot find the recent change of knowledge in data streams adaptively. The existing approach has many other limitations such as considering only forward references of web access sequences, suffers in the level-wise candidate generation-and-test methodology, needs several database scans, etc. In this paper, we propose a new approach for high utility web access sequence mining over data streams with a sliding window method. Our approach can not only handle large-scale data but also efficiently discover the recently generated information from data streams. Moreover, it can solve the other limitations of the existing algorithm over data streams. Extensive performance analyses show that our approach is very efficient and outperforms the existing algorithm.