• Knowledge Management Research
• /
• v.19 no.1
• /
• pp.119-132
• /
• 2018

Many advanced products and services are emerging in the market thanks to data-based technologies such as Internet (IoT), Big Data, and AI. The construction of a system for data processing under the IoT network environment is not simple in configuration, and has a lot of restrictions due to a high cost for constructing a high performance server environment. Therefore, in this paper, we will design a development environment for large data analysis computing platform using open source with low cost and practicality. Therefore, this study intends to implement a big data processing system using Raspberry Pi, an ultra-small PC environment, and open source API. This big data processing system includes building a portable server system, building a web server for web mining, developing Python IDE classes for crawling, and developing R Libraries for NLP and visualization. Through this research, we will develop a web environment that can control real-time data collection and analysis of web media in a mobile environment and present it as a curriculum for non-IT specialists.

• Journal of Information Processing Systems
• /
• v.15 no.5
• /
• pp.1141-1155
• /
• 2019

Today, most approaches used in the recommendation system provide correct data prediction similar to the data that users need. The method that researchers are paying attention and apply as a model in the recommendation system is the communities' detection in the big social network. The outputted result of this approach is effective in improving the exactness. Therefore, in this paper, the personalized movie recommendation system that combines data mining for the k-clique method is proposed as the best exactness data to the users. The proposed approach was compared with the existing approaches like k-clique, collaborative filtering, and collaborative filtering using k-nearest neighbor. The outputted result guarantees that the proposed method gives significant exactness data compared to the existing approach. In the experiment, the MovieLens data were used as practice and test data.

• Journal of Ship and Ocean Technology
• /
• v.11 no.4
• /
• pp.21-28
• /
• 2007

Recently the importance of the utilization of engineering data is gradually increasing. Engineering data contains the experiences and know-how of experts. Data mining technique is useful to extract knowledge or information from the accumulated existing data. This paper deals with generating optimal polynomials using genetic programming (GP) as the module of Data Mining system. Low order Taylor series are used to approximate the polynomial easily as a nonlinear function to fit the accumulated data. The overfitting problem is unavoidable because in real applications, the size of learning samples is minimal. This problem can be handled with the extended data set and function node stabilization method. The Data Mining system for the ship design based on polynomial genetic programming is presented.

• KSII Transactions on Internet and Information Systems (TIIS)
• /
• v.11 no.10
• /
• pp.5023-5038
• /
• 2017

Big data is an emerging technology which deals with wide range of data sets with sizes beyond the ability to work with software tools which is commonly used for processing of data. When we consider a huge network, we have to process a large amount of network information generated, which consists of both normal and abnormal activity logs in large volume of multi-dimensional data. Intrusion Detection System (IDS) is required to monitor the network and to detect the malicious nodes and activities in the network. Massive amount of data makes it difficult to detect threats and attacks. Sequential Pattern mining may be used to identify the patterns of malicious activities which have been an emerging popular trend due to the consideration of quantities, profits and time orders of item. Here we propose a sequential pattern mining algorithm with fuzzy logic feature selection and fuzzy weighted support for huge volumes of network logs to be implemented in Apache Hadoop YARN, which solves the problem of speed and time constraints. Fuzzy logic feature selection selects important features from the feature set. Fuzzy weighted supports provide weights to the inputs and avoid multiple scans. In our simulation we use the attack log from NS-2 MANET environment and compare the proposed algorithm with the state-of-the-art sequential Pattern Mining algorithm, SPADE and Support Vector Machine with Hadoop environment.

• The KIPS Transactions:PartD
• /
• v.8D no.2
• /
• pp.117-124
• /
• 2001

A data mining system includes various data mining functions such as aggregation, association and classification, among others. To express these data mining function, a powerful data mining query language is needed. In addition, a graphic user interface(GUI) based on the data mining query language is needed for users. In addition, processing a data mining query targeted for a data warehouse, which is the appropriate data repository for decision making, is needed. In this paper, we first build a GUI to enable users to easily define data mining queries. We then propose a data mining query processing framework that can be used to process a data mining query targeted for a data warehouse. We also implement a schema generate a data warehouse schema that is needed to build a data warehouse. Lastly, we show the implementation details of a query processor that can process queries that discover association rules.

• Proceedings of the IEEK Conference
• /
• 2003.07c
• /
• pp.2561-2564
• /
• 2003

Data mining is an effective method of the discovery of useful information such as rules and previously unknown patterns existing in large databases. The discovery of association rules is an important data mining problem. We have developed a new parallel mining called Distributed Frequent Pattern Tree (abbreviated by DFPT) algorithm on a distributed shared nothing parallel system to detect association rules. DFPT algorithm is devised for parallel execution of the FP-growth algorithm. It needs only two full disk data scanning of the database by eliminating the need for generating the candidate items. We have achieved good workload balancing throughout the mining process by distributing the work equally to all processors. We implemented the algorithm on a PC cluster system, and observed that the algorithm outperformed the Improved Count Distribution scheme.

• Proceedings of the KSRS Conference
• /
• 2004.10a
• /
• pp.716-718
• /
• 2004

The false alarm data in intrusion detection systems are divided into false positive and false negative. The false positive makes bad effects on the performance of intrusion detection system. And the false negative makes bad effects on the efficiency of intrusion detection system. Recently, the most of works have been studied the data mining technique for analysis of alert data. However, the false alarm data not only increase data volume but also change patterns of alert data along the time line. Therefore, we need a tool that can analyze patterns that change characteristics when we look for new patterns. In this paper, we focus on the false positives and present a framework for analysis of false alarm pattern from the alert data. In this work, we also apply incremental data mining techniques to analyze patterns of false alarms among alert data that are incremental over the time. Finally, we achieved flexibility by using dynamic support threshold, because the volume of alert data as well as included false alarms increases irregular.

• Proceedings of the Korea Inteligent Information System Society Conference
• /
• 2002.11a
• /
• pp.203-212
• /
• 2002

This study examined the predictive power of data mining algorithms by comparing the performance of logistic regression and decision tree algorithm, called CHAID (Chi-squared Automatic Interaction Detection), On the contrary to the previous studies, decision tree performed better than logistic regression. We have also developed a CDSS (Clinical Decision Support System) with three modules (doctor, nurse, and patient) based on data warehouse architecture. Data warehouse collects and integrates relevant information from various databases from hospital information system (HIS ). This system can help improve decision making capability of doctors and improve accessibility of educational material for patients.

• Proceedings of the Korean Society of Precision Engineering Conference
• /
• 2006.05a
• /
• pp.611-612
• /
• 2006

Many data mining techniques have been proved useful in revealing important patterns from large data sets. Especially, data mining techniques play an important role in a customer data analysis in a financial industry and an electronic commerce. Also, there are many data mining related research papers in a semiconductor industry and an automotive industry. In addition, data mining techniques are applied to the bioinformatics area. To satisfy customers' various requirements, each industry should develop new processes with more accurate production criteria. Also, they spend more money to guarantee their products' quality. In this manner, we apply data mining techniques to the production-related data such as a test data, a field claim data, and POP (point of production) data in the automotive parts industry. Data collection and transformation techniques should be applied to enhance the analysis results. Also, we classify various types of manufacturing processes and proposed an analysis scheme according to the type of manufacturing process. As a result, we could find inter- or intra-process relationships and critical features to monitor the current status of the each process. Finally, it helps an industry to raise their profit and reduce their failure cost.

• Journal of Korean Society of Industrial and Systems Engineering
• /
• v.23 no.54
• /
• pp.27-36
• /
• 2000

The problem of discovering association rules is an emerging research area, whose goal is to extract significant patterns or interesting rules from large databases and several algorithms for mining association rules have been applied to item-oriented sales transaction databases. Data warehouses and OLAP engines are expected to be widely available. OLAP and data mining are complementary; both are important parts of exploiting data. Our study shows that data cube is an efficient structure for mining association rules. OLAP databases are expected to be a major platform for data mining in the future. In this paper, we present an efficient and effective algorithm for mining association rules using data cube. The algorithm can be applicable to enhance the power of competitiveness of business organizations by providing rapid decision support and efficient database marketing through customer segmentation.