• 제목/요약/키워드: Distributed data mining

검색결과 110건 처리시간 0.023초

Students' Performance Prediction in Higher Education Using Multi-Agent Framework Based Distributed Data Mining Approach: A Review

  • M.Nazir;A.Noraziah;M.Rahmah
    • International Journal of Computer Science & Network Security
    • /
    • 제23권10호
    • /
    • pp.135-146
    • /
    • 2023
  • An effective educational program warrants the inclusion of an innovative construction which enhances the higher education efficacy in such a way that accelerates the achievement of desired results and reduces the risk of failures. Educational Decision Support System (EDSS) has currently been a hot topic in educational systems, facilitating the pupil result monitoring and evaluation to be performed during their development. Insufficient information systems encounter trouble and hurdles in making the sufficient advantage from EDSS owing to the deficit of accuracy, incorrect analysis study of the characteristic, and inadequate database. DMTs (Data Mining Techniques) provide helpful tools in finding the models or forms of data and are extremely useful in the decision-making process. Several researchers have participated in the research involving distributed data mining with multi-agent technology. The rapid growth of network technology and IT use has led to the widespread use of distributed databases. This article explains the available data mining technology and the distributed data mining system framework. Distributed Data Mining approach is utilized for this work so that a classifier capable of predicting the success of students in the economic domain can be constructed. This research also discusses the Intelligent Knowledge Base Distributed Data Mining framework to assess the performance of the students through a mid-term exam and final-term exam employing Multi-agent system-based educational mining techniques. Using single and ensemble-based classifiers, this study intends to investigate the factors that influence student performance in higher education and construct a classification model that can predict academic achievement. We also discussed the importance of multi-agent systems and comparative machine learning approaches in EDSS development.

Distributed Incremental Approximate Frequent Itemset Mining Using MapReduce

  • Mohsin Shaikh;Irfan Ali Tunio;Syed Muhammad Shehram Shah;Fareesa Khan Sohu;Abdul Aziz;Ahmad Ali
    • International Journal of Computer Science & Network Security
    • /
    • 제23권5호
    • /
    • pp.207-211
    • /
    • 2023
  • Traditional methods for datamining typically assume that the data is small, centralized, memory resident and static. But this assumption is no longer acceptable, because datasets are growing very fast hence becoming huge from time to time. There is fast growing need to manage data with efficient mining algorithms. In such a scenario it is inevitable to carry out data mining in a distributed environment and Frequent Itemset Mining (FIM) is no exception. Thus, the need of an efficient incremental mining algorithm arises. We propose the Distributed Incremental Approximate Frequent Itemset Mining (DIAFIM) which is an incremental FIM algorithm and works on the distributed parallel MapReduce environment. The key contribution of this research is devising an incremental mining algorithm that works on the distributed parallel MapReduce environment.

분산형 데이터마이닝 구현을 위한 의사결정나무 모델 전송 기술 (The Transfer Technique among Decision Tree Models for Distributed Data Mining)

  • 김충곤;우정근;백성욱
    • 디지털콘텐츠학회 논문지
    • /
    • 제8권3호
    • /
    • pp.309-314
    • /
    • 2007
  • 분산형 데이터마이닝을 위해 의사결정나무 알고리즘은 분산형 협업 환경에 적합하도록 변환되어야 한다. 본 논문에서 제시된 분산형 데이터마이닝 시스템은 각각의 사이트에서 부분적인 데이터를 위한 데이터마이닝 작업을 수행할 수 있는 에이전트와 여러 에이전트들의 협업을 통해 최종적인 의사결정나무 모델을 완성할 수 있도록 에이전트들 간의 통신을 중재하는 미디에이터로 구성되어 있다. 분산형 데이터마이닝의 장점 중에 하나는 여러 사이트에 분산되어 있는 대량의 데이터를 분산 처리하므로 데이터마이닝의 소요시간을 현저하게 줄일 수 있다는 점이다. 그러나 각 사이트들에 존재하고 있는 에이전트들 간의 통신에 부하가 과도하게 걸린다면, 효율적인 시스템으로의 활용도가 낮아질 것 이다. 본 논문은 에이전트들 간에 의사결정나무 모델의 전송량을 최소로 할 수 있는 방법론에 초점을 맞추었다.

  • PDF

분산 FTP 서버의 ACE 기반 로그 마이닝 시스템 (Distributed FTP Server for Log Mining System on ACE)

  • 민수홍;조동섭
    • 대한전기학회:학술대회논문집
    • /
    • 대한전기학회 2002년도 합동 추계학술대회 논문집 정보 및 제어부문
    • /
    • pp.465-468
    • /
    • 2002
  • Today large corporations are constructing distributed server environment. Many corporations are respectively operating Web server, FTP server, Mail server and DB server on heterogeneous operation. However, there is the problem that a manager must manage each server individually. In this paper, we present distributed FTP server for log mining system on ACE. Proposed log mining system is based upon ACE (Adaptive Communication Environment) framework and data mining techniques. This system provides a united operation with distributed FTP server.

  • PDF

분산형 FP트리를 활용한 병렬 데이터 마이닝 (Parallel Data Mining with Distributed Frequent Pattern Trees)

  • 조두산;김동승
    • 대한전자공학회:학술대회논문집
    • /
    • 대한전자공학회 2003년도 하계종합학술대회 논문집 V
    • /
    • pp.2561-2564
    • /
    • 2003
  • Data mining is an effective method of the discovery of useful information such as rules and previously unknown patterns existing in large databases. The discovery of association rules is an important data mining problem. We have developed a new parallel mining called Distributed Frequent Pattern Tree (abbreviated by DFPT) algorithm on a distributed shared nothing parallel system to detect association rules. DFPT algorithm is devised for parallel execution of the FP-growth algorithm. It needs only two full disk data scanning of the database by eliminating the need for generating the candidate items. We have achieved good workload balancing throughout the mining process by distributing the work equally to all processors. We implemented the algorithm on a PC cluster system, and observed that the algorithm outperformed the Improved Count Distribution scheme.

  • PDF

분산된 데이터마이닝을 위한 프레임워크의 설계 및 구현 (Design and Implementation of a Distributed Data Mining Framework)

  • Kadel, Prakash;Choi, Ho-Jin
    • 한국정보과학회:학술대회논문집
    • /
    • 한국정보과학회 2007년도 한국컴퓨터종합학술대회논문집 Vol.34 No.1 (C)
    • /
    • pp.336-340
    • /
    • 2007
  • We envisage that grid computing environments allow us to implement distributed data mining services, that is, those applications which analyze large sets of geographically distributed databases and information using the computational power and resources of a grid environment. This paper describes an experimental framework towards such a distributed data mining approach, including design considerations and a prototype implementation. Based on the "Knowledge Grid" architecture suggested by Cannataro et al., we identify four major components - user node, broker node, data node, and computation node - and define their individual roles. For implementing the prototype, we have investigated methods for utilizing distributed resources within a grid computing environment, e.g., communication and coordination among the various resources available.

  • PDF

A New Model to Enhance Efficiency in Distributed Data Mining Using Mobile Agent

  • Bardab, Saeed Ngmaldin;Ahmed, Tarig Mohamed
    • International Journal of Computer Science & Network Security
    • /
    • 제21권3호
    • /
    • pp.275-286
    • /
    • 2021
  • As a result of the vast amount of data that is geographically found in different locations. Distributed data mining (DDM) has taken a center stage in data mining. The use of mobile agents to enhance efficiency in DDM has gained the attention of industries, commerce and academia because it offers serious suggestions on how to solve inherent problems associated with DDM. In this paper, a novel DDM model has been proposed by using a mobile agent to enhance efficiency. The main idea behind the model is to use the Naive Bayes algorithm to give the mobile agent the ability to learn, compare, get and store the results on it from each server which has different datasets and we found that the accuracy increased roughly by 0.9% which is our main target.

이기종 분산환경에서 데이터마이닝을 위한 데이터준비 시스템 구현 (Implementation of Data Preparation System for Data Mining on Heterogenious Distributed Environment)

  • 이상희;이원섭
    • 한국컴퓨터정보학회논문지
    • /
    • 제9권3호
    • /
    • pp.109-113
    • /
    • 2004
  • 본 논문에서는 데이터 마이닝을 위한 데이터 준비 과정에 대하여 기존의 데이터 마이닝 도구들의 효율성을 비교하고, 새로운 효율적인 데이터 준비 시스템 설계 기준을 제안하고자 한다. 지역 및 원격 데이터베이스 접근방법 이기종 컴퓨터간의 정보 교환을 기준으로 기존의 데이터마이닝 도구들의 기능을 비교하였다. 본 논문에서는 앤서트리, 클레멘타인, 엔터프라이즈 마이너, 웨카를 비교하였다. 또한, 본 논문에서는 분산 네트워크 상에서 데이터 마이닝을 위한 효율적인 데이터 준비 시스템을 위한 설계기준을 제안한다.

  • PDF

분산 환경에서 신경망을 응용한 데이터 서버 마이닝 (Data Server Mining applied Neural Networks in Distributed Environment)

  • 박민기;김귀태;이재완
    • 한국정보통신학회:학술대회논문집
    • /
    • 한국해양정보통신학회 2003년도 춘계종합학술대회
    • /
    • pp.473-476
    • /
    • 2003
  • 오늘날 인터넷은 하나의 거대한 분산 정보 서비스센터의 역할을 수행하며 여러 가지 많은 정보들과 이를 관리 운영하는 데이터 베이스 서버들은 분산된 네트워크 환경 속에서 광범위하게 존재하고 있다. 그러나 우리는 데이터 특성에 따라 입력 데이터를 처리할 서버를 결정하는데 여러 가지 어려움을 겪고 있다. 본 논문에서는 분산 환경 속에 존재하는 수많은 데이터들 가운데 신경망을 이용해 입력 데이터 패턴을 가장 효율적으로 처리할 수 있는 목적지 서버를 마이닝하는 기법과 이를 기반으로 한 지능적 데이터 마이닝 시스템 구조를 설계하였다. 그 결과로서 새로운 입력 데이터패턴이 신경망으로 구현된 동적 바인딩 방법에 따라 목적지 서버를 결정한 후 처리됨을 보였다. 이 기법은 데이터 웨어하우스, 통신 및 전력부하패턴 분석, 인구센서스 분석, 의료데이터 분석에 활용될 수 있다.

  • PDF

Pattern mining for large distributed dataset: A parallel approach (PMLDD)

  • Pal, Amrit;Kumar, Manish
    • KSII Transactions on Internet and Information Systems (TIIS)
    • /
    • 제12권11호
    • /
    • pp.5287-5303
    • /
    • 2018
  • Handling vast amount of data found in large transactional datasets is an obvious challenge for the conventional data mining algorithms. Addressing this challenge, our paper proposes a parallel approach for proper decomposition of mining problem into sub-problems in order to find frequent patterns from these datasets. The proposed, Pattern Mining for Large Distributed Dataset (PMLDD) approach, ensures minimum dependencies as well as minimum communications among sub-problems. It establishes a linear aggregation of the intermediate results so that it can be adapted to large-scale programming models like MapReduce. In this context, an algorithmic structure for MapReduce programming model is presented. PMLDD guarantees an efficient load balancing among the sub-problems by a specific selection criterion. Further, it optimizes the number of required iterations over the dataset for mining frequent patterns as compared to the existing approaches. Finally, we believe that our approach is scalable enough to handle larger datasets in terms of performance evaluation, and the result analysis justifies all these mentioned concerns.