• Journal of Software Engineering Society
• /
• 제25권4호
• /
• pp.63-67
• /
• 2012

According to the growth of data being generated in man fields, a distributed programming model MapReduce has been introduced to handle it. In this paper, we build two cluster system with Solaris and Linux environment on SUN Blade150 respectively and then to evaluate the performance of a MapReduce application running on MapReduce middleware Hadoop in terms of its average elapse time and standard deviation. As a result of this experiment, we show that the overall performance of the MapReduce application based on Hadoop is affected by the configuration of the cluster system.

• The Journal of the Korea Contents Association
• /
• 제10권10호
• /
• pp.78-85
• /
• 2010

Map/Reduce is the programing model which can implement the Cloud Computing recently has been noticed. The model operates an application program processing amount of data using a lot of computers. It is important to plan the mechanism of separating the data in proper size and distributing that to a cluster consisted of computing node in efficient for using the computing nodes very well. Besides that, planning a process of Map phases and Reduce phases also influences the performance of Map/Reduce. This paper suggests the effectively distributing scheme that separates a huge data and operates Map task in the considering the performance of computing node and network status. And we make the Reduce task can be processed quickly through the tuning the mechanism of Map and Reduce task operation. Using the two Map/Reduce sample application, we experimented the suggestion and we evaluate suggestion considered it in how impact the Map/Reduce performance.

• IEMEK Journal of Embedded Systems and Applications
• /
• 제9권2호
• /
• pp.109-115
• /
• 2014

Analyzing large scale data has become an important activity for many organizations. Since MapReduce is a promising tool for processing the massive data sets, there are increasing studies to evaluate the performance of various algorithms related to MapReduce. In this paper, we first develop a simulation framework that includes MapReduce workload model, data center model, and the model of data access pattern. Then we propose two algorithms that can reduce the energy consumption of MapReduce systems. Using the simulation framework, we evaluate the performance of the proposed algorithms under different application characteristics and configurations of data centers.

• Electronics and Telecommunications Trends
• /
• 제28권6호
• /
• pp.156-166
• /
• 2013

MapReduce는 다양한 형식의 대용량 데이터를 병렬 처리하는데 있어 효과적인 도구로 인식되고 있다. 특히 MapReduce의 오픈 소스 구현인 Hadoop은 여러 분야에서 널리 이용되고 있으며, 가장 대표적인 빅데이터 솔루션으로 현재까지 많은 주목을 받아오고 있다. 하지만, MapReduce는 그 구조적 특정으로 인한 이점과 함께 여러 제약과 단점들을 가진다. 이에 따라 MapReduce의 개선을 위한 많은 연구와 시스템 개량이 학계와 산업계에서 동시에 수행되어 왔다. 본고에서는 대용량 데이터 분석을 위한 MapReduce 프레임워크의 특성과 이를 개선하기 위한 최근의 연구 내용들을 소개한다. 또한 향후의 대용량 데이터 처리는 어떠한 모습을 취하게 될 것인지를 예측해 본다.

• Proceedings of the Korea Information Processing Society Conference
• /
• 한국정보처리학회 2014년도 춘계학술발표대회
• /
• pp.301-304
• /
• 2014

비정형 게임 Big Data에서 고효율 정보를 추출하고, 신뢰 할 수 있는 클러스터 게임서버 환경을 위한 병렬 처리를 위해 MapReduce를 사용한다. 본 논문에서는 빈번하게 입력되는 신규 게임데이터 처리를 위해 함수 Demap을 사용하는 Incremental MapReduce를 적용하여 불필요한 중간 값 저장과 재계산 없이 점차적으로 MapReduce 함수를 실행한다.

• KIPS Transactions on Software and Data Engineering
• /
• 제5권11호
• /
• pp.569-584
• /
• 2016

In this paper, we present a spatial knowledge extractor implemented in Hadoop MapReduce parallel, distributed computing environment. From a large spatial dataset, this knowledge extractor automatically derives a qualitative spatial knowledge base, which consists of both topological and directional relations on pairs of two spatial objects. By using R-tree index and range queries over a distributed spatial data file on HDFS, the MapReduce-enabled spatial knowledge extractor, MRSPAKE, can produce a web-scale spatial knowledge base in highly efficient way. In experiments with the well-known open spatial dataset, Open Street Map (OSM), the proposed web-scale spatial knowledge extractor, MRSPAKE, showed high performance and scalability.

• Journal of Digital Contents Society
• /
• 제14권3호
• /
• pp.313-321
• /
• 2013

In recent, proliferation of mobile smart devices have led to big-data era, the importance of location-based services is increasing due to the exponential growth of trajectory related data. In order to process trajectory data, parallel processing platforms such as cloud computing and MapReduce are necessary. Currently, the researches based on MapReduce are on progress, but due to the MapReduce's properties in using batch processing and simple key-value structure, applying MapReduce framework for real time LBS is difficult. Therefore, in this research we propose a suitable system design on efficient indexing and search techniques for real time service based on detailed analysis on the properties of MapReduce.

• International Journal of Internet, Broadcasting and Communication
• /
• 제4권1호
• /
• pp.13-17
• /
• 2012

Indexing allows converting raw document collection into easily searchable representation. Web searching by Google or Yahoo provides subsecond response time which is made possible by efficient indexing of web-pages over the entire Web. Indexing process gets challenging when the scale gets bigger. Parallel techniques, such as MapReduce framework can assist in efficient large-scale indexing process. In this paper we propose PDFindexer, system for indexing scientific papers in PDF using MapReduce programming model. Unlike Web search engines, our target domain is scientific papers, which has pre-defined structure, such as title, abstract, sections, references. Our proposed system enables parsing scientific papers in PDF recreating their structure and performing efficient distributed indexing with MapReduce framework in a cluster of nodes. We provide the overview of the system, their components and interactions among them. We discuss some issues related with the design of the system and usage of MapReduce in parsing and indexing of large document collection.

• JSTS:Journal of Semiconductor Technology and Science
• /
• 제15권5호
• /
• pp.511-518
• /
• 2015

As the cost-per-byte of SSDs dramatically decreases, the introduction of SSDs to Hadoop becomes an attractive choice for high performance data processing. In this paper the cost-per-performance of SSD-based Hadoop cluster (SSD-Hadoop) and HDD-based Hadoop cluster (HDD-Hadoop) are evaluated. For this, we propose a MapReduce performance model using queuing network to simulate the execution time of MapReduce job with varying cluster size. To achieve an accurate model, the execution time distribution of MapReduce job is carefully profiled. The developed model can precisely predict the execution time of MapReduce jobs with less than 7% difference for most cases. It is also found that SSD-Hadoop is 20% more cost efficient than HDD-Hadoop because SSD-Hadoop needs a smaller number of nodes than HDD-Hadoop to achieve a comparable performance, according to the results of simulation with varying the number of cluster nodes.

• Proceedings of the Korea Information Processing Society Conference
• /
• 한국정보처리학회 2013년도 추계학술발표대회
• /
• pp.1068-1069
• /
• 2013

최근 정보 기술과 웹의 발전으로 많은 응용에서 데이터의 양이 급격이 증가하였다. MapReduce는 이러한 대용량 데이터를 처리하기 위해 구글에서 제안한 프레임워크이다. MapReduce 프레임워크는 데이터 전달 패러다임을 기반으로 한다. 이로부터, 데이터 처리 및 질의에 있어 I/O 비용이 전체 처리 비용에서 큰 부분을 차지한다. 본 논문에서는 MapReduce 프레임워크에서 I/O에 소요되는 비용을 확인하기 위해, 실제 데이터를 기반으로 실험을 수행하였다. 이를 통해, MapReduce 기반 시스템의 성능 예측이나 성능 향상을 위해 고려되어야 할 부분을 제시하고자 하였다.