Browse > Article

An Analysis of Utilization on Virtualized Computing Resource for Hadoop and HBase based Big Data Processing Applications  

Cho, Nayun (Computer Engineering, Konkuk University)
Ku, Mino (Computer Engineering, Konkuk University)
Kim, Baul (Computer Engineering, Konkuk University)
Xuhua, Rui (Computer Engineering, Konkuk University)
Min, Dugki (Computer Engineering, Konkuk University)
Publication Information
Journal of Information Technology and Architecture / v.11, no.4, 2014 , pp. 449-462 More about this Journal
Abstract
In big data era, there are a number of considerable parts in processing systems for capturing, storing, and analyzing stored or streaming data. Unlike traditional data handling systems, a big data processing system needs to concern the characteristics (format, velocity, and volume) of being handled data in the system. In this situation, virtualized computing platform is an emerging platform for handling big data effectively, since virtualization technology enables to manage computing resources dynamically and elastically with minimum efforts. In this paper, we analyze resource utilization of virtualized computing resources to discover suitable deployment models in Apache Hadoop and HBase-based big data processing environment. Consequently, Task Tracker service shows high CPU utilization and high Disk I/O overhead during MapReduce phases. Moreover, HRegion service indicates high network resource consumption for transfer the traffic data from DataNode to Task Tracker. DataNode shows high memory resource utilization and Disk I/O overhead for reading stored data.
Keywords
Big Data; Cloud Computing; Hadoop; HBase;
Citations & Related Records
연도 인용수 순위
  • Reference
1 S. Madden, "From Databases to Big Data," IEEE Internet Computing, Vol. 16, No. 3, pp. 4-6, 2012.
2 Ku, M. and Ku, M., "Are We Ready for the Era of Big Data?," The 2014 Association for Public Policy Analysis and Management Conference (APPAM), Apr. 2014.
3 Kaisler, S., Armour, F., Espinosa, J. A., and Money, W., "Big Data: Issues and Challenges Moving Forward," 46th Hawaii International Conference on System Sciences (HICSS), Jan, 2013, pp. 995-1004.
4 Russom, P., "Big data analytics," TDWI Best Practices Report, 4th Quarter 2011.
5 Hseush, W., Yi-Cheng Huang, Shih-Chang Hsu, and Pu, C., "Real-time collaborative planning with big data: Technical challenges and in-place computing (invited paper)," 2013 9th International Conference on Collaborative Computing: Networking, Applications and Worksharing, Oct, 201, pp. 96- 104.
6 Big Data Analytics, Gartner, Jan. 2011.
7 V. R. Borkar, M. J. Carey, and C. Li, "Big data platforms: what's next?," ACM Crossroads, Vol. 19, No. 1, pp. 44-49, 2012.   DOI
8 R. E. Bryant, R. H. Katz, and E. D. Lazowska, "Bigdata Computing: Creating Revolutionary Breakthroughs in Commerce, Science, and society," Computing Research Initiatives for the 21st Century, Computing Research Association, 2008. pp. 1-15.
9 Apache Hadoop, http://hadoop.apache.org/.
10 Apache Hadoop, http://hbase.apache.org/.
11 Bakshi, K., "Considerations for big data: Architecture and approach," International Conference on Aerospace, IEEE Publisher, March, 2012, pp. 1-7.
12 R. Taylor, "An overview of the Hadoop/MapReduce/ HBase framework and its current applications in bioinformatics," BMC Bioinformatics 11(Suppl 12): S1, 2010.
13 Padhy R P., "Big Data Processing with Hadoop- MapReduce in Cloud Systems," International Journal of Cloud Computing and Services Science, Vol. 2, No. 1, pp. 16-27, 2012.
14 Jeongrae Kim and Chanki Jeong, "A Study on Phon Call Big Data Analytics," Journal of Information Technology and Architecture, Vol. 10, No. 3, pp. 387-397, 2013.
15 Suan Lee, Sunhwa Jo and Jinho Kim, "An Iterative Algorithm for the Bottom Up Computation of the Data Cube using MapReduce," Journal of Information Technology and Architecture, Vol. 9, No. 4, pp. 455-464, 2012.
16 D. Agrawal, S. Das, and A. E. Abbadi., "Big Data and Cloud Computing: New Wine or just New Bottles?," The Proceedings of the VLDB Endowment (PVLDB), Vol. 3, No. 2, pp. 1647-1648, 2010.
17 S. Tsuchiya, Y. Sakamoto, Y. Tsuchimoto, and V. Lee, "Big data processing in Cloud environments," FUJITSU SCIENTIFIC & TECHNICAL JOURNAL, Vol. 48, No. 2, pp. 159-168, 2012.
18 S. Chaudhuri, "What Next? A Half-Dozen Data Management Research Goals for Big Data and the Cloud," The 31st Symposium on Principles of Database Systems (PODS), ACM, 2012, pp. 1-4.
19 J. Schad, J. Dittrich, and J.-A. Quiane-Ruiz., "Runtime Measurements in the Cloud: Observing, Analyzing, and Reducing Variance," The Proceedings of the VLDB Endowment (PVLDB), Vol. 3, No. 1, 2010.
20 J. Varia. Architecting for the cloud: Best practices. Technical report, Amazon, 2011.
21 W. Lloyd, S. Pallickara, O. David, J. Lyon, M. Arabi, and K. Rojas, "Performance Implications of Multi-tier Application Deployments on Infrastructureas- a-service Clouds: Towards Performance Modeling," Future Generation Computer Systems, Vol. 29, No. 5, pp. 1254-1264, 2012.
22 J. Dean,, and S. Ghemawat, "MapReduce: Simplified data processing on large clusters," The 6th Symposium on Operating System Design and Implementation, 2004, pp. 10-10.
23 Amaznon Elastic MapReduce, http://aws.amazon. com/ko/elasticmapreduce/.
24 E. Feller, L. Ramakrishnan, and C. Morin, "On the performance and energy efficiency of Hadoop deployment models," International Conference on Big Data, Oct, 2013, pp. 131-136.
25 Mino Ku, Eunmi Choi, and Dugki Min, "An analysis of performance factors on Esper-based stream big data processing in a virtualized environment," International Journal of Communication Systems, Vol. 27, No. 6, pp. 898-917, 2014.   DOI   ScienceOn
26 TASAS, California Department of Transportation, http://www.dot.ca.gov/
27 PeMS, California Department of Transportation, http://pems.dot.ca.gov/
28 Yeo H, Jang K, Skabardonis A, Kang S., "Impact of traffic states on freeway crash involvement rates," Accident Analysis and Prevention, Elsevier, Vol. 50, Jan, 2013, pp. 713-723.   DOI   ScienceOn
29 White T, Hadoop: The Definitive Guide, O'Reilly Media, 2009.
30 Lars George, HBase: The Definitive Guide, O'Reilly Media, 2011.