Browse > Article
http://dx.doi.org/10.13067/JKIECS.2017.12.2.367

Delayed Block Replication Scheme of Hadoop Distributed File System for Flexible Management of Distributed Nodes  

Ryu, Woo-Seok (Dept. of Health Care Management, Catholic University of Pusan)
Publication Information
The Journal of the Korea institute of electronic communication sciences / v.12, no.2, 2017 , pp. 367-374 More about this Journal
Abstract
This paper discusses management problems of Hadoop distributed node, which is a platform for big data processing, and proposes a novel technique for enabling flexible node management of Hadoop Distributed File System. Hadoop cannot configure Hadoop cluster dynamically because it judges temporarily unavailable nodes as a failure. Delayed block replication scheme proposed in this paper delays the removal of unavailable node as much as possible so as to be easily rejoined. Experimental results show that the proposed scheme increases flexibility of node management with little impact on distributed processing performance when the cluster size changes.
Keywords
Hadoop; HDFS; Block Replication; Fault Tolerance;
Citations & Related Records
Times Cited By KSCI : 3  (Citation Analysis)
연도 인용수 순위
1 H. Yoon, "Development of Contents on the Marine Meteorology Service by Meteorology and Climate Big Data," J. of The Korea Institute of Electronic Communication Sciences, vol. 11, no. 2, 2016, pp. 125-138.   DOI
2 H. Chen, R. Chiang, and V. C. Storey, "Business intelligence and analytics: From big data to big impact," MIS Quarterly, vol. 36, no. 4, 2012, pp. 1165-1188.
3 C. Ryu, "Context Inference and Sensor Data Classification of Big Data Stream Environment," J. of The Korea Institute of Electronic Communication Sciences, vol. 9, no. 10, 2014, pp. 1079-1085.   DOI
4 W. Raghupathi and V. Raghupathi, "Big data analytics in healthcare: promise and potential," Health Information Science and Systems, vol. 2, no. 1, 2014, pp. 1-10.   DOI
5 J. Choi, "Utilization value of medical Big Data created in operation of medical information system," J. of The Korea Institute of Electronic Communication Sciences, vol. 10, no. 12, 2015, pp. 1403-1410.   DOI
6 K. Shvachko, H. Kuang, S. Radia, and R. Chansler, "The Hadoop Distributed File System," In Proc. IEEE Symp. on Mass Storage Systems and Technologies (MSST), NV, USA, May 2010, pp. 1-10.
7 D. Borthakur, J. Sarma, and J. Gray, "Apache Hadoop Goes Realtime at Facebook, " In Proc. the 2011 ACM SIGMOD Int. Conf. on Management of data, Athens, Greece, 2011, pp. 1071-1080.
8 W. Ryu, "Flexible management of data nodes for hadoop distributed file system," In Proc. Int. Conf. on Big Data, Small Data, Linked Data and Open Data (ALLDATA 2017), Venice, Italy, 2017.
9 T. White, "Hadoop: The definitive guide, 4th Edition," O'Reilly Media, Inc., 2015.