Browse > Article

A Non-Shared Metadata Management Scheme for Large Distributed File Systems  

Yun, Jong-Byeon (충북대학교 정보통신공학과)
Park, Yang-Bun (충북대학교 정보통신공학과)
Lee, Seok-Jae (한국전자통신연구원 지식이러닝연구팀)
Jang, Su-Min (충북대학교 정보통신공학과)
Yoo, Jae-Soo (충북대학교 전기전자컴퓨터공학부)
Kim, Hong-Yeon (한국전자통신연구원 저장시스템연구팀)
Kim, Young-Kyun (한국전자통신연구원 저장시스템연구팀)
Publication Information
Journal of KIISE:Computer Systems and Theory / v.36, no.4, 2009 , pp. 259-273 More about this Journal
Abstract
Most of large-scale distributed file systems decouple a metadata operation from read and write operations for a file. In the distributed file systems, a certain server named a metadata server (MDS) maintains metadata information in file system such as access information for a file, the position of a file in the repository, the namespace of the file system, and so on. But, the existing systems used restrictive metadata management schemes, because most of the distributed file systems designed to focus on the distributed management and the input/output performance of data rather than the metadata. Therefore, in the existing systems, the metadata throughput and expandability of the metadata server are limited. In this paper, we propose a new non-shared metadata management scheme in order to provide the high metadata throughput and scalability for a cluster of MDSs. First, we derive a dictionary partitioning scheme as a new metadata distribution technique. Then, we present a load balancing technique based on the distribution technique. It is shown through various experiments that our scheme outperforms existing metadata management schemes in terms of scalability and load balancing.
Keywords
Distributed File System(DFS); Metadata Server(MDS); Non-Shared Metadata Management; Load Balancing;
Citations & Related Records
연도 인용수 순위
  • Reference
1 차명훈, 이상민, 김준, 김영균, 김명준, "대규모 분산 파일 시스템 환경의 메타 데이타 관리", 전자통신동향 분석, 제22권, 제3호, pp.154-165, 2007.
2 S.A. Brandt, L. Xue, E.L. Miller, and D.D.E. Long, "Efficient Metadata Management in Large Distributed File Systems," In Proc. of IEEE/11th NASA Goddard Conf. on Mass Storage Systems and Technologies, 2003.
3 Sage Weil, Kristal Pollack, Scott A. Brandt, and Ethan L. Miller, "Dynamic Metadata Management for Petabyte-Scale File Systems," In Proc. of the ACM/IEEE Conf. on Supercomputing, 2004.
4 장준호, 한세영, 박성용, "클러스터 파일시스템의 메타 데이타 서버를 위한 내용 기반 부하 분산 알고리즘", 정보처리학회논문지, 제13권, 제4호, pp.323-334, 2006.
5 FDTree, "https://computing.llnl.gov/?set=code&page=sio_downloads"
6 Sage Weil, Scott A. Brandt, Ethan L. Miller, Darrell D.E. Long, and Carlos Maltzahn, "Ceph : A Scalable, High-Performance Distributed File System," In Proc. of Conf. on Operating Systems Design and Implementation, pp.307-320, 2006.
7 Sanjay Ghemawat, Howard Gobioff, and Shun- Tak Leung, "The Google File System," In Proc. of ACM Symp. on Operating Systems Principles, pp.20-43, 2003.
8 D. Borthakur, "The Hadoop Distributed File System: Architecture and Design," 2005
9 Lustre, "Lustre : A Scalable High Performance File System," Cluster File System Inc., 2002.
10 D. Nagle, D. Serenyi, and A. Matthews, "The Panasas Active Scale Storage Cluster-Delivering Scalable High Bandwidth Storage," In Proc. of the ACM/IEEE SC2004 Conf., 2004.
11 민영수, 차명훈, 김영철, 진기성, 이상민, 정병권, 김준, "객체기반 저장 장치를 이용한 클러스터 파일시스템의 구현", 한국차세대컴퓨팅학회논문지, 제2권, 제4호, pp.42-52, 2006.
12 SPEC, "SFS 3.0 Documentation Version 1.0," Standard Performance Evaluation Corporation, 2001.
13 M. Mesnier, G. Ganger, and E. Riedel, "Object based Storage," In IEEE Communications Magazine, pp.84-90, 2003.
14 K. W. Preslan et al., "A 64bit, Shared Disk File System for Linux," Proceedings of the 16th IEEE Mass Storage Systems Symposium, pp.22-41, 1999.