Browse > Article
http://dx.doi.org/10.3745/KTCCS.2020.9.9.197

Deployment and Performance Analysis of Data Transfer Node Cluster for HPC Environment  

Hong, Wontaek (한국과학기술정보연구원)
An, Dosik (한국과학기술정보연구원 슈퍼컴퓨팅인프라센터)
Lee, Jaekook (한국과학기술정보연구원 슈퍼컴퓨팅인프라센터)
Moon, Jeonghoon (한국과학기술정보연구원)
Seok, Woojin (한국과학기술정보연구원)
Publication Information
KIPS Transactions on Computer and Communication Systems / v.9, no.9, 2020 , pp. 197-206 More about this Journal
Abstract
Collaborative research in science applications based on HPC service needs rapid transfers of massive data between research colleagues over wide area network. With regard to this requirement, researches on enhancing data transfer performance between major superfacilities in the U.S. have been conducted recently. In this paper, we deploy multiple data transfer nodes(DTNs) over high-speed science networks in order to move rapidly large amounts of data in the parallel filesystem of KISTI's Nurion supercomputer, and perform transfer experiments between endpoints with approximately 130ms round trip time. We have shown the results of transfer throughput in different size file sets and compared them. In addition, it has been confirmed that the DTN cluster with three nodes can provide about 1.8 and 2.7 times higher transfer throughput than a single node in two types of concurrency and parallelism settings.
Keywords
DTN Cluster; Wide Area Network; Data Transfer; Science DMZ; Parallel Filesystem;
Citations & Related Records
연도 인용수 순위
  • Reference
1 A. Khan, T. Kim, H. Byun, and Y. Kim, "SCISPACE: A scientific collaboration workspace for geo-distributed HPC data centers," International Journal of Future Generation Computer Systems, Vol.101, pp.398-409, 2019.   DOI
2 C. Laat, E. Radius, and S. Wallace, "The rationale of the current optical networking initiatives," International Journal of Future Generation Computer Systems, Vol.19, No.6, pp.999-1008, 2003.   DOI
3 E. Dart, L. Rotman, B. Tierney, M. Hester, and J. Zurawski, "The Science DMZ: A Network Design Pattern for Data-Intensive Science," Proceedings of IEEE/ACM Annual SuperComputing Conference (SC13), Denver, USA, Nov. 2013.
4 J. Crichigno, E. Bou-Harb, and N. Ghani, "A Comprehensive Tutorial on Science DMZ," IEEE Communications Surveys & Tutorials, Vol.21, No.2, pp.2041-2078, 2019.   DOI
5 Petascale DTN project [Internet], https://cs.lbl.gov /newsmedia/news/2017/esnets-petascale-dtn-project-speeds-up-data-transfers-between-leading-hpc-centers/.
6 E. Yildirim, E. Arslan, J. Kim, and T. Kosar, "Application-Level Optimization of Big Data Transfers through Pipelining, Parallelism and Concurrency," IEEE Transactions on Cloud Computing, Vol.4, No.1, pp.63-75, 2016.   DOI
7 Z. Liu, R. Kettimuthu, I. Foster, and N. Rao, "Cross-Geography Scientific Data Transferring Trends and Behavior," Proceedings of the 27th International Symposium on High-Performance Parallel and Distributed Computing, New York, USA, Jun. 2018.
8 Globus project [Internet], https://docs.globus.org/ globusconnect-server/.
9 ESnet DTNs [Internet], https://fasterdata.es.net/ performancetesting/DTNs/.
10 R. Kettimuthu, Z. Liu, D. Wheeler, I. Foster, K. Heitmann, and F. Cappello, "Transferring a Petabyte in a Day," International Journal of Future Generation Computer Systems, Vol.88, pp.191-198, 2018.   DOI
11 Z. Liu, R. Kettimuthu, I. Foster, and Y. Liu, "A Comprehensive Study of Wide Area Data Movement at a Scientific Computing Facility," Proceedings of IEEE International Conference on Distributed Computing Systems, Vienna, Austria, Jul. 2018.
12 Y. Liu, Z. Liu, R. Kettimuthu, N. Rao, Z. Chen, and I. Foster, "Data transfer between scientific facilities - bottleneck analysis, insights, and optimizations," Proceedings of the 19th IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing (CCGRID), Larnaca, Cyprus, May 2019.
13 L. Zhang, W. Wu, P. DeMar, and E. Pouyoul, "mdtmFTP and its evaluation on ESNET SDN testbed," International Journal of Future Generation Computer Systems, Vol.79, pp.199-204, 2018.   DOI
14 J. Bresnahan, M. Link, R. Kettimuthu, D. Fraser, and I. Foster, "GridFTP Pipelining," Proceedings of the TeraGrid2007 Conference, Madison, USA, Jun. 2007.