[KSCI] Korea Science Citation Index Service

http://dx.doi.org/10.9708/jksci.2020.25.09.011

Comparative Analysis of Container for High Performance Computing

Lee, Jaeryun (School of Computer Science and Engineering, Kyungpook National University)
Chae, Yunchang (School of Computer Science and Engineering, Kyungpook National University)
Tak, Byungchul (School of Computer Science and Engineering, Kyungpook National University)

Publication Information

Journal of the Korea Society of Computer and Information / v.25, no.9, 2020 , pp. 11-20 More about this Journal

Abstract

In this paper, we propose the possibility of using containers in the HPC ecosystem and the criteria for selecting a proper PMI library. Although demand for container has been growing rapidly in the HPC ecosystem, Docker container which is the most widely used has a potential security problem and is not suitable for the HPC. Therefore, several HPC containers have appeared to solve this problem and the chance of performance differences also emerged. For this reason, we measured the performance difference between each HPC container and Docker container through NAS Parallel Benchmark experiment and checked the effect of the type of PMI library. As a result, the HPC container and the Docker container showed almost the same performance as native, or in some cases, rather better performance was observed. In the result of comparison between PMI libraries showed that PMIx was not superior to PMI-2 in all conditions.

Keywords

Container; HPC; PMI; Singularity; Shifter; Charliecloud;

Citations & Related Records

Reference

1	M. de Bayser, and R. Cerqueira, "Integrating MPI with Docker for HPC", 2017 IEEE International Conference on Cloud Engineering (IC2E), p.p. 259-265, April 2017. DOI:10.1109/IC2E.2017.40
2	A. Azab, "Enabling Docker Containers for High-Performance and Many-Task Computing," 2017 IEEE International Conference on Cloud Engineering (IC2E), p.p. 279-285, April 2017. DOI:10.1109/IC2E.2017.52
3	J. Sparks, "Enabling Docker for HPC," Concurrency and computation, Vol. 31, No. 16, e5018, July 2018. DOI: 10.1002/cpe.5018 DOI
4	A.J. Younge, K. Pedretti, R.E. Grant and R. Brightwell, "A Tale of Two Systems Using Containers to Deploy HPC Applications on Supercomputers and Clouds," 2017 IEEE International Conference on Cloud Computing Technology and Science (CloudCom), p.p. 74-81, Dec. 2017. DOI: 10.1109/CloudCom.2017.40
5	O. Rudyy, M. Garcia-Gasulla, F. Mantovani, A. Santiago, R. Sirvent, and M. Vazquez, "Containers in hpc: A scalability and portability study in production biological simulations," 2019 IEEE International Parallel and Distributed Processing Symposium (IPDPS), p.p. 567-577, May 2019. DOI: 10.1109/IPDPS.2019.00066
6	A. Torrez, T. Randles and R. Priedhorsky, "HPC container runtimes have minimal or no performance impact," 2019 IEEE/ACM International Workshop on Containers and New Orchestration Paradigms for Isolated Environments in HPC (CANOPIE-HPC), pp.37-42, Nov. 2019 . DOI: 10.1109/CANOPIE-HPC49598.2019.00010
7	P. Balaji, D. Buntians, D. Goodell, W. Gropp, J. Krishna, E. Lusk and R. Thakur, "PMI: A scalable parallel process-management interface for extreme-scale systems," European MPI Users' Group Meeting, pp. 31-41, Nov. 2010. DOI: 10.1007/978-3-642-15646-5_4
8	D.H. Bailey, E. Barszcz, J.T. Barton, D.S. Browning, R.L. Carter, L. Dagum, R.A. Fatoohi, P.O. Frederickson, T.A. Lasinski, R.S. Schreiber, H.D. Simon, V. Venkatakrishnan and S.K. Weeratunga, "The nas parallel benchmarks," International Journal of High Performance Computing Applications, VOL. 5, No. 3, pp. 63-73, Sep. 1991.
9	R.H. Castain, J. Hursey, A. Bouteille and D. Solt , "PMIx: process management for exascale environments," Parallel Computing, Vol. 79, pp. 9-29, Nov. 2018. DOI: 10.1016/j.parco.2018.08.002 DOI
10	S. Chakraborty, H. Subramoni, J. Perkins and D.K. Panda, "SHMEMPMI -- Shared Memory Based PMI for Improved Performance and Scalability," 2016 16th IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing (CCGrid), pp.60-69, May. 2016 DOI: 10.1109/CCGrid.2016.99
11	S. Soltesz, H. Potzl, M.E. Fiuczynski A., Bavier, and L. Peterson, "Container-based operating system virtualization: A scalable, high-performance alternative to hypervisors," ACM SIGOPS Operating Systems Review, p.p. 275-287, Mar. 2007. DOI: 10.1145/1272996.1273025 DOI
12	M.G. Xavier, M.V. Neves, F.D. Rossi, T.C. Ferreto, T. Lange and C.A.F De Rose, "Performance Evaluation of Container-Based Virtualization for High Performance Computing Environments," 2013 21st Euromicro International Conference on Parallel, Distributed, and Network-Based Processing, p.p. 233-240, February 2013. DOI: 10.1109/PDP.2013.41
13	W. Felter, A. Ferreira, R. Rajamony and J. Rubio, "An updated performance comparison of virtual machines and linux containers," 2015 IEEE International Symposium on Performance Analysis of Systems and Software (ISPASS), p.p. 171-172, March 2015. DOI:10.1109/ISPASS.2015.7095802
14	L. Gerhardt, W. Bhimji, S. Cannon, M. Fasel, D. Jacobsen, M. Mustafa, J. Porter and V. Tsulaia, "Shifter: Containers for HPC," Journal of Physics: Conference Series, Vol. 898, No. 8, pp. 082021, Nov. 2017. DOI: 10.1088/1742-6596/898/8/082021 DOI
15	D.M. Jacobsen and R.S. Canon. "Contain this, unleashing docker for hpc," Proceedings of the Cray User Group, pp. 33-49, Apr. 2015.
16	G.M. Kurtzer, V. Sochat and M.W. Bauer. "Singularity: Scientific containers for mobility of compute," PloS one, Vol. 12, No. 5, e0177459, May. 2017. DOI: 10.1371/journal.pone.0177459 DOI
17	R. Priedhorsky and T. Randles, "Charliecloud: unprivileged containers for user-defined software stacks in HPC," Proceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis, SC 2017, pp. 1-10, Nov. 2017. DOI: 10.1145/3126908.3126925