Journal of Information Processing Systems

Korea Information Processing Society (한국정보처리학회)
권호 표지
DOI QR코드

DOI QR Code

Proposal of Container-Based HPC Structures and Performance Analysis

  • Yong, Chanho (Dept. of Computer Science and Engineering, Kyung Hee University) ;
  • Lee, Ga-Won (Dept. of Computer Science and Engineering, Kyung Hee University) ;
  • Huh, Eui-Nam (Dept. of Computer Science and Engineering, Kyung Hee University)
  • Received : 2018.05.28
  • Accepted : 2018.07.29
  • Published : 2018.12.31
Download PDF
⟨ Previous Next ⟩

Abstract

High-performance computing (HPC) provides to researchers a powerful ability to resolve problems with intensive computations, such as those in the math and medical fields. When an HPC platform is provided as a service, users may suffer from unexpected obstacles in developing and running applications due to restricted development environments and dependencies. In this context, operating system level virtualization can be a solution for HPC service to ensure lightweight virtualization and consistency in Dev-Ops environments. Therefore, this paper proposes three types of typical HPC structure for container environments built with HPC container and Docker. The three structures focus on smooth integration with existing HPC job framework, message passing interface (MPI). Lastly, the performance of the structures is analyzed with High Performance Linpack benchmark from the aspect of performance degradation in network communications under Docker.

Keywords

E1JBB0_2018_v14n6_1398_f0001.png 이미지

Fig. 1. HPC platform structure. (a) HPC container integration with existing HPC platform. (b) Docker case 1: container as MPI process of each job. (c) Docker case 2: container as HPC worker node.

E1JBB0_2018_v14n6_1398_f0002.png 이미지

Fig. 2. Performance test with HPL benchmark.

References

  1. D. M. Jacobsen and R. S. Canon, "Contain this, unleashing Docker for HPC," in Proceedings of the Cray User Group, Chicago, IL, 2015.
  2. M. de Bayser and R. Cerqueira, "Integrating MPI with Docker for HPC," in Proceedings of 2017 IEEE International Conference on Cloud Engineering (IC2E), 2017, pp. 259-265.
  3. J. Sparks, "HPC containers in use," in Proceedings of the Cray User Group, Redmond, WA, 2017.
  4. J. H. Huh and K. Seo, "Design and test bed experiments of server operation system using virtualization technology," Human-centric Computing and Information Sciences, vol. 6, article no. 1, 2016.
  5. L. Benedicic, F. A. Cruz, A. Madonna, and K. Mariotti, "Portable, high-performance containers for HPC," 2017 [Online]. Available: https://arxiv.org/abs/1704.03383.
  6. J. Higgins, V. Holmes, and C. Venters, "Orchestrating Docker containers in the HPC environment," in High Performance Computing. Cham: Springer, 2015, pp. 506-513.
  7. G. M. Kurtzer, V. Sochat, and M. W. Bauer, "Singularity: scientific containers for mobility of compute," PloS One, vol. 12, no. 5, article no. e0177459, 2017.
  8. J. P. Martin, A. Kandasamy, and K. Chandrasekaran, "Exploring the support for high performance applications in the container runtime environment," Human-centric Computing and Information Sciences, vol. 8, article no. 1, 2018.
  9. N. Nguyen and D. Bein, "Distributed MPI cluster with Docker swarm mode," in Proceedings of 2017 IEEE 7th Annual Computing and Communication Workshop and Conference (CCWC), Las Vegas, NV, 2017, pp. 1-7.
  10. B. Gerofi, R. Riesen, R. W. Wisniewski, and Y. Ishikawa, "Toward full specialization of the HPC software stack: reconciling application containers and lightweight multi-kernels," in Proceedings of the 7th International Workshop on Runtime and Operating Systems for Supercomputers (ROSS), Washington, DC, 2017.