• Journal of Internet Computing and Services
• /
• v.25 no.5
• /
• pp.41-51
• /
• 2024

This paper introduces a novel approach for managing and building customized container images in high-performance computing (HPC) environments, addressing the growing need for flexibility, scalability, and efficiency in computational workflows. Our contributions include the development and integration of a custom container image manager and builder within a container-based HPC infrastructure. This system enables users to effortlessly create, manage, and deploy personalized AI service platforms, significantly enhancing the user experience by reducing the time and effort required to configure essential packages and frameworks. The image manager we developed is capable of processing multiple user requests concurrently, distributing tasks efficiently to image builders operating on compute nodes. Meanwhile, the image builder is designed to handle queued tasks, generate customized container images based on active instances, and store these images in a private container registry, ensuring seamless access and reusability. We validated our system's effectiveness by implementing it on HPC cluster-based systems, including the Nurion supercomputer and the Neuron GPU system, demonstrating its scalability and interoperability in real-world environments. Additionally, we established an architecture and mechanism that ensures seamless integration with existing container-based supercomputing frameworks, underscoring our system's capability to optimize resource utilization and streamline the deployment of AI service platforms.

• KIPS Transactions on Computer and Communication Systems
• /
• v.6 no.10
• /
• pp.415-420
• /
• 2017

Virtualization technique of OS-level is a new paradigm for deploying applications, and is attracting attention as a technology to replace traditional virtualization technique, VM (Virtual Machine). Especially, docker containers are capable of distributing application images faster and more efficient than before by applying layered image structures and union mount point to existing linux container. These characteristics of containers can only be used in layered file systems that support snapshot functionality, so it is required to select appropriate layered file systems according to the characteristics of the containerized application. We examine the characteristics of representative layered file systems and conduct write performance evaluations of each layered file systems according to the operating principles of the layered file system, Allocate-on-Demand and Copy-up. We also suggest the method of determining a appropriate layered file system principle for unknown containerized application by learning block I/O usage history of each layered file system principles in artificial neural network. Finally we validate effectiveness of artificial neural network created from block I/O history of each layered file system principles.