• Title/Summary/Keyword: Docker container

Search Result 38, Processing Time 0.024 seconds

Empirical Performance Evaluation of Communication Libraries for Multi-GPU based Distributed Deep Learning in a Container Environment

  • Choi, HyeonSeong;Kim, Youngrang;Lee, Jaehwan;Kim, Yoonhee
    • KSII Transactions on Internet and Information Systems (TIIS)
    • /
    • v.15 no.3
    • /
    • pp.911-931
    • /
    • 2021
  • Recently, most cloud services use Docker container environment to provide their services. However, there are no researches to evaluate the performance of communication libraries for multi-GPU based distributed deep learning in a Docker container environment. In this paper, we propose an efficient communication architecture for multi-GPU based deep learning in a Docker container environment by evaluating the performances of various communication libraries. We compare the performances of the parameter server architecture and the All-reduce architecture, which are typical distributed deep learning architectures. Further, we analyze the performances of two separate multi-GPU resource allocation policies - allocating a single GPU to each Docker container and allocating multiple GPUs to each Docker container. We also experiment with the scalability of collective communication by increasing the number of GPUs from one to four. Through experiments, we compare OpenMPI and MPICH, which are representative open source MPI libraries, and NCCL, which is NVIDIA's collective communication library for the multi-GPU setting. In the parameter server architecture, we show that using CUDA-aware OpenMPI with multi-GPU per Docker container environment reduces communication latency by up to 75%. Also, we show that using NCCL in All-reduce architecture reduces communication latency by up to 93% compared to other libraries.

Implement HA using Container (컨테이너를 이용한 HA 구현)

  • Kim, Dong-hwan;Gwon, Ji-Seop;Han, Sung-Hwa
    • Proceedings of the Korean Institute of Information and Commucation Sciences Conference
    • /
    • 2022.10a
    • /
    • pp.341-343
    • /
    • 2022
  • Information service must ensure stability in service operation. For this purpose, many technologies related to HA (High-Availability) have been provided. Meanwhile, recently, a lot of information service implementations using containers are in progress. Therefore, it should be possible to implement HA in information service using containers. Docker, the most representative container platform, provides HA-related functions. In this study, we check how to implement HA using Docker and check the necessary settings. And based on the confirmed settings, the HA service using the container is implemented.

  • PDF

Comparative Analysis of Container for High Performance Computing

  • Lee, Jaeryun;Chae, Yunchang;Tak, Byungchul
    • Journal of the Korea Society of Computer and Information
    • /
    • v.25 no.9
    • /
    • pp.11-20
    • /
    • 2020
  • In this paper, we propose the possibility of using containers in the HPC ecosystem and the criteria for selecting a proper PMI library. Although demand for container has been growing rapidly in the HPC ecosystem, Docker container which is the most widely used has a potential security problem and is not suitable for the HPC. Therefore, several HPC containers have appeared to solve this problem and the chance of performance differences also emerged. For this reason, we measured the performance difference between each HPC container and Docker container through NAS Parallel Benchmark experiment and checked the effect of the type of PMI library. As a result, the HPC container and the Docker container showed almost the same performance as native, or in some cases, rather better performance was observed. In the result of comparison between PMI libraries showed that PMIx was not superior to PMI-2 in all conditions.

Design of Testbed for Agile Computing of MapReduce Applications by using Docker

  • Kang, Yunhee
    • International Journal of Contents
    • /
    • v.12 no.3
    • /
    • pp.29-33
    • /
    • 2016
  • Cloud computing makes extensive use of virtual machines that permit for workloads, as well as resource usage, to be isolated from one another, and a hypervisor can be used by a virtual machine to construct cloud computing infrastructure. However, the hypervisor has high resource usage when constructing virtual machines, which results in a waste of allocated resources when not activated. Docker provides a more light-weight method to obtain agile computing resources based on a container technique that handles this problem. In this study, we have chosen this specific tool due to the increasing popularity of MapReduce and cloud container technologies such as Docker. This study aims to automatically configure Twister workloads for container-driven clouds. Basically, this is the first attempt towards automatic configuration of Twister jobs on a container-based cloud platform VM for many workloads.

Method of Digital Forensic Investigation of Docker-Based Host (도커 기반 호스트에 대한 디지털 포렌식 조사 기법)

  • Kim, Hyeon Seung;Lee, Sang Jin
    • KIPS Transactions on Computer and Communication Systems
    • /
    • v.6 no.2
    • /
    • pp.75-86
    • /
    • 2017
  • Docker, which is one of the various virtualization technology in server systems, is getting popular as it provides more lightweight environment for service operation than existing virtualization technology. It supports easy way of establishment, update, and migration of server environment with the help of image and container concept. As the adoption of docker technology increases, the attack motive for the server for the distribution of docker images and the incident case of attacking docker-based hosts would also increase. Therefore, the method and procedure of digital forensic investigation of docker-based host including the way to extract the filesystem of containers when docker daemon is inactive are presented in this paper.

A Study on Integrity Protection of Edge Computing Application Based on Container Technology (컨테이너 기술을 활용한 엣지 컴퓨팅 환경 어플리케이션 무결성 보호에 대한 연구)

  • Lee, Changhoon;Shin, Youngjoo
    • Journal of the Korea Institute of Information Security & Cryptology
    • /
    • v.31 no.6
    • /
    • pp.1205-1214
    • /
    • 2021
  • Edge Computing is used as a solution to the cost problem and transmission delay problem caused by network bandwidth consumption that occurs when IoT/CPS devices are integrated into the cloud by performing artificial intelligence (AI) in an environment close to the data source. Since edge computing runs on devices that provide high-performance computation and network connectivity located in the real world, it is necessary to consider application integrity so that it is not exploited by cyber terrorism that can cause human and material damage. In this paper, we propose a technique to protect the integrity of edge computing applications implemented in a script language that is vulnerable to tampering, such as Python, which is used for implementing artificial intelligence, as container images and then digitally signed. The proposed method is based on the integrity protection technology (Docker Contents Trust) provided by the open source container technology. The Docker Client was modified and used to utilize the whitelist for container signature information so that only containers allowed on edge computing devices can be operated.

Comparative Analysis of the Virtual Machine and Containers Methods through the Web Server Configuration (웹 서버 구성을 통한 가상머신과 컨테이너 방식 비교 분석)

  • Bae, Yu-Mi;Jung, Sung-Jae;Soh, Woo-Young
    • Journal of the Korea Institute of Information and Communication Engineering
    • /
    • v.18 no.11
    • /
    • pp.2670-2677
    • /
    • 2014
  • The technique of virtual machine construction using hypervisor such as Xen and KVM is mainly used for implementation of cloud computing infrastructure. This technique is efficient in allocating and managing resources compared to the existing operation methods. However it requires high resource usage when constructing virtual machines and results in wasting of resources when not using the allocated resources. Docker is a technique based on the container method to resolve such a problem. This paper shows the container method such as Docker is efficient as a web construction technique by comparing virtual machine method to container method. It is shown to be especially useful when storing data into DB or storage devices in such environments of web server or program development. In the upcoming cloud computing environment the container method such as Docker is expected to improve the resource efficiency and the convenience of management.

Database PasS web service system using Docker

  • Hur, Tai-Sung
    • Journal of the Korea Society of Computer and Information
    • /
    • v.25 no.11
    • /
    • pp.51-58
    • /
    • 2020
  • Most of the students in computer-related departments work on projects, and it is essential to use a database for project execution. To use such a database, it is necessary to install a Database Management System. However, it takes several minutes (hours) to install a DBMS, and some DBMS require a difficult installation process. Therefore, in order to solve this problem, this study proposed a system that can easily install DBMS using Docker. Docker is an open source project that automates the deployment of Linux applications into software containers. Docker Container is wrapped in a complete file system that includes everything necessary for the execution of software, and includes code, runtime, system tools, system libraries, and anything that is installed on the server. This guarantees that they will always run the same regardless of the environment in which they are running. After creating a database using this proposed system, you can check the database access information on the web and check the server status in minutes. As a result of of implementing this proposed system and applying it to the projects of 10 teams, the installation time was reduced by 94.5% for Maria DBMS and 98.3% for Oracle DBMS than individual installation, confirming improved efficiency.

Implementation of Opensource-Based Automatic Monitoring Service Deployment and Image Integrity Checkers for Cloud-Native Environment (클라우드 네이티브 환경을 위한 오픈소스 기반 모니터링 서비스 간편 배포 및 이미지 서명 검사기 구현)

  • Gwak, Songi;Nguyen-Vu, Long;Jung, Souhwan
    • Journal of the Korea Institute of Information Security & Cryptology
    • /
    • v.32 no.4
    • /
    • pp.637-645
    • /
    • 2022
  • Cloud computing has been gaining popularity over decades, and container, a technology that is primarily used in cloud native applications, is also drawing attention. Although container technologies are lighter and more capable than conventional VMs, there are several security threats, such as sharing kernels with host systems or uploading/downloading images from the image registry. one of which can refer to the integrity of container images. In addition, runtime security while the container application is running is very important, and monitoring the behavior of the container application at runtime can help detect abnormal behavior occurring in the container. Therefore, in this paper, first, we implement a signing checker that automatically checks the signature of an image based on the existing Docker Content Trust (DCT) technology to ensure the integrity of the container image. Next, based on falco, an open source project of Cloud Native Computing Foundation(CNCF), we introduce newly created image for the convenience of existing falco image, and propose implementation of docker-compose and package configuration that easily builds a monitoring system.

Proposal of Container-Based HPC Structures and Performance Analysis

  • Yong, Chanho;Lee, Ga-Won;Huh, Eui-Nam
    • Journal of Information Processing Systems
    • /
    • v.14 no.6
    • /
    • pp.1398-1404
    • /
    • 2018
  • High-performance computing (HPC) provides to researchers a powerful ability to resolve problems with intensive computations, such as those in the math and medical fields. When an HPC platform is provided as a service, users may suffer from unexpected obstacles in developing and running applications due to restricted development environments and dependencies. In this context, operating system level virtualization can be a solution for HPC service to ensure lightweight virtualization and consistency in Dev-Ops environments. Therefore, this paper proposes three types of typical HPC structure for container environments built with HPC container and Docker. The three structures focus on smooth integration with existing HPC job framework, message passing interface (MPI). Lastly, the performance of the structures is analyzed with High Performance Linpack benchmark from the aspect of performance degradation in network communications under Docker.