• Title/Summary/Keyword: Container Orchestration

Search Result 12, Processing Time 0.016 seconds

Dynamic Resource Adjustment Operator Based on Autoscaling for Improving Distributed Training Job Performance on Kubernetes (쿠버네티스에서 분산 학습 작업 성능 향상을 위한 오토스케일링 기반 동적 자원 조정 오퍼레이터)

  • Jeong, Jinwon;Yu, Heonchang
    • KIPS Transactions on Computer and Communication Systems
    • /
    • v.11 no.7
    • /
    • pp.205-216
    • /
    • 2022
  • One of the many tools used for distributed deep learning training is Kubeflow, which runs on Kubernetes, a container orchestration tool. TensorFlow jobs can be managed using the existing operator provided by Kubeflow. However, when considering the distributed deep learning training jobs based on the parameter server architecture, the scheduling policy used by the existing operator does not consider the task affinity of the distributed training job and does not provide the ability to dynamically allocate or release resources. This can lead to long job completion time and low resource utilization rate. Therefore, in this paper we proposes a new operator that efficiently schedules distributed deep learning training jobs to minimize the job completion time and increase resource utilization rate. We implemented the new operator by modifying the existing operator and conducted experiments to evaluate its performance. The experiment results showed that our scheduling policy improved the average job completion time reduction rate of up to 84% and average CPU utilization increase rate of up to 92%.

IaC-VIMF: IaC-Based Virtual Infrastructure Mutagenesis Framework for Cyber Defense Training (IaC-VIMF: 사이버 공방훈련을 위한 IaC 기반 가상 인프라 변이 생성 프레임워크)

  • Joo-Young Roh;Se-Han Lee;Ki-Woong Park
    • Journal of the Korea Institute of Information Security & Cryptology
    • /
    • v.33 no.3
    • /
    • pp.527-535
    • /
    • 2023
  • To develop experts capable of responding to cyber security incidents, numerous institutions have established cyber training facilities to cultivate security professionals equipped with effective defense strategies. However, these challenges such as limited resources, scenario-based content development, and cost constraints. To address these issues, this paper proposes a virtual infrastructure variation generation framework. It provides customized, diverse IT infrastructure environments for each organization, allowing cyber defense trainers to accumulate a wide range of experiences. By leveraging Infrastructure-as-Code (IaC) containers and employing Word2Vec, a natural language processing model, mutable code elements are extracted and trained, enabling the generation of new code and presenting novel container environments.