DOI QR코드

DOI QR Code

Num Worker Tuner: An Automated Spawn Parameter Tuner for Multi-Processing DataLoaders

  • Synn, DoangJoo (Dept. of Electrical Engineering, Korea University) ;
  • Kim, JongKook (Dept. of Electrical Engineering, Korea University)
  • Published : 2021.11.04

Abstract

In training a deep learning model, it is crucial to tune various hyperparameters and gain speed and accuracy. While hyperparameters that mathematically induce convergence impact training speed, system parameters that affect host-to-device transfer are also crucial. Therefore, it is important to properly tune and select parameters that influence the data loader as a system parameter in overall time acceleration. We propose an automated framework called Num Worker Tuner (NWT) to address this problem. This method finds the appropriate number of multi-processing subprocesses through the search space and accelerates the learning through the number of subprocesses. Furthermore, this method allows memory efficiency and speed-up by tuning the system-dependent parameter, the number of multi-process spawns.

Keywords

Acknowledgement

This research was supported by Basic Science Research Program through the National Research Foundation of Korea(NRF) funded by the Ministry of Education(NRF-2016R1D1A1B04933156) This work was supported in part by the National Re- search Foundation of Korea (NRF) through the Basic Science Research Program funded by the Ministry of Education under Grant 2014R1A1A2059527, and in part by the Information Technology Research Center (ITRC), Ministry of Science and ICT (MSIT), South Korea, through a Support Program under Grant IITP-2020-2018-0-01433, supervised by the Institute for Infor- mation and Communications Technology Promotion (IITP)