Journal of the Korea Safety Management & Science (대한안전경영과학회지)

Korea Safety Management & Science (대한안전경영과학회)
권호 표지
DOI QR코드

DOI QR Code

Scheduling Algorithm, Based on Reinforcement Learning for Minimizing Total Tardiness in Unrelated Parallel Machines

이종 병렬설비에서 총납기지연 최소화를 위한 강화학습 기반 일정계획 알고리즘

  • Tehie Lee (Incheon National University Graduate School, Industrial & Management Engineering) ;
  • Jae-Gon Kim (Incheon National University, Dept. of Industrial & Management Engineering) ;
  • Woo-Sik Yoo (Incheon National University, Dept. of Industrial & Management Engineering)
  • 이태희 (인천대학교 산업경영공학과) ;
  • 김재곤 (인천대학교 산업경영공학과) ;
  • 유우식 (인천대학교 산업경영공학과)
  • Received : 2023.11.17
  • Accepted : 2023.12.18
  • Published : 2023.12.30
Download PDF
⟨ Previous Next ⟩

Abstract

This paper proposes an algorithm for the Unrelated Parallel Machine Scheduling Problem(UPMSP) without setup times, aiming to minimize total tardiness. As an NP-hard problem, the UPMSP is hard to get an optimal solution. Consequently, practical scenarios are solved by relying on operator's experiences or simple heuristic approaches. The proposed algorithm has adapted two methods: a policy network method, based on Transformer to compute the correlation between individual jobs and machines, and another method to train the network with a reinforcement learning algorithm based on the REINFORCE with Baseline algorithm. The proposed algorithm was evaluated on randomly generated problems and the results were compared with those obtained using CPLEX, as well as three scheduling algorithms. This paper confirms that the proposed algorithm outperforms the comparison algorithms, as evidenced by the test results.

Keywords

Acknowledgement

본 연구는 과학기술정보통신부 및 정보통신기획평가원의 학석사연계ICT핵심인재양성사업의 연구결과로 수행되었음(IITP-2023-RS-2023-00260175)

References

  1. S. Balin(2011a), "Non-identical parallel machine scheduling using genetic algorithm." Expert Systems with Applications, 38(6):6814-6821.  https://doi.org/10.1016/j.eswa.2010.12.064
  2. S. Balin(2011b), "Parallel machine scheduling with fuzzy processing times using a robust genetic algorithm and simulation." Information Sciences, 181(17):3551-3569.  https://doi.org/10.1016/j.ins.2011.04.010
  3. H. G. De-Alba, et al.(2022), "A mixed integer formulation and an efficient metaheuristic for the unrelated parallel machine scheduling problem: Total tardiness minimization." EURO Journal on Computational Optimization, 10:100034. 
  4. J. H. Holland(1975), Adaptation in natural and artificial system. Cambridge: MIT Press. 
  5. S. I. Kim, et al.(2007), "Scheduling algorithms for parallel machines with sequence-dependent set-up and distinct ready times: Minimizing total tardiness." Proceedings of the Institution of Mechanical Engineers, Part B: Journal of Engineering Manufacture, 221(6):1087-1096.  https://doi.org/10.1243/09544054JEM779
  6. Y. G. Kim(2017), Metaheuristics. Chonnam National University Press. 
  7. W. Kool, et al.(2019), "Attention, learn to solve routing problems!." International Conference on Learning Representations, 2019:1-25. 
  8. Y. D. Kwon, et al.(2020), "POMO: Policy optimization with multiple optima for reinforcement learning." Advances in Neural Information Processing Systems, 33:21188-21198. 
  9. Y. D. Kwon, et al.(2021), "Matrix encoding networks for neural combinatorial optimization." Advances in Neural Information Processing Systems, 34:5138-5149. 
  10. D. H. Lee, et al.(2023), "Deep reinforcement learning-based scheduler on parallel dedicated machine scheduling problem towards minimizing total tardiness." Sustainability, 15(4):2920. 
  11. T. H. Lee, W. S. Yoo(2023), "A study on memetic algorithm-based scheduling for minimizing makespan in unrelated parallel machines without setup time." Journal of the Korea Safety Management & Science, 25(2):1-8.  https://doi.org/10.12812/KSMS.2023.25.2.001
  12. Y. H. Lee, M. Pinedo(1997), "Scheduling jobs on parallel machines with sequence-dependent setup times." European Journal of Operational Research, 100(3):464-474.  https://doi.org/10.1016/S0377-2217(95)00376-2
  13. Y. K. Lin, F. Y. Hsieh(2014), "Unrelated parallel machine scheduling with setup times and ready times." International Journal of Production Research, 52(4):1200-1214.  https://doi.org/10.1080/00207543.2013.848305
  14. T. E. Morton, R. M. V. Rachamadugu(1981), "Myopic heuristics for the single machine weighted tardiness problem." Robotics Institute, Carnegie Mellon University, Technical Report, CMU-RI-TR-83-09. 
  15. P. Moscato(1989), "On evolution, search, optimization, genetic algorithms and martial arts: Towards memetic algorithms." Caltech Concurrent Computation Program 158-79, C3P Report, 826:37. 
  16. S. U. Randhawa, T. A. Smith(1995), "An experimental investigation of scheduling non-identical, parallel processors with sequence-dependent set-up times and due dates." The International Journal of Production Research, 33(1):59-69.  https://doi.org/10.1080/00207549508930137
  17. A. Vaswani, et al.(2017), "Attention is all you need." Advances in Neural Information Processing Systems, 30. 
  18. A. P. J. Vepsalainen, T. E. Morton(1987), "Priority rules for job shops with weighted tardiness costs." Management Science, 33(8):1035-1047.  https://doi.org/10.1287/mnsc.33.8.1035
  19. R. J. Williams(1992), "Simple statistical gradient-following algorithms for connectionist reinforcement learning." Machine Learning, 8(3):229-256. https://doi.org/10.1007/BF00992696