Journal of the Society of Naval Architects of Korea (대한조선학회논문집)

The Society of Naval Architects of Korea (대한조선학회)
권호 표지
DOI QR코드

DOI QR Code

Reinforcement Learning for Minimizing Tardiness and Set-Up Change in Parallel Machine Scheduling Problems for Profile Shops in Shipyard

조선소 병렬 기계 공정에서의 납기 지연 및 셋업 변경 최소화를 위한 강화학습 기반의 생산라인 투입순서 결정

  • So-Hyun Nam (Department of Naval Architecture and Ocean Engineering, Seoul National University) ;
  • Young-In Cho (Department of Naval Architecture and Ocean Engineering, Seoul National University) ;
  • Jong Hun Woo (Department of Naval Architecture and Ocean Engineering, Seoul National University)
  • 남소현 (서울대학교 조선해양공학과) ;
  • 조영인 (서울대학교 조선해양공학과) ;
  • 우종훈 (서울대학교 조선해양공학과)
  • Received : 2023.04.07
  • Accepted : 2023.05.23
  • Published : 2023.06.20
Download PDF
⟨ Previous Next ⟩

Abstract

The profile shops in shipyards produce section steels required for block production of ships. Due to the limitations of shipyard's production capacity, a considerable amount of work is already outsourced. In addition, the need to improve the productivity of the profile shops is growing because the production volume is expected to increase due to the recent boom in the shipbuilding industry. In this study, a scheduling optimization was conducted for a parallel welding line of the profile process, with the aim of minimizing tardiness and the number of set-up changes as objective functions to achieve productivity improvements. In particular, this study applied a dynamic scheduling method to determine the job sequence considering variability of processing time. A Markov decision process model was proposed for the job sequence problem, considering the trade-off relationship between two objective functions. Deep reinforcement learning was also used to learn the optimal scheduling policy. The developed algorithm was evaluated by comparing its performance with priority rules (SSPT, ATCS, MDD, COVERT rule) in test scenarios constructed by the sampling data. As a result, the proposed scheduling algorithms outperformed than the priority rules in terms of set-up ratio, tardiness, and makespan.

Keywords

Acknowledgement

본 연구는 국방과학연구소 선도형 핵심 기술 (응용연구) 사업의 자체 개발 이산 사건 시뮬레이션 방법에 의한 소티 생성률 산출 기술 개발 및 검증 과제의 도움을 받아 수행되었습니다.

References

  1. Chaudhry, I.A. and Elbadawi, I.A., 2017. Minimisation of total tardiness for identical parallel machine scheduling using genetic algorithm. Sadhana, 42(1), pp.11-21. https://doi.org/10.1007/s12046-016-0575-7
  2. Chen, C.L. and Chen, C.L., 2009. Hybrid metaheuristics for unrelated parallel machine scheduling with sequence-dependent setup times. The International Journal of Advanced Manufacturing Technology, 43(1), pp.161-169. https://doi.org/10.1007/s00170-008-1692-1
  3. Julaiti, J., Oh, S.C., Das, D. and Kumara, S., 2022. Stochastic parallel machine scheduling using reinforcement learning. Journal of Advanced Manufacturing and Processing, 4(4), pp.e10119.
  4. Kim, D.W., Kim, K.H., Jang, W. and Chen, F.F., 2002. Unrelated parallel machine scheduling with setup times using simulated annealing. Robotics and Computer-Integrated Manufacturing, 18(3-4), pp.223-231. https://doi.org/10.1016/S0736-5845(02)00013-3
  5. Lee, C.H., 2018. A dispatching rule and a random iterated greedy metaheuristic for identical parallel machine scheduling to minimize total tardiness. International journal of production research, 56(6), pp.2292-2308. https://doi.org/10.1080/00207543.2017.1374571
  6. Lee, J.H., Yu, J.M. and Lee, D.H., 2013. A tabu search algorithm for unrelated parallel machine scheduling with sequence-and machine-dependent setups: minimizing total tardiness. The International Journal of Advanced Manufacturing Technology, 69(9), pp.2081-2089. https://doi.org/10.1007/s00170-013-5192-6
  7. Lee, J.W. and Kim, H.J., 1995. Erection process planning & scheduling using genetic algorithm. Journal of the Society of Naval Architects of Korea, 32(1), pp.9-16.
  8. Paeng, B., Park, I.B. and Park, J., 2021. Deep reinforcement learning for minimizing tardiness in parallel machine scheduling with sequence dependent family setups. IEEE Access, 9, pp.101390-101401. https://doi.org/10.1109/ACCESS.2021.3097254
  9. Park, J.K. and Kim, M.K., 2020. Optimization of quantity allocation using integer linear programming in shipbuilding industry. Journal of the Society of Naval Architects of Korea, 57(1), pp.45-51. https://doi.org/10.3744/SNAK.2020.57.1.045
  10. Schulman, J., Wolski, F., Dhariwal, P., Radford, A. and Klimov, O., 2017. Proximal policy optimization algorithms. arXiv preprint arXiv:1707.06347.
  11. Son, J.R., Suh, H.W. and Ha, B.H, 2014. A heuristic algorithm for block storage planning in shipbuilding. Journal of the Society of Naval Architects of Korea, 51(3), pp.239-245. https://doi.org/10.3744/SNAK.2014.51.3.239
  12. Zhang, Z., Zheng, L., Li, N., Wang, W., Zhong, S. and Hu, K., 2012. Minimizing mean weighted tardiness in unrelated parallel machine scheduling with reinforcement learning. Computers & operations research, 39(7), pp.1315-1324. https://doi.org/10.1016/j.cor.2011.07.019
  13. Zhang, Z., Zheng, L. and Weng, M. X., 2007. Dynamic parallel machine scheduling with mean weighted tardiness objective by Q-Learning. The International Journal of Advanced Manufacturing Technology, 34(9), pp.968-980. https://doi.org/10.1007/s00170-006-0662-8