Journal of Korean Society of Industrial and Systems Engineering (산업경영시스템학회지)

Society of Korea Industrial and System Engineering (한국산업경영시스템학회)
권호 표지
DOI QR코드

DOI QR Code

A Reinforcement Learning Model for Dispatching System through Agent-based Simulation

에이전트 기반 시뮬레이션을 통한 디스패칭 시스템의 강화학습 모델

  • Minjung Kim (Department of Industrial and Management Engineering, Hanbat National University) ;
  • Moonsoo Shin (Department of Industrial and Management Engineering, Hanbat National University)
  • 김민정 (국립한밭대학교 산업경영공학과) ;
  • 신문수 (국립한밭대학교 산업경영공학과)
  • Received : 2024.05.31
  • Accepted : 2024.06.18
  • Published : 2024.06.30
Download PDF
⟨ Previous Next ⟩

Abstract

In the manufacturing industry, dispatching systems play a crucial role in enhancing production efficiency and optimizing production volume. However, in dynamic production environments, conventional static dispatching methods struggle to adapt to various environmental conditions and constraints, leading to problems such as reduced production volume, delays, and resource wastage. Therefore, there is a need for dynamic dispatching methods that can quickly adapt to changes in the environment. In this study, we aim to develop an agent-based model that considers dynamic situations through interaction between agents. Additionally, we intend to utilize the Q-learning algorithm, which possesses the characteristics of temporal difference (TD) learning, to automatically update and adapt to dynamic situations. This means that Q-learning can effectively consider dynamic environments by sensitively responding to changes in the state space and selecting optimal dispatching rules accordingly. The state space includes information such as inventory and work-in-process levels, order fulfilment status, and machine status, which are used to select the optimal dispatching rules. Furthermore, we aim to minimize total tardiness and the number of setup changes using reinforcement learning. Finally, we will develop a dynamic dispatching system using Q-learning and compare its performance with conventional static dispatching methods.

Keywords

Acknowledgement

This research was supported by the research fund of Hanbat National University in 2023.

References

  1. Cho, Y.I., Nam, S.H., and Woo, J.H., Development of the Reinforcement Learning-based Adaptive Scheduling Algorithm for Panel Block Shop, Korean Journal of Computational Design and Engineering, 2021, Vol. 26, No. 2, pp. 81-92.
  2. Cho, Y.I., Oh, S.H., Kwak, D.H., Choi, J.H., and Woo, J.H., Development of Quay Scheduling Algorithm Based on Reinforcement Learning, Korean Journal of Computational Design and Engineering, 2022, Vol. 27, No. 2, pp. 98-114.
  3. Choi, H.W., Byeon, H.J., Yoon, S.H., Kim, B.S., and Hong, S.D., Analysis of Workforce Scheduling Using Adjusted Man-machine Chart and Simulation, Journal of Korean Society of Industrial and Systems Engineering, 2024, Vol. 47, No. 1, pp. 20-27.
  4. Hanbat National University, https://www.youtube.com/watch?v=U-Zsqq0I2X8&t=400s.
  5. Kang, B.W., Kang, B.M., and Hong, S.D., A Dynamic OHT Routing Algorithm in Automated Material Handling Systems, Journal of Korean Society of Industrial and Systems Engineering, 2022, Vol. 45, No. 3, pp. 40-48.
  6. Kim, S.J., Yoo, W.S., and Kim, G.H., Real-time scheduling using CNN on a parallel machine with setup cost, Proceedings of the 2018 Korean Institute of Industrial Engineers Fall Conference, 2018, Seoul, pp. 1385-1401.
  7. Kim, H.H., Kim, J.H., Kong, J.H., and Kyung, J.H., Reinforcement Learning-based Dynamic Weapon Assignment to Multi-Caliber Long-Range Artillery Attacks, Journal of Korean Society of Industrial and Systems Engineering, 2022, Vol. 45, No. 4, pp. 42-52.
  8. Levine, S., Reinforcement Learning and Control as Probabilistic Inference: Tutorial and Review, 2018, arXiv Preprint:1805.00909.
  9. Nam, S.H., Cho, Y.I., and Woo, J.H., Reinforcement Learning for Minimizing Tardiness and Set-Up Change in Parallel Machine Scheduling Problems for Profile Shops in Shipyard, Journal of the Society of Naval Architects of Korea, 2023, Vol. 60, No. 3, pp.202-211.
  10. Priore, P., Gomez, A., Pino, R., and Rosillo, R., Dynamic scheduling of manufacturing systems using machine learning: An updated review, Artificial Intelligence for Engineering Design, Analysis and Manufacturing, 2014, Vol. 28, No. 1, pp. 83-97.
  11. The AnyLogic Company, https://www.anylogic.com.
  12. Waschneck, B., Reichstaller, A., Belzner, L., Altenmuller, T., Bauernhansl, T., Knapp, A., and Kyek, A., Deep reinforcement learning for semiconductor production scheduling, Proceedings of the 29th Annual SEMI Advanced Semiconductor Manufacturing Conference (ASMC), Saratoga Springs, NY, USA, 2018, pp. 301-306.
  13. Yoo, W.S., Kim, S.J., and Kim, G.H., Real-Time Scheduling Scheme based on Reinforcement Learning Considering Minimizing Setup Cost, The Journal of Society for e-Business Studies, 2020, Vol. 25, No. 2, pp.15-27.
  14. Zhang, Z., Wang, W., Zhong, S., and Kaishun, H.U., Flow Shop Scheduling with Reinforcement Learning, Asia-Pacific Journal of Operational Research, 2013.