International Journal of Internet, Broadcasting and Communication

  • 제16권3호
  • /
  • Pages.205-211
  • /
  • 2024
  • /
  • 2288-4920(pISSN)
  • /
  • 2288-4939(eISSN)
한국인터넷방송통신학회 (The Institute of Internet, Broadcasting and Communication)
권호 표지
DOI QR코드

DOI QR Code

Intelligent Warehousing: Comparing Cooperative MARL Strategies

  • Yosua Setyawan Soekamto (Department of Computer Engineering, Dongseo University) ;
  • Dae-Ki Kang (Department of Computer Engineering, Dongseo University)
  • 투고 : 2024.06.11
  • 심사 : 2024.06.23
  • 발행 : 2024.08.31
PDF 다운로드
⟨ 이전 논문 다음 논문 ⟩

초록

Effective warehouse management requires advanced resource planning to optimize profits and space. Robots offer a promising solution, but their effectiveness relies on embedded artificial intelligence. Multi-agent reinforcement learning (MARL) enhances robot intelligence in these environments. This study explores various MARL algorithms using the Multi-Robot Warehouse Environment (RWARE) to determine their suitability for warehouse resource planning. Our findings show that cooperative MARL is essential for effective warehouse management. IA2C outperforms MAA2C and VDA2C on smaller maps, while VDA2C excels on larger maps. IA2C's decentralized approach, focusing on cooperation over collaboration, allows for higher reward collection in smaller environments. However, as map size increases, reward collection decreases due to the need for extensive exploration. This study highlights the importance of selecting the appropriate MARL algorithm based on the specific warehouse environment's requirements and scale.

키워드

과제정보

This research was supported by "Regional Innovation Strategy (RIS)" through the National Research Foundation of Korea (NRF) funded by the Ministry of Education (MOE) (2023RIS-007) and the Basic Science Research Program through the National Research Foundation of Korea (NRF) funded by the Ministry of Science and ICT (NRF-2022R1A2C2012243).

참고문헌

  1. D. Ivanov, "Digital Supply Chain Management and Technology to Enhance Resilience by Building and Using End-to-End Visibility During the COVID-19 Pandemic," IEEE Trans Eng Manag, vol. 71, pp. 10485-10495, 2024, doi: 10.1109/TEM.2021.3095193.
  2. J. Gu, M. Goetschalckx, and L. F. McGinnis, "Research on warehouse operation: A comprehensive review," Eur J Oper Res, vol. 177, no. 1, pp. 1-21, Feb. 2007, doi: 10.1016/j.ejor.2006.02.025.
  3. M. C. Gombolay, R. J. Wilcox, and J. A. Shah, "Fast Scheduling of Multi-Robot Teams with Temporospatial Constraints," IEEE Transactions on Robotics, vol. 34, no. 1, pp. 220-239, 2018, doi: 10.1109/TRO.2018.2795034.
  4. Y. Liu, X. Tao, X. Li, A. W. Colombo, and S. Hu, "Artificial Intelligence in Smart Logistics Cyber-Physical Systems: State-of-The-Arts and Potential Applications," IEEE Transactions on Industrial Cyber-Physical Systems, vol. 1, pp. 1-20, Jun. 2023, doi: 10.1109/ticps.2023.3283230.
  5. M. Akbari and T. N. A. Do, "A systematic review of machine learning in logistics and supply chain management: current trends and future directions," Benchmarking, vol. 28, no. 10. Emerald Group Holdings Ltd., pp. 2977-3005, Nov. 05, 2021. doi: 10.1108/BIJ-10-2020-0514.
  6. L. Busoniu, R. Babuska, and B. De Schutter, "A comprehensive survey of multiagent reinforcement learning," IEEE Transactions on Systems, Man and Cybernetics Part C: Applications and Reviews, vol. 38, no. 2. pp. 156-172, Mar. 2008. doi: 10.1109/TSMCC.2007.913919.
  7. K. Zhang, Z. Yang, and T. Basar, "Multi-Agent Reinforcement Learning: A Selective Overview of Theories and Algorithms," Nov. 2019, [Online]. Available: http://arxiv.org/abs/1911.10635
  8. J. Schulman, F. Wolski, P. Dhariwal, A. Radford, and O. Klimov, "Proximal Policy Optimization Algorithms," Jul. 2017, [Online]. Available: http://arxiv.org/abs/1707.06347
  9. M. Zhou et al., "MALib: A Parallel Framework for Population-based Multi-agent Reinforcement Learning," Jun. 2021, [Online]. Available: http://arxiv.org/abs/2106.07551
  10. Y. Yang, R. Luo, M. Li, M. Zhou, W. Zhang, and J. Wang, "Mean Field Multi-Agent Reinforcement Learning," Feb. 2018, [Online]. Available: http://arxiv.org/abs/1802.05438
  11. R. Lowe, Y. Wu, A. Tamar, J. Harb, P. Abbeel, and I. Mordatch, "Multi-Agent Actor-Critic for Mixed Cooperative-Competitive Environments," Jun. 2017, [Online]. Available: http://arxiv.org/abs/1706.02275
  12. L. Panait and S. Luke, "Cooperative Multi-Agent Learning: The State of the Art," Autonomous Agents and Multi-Agent Systems, vol. 11, no. 1, pp. 378-434, 2005, doi: 10.1007/s10458-005-2631-2.
  13. A. OroojlooyJadid and D. Hajinezhad, "A Review of Cooperative Multi-Agent Deep Reinforcement Learning," Aug. 2019, [Online]. Available: http://arxiv.org/abs/1908.03963
  14. J. Foerster, G. Farquhar, T. Afouras, N. Nardelli, and S. Whiteson, "Counterfactual Multi-Agent Policy Gradients," May 2017, [Online]. Available: http://arxiv.org/abs/1705.08926
  15. G. Papoudakis, F. Christianos, L. Schafer, and S. V. Albrecht, "Benchmarking Multi-Agent Deep Reinforcement Learning Algorithms in Cooperative Tasks," Conference on Neural Information Processing Systems (NeurIPS), Jun. 2021.
  16. V. Mnih et al., "Asynchronous Methods for Deep Reinforcement Learning," International Conference on Machine Learning (ICML), Feb. 2016.
  17. J. Su, S. Adams, and P. A. Beling, "Value-Decomposition Multi-Agent Actor-Critics," AAAI Conference on Artificial Intelligence (AAAI), Jul. 2020.