Journal of the Korea Institute of Military Science and Technology (한국군사과학기술학회지)

The Korea Institute of Military Science and Technology (한국군사과학기술학회)
권호 표지
DOI QR코드

DOI QR Code

Battle Simulator for Multi-Robot Mission Simulation and Reinforcement Learning

다중로봇 임무모의 및 강화학습을 위한 전투급 시뮬레이터 연구

  • Jungho Bae (Defense AI Center, Agency for Defense Development) ;
  • Youngil Lee (Defense AI Center, Agency for Defense Development) ;
  • Dohyun Kim (Defense AI Center, Agency for Defense Development) ;
  • Heesoo Kim (R&D Center, REALTIMEVISUAL Inc.) ;
  • Myoungyoung Kim (R&D Center, REALTIMEVISUAL Inc.) ;
  • Myungjun Kim (R&D Center, REALTIMEVISUAL Inc.) ;
  • Heeyoung Kim (Defense AI Center, Agency for Defense Development)
  • 배정호 (국방과학연구소 국방AI센터) ;
  • 이영일 (국방과학연구소 국방AI센터) ;
  • 김도현 (국방과학연구소 국방AI센터) ;
  • 김희수 (리얼타임비쥬얼(주) 기술연구소) ;
  • 김명영 (리얼타임비쥬얼(주) 기술연구소) ;
  • 김명준 (리얼타임비쥬얼(주) 기술연구소) ;
  • 김희영 (국방과학연구소 국방AI센터)
  • Received : 2024.02.27
  • Accepted : 2024.08.30
  • Published : 2024.10.05
Download PDF
⟨ Previous Next ⟩

Abstract

As AI technology advances, interest in performing multi-robot autonomous missions for manned-unmanned teaming (MUM-T) is increasing. In order to develop autonomous mission performance technology for multiple robots, simulation technology that reflects the characteristics of real robots and can flexibly apply various missions is needed. Additionally, in order to solve complex non-linear tasks, an API must be provided to apply multi-robot reinforcement learning technology, which is currently under active research. In this study, we propose the campaign model to flexibly simulate the missions of multiple robots. We then discuss the results of developing a simulation environment that can be edited and run and provides a reinforcement learning API including acceleration performance. The proposed simulated control module and simulated environment were verified using an enemy infiltration scenario, and parallel processing performance for efficient reinforcement learning was confirmed through experiments.

Keywords

Acknowledgement

이 논문은 2024년 정부의 재원으로 수행된 연구임

References

  1. P. Bounker, and A. Volk, "NATO LCG UGV ToE Interoperability Experiences," in Proc. AVT-241 Specialists' Meeting on Technological and Operational Problems Connected with UGV Application for Future Military Operations, 2015.
  2. M. Quigley, K. Conley, B. Gerkey, J. Faust, T. Foote, J. Leibs, R. Wheeler, and A.Y. Ng, "ROS: An Open-Source Robot Operating System," ICRA Workshop on Open Source Software, Vol. 3, p. 5, 2009.
  3. K. Mathiassen, F. E. Schneider, P. Bounker, A. Tiderko, G.D. Cubber, M. Baksaas, J. Glowka, R. Kozik, T. Nussbaumer, J. Roning, J. Pellenz, and A. Volk, "Demonstrating Interoperability between Unmanned Ground Systems and Command and Control Systems," Int. J. Intelligent Defence Support Systems, Vol. 6, No. 2, pp. 100-129, 2021.
  4. O. Vinyals et al, "Grandmaster Level in Starcraft II using Multi-Agent Reinforcement Learning," Nature, Vol. 575, No. 7782, pp. 350-354, 2019.
  5. J. Raiman, S. Zhang, and F. Wolski, "Long-Term Planning and Situational Awarenessin OpenAI Five," arXive:1912.06721, 2019.
  6. J. Terry et al, "PettingZoo: gym for Multi-Agent Reinforcement Learning," Advances in Neural Information Processing Systems, Vol. 34, pp. 15032 -15043, 2021.
  7. M. Towers et al, "Gymnasium," Zenodo, 2023.
  8. K. Kurach, A. Raichuk, P. Stanczyk, and M. Zajac, "Google Resarch Football: A Novel Reinforcement Learning Environment," AAAI Conf. Artificial Intelligence, pp. 4501-4510, 2020.
  9. M. Samvelyan et al, "The Starcraft Multi-Agent Challenge," Int'l Conf. Autonomous Agents and MultiAgent Systems, pp. 2186-2188, 2019.
  10. N. Koening, and A. Howard, "Design and Use Paradigms for Gazebo, an Open-Source Multi-Robot Simulator," IEEE/RSJ Int'l Conf. Intelligent Robots and Systems, Vol. 3, pp. 2149-2154, 2004.
  11. E. Todorov, T. Erez, and Y. Tassa, "MuJoCo: A Physics Engine for Model-Based Control," IEEE/RSJ Int'l Conf. Intelligent Robots and Systems, pp. 5026-5033, 2012.
  12. S. Shah, D. Dey, C. Lovett, and A. Kapoor, "AirSim: High-Fidelity Visual and Physical Simulation for Autonomous Vehicles," Springer Proc. Advanced Robotics, Vol. 5, pp. 621-635, 2018.
  13. J. Liang, V. Makoviychuk, A. Handa, N. Chentanez, M. Macklin, and D. Fox, "GPU-Accelerated Robotic Simulation for Distributed Reinforcement Learning," Conf. Robot Learning, pp. 270-282, 2018.
  14. OMG, UML. http://www.uml.org
  15. A. Babushkin, jMAVSim. https://pixhawk.org/dev/hil/jmavsim
  16. E. Liang, et al, "Ray RLLib: A Composable and Scalable Reinforcement Learning Library," arXive:1712.09381, 2017.
  17. X. Wang, H. Zhao, and J. Zhu, "GRPC: A Communication Cooperation Mechanism in Distributed Systems," ACM SIGOPS Operating Systems Review, Vol. 27, No. 3, pp. 75-86, 1993.