Journal of the Korea Institute of Military Science and Technology (한국군사과학기술학회지)

The Korea Institute of Military Science and Technology (한국군사과학기술학회)
권호 표지
DOI QR코드

DOI QR Code

Mean Field Game based Reinforcement Learning for Weapon-Target Assignment

평균 필드 게임 기반의 강화학습을 통한 무기-표적 할당

  • Shin, Min Kyu (Department of Aerospace Engineering, Korea Advanced Institute of Science and Technology) ;
  • Park, Soon-Seo (Department of Aerospace Engineering, Korea Advanced Institute of Science and Technology) ;
  • Lee, Daniel (Department of Aerospace Engineering, Korea Advanced Institute of Science and Technology) ;
  • Choi, Han-Lim (Department of Aerospace Engineering, Korea Advanced Institute of Science and Technology)
  • 신민규 (한국과학기술원 항공우주공학과) ;
  • 박순서 (한국과학기술원 항공우주공학과) ;
  • 이단일 (한국과학기술원 항공우주공학과) ;
  • 최한림 (한국과학기술원 항공우주공학과)
  • Received : 2020.04.14
  • Accepted : 2020.06.26
  • Published : 2020.08.05
Download PDF
⟨ Previous Next ⟩

Abstract

The Weapon-Target Assignment(WTA) problem can be formulated as an optimization problem that minimize the threat of targets. Existing methods consider the trade-off between optimality and execution time to meet the various mission objectives. We propose a multi-agent reinforcement learning algorithm for WTA based on mean field game to solve the problem in real-time with nearly optimal accuracy. Mean field game is a recent method introduced to relieve the curse of dimensionality in multi-agent learning algorithm. In addition, previous reinforcement learning models for WTA generally do not consider weapon interference, which may be critical in real world operations. Therefore, we modify the reward function to discourage the crossing of weapon trajectories. The feasibility of the proposed method was verified through simulation of a WTA problem with multiple targets in realtime and the proposed algorithm can assign the weapons to all targets without crossing trajectories of weapons.

Keywords

References

  1. Lloyd, S. P. and Witsenhausen, H. S., "Weapon Allocation is NP-complete," Summer Computer Simulation Conference, 1986.
  2. Ahuja, R. K., Kumar, A., Jha, K. C., & Orlin, J. B., "Exact and Heuristic Algorithms for the Weapon-Target Assignment Problem," Operations Research, 55(6), pp. 1136-1146, 2007. https://doi.org/10.1287/opre.1070.0440
  3. Shin, M. K., Lee, D., & Choi, H. L., "Weapon-Target Assignment Problem with Interference Constraints using Mixed-Integer Linear Programming," 2019.
  4. Lee, Z. J., Su, S. F., Lee, C. Y., "Efficiently Solving General Weapon-Target Assignment Problem by Genetic Algorithms with Greedy Eugenics," IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics), Vol. 33, No. 1, pp. 113-121, 2003. https://doi.org/10.1109/TSMCB.2003.808174
  5. Lee, D., Shin, M. K., & Choi, H. L., "Weapon Target Assignment Problem with Interference Constraints," AIAA Scitech 2020 Forum. 2020.
  6. Cho, D. H., & Choi, H. L., "Greedy Maximization for Asset-based Weapon-Target Assignment with Time-Dependent Rewards," Cooperative Control of Multi-Agent Systems: Theory and Applications, pp. 115-139, 2017.
  7. Mnih, Volodymyr, et al., "Playing Atari with Deep Reinforcement Learning," arXiv preprint arXiv:1312.5602, 2013.
  8. Yang, Yaodong, et al., "Mean Field Multi-Agent Reinforcement Learning," arXiv preprint arXiv:1802.05438, 2018.
  9. M. Zhang, J. Zhang, G. Cheng, C. Chen and Z. Liu, "Fire Scheduling for Multiple Weapons Cooperative Engagement," 2016 10th International Conference on Software, Knowledge, Information Management & Applications(SKIMA), Chengdu, pp. 55-60, 2016.
  10. Mouton, H., Roodt, J., & Le Roux, H., "Applying Reinforcement Learning to the Weapon Assignment Problem in Air Defence," Scientia Militaria: South African Journal of Military Studies, 39(2), pp. 99-116, 2011.