Journal of the Korea Institute of Military Science and Technology (한국군사과학기술학회지)

The Korea Institute of Military Science and Technology (한국군사과학기술학회)
권호 표지
DOI QR코드

DOI QR Code

A Study on Reinforcement Learning Method for the Deception Behavior : Focusing on Marine Corps Amphibious Demonstrations

강화학습을 활용한 기만행위 모의방법 연구 : 해병대 상륙양동 사례를 중심으로

  • Park, Daekuk (Department of Defense Science, Korea National Defense University) ;
  • Cho, Namsuk (Department of Defense Science, Korea National Defense University)
  • 박대국 (국방대학교 국방과학학과) ;
  • 조남석 (국방대학교 국방과학학과)
  • Received : 2022.03.09
  • Accepted : 2022.07.29
  • Published : 2022.08.05
Download PDF
⟨ Previous Next ⟩

Abstract

Military deception is an action executed to deliberately mislead enemy's decision by deceiving friendly forces intention. In the lessons learned from war history, deception appears to be a critical factor in the battlefield for successful operations. As training using war-game simulation is growing more important, it is become necessary to implement military deception in war-game model. However, there is no logics or rules proven to be effective for CGF(Computer Generated Forces) to conduct deception behavior automatically. In this study, we investigate methodologies for CGF to learn and conduct military deception using Reinforcement Learning. The key idea of the research is to define a new criterion called a "deception index" which defines how agent learn the action of deception considering both their own combat objectives and deception objectives. We choose Korea Marine Corps Amphibious Demonstrations to show applicability of our methods. The study has an unique contribution as the first research that describes method of implementing deception behavior.

Keywords

References

  1. R.O.K Joint Chiefs of Staff Homepage, "Deception Operations," https://www.jcs.mil.kr/user/indexSub.action?codyMenuSeq=71157&siteId=jcs, accessed 1 Nov. 2021.
  2. S. B. Park, "The Importance of Military Deception and the Direction of Development of Our Military through the Transcription," Study Thesis, Korea National Defense University, pp. 1-2, 2019.
  3. R.O.K Joint Chiefs of Staff Homepage, "Demonstrations," https://www.jcs.mil.kr/user/indexSub.action? codyMenuSeq=71157&siteId=jcs, accessed 1 Nov. 2021.
  4. R.O.K Joint Chiefs of Staff Homepage, "Amphibious Demonstrations," https://www.jcs.mil.kr/user/indexSub.action? codyMenuSeq=71157&siteId=jcs, accessed 1 Nov. 2021.
  5. Institute for Military History, MND, "Inchon Landing Operation and Counterattack Operations," pp. 122-124, 2009.
  6. WIKIPEDIA Homepage, "Normandy Langings," https://en.wikipedia.org/wiki/Normandy_landings, accessed 1 Nov. 2021.
  7. K. S. Yoon, "The Direction of Joint Military Deception for Securing Initiative," Study Thesis, Korea National Defense University, pp. 1-10, 2014.
  8. S. K. Lee, et. al., "A Study on the Application of Computer Generated Forces Technology to Defense," Review of Korean Society for Internet Information, Vol. 13, No. 4, pp. 17-22, 2012.
  9. N. S. Cho, et. al., "The Study on CGF Behavior Modeling Methodologies for Defense M&S: Focusing on Survey and Future Direction," The Korea Society for Simulation, Vol. 29, No. 2, pp. 35-44, 2020.
  10. R.O.K Marine Corps, "Marine Corps Vision 2049," pp. 62-84, 2020.
  11. S. H. Jung, "A Goal Oriented Action Planning and Replanning method of Computer Generated Forces in Wargame," Journal of Korean Institute of Intelligent Systems, Vol. 23, No. 2, pp. 120-125, 2013. https://doi.org/10.5391/JKIIS.2013.23.2.120
  12. C. H. Han, et. al., "A Study for Autonomous Intelligence of Computer-Generated Forces," The Korea Society for Simulation, Vol. 21, No. 7, pp. 69-77, 2011.
  13. J. P. Hong, et. al., "A Case Study on Planning and Learning for Large-Scale CGFs with POMDPs," The Korean Institute of Information Scientists and Engineers Conference, pp. 983-985, 2016.
  14. K. H. Lee, et. al., "Modeling Combat Entity with POMDP and DEVS," Korean Institute Of Industrial Engineers, Vol. 39, No. 6, pp. 498-516, 2013. https://doi.org/10.7232/JKIIE.2013.39.6.498
  15. S. Y. Jang, et. al., "Research Trends on Deep Reinforcement Learning," Electronics and Telecommunications Trends, Vol. 34, No. 4, pp. 1-2, 2019.
  16. W. J. Jung, et. al., "Build Reinforcement Learning AI Process for Cooperative Play with Users," Journal of Korea Game Society, Vol. 20, No. 1, pp. 57-66, 2020. https://doi.org/10.7583/JKGS.2020.20.1.57
  17. H. V. Kim, et. al., "Trajectory Optimization of UAV Using Q Learning," The Korean Society for Aeronautical & Space Sciences Conference, pp. 1108-1109, 2019.
  18. R Core Team, "R: A Language and Environment for Statistical Computing," https://www.r-project.org/, 2020.
  19. R and RSudio Homepage, "Reinforcement Learning in R," https://cran.r-project.org/web/packages/ReinforcementLearning/vignettes/ReinforcementLearning.html, accessed 10 Nov. 2021.
  20. Wilensky, U, "Netlogo," http://ccl.northwestern.edu/netlogo/, 1999.
  21. Netlogo Homepage, "Reinforcement Learning Maze," http://ccl.northwestern.edu/netlogo/models/community/Reinforcement%Learning%Maze, accessed 10 Nov. 2021.
  22. Unity3D Homepage, "Unity 3D," https://unity.com/solutions/gaming-services, accessed 10 Nov. 2021.
  23. Unity3D Homepage, "ML-Agent Toolkit," https://unity.com/products/machine-learning-agents, accessed 10 Nov. 2021.
  24. MATLAB and Statistics Toolbox Release 2012b, The MathWorks, Inc, "MATLAB & Simulink - MathWorks," https://kr.mathworks.com/products/matlab.html
  25. MATLAB Homepage, "Reinforcement Learning Toolbox," https://kr.mathworks.com/products/reinforcement-learning.html
  26. B. G. Park, et. al., "Implementation of Horror Game Using Unity Game Engine," Proceedings of KIIT Conference, pp. 483-484, 2017.
  27. H. B. Choi, et. al., "Design and Implementation of Reinforcement Learning Environment Using Unity 3D-based ML-Agents Toolkit," KIPS Conference, Vol. 26, No. 1, pp. 548-551, 2019.
  28. Institute for Military History, MND, "Inchon Landing Operation and Counterattack Operations," pp. 87-143, 2009.
  29. K. S. Kim, "The Inchoen Landing Operation and Response of North Korean People's Army," Journal of Millitary History Studies, Vol. 130, pp. 69-120, 2010.
  30. J. S. Park, "Review of the Changsa Landing Operation during the Korean War," Northeast Asian History Foundation, Vol. 69, pp. 243-282, 2020.
  31. C. E. Lee, et. al., "Technical Trends of AI Military Staff to Support Decision-Making of Commanders," Electronics and Telecommunications Research Institute, Vol. 36, No. 1, pp. 89-98, 2021.