Journal of Korean Society of Industrial and Systems Engineering (산업경영시스템학회지)

Society of Korea Industrial and System Engineering (한국산업경영시스템학회)
권호 표지
DOI QR코드

DOI QR Code

Reinforcement Learning-based Dynamic Weapon Assignment to Multi-Caliber Long-Range Artillery Attacks

다종 장사정포 공격에 대한 강화학습 기반의 동적 무기할당

  • Received : 2022.09.15
  • Accepted : 2022.11.09
  • Published : 2022.12.31
Download PDF
⟨ Previous Next ⟩

Abstract

North Korea continues to upgrade and display its long-range rocket launchers to emphasize its military strength. Recently Republic of Korea kicked off the development of anti-artillery interception system similar to Israel's "Iron Dome", designed to protect against North Korea's arsenal of long-range rockets. The system may not work smoothly without the function assigning interceptors to incoming various-caliber artillery rockets. We view the assignment task as a dynamic weapon target assignment (DWTA) problem. DWTA is a multistage decision process in which decision in a stage affects decision processes and its results in the subsequent stages. We represent the DWTA problem as a Markov decision process (MDP). Distance from Seoul to North Korea's multiple rocket launchers positioned near the border, limits the processing time of the model solver within only a few second. It is impossible to compute the exact optimal solution within the allowed time interval due to the curse of dimensionality inherently in MDP model of practical DWTA problem. We apply two reinforcement-based algorithms to get the approximate solution of the MDP model within the time limit. To check the quality of the approximate solution, we adopt Shoot-Shoot-Look(SSL) policy as a baseline. Simulation results showed that both algorithms provide better solution than the solution from the baseline strategy.

Keywords

Acknowledgement

This study was partially supported by 2022 Industry-Academic Research Program of Hannam University and Hanwha systems.

References

  1. Bertsekas, D., Homer, M., Logan, D., Patek, S., and Sandell, N., Missile Defense and Interceptor Allocation by Neuro-Dynamic Programming, IEEE Transactions on Systems, Man, and Cybernetics, Part A: Systems and Humans, 2000, Vol. 30, No. 1, pp. 42-51.  https://doi.org/10.1109/3468.823480
  2. Bertsekas, D.P., Approximate Policy Iteration: A Survey and Some New Methods, Journal of Control Theory and Applications, 2011, Vol. 9, pp. 310-335.  https://doi.org/10.1007/s11768-011-1005-3
  3. Bertsekas, D.P., Dynamic Programming and Optimal Control: Approximate Dynamic Programming, 4th Edition, 2012, Athena Scientific, Belmont, MA. 
  4. Cha, Y.H. and Jeong, B., Exact Algorithm for the Weapon Target Assignment and Fire Scheduling Problem, Journal of the Society of Korea Industrial and Systems Engineering, 2019, Vol. 42, No. 1, pp. 143- 150.  https://doi.org/10.11627/jkise.2019.42.1.143
  5. Choi, B.W., Yoo, B.C., Kim, J.H., and Yim, D.S., A Study on the Flight Trajectory Prediction Method of Ballistic Missiles, 2020 Autumn Conference on Journal of Korean Society of Systems Engineering, 2020, pp. 131-140. 
  6. Chosun Media, https://www.chosun.com/politics/diplomacy-defense/2022/04/10/3AQITBNGDZHBFGUI4WFTS3A4RA/ (accessed 2022/08/30) 
  7. Davis, M.T., Robbins, M.J., and Lunday, B.J., Approximate Dynamic Programming for Missile Defense Interceptor Fire Control, European Journal of Operational Research, 2017, Vol. 259, pp. 873-886.  https://doi.org/10.1016/j.ejor.2016.11.023
  8. Hong, D.W., Yim, D.S., and Choi, B.W., Application and Determination of Defended Footprint Using a Simulation Model for Ballistic Missile Trajectory, Journal of the Korea Institute of Military Science and Technology, 2018, Vol. 21, No. 4, pp. 551-561.  https://doi.org/10.9766/KIMST.2018.21.4.551
  9. Hosein, P.A., A Class of Dynamic Nonlinar Resource Allocation Problems,[Ph. D. Dissertation], Massachusetts Institute of Technology, 1989. 
  10. Im, J.S., Yoo, B.C., Kim, J.H., and Choi, B.W., A Study of Multi-to-Majority Response on Threat Assessment and Weapon Assignment Algorithm: by Adjusting Ballistic Missiles and Long-Range Artillery Threat, Journal of Korean Society of Industrial and systems Engineering, 2021, Vol. 44, N0. 4, pp. 43-52. 
  11. Jang, J.G., Kim, K., Choi, B.W., and Suh, J.J., A Linear Approximation Model for an Asset-based Weapon Target Assignment Problem, Journal Society of Korea Industrial and System Engineering, 2015, Vol. 38, No. 3, pp. 108-116.  https://doi.org/10.11627/jkise.2015.38.3.108
  12. Jung, J.K., Uhm, H.S., and Lee, Y.H., Rolling-Horizon Scheduling Algorithm for Dynamic Weapon-Target Assignment in Air Defense Engagement, Journal of the Korean Institute of Industrial Engineering, 2020, Vol. 46, No. 1, pp. 11-24.  https://doi.org/10.7232/JKIIE.2020.46.1.011
  13. Kim, J.H., Kim, K., Choi, B.W., and Suh, J.J., An Application of Quantum-inspired Genetic Algorithm for Weapon Target Assignment Problem, Journal Society of Korea Industrial and System Engineering, 2017, Vol. 40, No. 4, pp. 260-267.  https://doi.org/10.11627/jkise.2017.40.4.260
  14. Kim, T., Yun, N., Kim, Y.J., Park, I., and Shim, D., Effect Analysis of Long-range Artillery Intercept System According to its Component Arrangement, Journal of Korean Society Industrial and System Engineering, 2022, Vol. 45, No. 1, pp. 41~52.  https://doi.org/10.11627/jksie.2022.45.1.041
  15. Lee, C., Kim, J.-H., Choi, B.W., and Kim, K., Approximate Dynamic Programming Based Interceptor Fire Control and Effectiveness Analysis for M-to-M Engagement, Journal of the Korean Society for Aeronautical & Space Science, 2022, Vol. 50, No. 4, pp. 287-295. 
  16. Lee, Z.J., Lee, C.Y., and Su, S.F., An Immunity Based Ant Colony Optimization Algorithm for Solving Weapon-Target Assignment Problem, Applied Soft Computing, 2002, Vol. 2, No. 1, pp. 39-47.  https://doi.org/10.1016/S1568-4946(02)00027-3
  17. McKenna, R.S., Robbins, M.J., Lunday, B.J., and McCormack, I.M., Approximate Dynamic Programming for the Military Inventory Routing Problem, Annals of Operational Research, 2020, Vol. 288, No. 1, pp. 391-416.  https://doi.org/10.1007/s10479-019-03469-8
  18. Ministry of National Defense, 2020 Defense White Paper, 2020, Seoul, Ministry of National Defense. 
  19. Naeem, H. and Masood, A., An Optimal Dynamic Threat Evaluation and Weapon Scheduling Technique, Knowledge-Based Systems, 2010, Vol. 23, No. 4, pp. 337-342.  https://doi.org/10.1016/j.knosys.2009.11.012
  20. Powell, W.B., Approximate Dynamic Programming: Solving the Curse of Dimensionality, Second Edition, 2011, John Wiley & Sons, Hoboken, NJ. 
  21. Powell, W. B., Perspectives of Approximate Dynamic Programming, Annals of Operations Research, 2012, Vol 13, No. 2, pp. 1-38. 
  22. Rempel, M. and Bai, J., A Review of Approximate Dynamic Programming Applications within Military Operations Research, Operational Research Perspectives, 2021, Vol. 8, 100204. 
  23. Shin, M.K., Park, S.-S., Lee, D., and Choi, H.-L., "Mean Field Game based Reinforcement Learning for Weapon-Target Assignment", Journal of the Korea Institute of Military Science and Technology, 2020, Vol. 23, No. 4, pp. 337-345.  https://doi.org/10.9766/KIMST.2020.23.4.337
  24. Summers, D.S., Robbins, M.J., and Lunday, B.J., An Approximate Dynamic Programming for Comparing Firing Policies in a Networked Air Defense Environment, Computers & Operations Research, 2020, Vol. 117, 104890. 
  25. Yonhapnews, https://www.yna.co.kr/view/AKR20220410019151504 (accessed 2022/08/30) 
  26. Yoo, B.C., Kim, J.H., Kwon, Y.S., and Choi, B.W., A Study on the Flight Trajectory Prediction Method of Ballistic Missiles, Journal of Korean Society of Systems Engineering, 2020, Vol. 16, No. 2, pp. 131-140. https://doi.org/10.14248/JKOSSE.2020.16.2.131