Browse > Article
http://dx.doi.org/10.11627/jksie.2022.45.4.042

Reinforcement Learning-based Dynamic Weapon Assignment to Multi-Caliber Long-Range Artillery Attacks  

Hyeonho Kim (Department of Industrial Engineering, Hannam University)
Jung Hun Kim (Hanwha Systems)
Joohoe Kong (CSB LAB)
Ji Hoon Kyung (Department of Industrial Engineering, Hannam University)
Publication Information
Journal of Korean Society of Industrial and Systems Engineering / v.45, no.4, 2022 , pp. 42-52 More about this Journal
Abstract
North Korea continues to upgrade and display its long-range rocket launchers to emphasize its military strength. Recently Republic of Korea kicked off the development of anti-artillery interception system similar to Israel's "Iron Dome", designed to protect against North Korea's arsenal of long-range rockets. The system may not work smoothly without the function assigning interceptors to incoming various-caliber artillery rockets. We view the assignment task as a dynamic weapon target assignment (DWTA) problem. DWTA is a multistage decision process in which decision in a stage affects decision processes and its results in the subsequent stages. We represent the DWTA problem as a Markov decision process (MDP). Distance from Seoul to North Korea's multiple rocket launchers positioned near the border, limits the processing time of the model solver within only a few second. It is impossible to compute the exact optimal solution within the allowed time interval due to the curse of dimensionality inherently in MDP model of practical DWTA problem. We apply two reinforcement-based algorithms to get the approximate solution of the MDP model within the time limit. To check the quality of the approximate solution, we adopt Shoot-Shoot-Look(SSL) policy as a baseline. Simulation results showed that both algorithms provide better solution than the solution from the baseline strategy.
Keywords
Reinforcement Learning; Dynamic Weapon Target Assignment; Long-Range Artillery;
Citations & Related Records
Times Cited By KSCI : 8  (Citation Analysis)
연도 인용수 순위
1 Bertsekas, D., Homer, M., Logan, D., Patek, S., and Sandell, N., Missile Defense and Interceptor Allocation by Neuro-Dynamic Programming, IEEE Transactions on Systems, Man, and Cybernetics, Part A: Systems and Humans, 2000, Vol. 30, No. 1, pp. 42-51.    DOI
2 Bertsekas, D.P., Approximate Policy Iteration: A Survey and Some New Methods, Journal of Control Theory and Applications, 2011, Vol. 9, pp. 310-335.    DOI
3 Bertsekas, D.P., Dynamic Programming and Optimal Control: Approximate Dynamic Programming, 4th Edition, 2012, Athena Scientific, Belmont, MA. 
4 Cha, Y.H. and Jeong, B., Exact Algorithm for the Weapon Target Assignment and Fire Scheduling Problem, Journal of the Society of Korea Industrial and Systems Engineering, 2019, Vol. 42, No. 1, pp. 143- 150.    DOI
5 Choi, B.W., Yoo, B.C., Kim, J.H., and Yim, D.S., A Study on the Flight Trajectory Prediction Method of Ballistic Missiles, 2020 Autumn Conference on Journal of Korean Society of Systems Engineering, 2020, pp. 131-140. 
6 Chosun Media, https://www.chosun.com/politics/diplomacy-defense/2022/04/10/3AQITBNGDZHBFGUI4WFTS3A4RA/ (accessed 2022/08/30) 
7 Davis, M.T., Robbins, M.J., and Lunday, B.J., Approximate Dynamic Programming for Missile Defense Interceptor Fire Control, European Journal of Operational Research, 2017, Vol. 259, pp. 873-886.    DOI
8 Hong, D.W., Yim, D.S., and Choi, B.W., Application and Determination of Defended Footprint Using a Simulation Model for Ballistic Missile Trajectory, Journal of the Korea Institute of Military Science and Technology, 2018, Vol. 21, No. 4, pp. 551-561.    DOI
9 Hosein, P.A., A Class of Dynamic Nonlinar Resource Allocation Problems,[Ph. D. Dissertation], Massachusetts Institute of Technology, 1989. 
10 Im, J.S., Yoo, B.C., Kim, J.H., and Choi, B.W., A Study of Multi-to-Majority Response on Threat Assessment and Weapon Assignment Algorithm: by Adjusting Ballistic Missiles and Long-Range Artillery Threat, Journal of Korean Society of Industrial and systems Engineering, 2021, Vol. 44, N0. 4, pp. 43-52. 
11 Jang, J.G., Kim, K., Choi, B.W., and Suh, J.J., A Linear Approximation Model for an Asset-based Weapon Target Assignment Problem, Journal Society of Korea Industrial and System Engineering, 2015, Vol. 38, No. 3, pp. 108-116.    DOI
12 Jung, J.K., Uhm, H.S., and Lee, Y.H., Rolling-Horizon Scheduling Algorithm for Dynamic Weapon-Target Assignment in Air Defense Engagement, Journal of the Korean Institute of Industrial Engineering, 2020, Vol. 46, No. 1, pp. 11-24.    DOI
13 Kim, J.H., Kim, K., Choi, B.W., and Suh, J.J., An Application of Quantum-inspired Genetic Algorithm for Weapon Target Assignment Problem, Journal Society of Korea Industrial and System Engineering, 2017, Vol. 40, No. 4, pp. 260-267.    DOI
14 Kim, T., Yun, N., Kim, Y.J., Park, I., and Shim, D., Effect Analysis of Long-range Artillery Intercept System According to its Component Arrangement, Journal of Korean Society Industrial and System Engineering, 2022, Vol. 45, No. 1, pp. 41~52.    DOI
15 Lee, C., Kim, J.-H., Choi, B.W., and Kim, K., Approximate Dynamic Programming Based Interceptor Fire Control and Effectiveness Analysis for M-to-M Engagement, Journal of the Korean Society for Aeronautical & Space Science, 2022, Vol. 50, No. 4, pp. 287-295. 
16 Lee, Z.J., Lee, C.Y., and Su, S.F., An Immunity Based Ant Colony Optimization Algorithm for Solving Weapon-Target Assignment Problem, Applied Soft Computing, 2002, Vol. 2, No. 1, pp. 39-47.    DOI
17 Powell, W.B., Approximate Dynamic Programming: Solving the Curse of Dimensionality, Second Edition, 2011, John Wiley & Sons, Hoboken, NJ. 
18 McKenna, R.S., Robbins, M.J., Lunday, B.J., and McCormack, I.M., Approximate Dynamic Programming for the Military Inventory Routing Problem, Annals of Operational Research, 2020, Vol. 288, No. 1, pp. 391-416.    DOI
19 Ministry of National Defense, 2020 Defense White Paper, 2020, Seoul, Ministry of National Defense. 
20 Naeem, H. and Masood, A., An Optimal Dynamic Threat Evaluation and Weapon Scheduling Technique, Knowledge-Based Systems, 2010, Vol. 23, No. 4, pp. 337-342.    DOI
21 Powell, W. B., Perspectives of Approximate Dynamic Programming, Annals of Operations Research, 2012, Vol 13, No. 2, pp. 1-38. 
22 Rempel, M. and Bai, J., A Review of Approximate Dynamic Programming Applications within Military Operations Research, Operational Research Perspectives, 2021, Vol. 8, 100204. 
23 Shin, M.K., Park, S.-S., Lee, D., and Choi, H.-L., "Mean Field Game based Reinforcement Learning for Weapon-Target Assignment", Journal of the Korea Institute of Military Science and Technology, 2020, Vol. 23, No. 4, pp. 337-345.    DOI
24 Summers, D.S., Robbins, M.J., and Lunday, B.J., An Approximate Dynamic Programming for Comparing Firing Policies in a Networked Air Defense Environment, Computers & Operations Research, 2020, Vol. 117, 104890. 
25 Yonhapnews, https://www.yna.co.kr/view/AKR20220410019151504 (accessed 2022/08/30) 
26 Yoo, B.C., Kim, J.H., Kwon, Y.S., and Choi, B.W., A Study on the Flight Trajectory Prediction Method of Ballistic Missiles, Journal of Korean Society of Systems Engineering, 2020, Vol. 16, No. 2, pp. 131-140.   DOI