Browse > Article
http://dx.doi.org/10.7472/jksii.2021.22.4.13

Formal Model of Extended Reinforcement Learning (E-RL) System  

Jeon, Do Yeong (Dept. of Software, Soongsil University)
Song, Myeong Ho (Dept. of Software, Soongsil University)
Kim, Soo Dong (Dept. of Software, Soongsil University)
Publication Information
Journal of Internet Computing and Services / v.22, no.4, 2021 , pp. 13-28 More about this Journal
Abstract
Reinforcement Learning (RL) is a machine learning algorithm that repeat the closed-loop process that agents perform actions specified by the policy, the action is evaluated with a reward function, and the policy gets updated accordingly. The key benefit of RL is the ability to optimze the policy with action evaluation. Hence, it can effectively be applied to developing advanced intelligent systems and autonomous systems. Conventional RL incoporates a single policy, a reward function, and relatively simple policy update, and hence its utilization was limited. In this paper, we propose an extended RL model that considers multiple instances of RL elements. We define a formal model of the key elements and their computing model of the extended RL. Then, we propose design methods for applying to system development. As a case stud of applying the proposed formal model and the design methods, we present the design and implementation of an advanced car navigator system that guides multiple cars to reaching their destinations efficiently.
Keywords
Reinforcement Learning (RL); Advanced RL; Formal Model; Design Methods; and Advanced Navigator System;
Citations & Related Records
연도 인용수 순위
  • Reference
1 F. Yao and L. Jia, "A Collaborative Multi-Agent Reinforcement Learning Anti-Jamming Algorithm in Wireless Networks," IEEE Wireless Communications Letters, Vol.8, No.4, pp.1024-1027, Aug. 2019. http://dx.doi.org/10.1109/LWC.2019.2904486   DOI
2 H. Huang and S. Li, "The Application of Reinforcement Learning in Amazons," in Proceedings of International Conference on Machine Learning, Big Data and Business Intelligence (MLBDBI), Taiyuan, China, pp.369-372, Jan. 2019. http://dx.doi.org/10.1109/MLBDBI48998.2019.00083   DOI
3 R. Sutton and A. Barto. Reinforcement Learning, second edition: An Introduction. Cambridge, Massachusetts: Bradford Books. 2018.
4 H. Schwartz. Multi-Agent Machine Learning: A Reinforcement Approach. Wiley. 2014.
5 J. Xu et al., "A Multi-Channel Reinforcement Learning Framework for Robotic Mirror Therapy," IEEE Robotics and Automation Letters, Vol.5, No.4, pp.5385-5392, Oct. 2020. http://dx.doi.org/10.1109/LRA.2020.3007408   DOI
6 S. Mukhopadhyay, O. Tilak and S. Chakrabarti, "Reinforcement Learning Algorithms for Uncertain, Dynamic, Zero-Sum Games," in Proceedings of 17th IEEE International Conference on Machine Learning and Applications (ICMLA 2018), Orlando, FL, USA, pp.48-54, Dec. 2018. http://dx.doi.org/10.1109/ICMLA.2018.00015   DOI
7 G. Sun, G. O. Boateng, et al., "A Reinforcement Learning Framework for Autonomous Cell Activation and Customized Energy-Efficient Resource Allocation in C-RANs," KSII Transactions on Internet and Information Systems, Vol.13, No.8, pp.3821-3841, Aug. 2019. http://dx.doi.org/10.3837/tiis.2019.08.001.   DOI
8 M. Wang, L. Wang and T. Yue, "An Application of Continuous Deep Reinforcement Learning Approach to Pursuit-Evasion Differential Game," in Proceedings of IEEE 3rd Information Technology, Networking, Electronic and Automation Control Conference (ITNEC 2019), Chengdu, China, pp.1150-1156, March 2019. http://dx.doi.org/10.1109/ITNEC.2019.8729310   DOI
9 T. Becsi, A. Szabo, et al., "Reinforcement Learning Based Control Design for a Floating Piston Pneumatic Gearbox Actuator," IEEE Access, Vol.8, pp.147295-147312, Aug. 2020. http://dx.doi.org/10.1109/ACCESS.2020.3015576   DOI
10 S. Choi, et al. "Controller Learning Method of Self-Driving Bicycle Using State-of-the-Art Deep Reinforcement Learning Algorithms," Journal of the Korea Society of Computer and Information, Vol.23, No.10, pp.23-31, Oct. 2018, http://dx.doi.org/10.9708/JKSCI.2018.23.10.023   DOI
11 D. Bertsekas. Reinforcement Learning and Optimal Control. Athena Scientific. 2019
12 H. Dong, Z. Ding, and S, Zhang. Deep Reinforcement Learning: Fundamentals, Research and Applications. Springer. 2020.
13 P. Wang, L. T. Yang, et al., "MMDP: A Mobile-IoT Based Multi-Modal Reinforcement Learning Service Framework," IEEE Transactions on Services Computing, Vol.13, No.10, pp.675-684, Aug. 2020. http://dx.doi.org/10.1109/TSC.2020.2964663   DOI
14 M. Kang, K. Kim, "Analysis of Reward Functions in Deep Reinforcement Learning for Continuous State Space Control, " Journal of KIISE, Vol.47, No.1, pp.78-87, Jan. 2020. http://dx.doi.org/10.5626/JOK.2020.47.1.78   DOI
15 Y. Li, W. Zheng and Z. Zheng, "Deep Robust Reinforcement Learning for Practical Algorithmic Trading," IEEE Access, Vol.7, pp.108014-108022, Aug. 2019. http://dx.doi.org/10.1109/ACCESS.2019.2932789   DOI
16 K. Shao, Y. Zhu, et al., "Cooperative Multi-Agent Deep Reinforcement Learning with Counterfactual Reward," in Proceedings of International Joint Conference on Neural Networks (IJCNN 2020), Glasgow, United Kingdom, pp.1-8, Sep. 2020. http://dx.doi.org/10.1109/IJCNN48605.2020.9207169   DOI
17 D. Choi and S. Park, "Reinforcement Learning based Single AI trained from Multi-game." Journal of The Korean Society for Computer Game, Vol.32, No.3, pp.59-64, Sep. 2019. http://dx.doi.org/10.21493/kscg.2016.29.2.1   DOI
18 T. Chu, J. Wang, et al., "Multi-Agent Deep Reinforcement Learning for Large-Scale Traffic Signal Control," IEEE Transactions on Intelligent Transportation Systems, Vol.21, No.3, pp.1086-1095, March 2020. http://dx.doi.org/10.1109/TITS.2019.2901791   DOI
19 D. Ha, "Reinforcement Learning for Improving Agent Design," Artificial Life, Vol.25, No.4, pp.352-365, Nov. 2019. http://dx.doi.org/10.1162/artl_a_00301   DOI
20 S. Woo and Y. Sung, "Dynamic Action Space Handling Method for Reinforcement Learning Models, " Journal of Information Processing Systems, Vol.16, No.5, pp.1223-1230, Oct. 2020. http://dx.doi.org/10.3745/JIPS.02.0146   DOI
21 A. Zai and B. Brown. Deep Reinforcement Learning in Action. Manning Publications. 2020.
22 T. Liu, B. Tian, et al., "Parallel reinforcement learning: a framework and case study," IEEE/CAA Journal of Automatica Sinica, Vol.5, No.4, pp.827-835, July 2018. http://dx.doi.org/10.1109/JAS.2018.7511144   DOI
23 N. Ebell and M. Pruckner, "Coordinated Multi-Agent Reinforcement Learning for Swarm Battery Control," in Proceedings of IEEE Canadian Conference on Electrical & Computer Engineering (CCECE 2018), Quebec City, QC, Canada, pp.1-4, May 2018. http://dx.doi.org/10.1109/CCECE.2018.8447851   DOI
24 M. A. Abd-Elmagid, H. S. Dhillon and N. Pappas, "A Reinforcement Learning Framework for Optimizing Age of Information in RF-Powered Communication Systems," IEEE Transactions on Communications, Vol.68, No.8, pp.4747-4760, Aug. 2020. http://dx.doi.org/10.1109/TCOMM.2020.2991992   DOI
25 W. Tang, B. Li, et al., "An Automatic Cost Learning Framework for Image Steganography Using Deep Reinforcement Learning," IEEE Transactions on Information Forensics and Security, Vol.16, pp.952-967, 2021. http://dx.doi.org/10.1109/TIFS.2020.3025438   DOI