Browse > Article
http://dx.doi.org/10.7472/jksii.2022.23.5.17

Modified Deep Reinforcement Learning Agent for Dynamic Resource Placement in IoT Network Slicing  

Ros, Seyha (Department of Software Convergence, Soonchunhyang University)
Tam, Prohim (Department of Software Convergence, Soonchunhyang University)
Kim, Seokhoon (Department of Software Convergence, Soonchunhyang University)
Publication Information
Journal of Internet Computing and Services / v.23, no.5, 2022 , pp. 17-23 More about this Journal
Abstract
Network slicing is a promising paradigm and significant evolution for adjusting the heterogeneous services based on different requirements by placing dynamic virtual network functions (VNF) forwarding graph (VNFFG) and orchestrating service function chaining (SFC) based on criticalities of Quality of Service (QoS) classes. In system architecture, software-defined networks (SDN), network functions virtualization (NFV), and edge computing are used to provide resourceful data view, configurable virtual resources, and control interfaces for developing the modified deep reinforcement learning agent (MDRL-A). In this paper, task requests, tolerable delays, and required resources are differentiated for input state observations to identify the non-critical/critical classes, since each user equipment can execute different QoS application services. We design intelligent slicing for handing the cross-domain resource with MDRL-A in solving network problems and eliminating resource usage. The agent interacts with controllers and orchestrators to manage the flow rule installation and physical resource allocation in NFV infrastructure (NFVI) with the proposed formulation of completion time and criticality criteria. Simulation is conducted in SDN/NFV environment and capturing the QoS performances between conventional and MDRL-A approaches.
Keywords
Deep reinforcement learning; network slicing; software-defined networking; network functions virtualization; edge computing;
Citations & Related Records
Times Cited By KSCI : 3  (Citation Analysis)
연도 인용수 순위
1 D. Sattar and A. Matrawy, "Optimal Slice Allocation in 5G Core Networks," in IEEE Networking Letters, vol. 1, no. 2, pp. 48-51, 2019. http://dx.doi.org/10.1109/LNET.2019.2908351   DOI
2 A. Moubayed and A. Shami, "Softwarization, Virtualization, and Machine Learning for Intelligent and Effective Vehicle-to-Everything Communications," in IEEE Intelligent Transportation Systems Magazine, vol. 14, no. 2, pp. 156-173, 2022. http://dx.doi.org/10.1109/MITS.2020.3014124   DOI
3 N. C. Luong et al., "Applications of Deep Reinforcement Learning in Communications and Networking: A Survey," in IEEE Communications Surveys & Tutorials, vol. 21, no. 4, pp. 3133-3174, 2019. http://dx.doi.org/10.1109/COMST.2019.2916583   DOI
4 S. Nath and J. Wu, "Deep reinforcement learning for dynamic computation offloading and resource allocation in cache-assisted mobile edge computing systems," in Intelligent and Converged Networks, vol. 1, no. 2, pp. 181-198, Sept. 2020. http://dx.doi.org/10.23919/ICN.2020.0014   DOI
5 P. Tam, S. Math and S. Kim, "Deep Neural Network-Based Critical Packet Inspection for Improving Traffic Steering in Software-Defined IoT," Journal of Internet Computing and Services, vol. 22, no. 6, pp. 1-8, 2021. http://dx.doi.org/10.7472/jksii.2021.22.6.1   DOI
6 Q. Ye, J. Li, K. Qu, W. Zhuang, X. S. Shen and X. Li, "End-to-End Quality of Service in 5G Networks: Examining the Effectiveness of a Network Slicing Framework," in IEEE Vehicular Technology Magazine, vol. 13, no. 2, pp. 65-74, 2018. http://dx.doi.org/10.1109/MVT.2018.2809473   DOI
7 A. Nassar and Y. Yilmaz, "Deep Reinforcement Learning for Adaptive Network Slicing in 5G for Intelligent Vehicular Systems and Smart Cities," in IEEE Internet of Things Journal, vol. 9, no. 1, pp. 222-235, 2022. http://dx.doi.org/10.1109/JIOT.2021.3091674   DOI
8 V. S. Mai, R. J. La, T. Zhang and A. Battou, "End-to-End Quality-of-Service Assurance with Autonomous Systems: 5G/6G Case Study," 2022 IEEE 19th Annual Consumer Communications & Networking Conference (CCNC), pp. 644-651, 2022 http://dx.doi.org/10.1109/CCNC49033.2022.9700514   DOI
9 A. Filali, Z. Mlika, S. Cherkaoui and A. Kobbane, "Dynamic SDN-Based Radio Access Network Slicing With Deep Reinforcement Learning for URLLC and eMBB Services," in IEEE Transactions on Network Science and Engineering, vol. 9, no. 4, pp. 2174-2187, 2022. http://dx.doi.org/10.1109/TNSE.2022.3157274.   DOI
10 K. Suh, S. Kim, Y. Ahn, S. Kim, H. Ju and B. Shim, "Deep Reinforcement Learning-Based Network Slicing for Beyond 5G," in IEEE Access, vol. 10, pp. 7384-7395, 2022. http://dx.doi.org/10.1109/ACCESS.2022.3141789.   DOI
11 T. Li, X. Zhu and X. Liu, "An End-to-End Network Slicing Algorithm Based on Deep Q-Learning for 5G Network," in IEEE Access, vol. 8, pp. 122229-122240, 2020. http://dx.doi.org/10.1109/ACCESS.2020.3006502   DOI
12 H. Hamzah, D. Le, M. Kim, and H. Choo, "Mobility-Aware Service Migration (MASM) Algorithms for Multi-Access Edge Computing," Journal of Internet Computing and Services, Vol. 21, No. 4, pp. 1-8, 2020. http://dx.doi.org/10.7472/jksii.2020.21.4.1   DOI
13 M. Bunyakitanon, X. Vasilakos, R. Nejabati and D. Simeonidou, "End-to-End Performance-Based Autonomous VNF Placement With Adopted Reinforcement Learning," in IEEE Transactions on Cognitive Communications and Networking, vol. 6, no. 2, pp. 534-547, 2020. http://dx.doi.org/10.1109/TCCN.2020.2988486   DOI
14 P. Tam, S. Math, and S. Kim, "Priority-Aware Resource Management for Adaptive Service Function Chaining in Real-Time Intelligent IoT Services," Electronics, vol. 11, no. 19, 2022. http://dx.doi.org/10.3390/electronics11192976   DOI
15 Y. Azimi, S. Yousefi, H. Kalbkhani and T. Kunz, "Energy-Efficient Deep Reinforcement Learning Assisted Resource Allocation for 5G-RAN Slicing," in IEEE Transactions on Vehicular Technology, vol. 71, no. 1, pp. 856-871, 2022. http://dx.doi.org/10.1109/TVT.2021.3128513   DOI
16 A. Nassar and Y. Yilmaz, "Deep Reinforcement Learning for Adaptive Network Slicing in 5G for Intelligent Vehicular Systems and Smart Cities," in IEEE Internet of Things Journal, vol. 9, no. 1, pp. 222-235, 2022. http://dx.doi.org/10.1109/JIOT.2021.3091674   DOI
17 P. Tam, S. Math and S. Kim, "Efficient Resource Slicing Scheme for Optimizing Federated Learning Communications in Software-Defined IoT Networks," Journal of Internet Computing and Services, vol. 22, no. 5, pp. 27-33, 2021. http://dx.doi.org/10.7472/jksii.2021.22.5.27   DOI
18 H. Ko, J. Lee and S. Pack, "Priority-Based Dynamic Resource Allocation Scheme in Network Slicing," 2021 International Conference on Information Networking (ICOIN), pp. 62-64, 2021. http://dx.doi.org/10.1109/ICOIN50884.2021.9333944   DOI