Browse > Article
http://dx.doi.org/10.6109/jkiice.2020.24.11.1500

A Distributed Scheduling Algorithm based on Deep Reinforcement Learning for Device-to-Device communication networks  

Jeong, Moo-Woong (Dept. of Information and Communication Engineering, Gyeongsang National University)
Kim, Lyun Woo (Dept. of Information and Communication Engineering, Gyeongsang National University)
Ban, Tae-Won (Dept. of Information and Communication Engineering, Gyeongsang National University)
Publication Information
Journal of the Korea Institute of Information and Communication Engineering / v.24, no.11, 2020 , pp. 1500-1506 More about this Journal
Abstract
In this paper, we study a scheduling problem based on reinforcement learning for overlay device-to-device (D2D) communication networks. Even though various technologies for D2D communication networks using Q-learning, which is one of reinforcement learning models, have been studied, Q-learning causes a tremendous complexity as the number of states and actions increases. In order to solve this problem, D2D communication technologies based on Deep Q Network (DQN) have been studied. In this paper, we thus design a DQN model by considering the characteristics of wireless communication systems, and propose a distributed scheduling scheme based on the DQN model that can reduce feedback and signaling overhead. The proposed model trains all parameters in a centralized manner, and transfers the final trained parameters to all mobiles. All mobiles individually determine their actions by using the transferred parameters. We analyze the performance of the proposed scheme by computer simulation and compare it with optimal scheme, opportunistic selection scheme and full transmission scheme.
Keywords
Device-to-Device communication; Machine learning; Reinforcement learning; DQN; Scheduling algorithm;
Citations & Related Records
연도 인용수 순위
  • Reference
1 M. Sheng, H. Sun, X. Wang, Y. Zhang, T. Q. S. Quek, J. Liu, and J. Li, "Ondemand scheduling: achieving QoS dierentiation for D2D communications," IEEE Communications Magazine, vol. 53, no. 7, pp. 162-170, Jul. 2015.   DOI
2 J. Lyu, Y. H. Chew, and W.-C. Wong, "A Stackelberg Game Model for Overlay D2D Transmission With Heterogeneous Rate Requirements," IEEE Transactions on Vehicular Technology, vol. 65, no. 10, pp. 8461-8475, Oct. 2016.   DOI
3 J. Xu and C. Guo, "Scheduling Stochastic Real-Time D2D Communications," IEEE Transactions on Vehicular Technology, vol. 68, no. 6, pp. 6022-6036, Jun. 2019.   DOI
4 T. Ban and B. C. Jung, "On the Link Scheduling for Cellular-Aided Device-to-Device Networks," IEEE Transactions on Vehicular Technology, vol. 65, no. 11, pp. 9404-9409, Nov. 2016. doi: 10.1109/TVT.2016.2519461.   DOI
5 F. Meng, P. Chen, L. Wu, and J. Cheng, "Power Allocation in Multi-User Cellular Networks: Deep Reinforcement Learning Approaches," IEEE Transactions on Wireless Communications, 2020. doi: 10.1109/TWC.2020.3001736.   DOI
6 Siba Narayan Swain, Rahul Thakur, and C. Siva Ram Murthy, "Design and stochastic geometric analysis of an efficient Q-Learning based physical resource block allocation scheme to maximize the spectral efficiency of Device-to-Device overlaid cellular networks," Computer Networks, vol. 119, pp. 71-85, Mar. 2017.   DOI
7 Z. Fan, X. Gu, S. Nie, and M. Chen, "D2D power control based on supervised and unsupervised learning," 2017 3rd IEEE International Conference on Computer and Communications (ICCC), Chengdu, pp. 558-563, 2017.
8 X. Fang, T. Zhang, Y. Liu, and Z. Zeng, "Multi-Agent Cooperative Alternating Q-Learning Caching in D2D-Enabled Cellular Networks," 2019 IEEE Global Communications Conference (GLOBECOM), Waikoloa, HI, USA, pp. 1-6, 2019.
9 J. Tang, H. Tang, X. Zhang, K. Cumanan, G. Chen, K.-K. Wong, J. A. Chambers, "Energy Minimization in D2D-Assisted Cache-Enabled Internet of Things: A Deep Reinforcement Learning Approach," IEEE Transactions on Industrial Informatics, vol. 16, no. 8, pp. 5412-5423, Aug. 2020.   DOI
10 J. Yin, L. Li, Y. Xu, W. Liang, H. Zhang, and Z. Han, "Joint Content Popularity Prediction and Content Delivery Policy for Cache- Enabled D2D Networks: A Deep Reinforcement Learning Approach," 2018 IEEE Global Conference on Signal and Information Processing (GlobalSIP), Anaheim, CA, USA, pp. 609-613, 2018.
11 R. Li, Y. Zhao, C. Wang, X. Wang, V. C. M. Leung, X. Li, T. Taleb, "Edge Caching Replacement Optimization for D2D Wireless Networks via Weighted Distributed DQN," 2020 IEEE Wireless Communications and Networking Conference (WCNC), Seoul, Korea (South), pp. 1-6, 2020.