Browse > Article
http://dx.doi.org/10.13067/JKIECS.2021.16.6.1153

A Routing Algorithm based on Deep Reinforcement Learning in SDN  

Lee, Sung-Keun (Dept. Multimedia Eng., Sunchon National University)
Publication Information
The Journal of the Korea institute of electronic communication sciences / v.16, no.6, 2021 , pp. 1153-1160 More about this Journal
Abstract
This paper proposes a routing algorithm that determines the optimal path using deep reinforcement learning in software-defined networks. The deep reinforcement learning model for learning is based on DQN, the inputs are the current network state, source, and destination nodes, and the output returns a list of routes from source to destination. The routing task is defined as a discrete control problem, and the quality of service parameters for routing consider delay, bandwidth, and loss rate. The routing agent classifies the appropriate service class according to the user's quality of service profile, and converts the service class that can be provided for each link from the current network state collected from the SDN. Based on this converted information, it learns to select a route that satisfies the required service level from the source to the destination. The simulation results indicated that if the proposed algorithm proceeds with a certain episode, the correct path is selected and the learning is successfully performed.
Keywords
QoS Aware Routing Algorithm; Software Defined Network; Deep Reinforcement Learning; QoS Profile;
Citations & Related Records
연도 인용수 순위
  • Reference
1 A. Yassine, H. Rahimi, and S. Shirmohammadi, ''Software defined network traffic measurement: Current trends and challenges,'' IEEE Instrum. Meas. Mag., vol. 18, no. 2, 2015, pp. 42-50.   DOI
2 V. Mnih, K. Kavukcuoglu and D. Silver, "Human-level control through deep reinforcement learning," Nature, vol. 518, no. 7540, 2015, pp. 529 - 535.   DOI
3 G. Stampa, M. Arias and A. Cabellos, "A deep-reinforcement learning approach for software defined networking routing optimization," arXiv preprint arXiv:1709.07080, 2017.
4 A. Valadarsky, M. Schapira, and A. Tamar, "Learning to route with deep reinforcement learning," in NIPS Deep Reinforcement Learning Symposium, 2017.
5 S. Q. Jalil, M. Husain Rehmani and S. Chalup, "DQR: Deep Q-Routing in Software Defined Networks," 2020 International Joint Conference on Neural Networks (IJCNN), 2020, pp. 1-8.
6 N. Feamste, J. Rexford, and E. Zegura, "The Road to SDN: An Intellectual History of Programmable Networks," SIGCOMM Comput. Commun. Rev., vol. 44, no. 2, 2014, pp. 87-98.
7 W. Guck, A. V. Bemten, M. Reisslein, and W. Kellerer, "Unicast QoS routing algorithms for SDN: A comprehensive survey and performance evaluation," IEEE Communications Surveys Tutorials, vol. 20, no. 1, 2018, pp. 388-415.   DOI
8 B. Guo, X. Zhang, Y. Wang, and H. Yang, "Deep Q-network based multimedia multi-service QoS optimization for mobile edge computing systems," IEEE Access, vol. 7, 2019, pp. 160961-160972.   DOI
9 N. C. Luong et al., "Applications of Deep Reinforcement Learning in Communications and Networking: A Survey," in IEEE Communications Surveys & Tutorials, vol. 21, no. 4, 2019, pp. 3133-3174   DOI
10 M. Karakus and A. Durresi, "Quality of service in software defined networking: A survey," Journal of Network and Computer Applications, vol. 80, 2017, pp. 200-218.   DOI
11 S.-C. Lin, I. F. Akyildiz, P. Wang, and M. Luo, "QoS-aware adaptive routing in multi-layer hierarchical software defined networks: A rein- forcement learning approach," in Services Computing (SCC), 2016 IEEE International Conference on. IEEE, 2016, pp. 25-33.
12 S. Lee, "Design and Application of LoRa-based Network Protocol in IoT Networks," J. of the Korea Institute of Electronic Communication Sciences, vol. 14, no. 6, 2019, pp. 1089-1096.   DOI
13 Xu, J. Tang, C. Yin, Y. Wang, and G. Xue, "Experience-driven congestion control: When multi-path tcp meets deep reinforcement learning," IEEE Journal on Selected Areas in Communications, vol. 37, no. 6, 2019, pp. 1325-1336.   DOI
14 S. Xiao, D. He, and Z. Gong, "Deep-q: Traffic-driven qos inference using deep generative network," in Proceedings of the 2018 Workshop on Network Meets AI & ML, 2018, pp. 67-73.
15 S. Q. Jalil, M. Husain Rehmani and S. Chalup, "DQR: Deep Q-Routing in Software Defined Networks," 2020 International Joint Conference on Neural Networks (IJCNN), 2020, pp. 1-8.
16 S. Jung and Lee, "A Queue Management Mechanism for Service groups based on Deep Reinforcement Learning," J. of the Korea Institute of Electronic Communication Sciences, vol. 15, no. 6, 2020, pp. 1099-1104.   DOI
17 X. Huang, T. Yuan, G. Qiao and Y. Ren, "Deep reinforcement learning for multimedia traffic control in software defined networking," IEEE Network, vol. 32, no. 6, 2018, pp. 35-41.   DOI