Browse > Article
http://dx.doi.org/10.3837/tiis.2021.09.003

Multicast Tree Generation using Meta Reinforcement Learning in SDN-based Smart Network Platforms  

Chae, Jihun (School of Computer Sciences and Engineering, Kyonggi University)
Kim, Namgi (School of Computer Sciences and Engineering, Kyonggi University)
Publication Information
KSII Transactions on Internet and Information Systems (TIIS) / v.15, no.9, 2021 , pp. 3138-3150 More about this Journal
Abstract
Multimedia services on the Internet are continuously increasing. Accordingly, the demand for a technology for efficiently delivering multimedia traffic is also constantly increasing. The multicast technique, that delivers the same content to several destinations, is constantly being developed. This technique delivers a content from a source to all destinations through the multicast tree. The multicast tree with low cost increases the utilization of network resources. However, the finding of the optimal multicast tree that has the minimum link costs is very difficult and its calculation complexity is the same as the complexity of the Steiner tree calculation which is NP-complete. Therefore, we need an effective way to obtain a multicast tree with low cost and less calculation time on SDN-based smart network platforms. In this paper, we propose a new multicast tree generation algorithm which produces a multicast tree using an agent trained by model-based meta reinforcement learning. Experiments verified that the proposed algorithm generated multicast trees in less time compared with existing approximation algorithms. It produced multicast trees with low cost in a dynamic network environment compared with the previous DQN-based algorithm.
Keywords
Multicast tree; Meta reinforcement learning; Multimedia routing; SDN; Deep learning;
Citations & Related Records
연도 인용수 순위
  • Reference
1 S. Sendra, A. Rego, J. Lloret, J. M. Jimenez and O. Romero, "Including Artificial Intelligence in a Routing Protocol using Software Defined Networks," in Proc. of IEEE International Conference on Communications Workshops (ICC Workshops), pp. 670-674, 2017.
2 F. K. Hwang and D. S. Richards, "Steiner tree problems," Networks, Vol. 22, No. 1, pp. 55-89, 1992.   DOI
3 T. Lee, L. Chang, W. Cheng, "Design and Implementation of SDN-based 6LBR with QoS Mechanism over Heterogeneous WSN and Internet," KSII Transactions on Internet and Information Systems, Vol. 11, No. 2, pp. 1070-1088, February 27, 2017.   DOI
4 A. Z. Zelikovsky, "An 11/6-Approximation Algorithm for the Network Steiner Problem," Algorithmica, Vol. 9, No. 5, pp. 463-470, 1993.   DOI
5 T. Hendriks, M. Camelo and S. Latre, "Q2-Routing: A QoS-Aware Q-Routing Algorithm for Wireless Ad Hoc Networks," in Proc. of IEEE International Conference on Wireless and Mobile Computing, Networking and Communications (WiMob), pp. 108-115, 2018.
6 P. Berman and V. Ramajyer, "Improved Approximations for the Steiner Tree Problem," Journal of Algorithms, Vol. 17, No. 3, pp. 381-408, 1994.   DOI
7 M. Karpinski and A. Zelikovsky, "New Approximation Algorithms for the Steiner Tree Problems," Journal of Combinatorial Optimization, Vol. 1, No. 1, pp. 47-65, 1997.   DOI
8 S.-C. Lin, I. F. Akyildiz, P. Wang and M. Luo, "QoS-Aware Adaptive Routing in Multi-Layer Hierarchical Software Defined Networks: A Reinforcement Learning Approach," in Proc. of IEEE International Conference on Services Computing (SCC), 2016.
9 H.-J. Heo, N. Kim and B.-D. Lee, "Multicast Tree Generation Technique using Reinforcement Learning in SDN Environments," IEEE SmartWorld, 2018.
10 C. Yu, J. Lan, Z. Guo and Y. Hu, "DROM: Optimizing the Routing in Software-Defined Networks with Deep Reinforcement Learning," IEEE Access, Vol. 6, pp. 64533-64539, 2018.   DOI
11 P. Pragyansmita and S. V. Raghavan, "Survey of Multicast Routing Algorithms and Protocols," in Proc. of the international conference on computer communication, Vol. 15, No. 3, p. 902. 2002.
12 YouTube, http://www.youtube.com/
13 Amazon Prime Video, http://www.primevideo.com/
14 Hulu, http://www.hulu.com
15 Jim Esch, "Software-Defined Networking: A Comprehensive Survey," Proc. of the IEEE, Vol. 103, No. 1, pp. 10-13, Jan. 2015.   DOI
16 Z. Zhao, X. Meng, S. Lu and Y. Su, "Different QoS Constraint Virtual SDN Embedding under Multiple Controllers," KSII Transactions on Internet and Information Systems, Vol. 12, No. 9, pp. 4144-4165, September 29, 2018.   DOI
17 C. Ren, S. Wang, J. Ren and X. Wang, "Traffic Engineering and Manageability for Multicast Traffic in Hybrid SDN," KSII Transactions on Internet and Information Systems, Vol. 12, No. 6, pp. 2492-2512, June 29, 2018.   DOI
18 X. Chen, J. Wu and T. Wu, "The Top-K QoS-aware Paths Discovery for Source Routing in SDN," KSII Transactions on Internet and Information Systems, Vol. 12, No. 6, pp. 2534-2553, June 29, 2018.   DOI
19 Netflix, http://www.netflix.com/
20 N. Kim and B.-D. Lee, "Analysis and Improvement of MPEG-DASH-based Internet Live Broadcasting Services in Real-world Environments," KSII Transactions on Internet and Information Systems, Vol. 13, No. 5, pp. 2544-2557, May 2019.   DOI
21 Y.-R. Chen, A. Rezapour, W. G. Tzeng and S. C. Tsai, "RL-Routing: An SDN Routing Algorithm based on Deep Reinforcement Learning," IEEE Transactions on Network Science and Engineering, Vol. 7, No. 4, pp. 3185-3199, 2020.   DOI
22 H. Takahashi and A. Matsuyama, "An Approximate Solution for the Steiner Problem in Graphs," Math. Japonica, Vol. 24, No. 6, pp. 573-577, 1980.
23 H. J. Promel and A. Steger, "RNC-Approximation Algorithms for the Steiner Problem," in Proc. of Annual Symposium on Theoretical Aspects of Computer Science, Springer, Berlin, Heidelberg, pp. 559-570, 1997.
24 Z. Yuan, P. Zhou, S. Wang and X. Zhang, "Research on Routing Optimization of SDN Network using Reinforcement Learning Method," in Proc. of IEEE International Conference on Safety Produce Information (IICSPI), pp. 443-445, 2019.
25 J. X. Wang, Z. Kurth-Nelson, D. Tirumala, H. Soyer, J. Z. Leibo, R. Munos, C. Blundell, D. Kumaran and M. Botvinick, "Learning to Reinforcement Learn," arXiv preprint arXiv:1611.05763, 2016.