Browse > Article
http://dx.doi.org/10.14372/IEMEK.2022.17.5.297

A DASH System Using the A3C-based Deep Reinforcement Learning  

Choi, Minje (Kyungpook National University)
Lim, Kyungshik (Kyungpook National University)
Publication Information
IEMEK Journal of Embedded Systems and Applications / v.17, no.5, 2022 , pp. 297-307 More about this Journal
Abstract
The simple procedural segment selection algorithm commonly used in Dynamic Adaptive Streaming over HTTP (DASH) reveals severe weakness to provide high-quality streaming services in the integrated mobile networks of various wired and wireless links. A major issue could be how to properly cope with dynamically changing underlying network conditions. The key to meet it should be to make the segment selection algorithm much more adaptive to fluctuation of network traffics. This paper presents a system architecture that replaces the existing procedural segment selection algorithm with a deep reinforcement learning algorithm based on the Asynchronous Advantage Actor-Critic (A3C). The distributed A3C-based deep learning server is designed and implemented to allow multiple clients in different network conditions to stream videos simultaneously, collect learning data quickly, and learn asynchronously, resulting in greatly improved learning speed as the number of video clients increases. The performance analysis shows that the proposed algorithm outperforms both the conventional DASH algorithm and the Deep Q-Network algorithm in terms of the user's quality of experience and the speed of deep learning.
Keywords
DASH; Deep Reinforcement Learning; A3C; Adaptive Video Streaming;
Citations & Related Records
연도 인용수 순위
  • Reference
1 T. R. Henderson, M. Lacage, G. F. Riley, "Network Simulations with the ns-3 Simulator," Proceeding of Association for Comput ing Machinery Conference on Special Interest Group on Data Communication, pp. 527, 2008.
2 M. Seufert, S. Egger, M. Slanina, T. Zinner, T. Hossfeld, P. Tran-Gia, "A Survey on Quality of Experience of HTTP Adaptive Streaming," IEEE Communications Surveys & Tutorials, Vol. 17, No. 1, pp. 469-492, 2015.   DOI
3 I. S. Kim, S. Hong, S. Jung, K. Lim, "An Intelligent Video Streaming Mechanism based on a Deep Q-Network for QoE Enhancement," Journal of Korea Multimedia Society, Vol. 21, No. 2, pp. 188-198, 2018.   DOI
4 V. Mnih, A. P. Badia, M. Mirza, A. Graves, T. Harley, T. P. Lillicrap, D. Silver, K. Kavukcuoglu, "Asynchronous Methods for Deep Reinforcement Learning," Proceedings of The 33rd International Conference on Machine Learning, PMLR 48:1928-1937, 2016.
5 T. Haarnoja, A. Zhou, P. Abbeel, S. Levine, "Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actor," Proceedings of the 35th International Conference on Machine Learning, PMLR 80:1861-1870, 2018.
6 V. Mnih, K. Kavukcuoglu, D. Silver, A. Graves, I. Antonoglou, "Playing Atari with Deep Reinforcement Learning," NIPS Deep Learning Workshop 2013, arXiv preprint arXiv:1312.5602, 2013.
7 P. Juluri, V. Tamarapalli, D. Medhi, "QoE Management in DASH Systems Using the Segment Aware Rate Adaptation Algorithm," Proceeding of IEEE/IFIP Network Operations and Management Symposium, pp. 129-136, 2016.
8 J. Kua, G. Armitage, P. Branch, "A Survey of Rate Adaptation Techniques for Dynamic Adaptive Streaming Over HTTP," IEEE Communications Surveys and Tutorials, Vol. 19, No. 3, pp. 1842-1866, 2017.   DOI
9 K. Miller, E. Quacchio, G. Gennari, A. Wolisz, "Adaptation Algorithm for Adaptive Streaming over HTTP," 2012 IEEE 19th International Packet Video Workshop, pp. 173-178, 2012.
10 I. S. Kim, K. Lim, "The Effect of Segment Size on Quality Selection in DQN-based Video Streaming Services," Journal of Korea Multimedia Society, Vol. 21, No. 10, pp. 1182-1194, 2018.   DOI