Browse > Article
http://dx.doi.org/10.9717/kmms.2018.21.10.1182

The Effect of Segment Size on Quality Selection in DQN-based Video Streaming Services  

Kim, ISeul (School of Computer Science and Engineering, Kyungpook National University)
Lim, Kyungshik (School of Computer Science and Engineering and Software Technology Research Center, Kyungpook National University)
Publication Information
Journal of Korea Multimedia Society / v.21, no.10, 2018 , pp. 1182-1194 More about this Journal
Abstract
The Dynamic Adaptive Streaming over HTTP(DASH) is envisioned to evolve to meet an increasing demand on providing seamless video streaming services in the near future. The DASH performance heavily depends on the client's adaptive quality selection algorithm that is not included in the standard. The existing conventional algorithms are basically based on a procedural algorithm that is not easy to capture and reflect all variations of dynamic network and traffic conditions in a variety of network environments. To solve this problem, this paper proposes a novel quality selection mechanism based on the Deep Q-Network(DQN) model, the DQN-based DASH Adaptive Bitrate(ABR) mechanism. The proposed mechanism adopts a new reward calculation method based on five major performance metrics to reflect the current conditions of networks and devices in real time. In addition, the size of the consecutive video segment to be downloaded is also considered as a major learning metric to reflect a variety of video encodings. Experimental results show that the proposed mechanism quickly selects a suitable video quality even in high error rate environments, significantly reducing frequency of quality changes compared to the existing algorithm and simultaneously improving average video quality during video playback.
Keywords
Dynamic Adaptive Streaming over HTTP(DASH); Reinforcement Learning; Deep Q-Network(DQN); Quality of Experience(QoE); Video Streaming; Deep Learning;
Citations & Related Records
Times Cited By KSCI : 1  (Citation Analysis)
연도 인용수 순위
1 M. Seufert, S. Egger, M. Slanina, T. Zinner, T. HoBfeld, and P. Tran-Gia, "A Survey on Quality of Experience of HTTP Adaptive Streaming," IEEE Communications Surveys & Tutorials, Vol. 17, No. 1, pp. 469-492, 2015.   DOI
2 D. Zegarra Rodriguez, R. Lopes Rosa, E. Costa Alfaia, J. Issy Abrahao, and G. Bressan, "Video Quality Metric for Streaming Service Using DASH Standard," IEEE Transactions on Broadcasting, Vol. 62, No. 3, pp. 628-639, 2016.   DOI
3 J. Schmidhuber, "Deep Learning in Neural Networks: An Overview," Neural Networks, Vol. 61, pp. 85-117, 2015.   DOI
4 Dash Industry Forum Reference Player Documentation(2017). https://github.com/Dash-Industry-Forum/dash.js/wiki (accessed Jan., 11, 2018).
5 M. Claeys, S. Latre, J. Famaey, and F. De Turck, “Design and Evaluation of a Self-Learning HTTP Adaptive Video Streaming Client,” IEEE Communications Letters, Vol. 18, No. 4, pp. 716-719, 2014.   DOI
6 L.P. Kaelbling, M.L. Littman, and A.W. Moore "Reinforcement Learning: A Survey," Journal of Artificial Intelligence Research, Vol. 4, pp. 237-285, 1996.   DOI
7 C.J.C.H. Watkins and P. Dayan, "Q-learning," Machine Learning, Vol. 8, No. 3-4, pp. 279-292, 1992.   DOI
8 V. Mnih, K. Kavukcuoglu, D. Silver, A.A. Rusu, J. Veness, M.G. Bellemare, et al., "Human-level Control Through Deep Reinforcement Learning," Nature, Vol. 518, No. 7540, pp. 529-533, 2015.   DOI
9 A.C. Dalal, D.R. Musicant, J. Olson, B. McMenamy, S. Benzaid, B. Kazez et al., "Predicting User-Perceived Quality Ratings from Streaming Media Data," Proceeding of IEEE International Conference on Communications, pp. 65-72, 2007.
10 P. Juluri, V. Tamarapalli, and D. Medhi, "QoE Management in Dash Systems Using the Segment Aware Rate Adaptation Algorithm," Proceeding of IEEE/IFIP Network Operations and Management Symposium, pp. 129-136, 2016.
11 H.V. Hasselt, A. Guez, and D. Silver, "Deep Reinforcement Learning with Double Q-Learning," Proceeding of AAAI Conference on Artificial Intelligence, pp. 2094-2100, 2016.
12 T. Hossfeld, S. Egger, R. Schatz, M. Fiedler, K. Masuch, and C. Lorentzen, "Initial Delay vs. Interruptions: Between the Devil and the Deep Blue Sea," Proceeding of Fourth International Workshop on Quality of Multimedia Experience, pp. 1-6, 2012.
13 J. Kua, G. Armitage, and P. Branch, “A Survey of Rate Adaptation Techniques for Dynamic Adaptive Streaming Over HTTP,” IEEE Communications Surveys & Tutorials, Vol. 19, No. 3, pp. 1842-1866, 2017.   DOI
14 A. Dalal, D.R. Musicant, J. Olson, B. Mc Menamy, S. Benzaid, B. Kazez et al., "Predicting User-Perceived Quality Ratings from Streaming Media Data," Proceeding of IEEE International Conference on Communications, pp. 65-67, 2007.
15 S. Adam, L. Busoniu, and R. Babuska, "Experience Replay for Real-Time Reinforcement Learning Control," IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews), Vol. 42, No. 2, pp. 201-212, 2012.   DOI
16 T.R. Henderson, M. Lacage, and G.F. Riley, "Network Simulations with the ns-3 Simulator," Proceeding of ACM Conference on Special Interest Group on Data Communication, pp. 17-22, 2008.
17 P. Juluri, V. Tamarapalli, and D. Medhi, "SARA: Segment Aware Rate Adaptation Algorithm for Dynamic Adaptive Streaming over HTTP," Proceeding of IEEE International Conference on Communications, pp. 1765-1770, 2015.
18 I. Kim, S. Hong, S. Jung, and K. Lim, "An Intelligent Video Streaming Mechanism based on a Deep Q-Network for QoE Enhancement," Journal of Korea Multimedia Society, Vol. 21, No. 2, pp. 188-198, 2018.   DOI
19 X. Yin, A. Jindal, V. Sekar, and B. Sinopoli, "A Control-Theoretic Approach for Dynamic Adaptive Video Streaming over HTTP," Proceeding of ACM Conference on Special Interest Group on Data Communication, pp. 325-338, 2015.
20 T. Stockhammer, "Dynamic Adaptive Streaming over HTTP-Standards and Design Principles," Proceeding of ACM Conference on Multimedia Systems, pp. 133-144, 2011.