Journal of Korea Multimedia Society (한국멀티미디어학회논문지)

Korea Multimedia Society (한국멀티미디어학회)
권호 표지
DOI QR코드

DOI QR Code

The Effect of Segment Size on Quality Selection in DQN-based Video Streaming Services

DQN 기반 비디오 스트리밍 서비스에서 세그먼트 크기가 품질 선택에 미치는 영향

  • Kim, ISeul (School of Computer Science and Engineering, Kyungpook National University) ;
  • Lim, Kyungshik (School of Computer Science and Engineering and Software Technology Research Center, Kyungpook National University)
  • Received : 2018.08.14
  • Accepted : 2018.08.30
  • Published : 2018.10.31
Download PDF
⟨ Previous Next ⟩

Abstract

The Dynamic Adaptive Streaming over HTTP(DASH) is envisioned to evolve to meet an increasing demand on providing seamless video streaming services in the near future. The DASH performance heavily depends on the client's adaptive quality selection algorithm that is not included in the standard. The existing conventional algorithms are basically based on a procedural algorithm that is not easy to capture and reflect all variations of dynamic network and traffic conditions in a variety of network environments. To solve this problem, this paper proposes a novel quality selection mechanism based on the Deep Q-Network(DQN) model, the DQN-based DASH Adaptive Bitrate(ABR) mechanism. The proposed mechanism adopts a new reward calculation method based on five major performance metrics to reflect the current conditions of networks and devices in real time. In addition, the size of the consecutive video segment to be downloaded is also considered as a major learning metric to reflect a variety of video encodings. Experimental results show that the proposed mechanism quickly selects a suitable video quality even in high error rate environments, significantly reducing frequency of quality changes compared to the existing algorithm and simultaneously improving average video quality during video playback.

Keywords

References

  1. X. Yin, A. Jindal, V. Sekar, and B. Sinopoli, "A Control-Theoretic Approach for Dynamic Adaptive Video Streaming over HTTP," Proceeding of ACM Conference on Special Interest Group on Data Communication, pp. 325-338, 2015.
  2. T. Stockhammer, "Dynamic Adaptive Streaming over HTTP-Standards and Design Principles," Proceeding of ACM Conference on Multimedia Systems, pp. 133-144, 2011.
  3. M. Seufert, S. Egger, M. Slanina, T. Zinner, T. HoBfeld, and P. Tran-Gia, "A Survey on Quality of Experience of HTTP Adaptive Streaming," IEEE Communications Surveys & Tutorials, Vol. 17, No. 1, pp. 469-492, 2015. https://doi.org/10.1109/COMST.2014.2360940
  4. D. Zegarra Rodriguez, R. Lopes Rosa, E. Costa Alfaia, J. Issy Abrahao, and G. Bressan, "Video Quality Metric for Streaming Service Using DASH Standard," IEEE Transactions on Broadcasting, Vol. 62, No. 3, pp. 628-639, 2016. https://doi.org/10.1109/TBC.2016.2570012
  5. J. Kua, G. Armitage, and P. Branch, “A Survey of Rate Adaptation Techniques for Dynamic Adaptive Streaming Over HTTP,” IEEE Communications Surveys & Tutorials, Vol. 19, No. 3, pp. 1842-1866, 2017. https://doi.org/10.1109/COMST.2017.2685630
  6. Dash Industry Forum Reference Player Documentation(2017). https://github.com/Dash-Industry-Forum/dash.js/wiki (accessed Jan., 11, 2018).
  7. M. Claeys, S. Latre, J. Famaey, and F. De Turck, “Design and Evaluation of a Self-Learning HTTP Adaptive Video Streaming Client,” IEEE Communications Letters, Vol. 18, No. 4, pp. 716-719, 2014. https://doi.org/10.1109/LCOMM.2014.020414.132649
  8. L.P. Kaelbling, M.L. Littman, and A.W. Moore "Reinforcement Learning: A Survey," Journal of Artificial Intelligence Research, Vol. 4, pp. 237-285, 1996. https://doi.org/10.1613/jair.301
  9. C.J.C.H. Watkins and P. Dayan, "Q-learning," Machine Learning, Vol. 8, No. 3-4, pp. 279-292, 1992. https://doi.org/10.1023/A:1022676722315
  10. V. Mnih, K. Kavukcuoglu, D. Silver, A.A. Rusu, J. Veness, M.G. Bellemare, et al., "Human-level Control Through Deep Reinforcement Learning," Nature, Vol. 518, No. 7540, pp. 529-533, 2015. https://doi.org/10.1038/nature14236
  11. A.C. Dalal, D.R. Musicant, J. Olson, B. McMenamy, S. Benzaid, B. Kazez et al., "Predicting User-Perceived Quality Ratings from Streaming Media Data," Proceeding of IEEE International Conference on Communications, pp. 65-72, 2007.
  12. P. Juluri, V. Tamarapalli, and D. Medhi, "QoE Management in Dash Systems Using the Segment Aware Rate Adaptation Algorithm," Proceeding of IEEE/IFIP Network Operations and Management Symposium, pp. 129-136, 2016.
  13. T. Hossfeld, S. Egger, R. Schatz, M. Fiedler, K. Masuch, and C. Lorentzen, "Initial Delay vs. Interruptions: Between the Devil and the Deep Blue Sea," Proceeding of Fourth International Workshop on Quality of Multimedia Experience, pp. 1-6, 2012.
  14. A. Dalal, D.R. Musicant, J. Olson, B. Mc Menamy, S. Benzaid, B. Kazez et al., "Predicting User-Perceived Quality Ratings from Streaming Media Data," Proceeding of IEEE International Conference on Communications, pp. 65-67, 2007.
  15. S. Adam, L. Busoniu, and R. Babuska, "Experience Replay for Real-Time Reinforcement Learning Control," IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews), Vol. 42, No. 2, pp. 201-212, 2012. https://doi.org/10.1109/TSMCC.2011.2106494
  16. T.R. Henderson, M. Lacage, and G.F. Riley, "Network Simulations with the ns-3 Simulator," Proceeding of ACM Conference on Special Interest Group on Data Communication, pp. 17-22, 2008.
  17. H.V. Hasselt, A. Guez, and D. Silver, "Deep Reinforcement Learning with Double Q-Learning," Proceeding of AAAI Conference on Artificial Intelligence, pp. 2094-2100, 2016.
  18. P. Juluri, V. Tamarapalli, and D. Medhi, "SARA: Segment Aware Rate Adaptation Algorithm for Dynamic Adaptive Streaming over HTTP," Proceeding of IEEE International Conference on Communications, pp. 1765-1770, 2015.
  19. I. Kim, S. Hong, S. Jung, and K. Lim, "An Intelligent Video Streaming Mechanism based on a Deep Q-Network for QoE Enhancement," Journal of Korea Multimedia Society, Vol. 21, No. 2, pp. 188-198, 2018. https://doi.org/10.9717/KMMS.2018.21.2.188
  20. J. Schmidhuber, "Deep Learning in Neural Networks: An Overview," Neural Networks, Vol. 61, pp. 85-117, 2015. https://doi.org/10.1016/j.neunet.2014.09.003