ETRI Journal

Electronics and Telecommunications Research Institute (한국전자통신연구원)
권호 표지
DOI QR코드

DOI QR Code

No-reference quality assessment of dynamic sports videos based on a spatiotemporal motion model

  • Kim, Hyoung-Gook (Department of Electronics Convergence Engineering, Kwangwoon University) ;
  • Shin, Seung-Su (Department of Electronics Convergence Engineering, Kwangwoon University) ;
  • Kim, Sang-Wook (Department of Software, Chung-Ang University) ;
  • Lee, Gi Yong (Department of Electronics Convergence Engineering, Kwangwoon University)
  • Received : 2020.04.14
  • Accepted : 2020.11.11
  • Published : 2021.06.01
Download PDF
⟨ Previous Next ⟩

Abstract

This paper proposes an approach to improve the performance of no-reference video quality assessment for sports videos with dynamic motion scenes using an efficient spatiotemporal model. In the proposed method, we divide the video sequences into video blocks and apply a 3D shearlet transform that can efficiently extract primary spatiotemporal features to capture dynamic natural motion scene statistics from the incoming video blocks. The concatenation of a deep residual bidirectional gated recurrent neural network and logistic regression is used to learn the spatiotemporal correlation more robustly and predict the perceptual quality score. In addition, conditional video block-wise constraints are incorporated into the objective function to improve quality estimation performance for the entire video. The experimental results show that the proposed method extracts spatiotemporal motion information more effectively and predicts the video quality with higher accuracy than the conventional no-reference video quality assessment methods.

Keywords

Acknowledgement

The work reported in this paper was conducted during the sabbatical year of Kwangwoon University in 2018. This study was supported by the BK21 Four project (Wellness Care Fusion Technology Based on Hyper-Connected Human Experiences) funded by the Ministry of Education, Department of Electronics, Convergence Engineering, Kwangwoon University, Rep. of Korea (F20YY8101058).

References

  1. Z. Akhtar and T. H. Falk, Audio-visual multimedia quality assessment: A comprehensive survey, IEEE Access 5 (2017), 21090-21117. https://doi.org/10.1109/ACCESS.2017.2750918
  2. K. Manasa and S. S. Channappayya, An optical flow-based full reference video quality assessment algorithm, IEEE Trans. Image Process. 25 (2016), 2480-2492. https://doi.org/10.1109/TIP.2016.2548247
  3. R. Soundararajan and A. C. Bovik, Video quality assessment by reduced reference spatio-temporal entropic differencing, IEEE Trans. Circuits Syst. Video Technol. 23 (2012), 684-694. https://doi.org/10.1109/TCSVT.2012.2214933
  4. C. Keimel, T. Oelbaum, and K. Diepold, No-reference video quality evaluation for high-definition video, in Proc. IEEE Int. Conf. Acoust., Speech Signal Process. (Taipei, Taiwan), Apr. 2009, pp. 1145-1148.
  5. H. Mao, R. Netravali, and M. Alizadeh, Neural adaptive video streaming with pensieve, in Proc. Conf. ACM Special Interest Group Data Commun. (New York, NY, USA), Aug. 2017, pp. 197-210.
  6. P. Yan and X. Mou, Video quality assessment based on correlation between spatiotemporal motion energies, in Proc. Appl. Digit. Image Process. XXXIX (San Diego, CA, USA), Sept. 2016, 997130: 1-12.
  7. T. R. Goodall, A. C. Bovik, and N. G. Paulter, Tasking on natural statistics of infrared images, IEEE Trans. Image Process. 25 (2016), 65-79. https://doi.org/10.1109/TIP.2015.2496289
  8. K. Seshadrinathan and A. C. Bovik, Motion tuned spatio-temporal quality assessment of natural videos, IEEE Trans. Image Process. 19 (2010), 335-350. https://doi.org/10.1109/TIP.2009.2034992
  9. M. A. Saad and A. C. Bovik, Blind quality assessment of videos using a model of natural scene statistics and motion coherency, in Proc. Conf. Rec. Asilomar Conf. Signals Syst. Comput. (Pacific Grove, CA, USA), Nov. 2012, pp. 332-336.
  10. M. T. Vega et al., Deep learning for quality assessment in live video streaming, IEEE Signal Process. Lett. 24 (2017), 736-740. https://doi.org/10.1109/LSP.2017.2691160
  11. Y. Li et al., No-reference video quality assessment with 3D shearlet transform and convolutional neural networks, IEEE Trans. Circuits Syst. Video Technol. 26 (2016), 1044-1057. https://doi.org/10.1109/TCSVT.2015.2430711
  12. J. Xu et al., No-reference video quality assessment via feature learning, in Proc. IEEE Int. Conf. Image Process. (Paris, France), Oct. 2014, pp. 491-495.
  13. S. W. Fu et al., Quality-net: An end-to-end non-intrusive speech quality assessment model based on BLSTM, in Proc. Interspeech (Hyderabad, India), Setp. 2018, pp. 1873-1877.
  14. C. Wang, L. Su, and Q. Huang, CNN-MR for no reference video quality assessment, in Proc. Int. Conf. Inf. Sci. Control Eng. (Changsha, China), July 2017, pp. 224-228.
  15. C. G. Bampis et al., Recurrent and dynamic models for predicting streaming video quality of experience, IEEE Trans. Image Process. 27 (2018), 3316-3331. https://doi.org/10.1109/TIP.2018.2815842
  16. Q. Bao, R. Huang, and X. Wei, Video quality assessment based on the improved LSTM model, in Image and Graphics, vol. 10667, Springer, Cham, Switzerland, 2017, pp. 313-324. https://doi.org/10.1007/978-3-319-71589-6_28
  17. M. Zhang et al., Deep residual-network-based quality assessment for SD-OCT retinal images: Preliminary study, in Proc. Medical Imaging 2019: Image Percept., Obs. Perform., Technol. Assess. (San Diego, CA, USA), Mar. 2019, 095214: 1-6.
  18. D. Nilsson and C. Sminchisescu, Semantic video segmentation by gated recurrent flow propagation, in Proc. IEEE Conf. Comput. Vis. Pattern Recognit. (Salt Lake City, UT, USA), June 2018, pp. 6819-6828.
  19. P. V. Vu and D. M. Chandler, ViS3: An algorithm for video quality assessment via analysis of spatial and spatiotemporal slices, J. Electron. Imaging 23 (2014), no. 1, article no. 01316.
  20. V. Hosu et al., The konstanz natural video database (KoNViD-1k), in Proc. Int. Conf. Qual. Multimedia Exper. (QoMEX) (Erfurt, Germany), June 2017, pp. 1-6.
  21. J. Sogaard, S. Forchhammer, and J. Korhonen, No-reference video quality assessment using codec analysis, IEEE Trans. Circuits Syst. Video Technol. 25 (2015), 1637-1650. https://doi.org/10.1109/TCSVT.2015.2397207
  22. K. Zhu et al., No-reference video quality assessment based on artifact measurement and statistical analysis, IEEE Trans. Circuits Syst. Video Technol. 25 (2014), 533-546. https://doi.org/10.1109/TCSVT.2014.2363737