Browse > Article
http://dx.doi.org/10.4218/etrij.2020-0160

No-reference quality assessment of dynamic sports videos based on a spatiotemporal motion model  

Kim, Hyoung-Gook (Department of Electronics Convergence Engineering, Kwangwoon University)
Shin, Seung-Su (Department of Electronics Convergence Engineering, Kwangwoon University)
Kim, Sang-Wook (Department of Software, Chung-Ang University)
Lee, Gi Yong (Department of Electronics Convergence Engineering, Kwangwoon University)
Publication Information
ETRI Journal / v.43, no.3, 2021 , pp. 538-548 More about this Journal
Abstract
This paper proposes an approach to improve the performance of no-reference video quality assessment for sports videos with dynamic motion scenes using an efficient spatiotemporal model. In the proposed method, we divide the video sequences into video blocks and apply a 3D shearlet transform that can efficiently extract primary spatiotemporal features to capture dynamic natural motion scene statistics from the incoming video blocks. The concatenation of a deep residual bidirectional gated recurrent neural network and logistic regression is used to learn the spatiotemporal correlation more robustly and predict the perceptual quality score. In addition, conditional video block-wise constraints are incorporated into the objective function to improve quality estimation performance for the entire video. The experimental results show that the proposed method extracts spatiotemporal motion information more effectively and predicts the video quality with higher accuracy than the conventional no-reference video quality assessment methods.
Keywords
3D shearlet transform; conditional constraints; deep residual bidirectional gated recurrent neural network; natural scene statistics; no-reference video quality assessment;
Citations & Related Records
연도 인용수 순위
  • Reference
1 C. Wang, L. Su, and Q. Huang, CNN-MR for no reference video quality assessment, in Proc. Int. Conf. Inf. Sci. Control Eng. (Changsha, China), July 2017, pp. 224-228.
2 M. Zhang et al., Deep residual-network-based quality assessment for SD-OCT retinal images: Preliminary study, in Proc. Medical Imaging 2019: Image Percept., Obs. Perform., Technol. Assess. (San Diego, CA, USA), Mar. 2019, 095214: 1-6.
3 V. Hosu et al., The konstanz natural video database (KoNViD-1k), in Proc. Int. Conf. Qual. Multimedia Exper. (QoMEX) (Erfurt, Germany), June 2017, pp. 1-6.
4 M. A. Saad and A. C. Bovik, Blind quality assessment of videos using a model of natural scene statistics and motion coherency, in Proc. Conf. Rec. Asilomar Conf. Signals Syst. Comput. (Pacific Grove, CA, USA), Nov. 2012, pp. 332-336.
5 M. T. Vega et al., Deep learning for quality assessment in live video streaming, IEEE Signal Process. Lett. 24 (2017), 736-740.   DOI
6 J. Xu et al., No-reference video quality assessment via feature learning, in Proc. IEEE Int. Conf. Image Process. (Paris, France), Oct. 2014, pp. 491-495.
7 S. W. Fu et al., Quality-net: An end-to-end non-intrusive speech quality assessment model based on BLSTM, in Proc. Interspeech (Hyderabad, India), Setp. 2018, pp. 1873-1877.
8 H. Mao, R. Netravali, and M. Alizadeh, Neural adaptive video streaming with pensieve, in Proc. Conf. ACM Special Interest Group Data Commun. (New York, NY, USA), Aug. 2017, pp. 197-210.
9 P. Yan and X. Mou, Video quality assessment based on correlation between spatiotemporal motion energies, in Proc. Appl. Digit. Image Process. XXXIX (San Diego, CA, USA), Sept. 2016, 997130: 1-12.
10 K. Seshadrinathan and A. C. Bovik, Motion tuned spatio-temporal quality assessment of natural videos, IEEE Trans. Image Process. 19 (2010), 335-350.   DOI
11 C. G. Bampis et al., Recurrent and dynamic models for predicting streaming video quality of experience, IEEE Trans. Image Process. 27 (2018), 3316-3331.   DOI
12 D. Nilsson and C. Sminchisescu, Semantic video segmentation by gated recurrent flow propagation, in Proc. IEEE Conf. Comput. Vis. Pattern Recognit. (Salt Lake City, UT, USA), June 2018, pp. 6819-6828.
13 P. V. Vu and D. M. Chandler, ViS3: An algorithm for video quality assessment via analysis of spatial and spatiotemporal slices, J. Electron. Imaging 23 (2014), no. 1, article no. 01316.
14 J. Sogaard, S. Forchhammer, and J. Korhonen, No-reference video quality assessment using codec analysis, IEEE Trans. Circuits Syst. Video Technol. 25 (2015), 1637-1650.   DOI
15 T. R. Goodall, A. C. Bovik, and N. G. Paulter, Tasking on natural statistics of infrared images, IEEE Trans. Image Process. 25 (2016), 65-79.   DOI
16 Z. Akhtar and T. H. Falk, Audio-visual multimedia quality assessment: A comprehensive survey, IEEE Access 5 (2017), 21090-21117.   DOI
17 K. Manasa and S. S. Channappayya, An optical flow-based full reference video quality assessment algorithm, IEEE Trans. Image Process. 25 (2016), 2480-2492.   DOI
18 C. Keimel, T. Oelbaum, and K. Diepold, No-reference video quality evaluation for high-definition video, in Proc. IEEE Int. Conf. Acoust., Speech Signal Process. (Taipei, Taiwan), Apr. 2009, pp. 1145-1148.
19 R. Soundararajan and A. C. Bovik, Video quality assessment by reduced reference spatio-temporal entropic differencing, IEEE Trans. Circuits Syst. Video Technol. 23 (2012), 684-694.   DOI
20 Q. Bao, R. Huang, and X. Wei, Video quality assessment based on the improved LSTM model, in Image and Graphics, vol. 10667, Springer, Cham, Switzerland, 2017, pp. 313-324.   DOI
21 Y. Li et al., No-reference video quality assessment with 3D shearlet transform and convolutional neural networks, IEEE Trans. Circuits Syst. Video Technol. 26 (2016), 1044-1057.   DOI
22 K. Zhu et al., No-reference video quality assessment based on artifact measurement and statistical analysis, IEEE Trans. Circuits Syst. Video Technol. 25 (2014), 533-546.   DOI