Browse > Article
http://dx.doi.org/10.3837/tiis.2011.09.007

Stereo Image Quality Assessment Using Visual Attention and Distortion Predictors  

Hwang, Jae-Jeong (Dept. of Radiocommunication Eng., Kunsan National University)
Wu, Hong Ren (School of Electrical and Computer Eng., RMIT University)
Publication Information
KSII Transactions on Internet and Information Systems (TIIS) / v.5, no.9, 2011 , pp. 1613-1631 More about this Journal
Abstract
Several metrics have been reported in the literature to assess stereo image quality, mostly based on visual attention or human visual sensitivity based distortion prediction with the help of disparity information, which do not consider the combined aspects of human visual processing. In this paper, visual attention and depth assisted stereo image quality assessment model (VAD-SIQAM) is devised that consists of three main components, i.e., stereo attention predictor (SAP), depth variation (DV), and stereo distortion predictor (SDP). Visual attention is modeled based on entropy and inverse contrast to detect regions or objects of interest/attention. Depth variation is fused into the attention probability to account for the amount of changed depth in distorted stereo images. Finally, the stereo distortion predictor is designed by integrating distortion probability, which is based on low-level human visual system (HVS), responses into actual attention probabilities. The results show that regions of attention are detected among the visually significant distortions in the stereo image pair. Drawbacks of human visual sensitivity based picture quality metrics are alleviated by integrating visual attention and depth information. We also show that positive correlation with ground-truth attention and depth maps are increased by up to 0.949 and 0.936 in terms of the Pearson and the Spearman correlation coefficients, respectively.
Keywords
Image quality assessment; stereo image processing; visual attention; 3D depth; distortion predictor;
Citations & Related Records

Times Cited By Web Of Science : 1  (Related Records In Web of Science)
Times Cited By SCOPUS : 2
연도 인용수 순위
  • Reference
1 S. Daly, "The Visible Differences Predictor: An Algorithm for the Assessment of Image Fidelity," Digital Image and Human Vision, Cambridge, MIT press, pp. 179-206, 1993.
2 B.A. Wandell, Foundations of vision, Sinauer Associates, Inc. Pub., 1995.
3 L. Itti, C. Koch, E. Niebur, "A Model of Saliency-based Visual Attention for Rapid Scene Analysis", IEEE Trans. PAMI, vol. 20, no. 11, pp. 1254-1259, 1998.   DOI   ScienceOn
4 C. Zitnick, T. Kanade, "A Cooperative Algorithm for Stereo Matching and Occlusion Detection, Robotics Institute Tech. Report, CMU-RI-TR-99-35, Carnegie Mellon University, Oct. 1999.
5 A.B. Watson, J. Hu, J.F. McGowan, "Digital Video Quality Metric based on Human Vision", J. of Electronic Imaging, vol. 10, no. 1, pp. 20-29, 2001.   DOI   ScienceOn
6 Adobe Creative Team, "Adobe Photoshop CS4 classroom in a book," Adobe Press, 2008.
7 A.B. Poirson, B.A. Wandell, "Appearance of Colored Patterns: Pattern-Color Separability," J. Opt. Soc. Am. A, vol. 10, no. 12, pp. 2458-2470, Dec. 1993.   DOI   ScienceOn
8 CCIR, "Encoding Parameters of Digital Television for Studios," CCIR Recommendation 601-2, Int. Radio Consult. Committee, Geneva, 1990.
9 W.H. Press, S.A. Teukolsky, W.T. Vetterling, B.P. Flannery, "Numerical recipes : the art of scientific computing," Ch. 14, Cambridge University Press, 2007.
10 E. Peli, "Contrast in complex images," J. Opt. Soc. Am. A, vol. 7, no. 10, pp. 2032-2040, Oct. 1990.   DOI
11 D.G. Lowe, "Object Recognition from Local Scale-Invariant Features," in Proc. of Int. Conf. on Computer Vision, vol. 2, pp. 1150-1157, 1999.
12 M.A. Fischler, R.C. Bolles, "Random Sample Consensus: A Paradigm for Model Fitting with Applications to Image Analysis and Automated Cartography," Comm. of the ACM, vol. 24, pp. 381-395, June 1981.   DOI   ScienceOn
13 A.M. Treisman, G. Gelade, "A feature-integration theory of attention," Cognitive Psychology., vol. 12, pp. 97-136, 1980.   DOI   ScienceOn
14 A. Boev, et al., "Modelling of the stereoscopic HVS," MOBILE 3DTV Technical Report D5.3, Apr. 2009.
15 Y. Zhang, et al., "Stereoscopic Visual Attention Model for 3D Video," Lecture Notes in Computer Sci., vol. 5916, pp. 314-324, Dec. 2009.
16 L. Itti, C. Koch, "Computational Modeling of Visual Attention," Nature Rev. Neuroscience, vol. 2, no. 11, pp. 194-203, Mar. 2001.   DOI
17 J.W. Crabtree, et al., "Contributions of Y- and W-cell Pathways to Response Properties of Cat Superior Colliculus Neurons: Comparison of Antibody- and Deprivation-induced Alterations," J. Neurophysiol., vol. 56, no. 4, pp. 1157-1173, 1986.   DOI
18 M. Mancas, B. Gosselin, B. Macq, "A Three-level Computational Attention model," in Proc. of ICVS Workshop on Comput. Attention & Appl., 2007.
19 A.K. Jain, Fundamentals of digital image processing, Prentice Hall, 1989.
20 Y. Zhai, M. Shah, "Visual Attention Detection in Video Sequences using Spatiotemporal Cues," in Proc. the 14th ACM Int. Conf. on Multimedia, pp. 815-824, Dec. 2006.
21 P. Seuntiënsa, L. Meestersa, W. IJsselsteijna, "Perceptual Evaluation of JPEG Coded Stereoscopic Images," SPIE Stereoscopic displays and virtual reality systems, vol. 5006, pp. 215-226, Jan. 2003.
22 ICIP2010 Special Session, WP.L1, 3D Video Quality Assessments, Hong Kong, Sept. 26-29, 2010.
23 Z.M.P. Sazzad, et al., "Stereoscopic Image Quality Prediction," in Proc. of Int. Work. on Quality of Multimedia Experience, pp. 180-185, July 2009.
24 D. Huang, M. Yu, Y. Yang, "Image Evaluation Algorithm for Right View of Stereoscopic Video," in Proc. of Int. Conf. on Signal Processing, pp. 1051-1054, Oct. 2008.
25 A. Smolic, et al., "Coding algorithms for 3DTV- A Survey," IEEE Trans. on Circuits and Systems for Video Technol., vol. 17, no. 11, pp. 1606-1620, 2007.   DOI
26 F. Lu, et al., "Quality Assessment of 3D Asymmetric View Coding using Spatial Frequency Dominance Model," in Proc. of 3DTV Conference: The True Vision - Capture, Transmission and Display of 3D Video, pp. 1-4, 2009.
27 M.G. Perkins, "Data Compression of Stereopairs," IEEE Trans. on Communication, vol. 40, no. 4, pp. 684-696, 1992.   DOI   ScienceOn
28 P.W. Gorley, N.S. Holliman, "Stereoscopic image quality metrics and compression," in Proc. of SPIE-IS&T Electronic Imaging, Stereoscopic Displays and Virtual Reality Systems, vol. 6803, Jan. 2008.
29 M. Ferre, R. Aracil, M. Sanchez-Uran, "Stereoscopic human interfaces," IEEE Robotics & Automation Mag., vol. 15, no. 4, pp. 50-57, Dec. 2008.   DOI
30 A. Awawdeh, G. Fan, "Pseudocepstrum for Assessing Stereo Quality of Retinal Images," in Proc. of Asilomar Conf. on Signals, Systems and Computers, vol. 2, pp. 1953-1957, Nov. 2003.
31 W.A. IJsselsteijn, H. de Ridder, J. Vliegen, "Subjective Evaluation of Stereoscopic Images: Effects of Camera Parameters and Display Duration," IEEE Trans. on Circuits and Systems for Video Technol., vol. 10, no. 2, pp. 225-233, Mar. 2000.   DOI   ScienceOn
32 J. You, L. Xing, A. Perkis, X. Wang, "Perceptual Quality Assessment for Stereoscopic Images based on 2D Image Quality Metrics and Disparity Analysis," in Proc. of 15th Int. Workshop on Video Processing and Quality Metrics for Consumer Electronics (VPQM), Jan. 13-15, 2010.
33 A. Benoit, et al., "Quality Assessment of Stereoscopic Images," EURASIP J. on Image and Video Process., Special issue on 3D Image and Video Process., vol. 2008, pp. 1-13, 2008.
34 Z. Wang, et al., "Image Quality Assessment: From Error Visibility to Structural Similarity," IEEE Trans. on Image Processing, vol. 13, no. 4, pp. 600-612, 2004.   DOI   ScienceOn
35 M. Carnec, P. Le Callet, D. Barba, "An Image Quality Assessment Method based on Perception of Structural Information," in Proc. of the IEEE Int. Conf. on Image Processing (ICIP '03), vol. 2, pp. 185-188, Sept. 2003.
36 P. Campisi, P.L. Callet, E. Marini, "Stereoscopic Images Quality Assessment," in Proc. of 15th European Signal Process. Conf. (EUSIPCO), pp. 2110-2114, Sep., 2007.
37 C.T.E.R. Hewage, et al., "Quality Evaluation of Color Plus Depth Map-based Stereoscopic Video," IEEE J. of Sel. Topics in Signal Process., vol. 3, no. 2, pp. 304-318, Apr. 2009.   DOI
38 L. Shen, J. Yang, Z. Zhang, "Quality Assessment of Stereo Images with Stereo Vision," Int. Congress on Image and Signal Processing, pp. 1-4, 2009.
39 J. Yang et al., "Objective Quality Assessment Method of Stereo Images," in Proc. of 3DTV Conference: The True Vision - Capture, Transmission and Display of 3D Video, pp. 1-4, 2009.
40 H. Shao, X. Cao, G. Er, "Objective Quality Assessment of Depth Image based Rendering in 3DTV System," in Proc. of 3DTV Conference: The True Vision - Capture, Transmission and Display of 3D Video, pp. 1-4, 2009.
41 ITU-R, Recommendation BT.500-11, "Methodology for the Subjective Assessment of the Quality of Television Pictures," 2002.
42 ITU-T, Recommendation P.910, "Subjective Video Quality Assessment Methods for Multimedia Applications," Apr. 2008.
43 ITU-T, Recommendation J.144, "Objective Perceptual Video Quality Measurement Techniques for Digital Cable Television in the Presence of a Full Reference," Geneva, Mar. 2004.
44 J. Caviedes, F. Oberti, "No-reference Quality Metric for Degraded and Enhanced Video," in Proc. SPIE, vol.5150, pp. 621-632, July 2003.
45 G. Sun, N.S. Holliman, "Evaluating Methods for Controlling Depth Perception in Stereoscopic Cinematography," Stereoscopic Displays and Virtual Reality Systems, SPIE, vol. 7237, Jan. 2009.
46 M.H. Pinson, S. Wolf, "A New Standardized Method for Objectively Measuring Video Quality," IEEE Trans. on Broadcasting, vol. 50, no. 3, pp. 312-322, Sep. 2004.   DOI   ScienceOn
47 F. Yang, S. Wan, Q. Xie, H.R. Wu, "No-reference Quality Assessment for Networked Video via Primary Analysis of Bit Stream," IEEE Trans. on Circuits and Sys. for Video Tech.., vol. 20, no. 11, pp. 1544-1554, Nov. 2010.   DOI
48 S. Narkhede, F. Golshani, "Stereoscopic imaging: a real-time, in depth look," IEEE Potentials, vol. 23, no. 1, pp. 38-42, Feb.-Mar. 2004.   DOI   ScienceOn
49 L.M.J. Meesters, W.A. IJsselsteijn, P.J.H. Seuntiens, "A Survey of Perceptual Evaluations and Requirements of Three-dimensional TV," IEEE Trans. on Circuits and Systems for Video Technol., vol. 14, no. 3, pp. 381-391, Mar. 2004.   DOI   ScienceOn
50 A. Kubota, et al. "Multiview Imaging and 3DTV," IEEE Signal Processing Mag., vol. 24, no. 6, pp. 10-21, Nov. 2007.