KSII Transactions on Internet and Information Systems (TIIS)

Korean Society for Internet Information (한국인터넷정보학회)
권호 표지
DOI QR코드

DOI QR Code

A Fast Kernel Regression Framework for Video Super-Resolution

  • Yu, Wen-Sen (College of Computer Science, Sichuan University) ;
  • Wang, Ming-Hui (College of Computer Science, Sichuan University) ;
  • Chang, Hua-Wen (College of Computer and Communication Engineering, Zhengzhou University of Light Industry) ;
  • Chen, Shu-Qing (College of Computer Science, Sichuan University)
  • Received : 2013.04.16
  • Accepted : 2014.01.04
  • Published : 2014.01.30
Download PDF
⟨ Previous Next ⟩

Abstract

A series of kernel regression (KR) algorithms, such as the classic kernel regression (CKR), the 2- and 3-D steering kernel regression (SKR), have been proposed for image and video super-resolution. In existing KR frameworks, a single algorithm is usually adopted and applied for a whole image/video, regardless of region characteristics. However, their performances and computational efficiencies can differ in regions of different characteristics. To take full advantage of the KR algorithms and avoid their disadvantage, this paper proposes a kernel regression framework for video super-resolution. In this framework, each video frame is first analyzed and divided into three types of regions: flat, non-flat-stationary, and non-flat-moving regions. Then different KR algorithm is selected according to the region type. The CKR and 2-D SKR algorithms are applied to flat and non-flat-stationary regions, respectively. For non-flat-moving regions, this paper proposes a similarity-assisted steering kernel regression (SASKR) algorithm, which can give better performance and higher computational efficiency than the 3-D SKR algorithm. Experimental results demonstrate that the computational efficiency of the proposed framework is greatly improved without apparent degradation in performance.

Keywords

References

  1. R. Y. Tsai and T. S. Huang, "Multi-frame image restoration and registration," Adv. Comput. Vis. Image Processing, vol. 1, pp. 317-339, 1984.
  2. S. P. Kim, N. K. Bose and H. M. Valenzuela, "Recursive reconstruction of high resolution image from noisy undersampled multiframes," IEEE Trans. Acoustics, Speech, Signal Processing, vol. 38, pp. 1013-1027,Jun,1990. https://doi.org/10.1109/29.56062
  3. E.Kaltenbacher and R.C.Hardie, "High Resolution Infrared Image Reconstruction Using Multiple, Low Resolution, Aliased Frames," in Proc. of IEEE Nat. Aerospace Electronics Conf, vol.2, pp.702-709, May, 1996.
  4. M.Irani and S.Peleg, "Improving resolution by image registration," CVGIP: Graphical Models and Image Processing, vol.53, no.3, pp.231-239, May, 1991. https://doi.org/10.1016/1049-9652(91)90045-L
  5. S.Farsiu, N.D, Robinson, M.Elad and P.Milanfar, "Fast and robust multiframe super resolution," IEEE Trans. Image Processing, vol. 13, no.10, pp. 1327-1344. October, 2004. https://doi.org/10.1109/TIP.2004.834669
  6. H.Shen, L.Zhang, B.Huang and P.Li, "A MAP Approach for Joint Motion Estimation, Segmentation, and Super Resolution," IEEE Trans. Image Processing. vol.16, no.2, pp.479-490, February, 2007. https://doi.org/10.1109/TIP.2006.888334
  7. G.Chantas, N.Galatsanos and N.Woods, "Super-resolution based on fast registration and maximum a posteriori reconstruction," IEEE Trans. Image Processing, vol.16, no.7, pp.1821-1830, July, 2007. https://doi.org/10.1109/TIP.2007.896664
  8. X.Li, Y.Hu, X.Gao and D.Tao, "A multi-frame image super-resolution method," Signal Processing,vol.90, no.2, pp.405-414, February, 2010. https://doi.org/10.1016/j.sigpro.2009.05.028
  9. L.Zhang, H.Zhang, H.Shen and P.Li, "A super-resolution reconstruction algorithm for surveillance images," Signal Processing, vol.90, no.3, pp.848-859, March, 2010. https://doi.org/10.1016/j.sigpro.2009.09.002
  10. A.M.Tekalp, M.K.Ozkan and M.I.Sezan, "High-resolution image reconstruction from lower-resolution image sequences and space-varying image restoration," in Proc. IEEE Int. Conf. Acousics, Speech, Signal Processing, vol.3, pp.169-172, March, 1992.
  11. C.Fan, J.Zhu, J.Gong and C.Kuang, "POCS super-resolution sequence image reconstruction based on improvement approach of keren registration method," Intelligent Systems Design and Applications, vol.2, pp.333-337, October, 2006. https://doi.org/10.1109/ISDA.2006.253857
  12. S.Borman and R.L.Strevenson, "Super-Resolution from Image Sequences - A Review," In Proc.1998 Midwest Symp.Circuits and Systems, pp.374-378, 1999.
  13. S.C.Park, M.K.Park and M.G.Kang, "Super-resolution image reconstruction: a technical overview," IEEE Trans. Signal Processing Magazine, vol.20, no.3, pp.21-36, 2003. https://doi.org/10.1109/MSP.2003.1203207
  14. S.Farsiu, M.Elad and P.Milanfar, "Video-to-Video Dynamic Super-Resolution for Grayscale and Color Sequences," EURASIP Journal on Applied Signal Processing, vol.2006, pp.1-15.
  15. R.Hardie, "A Fast Image Super-Resolution Algorithm Using an Adaptive Wiener Filter," IEEE Trans. Image Processing, vol.16, no.12, pp.2953-2964, December, 2007. https://doi.org/10.1109/TIP.2007.909416
  16. M.M.Islam, V.K.Asari, M.N. Islam and M.A.Karim, "Super-Resolution Enhancement Technique for Low Resolution Video," IEEE Trans.Consumer Electron., vol.56, no.2, pp.919-924, May, 2010. https://doi.org/10.1109/TCE.2010.5506020
  17. S.H.Keller, F.Lauze, and M.Nielsen, "Video Super-Resolution Using Simultaneous motion and Intensity Calculations," IEEE Trans. image processing, vol.20, no.7, pp.1870-1884, July, 2011. https://doi.org/10.1109/TIP.2011.2106793
  18. C.Liu and D.Sun, "A Bayesian Approach to Adaptive Video Super Resolution," in Proc. of 2011 IEEE Conference on Computer Vision and Pattern Recognition (CVPR 2011), pp.209-216, June, 2011.
  19. A.Danielyan, A.Foi, V.Katkovnik and K.Egiazarian, "Image and video super-resolution via spatially adaptive block-matching filtering," Presented at the Int.Workshop on Local and Non- Local Approximation in Image Processing, Lausanne, Switzerland, August, 2008.
  20. M.Protter, M.Elad, H.Takeda and P.Milanfar, "Generalizing the Nonlocal-Means to Super-Resolution Reconstruction," IEEE Trans. image processing, vol.18, no.1, pp.36-51, January, 2009. https://doi.org/10.1109/TIP.2008.2008067
  21. H.Takeda, P.Milanfar, M.Protter and M.Elad, "Super-Resolution without Explicit Subpixel Motion Estimation," IEEE Trans.image processing, vol.18, no.9, pp.1958-1975, September, 2009. https://doi.org/10.1109/TIP.2009.2023703
  22. H.Takeda, S.Farsiu and P.Milanfar, "Kernel Regression for Image Processing and Reconstruction," IEEE Trans.image processing, vol.16, no.2, pp.349-366, February, 2007. https://doi.org/10.1109/TIP.2006.888330
  23. K.Zhang, G.Mu, Y.Yan, X.Gao, and D.tao, "Video super-resolution with 3D adaptive normalized convolution," Neurocomputing, vol.94, no.1, pp.140-151, October 2012. https://doi.org/10.1016/j.neucom.2012.03.012
  24. H.Zhang, J.Yang, Y.Zhang and T.Huang, "Non-Local Kernel Regression for Image and Video Restoration.," Lecture Notes in Computer Science, 2010, vol.6313, Computer Vision-ECCV 2010, pp.566-579.
  25. H.Zhang, J.Yang, Y.Zhang and T.Huang, "Image and Video Restorations via Nonlocal Kernel Regression," IEEE Transactions on cybernetics, vol.43, no.3, pp. 1035 - 1046, June, 2013. https://doi.org/10.1109/TSMCB.2012.2222375
  26. H.Su, L.Tang, Y.Wu, D.Tretter and J.Zhou, "Spatially Adaptive Block-based Super-resolution," IEEE Transactions on Image Processing, vol.21, no.3, pp.1031-1045, March, 2012. https://doi.org/10.1109/TIP.2011.2166971
  27. P.Getreuer , 2012.[Online].Available:http://www.getreuer.info/home/tvreg.
  28. Z.Wang, A.C.Bovik, H.R.Sheikh, and E.P.Simoncelli, "Image quality assessment: From error visibility to structural similarity," IEEE Transactions on Image Process, vol.13, no.4, pp.600-612, Apr.2004. https://doi.org/10.1109/TIP.2003.819861

Cited by

  1. A depth-based Multi-view Super-Resolution Method Using Image Fusion and Blind Deblurring vol.10, pp.10, 2014, https://doi.org/10.3837/tiis.2016.10.027