Browse > Article
http://dx.doi.org/10.3837/tiis.2014.11.029

High-frame-rate Video Denoising for Ultra-low Illumination  

Tan, Xin (College of Information System and Management, National University of Defense Technology)
Liu, Yu (College of Information System and Management, National University of Defense Technology)
Zhang, Zheng (College of Information System and Management, National University of Defense Technology)
Zhang, Maojun (College of Information System and Management, National University of Defense Technology)
Publication Information
KSII Transactions on Internet and Information Systems (TIIS) / v.8, no.11, 2014 , pp. 4170-4188 More about this Journal
Abstract
In this study, we present a denoising algorithm for high-frame-rate videos in an ultra-low illumination environment on the basis of Kalman filtering model and a new motion segmentation scheme. The Kalman filter removes temporal noise from signals by propagating error covariance statistics. Regarded as the process noise for imaging, motion is important in Kalman filtering. We propose a new motion estimation scheme that is suitable for serious noise. This scheme employs the small motion vector characteristic of high-frame-rate videos. Small changing patches are intentionally neglected because distinguishing details from large-scale noise is difficult and unimportant. Finally, a spatial bilateral filter is used to improve denoising capability in the motion area. Experiments are performed on videos with both synthetic and real noises. Results show that the proposed algorithm outperforms other state-of-the-art methods in both peak signal-to-noise ratio objective evaluation and visual quality.
Keywords
High-frame-rate video; motion estimation; Kalman filter; ultra-low illumination; video denoising;
Citations & Related Records
연도 인용수 순위
  • Reference
1 F. Luiser, T. Blu and M. Unser, "SURE-LET for orthonormal wavelet-domain video denoising," IEEE Transactions on Circuits and Systems for Video Technology, vol. 20, no. 6, pp. 913-919, June, 2010.   DOI   ScienceOn
2 P. S. Negi and D. Labate, "3-D discrete shearlet transform and video processing," IEEE transactions on Image Processing, vol. 21, no. 6, pp. 2944-2954, June, 2012.   DOI   ScienceOn
3 M. Protter and M. Elad, "Image sequence denoising via sparse and redundant representations," IEEE transactions on Image Processing, vol. 18, no. 1, pp. 27-35, January, 2009.   DOI   ScienceOn
4 G. Varghese and Z. Wang, "Video denoising based on a spatiotemporal Gaussian scale mixture model," IEEE Transactions on Circuits and Systems for Video Technology, vol. 20, no. 7, pp. 1032-1040, July, 2010.   DOI   ScienceOn
5 M. Kim, D. Park, D. K. Han and H. Ko, "A novel framework for extremely low-light video enhancement," in Proc. of 2014 IEEE Int. Conf. Consumer Electronics, pp. 91-92, January 10-13, 2014.
6 F. Conte, A. Germani and G. Iannello, "A kalman filter approach for denoising and deblurring 3-d microscopy images," IEEE transactions on Image Processing, vol. 22, no. 12, pp. 5306-5321, December, 2013.   DOI   ScienceOn
7 M. Biloslavo, G. Ramponi, S. Olivieri and L. Albani, "Joint kalman-based noise filtering and motion compensated video coding for low bit rate videoconferencing," in Proc. of 2000 Int. Conf. Image Processing, pp. 992-995 vol. 1, September 10-13, 2000.
8 R. Dugad and N. Ahuja, "Video denoising by combing Kalman and wiener estimates," in Proc. 1999 Int. Conf. Image Processing, pp. 152-156 vol. 4, October 24-28, 1999.
9 A. Buades, B. Coll and J. Morel, "Nonlocal image and movie denoising," International Journal of Computer Vision, vol. 76, no. 2, pp. 123-139, February, 2008.   DOI   ScienceOn
10 K. Dabov, A. Foi, V. Katkovnik and K. Egiazarian, "Image denoising by sparse 3D transform-domain collaborative filtering," IEEE transactions on Image Processing, vol. 16, no. 8, pp. 2080-2095, August, 2007.   DOI   ScienceOn
11 K. Dabov, A. Foi and K. Egiazarian, "Video denoising by sparse 3-D transform-domain collaborative filtering," in Proc. of 15th European Signal Processing Conf., pp. 145-149, September 3-7, 2007.
12 M. Maggioni, G. Boracchi, A. Foi and K. Egiazarian, "Video denoising, deblocking and enhancement through separable 4-D nonlocal spatiotemporal transforms," IEEE transactions on Image Processing, vol. 21, no. 9, pp. 3952-3966, Septmeber, 2012.   DOI   ScienceOn
13 R. E. Kalman, "A new approach to linear filtering and prediction problems," Journal of Basic Engineering, vol. 82, no. 1, pp. 35-45, March, 1960.   DOI
14 X. Li and Y. Zheng, "Patch-based video processing: a variational Bayesian approach," IEEE Transactions on Circuits and Systems for Video Technology, vol. 19, no. 1, pp. 27-40, January, 2009.   DOI   ScienceOn
15 Y. Han and R. Chen, "Efficient video denoising based on dynamic nonlocal means," Image and Vision Computing, vol. 30, no. 2, pp. 78-85, February, 2012.   DOI   ScienceOn
16 H. Rabbani and S. Gazor, "Video denoising in three-dimensional complex wavelet domain using a doubly stochastic modelling," IET image processing, vol. 6, no. 9, pp. 1262-1274, December, 2012.   DOI   ScienceOn
17 Y. Kuang, L. Zhang and Z. Yi, "An adaptive rank-sparsity k-svd algorithm for image sequence denoising," Pattern Recognition Letters, vol. 45, pp. 46-54, August, 2014.   DOI   ScienceOn
18 I. W. Selesnick and K. Y. Li, "Video denoising using 2d and 3d dual-tree complex wavelet transforms," in Proc. of SPIE 5207, Wavelets: Applications in Signal and Image Processing, pp. 607-618, November 14, 2003.
19 D. Labate and P. S. Negi, "3D discrete shearlet transform and video denoising," in Proc. of SPIE 8138, Wavelets and Sparsity XIV, pp. 81381Y-81381Y-11, September, 2011.
20 G. Easley, D. Labate and W. Lim, "Sparse directional image representations using the discrete shearlet transform," Applied and Computational Harmonic Analysis, vol. 25, no. 1, pp. 25-46, July, 2008.   DOI   ScienceOn
21 C. Liu and W. T. Freeman, "A high-quality video denoising algorithm based on reliable motion estimation," in Proc. of 2010 European Conf. Computer Vision, vol. 6313, pp. 706-719, September 5-11, 2010.
22 T. Portz, L. Zhang and H. Jiang, "High-quality video denoising for motion-based exposure control," in Proc. of 2011 IEEE Int. Conf. Computer Vision Workshops, pp. 9-16, November 6-13, 2011.
23 R. C. Gonzales and R. E. Woods, Digital Image Processing, 2nd Edition, Prentice Hall, New Jersey, 2002.
24 V. Zlokolica, A. Pizurica and W. Philips, "Wavelet-domain video denoising based on reliability measures," IEEE Transactions on Circuits and Systems for Video Technology, vol. 16, no. 8, pp. 993-1007, August, 2006.   DOI   ScienceOn
25 C. Tomasi and R. Manduchi, "Bilateral filtering for gray and color images," in Proc. of 1998 6th Int. Conf. on Computer Vision, pp. 839-846, January 4-7, 1998.
26 Shearlet denoising toolbox [Online], http://www.math.uh.edu/-dlabate/software.html.
27 VBM3D denoising toolbox [Onlline], http://www.cs.tut.fi/-foi/GCF-BM3D/.
28 ST-GSM denoising toolbox [Onlline], https://ece.uwaterloo.ca/-z70wang/research/stgsm/.
29 H. Tan, F. Tian, Y. Qiu, S. Wang and J. Zhang, "Multihypothesis recursive video denoising based on separation of motion state," IET Image Processing, vol. 4, no. 4, pp. 261-268, August, 2010.   DOI   ScienceOn
30 M. Mahmoudi and G. Sapiro, "Fast image and video denoising via non-local means of similar neighborhoods," IEEE Signal Processing Letters, vol. 12, no. 12, pp. 839-842, December, 2005.   DOI   ScienceOn
31 Z. Cong, Z. Gao and X. Zhang, "A practical video denoising method based on hierarchical motion estimation," in Proc. of 2013 IEEE Int. Symposium on Broadband Multimedia Systems and Broadcasting, pp. 1-5, June 5-7, 2013.