Browse > Article
http://dx.doi.org/10.5909/JBE.2022.27.1.3

A Deep Learning-based Real-time Deblurring Algorithm on HD Resolution  

Shim, Kyujin (KAIST EE)
Ko, Kangwook (KAIST EE)
Yoon, Sungjoon (KAIST EE)
Ha, Namkoo (LIG Nex1 EO/IR R&D)
Lee, Minseok (LIG Nex1 EO/IR R&D)
Jang, Hyunsung (LIG Nex1 EO/IR R&D)
Kwon, Kuyong (LIG Nex1 EO/IR R&D)
Kim, Eunjoon (ADD)
Kim, Changick (KAIST EE)
Publication Information
Journal of Broadcast Engineering / v.27, no.1, 2022 , pp. 3-12 More about this Journal
Abstract
Image deblurring aims to remove image blur, which can be generated while shooting the pictures by the movement of objects, camera shake, blurring of focus, and so forth. With the rise in popularity of smartphones, it is common to carry portable digital cameras daily, so image deblurring techniques have become more significant recently. Originally, image deblurring techniques have been studied using traditional optimization techniques. Then with the recent attention on deep learning, deblurring methods based on convolutional neural networks have been actively proposed. However, most of them have been developed while focusing on better performance. Therefore, it is not easy to use in real situations due to the speed of their algorithms. To tackle this problem, we propose a novel deep learning-based deblurring algorithm that can be operated in real-time on HD resolution. In addition, we improved the training and inference process and could increase the performance of our model without any significant effect on the speed and the speed without any significant effect on the performance. As a result, our algorithm achieves real-time performance by processing 33.74 frames per second at 1280×720 resolution. Furthermore, it shows excellent performance compared to its speed with a PSNR of 29.78 and SSIM of 0.9287 with the GoPro dataset.
Keywords
Deblurring; Real-time; HD resolution; Deep-learning; Neural Network;
Citations & Related Records
Times Cited By KSCI : 2  (Citation Analysis)
연도 인용수 순위
1 S. Nah, T. Kim, and K. Lee, "Deep multi-scale convolutional neural net work for dynamic scene deblurring,". Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition(CVPR), Honolulu, Hawaii, USA, pp. 3883-3891, 2017.
2 P. Goyal, P. Doll'ar, R. B. Girshick, P. Noordhuis, L. Wesolowski, A. Kyrola, A. Tulloch, Y. Jia, and K. He, "Accurate, large minibatch SGD: training imagenet in 1 hour," CoRR, abs/1706.02677, 2017.
3 J. Pan, Z. Hu, Z. Su, and M. Yang, "Deblurring text images via L0-regularized intensity and gradient prior," IEEE Transactions on Pattern Analysis and Machine Intelligence(TPAMI), vol. 39, no. 2, pp. 342-355, Feb 2017.   DOI
4 T. Kim and K. Lee, "Segmentation-Free Dynamic Scene Deblurring," Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition(CVPR), Columbus, OH, USA, pp. 2766-2773, 2014
5 J. Sun, W. Cao, Z. Xu, and J. Ponce, "Learning a convolutional neural network for non-uniform motion blur removal," Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition(CVPR), Boston, MA, USA, pp. 769-777, 2015
6 T. Kim, K. Lee, B. Scholkopf, and M. Hirsch, "Online Video Deblurring via Dynamic Temporal Blending Network," Proceedings of the IEEE International Conference on Computer Vision(ICCV), Venice, Italy, pp. 4058-4067, 2017
7 X. Tao, H. Gao, X. Shen, J. Wang, and J. Jia, "Scale-recurrent network for deep image deblurring," Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition(CVPR), Salt Lake City, Utah, USA, pp. 8174-8182, 2018
8 H. Zhang, Y. Dai, H. Li, and P. Koniusz, "Deep stacked hierarchical multi-patch network for image deblurring," Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition(CVPR), Long Beach, CA, USA, pp. 5978-5986, 2019.
9 S. Nah, S. Son, and K. Lee, "Recurrent Neural Networks With Intra-Fra me Iterations for Video Deblurring," Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition(CVPR), Long Beach, CA, USA, pp. 8094-8103, 2019
10 Z. Zhong, Y. Gao, Y. Zheng, and B. Zheng. "Efficient spatio-temporal recurrent neural network for video deblurring," Proceedings of the 16th European Conference on Computer Vision(ECCV), pp. 191-207, 2020
11 K. He, X. Zhang, S. Ren, and J. Sun. "Deep residual learning for image recognition," Proceedings of the IEEE conference on Computer Vision and Pattern Recognition(CVPR), Las Vegas, Nevada, USA, pp. 770-778, 2016.
12 I. Loshchilov and F. Hutter, "Sgdr: stochastic gradient descent with rest arts," CoRR, abs/1608.03983, 2016.
13 A. L. Maas, A. Y. Hannun, and A. Y. Ng, "Rectifier nonlinearities improve neural network acoustic models," Proceedings of the 30th International Conference on Machine Learning Workshop on Deep Learning for Audio, Speech and Language Processing, Atlanta, GA, USA, 2013
14 K. Zhang, W. Zuo, Y. Chen, D. Meng, and L. Zhang, "Beyond a gaussi an denoiser: residual learning of deep CNN for image denoising," IEEE Transactions on Image Processing(TIP), vol. 26, no. 7, pp. 3142-3155, July 2017.   DOI
15 H. Zhao, O. Gallo, I. Frosio, and J. Kautz, "Loss functions for neural networks for image processing," IEEE Transactions on Computational Imaging(TCI), vol. 3, no. 1, pp. 47-57, March 2016.   DOI
16 J. Kim, J. K. Lee, and K. Lee, "Accurate image super-resolution using very deep convolutional networks," Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition(CVPR), Las Vegas, Nevada, USA, pp. 1646-1654, 2016.
17 X. Hu, W. Ren, K. Yu, K. Zhang, X. Cao, W. Liu, and B. Menze, "Pyramid Architecture Search for Real-Time Image Deblurring," Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 4298-4307, 2021.
18 W. Shi, J. Caballero, F. Huszar, J. Totz, A. P. Aitken, R. Bishop, D. Rueckert, and Z. Wang, "Real-time single image and video super-resoluti on using an efficient sub-pixel convolutional neural network," Proceed ings of the IEEE Conference on Computer Vision and Pattern Recognition(CVPR), Las Vegas, Nevada, USA, pp. 1874-1883, 2016.
19 T. He, Z. Zhang, H. Zhang, Z. Zhang, J. Xie, and M. Li, "Bag of tricks for image classification with convolutional neural networks," Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition(CVPR), Long Beach, CA, USA, pp. 558-567, 2019.
20 M. Noroozi, P. Chandramouli, and P. Favaro, "Motion deblurring in the wild," Proceedings of the 39th German Conference on Pattern Recognition(GCPR), Basel, Switzerland, pp. 65-77, 2017.