Browse > Article
http://dx.doi.org/10.6109/jkiice.2020.24.12.1604

Light Field Angular Super-Resolution Algorithm Using Dilated Convolutional Neural Network with Residual Network  

Kim, Dong-Myung (School of Electronics Engineering, Chung-Buk National University)
Suh, Jae-Won (School of Electronics Engineering, Chung-Buk National University)
Publication Information
Journal of the Korea Institute of Information and Communication Engineering / v.24, no.12, 2020 , pp. 1604-1611 More about this Journal
Abstract
Light field image captured by a microlens array-based camera has many limitations in practical use due to its low spatial resolution and angular resolution. High spatial resolution images can be easily acquired with a single image super-resolution technique that has been studied a lot recently. But there is a problem in that high angular resolution images are distorted in the process of using disparity information inherent among images, and thus it is difficult to obtain a high-quality angular resolution image. In this paper, we propose light field angular super-resolution that extracts an initial feature map using an dilated convolutional neural network in order to effectively extract the view difference information inherent among images and generates target image using a residual neural network. The proposed network showed superior performance in PSNR and subjective image quality compared to existing angular super-resolution networks.
Keywords
Light-field; Super-resolution; Deep learning; View synthesis;
Citations & Related Records
연도 인용수 순위
  • Reference
1 M. Levoy and P. Hanrahan, "Light field rendering," in Proceedings of the 23rd Annual Conference on Computer Graphics and Interactive Techniques, pp. 31-42, 1996.
2 J. Yu, "A Light-Field Journey to Virtual Reality," IEEE MultiMedia, vol. 24, no. 2, pp. 104-112, 2017.   DOI
3 Williem and I. K. Park, "Robust light field depth estimation for noisy scene with occlusion," in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 4396-4404, 2016.
4 H. Schilling, M. Diebold, C. Rother, and B. Jahne, "Trust your model: Light field depth estimation with inline occlusion handling," in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 4530-4538, 2018.
5 O. Johannsen, A. Sulc, N. Marniok, and B. Goldluecke, "Layered scene reconstruction from multiple light field camera views," in Asian Conference on Computer Vision, pp. 3-18, 2016.
6 A. Vianello, J. Ackermann, R. B. Campus, M. Diebold, and B. Jahne, "Robust hough transform based 3D reconstruction from circular light fields," in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7327-7335, 2018.
7 C. Dong, C. C. Loy, K. He, and X. Tang. "Image Super-Resolution Using Deep Convolutional Networks," IEEE transactions on pattern analysis and machine intelligence, vol. 38, no. 2, pp. 295-307, Feb. 2016.   DOI
8 B. Lim, S. Son, H. Kim, S. Nah and K. M. Lee, "Enhanced Deep Residual Networks for Single Image Super-Resolution," in Proceedings of the IEEE conference on computer vision and pattern recognition workshops, pp. 136-144, 2017.
9 M. Haris, G. Shakhnarovich, and N. Ukita, "Deep Back-Projection Networks for Super-Resolution," in Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 1664-1673, 2018.
10 N. K. Kalantari, T. C. Wang, and R. Ramamoorthi, "Learning-based view synthesis for light field cameras," ACM Transactions on Graphics, vol. 35, no. 6, pp. 193:1 -193:10, Nov. 2016.
11 A. Krizhevsky, I. Sutskever, and G. E. Hinton, "Imagenet classification with deep convolutional neural networks," in Advances in neural information processing systems, pp. 1097-1105, 2012.
12 Y. Yoon, H. G. Jeon, D. Yoo, J. Y. Lee, and I. S. Kweon, "Light-Field Image Super-Resolution Using Convolutional Neural Network," IEEE Signal Processing Letters, vol. 24, no. 6, pp. 848-852, Jun. 2017.   DOI
13 F. Yu and V. Koltun, "Multi-Scale Context Aggregation by Dilated Convolutions," in Proceedings of 4th International Conference on Learning Representations, 2016.
14 M. Rerabek and T. Ebrahimi, "New Light Field Image Dataset," in 8th International Conference on Quality of Multimedia Experience, 2016.
15 J. Zbontar and Y. Lecun, "Stereo matching by training a convolutional neural network to compare image patches," International Journal of Machine Learning Research, vol. 17, no. 1, pp. 2287-2318, Apr. 2016.