한국정보통신학회논문지 (Journal of the Korea Institute of Information and Communication Engineering)

  • 제24권12호
  • /
  • Pages.1604-1611
  • /
  • 2020
  • /
  • 2234-4772(pISSN)
  • /
  • 2288-4165(eISSN)
한국정보통신학회 (The Korea Institute of Information and Commucation Engineering)
권호 표지
DOI QR코드

DOI QR Code

잔차 신경망과 팽창 합성곱 신경망을 이용한 라이트 필드 각 초해상도 기법

Light Field Angular Super-Resolution Algorithm Using Dilated Convolutional Neural Network with Residual Network

  • Kim, Dong-Myung (School of Electronics Engineering, Chung-Buk National University) ;
  • Suh, Jae-Won (School of Electronics Engineering, Chung-Buk National University)
  • 투고 : 2020.10.16
  • 심사 : 2020.10.29
  • 발행 : 2020.12.31
PDF 다운로드
⟨ 이전 논문 다음 논문 ⟩

초록

마이크로렌즈 어레이 기반의 카메라로 촬영된 라이트필드 영상은 낮은 공간해상도 및 각해상도로 인하여 실제 사용하기에는 많은 제약이 따른다. 고해상도의 공간해상도 영상은 최근 많이 연구되고 있는 단일 영상 초해상도 기법으로 쉽게 얻을 수 있으나 고해상도의 각해상도 영상은 영상사이에 내재된 시점차 정보를 이용하는 과정에서 왜곡이 발생하여 좋은 품질의 각해상도 영상을 얻기 힘든 문제가 있다. 본 논문에서는 영상 사이에 내재된 시점차 정보를 효과적으로 추출하기 위해서 팽창 합성곱 신경망을 이용하여 초기 특징맵을 추출하고 잔차 신경망으로 새로운 시점 영상을 생성하는 라이트 필드 각 초해상도 영상 기법을 제안한다. 제안하는 네트워크는 기존의 각 초해상도 네트워크와 비교하여 PSNR 및 주관적 화질 비교에서 우수한 성능을 보였다.

Light field image captured by a microlens array-based camera has many limitations in practical use due to its low spatial resolution and angular resolution. High spatial resolution images can be easily acquired with a single image super-resolution technique that has been studied a lot recently. But there is a problem in that high angular resolution images are distorted in the process of using disparity information inherent among images, and thus it is difficult to obtain a high-quality angular resolution image. In this paper, we propose light field angular super-resolution that extracts an initial feature map using an dilated convolutional neural network in order to effectively extract the view difference information inherent among images and generates target image using a residual neural network. The proposed network showed superior performance in PSNR and subjective image quality compared to existing angular super-resolution networks.

키워드

참고문헌

  1. M. Levoy and P. Hanrahan, "Light field rendering," in Proceedings of the 23rd Annual Conference on Computer Graphics and Interactive Techniques, pp. 31-42, 1996.
  2. J. Yu, "A Light-Field Journey to Virtual Reality," IEEE MultiMedia, vol. 24, no. 2, pp. 104-112, 2017. https://doi.org/10.1109/MMUL.2017.24
  3. Williem and I. K. Park, "Robust light field depth estimation for noisy scene with occlusion," in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 4396-4404, 2016.
  4. H. Schilling, M. Diebold, C. Rother, and B. Jahne, "Trust your model: Light field depth estimation with inline occlusion handling," in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 4530-4538, 2018.
  5. O. Johannsen, A. Sulc, N. Marniok, and B. Goldluecke, "Layered scene reconstruction from multiple light field camera views," in Asian Conference on Computer Vision, pp. 3-18, 2016.
  6. A. Vianello, J. Ackermann, R. B. Campus, M. Diebold, and B. Jahne, "Robust hough transform based 3D reconstruction from circular light fields," in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7327-7335, 2018.
  7. C. Dong, C. C. Loy, K. He, and X. Tang. "Image Super-Resolution Using Deep Convolutional Networks," IEEE transactions on pattern analysis and machine intelligence, vol. 38, no. 2, pp. 295-307, Feb. 2016. https://doi.org/10.1109/TPAMI.2015.2439281
  8. B. Lim, S. Son, H. Kim, S. Nah and K. M. Lee, "Enhanced Deep Residual Networks for Single Image Super-Resolution," in Proceedings of the IEEE conference on computer vision and pattern recognition workshops, pp. 136-144, 2017.
  9. M. Haris, G. Shakhnarovich, and N. Ukita, "Deep Back-Projection Networks for Super-Resolution," in Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 1664-1673, 2018.
  10. N. K. Kalantari, T. C. Wang, and R. Ramamoorthi, "Learning-based view synthesis for light field cameras," ACM Transactions on Graphics, vol. 35, no. 6, pp. 193:1 -193:10, Nov. 2016.
  11. A. Krizhevsky, I. Sutskever, and G. E. Hinton, "Imagenet classification with deep convolutional neural networks," in Advances in neural information processing systems, pp. 1097-1105, 2012.
  12. Y. Yoon, H. G. Jeon, D. Yoo, J. Y. Lee, and I. S. Kweon, "Light-Field Image Super-Resolution Using Convolutional Neural Network," IEEE Signal Processing Letters, vol. 24, no. 6, pp. 848-852, Jun. 2017. https://doi.org/10.1109/LSP.2017.2669333
  13. F. Yu and V. Koltun, "Multi-Scale Context Aggregation by Dilated Convolutions," in Proceedings of 4th International Conference on Learning Representations, 2016.
  14. J. Zbontar and Y. Lecun, "Stereo matching by training a convolutional neural network to compare image patches," International Journal of Machine Learning Research, vol. 17, no. 1, pp. 2287-2318, Apr. 2016.
  15. M. Rerabek and T. Ebrahimi, "New Light Field Image Dataset," in 8th International Conference on Quality of Multimedia Experience, 2016.