Journal of Digital Contents Society (디지털콘텐츠학회 논문지)

Digital Contents Society (한국디지털콘텐츠학회)
권호 표지
DOI QR코드

DOI QR Code

A Study on Super Resolution Image Reconstruction for Acquired Images from Naval Combat System using Generative Adversarial Networks

생성적 적대 신경망을 이용한 함정전투체계 획득 영상의 초고해상도 영상 복원 연구

  • Received : 2018.05.30
  • Accepted : 2018.06.25
  • Published : 2018.06.30
⟨ Previous Next ⟩

Abstract

In this paper, we perform Single Image Super Resolution(SISR) for acquired images of EOTS or IRST from naval combat system. In order to conduct super resolution, we use Generative Adversarial Networks(GANs), which consists of a generative model to create a super-resolution image from the given low-resolution image and a discriminative model to determine whether the generated super-resolution image is qualified as a high-resolution image by adjusting various learning parameters. The learning parameters consist of a crop size of input image, the depth of sub-pixel layer, and the types of training images. Regarding evaluation method, we apply not only general image quality metrics, but feature descriptor methods. As a result, a larger crop size, a deeper sub-pixel layer, and high-resolution training images yield good performance.

본 논문에서는 함정전투체계의 EOTS나 IRST에서 획득한 영상을 초고해상도 영상으로 복원한다. 저해상도에서 초고해상도의 영상을 생성하는 생성 모델과 이를 판별하는 판별 모델로 구성된 생성적 적대 신경망을 이용하고, 다양한 학습 파라미터의 변화를 통한 최적의 값을 제안한다. 실험에 사용되는 학습 파라미터는 crop size와 sub-pixel layer depth, 학습 이미지 종류로 구성되며, 평가는 일반적인 영상 품질 평가 지표에 추가적으로 특징점 추출 알고리즘을 함께 사용하였다. 그 결과, Crop size가 클수록, Sub-pixel layer depth가 깊을수록, 고해상도의 학습이미지를 사용할수록 더 좋은 품질의 영상을 생성한다.

Keywords

References

  1. Linwei Yue, Huanfeng Shen, Jie Li, Qiangqiang Yuan, Hongyan Zhang, Liangpei Zhang, "Image super-resolution: The techniques, applications, and future," Signal Processing, Vol. 128, pp. 389-408, 2016. https://doi.org/10.1016/j.sigpro.2016.05.002
  2. Goodfellow, I., Pouget-Abadie, J., Mirza, M., Xu, B., Warde-Farley, D., Ozair, S., ... & Bengio, Y., "Generative adversarial nets," In Advances in neural information processing systems , pp. 2672-2680, 2014.
  3. Tsai, R. Y., "Multiframe image restoration and registration," Advance Computer Visual and Image Processing, Vol. 1, pp. 317-339, 1984..
  4. Villena, Salvador, et al., "Bayesian super-resolution image reconstruction using an ${\el}1$ prior," Image and Signal Processing and Analysis, 2009.
  5. Irani, Michal, and Shmuel Peleg., "Improving resolution by image registration," CVGIP: Graphical models and image processing, Vol. 53(3), pp. 231-239, 1991. https://doi.org/10.1016/1049-9652(91)90045-L
  6. D. Lee, Ho S. Lee, K. Lee, H. L, "Fast Very Deep Convolutional Neural Network with Deconvolution for Super-Resolution," Journal of Korea Multimedia Society, Vol 20(11), pp. 1750-1758, November 2017. https://doi.org/10.9717/KMMS.2017.20.11.1750
  7. Ledig, Christian, et al., "Photo-realistic single image super-resolution using a generative adversarial network." arXiv preprint, 2016.
  8. Reed, Scott, et al., "Generative adversarial text to image synthesis," arXiv preprint arXiv:1605.05396, 2016.
  9. Lotter, William, Gabriel Kreiman, and David Cox, "Deep predictive coding networks for video prediction and unsupervised learning," arXiv preprint arXiv:1605.08104, 2016.
  10. Lowe, David G., "Distinctive image features from scale-invariant keypoints." International journal of computer vision, Vol. 60(2), pp.91-110, 2014. https://doi.org/10.1023/B:VISI.0000029664.99615.94
  11. Bay, Herbert, Tinne Tuytelaars, and Luc Van Gool., "Surf: Speeded up robust features." European conference on computer vision. Springer, Berlin, Heidelberg, 2006.
  12. Rublee, Ethan, et al., "ORB: An efficient alternative to SIFT or SURF," Computer Vision (ICCV), 2011 IEEE international conference on. IEEE, 2011.
  13. Leutenegger, Stefan, Margarita Chli, and Roland Y. Siegwart, "BRISK: Binary robust invariant scalable keypoints," Computer Vision (ICCV), 2011 IEEE International Conference on. IEEE, 2011.
  14. Alahi, Alexandre, Raphael Ortiz, and Pierre Vandergheynst, "Freak: Fast retina keypoint," Computer vision and pattern recognition (CVPR), 2012 IEEE conference on. IEEE, 2012.
  15. He, Kaiming, et al., "Deep residual learning for image recognition," Proceedings of the IEEE conference on computer vision and pattern recognition. 2016.
  16. Ioffe, Sergey, and Christian Szegedy, "Batch normalization: Accelerating deep network training by reducing internal covariate shift," arXiv preprint arXiv:1502.03167, 2015.
  17. Shi, Wenzhe, et al., "Real-time single image and video super-resolution using an efficient sub-pixel convolutional neural network," Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2016.
  18. Agustsson, Eirikur, and Radu Timofte, "Ntire 2017 challenge on single image super-resolution: Dataset and study," The IEEE Conference on Computer Vision and Pattern Recognition (CVPR) Workshops. Vol. 3, 2017.
  19. Lynen, Jan Portmann Simon, "Ethz thermal infrared dataset," 2014.
  20. DDG-1000 Zumwalt image source [Internet]. Available: http://www.navy.mil/ah_online/zumwalt/images/gallery/151207-N-ZZ999-505%20cropped.jpg