Journal of the Institute of Electronics and Information Engineers (전자공학회논문지)

The Institute of Electronics and Information Engineers (대한전자공학회)
권호 표지
DOI QR코드

DOI QR Code

Infrared Image Sharpness Enhancement Method Using Super-resolution Based on Adaptive Dynamic Range Coding and Fusion with Visible Image

적외선 영상 선명도 개선을 위한 ADRC 기반 초고해상도 기법 및 가시광 영상과의 융합 기법

  • Received : 2016.07.15
  • Accepted : 2016.10.27
  • Published : 2016.11.25
Download PDF
⟨ Previous Next ⟩

Abstract

In general, infrared images have less sharpness and image details than visible images. So, the prior image upscaling methods are not effective in the infrared images. In order to solve this problem, this paper proposes an algorithm which initially up-scales an input infrared (IR) image by using adaptive dynamic range encoding (ADRC)-based super-resolution (SR) method, and then fuses the result with the corresponding visible images. The proposed algorithm consists of a up-scaling phase and a fusion phase. First, an input IR image is up-scaled by the proposed ADRC-based SR algorithm. In the dictionary learning stage of this up-scaling phase, so-called 'pre-emphasis' processing is applied to training-purpose high-resolution images, hence better sharpness is achieved. In the following fusion phase, high-frequency information is extracted from the visible image corresponding to the IR image, and it is adaptively weighted according to the complexity of the IR image. Finally, a up-scaled IR image is obtained by adding the processed high-frequency information to the up-scaled IR image. The experimental results show than the proposed algorithm provides better results than the state-of-the-art SR, i.e., anchored neighborhood regression (A+) algorithm. For example, in terms of just noticeable blur (JNB), the proposed algorithm shows higher value by 0.2184 than the A+. Also, the proposed algorithm outperforms the previous works even in terms of subjective visual quality.

일반적으로 적외선 열화상 영상은 가시광선 영상보다 약한 선명도를 가지며, 디테일 정보도 거의 없다. 그래서 종래 영상확대 알고리즘 방법으로 적외선 영상을 확대할 경우 가시광 영상에 비해 효과적이지 않다. 이런 문제점을 해결하기 위해 본 논문은 입력 적외선 영상을 ADRC 기반 초고해상도 기법으로 일차적으로 확대하고, 대응하는 가시광선 영상과 융합하는 방법을 제안한다. 제안하는 알고리즘은 크게 확대 과정과 융합 과정으로 나뉜다. 먼저 입력된 적외선 영상을 ADRC 기반의 초고해상도 알고리즘으로 확대한다. 사전의 학습과정에서 고해상도 영상들에 소위 pre-emphasis를 적용한 후 학습을 함으로써 선명도 향상을 꾀했다. 융합 과정에서는 먼저 입력 IR영상과 대응하는 가시광선 영상에서 고주파 정보를 추출하고, IR영상의 복잡도에 따라 적응적으로 상기 추출된 고주파 정보를 합성하는 방식으로 최종적인 확대 적외선 영상이 얻어진다. 모의 실험 결과 제안 알고리즘은 최신 SR기법 중 하나인 A+기법보다 JNB수치가 평균 0.2184만큼 높은 우수한 정량적 결과를 보인다. 뿐만 아니라 주관적 화질에서도 상당한 우위를 보인다.

Keywords

References

  1. Freedman et al., "Image and video upscaling from local self-examples", ACM T. Graphics, vol. 30, no. 2, 2011.
  2. C. Dong, C. C. Loy, K. He, and X. Tang. "Learning a deep convolutional network for image super-resolution," Proc. ECCV, 2014.
  3. C. F. Granda and E. J. Candes, "Super-resolution via transform-invariant group-sparse regularization," Proc. ICCV, pp. 3336-3343, 2013.
  4. H. Chang, D. Y. Yeung and Y. Xiong. "Superresolution through neighbor embedding," Proc. CVPR, pp. 275-282, 2004.
  5. S. Farsiu, M. D. Robinson, M. Elad, and P. Milanfar, "Fast and robust multiframe super resolution," IEEE Trans. Image Proc., vol. 13, no. 10, pp. 1327-1344, Oct. 2004.
  6. J. Yang, J. Wright, T. Huang, and Y. Ma, "Image superresolution via sparse representation", IEEE Trans. Image Proc., vol. 19, no. 11, pp. 2861-2873, 2010.
  7. R. Timofte, V. De and L. V. Gool, "Anchored neighborhood regression for fast example-based super-resolution", Proc. ICCV, pp. 1920-1927, 2013.
  8. I. H. Choi, Y. Nam, and B. C. Song, "A contentadaptive sharpness enhancement algorithm using 2D FIR filters trained by pre-emphasis," J. Vis. Commun. Image R., pp. 579-591, April 2013.
  9. K. Choi, C. Kim, M. H. Kang, and J. B. Ra, "Resolution improvement of infrared images using visible image information," IEEE Signal Processing Letters, vol. 18, no. 10, 2011.
  10. S. Yin, L. Cao, Q. Tan and G. Jin, "Infrared and visible image fusion based on NSCT and fuzzy logic," Proc. IEEE International Conference on Mechatronics and Automation, 2010.
  11. A. Gyaourova, G. Bebis and I. Pavlidis, "Fusion of infrared and visible images for face recognition," Proc. ECCV, vol. 3024, pp. 456-468, 2004.
  12. M. Irani and P. Anandan, "Robust multi-sensor image alignment," Proc. ICCV, pp. 959-966, 1998.
  13. J. H. Lee, Y. S. Kim, D. Lee, D. G. Kang and J. B. Ra, "Robust CCD and IR iamge registration using gradient-based statistical information," IEEE Signal Processing Letters, vol. 17, no. 4, 2010.
  14. S. Sonn, G. A. Bilodeau, P. Galinier, "Fast and accurate registration of visible and infrared videos," Proc. CVPR, 2013.
  15. A. Polesel, G. Ramponi, V. J. Mathews, "Image enhancement via adaptive unsharp masking", IEEE Trans. Image Proc., vol. 9, no. 3, pp. 505-510, 2000.
  16. T. Kondo, and K. Kawaguchi, "Adaptive dynamic range encoding method and apparatus," US-patent 5, 444, 487, 1995.
  17. R. Ferzli and L. J. Karam, "A no-reference objective image sharpness metric based on the notion of just noticeable blur (JNB)," IEEE Trans. Image Proc., vol. 18, no. 4, pp. 717-728, 2009.
  18. N. J. W. Morris et al., "Statistics of infrared images", Proc. CVPR, 2007.