Journal of the Korea Computer Graphics Society (한국컴퓨터그래픽스학회논문지)

Korea Computer Graphics Society (한국컴퓨터그래픽스학회)
권호 표지
DOI QR코드

DOI QR Code

Image Fusion Framework for Enhancing Spatial Resolution of Satellite Image using Structure-Texture Decomposition

구조-텍스처 분할을 이용한 위성영상 융합 프레임워크

  • Received : 2019.06.07
  • Accepted : 2019.06.24
  • Published : 2019.07.14
Download PDF
⟨ Previous Next ⟩

Abstract

This paper proposes a novel framework for image fusion of satellite imagery to enhance spatial resolution of the image via structure-texture decomposition. The resolution of the satellite imagery depends on the sensors, for example, panchromatic images have high spatial resolution but only a single gray band whereas multi-spectral images have low spatial resolution but multiple bands. To enhance the spatial resolution of low-resolution images, such as multi-spectral or infrared images, the proposed framework combines the structures from the low-resolution image and the textures from the high-resolution image. To improve the spatial quality of structural edges, the structure image from the low-resolution image is guided filtered with the structure image from the high-resolution image as the guidance image. The combination step is performed by pixel-wise addition of the filtered structure image and the texture image. Quantitative and qualitative evaluation demonstrate the proposed method preserves spectral and spatial fidelity of input images.

본 논문에서는 구조-텍스처 분할 기법을 기반으로 위성영상을 분할 융합하여 공간 해상도를 개선시키는 프레임워크를 제시한다. 위성영상은 센서가 감지하는 파장에 따라 다양한 공간해상도를 가진다. 전정 영상 (panchromatic image)은 일반적으로 높은 공간해상도를 가지지만 단일 흑백컬러를 가지고 있는 반면, 다중분광 영상 (multi-spectral image)나 적외선 영상은 전정 영상에 비해 낮은 공간해상도를 가지지만 다양한 분광 밴드정보와 열 정보를 가지고 있다. 본 논문에서는 다중분광 영상이나 적외선 영상의 공간 해상도를 향상시키기 위해 영상의 디테일이 텍스처 영상에만 존재한다는 것에 착안하여 본 프레임워크를 고안하였다. 고안된 프레임워크에서는 저해상도 영상과 고해상도 영상이 구조 영상과 텍스처 영상으로 분할된 뒤, 저해상도 구조영상은 고해상도 구조 영상을 참조하여 가이디드 필터링 된다. 구조-텍스처 영상 모델에 따라 필터링된 저해상도 영상의 구조 영역과 고해상도 영상의 텍스처 영역을 픽셀 단위로 더해져서 최종 영상이 생성된다. 생성된 영상은 저해상도 영상의 밴드와 고해상도 영상의 디테일을 포함한다. 제시하는 방법은 분광해상도와 공간해상도를 모두 보존할 수 있음을 실험적으로 확인하였다.

Keywords

References

  1. Tu T., Su S., Shyu H. and Huang P. S., "A new look at HIS-like image fusion methods," Information Fusion, vol. 2, pp. 177-186, 2001. https://doi.org/10.1016/S1566-2535(01)00036-7
  2. Rahmani S., Strait M., Merkurjev D., Moeller M., and Wittman T., "An Adaptive IHS Pan-Sharpening Method," IEEE Geoscience and Remote Sensing Letter, Vol. 7, No. 4, 2010.
  3. Chavez P. S., Jr. and Kwarteng A. W., "Extracting spectral contrast in Landsat thematic mapper image data using selective principal analysis," Photogramm. Eng. Remote Sens., vol. 58, no.5, pp.561-567, 1992
  4. Laban C. A. and Brower B. V., Process for enhancing the spatial resolution of multispectral imagery using pan-sharpening, US patenta 6,011,875, 2000.
  5. Aiazzi B., Baronti S., and Selva M., "Improving component substitution pansharpening through multivariate regression of MS+Pan data," IEEE Trans. Geosci. Remote Sens., vol. 45, no. 10, pp.3230-3239, 2007. https://doi.org/10.1109/TGRS.2007.901007
  6. Li H., Manjunath M. S. and Mitra S. K., "Multisensor Image Fusion Using the Wavelet Trasform," Graphics Models and Image Processing, vol. 57, pp. 234-245, 1995.
  7. M. N. Do and M. Vetterli, "The contourlet transform: An efficient directional multiresolution image representation," IEEE Trans. Image Process., vol. 14, no. 12, pp. 2091-2106, 2005. https://doi.org/10.1109/TIP.2005.859376
  8. F. Nencini, A. Garzelli, S. Baronti, and L. Alparone, "Remote sensing image fusion using the curvelet transform," Information fusion, Vol. 8, Issue 2, pp. 143-156, 2007 https://doi.org/10.1016/j.inffus.2006.02.001
  9. Y. Wei, Yuan Q., Shen H., and Zhang L., "Boosting the Accuracy of Multispectral Image Pansharpening by Learning a Deep Residual Network," IEEE Geoscience and Remote Sensing Letters, Vol. 14, Issue 10, 2017
  10. J. Yang, X. Fu, Y. Hu, Y. Huang, X. Ding, and J. Paisley, "PanNet: A Deep Network Architecture for Pan-Sahrpening," In Proc. ICCV 2017, pp. 5449-5457, 2017
  11. Tomasi C., Manduchi R., "Bilateral filtering for gray and color images," In Proc. ICCV 1998, pp. 839-846, 1998
  12. He K., Sun J. and Tang X., "Guided Image Filtering," Transaction on Pattern Analysis and Machine Intelligence, No. 06, Vol. 35, pp. 1397-1709, 2013. https://doi.org/10.1109/TPAMI.2012.213
  13. Cho H., Lee H., Kang H., and Lee S., "Bilateral Texture Filter," ACM Transaction on Graphics, Vol. 33, no. 4, 2014
  14. Xu L., Yan Q., Xia Y. and Jia J., "Structure Extraction from Texture via Relative Total Variation," ACM Transaction on Graphics, Vol. 31, 2012.
  15. Jeon J., Lee H., Kang H. and Lee S., "Scale-aware Structure-Preserving Texture Filtering," Computer Graphics Forum, Vol. 35, No. 7, 2016.
  16. Lee H., Jeon J., Kim J. and Lee S., "Structure-Texture Decomposition of Images with Interval Gradient," Computer Graphics Forum, Vol. 36, 2017.
  17. Wang Z., Bovik A. C., Sheikh H. R., and Simoncelli E. P., "Image quality assessment: from error visibility to structural similarity," IEEE Transactions on Image Processing, Vol. 13, Issue 4, pp. 600-612, 2004. https://doi.org/10.1109/TIP.2003.819861
  18. Wald L., Ranchin T., and Mangolini M., "Fusion of satellite images of different spatial resolutions: Assessing the quality of resulting images," Photogrammetric engineering and remote sensing, Vol. 63, pp. 691-699, 1997.