Journal of the Korean Society of Surveying, Geodesy, Photogrammetry and Cartography (한국측량학회지)

Korean Society of Surveying, Geodesy, Photogrammetry and Cartography (한국측량학회)
권호 표지
DOI QR코드

DOI QR Code

A Comparison of Pan-sharpening Algorithms for GK-2A Satellite Imagery

천리안위성 2A호 위성영상을 위한 영상융합기법의 비교평가

  • Lee, Soobong (National Meteorological Satellite Center, Korea Meteorological Administration) ;
  • Choi, Jaewan (Dept. of Civil Engineering, Chungbuk National University)
  • Received : 2022.07.29
  • Accepted : 2022.08.25
  • Published : 2022.08.31
Download PDF
⟨ Previous Next ⟩

Abstract

In order to detect climate changes using satellite imagery, the GCOS (Global Climate Observing System) defines requirements such as spatio-temporal resolution, stability by the time change, and uncertainty. Due to limitation of GK-2A sensor performance, the level-2 products can not satisfy the requirement, especially for spatial resolution. In this paper, we found the optimal pan-sharpening algorithm for GK-2A products. The six pan-sharpening methods included in CS (Component Substitution), MRA (Multi-Resolution Analysis), VO (Variational Optimization), and DL (Deep Learning) were used. In the case of DL, the synthesis property based method was used to generate training dataset. The process of synthesis property is that pan-sharpening model is applied with Pan (Panchromatic) and MS (Multispectral) images with reduced spatial resolution, and fused image is compared with the original MS image. In the synthesis property based method, fused image with desire level for user can be produced only when the geometric characteristics between the PAN with reduced spatial resolution and MS image are similar. However, since the dissimilarity exists, RD (Random Down-sampling) was additionally used as a way to minimize it. Among the pan-sharpening methods, PSGAN was applied with RD (PSGAN_RD). The fused images are qualitatively and quantitatively validated with consistency property and the synthesis property. As validation result, the GSA algorithm performs well in the evaluation index representing spatial characteristics. In the case of spectral characteristics, the PSGAN_RD has the best accuracy with the original MS image. Therefore, in consideration of spatial and spectral characteristics of fused image, we found that PSGAN_RD is suitable for GK-2A products.

기후변화 감시에 위성 자료 활용을 위해 GCOS (Global Climate Observing System)는 시공간 해상도, 시간 변화에 따른 안정성, 불확실도 등의 요구사항을 제시하고 있다. 천리안위성 2A호의 경우, 센서의 한계로 인해 산출물들이 공간해상도 조건에 충족하지 못하는 경우가 많다. 따라서 본 연구에서는 영상융합 기법들을 천리안위성 2A호 영상에 적용하여 산출물 생성 시 활용될 수 있는 최적의 기법을 찾고자 한다. 이를 위해 CS (Component Substitution), MRA (Multiresolution Analysis), VO (Variational Optimization), DL (Deep Learning)에 포함되는 총 6가지 영상융합 기법을 활용하였다. DL의 경우 합성적(Synthesis) 특성 기반 방법을 훈련자료 구축에 사용하였다. 합성적 특성 기반 방법의 과정은 PAN (Panchromatic)과 MS (Multispectral) 영상의 공간해상도 차이만큼 두 영상의 해상도를 낮춰 융합 영상을 생성한 후 원본 MS 영상과 비교한다. 합성적 특성 기반 방법은 공간해상도를 저하시킨 PAN 영상과 MS 영상 간 기하 특성이 같아야 사용자가 원하는 수준의 융합 영상을 제작할 수 있다. 하지만, 훈련자료 구축 시 비유사성이 존재하기에 이를 최소화하는 방법으로 무작위 비율을 활용한 PSGAN 모델(PSGAN_RD)을 추가로 활용하였다. 융합 영상의 검증은 일관성(consistency) 및 합성적 특성 기반 정성적, 정량적 분석을 수행하였다. 분석 결과, 영상융합 알고리즘 중 GSA가 공간 유사도를 나타내는 평가지수에서 가장 높은 수치를 보였으며, 분광 유사도를 나타내는 지수들은 PSGAN_RD 모델의 정확도가 가장 높았다. 융합 영상의 공간 및 분광 특성을 모두 고려한다면 PSGAN_RD 모델이 천리안위성 2A호 산출물 제작에 가장 최적일 것으로 판단하였다.

Keywords

Acknowledgement

이 논문은 과학기술정보통신부 및 정보통신산업진흥원의 '2022년 고성능 컴퓨팅 지원' 사업으로부터 지원받아 수행하였으며, 2020년도 정부(교육부)의 제원으로 한국연구재단의 지원을 받아 수행되었음(NRF-2020R1I1A3A04037483).

References

  1. Aiazzi, B., Alparone, L., Baronti, S., Garzelli, A., and Selva, M. (2006), MTF-tailored multiscale fusion of high-resolution MS and Pan imagery, Photogrammetric Engineering & Remote Sensing, Vol. 72, No. 5, pp. 591-596. https://doi.org/10.14358/PERS.72.5.591
  2. Aiazzi, B., Baronti, S., and Selva, M. (2007), Improving component substitution pansharpening through multivariate regression of MS + Pan data, IEEE Transactions on Geoscience and Remote Sensing, Vol. 45, No. 10, pp. 3230-3239. https://doi.org/10.1109/TGRS.2007.901007
  3. Alparone, L., Wald, L., Chanussot, J., Thomas, C., Gamba, P., and Bruce, L.M. (2007), Comparison of pansharpening algorithms: Outcome of the 2006 GRS-S data-fusion contest, IEEE Transactions on Geoscience and Remote Sensing, Vol. 45, No. 10, pp. 3012-3021. https://doi.org/10.1109/TGRS.2007.904923
  4. Ballester, C., Caselles, V., Igual, L., Verdera, J., and Rouge, B. (2006), A variational model for P+ XS image fusion, International Journal of Computer Vision, Vol. 69, No. 1, pp. 43-58. https://doi.org/10.1007/s11263-006-6852-x
  5. Balogh, W., Canturk, L., Chernikov, S., Doi, T., Gadimova, S., Haubold, H., and Kotelnikov, V. (2010), The United Nations programme on space applications: Status and direction for 2010, Space Policy, Vol. 26, No. 3, pp. 185-188. https://doi.org/10.1016/j.spacepol.2010.03.008
  6. Choi, J., Yu, K., and Kim, Y. (2010), A new adaptive component-substitution-based satellite image fusion by using partial replacement, IEEE Transactions on Geoscience and Remote Sensing, Vol. 49, No. 1, pp. 295-309. https://doi.org/10.1109/TGRS.2010.2051674
  7. Cohen, J., Screen, J.A., Furtado, J.C., Barlow, M., Whittleston, D., Coumou, D., Francis, J., Dethloff, K., Entekhabi, D., and Overland, J. (2014), Recent arctic amplification and extreme mid-latitude weather, Nature Geoscience, Vol. 7, No. 9, pp. 627-637. https://doi.org/10.1038/ngeo2234
  8. Dong, C., Loy, C.C., and Tang, X. (2016), Accelerating the super-resolution convolutional neural network, Computer Vision-ECCV 2016, Vol. 9906, pp. 391-407. https://doi.org/10.1007/978-3-319-46475-6_25
  9. Dou, W. (2018), Image degradation for quality assessment of pan-sharpening methods, Remote Sensing, Vol. 10, No. 1, pp. 154. https://doi.org/10.3390/rs10010154
  10. Eghbalian, S., and Ghassemian, H. (2018), Multi spectral image fusion by deep convolutional neural network and new spectral loss function, International Journal of Remote Sensing, Vol. 39, No. 12, pp. 3983-4002. https://doi.org/10.1080/01431161.2018.1452074
  11. Fasbender, D., Radoux, J., and Bogaert, P. (2008), Bayesian data fusion for adaptable image pansharpening, IEEE Transactions on Geoscience and Remote Sensing, Vol. 46, No. 6, pp. 1847-1857. https://doi.org/10.1109/TGRS.2008.917131
  12. GCOS (2021), The Status of the Global Climate Observing System 2021: The GCOS Status Report, GCOS-240, WMO, Geneva, Switzerland, pp. 1-384.
  13. IPCC (2021), Climate Change 2021: The Physical Science Basis. Contribution of Working Group I to the Sixth Assessment Report of the Intergovernmental Panel on Climate Change, Cambridge University Press, Cambridge, United Kingdom and New York, NY, USA, pp. 3-32.
  14. Jeong, N.K., Jung, H.S., Oh, K.Y., Park, S.H., and Lee, S.C. (2016), Comparison analysis of quality assessment protocols for image fusion of KOMPSAT-2/3/3A, Korean Journal of Remote Sensing, Vol. 32, No. 5, pp. 453-469. https://doi.org/10.7780/kjrs.2016.32.5.5
  15. Kruse, F.A., Lefkoff, A., Boardman, J., Heidebrecht, K., Shapiro, A., Barloon, P., and Goetz, A. (1993), The spectral image processing system (SIPS)-interactive visualization and analysis of imaging spectrometer data, Remote Sensing of Environment, Vol. 44, No. 2-3, pp. 145-163. https://doi.org/10.1016/0034-4257(93)90013-N
  16. Kug, J.S., Jeong, J.H., Jang, Y.S., Kim, B.M., Folland, C.K., Min, S.K., and Son, S.W. (2015), Two distinct influences of Arctic warming on cold winters over North America and East Asia, Nature Geoscience, Vol. 8, No. 10, pp. 759-762. https://doi.org/10.1038/ngeo2517
  17. Li, S., and Yang, B. (2010), A new pan-sharpening method using a compressed sensing technique, IEEE Transactions on Geoscience and Remote Sensing, Vol. 49, No. 2, pp. 738-746. https://doi.org/10.1109/TGRS.2010.2067219
  18. Liu, Q., Zhou, H., Xu, Q., Liu, X., and Wang, Y. (2020), PSGAN: A generative adversarial network for remote sensing image pan-sharpening, IEEE Transactions on Geoscience and Remote Sensing, Vol. 59, No. 12, pp. 10227-10242. https://doi.org/10.1109/ICIP.2018.8451049
  19. Mahyari, A.G., and Yazdi, M. (2011), Panchromatic and multispectral image fusion based on maximization of both spectral and spatial similarities, IEEE Transactions on Geoscience and Remote Sensing, Vol. 49, No. 6, pp. 1976-1985. https://doi.org/10.1109/TGRS.2010.2103944
  20. Masi, G., Cozzolino, D., Verdoliva, L., and Scarpa, G. (2016), Pansharpening by convolutional neural networks, Remote Sensing, Vol. 8, No. 7, pp. 594. https://doi.org/10.3390/rs8070594
  21. Otazu, X., Gonzalez-Audicana, M., Fors, O., and Nunez, J. (2005), Introduction of sensor spectral response into image fusion methods. Application to wavelet-based methods, IEEE Transactions on Geoscience and Remote Sensing, Vol. 43, No. 10, pp. 2376-2385. https://doi.org/10.1109/TGRS.2005.856106
  22. Ozcelik, F., Alganci, U., Sertel, E., and Unal, G. (2020), Rethinking CNN-based pansharpening: Guided colorization of panchromatic images via GANs, IEEE Transactions on Geoscience and Remote Sensing, Vol. 59, No. 4, pp. 3486-3501. https://doi.org/10.1109/TGRS.2020.3010441
  23. Palsson, F., Sveinsson, J.R., and Ulfarsson, M.O. (2013), A new pansharpening algorithm based on total variation, IEEE Geoscience and Remote Sensing Letters, Vol. 11, No. 1, pp. 318-322. https://doi.org/10.1109/LGRS.2013.2257669
  24. Palsson, F., Sveinsson, J.R., Ulfarsson, M.O., and Benediktsson, J.A. (2015), Quantitative quality evaluation of pansharpened imagery: Consistency versus synthesis, IEEE Transactions on Geoscience and Remote Sensing, Vol. 54, No. 3, pp. 1247-1259. https://doi.org/10.1109/TGRS.2015.2476513
  25. Ranchin, T., and Wald, L. (2000), Fusion of high spatial and spectral resolution images: The ARSIS concept and its implementation, Photogrammetric Engineering and Remote Sensing, Vol. 66, No. 1, pp. 49-61.
  26. Rebecca, L., and Michon, S. (2020), Climate change: Arctic sea ice summer minimum, NOAA, https://www.climate.gov/news-features/understanding-climate/climate-change-arctic-sea-ice-summer-minimum (last date accessed: 8 September 2020).
  27. Tian, X., Chen, Y., Yang, C., and Ma, J. (2021), Variational pansharpening by exploiting cartoon-texture similarities, IEEE Transactions on Geoscience and Remote Sensing, Vol. 60, No. pp. 1-16. https://doi.org/10.1109/TGRS.2020.3048257
  28. Vicinanza, M.R., Restaino, R., Vivone, G., Dalla Mura, M., and Chanussot, J. (2014), A pansharpening method based on the sparse representation of injected details, IEEE Geoscience and Remote Sensing Letters, Vol. 12, No. 1, pp. 180-184. https://doi.org/10.1109/LGRS.2014.2331291
  29. Vivone, G., Dalla Mura, M., Garzelli, A., Restaino, R., Scarpa, G., Ulfarsson, M.O., Alparone, L., and Chanussot, J. (2020), A new benchmark based on recent advances in multispectral pansharpening: Revisiting pansharpening with classical and emerging pansharpening methods, IEEE Geoscience and Remote Sensing Magazine, Vol. 9, No. 1, pp. 53-81. https://doi.org/10.1109/MGRS.2020.3019315
  30. Vivone, G., Restaino, R., Dalla Mura, M., Licciardi, G., and Chanussot, J. (2013), Contrast and error-based fusion schemes for multispectral image pansharpening, IEEE Geoscience and Remote Sensing Letters, Vol. 11, No. 5, pp. 930-934. https://doi.org/10.1109/LGRS.2013.2281996
  31. Wald, L., Ranchin, T., and Mangolini, M. (1997), Fusion of satellite images of different spatial resolutions: Assessing the quality of resulting images, Photogrammetric Engineering and Remote Sensing, Vol. 63, No. 6, pp. 691-699.
  32. Wang, Z., Bovik, A.C., Sheikh, H.R., and Simoncelli, E.P. (2004), Image quality assessment: from error visibility to structural similarity, IEEE Transactions on Image Processing, Vol. 13, No. 4, pp. 600-612. https://doi.org/10.1109/TIP.2003.819861
  33. WMO (2010), Space and Climate Change: Use of Space-based Technologies in the United Nations System, WMO-No. 1081, WMO, Geneva, Switzerland, pp. 1-56.
  34. Xu, S., Zhang, J., Zhao, Z., Sun, K., Liu, J., and Zhang, C. (2021), Deep gradient projection networks for pan-sharpening, Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 20-25 June 2021, Nashville, TN, USA, pp. 1366-1375.
  35. Yuan, Q., Wei, Y., Meng, X., Shen, H., and Zhang, L. (2018), A multiscale and multidepth convolutional neural network for remote sensing imagery pan-sharpening, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, Vol. 11, No. 3, pp. 978-989. https://doi.org/10.1109/JSTARS.2018.2794888
  36. Yuhas, R.H., Goetz, A.F., and Boardman, J.W. (1992), Discrimination among semi-arid landscape endmembers using the spectral angle mapper (SAM) algorithm, The Third Annual JPL Airborne Geoscience Workshop. Volume 1: AVIRIS Workshop, 1-2 June 1992, Pasadena, California, USA, pp. 147-149.