Browse > Article
http://dx.doi.org/10.7780/kjrs.2022.38.6.1.28

Combining Conditional Generative Adversarial Network and Regression-based Calibration for Cloud Removal of Optical Imagery  

Kwak, Geun-Ho (Geoinformatic Engineering Research Institute, Inha University)
Park, Soyeon (Department of Geoinformatic Engineering, Inha University)
Park, No-Wook (Department of Geoinformatic Engineering, Inha University)
Publication Information
Korean Journal of Remote Sensing / v.38, no.6_1, 2022 , pp. 1357-1369 More about this Journal
Abstract
Cloud removal is an essential image processing step for any task requiring time-series optical images, such as vegetation monitoring and change detection. This paper presents a two-stage cloud removal method that combines conditional generative adversarial networks (cGANs) with regression-based calibration to construct a cloud-free time-series optical image set. In the first stage, the cGANs generate initial prediction results using quantitative relationships between optical and synthetic aperture radar images. In the second stage, the relationships between the predicted results and the actual values in non-cloud areas are first quantified via random forest-based regression modeling and then used to calibrate the cGAN-based prediction results. The potential of the proposed method was evaluated from a cloud removal experiment using Sentinel-2 and COSMO-SkyMed images in the rice field cultivation area of Gimje. The cGAN model could effectively predict the reflectance values in the cloud-contaminated rice fields where severe changes in physical surface conditions happened. Moreover, the regression-based calibration in the second stage could improve the prediction accuracy, compared with a regression-based cloud removal method using a supplementary image that is temporally distant from the target image. These experimental results indicate that the proposed method can be effectively applied to restore cloud-contaminated areas when cloud-free optical images are unavailable for environmental monitoring.
Keywords
Cloud removal; Conditional generative adversarial networks (cGANs); Random forest; SAR;
Citations & Related Records
Times Cited By KSCI : 4  (Citation Analysis)
연도 인용수 순위
1 Zeng, C., H. Shen, and L. Zhang, 2013. Recovering missing pixels for Landsat ETM + SLC-off imagery using multi-temporal regression analysis and a regularization method, Remote Sensing of Environment, 131: 182-194. https://doi.org/10.1016/j.rse.2012.12.012   DOI
2 Goodfellow, I., J. Pouget-Abadie, M. Mirza, B. Xu, D. Warde-Farley, S. Ozair, A. Courvile, and Y. Bengio, 2020. Generative adversarial networks, Communications of the ACM, 63(11): 139-144. https://doi.org/10.1145/3422622   DOI
3 Isola, P., J.Y. Zhu, T. Zhou, and A.A. Efros, 2017. Image-to-image translation with conditional adversarial networks, Proc. of the IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, HI, USA, Jul. 21-26, pp. 1125-1134. https://doi.org/10.48550/arXiv.1611.07004   DOI
4 King, M.D., S. Platnick, W.P. Menzel, S.A. Ackerman, and P.A. Hubanks, 2013. Spatial and temporal distribution of clouds observed by MODIS onboard the Terra and Aqua satellites, IEEE Transactions on Geoscience and Remote Sensing, 51(7): 3826-3852. https://doi.org/10.1109/TGRS.2012.2227333   DOI
5 Li, W., Y. Li, and J.C.W. Chan, 2019. Thick cloud removal with optical and SAR imagery via convolutional-mapping-deconvolutional network, IEEE Transactions on Geoscience and Remote Sensing, 58(4): 2865-2879. https://doi.org/10.1109/TGRS.2019.2956959   DOI
6 Zhang, C., W. Li, and D. Travis, 2007. Gaps-fill of SLC-off Landsat ETM+ satellite image using a geostatistical approach, International Journal of Remote Sensing, 28(22): 5103-5122. https://doi.org/10.1080/01431160701250416   DOI
7 Darbaghshahi, F.N., M.R. Mohammadi, and M. Soryani, 2021. Cloud removal in remote sensing images using generative adversarial networks and SAR-to-optical image translation, IEEE Transactions on Geoscience and Remote Sensing, 60: 1-9. https://doi.org/10.1109/TGRS.2021.3131035   DOI
8 Lee, Y.S., W.K. Baek, and H.S. Jung, 2019. Forest vertical structure classification in Gongju city, Korea from optic and RADAR satellite images using artificial neural network, Korean Journal of Remote Sensing, 35(3): 447-455 (in Korean with English abstract). https://doi.org/10.7780/kjrs.2019.35.3.8   DOI
9 Moon, J. and K.J. Lee, 2019. A comparative study on the possibility of land cover classification of the mosaic images on the Korean Peninsula, Korean Journal of Remote Sensing, 35(6-4): 1319-1326 (in Korean with English abstract). https://doi.org/10.7780/kjrs.2019.35.6.4.2   DOI
10 Shen, H., X. Li, Q. Cheng, C. Zeng, G. Yang, H. Li, and L. Zhang, 2015. Missing information reconstruction of remote sensing data: A technical review, IEEE Geoscience and Remote Sensing Magazine, 3(3): 61-85. https://doi.org/10.1109/mgrs.2015.2441912   DOI
11 Gao, J., Y. Yi, T. Wei, and G. Zhang, 2021. Sentinel-2 cloud removal considering ground changes by fusing multitemporal SAR and optical images, Remote Sensing, 13: 3998. https://doi.org/10.3390/rs13193998   DOI
12 Ju, J. and D.P. Roy, 2008. The availability of cloud-free Landsat ETM+ data over the conterminous United States and globally, Remote Sensing of Environment, 112(3): 1196-1211. https://doi.org/10.1016/j.rse.2007.08.011   DOI
13 Shen, H. and L. Zhang, 2009. A MAP-based algorithm for destriping and inpainting of remotely sensed images, IEEE Transactions on Geoscience and Remote Sensing, 47(5): 1492-1502. https://doi.org/10.1109/TGRS.2008.2005780   DOI
14 Kim, S.H. and J. Eun, 2021. Development of cloud and shadow detection algorithm for periodic composite of Sentinel-2A/B satellite images, Korean Journal of Remote Sensing, 37(5-1): 989-998 (in Korean with English abstract). https://doi.org/10.7780/kjrs.2021.37.5.1.13   DOI
15 Covello, F., F. Battazza, A. Coletta, E. Lopinto, C. Fiorentino, L. Pietranera, G. Valentini, and S. Zoffoli, 2010. COSMO-SkyMed an existing opportunity for observing the Earth, Journal of Geodynamics, 49(3-4): 171-180. https://doi.org/10.1016/j.jog.2010.01.001   DOI
16 Filipponi, F., 2019. Sentinel-1 GRD preprocessing workflow, Multidisciplinary Digital Publishing Institute Proceedings, 18(1): 11. https://doi.org/10.3390/ECRS-3-06201   DOI
17 Fuentes Reyes, M., S. Auer, N. Merkle, C. Henry, and M. Schmitt, 2019. SAR-to-optical image translation based on conditional generative adversarial networks-Optimization, opportunities and limits, Remote Sensing, 11(17): 2067. https://doi.org/10.3390/rs11172067   DOI
18 Meraner, A., P. Ebel, X.X. Zhu, and M. Schmitt, 2020. Cloud removal in Sentinel-2 imagery using a deep residual neural network and SAR-optical data fusion, ISPRS Journal of Photogrammetry and Remote Sensing, 166: 333-346. https://doi.org/10.1016/j.isprsjprs.2020.05.013   DOI
19 Park, S. and N.-W. Park, 2022. Cloud removal using Gaussian process regression for optical image reconstruction, Korean Journal of Remote Sensing, 38(4): 327-341 (in Korean with English abstract). https://doi.org/10.7780/kjrs.2022.38.4.1   DOI