Browse > Article
http://dx.doi.org/10.7780/kjrs.2022.38.1.3

An Experiment on Image Restoration Applying the Cycle Generative Adversarial Network to Partial Occlusion Kompsat-3A Image  

Won, Taeyeon (Department of Advanced Technology Fusion, Konkuk University)
Eo, Yang Dam (Department of Civil and Environmental Engineering, Konkuk University)
Publication Information
Korean Journal of Remote Sensing / v.38, no.1, 2022 , pp. 33-43 More about this Journal
Abstract
This study presents a method to restore an optical satellite image with distortion and occlusion due to fog, haze, and clouds to one that minimizes degradation factors by referring to the same type of peripheral image. Specifically, the time and cost of re-photographing were reduced by partially occluding a region. To maintain the original image's pixel value as much as possible and to maintain restored and unrestored area continuity, a simulation restoration technique modified with the Cycle Generative Adversarial Network (CycleGAN) method was developed. The accuracy of the simulated image was analyzed by comparing CycleGAN and histogram matching, as well as the pixel value distribution, with the original image. The results show that for Site 1 (out of three sites), the root mean square error and R2 of CycleGAN were 169.36 and 0.9917, respectively, showing lower errors than those for histogram matching (170.43 and 0.9896, respectively). Further, comparison of the mean and standard deviation values of images simulated by CycleGAN and histogram matching with the ground truth pixel values confirmed the CycleGAN methodology as being closer to the ground truth value. Even for the histogram distribution of the simulated images, CycleGAN was closer to the ground truth than histogram matching.
Keywords
Deep learning; CycleGAN; High-resolution satellite image;
Citations & Related Records
Times Cited By KSCI : 1  (Citation Analysis)
연도 인용수 순위
1 Goodfellow, I.,J. Pouget-Abadie, M. Mirza, B. Xu, D. Warde-Farley, S. Ozair, A. Courville, and Y. Bengio, 2014. Generative adversarial nets, Advances in Neural Information Processing Systems, 27.
2 Zhang, Q., Q. Yuan, C. Zeng, X. Li, and Y. Wei, 2018. Missing data reconstruction in remote sensing image with a unified spatial-temporal-spectral deep convolutional neural network, IEEE Transactions on Geoscience and Remote Sensing, 56(8): 4274-4288.   DOI
3 Chen, Y., K. Sun, D. Li, T. Bai, and W. Li, 2018. Improved relative radiometric normalization method of remote sensing images for change detection, Journal of Applied Remote Sensing, 12(4): 045018.
4 Ghamisi, P. and N. Yokoya, 2018. Img2dsm: Height simulation fromsingle imagery using conditional generative adversarial net, IEEE Geoscience and Remote Sensing Letters, 15(5): 794-798.   DOI
5 Helmer, E. H. and B. Ruefenacht, 2005. Cloud-free satellite image mosaics with regression trees and histogram matching, Photogrammetric Engineering & Remote Sensing, 71(9): 1079-1089.   DOI
6 LeCun, Y., B. Boser, J.S. Denker, D. Henderson, R.E. Howard, W. Hubbard, and L.D. Jackel, 1989. Backpropagation applied to handwritten zip code recognition, Neural Computation, 1(4): 541-551.   DOI
7 Lee, S.B., W.Y. Park, Y.D. Eo, M.W. Pyeon, S. Han, S.H. Yeon, and B.K. Lee, 2017.Analysis on the applicability of simulated image from SPOT 4 HRVIR image, KSCE Journal of Civil Engineering, 21(4): 1434-1442.   DOI
8 Su, N., Y. Zhang, S. Tian, Y. Yan, and X. Miao, 2016. Shadow detection and removal for occluded object information recovery in urban high-resolution panchromatic satellite images, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 9(6): 2568-2582.   DOI
9 Badrinarayanan, V., A. Kendall, and R. Cipolla, 2017. Segnet: A deep convolutional encoder-decoder architecture for image segmentation, IEEE Transactions on Pattern Analysis and Machine Intelligence, 39(12): 2481-2495.   DOI
10 Andrade, H.J. and B.J. Fernandes, 2020. Synthesis of Satellite-Like Urban Images From Historical Maps Using Conditional GAN, IEEE Geoscience and Remote Sensing Letters, 19:1-4.   DOI
11 Choi, H.W., S.H. Lee, H.H. Kim, and Y.C. Suh, 2020. A Study on the Complementary Method of Aerial Image Learning Dataset Using Cycle Generative Adversarial Network, Journal of the Korean Society of Surveying, Geodesy, Photogrammetry and Cartography, 38(6): 499-509 (in Korean with English abstract).   DOI
12 Du, Y., P.M. Teillet, and J. Chihlar, 2002. Radiometric normalization of multitemporal high-resolution satellite images with quality control for land cover change detection, Remote Sensing of Environment, 82(1): 123-134.   DOI
13 Guo, Q., M. He, and A. Li, 2018. High-resolution remote-sensing image registration based on angle matching of edge point features, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 11(8): 2881-2895.   DOI
14 Isola, P., J.Y. Zhu, T. Zhou, and A. A. Efros, 2017. Image-to-image translation with conditional adversarial networks, Proc. of the 2017 IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, Hawaii,Jul. 22-25, pp. 1125-1134.
15 Zhou, H., S. Liu, J. He, Q. Wen, L. Song, and Y. Ma, 2016. A new model for the automatic relative radiometric normalization of multiple images with pseudo-invariant features, International Journal of Remote Sensing, 37(19): 4554-4573.   DOI
16 Kim, H.J., D.K. Seo, Y.D. Eo, M.C. Jeon, and W.Y. Park, 2019.Multi-temporal nonlinear regression method for landsat image simulation, KSCE Journal of Civil Engineering, 23(2): 777-787.   DOI
17 Seo, D.K. and Y.D. Eo, 2019. Local-Based Iterative Histogram Matching for Relative Radiometric Normalization, Journal of the Korean Society of Surveying, Geodesy, Photogrammetry and Cartography, 37(5): 323-330.   DOI
18 Lee, J.M. and K.H. Bae, 2021.ApplicationTechnology of Image Inpainting Algorithm for Occlusion on Texture Image, Journal of Korean Society for GeospatialInformation Science, 29(4): 147-156 (in Korean with English abstract).   DOI
19 Lee, M.H., S.B. Lee, Y.D. Eo, S.W. Kim, J.H. Woo, and S.H. Han, 2017. A comparative study on generating simulated Landsat NDVI images using data fusion and regression method-the case of the Korean Peninsula, Environmental Monitoring and Assessment, 189(7): 1-13.   DOI
20 Seo, D.K. and Y.D. Eo, 2018. Relative radiometric normalization for high-spatial resolution satellite imagery based on multilayer perceptron, Journal of the Korean Society of Surveying, Geodesy, Photogrammetry and Cartography, 36(6): 515-523 (in Korean with English abstract).   DOI
21 Yoo, E.J. and D.C. Lee, 2010. Patch-based processing and occlusion area recovery for true orthoimage generation, Journal of the Korean Society of Surveying, Geodesy, Photogrammetry and Cartography, 28(1): 83-92 (in Korean with English abstract).
22 Yu,J., Z. Lin,J.Yang, X. Shen, X. Lu, andT. S. Huang, 2019. Free-form image inpainting with gated convolution, Proc. of the 2019 IEEE/CVF International Conference on Computer Vision, Long Beach, CA, Jun. 15-20, pp. 4471-4480.
23 Yuan, X., J. Tian, and P. Reinartz, 2020. Generating artificial near infrared spectral band from rgb image using conditional generative adversarial network, ISPRS Annals of the Photogrammetry, Remote Sensing and Spatial Information Sciences, 3: 279-285.
24 Zhu, J.Y., T. Park, P. Isola, and A.A. Efros, 2017. Unpaired image-to-image translation using cycle-consistent adversarial networks, Proc. of the 2017 IEEE International Conference on Computer Vision, Venice, Italy, Oct. 22-29, pp. 2223-2232.