Browse > Article
http://dx.doi.org/10.7780/kjrs.2022.38.2.3

A Study on Lightweight CNN-based Interpolation Method for Satellite Images  

Kim, Hyun-ho (National Satellite Operation & Application Center, Korea Aerospace Research Institute)
Seo, Doochun (National Satellite Operation & Application Center, Korea Aerospace Research Institute)
Jung, JaeHeon (National Satellite Operation & Application Center, Korea Aerospace Research Institute)
Kim, Yongwoo (Department of System Semiconductor Engineering, Sangmyung University)
Publication Information
Korean Journal of Remote Sensing / v.38, no.2, 2022 , pp. 167-177 More about this Journal
Abstract
In order to obtain satellite image products using the image transmitted to the ground station after capturing the satellite images, many image pre/post-processing steps are involved. During the pre/post-processing, when converting from level 1R images to level 1G images, geometric correction is essential. An interpolation method necessary for geometric correction is inevitably used, and the quality of the level 1G images is determined according to the accuracy of the interpolation method. Also, it is crucial to speed up the interpolation algorithm by the level processor. In this paper, we proposed a lightweight CNN-based interpolation method required for geometric correction when converting from level 1R to level 1G. The proposed method doubles the resolution of satellite images and constructs a deep learning network with a lightweight deep convolutional neural network for fast processing speed. In addition, a feature map fusion method capable of improving the image quality of multispectral (MS) bands using panchromatic (PAN) band information was proposed. The images obtained through the proposed interpolation method improved by about 0.4 dB for the PAN image and about 4.9 dB for the MS image in the quantitative peak signal-to-noise ratio (PSNR) index compared to the existing deep learning-based interpolation methods. In addition, it was confirmed that the time required to acquire an image that is twice the resolution of the 36,500×36,500 input image based on the PAN image size is improved by about 1.6 times compared to the existing deep learning-based interpolation method.
Keywords
Interpolation; Super-resolution; Remote sensing; Satellite images;
Citations & Related Records
연도 인용수 순위
  • Reference
1 Hu, J., L. Shen, and G. Sun, 2018. Squeeze-and-excitation networks, Proc. of the IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, Jun. 18-23, pp. 7132-7141. https://doi.org/10.1109/CVPR.2018.00745   DOI
2 Hui, Z., X. Wang, and X. Gao, 2018. Fast and accurate single image super-resolution via information distillation network, Proc. of the IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, Jun. 18-23, pp. 723-731. https://doi.org/10.1109/CVPR.2018.00082   DOI
3 Lanaras, C., J. Bioucas-Dias, S. Galliani, E. Baltsavias, and K. Schindler, 2018. Super-resolution of Sentinel2 images: Learning a globally applicable deep neural network, ISPRS Journal of Photogrammetry and Remote Sensing, 146: 305-319. https://doi.org/10.1016/j.isprsjprs.2018.09.018   DOI
4 Tuna, C., G. Unal, and E. Sertel, 2018. Single-frame super resolution of remote-sensing images by convolutional neural networks, International Journal of Remote Sensing, 39(8): 2463-2479. https://doi.org/10.1080/01431161.2018.1425561   DOI
5 Bosch, M., C.M. Gifford, and P.A. Rodriguez, 2018. Super-resolution for overhead imagery using densenets and adversarial learning, Proc. of 2018 IEEE Winter Conference on Applications of Computer Vision (WACV), Lake Tahoe, NV, Mar. 12-15, pp. 1414-1422. https://doi.org/10.1109/WACV.2018.00159   DOI
6 Dong, C., C.C. Loy, K. He, and X. Tang, 2014. Learning a deep convolutional network for image super-resolution, Proc. of the European Conference on Computer Vision, Zurich, Switzerland, Sep. 6-12, vol. 8692, pp. 184-199. https://doi.org/10.1007/978-3-319-10593-2_13   DOI
7 Lai, W.S., J.B. Huang, N. Ahuja, and M.H. Yang, 2018. Fast and accurate image super-resolution with deep laplacian pyramid networks, IEEE Transactions on Pattern Analysis and Machine Intelligence, 41(11): 2599-2613. https://doi.org/10.1109/TPAMI.2018.2865304   DOI
8 Huang, G., Z. Liu, L. Van Der Maaten, and K.Q. Weinberger, 2017. Densely connected convolutional networks, Proc. of the IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, HI, Jun. 21-26, pp. 4700-4708. https://doi.org/10.1109/CVPR.2017.243   DOI
9 Hui, Z., X. Gao, Y. Yang, and X. Wang, 2019. Lightweight image super-resolution with information multi-distillation network, Proc. of the 27th ACM International Conference on Multimedia, Nice, France, Oct. 21-25, pp. 2024-2032. https://doi.org/10.1145/3343031.3351084   DOI
10 Keys, R., 1981. Cubic convolution interpolation for digital image processing, IEEE Transactions on Acoustics, Speech, and Signal Processing, 29(6): 1153-1160. https://doi.org/10.1109/TASSP.1981.1163711   DOI
11 Yan, Q., Y. Xu, X. Yang, and T.Q. Nguyen, 2015. Single image super resolution based on gradient profile sharpness, IEEE Transactions on Image Processing, 24(10): 3187-3202. https://doi.org/10.1109/TIP.2015.2414877   DOI
12 Ledig, C., L. Theis, F. Huszar, J. Caballero, A. Cunningham, A. Acosta, and W. Shi, 2017. Photo-realistic single image super-resolution using a generative adversarial network, Proc. of the IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, HI, Jun. 21-26, pp. 4681-4690. https://doi.org/10.1109/CVPR.2017.19   DOI
13 Liebel, L. and M. Korner, 2016. Single-image super resolution for multispectral remote sensing data using convolutional neural networks, International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, 41(B3): 883-890. https://doi.org/10.5194/isprs-archives-XLI-B3-883-2016   DOI
14 Wang, Z., A.C. Bovik, and H.R. Sheikh, 2004. Image quality assessment: from error visibility to structural similarity, IEEE Transactions on Image Processing, 13(4): 600-612. https://doi.org/10.1109/TIP.2003.819861   DOI