Browse > Article
http://dx.doi.org/10.7780/kjrs.2022.38.5.1.10

Impact Analysis of Deep Learning Super-resolution Technology for Improving the Accuracy of Ship Detection Based on Optical Satellite Imagery  

Park, Seongwook (Nara Space Technology)
Kim, Yeongho (Nara Space Technology)
Kim, Minsik (Nara Space Technology)
Publication Information
Korean Journal of Remote Sensing / v.38, no.5_1, 2022 , pp. 559-570 More about this Journal
Abstract
When a satellite image has low spatial resolution, it is difficult to detect small objects. In this research, we aim to check the effect of super resolution on object detection. Super resolution is a software method that increases the resolution of an image. Unpaired super resolution network is used to improve Sentinel-2's spatial resolution from 10 m to 3.2 m. Faster-RCNN, RetinaNet, FCOS, and S2ANet were used to detect vessels in the Sentinel-2 images. We experimented the change in vessel detection performance when super resolution is applied. As a result, the Average Precision (AP) improved by at least 12.3% and up to 33.3% in the ship detection models trained with the super-resolution image. False positive and false negative cases also decreased. This implies that super resolution can be an important pre-processing step in object detection, and it is expected to greatly contribute to improving the accuracy of other image-based deep learning technologies along with object detection.
Keywords
Ship detection; Super-resolution; Sentinel-2; Remote sensing; Deep learning;
Citations & Related Records
Times Cited By KSCI : 2  (Citation Analysis)
연도 인용수 순위
1 Lim, B., S.H. Son, H.W. Kim, S.J. Nah, and K.M. Lee, 2017. Enhanced deep residual networks for single image super-resolution, Proc of 2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), Honolulu, HI, USA, Jul. 21-26, pp. 136-144. https://doi.org/10.1109/CVPRW.2017.151   DOI
2 Maeda, S., 2020. Unpaired image super-resolution using pseudo-supervision, Proc. of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Seattle, WA, USA, Jun 13-19, pp. 288-297. https://doi.org/10.1109/CVPR42600.2020.00037   DOI
3 Agustsson, E. and R. Timofte, 2017. NTIRE 2017 challenge on single image super-resolution: Dataset and study, Proc. of 2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), Honolulu, HI, USA, Jul. 21-26, pp. 1122-1131. https://doi.org/10.1109/CVPRW.2017.150   DOI
4 Bo, L., X. Xiaoyang, W. Xingxing, and T. Wenting, 2021. Ship detection and classification from optical remote sensing images: A survey, Chinese Journal of Aeronautics, 34(3): 145-163. https://doi.org/10.1016/j.cja.2020.09.022   DOI
5 He, S., H. Zou, Y. Wang, R. Li, F. Cheng, X. Cao, and M. Li, 2021b. Enhancing Mid-Low-Resolution Ship Detection With High-Resolution Feature Distillation, IEEE Geoscience and Remote Sensing Letters, 19: 1-5. https://doi.org/10.1109/LGRS.2021.3110404   DOI
6 Bose, P., D. Halder, O. Rahman, and T. Pial, 2022. Effectivity of super resolution convolutional neural network for the enhancement of land cover classification from medium resolution satellite images, arXiv preprint arXiv:2207. 02301. https://doi.org/10.48550/arXiv.2207.02301   DOI
7 Cao, L., C. Wang, and J. Li, 2016. Vehicle detection from highway satellite images via transfer learning, Information Sciences, 366: 177-187. https://doi.org/10.1016/j.ins.2016.01.004   DOI
8 Han, J., J. Ding, J. Li, and G.S. Xia, 2021. Align deep features for oriented object detection, IEEE Transactions on Geoscience and Remote Sensing, 60: 1-11. https://doi.org/10.1109/TGRS.2021.3062048   DOI
9 Lin, T. Y., P. Goyal, R. Girshick, K. He, and P. Dollar, 2017. Focal loss for dense object detection, Proc. of 2017 IEEE International Conference on Computer Vision (ICCV), Venice, Italy, Oct. 22-29, pp. 2999-3007. https://doi.org/10.1109/ICCV.2017.324   DOI
10 Park, J.J., S.W. Oh, K.-A. Park, M.-S. Lee, J.-C. Jang, and M.J. Lee, 2018. A methodology of ship detection using high-resolution satellite optical image, Journal of the Korean Earth Science Society, 39(3): 241-249 (in Korean with English abstract). https://doi.org/10.5467/JKESS.2018.39.3.241   DOI
11 Stofa, M.M., M.A. Zulkifley, and S.Z.M. Zaki, 2020. A deep learning approach to ship detection using satellite imagery, IOP Conference Series: Earth and Environmental Science, 540: 012049. https://doi.org/10.1088/1755-1315/540/1/012049   DOI
12 Xie, X., L. Li, Z. An, G. Lu, and Z. Zhou, 2022. Small Ship Detection Based on Hybrid Anchor Structure and Feature Super-Resolution, Remote Sensing, 14(15): 3530. https://doi.org/10.3390/rs14153530   DOI
13 Yang, X., H. Sun, K. Fu, J. Yang, X. Sun, M. Yan, and Z. Guo, 2018. Automatic ship detection in remote sensing images from google earth of complex scenes based on multiscale rotation dense feature pyramid networks, Remote Sensing, 10(1): 132. https://doi.org/10.3390/rs10010132   DOI
14 Tian, Z., C. Shen, H. Chen, and T. He, 2019. FCOS: Fully convolutional one-stage object detection, Proc. of 2019 IEEE/CVF International Conference on Computer Vision (ICCV), Seoul, South Korea, Oct. 27-Nov. 2, pp. 9626-9635. https://doi.org/10.1109/ICCV.2019.00972   DOI
15 Padilla, R., S.L. Netto, and E.A. Da Silva, 2020. A survey on performance metrics for object-detection algorithms, Proc. of 2020 International Conference on Systems, Signals and Image Processing (IWSSIP), Niteroi, Brazil, Jul. 1-3, pp. 237-242. https://doi.org/10.1109/IWSSIP48289.2020.9145130   DOI
16 Ren, S., K. He, R. Girshick, and J. Sun, 2015. Faster R-CNN: Towards real-time object detection with region proposal networks, arXiv preprint arXiv: 1506.01497. https://doi.org/10.48550/arXiv.1506.01497   DOI
17 Shermeyer, J. and A. Van Etten, 2019. The effects of super-resolution on object detection performance in satellite imagery, Proc. of 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), Long Beach, CA, USA, Jun 16-17, pp. 1432-1441. https://doi.org/10.1109/CVPRW.2019.00184   DOI
18 Wang, Z., A.C. Bovik, H.R. Sheikh, and E.P. Simoncelli, 2004. Image quality assessment: from error visibility to structural similarity, IEEE Transactions on Image Processing, 13(4): 600-612. https://doi.org/10.1109/TIP.2003.819861   DOI
19 Yoo, J., N. Ahn, and K.-A. Sohn, 2020. Rethinking data augmentation for image super-resolution: A comprehensive analysis and a new strategy, Proc. of 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Seattle, WA, USA, Jun 13-19, pp. 8372-8381. https://doi.org/10.1109/CVPR42600.2020.00840   DOI
20 Yang, W., X. Zhang, Y. Tian, W. Wang, J.H. Xue, and Q. Liao, 2019. Deep learning for single image super-resolution: A brief review, IEEE Transactions on Multimedia, 21(12): 3106-3121. https://doi.org/10.1109/TMM.2019.2919431   DOI
21 He, S., H. Zou, Y. Wang, R. Li, and F. Cheng, 2021a. ShipSRDet: An end-to-end remote sensing ship detector using super-resolved feature representation. Proc. of 2021 IEEE International Geoscience and Remote Sensing Symposium IGARSS, Brussels, Belgium, Jul. 11-16, pp. 3541-3544. https://doi.org/10.1109/IGARSS47720.2021.9554079   DOI
22 Choi, Y. J., M.S. Kim, Y.W. Kim, and S.H. Han, 2020. A study of CNN-based super-resolution method for remote sensing image, Korean Journal of Remote Sensing, 36(3): 449-460 (in Korean with English abstract). https://doi.org/10.7780/kjrs.2020.36.3.5   DOI
23 Chen, H., X. He, L. Qing, Y. Wu, C. Ren, R.E. Sheriff, and C. Zhu, 2022. Real-world single image superresolution: A brief review, Information Fusion, 79: 124-145. https://doi.org/10.48550/arXiv.2103.02368   DOI
24 Haris, M., G. Shakhnarovich, and N. Ukita, 2018. Taskdriven super resolution: Object detection in lowresolution images, arXiv preprint arXiv:1803. 11316. https://doi.org/10.48550/arXiv.1803.11316   DOI
25 Li, L., Z. Zhou, B. Wang, L. Miao, and H. Zong, 2020. A novel CNN-based method for accurate ship detection in HR optical remote sensing images via rotated bounding box, IEEE Transactions on Geoscience and Remote Sensing, 59(1): 686-699. https://doi.org/10.1109/TGRS.2020.2995477   DOI