Browse > Article
http://dx.doi.org/10.9717/kmms.2022.25.5.685

Few-shot Aerial Image Segmentation with Mask-Guided Attention  

Kwon, Hyeongjun (School of Electrical and Electronic Engineering, Yonsei University)
Song, Taeyong (School of Electrical and Electronic Engineering, Yonsei University)
Lee, Tae-Young (Intelligence SW Team, Hanwha Systems)
Ahn, Jongsik (Intelligence SW Team, Hanwha Systems)
Sohn, Kwanghoon (School of Electrical and Electronic Engineering, Yonsei University)
Publication Information
Journal of Korea Multimedia Society / v.25, no.5, 2022 , pp. 685-694 More about this Journal
Abstract
The goal of few-shot semantic segmentation is to build a network that quickly adapts to novel classes with extreme data shortage regimes. Most existing few-shot segmentation methods leverage single or multiple prototypes from extracted support features. Although there have been promising results for natural images, these methods are not directly applicable to the aerial image domain. A key factor in few-shot segmentation on aerial images is to effectively exploit information that is robust against extreme changes in background and object scales. In this paper, we propose a Mask-Guided Attention module to extract more comprehensive support features for few-shot segmentation in aerial images. Taking advantage of the support ground-truth masks, the area correlated to the foreground object is highlighted and enables the support encoder to extract comprehensive support features with contextual information. To facilitate reproducible studies of the task of few-shot semantic segmentation in aerial images, we further present the few-shot segmentation benchmark iSAID-, which is constructed from a large-scale iSAID dataset. Extensive experimental results including comparisons with the state-of-the-art methods and ablation studies demonstrate the effectiveness of the proposed method.
Keywords
Semantic Segmentation; Aerial Image; Deep Learning; Few-shot Learning;
Citations & Related Records
연도 인용수 순위
  • Reference
1 S. Park and Y. S. Heo, "Multi-Path Feature Fusion Module for Semantic Segmentation," Journal of Korea Multimedia Society, Vol. 24, No. 1, pp. 1-12, 2021.   DOI
2 J. Wang, C. Ding, S. Chen, C. He, and B. Luo, "Semi-Supervised Remote Sensing Image Semantic Segmentation via Consistency Regularization and Average Update of Pseudo-Label," Remote Sensing, Vol. 12, No. 21, pp. 3603, 2020.   DOI
3 B. Yang, C. Liu, B. Li, J. Jiao, and Q. Ye, "Prototype Mixture Models for Few-Shot Semantic Segmentation," European Conference on Computer Vision, pp. 763-778, 2020.
4 S. Paisitkriangkrai, J. Sherrah, P. Janney, and A.V.-D. Hengel, "Effective Semantic Pixel Labelling With Convolutional Networks and Conditional Random Fields," IEEE Conference on Computer Vision and Pattern Recognition Workshops, pp. 36-43, 2015.
5 C. Zhang, I. Sargent, X. Pan, A. Gardiner, J. Hare and P.M. Atkinson, "VPRS-Based Regional Decision Fusion of CNN and MRF Classifications for Very Fine Resolution Remotely Sensed Images," IEEE Transactions on Geoscience and Remote Sensing, Vol. 56, No. 8, pp. 4507-4521, 2018.   DOI
6 D. Marmanis, K. Schindler, J.D. Wegner, S. Galliani, M. Datcu, and U. Stilla, "Classification with an Edge: Improving Semantic Image Segmentation with Boundary Detection," ISPRS Journal of Photogrammetry and Remote Sensing, Vol. 135, pp. 158-172, 2018.   DOI
7 C. Zhang, G. Lin, F. Liu, J. Guo, Q. Wu, and R. Yao, "Pyramid Graph Networks with Connection Attentions for Region-Based OneShot Semantic Segmentation," IEEE/CVF International Conference on Computer Vision, pp. 9586-9594, 2019.
8 K. Wang, J.H. Liew, Y. Zou, D. Zhou, and J. Feng, "PANet: Few-Shot Image Semantic Segmentation with Prototype Alignment," IEEE/CVF International Conference on Computer Vision, pp. 9196-9205, 2019.
9 Z. Shao, W. Zhou, X. Deng, M. Zhang, and Q. Cheng, "Multilabel Remote Sensing Image Retrieval Based on Fully Convolutional Network," IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, Vol. 13, pp. 318-328, 2020.   DOI
10 C. Zhang, G. Lin, F. Liu, R. Yao, and C. Shen, "CANet: Class-Agnostic Segmentation Networks with Iterative Refinement and Attentive Few-Shot Learning," IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 5212-5221, 2019.
11 M. Siam, B. Oreshkin, and M. Jagersand, "AMP: Adaptive Masked Proxies for FewShot Segmentation," IEEE/CVF International Conference on Computer Vision, pp. 5248-5257, 2019.
12 B. Huang, B. Zhao, and Y. Song, "Urban Land-Use Mapping Using a Deep Convolutional Neural Network with High Spatial Resolution Multispectral Remote Sensing Imagery," Remote Sensing of Environment, Vol. 214, pp. 73-86, 2018.   DOI
13 L. Ding, H. Tang, and L. Bruzzone, "LANet: Local Attention Embedding to Improve the Semantic Segmentation of Remote Sensing Images," IEEE Transactions on Geoscience and Remote Sensing, Vol. 59, No. 1, pp. 426-435, 2021.   DOI
14 B. Yu, L. Yang, and F. Chen, "Semantic Segmentation for High Spatial Resolution Remote Sensing Images Based on Convolution Neural Network And Pyramid Pooling Module," IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, Vol. 11, No. 9, pp. 3252-3261, 2018.   DOI
15 L. Ding, J. Zhang, and L. Bruzzone, "Semantic Segmentation of Large-Size VHR Remote Sensing Images Using a Two-Stage Multiscale Training Architecture," IEEE Transactions on Geoscience and Remote Sensing, Vol. 58, No. 8, pp. 5367-5376, 2020.   DOI
16 B. Chaudhuri, B. Demir, S. Chaudhuri, and L. Bruzzone, "Multilabel Remote Sensing Image Retrieval Using a Semisupervised GraphTheoretic Method," IEEE Transactions on Geoscience and Remote Sensing, Vol. 56, No. 2, pp. 1144-1158, 2018.   DOI
17 J. Castillo-Navarro, B. Le Saux, A. Boulch, N. Audebert, and S. Lefevre, "Semi-Supervised Semantic Segmentation in Earth Observation: The Minifrance Suite, Dataset Analysis and Multi-Task Network Study," Machine Learning, pp. 1-36, 2021.
18 X. Zhang, Y. Wei, Y. Yang, and T.S. Huang, "SG-One: Similarity Guidance Network for One-Shot Semantic Segmentation," IEEE Transactions on Cybernetics, Vol. 50, No. 9, pp. 3855-3865, 2020.   DOI
19 Z. Tian, H. Zhao, M. Shu, Z. Yang, R. Li, and J. Jia, "Prior Guided Feature Enrichment Network for Few-Shot Segmentation." IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 44, No. 2, pp. 1050-1065, 2022.   DOI
20 O. Vinyals, C. Blundell, T. Lillicrap, K. Kavukcuoglu, and D. Wierstra, "Matching Networks for One Shot Learning," Advances in Neural Information P rocessing Systems, Vol. 29, pp. 3630-3638, 2016.
21 S.W. Zamir et al., "iSAID: A Large-Scale Dataset for Instance Segmentation in Aerial Images," IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshop, pp. 28-37, 2019.
22 G. Moser, S.B. Serpico, and J.A. Benediktsson, "Land-Cover Mapping by Markov Modeling of Spatial-Contextual Information in Very-High-Resolution Remote Sensing Images," Proceedings of the IEEE, Vol. 101, No. 3, pp. 631-651, 2013.   DOI
23 L. Mi and Z. Chen. "Superpixel-Enhanced Deep Neural Forest for Remote Sensing Image Semantic Segmentation," ISPRS Journal of Photogrammetry and Remote Sensing, Vol. 159, pp. 140-152, 2020.   DOI
24 M. Dalponte, L. Bruzzone, and D. Gianelle, "Tree Species Classification in the South9 Alps based on the Fusion of Very High Geometrical Resolution Multispectral/ Hyperspectral Images and LiDAR Data," Remote Sensing of Environment, Vol. 123, pp. 258-270, 2012.   DOI
25 J.V. Solrzano, J.A. Meave, J.A. Gallardo- Cruz, E.J. Gonzlez, and J.L. Hernndez- Stefanoni, "Predicting Old-Growth Tropical Forest Attributes from Very High Resolution (VHR)-Derived Surface Metrics," International Journal of Remote Sensing, Vol. 38, No. 2, pp. 492-513, 2017.   DOI
26 H. Zhou, H. Kong, L. Wei, D. Creighton, and S. Nahavandi, "On Detecting Road Regions in a Single UAV Image," IEEE Transactions on Intelligent Transportation Systems, Vol. 18, No. 7, pp. 1713-1722, 2017.   DOI
27 S. Liu, W. Ding, C. Liu, Y. Liu, Y. Wang, and H. Li, "ERN: Edge Loss Reinforced Semantic Segmentation Network for Remote Sensing Images," Remote Sensing, Vol. 10, No. 9, p. 1339, 2018.   DOI
28 K. He, X. Zhang, S. Ren, and J. Sun, "Deep Residual Learning for Image Recognition," IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 770-778, 2016.
29 Y. Yi, Z. Zhang, and W. Zhang, "Building Segmentation of Aerial Images in Urban Areas with Deep Convolutional Neural Networks," Conference of the Arabian Journal of Geosciences, pp. 61-64, 2018.