Journal of Korea Multimedia Society (한국멀티미디어학회논문지)

Korea Multimedia Society (한국멀티미디어학회)
권호 표지
DOI QR코드

DOI QR Code

Satellite Building Segmentation using Deformable Convolution and Knowledge Distillation

변형 가능한 컨볼루션 네트워크와 지식증류 기반 위성 영상 빌딩 분할

  • Received : 2022.06.10
  • Accepted : 2022.07.15
  • Published : 2022.07.31
Download PDF
⟨ Previous Next ⟩

Abstract

Building segmentation using satellite imagery such as EO (Electro-Optical) and SAR (Synthetic-Aperture Radar) images are widely used due to their various uses. EO images have the advantage of having color information, and they are noise-free. In contrast, SAR images can identify the physical characteristics and geometrical information that the EO image cannot capture. This paper proposes a learning framework for efficient building segmentation that consists of a teacher-student-based privileged knowledge distillation and deformable convolution block. The teacher network utilizes EO and SAR images simultaneously to produce richer features and provide them to the student network, while the student network only uses EO images. To do this, we present objective functions that consist of Kullback-Leibler divergence loss and knowledge distillation loss. Furthermore, we introduce deformable convolution to avoid pixel-level noise and efficiently capture hard samples such as small and thin buildings at the global level. Experimental result shows that our method outperforms other methods and efficiently captures complex samples such as a small or narrow building. Moreover, Since our method can be applied to various methods.

Keywords

Acknowledgement

This research was supported by a grant-in-aid of HANHWA SYSTEMS.

References

  1. M. Kim, S. Kim, D. Lee, and J. Gahm, "Comparative Study of Deep Learning Model for Semantic Segmentation of Water System in SAR Images of KOMPSAT-5," Journal of Korea Multimedia Society, Vol. 25, No. 2, pp. 206-214, 2022. https://doi.org/10.9717/KMMS.2022.25.2.206
  2. J. Long, E. Shelhamer, and T. Darrell, "Fully Convolutional Networks for Semantic Segmentation," Proceedings of the IEEE Conference on Computer Vision and P attern Recognition, pp. 3431-3440, 2015.
  3. O. Ronneberger, P. Fischer, and T. Brox, "U-net: Convolutional Networks for Biomedical Image Segmentation," International Conference on Medical Image Computing and Computer-assisted Intervention, pp. 234-241, 2015.
  4. H. Zhao, J. Shi, X. Qi, X. Wang, and J. Jia, "Pyramid Scene Parsing Network," Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2881-2890, 2017.
  5. L.C. Chen, G. Papandreou, F. Schroff, and H. Adam, "Rethinking Atrous Convolution for Semantic Image Segmentation," arXiv P reprint, arXiv:1706.05587, 2017.
  6. H. Kwon, T. Song, T. Lee, J. Ahn, and K. Sohn, "Few-shot Aerial Image Segmentation with Mask-Guided Attention," Journal of Korea Multimedia Society, Vol. 25, No. 5, pp. 685-694, 2022. https://doi.org/10.9717/KMMS.2022.25.5.685
  7. N. Girard, D. Smirnov, J. Solomon, and Y. Tarabalka, "Polygonal Building Extraction by Frame Field Learning," Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 5891-5900, 2021.
  8. V. Vapnik and R. Izmailov, "Learning Using Privileged Information: Similarity Control and Knowledge Transfer," Journal of Machine Learning Research, Vol. 16, No. 1, pp. 2023-2049, 2015.
  9. G. Hinton, O. Vinyals, and J. Dean, "Distilling the Knowledge in a Neural Network," arXiv P reprint, arXiv:1503.02531, 2015.
  10. J. Dai, H. Qi, Y. Xiong, Y. Li, G. Zhang, H. Hu, and Y. Wei, "Deformable Convolutional Networks," Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 764-773, 2017.
  11. A.G. Roy, N. Navab, and C. Wachinger, "Recalibrating Fully Convolutional Networks with Spatial and Channel "Squeeze and Excitation" Blocks," IEEE Transactions on Medical Imaging, Vol. 38, No. 2, pp. 540-549, 2018. https://doi.org/10.1109/tmi.2018.2867261
  12. F. Milletari, N. Navab, and S.A. Ahmadi, "V-net: Fully Convolutional Neural Networks for Volumetric Medical Image Segmentation," International Conference on 3D Vision, pp. 565-571, 2016.
  13. J. Shermeyer, D. Hogan, J. Brown, A.V. Etten, N. Weir, F. Pacifici, et al., "Spacenet 6: Multi-Sensor All Weather Mapping Dataset," Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 196-197, 2020.
  14. K. He, X. Zhang, S. Ren, and J. Sun, "Deep Residual Learning for Image Recognition," Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 770-778, 2016.
  15. A.V. Etten, D. Lindenbaum, and T.M. Bacastow, "Spacenet: A Remote Sensing Dataset and Challenge Series," arXiv P reprint, arXiv: 1807.01232, 2018.