Journal of the Korean Society of Surveying, Geodesy, Photogrammetry and Cartography (한국측량학회지)

Korean Society of Surveying, Geodesy, Photogrammetry and Cartography (한국측량학회)
권호 표지
DOI QR코드

DOI QR Code

Change Detection for High-resolution Satellite Images Using Transfer Learning and Deep Learning Network

전이학습과 딥러닝 네트워크를 활용한 고해상도 위성영상의 변화탐지

  • Song, Ah Ram (Dept. of Civil and Environmental Engineering, Seoul National University) ;
  • Choi, Jae Wan (School of Civil Engineering, Chungbuk National University) ;
  • Kim, Yong Il (Dept. of Civil and Environmental Engineering, Seoul National University)
  • Received : 2019.06.04
  • Accepted : 2019.06.21
  • Published : 2019.06.30
Download PDF
⟨ Previous Next ⟩

Abstract

As the number of available satellites increases and technology advances, image information outputs are becoming increasingly diverse and a large amount of data is accumulating. In this study, we propose a change detection method for high-resolution satellite images that uses transfer learning and a deep learning network to overcome the limit caused by insufficient training data via the use of pre-trained information. The deep learning network used in this study comprises convolutional layers to extract the spatial and spectral information and convolutional long-short term memory layers to analyze the time series information. To use the learned information, the two initial convolutional layers of the change detection network are designed to use learned values from 40,000 patches of the ISPRS (International Society for Photogrammertry and Remote Sensing) dataset as initial values. In addition, 2D (2-Dimensional) and 3D (3-dimensional) kernels were used to find the optimized structure for the high-resolution satellite images. The experimental results for the KOMPSAT-3A (KOrean Multi-Purpose SATllite-3A) satellite images show that this change detection method can effectively extract changed/unchanged pixels but is less sensitive to changes due to shadow and relief displacements. In addition, the change detection accuracy of two sites was improved by using 3D kernels. This is because a 3D kernel can consider not only the spatial information but also the spectral information. This study indicates that we can effectively detect changes in high-resolution satellite images using the constructed image information and deep learning network. In future work, a pre-trained change detection network will be applied to newly obtained images to extend the scope of the application.

운용 가능한 위성의 수가 증가하고 기술이 진보함에 따라 영상정보의 성과물이 다양해지고 많은 양의 자료가 축적되고 있다. 본 연구에서는 기구축된 영상정보를 활용하여 부족한 훈련자료의 문제를 극복하고 딥러닝(deep learning) 기법의 장점을 활용하고자 전이학습과 변화탐지 네트워크를 활용한 고해상도 위성영상의 변화탐지를 수행하였다. 본 연구에서 활용한 딥러닝 네트워크는 공간 및 분광 정보를 추출하는 합성곱 레이어(convolutional layer)와 시계열 정보를 분석하는 합성곱 장단기 메모리 레이어(convolutional long short term memory layer)로 구성되었으며, 고해상도 다중분광 영상에 최적화된 정보를 추출하기 위하여 커널(kernel)의 차원에 따른 정확도를 비교하였다. 또한, 학습된 커널 정보를 활용하기 위하여 변화탐지 네트워크의 초기 합성곱 레이어를 고해상도 항공영상인 ISPRS (International Society for Photogrammetry and Remote Sensing) 데이터셋에서 추출된 40,000개의 패치로 학습된 값으로 초기화하였다. 다시기 KOMPSAT-3A (KOrean Multi-Purpose SATllite-3A) 영상에 대한 실험 결과, 전이학습과 딥러닝 네트워크를 활용할 경우 기복 변위 및 그림자 등으로 인한 변화에 덜 민감하게 반응하며 분류 항목이 달라진 지역의 변화를 보다 효과적으로 추출할 수 있었으며, 2차원 커널보다 3차원 커널을 사용할 때 변화탐지의 정확도가 높았다. 3차원 커널은 공간 및 분광정보를 모두 고려하여 특징 맵(feature map)을 추출하기 때문에 고해상도 영상의 분류뿐만 아니라 변화탐지에도 효과적인 것을 확인하였다. 본 연구에서는 고해상도 위성영상의 변화탐지를 위한 전이학습과 딥러닝 기법의 활용 가능성을 제시하였으며, 추후 훈련된 변화탐지 네트워크를 새롭게 취득된 영상에 적용하는 연구를 수행하여 제안기법의 활용범위를 확장할 예정이다.

Keywords

GCRHBD_2019_v37n3_199_f0001.png 이미지

Fig. 1. The example of ISPRS 2D Semantic Labelling Challenge datasets (http://www2.isprs.org/commissions/comm3/wg4/semantic-labeling.html)

GCRHBD_2019_v37n3_199_f0002.png 이미지

Fig. 2. Two study sites with multi-temporal KOMPSAT-3A CIR (Color InfraRed) imagery; site 1 acquired at times (a) T1 , (b) T2 and site 2 acquired at times (c)T1, (d) T2

GCRHBD_2019_v37n3_199_f0003.png 이미지

Fig. 3. The framework of transfer learning and change detection network

GCRHBD_2019_v37n3_199_f0004.png 이미지

Fig. 4. Comparison of (a) 2D and (b) 3D (three-dimensional) convolution operation (Song, 2019)

GCRHBD_2019_v37n3_199_f0005.png 이미지

Fig. 5. The fully convolutional network for multi-spectral image classification

GCRHBD_2019_v37n3_199_f0006.png 이미지

Fig. 6. Change detection maps obtained from the proposed method and other methods for site 1 (a) PCA with SVM, (b) fully connected LSTM, (c) CD network with 2d kernel, (d) CD network with 3d kernel, and (e) ground truth (white color represents changed areas and black color represents unchanged areas)

GCRHBD_2019_v37n3_199_f0007.png 이미지

Fig. 7. Change detection maps obtained from the proposed method and other methods for site 2 (a) PCA with SVM, (b) fully connected LSTM, (c) CD network with 2d kernel, (d) CD network with 3d kernel, and (e) ground truth (white color represents changed areas and black color represents unchanged areas)

GCRHBD_2019_v37n3_199_f0008.png 이미지

Fig. 8. The enlarged images of the change detection results (a) site 1-A, (b) site 1-B, (c) site 1-C, and (d) site 2-A (white color represents changed areas and black color represents unchanged areas)

Table 1. Accuracy comparison of change detection results on site 1 and site 2

GCRHBD_2019_v37n3_199_t0001.png 이미지

References

  1. Choi, J. (2015), Unsupervised change detection for very high-spatial resolution satellite imagery by using object-based IR-MAD algorithm, Journal of the Korean Society of Surveying, Geodesy, Photogrammetry and Cartography, Vol. 33, No. 4, pp. 297-304. (in Korean with English abstract) https://doi.org/10.7848/ksgpc.2015.33.4.297
  2. Choi, J., Kim, G., Park, N., Park, H., and Choi, S. (2017), A hybrid pansharpening algorithm of VHR satellite images that employs injection gains based on NDVI to reduce computational costs, Remote Sensing, Vol. 9, No. 10. pp. 976. https://doi.org/10.3390/rs9100976
  3. Chung, M., Han, Y., Choi, J., and Kim, Y. (2018), Optimal parameter analysis and evaluation of change detection for SLIC-based superpixel techniques using KOMPSAT Data, Korean Journal of Remote Sensing, Vol. 34, No. 6_3, pp. 1427-1443. (in Korean with English abstract) https://doi.org/10.7780/KJRS.2018.34.6.3.9
  4. Dellinger, F., Delon, J., Gousseau, Y., Michel, J., and Tupin, F. (2014), Change detection for high resolution satellite images, based on SIFT descriptors and an a contrario approach, In 2014 IEEE Geoscience and Remote Sensing Symposium 2014, 13-18 July, Quebec, Canada, pp. 1281-1284.
  5. Deng, J.S., Wang, K., Deng, Y.H., and Qi, G.J. (2008), PCA-based land-use change detection and analysis using multitemporal and multisensor satellite data, International Journal of Remote Sensing, Vol. 29, No. 16, pp. 4823-4838. https://doi.org/10.1080/01431160801950162
  6. Han, Y., Kim, T., Han, S., and Song, J. (2017), Change detection of urban development over large area using KOMPSAT optical imagery, Korean Journal of Remote Sensing, Vol. 33, No. 6-3, pp. 1223-1232. (in Korean with English abstract) https://doi.org/10.7780/KJRS.2017.33.6.3.6
  7. Hussain, M., Chen, D., Cheng, A., Wei, H., and Stanley, D. (2013), Change detection from remotely sensed images: from pixel-based to object-based approaches, ISPRS Journal of photogrammetry and remote sensing, Vol. 80, pp. 91-106. https://doi.org/10.1016/j.isprsjprs.2013.03.006
  8. Jeong, J. (2005), Developments of urban change detection methods according to spatial resolution of satellite images -application of KOMPSAT 1 images into urban area-, The Geographical Journal of Korea, Vol. 39, No. 1, pp. 161-170. (in Korean with English abstract)
  9. Ji, S., Zhang, C., Xu, A., Shi, Y., and Duan, Y. (2018), 3D convolutional neural networks for crop classification with multi-temporal remote sensing images, Remote Sensing, Vol. 10, No. 1, pp. 75. https://doi.org/10.3390/rs10010075
  10. Lim, K., Jin, D., and Kim, C.S. (2018), Change detection in high resolution satellite images using an ensemble of convolutional neural networks, In 2018 Asia-Pacific Signal and Information Processing Association Annual Summit and Conference (APSIPA ASC), 12-15 November, Honolulu, Hawaii, USA, pp. 509-515.
  11. Li, Y., Zhang, H., Xue, X., Jiang, Y., and Shen, Q. (2018), Deep learning for remote sensing image classification: A survey, Wiley Interdisciplinary Reviews: Data Mining and Knowledge Discovery, Vol. 8, No. 6, pp. e1264.
  12. Liu, B., Yu, X., Yu, A., and Wan, G. (2018), Deep convolutional recurrent neural network with transfer learning for hyperspectral image classification, Journal of Applied Remote Sensing, Vol. 12, No. 2, pp. 026028.
  13. Lyu, H., Lu, H., and Mou, L. (2016), Learning a transferable change rule from a recurrent neural network for land cover change detection, Remote Sensing, Vol. 8, No. 6, pp. 506. https://doi.org/10.3390/rs8060506
  14. Mei, S., Yuan, X., Ji, J., Zhang, Y., Wan, S., and Du, Q. (2017), Hyperspectral image spatial super-resolution via 3D full convolutional neural network, Remote Sensing, Vol. 9, No. 11, pp. 1139. https://doi.org/10.3390/rs9111139
  15. Mou, L., Bruzzone, L., and Zhu, X.X. (2019), Learning spectral-spatial-temporal features via a recurrent convolutional neural network for change detection in multispectral imagery, IEEE Transactions on Geoscience and Remote Sensing, Vol. 57, No. 2, pp. 924-935. https://doi.org/10.1109/TGRS.2018.2863224
  16. Song, A., Choi, J., Han, Y., and Kim, Y. (2018), Change detection in hyperspectral images using recurrent 3D fully convolutional networks, Remote Sensing, Vol. 10, No. 11, pp. 1827. https://doi.org/10.3390/rs10111827
  17. Song, A. (2019), A Novel Deep Learning Framework for Multi-Class Change Detection of Hyperspectral Images, Doctoral Thesis, Seoul National University, Seoul, Korea, 137p.
  18. Tan, K., Zhang, Y., Wang, X., and Chen, Y. (2019), Object-based change detection using multiple classifiers and multi-scale uncertainty analysis, Remote Sensing, Vol. 11, No. 3, pp. 359. https://doi.org/10.3390/rs11030359
  19. Wang, W., Dou, S., Jiang, Z., and Sun, L. (2018), A fast dense spectral-spatial convolution network framework for hyperspectral images classification, Remote Sensing, Vol. 10, No. 7, pp. 1068. https://doi.org/10.3390/rs10071068
  20. Wang, Q., Yuan, Z., Du, Q., and Li, X. (2018), Getnet: A general end-to-end 2-d CNN framework for hyperspectral image change detection, IEEE Transactions on Geoscience and Remote Sensing, Vol. 57, No. 1, pp. 3-13. https://doi.org/10.1109/tgrs.2018.2849692
  21. Wang, Q., Zhang, X., Chen, G., Dai, F., Gong, Y., and Zhu, K. (2018), Change detection based on Faster R-CNN for high-resolution remote sensing images, Remote sensing letters, Vol. 9, No. 10, pp. 923-932. https://doi.org/10.1080/2150704X.2018.1492172
  22. Wiratama, W., Lee, J., Park, S. E., and Sim, D. (2018), Dual-dense convolution network for change detection of high-resolution panchromatic imagery, Applied Sciences, Vol. 8, No. 10, pp. 1785. https://doi.org/10.3390/app8101785
  23. Xingjian, S.H.I., Chen, Z., Wang, H., Yeung, D.Y., Wong, W.K., and Woo, W.C. (2015), Convolutional LSTM network: A machine learning approach for precipitation nowcasting, In Advances in Neural Information Processing Systems, 7-12 December, Montreal, Canada, pp. 802-810.
  24. Yu, H., Yang, W., Hua, G., Ru, H., and Huang, P. (2017), Change detection using high resolution remote sensing images based on active learning and markov random fields, Remote Sensing, Vol. 9, No. 12, pp. 1233. https://doi.org/10.3390/rs9121233
  25. Zhang, C., Wei, S., Ji, S., and Lu, M. (2019), Detecting large-scale urban land cover changes from very high resolution remote sensing images using CNN-based classification, ISPRS International Journal of Geo-Information, Vol. 8, No. 4, pp. 189. https://doi.org/10.3390/ijgi8040189

Cited by

  1. 최근 다목적실용위성 시리즈 활용 현황 vol.36, pp.6, 2019, https://doi.org/10.7780/kjrs.2020.36.6.2.1
  2. SPADE 기반 U-Net을 이용한 고해상도 위성영상에서의 도시 변화탐지 vol.36, pp.6, 2019, https://doi.org/10.7780/kjrs.2020.36.6.2.8
  3. 원격 탐사 변화 탐지를 위한 변화 주목 기반의 덴스 샴 네트워크 vol.26, pp.1, 2021, https://doi.org/10.5909/jbe.2021.26.1.14