Journal of the Korean Society of Surveying, Geodesy, Photogrammetry and Cartography (한국측량학회지)

Korean Society of Surveying, Geodesy, Photogrammetry and Cartography (한국측량학회)
권호 표지
DOI QR코드

DOI QR Code

Comparative Experiment of Cloud Classification and Detection of Aerial Image by Deep Learning

딥러닝에 의한 항공사진 구름 분류 및 탐지 비교 실험

  • Song, Junyoung (Dept. of Advanced Technology Fusion, Konkuk University) ;
  • Won, Taeyeon (Dept. of Advanced Technology Fusion, Konkuk University) ;
  • Jo, Su Min (Dept. of Advanced Technology Fusion, Konkuk University) ;
  • Eo, Yang Dam (Dept. of Civil and Environmental Engineering, Konkuk University) ;
  • Park, So young (Geographic Information Division, National Geographic Information Institute, Ministry of Land, Infrastructure and Transport) ;
  • Shin, Sang ho (Geographic Information Division, National Geographic Information Institute, Ministry of Land, Infrastructure and Transport) ;
  • Park, Jin Sue (Project Development Division, ALLforLAND.Co.Ltd) ;
  • Kim, Changjae (Dept. of Civil and Environmental Engineering, Myongji University,)
  • Received : 2021.11.14
  • Accepted : 2021.12.18
  • Published : 2021.12.31
Download PDF
⟨ Previous Next ⟩

Abstract

As the amount of construction for aerial photography increases, the need for automation of quality inspection is emerging. In this study, an experiment was performed to classify or detect clouds in aerial photos using deep learning techniques. Also, classification and detection were performed by including satellite images in the learning data. As algorithms used in the experiment, GoogLeNet, VGG16, Faster R-CNN and YOLOv3 were applied and the results were compared. In addition, considering the practical limitations of securing erroneous images including clouds in aerial images, we also analyzed whether additional learning of satellite images affects classification and detection accuracy in comparison a training dataset that only contains aerial images. As results, the GoogLeNet and YOLOv3 algorithms showed relatively superior accuracy in cloud classification and detection of aerial images, respectively. GoogLeNet showed producer's accuracy of 83.8% for cloud and YOLOv3 showed producer's accuracy of 84.0% for cloud. And, the addition of satellite image learning data showed that it can be applied as an alternative when there is a lack of aerial image data.

항공사진 촬영량이 증가함에 따라 품질검사 자동화의 필요성이 대두되고 있다. 본 연구에서는 딥러닝 기법으로 항공사진 내 구름을 분류 또는 탐지하는 실험을 수행하였고, 또한 위성영상을 학습자료에 포함시켜 분류 및 탐지를 수행하였다. 실험에 사용한 알고리즘으로는 GoogLeNet, VGG16, Faster R-CNN과 YOLOv3을 적용하여 결과를 비교하였다. 또한 구름이 포함된 오류영상 확보의 현실적 제한을 고려하여 항공영상만 존재하는 학습 데이터세트에서 위성영상을 활용한 추가학습이 분류 및 탐지정확도에 영향을 미치는지도 분석하였다. 실험결과, 항공사진의 구름 분류와 탐지에서 각각 GoogLeNet과 YOLOv3 알고리즘이 상대적으로 우월한 정확도를 나타냈고, GoogLeNet은 구름에 대한 생산자정확도 83.8% 그리고 YOLOv3는 구름에 대한 생산자정확도 84.0%를 보여주었다. 또한, 위성영상 학습자료 추가가 항공사진 자료의 부족 시 대안으로 적용가능 함을 보여주었다.

Keywords

Acknowledgement

본 연구는 2021년 국토지리정보원 "항공영상 품질검사 자동화체계 연구"사업의 지원을 받아 수행된 연구임.

References

  1. Cho, W., Lee, S.H., Choi, S.S., Hwang, H.D., and Lee, H.J. (2003), Quality Improvement on Aerial Photographs, Journal of the Korean Society of Surveying, Geodesy, Photogrammetry and Cartography, Vol. 21, No. 3, pp. 277-287. (in Korean with English abstract)
  2. Girshick, R., Donahue, J., Darrell, T., and Malik, J. (2014), Rich Feature Hierarchies for Accurate Object Detection and Semantic Segmentation, IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 24-27 June, Columbus, Ohio, pp. 580-587.
  3. He, K., Gkioxari, G., Dollar, P., and Girshick, R. (2017), Mask R-CNN, IEEE International Conference on Computer Vision (ICCV), 22-29 October, Venice, Italy, pp. 2961-2969.
  4. He, K., Zhang, X., Ren, S., and Sun, J. (2015), Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification, IEEE International Conference on Computer Vision (ICCV), 13-16 December, Santiago, Chile, pp. 1026-1034.
  5. Kim, B.H., Kim, Y.H., Han, Y.K., Choi, W.S., and Kim, Y.I. (2014), Fully Automated Generation of Cloud-free Imagery Using Landsat-8, Journal of the Korean Society of Surveying, Geodesy, Photogrammetry and Cartography, Vol. 32, No. 2, pp. 133-142. (in Korean with English abstract) https://doi.org/10.7848/KSGPC.2014.32.2.133
  6. Kim, B., Kim, G., Jin, S., and Cho, S. (2019), A Comparative Study on Performance of Deep Learning Models for Vision-based Concrete Crack Detection according to Model Types, Journal of the Korean Society of Safety, Vol. 34, No. 6, pp. 50-57. (in Korean with English abstract) https://doi.org/10.14346/JKOSOS.2019.34.6.50
  7. Krizhevsky, A., Sutskever, I., and Hinton, G. E. (2012), ImageNet classification with deep convolutional neural networks, In Proceedings of NIPS, pp. 1106-1114.
  8. Lecun, Y., Bottou, L., Bengio, Y., and Haffner, P. (1998), Gradient-based learning applied to document recognition, IEEE, vol. 86, no. 11, pp. 2278-2324. https://doi.org/10.1109/5.726791
  9. Lee, M.H., Cheon, E.J., and Eo, Y.D. (2019), Cloud Detection and Restoration of Landsat-8 using STARFM, Korean Journal of Remote Sensing, Vol. 35, No. 5-2, pp. 861-871. (in Korean with English abstract)
  10. Lee, J.O., and Sung, S.M. (2019), Quality Evaluation of UAV Images Using Resolution Target, Journal of the Korean Association of Geographic Information Studies, Vol. 22, No. 1, pp. 103-113. (in Korean with English abstract) https://doi.org/10.11108/KAGIS.2019.22.1.103
  11. Lin, T.Y., Goyal, P., Girshick, R., He, K., and Dollar, P. 2017, Focal Loss for Dense Object Detection, IEEE International Conference on Computer Vision (ICCV), 2017, 22-29 October, Venice, Italy, pp. 2980-2988.
  12. Liu, W., Anguelov, D., Erhan, D., Szegedy, C., Reed, S., Fu, C.Y., and Berg, A. C. 2016, SSD: Single Shot MultiBox Detector, In: Leibe, B., Matas, J., Sebe, N., Welling, M. (eds.), Computer Vision - ECCV 2016, Springer, Cham. pp. 21-37.
  13. Liu, Z., Du, J., Tian, F., and Wen, J. (2019), MR-CNN: A Multi-Scale Region-Based Convolutional Neural Network for Small Traffic Sign Recognition, IEEE Access, vol. 7, pp. 57120-57128. https://doi.org/10.1109/access.2019.2913882
  14. Mittal, A., Moorthy, A.K., and Bovik, A.C. (2012), No-Reference Image Quality Assessment in the Spatial Domain, IEEE TRANSACTIONS ON IMAGE PROCESSING, Vol. 21, No. 12, pp. 4695-4708. https://doi.org/10.1109/TIP.2012.2214050
  15. Redmon, J., Divvala, S., Girshick, R., and Farhadi, A. (2016), You Only Look Once: Unified, Real-Time Object Detection, 29th IEEE Conference on Computer Vision and Pattern Recognition, 27-30 June, Las Vegas, Nevada, USA, pp. 779-788.
  16. Redmon, J., and Farhadi, A. (2018), YOLOv3: An Incremental Improvement, arXiv, https://arxiv.org/abs/1804.02767 (last date accessed: 17 December 2021).
  17. Ren, S., He, K., Girshick, R., and Sun, J. (2015), Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks, IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 39, No. 6, pp. 1137-1149. https://doi.org/10.1109/TPAMI.2016.2577031
  18. Seong, S.K., Choi, S.K., and Choi, J.W., Cloud Detection of PlanetScope Imagery Based on Deeplab-V3+ by Using Transfer Learning, Journal of Korean Society for Geospatial Information Science, Vol. 28, No. 4, pp. 25-32. (in Korean with English abstract) https://doi.org/10.7319/kogsis.2020.28.4.025
  19. Simonyan, K. and Zisserman, A. (2015), Very Deep Convolutional Networks For Large-Scale Image Recognition, Computer Vision and Pattern Recognition, Cornell University, Ithaca, NY, USA.
  20. National Geographic Information Institute (2015), Data Quality for Aerial Photograph, NGII-STD.2015-7/2020, National Geographic Information Institute, Suwon, Korea.
  21. Song, H.J., Lee, E.B., Jo, H.J., Park, S.Y., Kim, S.Y., Kim, H.J., and Hong, J.W. (2020), Evaluation of Classification and Accuracy in Chest X-ray Images using Deep Learning with Convolution Neural Network, Journal of the Korean Society of Radiology, Vol. 14, No. 1, pp. 39-44. (in Korean with English abstract) https://doi.org/10.7742/JKSR.2019.14.1.39
  22. Sun, L., Mi, X., Wei, J., Wang, J., Tian, X., Yu, H., and Gan, P. (2017), A cloud detection algorithm generating method for remote sensing data at visible to short-wave infrared wavelengths, ISPRS Journal of Photogrammetry and Remote Sensing, Vol. 124, pp. 70-88. https://doi.org/10.1016/j.isprsjprs.2016.12.005
  23. Szegedy, C., Liu, W., Jia, Y., Sermanet, P., Reed, S., Anguelov, D., Erhan, D., Vanhoucke, V., and Rabinovich, A. (2015), Going Deeper with Convolutions, 2015 IEEE Conference on Computer Vision and Pattern Recognition, 7-12 June, Boston, MA, USA, pp. 1-9.
  24. Zhang, H., Li, B., Zhang, J., and Xu, F. (2014), Aerial Image Series Quality Assessment, 35th International Symposium on Remote Sensing of Environment-2013, 22-26 April, Beijing, China, Vol. 17, pp. 012183.
  25. Zhang, G.F., Wang, L., Zheng, Z.N., Chen, Y.F., Zhou, Z.B., and Zhao K. (2016), No-reference aerial image quality assessment based on natural scene statistics and color correlation blur metric, 13th International Conference on Transmission & Distribution Construction, Operation & Live-Line Maintenance, 21 August, Columbus, OH, USA, pp. 1-4.
  26. Zeiler, M.D., and Fergus, R. (2014), Visualizing and understanding convolutional networks, In: Fleet, D., Pajdla, T., Schiele, B., Tuytelaars, T. (eds.), ECCV 2014, Springer, Cham, vol. 8689, pp. 818-833.