Browse > Article
http://dx.doi.org/10.7780/kjrs.2021.37.5.1.4

A Study of Establishment and application Algorithm of Artificial Intelligence Training Data on Land use/cover Using Aerial Photograph and Satellite Images  

Lee, Seong-hyeok (Division for Environmental Planning, Korea Environment Institute)
Lee, Moung-jin (Center for Environmental Data Strategy, Korea Environment Institute)
Publication Information
Korean Journal of Remote Sensing / v.37, no.5_1, 2021 , pp. 871-884 More about this Journal
Abstract
The purpose of this study was to determine ways to increase efficiency in constructing and verifying artificial intelligence learning data on land cover using aerial and satellite images, and in applying the data to AI learning algorithms. To this end, multi-resolution datasets of 0.51 m and 10 m each for 8 categories of land cover were constructed using high-resolution aerial images and satellite images obtained from Sentinel-2 satellites. Furthermore, fine data (a total of 17,000 pieces) and coarse data (a total of 33,000 pieces) were simultaneously constructed to achieve the following two goals: precise detection of land cover changes and the establishment of large-scale learning datasets. To secure the accuracy of the learning data, the verification was performed in three steps, which included data refining, annotation, and sampling. The learning data that wasfinally verified was applied to the semantic segmentation algorithms U-Net and DeeplabV3+, and the results were analyzed. Based on the analysis, the average accuracy for land cover based on aerial imagery was 77.8% for U-Net and 76.3% for Deeplab V3+, while for land cover based on satellite imagery it was 91.4% for U-Net and 85.8% for Deeplab V3+. The artificial intelligence learning datasets on land cover constructed using high-resolution aerial and satellite images in this study can be used as reference data to help classify land cover and identify relevant changes. Therefore, it is expected that this study's findings can be used in the future in various fields of artificial intelligence studying land cover in constructing an artificial intelligence learning dataset on land cover of the whole of Korea.
Keywords
Land cover; Dataset for AI; Fine annotation; Coarse annotation; Machine learning;
Citations & Related Records
연도 인용수 순위
  • Reference
1 Brostow, G.J., J. Fauqueur, and R. Cipolla, 2009. Semantic object classes in video: A high-definition ground truth database, Pattern Recognition Letters, 30(2): 88-97.   DOI
2 Long, J., E. Shelhamer, and T. Darrell, 2015. Fully convolutional networks for semantic segmentation, Proc. of the IEEE conference on computer vision and pattern recognition, Boston, MA, Jun. 7-12. pp. 3431-3440.
3 Luo, Y., Z. Wang, Z. Huang, Y. Yang, and C. Zhao, 2018. Coarse-to-fine annotation enrichment for semantic segmentation learning, Proc. of the 27th ACM International Conference on Information and Knowledge Management, Torino, IT, Oct. 22-26. pp. 237-246.
4 Lyu, H., H. Lu, and L. Mou, 2016. Learning a transferable change rule from a recurrent neural network for land cover change detection, Remote Sensing, 8(6): 506.   DOI
5 Guidici, D. and M.L. Clark, 2017. One-Dimensional convolutional neural network land-cover classification of multi-seasonal hyperspectral imagery in the San Francisco Bay Area, California, Remote Sensing, 9(6): 629.   DOI
6 Gangwon province home page, https://provin.gangwon.kr, Accessed on Jun. 18, 2021.
7 Gyeonggi province home page, https://www.gg.go.kr/, Accessed on Aug. 10, 2021.
8 Krizhevsky, A., I. Sutskever, and G.E. Hinton, 2012. Imagenet classification with deep convolutional neural networks, Advances in Neural Information Processing Systems, 25: 1097-1105.
9 Kussul, N., M. Lavreniuk, S. Skakun, and A. Shelestov, 2017. Deep learning classification of land cover and crop types using remote sensing data, IEEE Geoscience and Remote Sensing Letters, 14(5): 778-782.   DOI
10 Kussul, N.N., N.S. Lavreniuk, A.Y. Shelestov, B.Y. Yailymov, and I.N. Butko, 2016. Land cover changes analysis based on deep machine learning technique, Journal of Automation and Information Sciences, 48(5): 42-54.   DOI
11 Kwan, C., B. Ayhan, B. Budavari, Y. Lu, D. Perez, J. Li, S. Bernabe, and A. Plaza, 2020. Deep learning for Land Cover Classification using only a few bands, Remote Sensing, 12(12): 1-17.
12 Kaggle home page, https://Kaggle.com/datasets, Accessed on Aug. 5, 2021.
13 Korea Environmental Geospatial Information Service home page, https://egis.me.go.kr/main.do, Accessed on Jul. 12, 2021.
14 ETRI AI API, Data home page, https://aiopen.etri.re.kr/, Accessed on Aug. 5, 2021.
15 Mohamadou, Y., A. Halidou, and P.T. Kapen, 2020. A review of mathematical modeling, artificial intelligence and datasets used in the study, prediction and management of COVID-19, Applied Intelligence, 50(11): 3913-3925.   DOI
16 Cordts, M., M. Omran, S. Ramos, T. Rehfeld, M. Enzweiler, R. Benenson, U. Franke, S. Roth, and B. Schiele, 2016. The cityscapes dataset for semantic urban scene understanding, Proc. of the IEEE conference on computer vision and pattern recognition, Las Vegas, NV, Jun. 27-30. pp. 3213-3223.
17 Common Objects in Context home page, https://cocodataset.org/, Accessed on Aug. 4, 2021.
18 Everingham, M., S.A. Eslami, L. Van Gool, C.K. Williams, J. Winn, and A. Zisserman, 2015. The pascal visual object classes challenge: A retrospective, International Journal of Computer Vision, 111(1): 98-136.   DOI
19 Everingham, M., L. Van Gool, C.K. Williams, J. Winn, and A. Zisserman, 2010. The pascal visual object classes (voc) challenge, International Journal of Computer Vision, 88(2): 303-338.   DOI
20 Najafabadi, M.M., F. Villanustre, T.M. Khoshgoftaar, N. Seliya, R. Wald, and E. Muharemagic, 2015. Deep learning applications and challenges in big data analytics, Journal of big data, 2(1): 1-21.   DOI
21 Korea Law Information Center, https://www.law.go.kr, Accessed on Jul. 23, 2021.
22 Lin, T., M. Maire, S. Belongie, J. Hays, P. Perona, D. Ramanan, P. Dollar, and C.L. Zitnick, 2014. Microsoft coco: Common objects in context, European conference on computer vision, Zurich, Switzerland, CH, Sep. 6-12. pp. 740-755.
23 Lee, S., K. Han, K. Lee, K. Lee, K. Oh, and M. Lee, 2020. Classification of Landscape Affected by Deforestation Using High-Resolution Remote Sensing Data and Deep-Learning Techniques, Remote Sensing, 12(20): 3372.   DOI
24 Lee, S. and J. Kim, 2019. Land cover classification using sematic image segmentation with deep learning, Korean Journal of Remote Sensing, 35(2): 279-288 (in Korean with English abstract).   DOI
25 Lee, S. and M. Lee, 2020. A Study on Deep Learning Optimization by Land Cover Classification Item Using Satellite Imagery, Korean Journal of Remote Sensing, 36(6-2): 1591-1604 (in Korean with English abstract).   DOI
26 Zhang, C., I. Sargent, X. Pan, H. Li, A. Gardiner, J. Hare, and P.M. Atkinson, 2019. Joint Deep Learning for land cover and land use classification, Remote Sensing of Environment, 221: 173-187.   DOI
27 Chen, L., Y. Zhu, G. Papandreou, F. Schroff, and H. Adam, 2018. Encoder-decoder with atrous separable convolution for semantic image segmentation, Proc. of the European conference on computer vision (ECCV), Munich, DE, Sep. 8-14. pp. 801-818.
28 Helber, P., B. Bischke, A. Dengel, and D. Borth, 2019. Eurosat: A novel dataset and deep learning benchmark for land use and land cover classification, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 12(7): 2217-2226.   DOI
29 Wang, M., H. Zhang, W. Sun, S. Li, F. Wang, and G. Yang, 2020. A Coarse-to-Fine deep learning based land use change detection method for high-resolution remote sensing images, Remote Sensing, 12(12): 1933.   DOI
30 Zhang, C., P.A. Harrison, X. Pan, H. Li, I. Sargent, and P.M. Atkinson, 2020. Scale Sequence Joint Deep Learning (SS-JDL) for land use and land cover classification, Remote Sensing of Environment, 237: 111593.   DOI
31 Zhang, W., K. Song, X. Rong and Y. Li, 2018. Coarse-to-fine uav target tracking with deep reinforcement learning, IEEE Transactions on Automation Science and Engineering, 16(4): 1522-1530.   DOI
32 Nijhawan, R., D. Joshi, N. Narang, A. Mittal, and A. Mittal, 2019. A futuristic deep learning framework approach for land use-land cover classification using remote sensing imagery, Advanced Computing and Communication Technologies, 2019: 87-96.
33 Noh, H., S. Hong, and B. Han, 2015. Learning deconvolution network for semantic segmentation, Proc. of the IEEE international conference on computer vision, Santiago, CL, Dec. 7-13. pp. 1520-1528.
34 Shrestha, A. and A. Mahmood, 2019. Review of deep learning algorithms and architectures, IEEE Access, 7: 53040-53065.   DOI
35 Aviles-Cruz, C., A. Ferreyra-Ramirez, A. Zuniga-Lopez, and J. Villegas-Cortez, 2019. Coarse-fine convolutional deep-learning strategy for human activity recognition, Sensors, 19(7): 1556.   DOI
36 Azimi, S.M., C. Henry, L. Sommer, A. Schumann, and E. Vig, 2019. Skyscapes fine-grained semantic understanding of aerial scenes, Proc. of the IEEE/CVF International Conference on Computer Vision, Seoul, KR, Oct. 27-Nov. 2, pp. 7393-7403.
37 Rakhlin, A., A. Davydow, and S. Nikolenko, 2018. Land cover classification from satellite imagery with u-net and lovasz-softmax loss, Proc. of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, Salt Lake City, UT, Jun. 18-22. pp. 262-266.
38 Paul, S. and L. Singh, 2015. A review on advances in deep learning, 2015 IEEE Workshop on Computational Intelligence: Theories, Applications and Future Directions (WCI), Kochi, IN, Aug. 10-13. pp. 1-6.
39 Ronneberger, O., P. Fischer, and T. Brox, 2015. U-net: Convolutional networks for biomedical image segmentation, Proc. of International Conference on Medical image computing and computer-assisted intervention, Munich, GER, Oct. 5-9. pp. 234-241.
40 Russwurm, M. and M. Korner, 2017. Multi-temporal Land Cover Classification with Long Short-Term Memory Neural Networks, International Archives of the Photogrammetry, Remote Sensing & Spatial Information Sciences, 42: 551-558.   DOI
41 The province of chungchengbuk-do home page, https://chungbuk.go.kr, Accessed on Jul. 12, 2021.
42 The province of chungchengnam-do home page, https://chungnam.go.kr, Accessed on Jul. 23, 2021.
43 Vargas, R., A. Mosavi and R. Ruiz, 2017. Deep learning: a review. Queensland University of Technology, Brisbane, Queensland, AU.
44 Wang, M., H. Zhang, W. Sun, S. Li, F. Wang, and G. Yang, 2020. A Coarse-to-Fine deep learning based land use change detection method for high-resolution remote sensing images, Remote Sensing, 12(12): 1933.   DOI
45 Xia, G., X. Bai, J. Ding, Z. Zhu, S. Belongie, J. Luo, M. Datcu, M. Pelillo, and L. Zhang, 2018. DOTA: A large-scale dataset for object detection in aerial images, Proc. of the IEEE conference on computer vision and pattern recognition, Salt Lake City, UT, Jun. 18-23. pp. 3974-3983.
46 Bondi, E., R. Jain, P. Aggrawal, S. Anand, R. Hannaford, A. Kapoor, J. Piavis, S. Shah, L. Joppa, and B. Dilkina, 2020. BIRDSAI: A dataset for detection and tracking in aerial thermal infrared videos, Proc. of the IEEE/CVF Winter Conference on Applications of Computer Vision, Snowmass Village, Snowmass, CO, USA, Mar. 1-5, pp. 1747-1756.
47 Azimi, S.M., M. Kraus, R. Bahmanyar, and P. Reinartz, 2021. Multiple Pedestrians and Vehicles Tracking in Aerial Imagery Using a Convolutional Neural Network, Remote Sensing, 13(10): 1953.   DOI
48 AI hub home page, https://aihub.or.kr/, Accessed on Aug. 10, 2021.
49 Boguszewski, A., D. Batorski, N. Ziemba-Jankowska, A. Zambrzycka, and T. Dziedzic, 2020. LandCover. ai: Dataset for Automatic Mapping of Buildings, Woodlands and Water from Aerial Imagery, arXiv preprint, arXiv: 2005.02264.