Establishment of Priority Update Area for Land Coverage Classification Using Orthoimages and Serial Cadastral Maps

Abstract

This paper introduces a method of selecting priority update areas for subdivided land cover maps by training orthoimages and serial cadastral maps in a deep learning model. For the experiment, orthoimages and serial cadastral maps were obtained from the National Spatial Data Infrastructure Portal. Based on the VGG-16 model, 51,470 images were trained on 33 subdivided classifications within the experimental area and an accuracy evaluation was conducted. The overall accuracy was 61.42%. In addition, using the differences in the classification prediction probability of the misclassified polygon and the cosine similarity that numerically expresses the similarity of the land category features with the original subdivided land cover class, the cases were classified and the areas in which the boundary setting was incorrect and in which the image itself was determined to have a problem were identified as the priority update polygons that should be checked by operators.

1. Introduction

Land cover maps are 2D representations of physical materials on the surface of the earth, valuable as a scientific basis for establishing environmental policies, and are also used as research materials for ecosystem surveys in academics (Park et al., 2007). The Ministry of Environment of Korea established a classified land cover map for the entire country in 2010 and has since performed a continuous revision project. However, as land cover is changing faster than the mapping update cycle due to urban development and climate change, land cover maps must be updated to provide timely and accurate land cover data describing surface changes (Jo et al., 2019).

Land cover mapping and update technology using remote sensing has been studied recently, and numerous experimental studies have been conducted owing to the rapid development of the spectral and spatial resolutions of sensors. In terms of spatial resolution, land cover classification has been performed using high-resolution satellite images, aerial images, and unmanned aerial vehicle (UAV) images, and multiple data have been fused to improve the accuracy of the classification results (Hamid et al., 2021; Moon et al., 2017). In terms of spectral resolution, the spectral reflectance characteristics of land cover have been further subdivided into wavelengths using multispectral and hyperspectral images (Kim et al., 2017; Natesan et al., 2018). In terms of classification techniques, various attempts have been made to compensate for the shortcomings of the existing statistical methods. The fuzzy unordered rule induction algorithm has also been applied to object-based image analysis (Bahareh et al., 2017), including in image segmentation by setting weights such as the normalized difference validation index, shape, compactness, and color in a step-wise manner (Moon et al., 2017). For fusion of UAV images (20 cm resolution) and Landsat 8 images (30 m resolution), research has been conducted using high-pass filters, wavelets, principle component transformations, Brovey, intensity-hue-saturation, and Gram-Schmidt techniques to classify cropped land (Hamid et al., 2021). Despite the numerous aforementioned attempts, owing to the complexity of land cover characteristics, the classification results in cases with numerous classes remain at the experimental level (Won et al., 2020).

Owing to the rapid development of information technology devices, deep learning techniques have been applied, and there have been many achievements in the field of remote sensing. Experiments have been conducted to reclassify data from 19, 271 items of the Global Geo-Referenced Field Photo Library into 11 land cover classes using the Inception-V3 model synthesized using pre-trained convolutional neural networks as feature extractors and multinomial logistic regression as a feature classifier (Guang et al., 2017). Utilizing the joint deep learning technique, Zhang et al. (2019) conducted classification experiments on 12 classes of land cover and 11 classes of land use in Great Britain, and the authors demonstrated the potential for convergence of land cover and land use in classification using remote sensing. Experiments have been conducted on GeoManitoba in Canada, generating Landsat 5/7 multispectral satellite images to create a land-use/land-cover map and categorize it into 18 classes, including cloud, land cover, and land-use classes, using three p retrained deep convolutional neural networks (VGGNet, GoogLeNet, and ResNet) (Alhassan et al., 2020).

Based on classification utilizing 33 of the 41 subdivided classes in South Korea to SegNet, this experiment has an overall accuracy of approximately 30%. This result indicates that the problems of seasonal and temporal changes and increasing error with increasing number of classification items in remote sensing images remain to be addressed fully and that land-use aspects should be considered when defining and deriving land cover classes. These realistic issues make it difficult to automate the production of land cover maps fully, and the subsequent update process is subjectively performed by workers by referring to auxiliary data such as serial cadastral maps and aerial photographs. Serial cadastral maps have only been studied for land cover classification using satellite images and the nonconformities between land cover classification and land use. As no research has been conducted on the establishment of relationships between land use and land cover class, it is necessary to assess the importance of auxiliary data by checking the contribution (Sung et al., 2008).

This paper proposes a method of selecting prioritized workspaces for subdivided land cover map update tasks by training aerial photography and serial cadastral maps using deep learning techniques.

2. Experimental Algorithms and Proposed Methods

Convolutional neural networks (CNNs), which have been used in various fields recently, were first introduced by Lecun et al. (1989) and specifically designed by Behnke (2003).

A fully connected neural network (FCN), which is used for early deep learning, enables the classification problem to be solved through a neural network that is completely connected between nodes by inputting images in series. Consequently, as the image size increases, the number of input nodes increases, and the number of nodes in the hidden layer increases, resulting in enormous numbers of parameters and computations. In addition, the overfitting problem can easily arise because it has a strong dependence on the input image for all nodes. CNNs have been studied as a means of overcoming these computational loads and overfitting problems (Behnke, 2003).

A CNN involves convolution and pooling processes as well as an FCN. Convolutional processing is performed on the original image to generate a feature map, which is down-sampled by pooling. We entered this generated feature map into the FCN to solve the classification problem for the various image features. Owing to the convolution and pooling processes, the overfitting problem experienced by FCNs is solved, and the computation volume associated with learning and prediction is effectively reduced. CNNs are mainly employed to extract features from data that have geometric, texture, and spatial associations between them, and various CNN-based models have been studied using these features (Badrinarayanan et al., 2017; He et al., 2015; Krizhevsky et al., 2012; Lecun et al., 1998; Ronneberger et al., 2015; Simonyan et al., 2015; Szegedy et al., 2015; Zeiler et al., 2014; Zhao et al., 2019).

In this study, we reduced the convolutional layer of VGG-16 and increased the depth of the FCN. Furthermore, we set the error function to binary cross entropy so that the estimation plots for land cover multiple labels could be derived. In particular, when using the automatic classification employed so far, it is complicated to maintain domain-specific boundaries within classification land cover maps, and we excluded theobjectsegmentation approach fromthe scope of this study by utilizing existing land cover maps (Jensen, 2005). We obtained the same orthoimages and serial cadastral maps as the subdivided land cover maps from the National Geographic Information Institute (https://map.ngii.go.kr/) and the National Spatial Data Infrastructure Portal (https://www.nsdi.go.kr/), respectively. For the experiments, we clipped the orthoimages and serial cadastral maps into polygon units based on the class code of the subdivided land cover maps, as shown in Fig. 1, and utilized them as training and test datasets. After classification, we trained the training dataset using the deep learning model and entered the test dataset into the learned model to predict the classification. In learning, the RGB distribution of the orthoimage and the area ratio of each class according to the serial cadastral map clipped by the polygon were included. The result is a probabilistic representation of the class codes for the deep learning model that is best suited to the test data. For the appropriate update proposal, analysis based on the experimental results was performed in several cases. For example, we analyzed the phenomenon in which classes of misclassified data are biased toward a particular class because there are classes that contain image features of different classes. In addition, we determined that the difference between the top one, two and three prediction values was set in thresholds to analyze the data classification.

Fig. 1. Experimental overview.

3. Experimental Data and Preprocessing

1) Experimental areas and materials

The experimental area wasselected where agricultural land, urban areas, mountain areas, and water areas exist evenly because various classification items were availablefor learning. The land cover classes used in this study were the subdivided land cover classes designated by the Ministry of Environment. A total of 41 classes exist, as summarized in Table 1. The legend for the subdivided land cover map was written with reference to Table 1.

Table 1. Subdivided land cover classes used by the Ministry of Environment

The area selected for the experiment is part of Jeonju, Jeollabuk-do and is shown in Fig. 2. Some of the class items that accounted for more than 5% of the selected areasincludedRoad (20.31%), Readjusted Paddy Field (7.19%), Not Readjusted Field (13.08%), Deciduous Forest (6.95%), Other Grassland (14.62%), and Other Bare Site (5.76%). These findings demonstrate that the experimental area has more agricultural land cover than urban areas. Among the 41 subclass items, 33 items were found in the experimental area, excludingAirport, Port, Railway, Natural Grassland, Tidal Flat, Salt Field, Beach, Mining Site, and Ocean. The scale of the orthographic image in the experimental area is 1:5000, the experimental area ranges from latitude 35°52’30″ to 35°48’00″ and longitude 127°00’00″ to 127°09’00″.

Fig. 2. Experimental area in Jeonju, Jeollabuk-do.

Fig. 3 provides subdivided land cover maps, serial cadastral maps, and orthoimages of the experimental area.

Fig. 3. Experimental region data: (a) subdivided land cover map, (b) examples of subdivided legend data for some regions, (c) serial cadastral map, and (d) orthoimage.

2) Data preprocessing

Registration was performed based on the subdivided land cover map. Furthermore, because the ellipsoid of the subdivided land cover map provided by the Ministry of Environment is the GRS80 ellipsoid and that of the serial cadastral map provided by the National Spatial Data Infrastructure Portal is the Bessel ellipsoid, we performed coordinate conversion for correction between the ellipsoids based on the subdivided land cover map. Fig. 4 shows the coordinate conversion performed to match the subdivided land cover map with the serial cadastral map.

Fig. 4. Shapes (a) before and (b) after coordinate transformation of the subdivided land cover and serial cadastral map in the same region.

The experimental area included a total of 26 land categories, excluding Mineral Spring Site and Saltern. The class of serial cadastral maps was unified by first modifying the attribute table to clip serial cadastral maps to the boundaries of the subdivided land cover classes. Because each serial cadastral map file was in .shp format, rasterization was conducting using the polygon-to-raster tool. To unify the values of the rasterized data and assign different values for each land category, the task was performed using the reclassify feature. The partial land category of each file was integrated into a single .tiffile using the mosaic feature as a new raster tool. Subsequently, the grouping process was performed by compiling sets by codes ofsubdivided land cover maps and selecting them by class code to proceed with clipping by polygon. The process of clipping per polygon was conducted using the model builder of ArcGIS, as shown in Fig. 5.

Fig. 5. Model builder flowchart used to perform clipping on ArcGIS.

3) Training dataset configuration

In total, 51, 070 datasetsthat contain orthoimage and land category were created for all orthoimage and serial cadastral map by using the boundary of polygon of subdivided cover map, and random sampling was performed in a 9:1 ratio to classify them into training dataset that was composed of 45, 963 datasets and test dataset that was composed of 5, 107 datasets. The reason for using a large proportion of training data was to learn as many cases as possible by considering the features of subdivided land cover maps with varying polygons. Fig. 6 presents the orthoimages and serial cadastral maps clipped by polygons ofsubdivided land cover maps.

Fig. 6. Examples of the orthoimages and serial cadastral maps clipped in polygon units of the subdivided land cover map for the (a) Single-Family Facility, (b) Readjusted Paddy Field, (c) Culture/Sport/Recreation Facility, and (d) Other Grassland categories.

Table 2 lists the proportions of the land in the experimental area corresponding to the majorsubdivided land cover classes (those accounting for at least 5% of the experimental area).

Table 2. Proportions of land in major subdivided land cover classes

Among the land categories accounting for more than 5% in the entire region, the Miscellaneous Land, Site, Forestry, and Paddy Field classes account for 39.57%, 14.11%, 13.87%, and 12.31%ofthe land, respectively. Some of the land categories constituting lessthan 0.1% of the total are the Religion Site, Water Supply Site, Fish Farm, Recreation Area, Gas Station Site, and Parking Lot classes, which comprise 0.05%, 0.09%, 0.04%, 0.05%, 0.03%, and 0.08%, respectively.

4. Results and Analysis

1) Experimental results

The overall accuracy was measured on a 10-step basis, with up to 500 epochs in total. The overall accuracy gradually increased as the number of lessons increased, with the highest overall accuracy of 61.43% in epoch 400; subsequently, the overall accuracy did not increase even with an increase in the number of lessons. The experimental results were analyzed using the cosine similarity formula to evaluate the similarity between the distributions of the land categories within the subdivided classes and those within specific polygons within the subdivided classes. The cosine similarity isthe cosine of the angle between two vectors in the inner space if the compared targets are the same size, indicating the similarity between the two vectors (Korenius et al., 2007). The following is the formula for the cosine similarity:

\(\text { cos similarity }=\frac{\sum_{i=1}^{n} A_{i} \times B_{i}}{\sqrt{\sum_{i=1}^{n}\left(A_{i}\right)^{2}} \times \sqrt{\sum_{i}^{n}\left(B_{i}\right)^{2}}}\)       (1)

A= Ratio of each cadastral in a subdivided class

B= Ratio of each cadastral in a specific pologon

The top one, two, and three predictions are derived when a test orthoimage polygon is entered into the trained model, indicating how the polygon fits the subdivided class. Thus, the higher the value, the more likely it is to be a corresponding class item.

2) Analysis

Table 3 summarizesthe producer and user accuracies by subdivided class. Among the subdivided classes, those with lessthan 50clippings are the Railway, Other Transport/Communication Facility, Environmental Infrastructure, Golf Course, Riparian, and Rock classes. These items were not analyzed in this study.

Fig. 7 depicts the land category distribution ratios for some of the subdivided land cover classes used in the experiment.

Table 3. Producer and user accuracies by subdivided class

Fig. 7. Land category distribution ratios of some subdivided land cover classes: (a) Cemetery, Other Grassland, and Playground and (b) Road, Other Cultivation Site, and Lake.

Figs. 7(a) and (b) respectively depict the land category rates of the subdivided classes with user and producer accuracies above 70% and with either user or producer accuracies below 30%. For the Cemetery class in Fig. 7(a), 17 of the 26 land categories were included, with Forestry accounting for 55% and Miscellaneous Land accounting for 32%. In the case of Other Grassland, all 26 points are included, but Miscellaneous Land accounts for 59%. Meanwhile, the Playground class contains only 12 points, and among them, the School Site category accounts for 77%. In the case ofthe Road class in Fig. 7(b), Miscellaneous Land has the largest proportion at only 38%, including 25 points. In the Other Cultivation Site case, the Dry Paddy Field, Forestry, Miscellaneous Land, and Paddy Field categories account for 17%, 24%, 20%, and 26%, respectively, showing a relatively diverse range ofland categories. The Lake class contains only 18 points, where the Marsh and Miscellaneous Land categories account for 42% and 39%, respectively, indicating that no one point prevails. The subdivided classes that contain relatively diverse land categories or that do not have one dominant land category tend to have low accuracies compared to the organized subdivided classes.

Aside from the classification accuracy of the subdivided classes, there are polygons whose top predictions differed fromthe general trends depending on the shape and land category distribution of the test polygon. Fig. 8 provides an example of data misclassification among polygons belonging to the NotReadjustedPaddyFieldclass.

Fig. 8. Polygon misclassification in the Readjusted Paddy Field category: (a) orthoimage of polygon, (b) serial cadastral map of polygon, and (c) polygon analysis information.

In the Not Readjusted Paddy Field class, the most common land category is Paddy Field and the second is the Miscellaneous Site category. In this class, Paddy Field accounts for 59.68% of the total land area, whereas Miscellaneous Site accounts for 22.44%. On the other hand, in the case of the misclassified Not Readjusted Paddy Field polygons in Fig. 8, Dry Paddy Field (value 17 in Fig. 8(b)) accounts for the majority. Numerically, Dry Paddy Field (value 17 in Fig. 8(b)) occupies the largest portion at 79.84%, followed by Paddy Field (value 4 in Fig 8. (b)) at 15.97%. The top one, two, and three predictions of the polygon are Not Readjusted Field (87.90%), Other Bare Site (11.96%), and NotReadjusted Paddy Field (0.62%). Among these misclassified data, we identified polygons that differed from the land category ratios of the subdivided classes to which the polygons belonged. In this case, the cosine similarity is relatively low compared to the subdivided class to which the polygon belongs, with a value of 0.34 for the polygon in Fig. 8. In the system established in this experiment, we determined that there was a problem with boundary selection of the polygon in such cases and proposed boundary modification. Ofthe total 1, 970 misclassified data, 952 data accounted for less than 50% of the most land category rates of the subdivided classes to which the polygons belonged, representing 48.32%ofthe total. On the other hand, there were 1, 018 data that contributed more than 80% of the most land category rates of the subdivided land cover classes to which the polygons belong. However, in this experiment, only five of the 33 subdivided classes accounted for more than 70% of the most land categories rate of the classes. Therefore, in this experiment, it will be necessary to place greater significance on classification accuracy analysis based on the land category distribution within the polygon rather than classification accuracy analysis based on the land category distribution throughout the subdivided classes.

Although polygons are correctly formed and even organized, there have been cases in which they are misclassified due to temporal variable. Fig. 9 presents orthoimages and serial cadastral maps of two test polygons in the Lake category.

Fig. 9. (a) Orthoimage of Lake A, (b) serial cadastral map of Lake A, (c) orthoimage of Lake B, (d) serial cadastral map of Lake B, and (e) Lake B polygon analysis information.

In Fig. 9(a), water can be identified in the orthoimage, and the identified land category data confirm that Marsh (value 13) accounts for the majority. A value of 0.72 was derived for the cosine similarity with the Lake classto which the polygon belongs. Thus, there is high similarity in the land category distribution between the polygons and classes. Based on the classification prediction of Lake A, the top one prediction was classified as Lake 100%. In the case of Lake B, the land category distribution is the same as that of Lake A, with Marsh (value 13) accounting for the majority. The cosine similarity of Lake B is also 0.72, but the classification prediction of Lake B shows that the top one prediction is the Not Readjusted Field (65.14%), while the top two prediction is the Orchard (34.54%). Although these results are reasonable from the perspective of land category distribution, they appear to have been influenced by the fact that Fig. 9(c) is no RGB distribution corresponding to water within the domain due to timing variables, unlike polygons in the general Lake class. In the system established in this experiment, such cases are determined to have problems with the orthoimage itself rather than problems with the polygon boundary. To determine whether the cosine similarity can be used when classifying misclassified data into a boundary problem or an image data problem, Table 4 presents the cosine similarity values compared with the land category distributions of misclassified data and subdivided classes to which the data belong.

Table 4. Cosine similarity distribution of misclassified data in experimental areas

The proportions of cosine similarity values of at least 0 and less than 0.4 were determined to have low correlations with the land category distribution of subdivided classes containing misclassified polygons and cosine similarity values of at least 0.7 and less than 1 were determined to have high correlations constituting86.95% of the total. It can be seen that almost all misclassified data were included. Therefore, whether the problem of misclassified data is due to the gap in the overall characteristics of the subdivided class due to an error in setting the polygon boundary of the data, or whether the boundary is set normally, there is a problem in the image itself by calculating the cosine similarity. First, the operator checks the polygon classification results and the polygons that do not have large differences between the top one, two, and three predictions. In addition, when checking the polygons, the cosine similarity is also checked. If the cosine similarity is less than 0.4, the differences between the top one, two, and three predictions of misclassified polygons are considered not to be large because the misclassified polygon was confused with other subdivided classes with similar distributions of land categories and RGB characteristics. Therefore, we propose a boundary modification to ensure similarity in the distribution of land categories with that of the subdivided class containing misclassified polygons. Next, we checked the polygons where the top one prediction had a relatively high value compared to the top two and three predictions. When the polygon was checked, if the cosine similarity was higher than 0.7, the misclassified polygon was similar to the land category characteristic of the subdivided class to which the polygon belonged, but it was determined that the polygon had been misclassified due to the large differences in the RGB characteristics, indicating that the image should be reconsidered. Such a case classification is expected to facilitate early classification of the common causes of misclassification data.

3) Priority update region

Based on the above analysis, we established a system that identifies misclassified polygons, offers information and proposes update methods for each polygon. Fig. 10 presents a screen that appears the misclassified data are selected. If a polygon is selected, it is magnified to show the orthoimage, serial cadastral map, graph of the percentage of the land category distributions of the subdivided class to which the misclassified polygon belongs and percentage of the total land category distributions of the selected polygons, top one and two predictions according to deep learning, and cosine similarity.

Fig. 10. Example of data displayed after selecting a specific polygon.

We propose systematic boundary modification, considering that the orthoimage itselfis not problematic if the cosine similarity is less than 0.4, and suggest a revision of the orthoimage, considering that the land category distribution is not abnormal rather misclassified if the similarity is higher than 0.7.

5. Conclusion

We conducted classification of subdivided land cover classes using RGB data from orthoimages and the land categories of serial cadastral maps. Then, we performed an experiment to develop a case-by-case update strategy through analysis based on the land category distributions of the misclassified regions. In deep learning training, we used RGB data from orthoimages and the land category of serial cadastral maps as training data. Utilizing this approach, the system provides orthoimages of the misclassified data, land category distributions of subdivided classes, land category distributions of polygons, polygon classification prediction result, and similarity between the land category distribution of subdivided class and that of the polygon. Using these analysis data, it is possible to determine the problem in the misclassified polygon by considering the differences between the top one, two, and three predictions of the misclassified polygon and the cosine similarity. In addition, it is expected that this approach will facilitate the automation of error polygons update of quality inspection based on objective and numerical criteria pertaining to the direction of correction that will be useful for the operator. However, in the system proposed in this paper, the operator’s subjectivity enters into the threshold setting of the cosine similarity value and the difference between the top one, two and three classification prediction probabilities that determine the update direction of misclassified polygons. Therefore, the proposed system has a limitation that it is not a fully automated system. As a future research, it will be possible to automatically set the threshold that can be standard of update direction according to the characteristics of the test dataset and the classification results. Also, future research will enable the analysis of the land category distributions of misclassified data and assess whether these affect misclassification according to the combination of the land category within the polygonal domain.