Browse > Article
http://dx.doi.org/10.7780/kjrs.2018.34.5.9

Performance Evaluation of Machine Learning and Deep Learning Algorithms in Crop Classification: Impact of Hyper-parameters and Training Sample Size  

Kim, Yeseul (Department of Geoinformatic Engineering, Inha University)
Kwak, Geun-Ho (Department of Geoinformatic Engineering, Inha University)
Lee, Kyung-Do (National Institute of Agriculture Sciences, Rural Development Administration)
Na, Sang-Il (National Institute of Agriculture Sciences, Rural Development Administration)
Park, Chan-Won (National Institute of Agriculture Sciences, Rural Development Administration)
Park, No-Wook (Department of Geoinformatic Engineering, Inha University)
Publication Information
Korean Journal of Remote Sensing / v.34, no.5, 2018 , pp. 811-827 More about this Journal
Abstract
The purpose of this study is to compare machine learning algorithm and deep learning algorithm in crop classification using multi-temporal remote sensing data. For this, impacts of machine learning and deep learning algorithms on (a) hyper-parameter and (2) training sample size were compared and analyzed for Haenam-gun, Korea and Illinois State, USA. In the comparison experiment, support vector machine (SVM) was applied as machine learning algorithm and convolutional neural network (CNN) was applied as deep learning algorithm. In particular, 2D-CNN considering 2-dimensional spatial information and 3D-CNN with extended time dimension from 2D-CNN were applied as CNN. As a result of the experiment, it was found that the hyper-parameter values of CNN, considering various hyper-parameter, defined in the two study areas were similar compared with SVM. Based on this result, although it takes much time to optimize the model in CNN, it is considered that it is possible to apply transfer learning that can extend optimized CNN model to other regions. Then, in the experiment results with various training sample size, the impact of that on CNN was larger than SVM. In particular, this impact was exaggerated in Illinois State with heterogeneous spatial patterns. In addition, the lowest classification performance of 3D-CNN was presented in Illinois State, which is considered to be due to over-fitting as complexity of the model. That is, the classification performance was relatively degraded due to heterogeneous patterns and noise effect of input data, although the training accuracy of 3D-CNN model was high. This result simply that a proper classification algorithms should be selected considering spatial characteristics of study areas. Also, a large amount of training samples is necessary to guarantee higher classification performance in CNN, particularly in 3D-CNN.
Keywords
Crop classification; Machine learning; Deep learning; Support vector machine; Convolutional neural network;
Citations & Related Records
Times Cited By KSCI : 4  (Citation Analysis)
연도 인용수 순위
1 Zhong, Z., J. Li, Z. Luo, and M. Chapman, 2018. Spectral-spatial residual network for hyperspectral image classification: a 3-D deep learning framework, IEEE Transactions on Geoscience and Remote Sensing, 56(2): 847-858.   DOI
2 Cai, Y., K. Guan, J. Peng, S. Wang, C. Seifert, B. Wardlow, and Z. Li, 2018. A high-performance and in-season classification system of field-level crop types using time-series Landsat data and a machine learning approach, Remote Sensing of Environment, 210: 35-47.   DOI
3 Castro, J.B., R.Q. Feitosa, L.C.L. Rosa, P.A. Diaz, and I. Sanches, 2017. A comparative analysis of deep learning techniques for sub-tropical crop types recognition from multitemporal optical/SAR image sequences, Proc. of 2017 30th SIBGRAPI Conference on Graphics, Patterns and Images, Niteroi, Oct. 17-20, pp. 382-389.
4 Dang, L.M., S.I. Hassan, I. Suhyeon, A.K. Sangaiah, I. Mehmood, S. Rho, S. Seo, and H. Moon, 2018. UAV based wilt detection system via convolutional neural networks, Sustainable Computing: Informatics and Systems, https://doi.org/10.1016/j.suscom.2018.05.010.   DOI
5 Deng, L., Z. Mao, X. Li, Z. Hu, F. Duan, and Y. Yan, 2018. UAV-based multispectral remote sensing for precision agriculture: a comparison between different cameras, ISPRS Journal of Photogrammetry and Remote Sensing, 146: 124-136.   DOI
6 Gao, Q., S. Lim, and X. Jia, 2018. Hyperspectral image classification using convolutional neural networks and multiple feature learning, Remote Sensing, 10(2): 299.   DOI
7 Garcia-Garcia, A., J. Garcia-Rodriguez, S. Orts-Escolano, S. Oprea, F. Gomez-Donoso, and M. Cazorla, 2017. A study of the effect of noise and occlusion on the accuracy of convolutional neural networks applied to 3D object recognition, Computer Vision and image Understanding, 164: 124-134.   DOI
8 Ishida, T., J. Kurihara, F.A. Viray, S.B. Namuco, E.C. Paringit, G.J. Perez, Y. Takahashi, and J.J. Marciano Jr., 2018. A novel approach for vegetation classification using UAV-based hyperspectral imaging, Computers and Electronics in Agriculture, 144: 80-85.   DOI
9 Hall, O., S. Dahlin, H. Marstorp, M.F.A. Bustos, I. Oborn, and M. Jirstrom, 2018. Classification of maize in complex smallholder farming systems using UAV imagery, Remote Sensing, 2(3): 22.
10 Hsu, C.-W., C.-C. Chang, and C.-J. Lin, 2003. A practical guide to support vector classification, Department of Computer Science, National Taiwan University, Taipei City, Taiwan.
11 Karpathy, A., G. Toderici, S. Shetty, T. Leung, R. Sukthankar, and L. Fei-Fei, 2014. Large-scale video classification with convolutional neural networks, Proc. of 2014 IEEE conference on Computer vision and Pattern Recognition, Columbus, OH, Jun. 23-28, pp. 1725-1732.
12 Ji, S., C. Zhang, A. Xu, Y. Shi, and Y. Duan, 2018. 3D convolutional neural networks for crop classification with multi-temporal remote sensing images, Remote Sensing, 10(1): 75.   DOI
13 Ji, S., W. Xu, M. Yang, and K. Yu, 2013. 3D convolutional neural networks for human action recognition, IEEE Transactions on Pattern Analysis and Machine Intelligence, 35(1): 221-231.   DOI
14 Kamilaris, A. and F.X. Prenafeta-Boldu, 2018. Deep learning in agriculture: a survey, Computers and Electronics in Agriculture, 147: 70-90.   DOI
15 Kim, Y., N.-W. Park, and K.-D. Lee, 2017. Self-learning based land-cover classification using sequential class patterns from past land-cover maps, Remote Sensing, 9(9): 921.   DOI
16 Kussul, N., M. Lavreniuk, S. Skakun, and A. Shelestov, 2017. Deep learning classification of land cover and crop types using remote sensing data, IEEE Geoscience and Remote Sensing Letters, 14(5): 778-782.   DOI
17 Liu, T., A. Abd-Elrahman, J. Morton, and V.L. Wilhelm, 2018. Comparing fully convolutional networks, random forest, support vector machine, and patch-based deep convolutional neural networks for object-based wetland mapping using images from small unmanned aircraft system, GIScience & Remote Sensing, 55(2): 243-264.   DOI
18 Kwak, G.-H., S. Park, H.Y. Yoo, and N.-W. Park, 2017. Updating land cover maps using object segmentation and past land cover information, Korean Journal of Remote Sensing, 33(6-2): 1089-1100 (in Korean with English abstract).   DOI
19 LeCun, Y., Y. Bengio, and G. Hinton, 2015. Deep learning, Nature, 521(7553): 436.   DOI
20 Lee, K.-D., Y.E. Lee, C.-W. Park, and S.-I. Na, 2016. A comparative study of image classification method to classify onion and garlic using unmanned aerial vehicle (UAV) imagery, Korean Society of Soil Science and Fertilizer, 49(6): 743-750 (in Korean with English abstract).   DOI
21 Maulik, U. and D. Chakraborty, 2017. Remote sensing image classification: a survey of support-vector-machine-based advanced techniques, IEEE Geoscience and Remote Sensing, 5(1): 33-52.   DOI
22 Maxwell, A.E., T.A. Warner, and F. Fang, 2018. Implementation of machine-learning classification in remote sensing: an applied review, International Journal of Remote Sensing, 39(9): 2784-2817.   DOI
23 Melgani, F. and L. Bruzzone, 2004. Classification of hyperspectral remote sensing images with support vector machines, IEEE Transactions on Geoscience and Remote Sensing, 42(8): 1778-1790.   DOI
24 Montero, P. and J.A. Vilar, 2014. TSclust: an R package for time series clustering, Journal of Statistical Software, 62(1): 1-43.
25 Sameen, M.I., B. Pradhan, and O.S. Aziz, 2018. Classification of very high resolution aerial photos using spectral-spatial convolutional neural networks, Journal of Sensors, https://doi.org/10.1155/2018/7195432.   DOI
26 Onojeghuo, A.O., G.A. Blackburn, Q. Wang, P.M. Atkinson, D. Kindred, and Y. Miao, 2018. Mapping paddy rice fields by applying machine learning algorithms to multi-temporal Sentinel-1A and Landsat data, International Journal of Remote Sensing, 39(4): 1042-1067.   DOI
27 Park, J.K. and J.H. Park, 2015. Crops classification using imagery of unmanned aerial vehicle (UAV), Journal of the Korean Society of Agricultural Engineers, 57(6): 91-97 (in Korean with English abstract).   DOI
28 Prosekov, A.Y. and S.A. Ivanova, 2018. Food security: the challenge of the present, Geoforum, 91: 73-77.   DOI
29 Sinha, T., B. Verma, and A. Haidar, 2017. Optimization of convolutional neural network parameters for image classification, Proc. of 2017 IEEE Symposium Series on Computational Intelligence, Honolulu, HI, Nov. 27-Dec. 1, pp. 1-7.
30 Sharma, A., X. Liu, X. Yang, and D. Shi, 2017. A patch-based convolutional neural network for remote sensing image classification, Neural Networks, 95: 19-28.   DOI
31 Skakun, S., B. Franch, E. Vermote, J.-C. Roger, I. Becker-Reshef, C. Justice, and N. Kussul, 2017. Early season large-area winter crop mapping using MODIS NDVI data, growing degree days information and a Gaussian mixture model, Remote Sensing of Environment, 195: 244-258.   DOI
32 Song, A. and Y. Kim, 2017. Deep learning-based hyperspectral image classification with application to environmental geographic information systems, Korean Journal of Remote Sensing, 33(6-2): 1061-1073 (in Korean with English abstract).   DOI
33 Zhang, X., N. Chen, J. Li, Z. Chen, and D. Niyogi, 2017. Multi-sensor integrated framework and index for agricultural drought monitoring, Remote Sensing of Environment, 188: 141-163.   DOI
34 Torbick, N., X. Huang, B. Ziniti, D. Johnson, J. Masek, and M. Reba, 2018. Fusion of moderate resolution earth observations for operational crop type mapping, Remote Sensing, 10(7): 1058.   DOI
35 Wu, H. and S. Prasad, 2017. Convolutional recurrent neural networks for hyperspectral data classification, Remote Sensing, 9(3): 298.   DOI
36 Xu, X., X. Ji, J. Jiang, X. Yao, Y. Tian, Y. Zhu, W. Cao, Q. Cao, H. Yang, Z. Shi, and T. Cheng, 2018. Evaluation of one-class support vector classification for mapping the paddy rice planting area in Jiangsu Province of China from Landsat 8 OLI imagery, Remote Sensing, 10(4): 546.   DOI
37 Zhong, L., P. Gong, and G.S. Biging, 2014. Efficient corn and soybean mapping with temporal extendability: a multi-year experiment using Landsat imagery, Remote Sensing of Environment, 140: 1-13.   DOI