Journal of Information Processing Systems

Korea Information Processing Society (한국정보처리학회)
권호 표지
DOI QR코드

DOI QR Code

A Survey of Deep Learning in Agriculture: Techniques and Their Applications

  • Ren, Chengjuan (Dept. of Software Convergence Engineering, Kunsan National University) ;
  • Kim, Dae-Kyoo (Dept. of Computer Science and Engineering, Oakland University) ;
  • Jeong, Dongwon (Dept. of Software Convergence Engineering, Kunsan National University)
  • Received : 2020.01.15
  • Accepted : 2020.04.07
  • Published : 2020.10.31
Download PDF
⟨ Previous Next ⟩

Abstract

With promising results and enormous capability, deep learning technology has attracted more and more attention to both theoretical research and applications for a variety of image processing and computer vision tasks. In this paper, we investigate 32 research contributions that apply deep learning techniques to the agriculture domain. Different types of deep neural network architectures in agriculture are surveyed and the current state-of-the-art methods are summarized. This paper ends with a discussion of the advantages and disadvantages of deep learning and future research topics. The survey shows that deep learning-based research has superior performance in terms of accuracy, which is beyond the standard machine learning techniques nowadays.

Keywords

References

  1. A. Singh, B. Ganapathysubramanian, A. K. Singh, and S. Sarkar, "Machine learning for high-throughput stress phenotyping in plants," Trends in Plant Science, vol. 21, no. 2, pp. 110-124, 2016. https://doi.org/10.1016/j.tplants.2015.10.015
  2. S. Park, J. Im, E. Jang, and J. Rhee, "Drought assessment and monitoring through blending of multi-sensor indices using machine learning approaches for different climate regions," Agricultural and Forest Meteorology, vol. 216, pp. 157-169, 2016. https://doi.org/10.1016/j.agrformet.2015.10.011
  3. D. C. Duro, S. E. Franklin, and M. G. Dube, "A comparison of pixel-based and object-based image analysis with selected machine learning algorithms for the classification of agricultural landscapes using SPOT-5 HRG imagery," Remote Sensing of Environment, vol. 118, pp. 259-272, 2012. https://doi.org/10.1016/j.rse.2011.11.020
  4. R. Lang, R. Lu, C. Zhao, H. Qin, and G. Liu, "Graph-based semi-supervised one class support vector machine for detecting abnormal lung sounds," Applied Mathematics and Computation, vol. 364, article no. 124487, 2020.
  5. Y. Liu, S. Zhou, W. Han, W. Liu, Z. Qiu, and C. Li, "Convolutional neural network for hyperspectral data analysis and effective wavelengths selection," Analytica Chimica Acta, vol. 1086, pp. 46-54, 2019. https://doi.org/10.1016/j.aca.2019.08.026
  6. J. Greeff and T. Belpaeme, "Why robots should be social: enhancing machine learning through social human-robot interaction," PLoS ONE, vol. 10, no. 9, article no. e0138061, 2015.
  7. E. Senft, P. Baxter, J. Kennedy, S. Lemaignan, and T. Belpaeme, "Supervised autonomy for online learning in human-robot interaction," Pattern Recognition Letters, vol. 99, pp. 77-86, 2017. https://doi.org/10.1016/j.patrec.2017.03.015
  8. G. Canal, S. Escalera, and C. Angulo, "A real-time Human-Robot Interaction system based on gestures for assistive scenarios," Computer Vision and Image Understanding, vol. 149, pp. 65-77, 2016. https://doi.org/10.1016/j.cviu.2016.03.004
  9. M. S. Hinton, The State on the Streets: Police and Politics in Argentina and Brazil. Boulder, CO: Lynne Rienner Publishers, 2006.
  10. I. Sutskever, G. E. Hinton, and G. W. Taylor, "The recurrent temporal restricted Boltzmann machine," Advances in Neural Information Processing System, vol. 21, pp. 1601-1608, 2009.
  11. X. Lu, Y. Tsao, S. Matsuda, and C. Hori, "Speech enhancement based on deep denoising autoencoder," in Proceedings of the 14th Annual Conference of the International Speech Communication Association, Lyon, France, 2013, pp. 436-440.
  12. Y. Kim, "Convolutional neural networks for sentence classification," 2014 [Online]. Available: https://arxiv.org/abs/1408.5882.
  13. A. Graves, A. Mohamed, and G. Hinton, "Speech recognition with deep recurrent neural networks," in Proceedings of 2013 IEEE International Conference on Acoustics, Speech and Signal Processing, Vancouver, Canada, 2013, pp. 6645-6649.
  14. Y. Chen, Z. Lin, X. Zhao, G. Wang, and Y. Gu, "Deep learning-based classification of hyperspectral data," IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, vol. 7, no. 6, pp. 2094-2107, 2014. https://doi.org/10.1109/JSTARS.2014.2329330
  15. C. Kalyoncu and O. Toygar, "Geometric leaf classification," Computer Vision and Image Understanding, vol. 133, pp. 102-109, 2015. https://doi.org/10.1016/j.cviu.2014.11.001
  16. A. Hannun, C. Case, J. Casper, B. Catanzaro, G. Diamos, E. Elsen, et al., "Deep Speech: scaling up end-to-end speech recognition," 2014 [Online]. Available: https://arxiv.org/abs/1412.5567.
  17. T. Chan, K. Jia, S. Gao, J. Lu, Z. Zeng, and Y. Ma, "PCANet: a simple deep learning baseline for image classification?," IEEE Transactions on Image Processing, vol. 24, no. 12, pp. 5017-5032, 2015. https://doi.org/10.1109/TIP.2015.2475625
  18. J. B. Robinson, D. M. Silburn, D. Rattray, D. M. Freebairn, A. Biggs, D. McClymont, and N. Christodoulou, "Modelling shows that the high rates of deep drainage in parts of the Goondoola Basin in semi-arid Queensland can be reduced with changes to the farming systems," Australian Journal of Soil Research, vol. 48, no. 1, pp. 58-68, 2010. https://doi.org/10.1071/SR09067
  19. A. Kamilaris and F. X. Prenafeta-Boldu, "Deep learning in agriculture: a survey," Computers and Electronics in Agriculture, vol. 147, pp. 70-90, 2018. https://doi.org/10.1016/j.compag.2018.02.016
  20. A. Chlingaryan, S. Sukkarieh, and B. Whelan, "Machine learning approaches for crop yield prediction and nitrogen status estimation in precision agriculture: a review," Computers and Electronics in Agriculture, vol. 151, pp. 61-69, 2018. https://doi.org/10.1016/j.compag.2018.05.012
  21. A. Kamilaris and F. X. Prenafeta-Boldu, "A review of the use of convolutional neural networks in agriculture," The Journal of Agricultural Science, vol. 156, no. 3, pp. 312-322, 2018. https://doi.org/10.1017/s0021859618000436
  22. D. I. Patricio and R. Rieder, "Computer vision and artificial intelligence in precision agriculture for grain crops: a systematic review," Computers and Electronics in Agriculture, vol. 153, pp. 69-81, 2018. https://doi.org/10.1016/j.compag.2018.08.001
  23. A. Gongal, S. Amatya, M. Karkee, Q. Zhang, and K. Lewis, "Sensors and systems for fruit detection and localization: a review," Computers and Electronics in Agriculture, vol. 116, pp. 8-19, 2015. https://doi.org/10.1016/j.compag.2015.05.021
  24. K. G. Liakos, P. Busato, D. Moshou, S. Pearson, and D. Bochtis, "Machine learning in agriculture: a review," Sensors, vol. 18, no. 8, pp. 1-29, 2018. https://doi.org/10.1109/JSEN.2017.2772700
  25. S. Mishra, D. Mishra, and G. H. Santra, "Applications of machine learning techniques in agricultural crop production: a review paper," Indian Journal of Science and Technology, vol. 9, no. 38, pp. 1-14, 2016.
  26. N. Zhu, X. Liu, Z. Liu, K. Hu, Y. Wang, J. Tan, et al., "Deep learning for smart agriculture: concepts, tools, applications, and opportunities," International Journal of Agricultural and Biological Engineering, vol. 11, no. 4, pp. 32-44, 2018.
  27. M. Weiss, F. Jacob, and G. Duveiller, "Remote sensing for agricultural applications: a meta-review," Remote Sensing of Environment, vol. 236, 111402, 2020.
  28. T. Mikolov, A. Deoras, D. Povey, L. Burget, and J. Cernocky, "Strategies for training large scale neural network language models," in Proceedings of 2011 IEEE Workshop on Automatic Speech Recognition & Understanding, Waikoloa, HI, 2011, pp. 196-201.
  29. G. Hinton, L. Deng, D. Yu, G. E. Dahl, A. Mohamed, N. Jaitly, et al., "Deep neural networks for acoustic modeling in speech recognition: the shared views of four research groups," IEEE Signal Processing Magazine, vol. 29, no. 6, pp. 82-97, 2012. https://doi.org/10.1109/MSP.2012.2205597
  30. T. N. Sainath, A. Mohamed, B. Kingsbury, and B. Ramabhadran, "Deep convolutional neural networks for LVCSR," in Proceedings of 2013 IEEE International Conference on Acoustics, Speech and Signal Processing, Vancouver, BC, Canada, 2013, pp. 8614-8618.
  31. P. Y. Huang, F. Liu, S. R. Shiang, J. Oh, and C. Dyer, "Attention-based multimodal neural machine translation," in Proceedings of the 1st Conference on Machine Translation, Volume 2: Shared Task Papers, Berlin, Germany, 2016, pp. 639-645.
  32. J. Bastings, I. Titov, W. Aziz, D. Marcheggiani, and K. Sima'an, "Graph convolutional encoders for syntax-aware neural machine translation," in Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, Copenhagen, Denmark, 2017, pp. 1957-1967.
  33. T. Shen, T. Zhou, G. Long, J. Jiang, S. Pan, and C. Zhang, "DiSAN: directional self-attention network for RNN/CNN-free language understanding," in Proceedings of the 32nd AAAI Conference on Artificial Intelligence, Palo Alto, CA, 2018, pp. 5446-5455.
  34. A. Krizhevsky, I. Sutskever, and G. E. Hinton, "ImageNet classification with deep convolutional neural networks," Advances in Neural Information Processing Systems, vol. 25, pp. 1097-1105, 2012.
  35. C. Farabet, C. Couprie, L. Najman, and Y. LeCun, "Learning hierarchical features for scene labeling," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 35, no. 8, pp. 1915-1929, 2013. https://doi.org/10.1109/TPAMI.2012.231
  36. J. Tompson, A. Jain, Y. LeCun, and C. Bregler, "Joint training of a convolutional network and a graphical model for human pose estimation," Advances in Neural Information Processing Systems, vol. 27, pp. 1799-1807, 2014.
  37. C. Szegedy, W. Liu, Y. Jia, P. Sermanet, S. Reed, D. Anguelov, D. Erhan, V. Vanhoucke, and A. Rabinovich, "Going deeper with convolutions," in Proceedings of 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Boston, MA, 2015, pp. 1-9.
  38. A. N. Lam, A. T. Nguyen, H. A. Nguyen, and T. N. Nguyen, "Combining Deep Learning with Information Retrieval to Localize Buggy Files for Bug Reports (N)," in Proceedings of 2015 30th IEEE/ACM International Conference on Automated Software Engineering (ASE), Lincoln, NE, 2015, pp. 476-481.
  39. P. S. Huang, X. He, J. Gao, L. Deng, A. Acero, and L. Heck, "Learning deep structured semantic models for web search using clickthrough data," in Proceedings of the 22nd ACM international conference on Information & Knowledge Management, San Francisco, CA, 2013, pp. 2333-2338.
  40. P. Hamel and D. Eck, "Learning features from music audio with deep belief networks," in Proceedings of the11th International Society for Music Information Retrieval Conference (ISMIR), Utrecht, The Netherlands, 2010, pp. 339-344.
  41. N. Srivastava and R. R. Salakhutdinov, "Multimodal learning with deep Boltzmann machines," Advances in Neural Information Processing Systems, vol. 25, pp. 2222-2230, 2012.
  42. J. G. Ha, H. Moon, J. T. Kwak, S. I. Hassan, M. Dang, O. N. Lee, and H. Y. Park, "Deep convolutional neural network for classifying Fusarium wilt of radish from unmanned aerial vehicles," Journal of Applied Remote Sensing, vol. 11, no. 4, article no. 042621, 2017.
  43. J. Ma, K. Du, F. Zheng, L. Zhang, Z. Gong, and Z. Sun, "A recognition method for cucumber diseases using leaf symptom images based on deep convolutional neural network," Computers and Electronics in Agriculture, vol. 154, pp. 18-24, 2018. https://doi.org/10.1016/j.compag.2018.08.048
  44. Y. Lu, S. Yi, N. Zeng, Y. Liu, and Y. Zhang, "Identification of rice diseases using deep convolutional neural networks," Neurocomputing, vol. 267, pp. 378-384, 2017. https://doi.org/10.1016/j.neucom.2017.06.023
  45. B. Liu, Y. Zhang, D. He, and Y. Li, "Identification of apple leaf diseases based on deep convolutional neural networks," Symmetry, vol. 10, no. 1, pp. 1-16, 2017. https://doi.org/10.3390/sym10010001
  46. S. P. Mohanty, D. P. Hughes, and M. Salathe, "Using deep learning for image-based plant disease detection," Frontiers in Plant Science, vol. 7, article no. 1419, 2016.
  47. T. T. Tran, J. W. Choi, T. T. H. Le, and J. W. Kim, "A comparative study of deep CNN in forecasting and classifying the macronutrient deficiencies on development of tomato plant," Applied Sciences, vol. 9, no. 8, article no. 1601, 2019.
  48. A. Fuentes, S. Yoon, S. C. Kim, and D. S. Park, "A robust Deep-Learning-based detector for real-time tomato plant diseases and pests recognition," Sensors, vol. 17, no. 9, article no. 2022, 2017
  49. G. Wang, Y. Sun, and J. Wang, "Automatic image-based plant disease severity estimation using deep learning," Computational Intelligence and Neuroscience, vol. 2017, article no. 2917536, 2017.
  50. L. Zhong, L. Hu, and H. Zhou, "Deep learning based multi-temporal crop classification," Remote Sensing of Environment, vol. 221, pp. 430-443, 2019. https://doi.org/10.1016/j.rse.2018.11.032
  51. A. Milioto, P. Lottes, and C. Stachniss, "Real-time blob-wise sugar beets vs weeds classification for monitoring fields using convolutional neural networks," in Proceedings of ISPRS Annals of the Photogrammetry, Remote Sensing and Spatial Information Sciences, Bonn, Germany, 2017, pp. 41-48.
  52. M. M. Ghazi, B. Yanikoglu, and E. Aptoula, "Plant identification using deep neural networks via optimization of transfer learning parameters," Neurocomputing, vol. 235, pp. 228-235, 2017. https://doi.org/10.1016/j.neucom.2017.01.018
  53. X. Zhu, M. Zhu, and H. Ren, "Method of plant leaf recognition based on improved deep convolutional neural network," Cognitive Systems Research, vol. 52, pp. 223-233, 2018. https://doi.org/10.1016/j.cogsys.2018.06.008
  54. P. A. Dias, A. Tabb, and H. Medeiros, "Apple flower detection using deep convolutional networks," Computers in Industry, vol. 99, pp. 17-28, Aug. 2018. https://doi.org/10.1016/j.compind.2018.03.010
  55. Y. Tian, G. Yang, Z. Wang, H. Wang, E. Li, and Z. Liang, "Apple detection during different growth stages in orchards using the improved YOLO-V3 model," Computers and Electronics in Agriculture, vol. 157, pp. 417-426, 2019. https://doi.org/10.1016/j.compag.2019.01.012
  56. M. Rahnemoonfar and C. Sheppard, "Deep count: fruit counting based on deep simulated learning," Sensors, vol. 17, no. 4, article no. 905, 2017.
  57. A. Yang, H. Huang, X. Zhu, X. Yang, P. Chen, S. Li, and Y. Xue, "Automatic recognition of sow nursing behaviour using deep learning-based segmentation and spatial and temporal features," Biosystems Engineering, vol. 175, pp. 133-145, 2018. https://doi.org/10.1016/j.biosystemseng.2018.09.011
  58. Y. Qiao, M. Truman, and S. Sukkarieh, "Cattle segmentation and contour extraction based on Mask R-CNN for precision livestock farming," Computers and Electronics in Agriculture, vol. 165, article no. 104958, 2019.
  59. S. Kumar, A. Pandey, K. S. R. Satwik, S. Kumar, S. K. Singh, A. K. Singh, and A. Mohan, "Deep learning framework for recognition of cattle using muzzle point image pattern," Measurement, vol. 116, pp. 1-17, 2018. https://doi.org/10.1016/j.measurement.2017.10.064
  60. M. F. Hansen, M. L. Smith, L. N. Smith, M. G. Salter, E. M. Baxter, M. Farish, and B. Grieve, "Towards on-farm pig face recognition using convolutional neural networks," Computers in Industry, vol. 98, pp. 145-152, 2018. https://doi.org/10.1016/j.compind.2018.02.016
  61. S. A. Jwade, A. Guzzomi, and A. Mian, "On farm automatic sheep breed classification using deep learning," Computers and Electronics in Agriculture, vol. 167, Article 105055, 2019.
  62. M. Tian, H. Guo, H. Chen, Q. Wang, C. Long, and Y. Ma, "Automated pig counting using deep learning," Computers and Electronics in Agriculture, vol. 163, article no. 104840, 2019.
  63. N. Kussul, M. Lavreniuk, S. Skakun, and A. Shelestov, "Deep learning classification of land cover and crop types using remote sensing data," IEEE Geoscience and Remote Sensing Letters, vol. 14, no. 5, pp. 778-782, 2017. https://doi.org/10.1109/LGRS.2017.2681128
  64. R. Gaetano, D. Ienco, K. Ose, and R. Cresson, "A two-branch CNN architecture for land cover classification of PAN and MS imagery," Remote Sensing, vol. 10, no. 11, article no. 1746, 2018.
  65. G. J. Scott, M. R. England, W. A. Starms, R. A. Marcum, and C. H. Davis, "Training deep convolutional neural networks for land-cover classification of high-resolution imagery," IEEE Geoscience and Remote Sensing Letters, vol. 14, no. 4, pp. 549-553, 2017. https://doi.org/10.1109/LGRS.2017.2657778
  66. H. Xing, Y. Meng, Z. Wang, K. Fan, and D. Hou, "Exploring geo-tagged photos for land cover validation with deep learning," ISPRS Journal of Photogrammetry and Remote Sensing, vol. 141, pp. 237-251, 2018. https://doi.org/10.1016/j.isprsjprs.2018.04.025
  67. M. Mahdianpari, B. Salehi, M. Rezaee, F. Mohammadimanesh, and Y. Zhang, "Very deep convolutional neural networks for complex land cover mapping using multispectral remote sensing imagery," Remote Sensing, vol. 10, no, 7, article no. 1119, 2018.
  68. P. Christiansen, L. N. Nielsen, K. A. Steen, R. N. Jorgensen, and H. Karstoft, "DeepAnomaly: combining background subtraction and deep learning for detecting obstacles and anomalies in an agricultural field," Sensors, vol. 16, no. 11, article no. 1904, 2016.
  69. K. A. Steen, P. Christiansen, H. Karstoft, and R. N. Jorgensen, "Using deep learning to challenge safety standard for highly autonomous machines in agriculture," Journal of Imaging, vol. 2, no. 1, article no. 6, 2016.
  70. Z. Khan, V. Rahimi-Eichi, S. Haefele, T. Garnett, and S. J. Miklavcic, "Estimation of vegetation indices for high-throughput phenotyping of wheat using aerial imaging," Plant Methods, vol. 14, article no. 20, 2018.
  71. Y. Kaneda, S. Shibata, and H. Mineno, "Multi-modal sliding window-based support vector regression for predicting plant water stress," Knowledge-Based Systems, vol. 134, pp. 135-148, 2017. https://doi.org/10.1016/j.knosys.2017.07.028
  72. X. Song, G. Zhang, F. Liu, D. Li, Y. Zhao, and J. Yang, "Modeling spatio-temporal distribution of soil moisture by deep learning-based cellular automata model," Journal of Arid Land, vol. 8, no. 5, pp. 734-748, 2016. https://doi.org/10.1007/s40333-016-0049-0
  73. Z. Wang, M. Hu, and G. Zhai, "Application of deep learning architectures for accurate and rapid detection of internal mechanical damage of blueberry using hyperspectral transmittance data," Sensors, vol. 18, no. 4, Article 1126, 2018.
  74. M. K. Saggi and S. Jain, "Reference evapotranspiration estimation and modeling of the Punjab Northern India using deep learning," Computers and Electronics in Agriculture, vol. 156, pp. 387-398, 2019. https://doi.org/10.1016/j.compag.2018.11.031
  75. A. Graves and J. Schmidhuber, "Framewise phoneme classification with bidirectional LSTM and other neural network architectures," Neural Networks, vol. 18, no. 5-6, pp. 602-610, 2005. https://doi.org/10.1016/j.neunet.2005.06.042
  76. A. Jain, A. R. Zamir, S. Savarese, and A. Saxena, "Structural-RNN: deep learning on spatio-temporal graphs," in Proceedings of 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, NV, 2016, pp. 5308-5317.
  77. Y. R. Chen, K. Chao, and M. S. Kim. "Machine vision technology for agricultural applications," Computers and Electronics in Agriculture, vol. 36, no. 2-3, pp. 173-191, 2002. https://doi.org/10.1016/S0168-1699(02)00100-X