Browse > Article
http://dx.doi.org/10.12673/jant.2021.25.1.115

Deep Learning Based Pine Nut Detection in UAV Aerial Video  

Kim, Gyu-Min (School of Electronics and Information Engineering, Korea Aerospace University)
Park, Sung-Jun (School of Electronics and Information Engineering, Korea Aerospace University)
Hwang, Seung-Jun (School of Electronics and Information Engineering, Korea Aerospace University)
Kim, Hee Yeong (LinktoTo Co. Ltd)
Baek, Joong-Hwan (School of Electronics and Information Engineering, Korea Aerospace University)
Publication Information
Journal of Advanced Navigation Technology / v.25, no.1, 2021 , pp. 115-123 More about this Journal
Abstract
Pine nuts are Korea's representative nut forest products and profitable crops. However, pine nuts are harvested by climbing the trees themselves, thus the risk is high. In order to solve this problem, it is necessary to harvest pine nuts using a robot or an unmanned aerial vehicle(UAV). In this paper, we propose a deep learning based detection method for harvesting pine nut in UAV aerial images. For this, a video was recorded in a real pine forest using UAV, and a data augmentation technique was used to supplement a small number of data. As the data for 3D detection, Unity3D was used to model the virtual pine nut and the virtual environment, and the labeling was acquired using the 3D transformation method of the coordinate system. Deep learning algorithms for detection of pine nuts distribution area and 2D and 3D detection of pine nuts objects were used DeepLabV3+, YOLOv4, and CenterNet, respectively. As a result of the experiment, the detection rate of pine nuts distribution area was 82.15%, the 2D detection rate was 86.93%, and the 3D detection rate was 59.45%.
Keywords
Unmanned aerial vehicle; Pine nut; Segmentation; Detection;
Citations & Related Records
연도 인용수 순위
  • Reference
1 W. J. Kim and J. H. Kang, "ToA-based localization algorithm for mitigating positioning error in NLoS channel," Journal of Institute of Control, Robotics and Systems, Vol. 24, No. 11. pp. 1043-1047, Oct. 2010.   DOI
2 C. Ju and H. I. Son, "Design of supervisors for the control of agricultural multiple UAV systems: A preliminary result," Journal of Institute of Control, Robotics and Systems, Vol. 24, No. 11, pp. 1005-1013, 2018.   DOI
3 U. R. Mogili and B. B. V. L. Deepark, "Review on application of drone systems in precision agriculture," Procedia Computer Science, Vol. 133, pp. 502-509, July. 2018.   DOI
4 Liu, W. Jia, C. Ruan, D. Zhao, Y. Gu, and W. Chen, "The recognition of apple fruits in plastic bags based on block classification," Precision Agriculture, Vol. 19, No. 4, pp. 735-749, Aug. 2018.   DOI
5 D. Choi, W. S. Lee, J. K. Schueller, R. Ehsani, F. Roka, and J. Diamond, "A performance comparison of RGB, NIR, and depth images in immature citrus detection using deep learning algorithms for yield prediction," in ASABE Annual International Meeting, Kentucky: KY, pp. 1, 2017.
6 Fu, M. Gong, C. Wang, K. Batmanghelich, and D. Tao, "Deep ordinal regression network for monocular depth estimation," in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City: SLC, pp. 2002-2011, 2018.
7 Y. Tian, G. Yang, Z. Wang, H. Wang, E. Li, and Z. Liang, " Apple detection during different growth stages in orchards using the improved YOLO-V3 model," Computers and Electronics in Agriculture, Vol. 157, pp. 417-426, Jan. 2019.   DOI
8 Y. Yu, K. Zhang, L. Yang, and D. Zhang, "Fruit detection for strawberry harvesting robot in non-structural environment based on Mask-RCNN," Computers and Electronics in Agriculture, Vol. 163, Jun. 2019.
9 X. Chen, K. Kundu, Z. Zhang, H. Ma, S. Fidler, and R. Urtasun, "Monocular 3d object detection for autonomous driving," in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas: LAS, pp. 2147-2156, 2016.
10 Ku, A. D. Pon and S. L.Waslander, "Monocular 3d object detection leveraging accurate proposals and shape reconstruction," in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, California: CA, pp. 11867-11876, 2019.
11 Z. Yang, Y. Sun, S. Liu, X. Shen, and J. Jia, "Std: sparse-to-dense 3d object detector for point cloud," in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, California: CA, pp. 1951-1960, 2019.
12 Y. You, Y. Wang, W. L. Chao, D. Garg, G. Pleiss, B. Hariharan, M. Campbell, and K. Q. Weinberger, "Pseudo -lidar++: accurate depth for 3d object detection in autonomous driving," in International Conference on Learning Representations, Addis Abada: ADD, pp. 1-22, 2020.
13 Karlsruche Institute of Technology. The KITTI Vision Benchmark Suite [Internet]. Available: http://www.cvlibs.net/datasets/kitti/eval_object.php?obj_benchmark=3d.
14 L. C. Chen, G. Papandreou, F. Schroff, and H. Adam, " Rethinking atrous convolution for semantic image segmentation," arXiv preprint arXiv:1706.05587, 2017.
15 A. Bochkovskiy, C. Y. Wang, and H. Y. M. Liao, "Yolov4: optimal speed and accuracy of object detection," arXiv preprint arXiv:2004.10934, 2020.
16 X. Zhou, D. Wang and P. Krahenbuhl, "Objects as points," arXiv preprint arXiv:1904.07850, 2019.
17 Y. S. Kim, B. R. Min, W. Kim, D. W. Kim, K. W. Seo, H. T. Kim and D. W. Lee, "Development of image processing system for tracking location of moving objects," Proceedings of the Korean Society for Agricultural Machinery Conference, pp. 292-297, 2002.
18 S. Shetty, "Application of convolutional neural network for image classification on Pascal VOC challenge 2012 dataset," arXiv preprint arXiv:1607.03785, 2016.