한국항행학회논문지 (Journal of Advanced Navigation Technology)

  • 제25권1호
  • /
  • Pages.115-123
  • /
  • 2021
  • /
  • 1226-9026(pISSN)
  • /
  • 2288-842X(eISSN)
한국항행학회 (The Korean Navigation Institute)
권호 표지
DOI QR코드

DOI QR Code

UAV 항공 영상에서의 딥러닝 기반 잣송이 검출

Deep Learning Based Pine Nut Detection in UAV Aerial Video

  • 김규민 (한국항공대학교 항공전자정보공학부) ;
  • 박성준 (한국항공대학교 항공전자정보공학부) ;
  • 황승준 (한국항공대학교 항공전자정보공학부) ;
  • 김희영 ((주)링크투) ;
  • 백중환 (한국항공대학교 항공전자정보공학부)
  • Kim, Gyu-Min (School of Electronics and Information Engineering, Korea Aerospace University) ;
  • Park, Sung-Jun (School of Electronics and Information Engineering, Korea Aerospace University) ;
  • Hwang, Seung-Jun (School of Electronics and Information Engineering, Korea Aerospace University) ;
  • Kim, Hee Yeong (LinktoTo Co. Ltd) ;
  • Baek, Joong-Hwan (School of Electronics and Information Engineering, Korea Aerospace University)
  • 투고 : 2021.01.31
  • 심사 : 2021.02.26
  • 발행 : 2021.02.28
PDF 다운로드
⟨ 이전 논문 다음 논문 ⟩

초록

잣은 우리나라 대표적인 견과류 임산물이자 수익형 작물이다. 그러나 잣송이는 사람이 직접 나무 위로 올라가 수확하기 때문에 위험성이 높다. 이러한 문제를 해결하기 위해서 로봇 또는 UAV(unmanned aerial vehicle)를 이용한 잣송이 수확이 필요하다. 본 논문에서는 UAV를 이용한 잣송이 수확을 위해 UAV 항공 영상에서 딥러닝(deep learning) 기반의 잣송이 검출 기법을 제안한다. 이를 위해, UAV를 이용하여 실제 잣나무 숲에서 동영상을 촬영했으며, 적은 수의 데이터 보완을 위해 데이터 증강기법을 사용했다. 3D 검출을 위한 데이터로는 Unity3D을 이용하여 가상 잣송이 및 가상환경을 3D 모델링 하였으며 라벨링은 좌표계의 3차원 변환법을 이용해 구축했다. 잣 분포 영역 검출, 잣 객체에 대한 2D 및 3D 검출을 위한 딥러닝 알고리즘은 DeepLabV3, YOLOv4, CenterNet을 각각 이용하였다. 실험 결과, 잣송이 분포 영역 검출률은 82.15%, 2D 검출률은 86.93%, 3D 검출률은 59.45%이었다.

Pine nuts are Korea's representative nut forest products and profitable crops. However, pine nuts are harvested by climbing the trees themselves, thus the risk is high. In order to solve this problem, it is necessary to harvest pine nuts using a robot or an unmanned aerial vehicle(UAV). In this paper, we propose a deep learning based detection method for harvesting pine nut in UAV aerial images. For this, a video was recorded in a real pine forest using UAV, and a data augmentation technique was used to supplement a small number of data. As the data for 3D detection, Unity3D was used to model the virtual pine nut and the virtual environment, and the labeling was acquired using the 3D transformation method of the coordinate system. Deep learning algorithms for detection of pine nuts distribution area and 2D and 3D detection of pine nuts objects were used DeepLabV3+, YOLOv4, and CenterNet, respectively. As a result of the experiment, the detection rate of pine nuts distribution area was 82.15%, the 2D detection rate was 86.93%, and the 3D detection rate was 59.45%.

키워드

참고문헌

  1. W. J. Kim and J. H. Kang, "ToA-based localization algorithm for mitigating positioning error in NLoS channel," Journal of Institute of Control, Robotics and Systems, Vol. 24, No. 11. pp. 1043-1047, Oct. 2010. https://doi.org/10.5302/j.icros.2018.18.0175
  2. C. Ju and H. I. Son, "Design of supervisors for the control of agricultural multiple UAV systems: A preliminary result," Journal of Institute of Control, Robotics and Systems, Vol. 24, No. 11, pp. 1005-1013, 2018. https://doi.org/10.5302/j.icros.2018.18.0161
  3. U. R. Mogili and B. B. V. L. Deepark, "Review on application of drone systems in precision agriculture," Procedia Computer Science, Vol. 133, pp. 502-509, July. 2018. https://doi.org/10.1016/j.procs.2018.07.063
  4. Liu, W. Jia, C. Ruan, D. Zhao, Y. Gu, and W. Chen, "The recognition of apple fruits in plastic bags based on block classification," Precision Agriculture, Vol. 19, No. 4, pp. 735-749, Aug. 2018. https://doi.org/10.1007/s11119-017-9553-2
  5. D. Choi, W. S. Lee, J. K. Schueller, R. Ehsani, F. Roka, and J. Diamond, "A performance comparison of RGB, NIR, and depth images in immature citrus detection using deep learning algorithms for yield prediction," in ASABE Annual International Meeting, Kentucky: KY, pp. 1, 2017.
  6. Y. Tian, G. Yang, Z. Wang, H. Wang, E. Li, and Z. Liang, " Apple detection during different growth stages in orchards using the improved YOLO-V3 model," Computers and Electronics in Agriculture, Vol. 157, pp. 417-426, Jan. 2019. https://doi.org/10.1016/j.compag.2019.01.012
  7. Y. Yu, K. Zhang, L. Yang, and D. Zhang, "Fruit detection for strawberry harvesting robot in non-structural environment based on Mask-RCNN," Computers and Electronics in Agriculture, Vol. 163, Jun. 2019.
  8. X. Chen, K. Kundu, Z. Zhang, H. Ma, S. Fidler, and R. Urtasun, "Monocular 3d object detection for autonomous driving," in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas: LAS, pp. 2147-2156, 2016.
  9. Fu, M. Gong, C. Wang, K. Batmanghelich, and D. Tao, "Deep ordinal regression network for monocular depth estimation," in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City: SLC, pp. 2002-2011, 2018.
  10. Ku, A. D. Pon and S. L.Waslander, "Monocular 3d object detection leveraging accurate proposals and shape reconstruction," in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, California: CA, pp. 11867-11876, 2019.
  11. Z. Yang, Y. Sun, S. Liu, X. Shen, and J. Jia, "Std: sparse-to-dense 3d object detector for point cloud," in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, California: CA, pp. 1951-1960, 2019.
  12. Y. You, Y. Wang, W. L. Chao, D. Garg, G. Pleiss, B. Hariharan, M. Campbell, and K. Q. Weinberger, "Pseudo -lidar++: accurate depth for 3d object detection in autonomous driving," in International Conference on Learning Representations, Addis Abada: ADD, pp. 1-22, 2020.
  13. L. C. Chen, G. Papandreou, F. Schroff, and H. Adam, " Rethinking atrous convolution for semantic image segmentation," arXiv preprint arXiv:1706.05587, 2017.
  14. A. Bochkovskiy, C. Y. Wang, and H. Y. M. Liao, "Yolov4: optimal speed and accuracy of object detection," arXiv preprint arXiv:2004.10934, 2020.
  15. X. Zhou, D. Wang and P. Krahenbuhl, "Objects as points," arXiv preprint arXiv:1904.07850, 2019.
  16. Y. S. Kim, B. R. Min, W. Kim, D. W. Kim, K. W. Seo, H. T. Kim and D. W. Lee, "Development of image processing system for tracking location of moving objects," Proceedings of the Korean Society for Agricultural Machinery Conference, pp. 292-297, 2002.
  17. S. Shetty, "Application of convolutional neural network for image classification on Pascal VOC challenge 2012 dataset," arXiv preprint arXiv:1607.03785, 2016.
  18. Karlsruche Institute of Technology. The KITTI Vision Benchmark Suite [Internet]. Available: http://www.cvlibs.net/datasets/kitti/eval_object.php?obj_benchmark=3d.