Journal of the Korean Society for Aeronautical & Space Sciences (한국항공우주학회지)

The Korean Society for Aeronautical and Space Sciences (한국항공우주학회)
권호 표지
DOI QR코드

DOI QR Code

A Study on Realtime Drone Object Detection Using On-board Deep Learning

온-보드에서의 딥러닝을 활용한 드론의 실시간 객체 인식 연구

  • Received : 2021.05.24
  • Accepted : 2021.09.27
  • Published : 2021.10.01
Download PDF
⟨ Previous Next ⟩

Abstract

This paper provides a process for developing deep learning-based aerial object detection models that can run in realtime on onboard. To improve object detection performance, we pre-process and augment the training data in the training stage. In addition, we perform transfer learning and apply a weighted cross-entropy method to reduce the variations of detection performance for each class. To improve the inference speed, we have generated inference acceleration engines with quantization. Then, we analyze the real-time performance and detection performance on custom aerial image dataset to verify generalization.

본 논문에서는 드론을 활용한 감시정찰 임무의 효율성을 향상하기 위해 드론 탑재장비에서 실시간으로 구동 가능한 딥러닝 기반의 객체 인식 모델을 개발하는 연구를 수행하였다. 드론 영상 내 객체 인식 성능을 높이는 목적으로 학습 단계에서 학습 데이터 전처리 및 증강, 전이 학습을 수행하였고 각 클래스 별 성능 편차를 줄이기 위해 가중 크로스 엔트로피 방법을 적용하였다. 추론 속도를 개선하기 위해 양자화 기법이 적용된 추론 가속화 엔진을 생성하여 실시간성을 높였다. 마지막으로 모델의 성능을 확인하기 위해 학습에 참여하지 않은 드론 영상 데이터에서 인식 성능 및 실시간성을 분석하였다.

Keywords

References

  1. Jiao, L., Zhang, F., Liu, F., Yang, S., Li, L., Feng, Z. and Qu, R., "A survey of deep learningbased object detection," IEEE Access, Vol. 7, 2019, pp. 128837~128868. https://doi.org/10.1109/ACCESS.2019.2939201
  2. Vaddi, S., "Efficient object detection model for real-time UAV applications," Doctoral dissertation Iowa State University, 2019.
  3. Lee, J., Wang, J., Crandall, D., Sabanovic, S. and Fox, G., "Real-time, cloud-based object detection for unmanned aerial vehicles," International Conference on Robotic Computing (IRC) IEEE, April 2017, pp. 36~43.
  4. Du, D., et al, "The unmanned aerial vehicle benchmark: Object detection and tracking," Proceedings of the European Conference on Computer Vision (ECCV), 2018, pp. 370~386.
  5. Xia, G. S., et al, "DOTA: A large-scale dataset for object detection in aerial images," Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2018, pp. 3974~3983.
  6. Du, D., et al, "VisDrone-DET2019: The vision meets drone object detection in image challenge results," Proceedings of the IEEE/CVF International Conference on Computer Vision Workshops, 2019.
  7. Panchapagesan, S., Sun, M., Khare, A., Matsoukas, S., Mandal, A., Hoffmeister, B. and Vitaladevuni, S., "Multi-task learning and weighted cross-entropy for DNN-based keyword spotting," Interspeech, Vol. 9, September 2016, pp. 760~764.
  8. Redmon, J. and Farhadi, A., "Yolov3: An incremental improvement," Computer Vision and Pattern Recognition, April 2018.
  9. Ren, S., He, K., Girshick, R. and Sun, J., "Faster r-cnn: Towards real-time object detection with region proposal networks," Advances in Neural Information Processing Systems, Vol. 28, 2015, pp. 91~99.
  10. Shorten, C. and Khoshgoftaar, T. M., "A survey on image data augmentation for deep learning," Journal of Big Data, Vol. 6, No. 1, 2019, pp. 1~48. https://doi.org/10.1186/s40537-018-0162-3
  11. Yosinski, J., Clune, J., Bengio, Y. and Lipson, H., "How transferable are features in deep neural networks?," arXiv preprint arXiv:1411.1792, 2014.
  12. Salman, H., Ilyas, A., Engstrom, L., Kapoor, A. and Madry, A., "Do adversarially robust imagenet models transfer better?," arXiv preprint arXiv:2007.08489, 2020.
  13. Hanif, A. and Azhar, N., "Resolving class imbalance and feature selection in customer churn dataset," International Conference on Frontiers of Information Technology (FIT), 2017, pp. 82~86.
  14. Yoo, S. M., Lee, K. H., Park, J., Yoon, S. J., Cho, C., Jung, Y. J. and Cho, I. Y., "Trends in Deep Learning Inference Engines for Embedded Systems," Electronics and Telecommunications Trends, Vol. 34, No. 4, 2019, pp. 23~31.
  15. Padilla, R., Netto, S. L. and da Silva, E. A., "A survey on performance metrics for object-detection algorithms.," International Conference on Systems, Signals and Image Processing (IWSSIP), 2020, pp. 237~242.
  16. Lin, T. Y., et al, "Microsoft coco: Common objects in context," European Conference on Computer Vision, 2014, pp. 740~755.