KIPS Transactions on Software and Data Engineering (정보처리학회논문지:소프트웨어 및 데이터공학)

Korea Information Processing Society (한국정보처리학회)
권호 표지
DOI QR코드

DOI QR Code

A Study of Tram-Pedestrian Collision Prediction Method Using YOLOv5 and Motion Vector

YOLOv5와 모션벡터를 활용한 트램-보행자 충돌 예측 방법 연구

  • 김영민 (인천대학교 경제학과/컴퓨터공학부) ;
  • 안현욱 (고려대학교 전기전자공학부) ;
  • 전희균 (한국철도기술연구원 스마트트램연구실) ;
  • 김진평 (차세대융합기술연구원) ;
  • 장규진 (차세대융합기술연구원 컴퓨터비전 및 인공지능연구실) ;
  • 황현철 (한국철도기술연구원 스마트트램연구실)
  • Received : 2021.04.07
  • Accepted : 2021.07.11
  • Published : 2021.12.31
Download PDF
⟨ Previous Next ⟩

Abstract

In recent years, autonomous driving technologies have become a high-value-added technology that attracts attention in the fields of science and industry. For smooth Self-driving, it is necessary to accurately detect an object and estimate its movement speed in real time. CNN-based deep learning algorithms and conventional dense optical flows have a large consumption time, making it difficult to detect objects and estimate its movement speed in real time. In this paper, using a single camera image, fast object detection was performed using the YOLOv5 algorithm, a deep learning algorithm, and fast estimation of the speed of the object was performed by using a local dense optical flow modified from the existing dense optical flow based on the detected object. Based on this algorithm, we present a system that can predict the collision time and probability, and through this system, we intend to contribute to prevent tram accidents.

최근 자율주행에 관한 기술은 고부가가치 신기술로서 주목받고 있으며 활발히 연구가 진행되고 있는 분야이다. 상용화 가능한 자율주행을 위해서는 실시간으로 정확하게 진입하는 객체를 탐지하고 이동속도를 추정해야 한다. CNN(Convolutional Neural Network) 기반 딥러닝 알고리즘과 밀집광학흐름(Dense Optical Flow)을 사용하는 기존 방식은 실행 속도가 느려 실시간으로 객체를 탐지하고 이동속도를 추정하기에는 한계가 존재한다. 본 논문에서는 트램에 설치된 카메라를 통해 획득된 주행영상에서 딥러닝 알고리즘인 YOLOv5 알고리즘을 활용하여 실시간으로 객체를 탐지를 수행하고, 탐지된 객체영역에서 기존의 밀집광학흐름(Dense Optical Flow) 대신 연산량을 개선한 부분 밀집광학흐름(Local Dense Optical Flow)을 사용하여 객체의 진행 방향과 속력을 빠르게 추정하는 방식을 제안한다. 이를 바탕으로 충돌 시간과 충돌 지점을 예측할 수 있는 모델을 설계하였으며, 이를 통해 트램(Tram)의 주행 중 전방 충돌사고를 방지할 수 있는 시스템에 적용하고자 한다.

Keywords

Acknowledgement

이 논문은 한국철도기술연구원 자율주행 트램 기술고도화 및 시험운행과제의 지원을 받아 수행된 연구임(PK2103C5). 이 논문은 행정안전부 극한재난대응기반기술개발사업의 지원을 받아 수행된 연구임(2020-MOIS31-014).

References

  1. S. Lee, M. Myung, and T. Kim, "A study on tram traffic accidents characteristics and safety measures," Vol.39, No.4, pp.505-512, 2019.
  2. A. Manglik, X. Weng, E. Ohn-Bar, and K. M. Kitanil, "Forecasting time-to-collision from monocular video: Feasibility, dataset, and challenges," In 2019 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp.8081-8088, 2019.
  3. H. Kataoka, T. Suzuki, K. Nakashima, Y. Satoh, and Y. Aoki, "Joint pedestrian detection and risk-level prediction with motion-representation-by-detection," In 2020 IEEE International Conference on Robotics and Automation (ICRA). IEEE, pp.1021-1027, 2020.
  4. Y. Cai, L. Dai, H. Wang, L. Chen, Y. Li, M. A. Sotelo, and Z. Li, "Pedestrian motion trajectory prediction in intelligent driving from far shot first-person perspective video," IEEE Transactions on Intelligent Transportation Systems, pp.1-16, 2021.
  5. J. Redmon, S. Divvala, R. Girshick, and A. Farhadi, "You only look once: Unified, real-time object detection," In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp.779-788, 2016.
  6. C. Y. Wang, H. Y. M. Liao, Y. H. Wu, P. Y. Chen, J. W. Hsieh, and I. H. Yeh, "CSPNet: A new backbone that can enhance learning capability of CNN," Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp.390-391, 2020.
  7. J. Park, S. Woo, J. Y. Lee, and I. S. Kweon, "Bam: Bottleneck attention module," arXiv preprint arXiv:1807.06514, 2018.
  8. S. Elfwing, E. Uchibe, and K. Doya, "Sigmoid-weighted linear units for neural network function approximation in reinforcement learning," Neural Networks, Vol.107, pp.3-11, 2018. https://doi.org/10.1016/j.neunet.2017.12.012
  9. S. Liu, L. Qi, H. Qin, J. Shi, and J. Jia, "Path aggregation network for instance segmentation," Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp.8759-8768, 2018.
  10. J. Redmon and A. Farhadi, "Yolov3: An incremental improvement," In Computer Vision and Pattern Recognition, pp.1804-2767, 2018.
  11. T. Y. Lin, et al., "Microsoft coco: Common objects in context," European Conference on Computer Vision. Springer, Cham, pp.740-755, 2014.
  12. Ultralytics. YOLOv5 [Internet] https://github.com/ultralytics/yolov5
  13. B. K. Horn and B. G. Schunck, "Determining optical flow," Artificial Intelligence, Vol.17, No.1-3, pp.185-203, 1981. https://doi.org/10.1016/0004-3702(81)90024-2
  14. A. B. Khalifa, I. Alouani, M. A. Mahjoub, and N. E. B. Amara, "Pedestrian detection using a moving camera: A novel framework for foreground detection," Cognitive Systems Research, Vol.60, pp.77-96, 2020. https://doi.org/10.1016/j.cogsys.2019.12.003
  15. A. Bruhn, J. Weickert, and C. Schnorr, "Lucas/Kanade meets Horn/Schunck: Combining local and global optic flow methods," International Journal of Computer Vision, Vol.61, No.3, pp.211-231, 2005. https://doi.org/10.1023/B:VISI.0000045324.43199.43
  16. G. Farneback, "Two-frame motion estimation based on polynomial expansion," Scandinavian Conference on Image Analysis. Springer, Berlin, Heidelberg, 2003.
  17. A. Dosovitskiy, et al., "Flownet: Learning optical flow with convolutional networks," In Proceedings of the IEEE International Conference on Computer Vision, pp.2758-2766, 2015.
  18. E. Ilg, N. Mayer, T. Saikia, M. Keuper, A. Dosovitskiy, and T. Brox, "Flownet 2.0: Evolution of optical flow estimation with deep networks," In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp.2462-2470, 2017.
  19. M. Bevilacqua, A. Tsourdos, and A. Starr, "Egomotion estimation for monocular camera visual odometer," IEEE International Instrumentation and Measurement Technology Conference Proceedings, pp.1-6, 2016.
  20. J. Lim and B. Han, "Generalized background subtraction using superpixels with label integrated motion estimation," European Conference on Computer Vision, Springer, Cham, pp.173-187, 2014.
  21. L. Kurnianggoro, A. Shahbaz, and K. H. Jo, "Dense optical flow in stabilized scenes for moving object detection from a moving camera," 2016 16th International Conference on Control, Automation and Systems (ICCAS), pp.704-708, 2016.
  22. J. Mao, H. Yu, , X. Gao, and L. Yu, "Complementary motion vector for motion prediction in video coding with longterm reference," In 2019 Picture Coding Symposium (PCS), pp.1-5, 2019.
  23. M. Muhlhausen, L. Wohler, G. Albuquerque, and M. Magnor, "Iterative optical flow refinement for high resolution images," In IEEE International Conference on Image Processing (ICIP), pp.1282-1286, 2019.
  24. Y. C. Kuo, C. T. Tsai, and C. H. Chang, "Fast estimation of pedestrian movement," Sensors and Materials, Vol.29, No.6, pp.713-726, 2017,