KIPS Transactions on Software and Data Engineering (정보처리학회논문지:소프트웨어 및 데이터공학)

Korea Information Processing Society (한국정보처리학회)
권호 표지
DOI QR코드

DOI QR Code

A Study on the Estimation of Multi-Object Social Distancing Using Stereo Vision and AlphaPose

Stereo Vision과 AlphaPose를 이용한 다중 객체 거리 추정 방법에 관한 연구

  • 이주민 (성균관대학교 인공지능융합학부) ;
  • 배현재 (차세대융합기술연구원 컴퓨터비전 및 인공지능 연구실) ;
  • 장규진 (차세대융합기술연구원 컴퓨터비전 및 인공지능 연구실) ;
  • 김진평 (차세대융합기술연구원 컴퓨터비전 및 인공지능 연구실)
  • Received : 2021.01.19
  • Accepted : 2021.04.27
  • Published : 2021.07.31
Download PDF
⟨ Previous Next ⟩

Abstract

Recently, We are carrying out a policy of physical distancing of at least 1m from each other to prevent the spreading of COVID-19 disease in public places. In this paper, we propose a method for measuring distances between people in real time and an automation system that recognizes objects that are within 1 meter of each other from stereo images acquired by drones or CCTVs according to the estimated distance. A problem with existing methods used to estimate distances between multiple objects is that they do not obtain three-dimensional information of objects using only one CCTV. his is because three-dimensional information is necessary to measure distances between people when they are right next to each other or overlap in two dimensional image. Furthermore, they use only the Bounding Box information to obtain the exact coordinates of human existence. Therefore, in this paper, to obtain the exact two-dimensional coordinate value in which a person exists, we extract a person's key point to detect the location, convert it to a three-dimensional coordinate value using Stereo Vision and Camera Calibration, and estimate the Euclidean distance between people. As a result of performing an experiment for estimating the accuracy of 3D coordinates and the distance between objects (persons), the average error within 0.098m was shown in the estimation of the distance between multiple people within 1m.

최근 COVID-19 확산 방지를 위한 공공장소에서는 최소 1m 이상을 유지하는 물리적 거리두기 정책을 실행하고 있다. 본 논문에서는 드론과 CCTV가 취득한 스테레오 영상에서 실시간으로 사람들 간의 거리를 추정하는 방법과 추정된 거리에서 1m 이내의 객체를 인식하는 자동화 시스템을 제안한다. 기존의 CCTV를 이용하여 다중 객체 간의 거리 추정에 사용되었던 방법의 문제점으로는 한 대의 CCTV만을 이용하여 객체의 3차원 정보를 얻지 못한다는 것이다. 선, 후행하거나 겹쳐진 사람 간의 거리를 구하기 위해서는 3차원 정보가 필요하기 때문이다. 또한, 일반적인 Detected Bounding Box를 사용하여 영역 안에서 사람이 존재하는 정확한 좌표를 얻지 못한다. 따라서 사람이 존재하는 정확한 위치 정보를 얻기 위해 스켈레톤 추출하여 관절 키포인트의 2차원 좌표를 획득한 후, Stereo Vision을 이용한 카메라 캘리브레이션을 적용하여 3차원 좌표로 변환한다. 3차원으로 변환된 관절 키포인트의 중심좌표를 계산하고 객체 간 사이의 거리를 추정한다. 3차원 좌표의 정확성과 객체(사람) 간의 거리 추정 실험을 수행한 결과, 1m 이내에 존재하는 다수의 사람 간의 거리 추정에서 0.098m 이내 평균오차를 보였다.

Keywords

Acknowledgement

이 논문은 행정안전부 극한재난대응기반기술개발사업의 지원을 받아 수행된 연구임(2020-MOIS31-014).

References

  1. T. P. B. Thu, T. P. N. H. Ngoca, N. M. Hai, and L. A.Tuanc, "Effect of the social distancing measures on the spread of COVID-19 in 10 highly infected countries," Proceeding of the Science of The Total Environment, 2020.
  2. L. S. Liebst, W. Bernasco, M. R. Lindegaard, and E. Hoeben, "Social distancing compliance: A video observational analysis," Proceeding of the PLoS ONE. 2021.
  3. J. Redmon, S. Divvala, R. Girshick, and A. Farhadi, "You only look once: Unified, real-time object detection," Proceedings of the IEEE International Conference on Computer Vision, 2016.
  4. H. Kaiming, G. Gkioxari, P. Dollar, and R. Girshick, "Mask R-CNN," Proceedings of the IEEE International Conference on Computer Vision, 2017.
  5. H. S. Fang, S. Xie, Y. W. Tai, and C. Lu, "Rmpe: Regional multi-person pose estimation," Proceedings of the IEEE International Conference on Computer Vision, 2017.
  6. J. Redmon and A. Farhadi, "Yolov3: An incremental improvement," arXiv preprint arXiv:1804.02767. 2018.
  7. H. Fu, M. Gong, C. Wang, K. Batmanghelich, and D. Tao, "Deep ordinal regression network for monocular depth estimation," Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2018.
  8. J. R. Chang and Y. S. Chen, "Pyramid stereo matching network," Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2018.
  9. Y. Wang, W. L. Chao, D. Garg, B. Hariharan, M. Campbell, and K. Q. Weinberger, "Pseudo-LiDAR from visual depth estimation: Bridging the gap in 3D object detection for autonomous driving," Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2019.
  10. I. Ahmed, M. Ahmad, J. P. C. Rodrigues, G. Jeon, and S. Din, "A deep learning-based social distance monitoring framwork for COVID-19," Sustainable Cities and Society, Vol.65, Article 10257r, 2020.
  11. M. Rezaei and M. Azarmi, "DeepSOCIAL: Social distancing monitoring and infection risk assessment in COVID-19 pandemic," Proceedings for the Applied Sciences, Vol.10, Article 7514, 2020.
  12. N. Neshov, A. Manolova, K. Tonchev, and V. Poulkov, "Real-time estimation of distance between people and/or objects in video surveillance," Proceedings of the International Symposium on Wireless Personal Multimedia Communications, 2020.
  13. L. Narupiyakul and N. Srisrisawang, "A comparison between skeleton and bounding box models for falling direction recognition," Proceedings of the International Conference on Robotics and Machine Vision, 2017.
  14. Z. Cao, G. Hidalgo, T. Simon, S. E. Wei, and Y. Sheikh, "OpenPose: Realtime multi-person 2D pose estimation using part affinity fields," Proceedings of the IEEE International Conference on Computer Vision, 2019.
  15. T. Y. Lin, M. Maire, S. Belongie, L. Bourdev, R. Girshick, J. Hays, et al. "Microsoft COCO: Common objects in context," Proceedings of the European Conference on Computer Vision, 2014.
  16. M. Andriluka, L. Pishchulin, P. Gehler, and B. Schiele, "2D human pose estimation: New benchmark and state of the art analysis," Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2014.
  17. Human Pose Estimation Image AI Data [Internet], https://aihub.or.kr/aidata/138
  18. D. Bolya, C. Zhou, F. Xiao, and Y. J. Lee, "YOLACT++: Better real-time instance segmentation," Proceedings of the ICCV Conference on Computer Vision, 2019.