KIPS Transactions on Software and Data Engineering (정보처리학회논문지:소프트웨어 및 데이터공학)

Korea Information Processing Society (한국정보처리학회)
권호 표지
DOI QR코드

DOI QR Code

Deep Learning-Based Outlier Detection and Correction for 3D Pose Estimation

3차원 자세 추정을 위한 딥러닝 기반 이상치 검출 및 보정 기법

  • 주찬양 (한양대학교 인공지능융합학과 바이오인공지능융합전공) ;
  • 박지성 (한양대학교 인공지능융합학과 바이오인공지능융합전공) ;
  • 이동호 (한양대학교 인공지능융합학과 바이오인공지능융합전공)
  • Received : 2021.12.24
  • Accepted : 2022.03.24
  • Published : 2022.10.31
Download PDF
⟨ Previous Next ⟩

Abstract

In this paper, we propose a method to improve the accuracy of 3D human pose estimation model in various move motions. Existing human pose estimation models have some problems of jitter, inversion, swap, miss that cause miss coordinates when estimating human poses. These problems cause low accuracy of pose estimation models to detect exact coordinates of human poses. We propose a method that consists of detection and correction methods to handle with these problems. Deep learning-based outlier detection method detects outlier of human pose coordinates in move motion effectively and rule-based correction method corrects the outlier according to a simple rule. We have shown that the proposed method is effective in various motions with the experiments using 2D golf swing motion data and have shown the possibility of expansion from 2D to 3D coordinates.

본 논문에서는 다양한 운동 모션에서 3차원 사람 자세 추정 모델의 정확도를 향상하는 방법을 제안한다. 기존의 사람 자세 추정 모델은 사람의 자세를 추정할 때 좌표 오차를 유발하는 흔들림, 반전, 교환, 오검출 등의 문제가 발생한다. 이러한 문제는 사람 자세 추정 모델의 정확한 자세 추정을 어렵게 한다. 이를 해결하기 위해 본 논문에서는 딥러닝 기반 이상치 검출 및 보정 방법을 제안한다. 딥러닝 기반의 이상치 검출 방법은 여러 모션에서 좌표의 이상치를 효과적으로 검출하고, 모션의 특징을 활용한 규칙 기반 보정 방법을 통해 이상치를 보정한다. 다양한 실험과 분석을 통하여 제안하는 방법이 골프 스윙 모션과 다양한 운동 모션에서도 사람의 자세를 정확히 추정할 수 있고, 3차원 좌표 데이터에서도 확장 가능함을 보인다.

Keywords

Acknowledgement

이 논문은 2022년도 정부(과학기술정보통신부)의 재원으로 정보통신기획평가원의 지원을 받아 수행된 연구임(No.2020-0-01343, 인공지능융합연구센터지원(한양대학교 ERICA).

References

  1. M. R. Ronchi and P. Perona "Matteo; PERONA, Pietro. Benchmarking and error diagnosis in multi-instance pose estimation," In Proceedings of the IEEE International Conference on Computer Vision, pp.369-378, 2017.
  2. C. Y. Ju, J. S. Park, G. S. Oh, H. J. Choi, and D. H. Lee. "An efficient Bi-LSTM based method for outlier detection and correction in golf swing motion estimation." Proceedings of the Korea Information Processing Society Conference, pp.787-790, 2021.
  3. Q. Dang, J. Yin, B. Wang, and W. Zheng, "Deep learning based 2d human pose estimation: A survey," Tsinghua Science and Technology, Vol.24, No.6, pp.663-676, 2019. https://doi.org/10.26599/TST.2018.9010100
  4. V. Bazarevsky, L. Grishchenko, K. Raveendran, T. Zhu, F. Zhang, and M. Grundmann, "BlazePose: On-device real-time body pose tracking," arXiv preprint arXiv:2006.10204, 2020.
  5. K. Sun, B. Xiao, D. Liu, and J. Wang, "Deep high-resolution representation learning for human pose estimation," In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp.5693-5703, 2019.
  6. Z. Cao, G. Hidalgo, T. Simon, S. E. Wei, and Y. Sheikh, "OpenPose: Realtime multi-person 2D pose estimation using part affinity fields," IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol.43, No.1, pp.172-186, 2019.
  7. Y. Li, D. Yang, Y. Chen, C. Peng, Z. Sun, and L. Jiao, "A lightweight top-down multi-person pose estimation method based on symmetric transformation and global matching," IEEE Access, Vol.10, pp.22112-22122, 2022. https://doi.org/10.1109/ACCESS.2022.3151136
  8. G. Moon, J. Y. Chang, and K. M. Lee, "Posefix: Model-agnostic general human pose refinement network," In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp.7773-7781, 2019.
  9. A. Newell, K. Yang, and J. Deng, "Stacked hourglass networks for human pose estimation," In European Conference on Computer Vision, Springer, Cham, pp.483-499, 2016.
  10. A. Bulat and G. Tzimiropoulos, "Human pose estimation via convolutional part heatmap regression," In European Conference on Computer Vision, Springer, Cham, pp.717-732, 2016.
  11. S. E. Wei, V. Ramakrishna, T. Kanade, and Y. Sheikh, "Convolutional pose machines," In Proceedings of the IEEE conference on Computer Vision and Pattern Recognition, pp.4724-4732, 2016.
  12. Y. Chen, Z. Wang, Y. Peng, Z. Zhang, G. Yu, and J. Sun, "Cascaded pyramid network for multi-person pose estimation," In Proceedings of the IEEE conference on computer vision and pattern recognition, pp.7103-7112, 2018.
  13. J. Carreira, P. Agrawal, K. Fragkiadaki, and J. Malik, "Human pose estimation with iterative error feedback," In Proceedings of the IEEE conference on computer vision and pattern recognition, pp.4733-4742, 2016.
  14. Y. Cai et al."Learning delicate local representations for multiperson pose estimation," In European Conference on Computer Vision, pp.455-472, 2020.
  15. J. Wang, X. Long, Y. Gao, E. Ding, and S. Wen, "Graph-pcnn: Two stage human pose estimation with graph pose refinement," In European Conference on Computer Vision, pp.492-508, 2020.
  16. M. Fieraru, A. Khoreva, L. Pishchulin, and B. Schiele, "Learning to refine human pose estimation," In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, pp.205-214, 2018.
  17. S. Li, J. Yi, Y. A. Farha, and J. Gall, "Pose refinement graph convolutional network for skeleton-based action recognition," IEEE Robotics and Automation Letters, Vol.6, No.2, pp.1028-1035, 2021. https://doi.org/10.1109/LRA.2021.3056361
  18. T. V. Marcard, R. Henschel, M. J. Black, B. Rosenhahn, and G. Pons-Moll, "Recovering accurate 3d human pose in the wild using imus and a moving camera," In Proceedings of the European Conference on Computer Vision (ECCV), pp.601-617, 2018.