Journal of the Korea Computer Graphics Society (한국컴퓨터그래픽스학회논문지)

Korea Computer Graphics Society (한국컴퓨터그래픽스학회)
권호 표지
DOI QR코드

DOI QR Code

Climbing Motion Synthesis using Reinforcement Learning

강화학습을 이용한 클라이밍 모션 합성

  • Kyungwon Kang (Dept. of Computer and Software, Hanyang University) ;
  • Taesoo Kwon (Dept. of Computer and Software, Hanyang University)
  • 강경원 (한양대학교 일반대학원 컴퓨터소프트웨어학과) ;
  • 권태수 (한양대학교 일반대학원 컴퓨터소프트웨어학과)
  • Received : 2024.04.08
  • Accepted : 2024.05.13
  • Published : 2024.06.01
Download PDF
⟨ Previous Next ⟩

Abstract

Although there is an increasing demand for capturing various natural motions, collecting climbing motion data is difficult due to technical complexities, related to obscured markers. Additionally, scanning climbing structures and preparing diverse routes further complicate the collection of necessary data. To tackle this challenge, this paper proposes a climbing motion synthesis using reinforcement learning. The method comprises two learning stages. Firstly, the hanging policy is trained to grasp holds in a natural posture. Once the policy is obtained, it is used to extract the positions of the holds, postures, and gripping states, thus forming a dataset of favorable initial poses. Subsequently, the climbing policy is trained to execute actual climbing maneuvers using this initial state dataset. The climbing policy allows the character to move to the target location using limbs more evenly in a natural posture. Experiments have shown that the proposed method can effectively explore the space of good postures for climbing and use limbs more evenly. Experimental results demonstrate the effectiveness of the proposed method in exploring optimal climbing postures and promoting balanced limb utilization.

최근 자연스러운 모션 데이터에 대한 수요가 늘고 있지만, 클라이밍 모션을 정확하게 캡처하는 것은 가려진 부분이 많은 클라이밍 동작의 특성상 쉽지 않다. 또한 벽 구조물의 스캔이나 다양한 암벽 코스 준비 등 필요한 데이터를 수집하는 과정이 쉽지 않다. 본 논문에서는 강화학습을 이용한 클라이밍 모션 합성 방법론을 제안한다. 학습 과정은 두 단계의 난이도로 구성되어 있다. 첫 번째 단계는 매달리기 정책을 학습하는 것이다. 매달리기 정책은 자연스러운 자세로 홀드를 잡는 방법을 학습한다. 이후 추론 단계를 통해 위치, 자세, 잡기 상태를 다양하게 추출한 초기 상태 데이터세트를 만든다. 두 번째 단계에서는 이 초기 상태 데이터세트를 사용해서 실제 클라이밍을 수행하는 태스크를 학습한다. 클라이밍 정책은 자연스러운 자세로 타겟 위치로 이동하는 방법을 학습한다. 실험을 통해 제안하는 방법이 클라이밍 하기 위한 좋은 자세를 효과적으로 탐색할 수 있는 것을 보였다.

Keywords

Acknowledgement

이 논문은 2024년도 정부(과학기술정보통신부)의 재원으로 정보통신기획평가원의 지원(No. 2021-0-00320, 실 공간 대상 XR 생성 및 변형/증강 기술 개발)과 정부(과학기술정보통신부)의 재원으로 한국연구재단의 지원을 받아 수행된 연구임(RS-2023-00222776).

References

  1. S. Maheshwari, R. Narain, and R. Hebbalaguppe, "Transfer4d: A framework for frugal motion capture and deformation transfer," Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 12836-12846, 2023.
  2. X. Yi, Y. Zhou, and F. Xu, "Transpose: Real-time 3d human translation and pose estimation with six inertial sensors," ACM Transactions on Graphics, vol. 40, no. 4, 2021.
  3. X. B. Peng, P. Abbeel, S. Levine, and M. van de Panne, "Deepmimic: Example-guided deep reinforcement learning of physics-based character skills," ACM Trans. Graph., vol. 37, no. 4, pp. 143:1-143:14, July 2018.
  4. X. B. Peng, Z. Ma, P. Abbeel, S. Levine, and A. Kanazawa, "Amp: Adversarial motion priors for stylized physics-based character control," ACM Trans. Graph., vol. 40, no. 4, July 2021.
  5. X. B. Peng, Y. Guo, L. Halper, S. Levine, and S. Fidler, "Ase: Large-scale reusable adversarial skill embeddings for physically simulated characters," ACM Trans. Graph., vol. 41, no. 4, July 2022.
  6. J. Ho and S. Ermon, "Generative adversarial imitation learning," in Advances in Neural Information Processing Systems, vol. 29, 2016.
  7. C. Tessler, Y. Kasten, Y. Guo, S. Mannor, G. Chechik, and X. B. Peng, "Calm: Conditional adversarial latent models for directable virtual characters," in ACM SIGGRAPH 2023 Conference Proceedings. New York, NY, USA: Association for Computing Machinery, 2023.
  8. Z. Dou, X. Chen, Q. Fan, T. Komura, and W. Wang, "C.ase: Learning conditional adversarial skill embeddings for physics-based characters," arXiv preprint arXiv:2309.11351, 2023.
  9. Q. Zhu, H. Zhang, M. Lan, and L. Han, "Neural categorical priors for physics-based character control," ACM Trans. Graph., vol. 42, no. 6, dec 2023. [Online]. Available: https://doi.org/10.1145/3618397
  10. K. Naderi, J. Rajamaki, and P. Hamalainen, "Discovering and synthesizing humanoid climbing movements," ACM Trans. Graph., vol. 36, no. 4, jul 2017.
  11. K. Naderi, A. Babadi, and P. Hamalainen, "Learning physically based humanoid climbing movements," Computer Graphics Forum, vol. 37, no. 8, pp. 69-80, 2018.
  12. K. Naderi, A. Babadi, S. Roohi, and P. Hamalainen, "A reinforcement learning approach to synthesizing climbing movements," in 2019 IEEE Conference on Games (CoG), 2019, pp. 1-7.
  13. Adobe, "mixamo," https://www.mixamo.com, 2020.
  14. M. Yan, X. Wang, Y. Dai, S. Shen, C. Wen, L. Xu, Y. Ma, and C. Wang, "Cimi4d: A large multimodal climbing motion dataset under human-scene interactions," in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), June 2023, pp. 12977-12988.
  15. J. Schulman, F. Wolski, P. Dhariwal, A. Radford, and O. Klimov, "Proximal policy optimization algorithms," CoRR, vol. abs/1707.06347, 2017. [Online]. Available: http://arxiv.org/abs/1707.06347
  16. V. Makoviychuk, L. Wawrzyniak, Y. Guo, M. Lu, K. Storey, M. Macklin, D. Hoeller, N. Rudin, A. Allshire, A. Handa, and G. State, "Isaac gym: High performance gpu-based physics simulation for robot learning," CoRR, vol. abs/2108.10470, 2021. [Online]. Available: https://arxiv.org/abs/2108.10470 108.10470