로봇학회논문지 (The Journal of Korea Robotics Society)

  • 제18권2호
  • /
  • Pages.143-154
  • /
  • 2023
  • /
  • 1975-6291(pISSN)
  • /
  • 2287-3961(eISSN)
한국로봇학회 (Korea Robotics Society)
권호 표지
DOI QR코드

DOI QR Code

심층 강화학습을 이용한 휠-다리 로봇의 3차원 장애물극복 고속 모션 계획 방법

Fast Motion Planning of Wheel-legged Robot for Crossing 3D Obstacles using Deep Reinforcement Learning

  • Soonkyu Jeong (Department of Mechatronics Engineering, Chungnam National University) ;
  • Mooncheol Won (Department of Mechatronics Engineering, Chungnam National University)
  • 투고 : 2023.02.17
  • 심사 : 2023.02.27
  • 발행 : 2023.05.31
PDF 다운로드
⟨ 이전 논문 다음 논문 ⟩

초록

In this study, a fast motion planning method for the swing motion of a 6x6 wheel-legged robot to traverse large obstacles and gaps is proposed. The motion planning method presented in the previous paper, which was based on trajectory optimization, took up to tens of seconds and was limited to two-dimensional, structured vertical obstacles and trenches. A deep neural network based on one-dimensional Convolutional Neural Network (CNN) is introduced to generate keyframes, which are then used to represent smooth reference commands for the six leg angles along the robot's path. The network is initially trained using the behavioral cloning method with a dataset gathered from previous simulation results of the trajectory optimization. Its performance is then improved through reinforcement learning, using a one-step REINFORCE algorithm. The trained model has increased the speed of motion planning by up to 820 times and improved the success rates of obstacle crossing under harsh conditions, such as low friction and high roughness.

키워드

참고문헌

  1. S. Jeong and M. Won, "Motion Planning and Control of Wheel-legged Robot for Obstacle Crossing," The Journal of Korea Robotics Society, vol.17, no.4, pp. 500-507, Dec., 2022, DOI: 10.7746/jkros.2022.17.4.500.
  2. X. B. P eng, G. Berseth, K. Yin, and M. Van De P anne, "DeepLoco: Dynamic locomotion skills using hierarchical deep reinforcement learning," ACM Transactions on Graphics, vol. 36, no. 4, pp. 1-13, Jul., 2017, DOI: 10.1145/3072959.3073602.
  3. V. Tsounis, M. Alge, J. Lee, F. Farshidian, and M. Hutter, "DeepGait: Planning and control of quadrupedal gaits using deep reinforcement learning," IEEE Robotics and Automation Letters, vol. 5, no. 2, pp. 3699-3706, Apr., 2020, DOI: 10.1109/LRA.2020.2979660.
  4. N. Rudin, D. Hoeller, P. Reist, and M. Hutter, "Learning to Walk in Minutes Using Massively Parallel Deep Reinforcement Learning," Robotics, 2021, DOI: 10.48550/arXiv.2109.11978.
  5. K. Arulkumaran, M. P . Deisenroth, M. Brundage, and A. A. Bharath, "Deep reinforcement learning: a brief survey," IEEE Signal Processing Magazine, vol. 34, no. 6, pp. 26-38, Nov., 2017, DOI: 10.1109/MSP.2017.2743240.
  6. M. Bain and C. Sammut, "A framework for behavioural cloning," Machine Intelligence 15, 1995, [Online], https://www.semanticscholar.org/paper/A-Framework-for-Behavioural-Cloning-Bain-Sammut/1f4731d5133cb96ab30e08bf39dffa874aebf487, Accessed: Feb., 13, 2023.
  7. J. D. Chang, M. Uehara, D. Sreenivas, R. Kidambi, and W. Sun, "Mitigating covariate shift in imitation learning via offline data with partial coverage," Machine Learning, 2021, DOI: 10.48550/arXiv.2106.03207.
  8. J. Peters and S. Schaal, "Reinforcement learning of motor skills with policy gradients," Neural networks, vol. 21, no. 4, pp. 682-697, May, 2008, DOI: 10.1016/j.neunet.2008.02.003.
  9. M. S. Jazayeri and A. Jahangiri, "Utilizing b-spline curves and neural networks for vehicle trajectory prediction in an inverse reinforcement learning framework," Journal of Sensor and Actuator Networks, vol. 11, no. 1, Feb., 2022, DOI: 10.3390/jsan11010014.
  10. M. Hasanzade and E. Koyuncu, "A dynamically feasible fast replanning strategy with deep reinforcement learning," Journal of Intelligent & Robotic Systems, vol. 101, 2021, DOI: 10.1007/s10846-020-01274-1.
  11. E. Todorov, T. Erez, and Y. Tassa, "MuJoCo: A physics engine for model-based control," IEEE International Workshop on Intelligent Robots and Systems (IROS), Vilamoura-Algarve, Portugal, 2012, DOI: 10.1109/IROS.2012.6386109.
  12. pybind11 - Seamless operability between C++11 and Python, [Online], https://github.com/pybind/pybind11, Accessed: Nov. 1, 2022.
  13. Eigen is a C++ template library for linear algebra: matrices, vectors, numerical solvers, and related algorithms, [Online], https://eigen.tuxfamily.org, Accessed: Nov. 1, 2022.