The Journal of Korea Robotics Society (로봇학회논문지)

Korea Robotics Society (한국로봇학회)
권호 표지
DOI QR코드

DOI QR Code

Singularity Avoidance Path Planning on Cooperative Task of Dual Manipulator Using DDPG Algorithm

DDPG 알고리즘을 이용한 양팔 매니퓰레이터의 협동작업 경로상의 특이점 회피 경로 계획

  • Received : 2020.12.21
  • Accepted : 2021.02.26
  • Published : 2021.05.31
Download PDF
⟨ Previous Next ⟩

Abstract

When controlling manipulator, degree of freedom is lost in singularity so specific joint velocity does not propagate to the end effector. In addition, control problem occurs because jacobian inverse matrix can not be calculated. To avoid singularity, we apply Deep Deterministic Policy Gradient(DDPG), algorithm of reinforcement learning that rewards behavior according to actions then determines high-reward actions in simulation. DDPG uses off-policy that uses 𝝐-greedy policy for selecting action of current time step and greed policy for the next step. In the simulation, learning is given by negative reward when moving near singulairty, and positive reward when moving away from the singularity and moving to target point. The reward equation consists of distance to target point and singularity, manipulability, and arrival flag. Dual arm manipulators hold long rod at the same time and conduct experiments to avoid singularity by simulated path. In the learning process, if object to be avoided is set as a space rather than point, it is expected that avoidance of obstacles will be possible in future research.

Keywords

Acknowledgement

This research is based upon work supported by the Ministry of Trade, Industry & Energy (MOTIE, Korea) under Industrial Technology Innovation Program. No.10073147

References

  1. S. Y. Chung, H. S. Kang, and M. J. Hwang, "Singularity Avoidance for Linear Motion of 6-DOF Manipulator," Journal of Institute of Control Robotics and Systems, vol. 24, no. 11, pp. 1025-1032, 2018, DOI: 10.5302/J.ICROS.2018.18.0164.
  2. D.-J. Park, D.-E. Kim, J.-H. Park, and J.-M. Lee, "Singularity and Collision Avoidance Path Planning based upon Artificial Potential Field and Manipulability Measure," Journal of the Institute of Electronics and Information Engineers, vol. 55, no. 5, pp. 89-96, 2018, DOI: 10.5573/ieie.2018.55.5.89.
  3. S. Kim and J. Park, "Singularity Avoidance Algorithms for Controlling Robot Manipulator: A Comparative Study," The Journal of Korea Robotics Society, vol. 12, no. 1, pp. 42-54, Jan., 2019, DOI: 10.7746/jkros.2017.12.1.042.
  4. T. Yoshikawa, "Analysys and control of robot manipulators with redundancy. Robotic Research," The First International Symposium, New Hampshire, USA, pp. 735-747, 1984, [Online], https://ci.nii.ac.jp/naid/80014489288.
  5. H. A. Park, M. A. Ali, and C. S. G. Lee, "Closed-Form Inverse Kinematic Position Solution for Humanoid Robots," International Journal of Humanoid Robotics, vol. 9, no. 3, 2012, DOI: 10.1142/S0219843612500223.
  6. P. P. R. Filho, S. P. P. da Silva, V. N. Praxedes, J. Hemanth, V. Hugo C. de Albuquerque, "Control of singularity trajectory tracking for robotic manipulator by genetic algorithms," Journal of Computational Science, vol. 30, pp. 55-64, 2019, DOI: 10.1016/j.jocs.2018.11.006.
  7. Y. Nakamura and H. Hanafusa, "Inverse kinematic solutions with singularity robustness for robot manipulator control," ASME Journal of Dynamic Systems, Measurement and Control, vol. 108, pp. 163-171, 1986, DOI: 10.1115/1.3143764.
  8. C. W. Wampler, "Manipulator Inverse Kinematic Solutions Based on Vector Formulations and Damped Least-Squares Methods," IEEE Transactions on Systems, Man, and Cybernetics, vol. 16, no. 1 pp. 93-101, 1986, DOI: 10.1109/TSMC.1986.289285.
  9. D. E. Whitney, "The mathematics ofcoordinated control of prosthetic arms andmanipulators," Journal of Dynamic Systems, Measurement, and Control, vol. 94, no. 4, pp. 303-309, 1972, DOI: 10.1115/1.3426611.
  10. W. Wang, M. Zhu, X. Wang, S. He, J. He, and Z. Xu, "An Improved Artificial Potential Field Method of Trajectory Planning and Obstacle Avoidance for Redundant Manipulators," International Journal of Advanced Robotic Systems, vol. 15, no. 5, pp. 1-13, 2018, DOI: 10.1177/1729881418799562.
  11. S. R. Buss, "Introduction to inverse kinematics with jacobian transpose, pseudoinverse and damped least squares methods," IEEE Journal of Robotics and Automation, vol. 17, pp. 1-19, 2004, [Online], http://graphics.cs.cmu.edu/nsp/course/15464-s17/lectures/iksurvey.pdf.
  12. Alex Owen-Hill, Robotiq Inc., 3 types of robot singularities and how to avoid them, [Online], https://robohub.org/3-types-of-robot-singularities-and-how-to-avoid-them, Accessed: Nov. 29, 2020.
  13. S. Phaniteja, P. Dewangan, P. Guhan, A. Sarkar, and K. M. Krishna, "A deep reinforcement learning approach for dynamically stable inverse kinematics of humanoid robots," 2017 IEEE International Conference on Robotics and Biomimetics (ROBIO), Macau, China, pp. 1818-1823, 2017, DOI: 10.1109/ROBIO.2017.8324682.
  14. Y.-G. Kang and C.-S. Lee, "Deep Reinforcement Learning of Ball Throwing Robot's Policy Prediction," The Journal of Korea Robotics Society, vol. 15, no. 4 pp. 398-403, Dec., 2020, DOI: 10.7746/jkros.2020.15.4.398.
  15. S. Kim, S. A. Kim, R. de Lima, and J. Choi, "Implementation of End-to-End Training of Deep Visuomotor Policies for Manipulation of a Robotic Arm of Baxter Research Robot," The Journal of Korea Robotics Society, vol. 14, no. 1 pp. 40-49, 2019, DOI: 10.7746/jkros.2019.14.1.040.
  16. T. Kim, Y. Park, J. B. Kim, Y. Park, and I. H. Suh, "Development of an Actor-Critic Deep Reinforcement Learning Platform for Robotic Grasping in Real World," The Journal of Korea Robotics Society, vol. 15, no. 2, pp. 197-204, Jun., 2020, DOI: 10.7746/jkros.2020.15.2.197.
  17. M. Duguleana, F. G. Barbuceanu, A. Teirelbar, and G. Mogan, "Obstacle avoidance of redundant manipulators using neural networks based reinforcement learning," Robotics and ComputerIntegrated Manufacturing, vol. 28, no. 2, pp. 132-146, 2012, DOI: 10.1016/j.rcim.2011.07.004.
  18. D. L. Pieper, "The kinematics of manipulators under computer control," Ph.D. dissertation, Stanford University, USA, 1968, [Online], https://www.saildart.org/AIM/072/pdf.
  19. L. Rozo, N. Jaquier, S. Calinon and D. G. Caldwell, "Learning manipulability ellipsoids for task compatibility in robot manipulation," 2017 IEEE/RSJ International Conference on In-telligent Robots and Systems (IROS), Vancouver, BC, Canada, pp. 3183-3189, 2017 , DOI: 10.1109/IROS.2017.8206150.