Browse > Article
http://dx.doi.org/10.7746/jkros.2022.17.1.058

Learning Optimal Trajectory Generation for Low-Cost Redundant Manipulator using Deep Deterministic Policy Gradient(DDPG)  

Lee, Seunghyeon (Pusan National University)
Jin, Seongho (Pusan National University)
Hwang, Seonghyeon (Pusan National University)
Lee, Inho (Dept of Electronics Engineering Pusan National University)
Publication Information
The Journal of Korea Robotics Society / v.17, no.1, 2022 , pp. 58-67 More about this Journal
Abstract
In this paper, we propose an approach resolving inaccuracy of the low-cost redundant manipulator workspace with low encoder and low stiffness. When the manipulators are manufactured with low-cost encoders and low-cost links, the robots can run into workspace inaccuracy issues. Furthermore, trajectory generation based on conventional forward/inverse kinematics without taking into account inaccuracy issues will introduce the risk of end-effector fluctuations. Hence, we propose an optimization for the trajectory generation method based on the DDPG (Deep Deterministic Policy Gradient) algorithm for the low-cost redundant manipulators reaching the target position in Euclidean space. We designed the DDPG algorithm minimizing the distance along with the jacobian condition number. The training environment is selected with an error rate of randomly generated joint spaces in a simulator that implemented real-world physics, the test environment is a real robotic experiment and demonstrated our approach.
Keywords
Trajectory Optimization; Low-Cost Manipulator; DDPG;
Citations & Related Records
Times Cited By KSCI : 5  (Citation Analysis)
연도 인용수 순위
1 T. P. Lillicrap, J. J. Hunt, A. Pritzel, N. Heess, T. Erez, Y. Tassa, D. Silver, and D. Wierstra, "Continuous control with deep reinforcement learning," arXiv:1509.02971 [cs.LG], 2015, [Online], https://arxiv.org/abs/1509.02971.
2 J. Lee, K. Kim, Y. Kim, and J. Lee, "Singularity Avoidance Path Planning on Cooperative Task of Dual Manipulator Using DDPG Algorithm," The Journal of Korea Robotics Society, vol. 16, no. 2, pp. 137-146, Jun., 2021, DOI: 10.7746/jkros.2021.16.2.137.   DOI
3 M. Tokic, "Adaptive ε-Greedy Exploration in Reinforcement Learning Based on Value Differences," Annual Conference on Artificial Intelligence, pp. 203-210, 2010, DOI: 10.1007/978-3-642-16111-7_23.   DOI
4 V. Mnih, K. Kavukcuoglu, D. Silver, A. Graves, I. Antonoglou, D. Wierstra, and M. Riedmiller, "Playing Atari with Deep Reinforcement Learning," arXiv:1312.5602 [cs.LG], 2013, [Online], https://arxiv.org/abs/1312.5602.
5 D. Silver, G. Lever, N. Heess, T. Degris, D. Wierstra, and M. Riedmiller, "Deterministic Policy Gradient Algorithms," 31st International Conference on Machine Learning, vol. 32, no.1 pp. 387-395, 2014, [Online], http://proceedings.mlr.press/v32/silver14.html.
6 C. C. White III and D. J. White, "Markov decision processes," European Journal of Operational Research, vol. 39, no. 1, pp. 1-16, 1989, DOI: 10.1016/0377-2217(89)90348-2.   DOI
7 G. Farneback, "Two-frame motion estimation based on polynomial expansion," Scandinavian conference on Image analysis, pp. 363-370. 2003, DOI: 10.1007/3-540-45103-X_50.   DOI
8 F. Dellaert, D. Fox, W. Burgard, and S. Thrun, "Monte Carlo localization for mobile robots," 1999 IEEE International Conference on Robotics and Automation, vol. 2, pp. 1322-1328, 1999, DOI: 10.1109/ROBOT.1999.772544.   DOI
9 M. A. Ali, H. Andy Park and C. S. G. Lee, "Closed-form inverse kinematic joint solution for humanoid robots," 2010 IEEE/RSJ International Conference on Intelligent Robots and Systems, Taipei, Taiwan, 2010, DOI:10.1109/IROS.2010.5649842.   DOI
10 D. L. Pieper, "The Kinematics of Manipulators Under Computer Control," Stanford University, 1968, [Online], https://www.proquest.com.
11 D. I. Park, C. H. Park, D. H. Kim, and J. H. Kyung, "Analysis and Design of the Dual Arm Manipulator for Rescue Robot," The Journal of Korea Robotics Society, vol. 11, no. 4, pp. 235-241, Dec., 2016, DOI:10.7746/jkros.2016.11.4.235.   DOI
12 Z. Bingul, H. M. Ertunc, and C. Oysu, "Applying Neural Network to Inverse Kinematic Problem for 6R Robot Manipulator with Offset Wrist," Adaptive and Natural Computing Algorithms, pp. 112-115, 2005, DOI: 10.1007/3-211-27389-1_27.   DOI
13 D. Jang and S. Yoo, "Integrated System of Mobile Manipulator with Speech Recognition and Deep Learning-based Object Detection," The Journal of Korea Robotics Society, vol. 16, no. 3, pp. 270-275, Sept., 2021, DOI:10.7746/jkros.2021.16.3.270.   DOI
14 A. Poroykov, P. Kalugin, S. Shitov, and I. Lapitskaya, "Modeling ArUco Markers Images for Accuracy Analysis of Their 3D Pose Estimation," 30th International Conference on Computer Graphics and Machine Vision, vol. 2, 2020, DOI: 10.51130/graphicon-2020-2-4-14.   DOI
15 J. P. Merlet, "Jacobian, Manipulability, Condition Number, and Accuracy of Parallel Robots," Journal of Mechanical Design, vol. 128, no. 1, pp. 199-206, 2005, DOI: 10.1115/1.2121740.   DOI
16 M. Duguleana, F. G. Barbuceanu, A. Teirelbar, and G. Mogan, "Obstacle avoidance of redundant manipulators using neural networks based reinforcement learning," Robotics and Computer-Integrated Manufacturing, vol. 28, no. 2, pp. 132-146, Apr., 2012, DOI:10.1016/j.rcim.2011.07.004.   DOI
17 E. Prianto, M. S. Kim, J.-H. Park, J.-H. Bae, and J.-S. Kim, "Path Planning for Multi-Arm Manipulators Using Deep Reinforcement Learning: Soft Actor-Critic with Hindsight Experience Replay," Sensors 2020, vol. 20, no. 20, 2020, DOI: 10.3390/s20205911.   DOI
18 J.-Y. Moon, J.-H. Moon, and S.-H. Bae, "Control f or Manipulator of an Underwater Robot Using Meta Reinforcement Learning," The Journal of the Korea institute of electronic communication sciences, vol. 16, no. 1, pp. 95-100, 2021, DOI: 10.13067/JKIECS.2021.16.1.95.   DOI
19 J. D. Norwood, "A neural network approach to the redundant robot inverse kinematic problem in the presence of obstacles," PhD thesis. Rice University, 1991, [Online], https://www.proquest.com.
20 M. M. Fateh and H. Farhangfard, "On the Transf orming of Control Space by Manipulator Jacobian," Institute of Control, Robotics and Systems, vol. 6, no.1, pp. 101-108, 2008, [Online], https://www.koreascience.or.kr/article/JAKO200809906440883.pdf.
21 Y.-H. Kim, H. Kang, and H.-T. Jeon, "Planning a minimum time path for robot manipulator using Genetic Algorithm," The Institute of Electronics and Information Engineers, pp. 698-702, 1992, [Online], https://www.koreascience.or.kr/article/CFKO199211919700824.pdf.
22 A. Iriondo, E. Lazkano, L. Susperregi, J. Urain, A. Fernandez, and J. Molina, "Pick and Place Operations in Logistics Using a Mobile Manipulator Controlled with Deep Reinforcement Learning," Applied Sciences, vol. 9, no. 2, 2019, DOI:10.3390/app9020348.   DOI