The Journal of Korea Robotics Society (로봇학회논문지)

Korea Robotics Society (한국로봇학회)
권호 표지
DOI QR코드

DOI QR Code

Variational Autoencoder-based Assembly Feature Extraction Network for Rapid Learning of Reinforcement Learning

강화학습의 신속한 학습을 위한 변이형 오토인코더 기반의 조립 특징 추출 네트워크

  • Jun-Wan Yun (Mechanical Engineering, Korea University) ;
  • Minwoo Na (Mechanical Engineering, Korea University) ;
  • Jae-Bok Song (Mechanical Engineering, Korea University)
  • Received : 2023.05.31
  • Accepted : 2023.07.17
  • Published : 2023.08.31
Download PDF
⟨ Previous Next ⟩

Abstract

Since robotic assembly in an unstructured environment is very difficult with existing control methods, studies using artificial intelligence such as reinforcement learning have been conducted. However, since long-time operation of a robot for learning in the real environment adversely affects the robot, so a method to shorten the learning time is needed. To this end, a method based on a pre-trained neural network was proposed in this study. This method showed a learning speed about 3 times than the existing methods, and the stability of reward during learning was also increased. Furthermore, it can generate a more optimal policy than not using a pre-trained neural network. Using the proposed reinforcement learning-based assembly trajectory generator, 100 attempts were made to assemble the power connector within a random error of 4.53 mm in width and 3.13 mm in length, resulting in 100 successes.

Keywords

Acknowledgement

This research was funded by the MOTIE under the Industrial Foundation Technology Development Program supervised by the KEIT (No. 20008613)

References

  1. S. R. Chhatpar and M. S. Branicky, "Search strategies for peg-in-hole assemblies with position uncertainty," IEEE International Workshop on Intelligent Robots and Systems (IROS), Maui, USA, pp. 1465-1470, 2001, DOI: 10.1109/IROS.2001.977187.
  2. L. Xie, H. Yu, Y. Zhao, H. Zhang, Z. Zhou, M. Wang, and R. Xiong, "Learning to Fill the Seam by Vision: Sub-millimeter Peg-in-hole on Unseen Shapes in Real World," IEEE International Conference on Robotics and Automation (ICRA), Philadelphia, USA, pp. 2982-2988, 2022, DOI: 10.1109/ICRA46639.2022.9812429.
  3. J. Jiang, L. Yao, Z. Huang, G. Yu, L. Wang, and Z. Bi, "The state of the art of search strategies in robotic assembly," Journal of Industrial Information Integration, vol 26, Mar., 2022, DOI: 10.1016/j.jii.2021.100259.
  4. G. Schoettler, A. Nair, J. Luo, S. Bahl, J. A. Ojea, E. Solowjow, and S. Levine, "Deep Reinforcement Learning for Industrial Insertion Tasks with Visual Inputs and Natural Rewards," IEEE International Workshop on Intelligent Robots and Systems (IROS), Las Vegas, USA, pp. 5548-5555, 2020, DOI: 10.1109/IROS45743.2020.9341714.
  5. X. Zhao, H. Zhao, P. Chen, and H. Ding, "Model accelerated reinforcement learning for high precision robotic assembly," International Journal of Intelligent Robotics and Applications, vol. 4, pp. 202-216, Jun., 2020, DOI: 10.1007/s41315-020-00138-z.
  6. Y.-G. Kim, M. Na, and J.-B. Song, "Reinforcement Learning-based Sim-to-Real Impedance Parameter Tuning for Robotic Assembly," International Conference on Control, Automation and Systems (ICCAS), Jeju, Republic of Korea, pp. 833-836, 2021, DOI: 10.23919/ICCAS52745.2021.9649923.
  7. J. Luo, O. Sushkov, R. Pevceviciute, W. Lian, C. Su, M. Vecerik, N. Ye, S. Schaal, and J. Scholz, "Robust Multi-Modal Policies for Industrial Assembly via Reinforcement Learning and Demonstrations: A Large-Scale Study," Robotics: Science and Systems, 2021, DOI: 10.15607/RSS.2021.XVII.088.
  8. N. Hogan, "Impedance control: An approach to manipulation: Part II-Implementation," ASME Journal of Dynamic Systems Measurement and Control, vol. 107, no. 1, pp. 8-16, Mar., 1985, DOI: 10.1115/1.3140713.
  9. T. Haarnoja, A. Zhou, P. Abbeal, and S. Levine, "Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actor," International Conference on Machine Learning (ICML), Stockholm, Sweden, 2018, DOI: 10.48550/arXiv.1801.01290.
  10. D. P. Kingma and M. Weling, "Auto-Encoding Variational Bayes," International Conference on Learning Representations (ICLR), Banff, Canada, 2014, DOI: 10.48550/arXiv.1312.6114.
  11. K. Cho, B. V. Merrienboer, C. Gulcehre, D. Bahdanau, F. Bougares, H. Schwenk, and Y. Bengio, "Learning phrase representations using RNN encoder-decoder for statistical machine translation," Conference on Empirical Methods in Natural Language Processing (EMNLP), Doha, Qatar, 2014, DOI: 10.48550/arXiv.1406.1078.