DOI QR코드

DOI QR Code

A Study on the Implementation of Crawling Robot using Q-Learning

  • Hyunki KIM (Shinnam Information & Communication) ;
  • Kyung-A KIM (Dept. of Medical Artificial Intelligence, Eulji University) ;
  • Myung-Ae CHUNG (Dept. of BigData Medical Convergence, Eulji University) ;
  • Min-Soo KANG (Dept. of BigData Medical Convergence, Eulji University)
  • Received : 2022.11.04
  • Accepted : 2023.03.17
  • Published : 2023.12.30

Abstract

Machine learning is comprised of supervised learning, unsupervised learning and reinforcement learning as the type of data and processing mechanism. In this paper, as input and output are unclear and it is difficult to apply the concrete modeling mathematically, reinforcement learning method are applied for crawling robot in this paper. Especially, Q-Learning is the most effective learning technique in model free reinforcement learning. This paper presents a method to implement a crawling robot that is operated by finding the most optimal crawling method through trial and error in a dynamic environment using a Q-learning algorithm. The goal is to perform reinforcement learning to find the optimal two motor angle for the best performance, and finally to maintain the most mature and stable motion about EV3 Crawling robot. In this paper, for the production of the crawling robot, it was produced using Lego Mindstorms with two motors, an ultrasonic sensor, a brick and switches, and EV3 Classroom SW are used for this implementation. By repeating 3 times learning, total 60 data are acquired, and two motor angles vs. crawling distance graph are plotted for the more understanding. Applying the Q-learning reinforcement learning algorithm, it was confirmed that the crawling robot found the optimal motor angle and operated with trained learning, and learn to know the direction for the future research.

Keywords

Acknowledgement

This work was supported by the research grant of the KODISA Scholarship Foundation in 2023.

References

  1. Angel Martinez-Tenor, Juan-Antonio Fernandez-Madrigal, Ana Cruz-Martin. LEGO Mindstorms nxt and q-learning: a teaching approach for robotics in engineering. 7th international Conference of Education, Research and Innovation (ICERI) 2014.
  2. Ke X., Wu, F., Zhao, J. (2015). Simplified Online Q-Learning for LEGO EV3 Robot. IEEE International Conference on Control System, Computing and Engineering, 27-29 November 2015, Penang Malaysia. https://doi.org/https://doi.org/10.1109/ICCSCE.2015.7482161
  3. Kim, B. C., Kim, S.K., Yoon B. J. (2002). Online Reinforcement Learning to Search the Shortest Path in Maze Environments. Journal of Korea Information Processing Society, 9-B, 155-162. https://doi.org/10.3745/KIPSTB.2002.9B.2.155
  4. LEGO Mindstorms. http://mindstorms.lego.com/. Retrieved Jul. 2015
  5. LEGO, Mindstorms. https://www.lego.com/en-gb/themes/mindstorms/. (2022)
  6. Oh, I. S. (2017), Machine Learning. Published by Hanbit Academy, Inc. Printed in Korea. ISBN : 979-11-5664-158-2
  7. Park, S., Lee, S., Kim, H. (2021), Study on Sensing-based Robot Control System in Constrained Environment, Spring Conference Korean Institute of Next Generation Computing. 102-103, 2021
  8. Watkins C.J.C.H. (1992), Q-Learning. Technical Note. Machine Learning, 8, 279-292. Kluwer Academic Publishers, Boston https://doi.org/10.1023/A:1022676722315