Journal of IKEEE (전기전자학회논문지)

Institute of Korean Electrical and Electronics Engineers (한국전기전자학회)
권호 표지
DOI QR코드

DOI QR Code

Map-Based Obstacle Avoidance Algorithm for Mobile Robot Using Deep Reinforcement Learning

심층 강화학습을 이용한 모바일 로봇의 맵 기반 장애물 회피 알고리즘

  • Sunwoo, Yung-Min (Dept. of Smart Robot Convergence and Application Engineering, Pukyong National University) ;
  • Lee, Won-Chang (Dept. of Electronic Engineering, Pukyong National University)
  • Received : 2021.05.14
  • Accepted : 2021.06.25
  • Published : 2021.06.30
Download PDF
⟨ Previous Next ⟩

Abstract

Deep reinforcement learning is an artificial intelligence algorithm that enables learners to select optimal behavior based on raw and, high-dimensional input data. A lot of research using this is being conducted to create an optimal movement path of a mobile robot in an environment in which obstacles exist. In this paper, we selected the Dueling Double DQN (D3QN) algorithm that uses the prioritized experience replay to create the moving path of mobile robot from the image of the complex surrounding environment. The virtual environment is implemented using Webots, a robot simulator, and through simulation, it is confirmed that the mobile robot grasped the position of the obstacle in real time and avoided it to reach the destination.

심층 강화학습은 학습자가 가공되지 않은 고차원의 입력 데이터를 기반으로 최적의 행동을 선택할 수 있게 하는 인공지능 알고리즘이며, 이를 이용하여 장애물들이 존재하는 환경에서 모바일 로봇의 최적 이동 경로를 생성하는 연구가 많이 진행되었다. 본 논문에서는 복잡한 주변 환경의 이미지로부터 모바일 로봇의 이동 경로를 생성하기 위하여 우선 순위 경험 재사용(Prioritized Experience Replay)을 사용하는 Dueling Double DQN(D3QN) 알고리즘을 선택하였다. 가상의 환경은 로봇 시뮬레이터인 Webots를 사용하여 구현하였고, 시뮬레이션을 통해 모바일 로봇이 실시간으로 장애물의 위치를 파악하고 회피하여 목표 지점에 도달하는 것을 확인하였다.

Keywords

Acknowledgement

This work was supported by a Research Grant of Pukyong National University(2021)

References

  1. K. Zhu and T. Zhang, "Deep reinforcement learning based mobile robot navigation: A review," in Tsinghua Science and Technology, vol.26, no.5, pp.674-691, 2021. DOI: 10.26599/TST.2021.9010012.
  2. X. Xue, Z. Li, D. Zhang and Y. Yan, "A Deep Reinforcement Learning Method for Mobile Robot Collision Avoidance based on Double DQN," 2019 IEEE 28th International Symposium on Industrial Electronics (ISIE), pp.2131-2136, 2019. DOI: 10.1109/ISIE.2019.8781522.
  3. Kober, Jens, J. Andrew Bagnell, and Jan Peters. "Reinforcement learning in robotics: A survey," The International Journal of Robotics Research, Vol.32, No.11, pp.1238-1274, 2013. DOI: 10.1177/0278364913495721
  4. Otte, Michael W. "A survey of machine learning approaches to robotic path-planning," University of Colorado at Boulder, Boulder 2015. DOI: 10.1109/RISE.2017.8378228
  5. S. Zhou, X. Liu, Y. Xu and J. Guo, "A Deep Q-network (DQN) Based Path Planning Method for Mobile Robots," 2018 IEEE International Conference on Information and Automation (ICIA), pp.366-371, 2018. DOI: 10.1109/ICInfA.2018.8812452.
  6. Mnih, Volodymyr, et al. "Playing atari with deep reinforcement learning," arXiv preprint arXiv:1312.5602, 2013.
  7. J. Xin, H. Zhao, D. Liu and M. Li, "Application of deep reinforcement learning in mobile robot path planning," 2017 Chinese Automation Congress (CAC), pp.7112-7116, 2017. DOI: 10.1109/CAC.2017.8244061.
  8. X. Ruan, D. Ren, X. Zhu and J. Huang, "Mobile Robot Navigation based on Deep Reinforcement Learning," 2019 Chinese Control And Decision Conference (CCDC), pp.6174-6178, 2019. DOI: 10.1109/CCDC.2019.8832393.
  9. H. Sasaki, T. Horiuchi and S. Kato, "A study on vision-based mobile robot learning by deep Q-network," 2017 56th Annual Conference of the Society of Instrument and Control Engineers of Japan (SICE), pp.799-804, 2017. DOI: 10.23919/SICE.2017.8105597.
  10. Van Hasselt, Hado, Arthur Guez, and David Silver. "Deep reinforcement learning with double q-learning," Proceedings of the AAAI Conference on Artificial Intelligence. Vol.30, No.1, 2016.
  11. Wang, Ziyu, et al. "Dueling network architectures for deep reinforcement learning." International conference on machine learning. PMLR, 2016.
  12. Schaul, Tom, et al. "Prioritized experience replay." arXiv preprint arXiv:1511.05952, 2015.
  13. "Webots: robot simulator" https://cyberbotics.com/#features
  14. Kirtas, M., et al. "Deepbots: A Webots-Based Deep Reinforcement Learning Framework for Robotics." IFIP International Conference on Artificial Intelligence Applications and Innovations. Springer, Cham, 2020.