Journal of Digital Contents Society (디지털콘텐츠학회 논문지)

Digital Contents Society (한국디지털콘텐츠학회)
권호 표지
DOI QR코드

DOI QR Code

Multi Behavior Learning of Lamp Robot based on Q-learning

강화학습 Q-learning 기반 복수 행위 학습 램프 로봇

  • Kwon, Ki-Hyeon (Department of Information & Communication Engineering, Kangwon National University) ;
  • Lee, Hyung-Bong (Department of Computer Science & Engineering, Gangneung-Wonju National University)
  • 권기현 (강원대학교 정보통신공학과) ;
  • 이형봉 (강릉원주대학교 컴퓨터공학과)
  • Received : 2017.12.07
  • Accepted : 2018.01.29
  • Published : 2018.01.31
Download PDF
⟨ Previous Next ⟩

Abstract

The Q-learning algorithm based on reinforcement learning is useful for learning the goal for one behavior at a time, using a combination of discrete states and actions. In order to learn multiple actions, applying a behavior-based architecture and using an appropriate behavior adjustment method can make a robot perform fast and reliable actions. Q-learning is a popular reinforcement learning method, and is used much for robot learning for its characteristics which are simple, convergent and little affected by the training environment (off-policy). In this paper, Q-learning algorithm is applied to a lamp robot to learn multiple behaviors (human recognition, desk object recognition). As the learning rate of Q-learning may affect the performance of the robot at the learning stage of multiple behaviors, we present the optimal multiple behaviors learning model by changing learning rate.

강화학습기반 Q-learning 알고리즘은 이산적인 상태와 액션의 조합을 사용하여, 한 번에 하나의 행위에 대한 목표를 학습하는데 유용하다. 여러 액션을 학습하기 위해서는 행위 기반 아키텍처를 적용하고 적절한 행위 조절 방법을 사용하면 로봇으로 하여금 빠르고 신뢰성 있는 액션을 가능하게 할 수 있다. Q-learning은 인기 있는 강화학습 방법으로 단순하고, 수렴성이 있고 사전 훈련 환경에 영향을 덜 받는 특성(off-policy)으로 인해 로봇 학습에 많이 사용되고 있다. 본 논문에서는 Q-learning 알고리즘을 램프 로봇에 적용하여 복수 행위(사람인식, 책상의 물체 인식)를 학습시키는데 사용하였다. Q-learning의 학습속도(learning rate)는 복수 행위 학습 단계의 로봇 성능에 영향을 줄 수 있으므로 학습속도 변경을 통해 최적의 복수 행위 학습 모델을 제시한다.

Keywords

References

  1. R. Brooks, "A Robust Layered Control System For a Mobile Robot," IEEE Journal of Robotics and Automation, Vol. 2, No. 1, pp. 14-23, 1986. https://doi.org/10.1109/JRA.1986.1087032
  2. R. Hafner, and M. Riedmiller, "Reinforcement Learning on a Omnidirectional Mobile Robot," in Proceeding of 2003 IEEE/RSJ International Conference on Intelligent Robots and Systems, Vol. 1, Las Vegas, pp. 418-423, 2003.
  3. R.S. Sutton, and A.G. Barto, "Reinforcement Learning, an Introduction," MIT Press, Massachusets, 1998.
  4. H. Wicaksono, Prihastono, K. Anam, S. Kuswadi, R. Effendie, A. Jazidie, I. A. Sulistijono, M. Sampei, "Modified Fuzzy Behavior Coordination for Autonomous Mobile Robot Navigation System," in Proceeding of ICCAS-SICE, 2009.
  5. C. Watkins and P. Dayan, "Q-learning, Technical Note," Machine Learning, Vol 8, pp. 279-292, 1992.
  6. Y. G. Seo, "LoRa Network based Parking Dispatching System : Queuing Theory and Q-learning Approach," The Journal of Digital Contents Society, Vol. 18, No. 7, pp. 1443-1450, June 2017. https://doi.org/10.9728/DCS.2017.18.7.1443
  7. K. Anam, S. Kuswadi, "Behavior Based Control and Fuzzy Q-Learning For Autonomous Mobile Robot Navigation," in Proceeding of The 4th International Conference on Information & Communication Technology and Systems (ICTS), 2008.
  8. S. M. Rho, "LoRa Network based Parking Dispatching System : Queuing Theory and Q-learning Approach," The Journal of Digital Contents Society, Vol. 18, No. 7, pp. 1443-1450, June 2017. https://doi.org/10.9728/DCS.2017.18.7.1443
  9. M.C. Perez, A Proposal of Behavior Based Control Architecture with Reinforcement Learning for an Autonomous Underwater Robot, Ph.D. Dissertation, University of Girona, Girona, 2003.
  10. L. Khriji, F. Touati, K. Benhmed, A.A. Yahmedi, "Q-Learning Based Mobile robot behaviors Coordination," in Proceeding of International Renewable Energy Congress (IREC), 2010.
  11. C. J. C. H.Watkins, Learning from delayed rewards, Ph.D. dissertation, Dept. Psychol., Univ. Cambridge, Cambridge, U.K., 1989.
  12. H.Wicaksono, "Q Learning Behavior on Autonomous Navigation of Physical Robot," The 8th International Conference on Ubiquitous Robots and Ambient Intelligence (URAI 2011), in Songdo Convention, Incheon, Korea, Nov. 23-26, 2011.
  13. C. F. Touzet, "Q-learning for robot,'' in The Handbook of Brain Theory and Neural Networks, M. A. Arbib, Ed. Cambridge, MA, USA: MIT Press, pp. 934-937, 2003.
  14. J.L LIN, K.S. HWANG, W.C. JIANG, and Y.J. CHEN, "Gait Balance and Acceleration of a Biped Robot Based on Q-Learning," IEEE Access, Vol. 4, pp. 2439-2449, 2016. https://doi.org/10.1109/ACCESS.2016.2570255

Cited by

  1. 강화학습을 기반으로 하는 열사용자 기계실 설비의 열효율 향상에 대한 연구 vol.27, pp.2, 2018, https://doi.org/10.5855/energy.2018.27.2.026