The Journal of the Institute of Internet, Broadcasting and Communication (한국인터넷방송통신학회논문지)

The Institute of Internet, Broadcasting and Communication (한국인터넷방송통신학회)
권호 표지

On-line Reinforcement Learning for Cart-pole Balancing Problem

카트-폴 균형 문제를 위한 실시간 강화 학습

  • 김병천 (한경대학교 웹정보공학과) ;
  • 이창훈 (한경대학교 컴퓨터공학과)
  • Received : 2010.06.13
  • Accepted : 2010.08.13
  • Published : 2010.08.31
Download PDF
⟨ Previous Next ⟩

Abstract

The cart-pole balancing problem is a pseudo-standard benchmark problem from the field of control methods including genetic algorithms, artificial neural networks, and reinforcement learning. In this paper, we propose a novel approach by using online reinforcement learning(OREL) to solve this cart-pole balancing problem. The objective is to analyze the learning method of the OREL learning system in the cart-pole balancing problem. Through experiment, we can see that approximate faster the optimal value-function than Q-learning.

Cart-pole 균형 문제는 유전자 알고리즘, 인공신경망, 강화학습 등을 이용한 제어 전략 분야의 표준 문제이다. 본 논문에서는 cart-pole 균형문제를 해결하기 위해 실시간 강화 학습을 이용한 접근 방법을 제안하였다. 본 논문의 목적은 cart-pole 균형 문제에서 OREL 학습 시스템의 학습 방법을 분석하는데 있다. 실험을 통해, 본 논문에서 제안한 OREL 학습 방법은 Q-학습보다 최적 값 함수에 더 빠르게 접근함을 알 수 있었다.

Keywords

References

  1. 김병천, 윤병주, "복수전략학습", 정보과학회지, 13권, 5호, pp45-52, 1995.
  2. M.L.Minsky Theory of Neural-Analog Reinforcement Systems and Application to the Brain-Model Problem, Ph.D. Thesis, Princeton University, Princeton, 1954.
  3. A. G. Barto, D. A. White and D. A. Sofge, "Reinforcement Learning and adaptive critic model", Handbook of Intelligent Control, pp. 469-491,1992.
  4. C. W. Anderson, "Learning to control an inverted pendulum using neural networks", IEEE Control Systems Magazine, pp.31-37, 1989.
  5. O. Pinngern and T. H. Nguyen, "International Symposium on Electrical & Electronics Engineering", HCM City, Vietnam, 2007.
  6. As'ad Salkham, Raymond Cunningham, Anurag Garg, and Vinny Cahill, "A Collaborative Reinforcement Learning Approach to Urban Traffic Control", IEEE/WIC/ACM International Conference, Vol. 2 (2008), pp. 560-566.
  7. T. Walczak and P. Cichosz. "A distributed learning control system for elevator groups", Artificial Intelligence and Soft Computing (ICAISC-06), volume 4029 of Lecture Notes in Computer Science, pp.1223–232. Springer, 2006.
  8. K Conn and R A Peters, ""Reinforcement Learning with a Supervisor for a Mobile Robot in a Real world Environment", Computational Intelligence in Robotics and Automation, pp. 73-78, 2007
  9. G. Cybenko, R. Gray, and K. Moizumi, "Q-learning : A Tutorial and Extensions", Mathematics of Artificial Neural Networks, Oxford University, July, 1995.