International Journal of Control, Automation, and Systems

Institute of Control, Robotics and Systems (제어로봇시스템학회)
권호 표지

Multi-Dimensional Reinforcement Learning Using a Vector Q-Net - Application to Mobile Robots

  • Published : 2003.03.01
Download PDF
⟨ Previous Next ⟩

Abstract

Reinforcement learning is considered as an important tool for robotic learning in unknown/uncertain environments. In this paper, we propose an evaluation function expressed in a vector form to realize multi-dimensional reinforcement learning. The novel feature of the proposed method is that learning one behavior induces parallel learning of other behaviors though the objectives of each behavior are different. In brief, all behaviors watch other behaviors from a critical point of view. Therefore, in the proposed method, there is cross-criticism and parallel learning that make the multi-dimensional learning process more efficient. By ap-plying the proposed learning method, we carried out multi-dimensional evaluation (reward) and multi-dimensional learning simultaneously in one trial. A special neural network (Q-net), in which the weights and the output are represented by vectors, is proposed to realize a critic net-work for Q-learning. The proposed learning method is applied for behavior planning of mobile robots.

Keywords

References

  1. Science v.275 A neural substrate of prediction and reward W. Schultz;P. Dayan;P.R. Montague https://doi.org/10.1126/science.275.5306.1593
  2. Aartificial Intelligence Memo 887 Estimation of internal parameters of rigid body links of manipulators C. H. An;C. G. Atkeson;J. M. Hollerbach
  3. Reinforcement Learning R. S. Sutton;A. G. Barto
  4. Ph. D. Dissertation, Cambridge University Learning from delayed rewards C. J. C. H Watkins
  5. Machine Learning v.23 Purposive behavior acquisition for a real robot by vision-based reinforcement learning M. Asada;S. Noda;S. Tawaratumida;K. Hosoda
  6. Proc. of IEEE International Conference on Robotics and Automation Learning architecture for real robotic systems Extension of connectionist q-learning for continuous robot control domain F. Saito;T. Fukuda
  7. Proc. of 9th National Conf. on Artificial Intelligence Automatic programming of behavior-based robots using reinforcement learning S. Mahadevan;J. Connell
  8. Ph. D. Dissertation, Cambridge Mellon University Reinforcement learning for robots using neural networks L. J. Lin
  9. Ph. D. Dissertation, MIT Interaction and intelligeng behavior M. J. Mataric
  10. IEEE Control Systems Magazine v.14 no.1 Acquiring robot skills via reinforcement learning V. Gullapalli;J. A. Franklin;H. Benbrahim https://doi.org/10.1109/37.257890
  11. IEEE Trans. on Systems, Man, and Cybernetics v.25 A sensor-based navigation for a mobile robot using fuzzy logic and reinforcement learning H. K. Beom;H. S. Cho https://doi.org/10.1109/21.364859
  12. Machine Learning v.31 Module-based reinforcement learning: experiments with a real robot Z. Kalmar;C. Szepesvari;A. Lorincz https://doi.org/10.1023/A:1007440607681
  13. Proc. of IEEE International Conf. on Systems, Man, and Cybernetics Multiple reward criterion for cooperative behavior acquisition in a multiagent environment E. Uchibe;M. Asada