The Transactions of the Korean Institute of Electrical Engineers D (대한전기학회논문지:시스템및제어부문D)

The Korean Institute of Electrical Engineers (대한전기학회)
권호 표지

Design of Reinforcement Learning Controller with Self-Organizing Map

자기 조직화 맵을 이용한 강화학습 제어기 설계

  • 이재강 (강원대학교 제어계측공학과) ;
  • 김일환 (강원대학교 전기전자정보통신공학부)
  • Published : 2004.05.01
Download PDF
⟨ Previous Next ⟩

Abstract

This paper considers reinforcement learning control with the self-organizing map. Reinforcement learning uses the observable states of objective system and signals from interaction of the system and environment as input data. For fast learning in neural network training, it is necessary to reduce learning data. In this paper, we use the self-organizing map to partition the observable states. Partitioning states reduces the number of learning data which is used for training neural networks. And neural dynamic programming design method is used for the controller. For evaluating the designed reinforcement learning controller, an inverted pendulum on the cart system is simulated. The designed controller is composed of serial connection of self-organizing map and two Multi-layer Feed-Forward Neural Networks.

Keywords

References

  1. Richard S. Sutton, and Andrew G. Barto, 'Reinforcement Learning : An Introduction,' MIT Press, Cmabrige, MA, 1998
  2. Charles W. Anderson, 'Strategy Learning with Multilayer Connectionist Representations,' Proceedings of the 4th International Workshop on Machine Learning, pp. 103-114, 1987
  3. Charles W. Anderson, 'Learning to Control an Inverted Pendulum Using Neural Network,' IEEE Control Systems Magazine, Vol. 9, No. 3, pp. 31-37. 1989 https://doi.org/10.1109/37.24809
  4. Andrew G. Barto, Richard S. Sutton, Charles W. Anderson, 'Neuronlike Adaptive Elements That Can Solve Difficult Learning Control Problems,' IEEE Transactions on Systems, Man, and Cybernetics, Vol. SMC-13, No. 5, 1983
  5. J. S. Albus, 'A New Approach to Manipulator control: The Cerebellar Model Articulation Controller(CMAC),' Journal of Dynamics Systems, Measurement, and Control, pp. 220-227, 1975
  6. Dean F. Hougen, Maria Gini, and James Slagle, 'Partitioning input space for reinforcement learning for control,' Proceedings of the IEEE International Conference on Roborics and Autonation, pp. 1917-1922, April, 1996
  7. Andrew James Smith, 'Applications of the self-organizing map to reinforcement learning,' In Neural Network (Special Issue), 15 pp. 1107-1124, 2002 https://doi.org/10.1016/S0893-6080(02)00083-7
  8. T. Kohonen, 'Self organizing maps,' Berlin: Springer
  9. P. Werbos, 'Advanced forecasting methods for global crisis warning and models of intelligence,' General System Yearbook, Vol. 22, pp. 25-38, 1977
  10. Richard S. Sutton, 'Learning to predict by the methods of temporal difference,' Machine Learning, Vol. 3, pp. 9-44, 1988
  11. Jennie Si, and Yu-Tsung Wang, 'On-Line Learning Control by Association and Reinforcement,' IEEE Transactions on Neural Networks, Vol. 12, No. 2, pp.264-276, 2001 https://doi.org/10.1109/72.914523