Browse > Article

Design of Reinforcement Learning Controller with Self-Organizing Map  

이재강 (강원대학교 제어계측공학과)
김일환 (강원대학교 전기전자정보통신공학부)
Publication Information
The Transactions of the Korean Institute of Electrical Engineers D / v.53, no.5, 2004 , pp. 353-360 More about this Journal
Abstract
This paper considers reinforcement learning control with the self-organizing map. Reinforcement learning uses the observable states of objective system and signals from interaction of the system and environment as input data. For fast learning in neural network training, it is necessary to reduce learning data. In this paper, we use the self-organizing map to partition the observable states. Partitioning states reduces the number of learning data which is used for training neural networks. And neural dynamic programming design method is used for the controller. For evaluating the designed reinforcement learning controller, an inverted pendulum on the cart system is simulated. The designed controller is composed of serial connection of self-organizing map and two Multi-layer Feed-Forward Neural Networks.
Keywords
Reinforcement Learning; Self-Organizing Map; Neural Dynamic Programming;
Citations & Related Records
연도 인용수 순위
  • Reference
1 P. Werbos, 'Advanced forecasting methods for global crisis warning and models of intelligence,' General System Yearbook, Vol. 22, pp. 25-38, 1977
2 Richard S. Sutton, 'Learning to predict by the methods of temporal difference,' Machine Learning, Vol. 3, pp. 9-44, 1988
3 Jennie Si, and Yu-Tsung Wang, 'On-Line Learning Control by Association and Reinforcement,' IEEE Transactions on Neural Networks, Vol. 12, No. 2, pp.264-276, 2001   DOI   ScienceOn
4 Richard S. Sutton, and Andrew G. Barto, 'Reinforcement Learning : An Introduction,' MIT Press, Cmabrige, MA, 1998
5 Charles W. Anderson, 'Strategy Learning with Multilayer Connectionist Representations,' Proceedings of the 4th International Workshop on Machine Learning, pp. 103-114, 1987
6 Dean F. Hougen, Maria Gini, and James Slagle, 'Partitioning input space for reinforcement learning for control,' Proceedings of the IEEE International Conference on Roborics and Autonation, pp. 1917-1922, April, 1996
7 Charles W. Anderson, 'Learning to Control an Inverted Pendulum Using Neural Network,' IEEE Control Systems Magazine, Vol. 9, No. 3, pp. 31-37. 1989   DOI   ScienceOn
8 Andrew G. Barto, Richard S. Sutton, Charles W. Anderson, 'Neuronlike Adaptive Elements That Can Solve Difficult Learning Control Problems,' IEEE Transactions on Systems, Man, and Cybernetics, Vol. SMC-13, No. 5, 1983
9 J. S. Albus, 'A New Approach to Manipulator control: The Cerebellar Model Articulation Controller(CMAC),' Journal of Dynamics Systems, Measurement, and Control, pp. 220-227, 1975
10 T. Kohonen, 'Self organizing maps,' Berlin: Springer
11 Andrew James Smith, 'Applications of the self-organizing map to reinforcement learning,' In Neural Network (Special Issue), 15 pp. 1107-1124, 2002   DOI   ScienceOn