Browse > Article

Reinforcement Learning with Clustering for Function Approximation and Rule Extraction  

이영아 (경희대학교 컴퓨터공학과)
홍석미 (경희대학교 컴퓨터공학과)
정태충 (경희대학교 컴퓨터공학과)
Publication Information
Journal of KIISE:Software and Applications / v.30, no.11, 2003 , pp. 1054-1061 More about this Journal
Abstract
Q-Learning, a representative algorithm of reinforcement learning, experiences repeatedly until estimation values about all state-action pairs of state space converge and achieve optimal policies. When the state space is high dimensional or continuous, complex reinforcement learning tasks involve very large state space and suffer from storing all individual state values in a single table. We introduce Q-Map that is new function approximation method to get classified policies. As an agent learns on-line, Q-Map groups states of similar situations and adapts to new experiences repeatedly. State-action pairs necessary for fine control are treated in the form of rule. As a result of experiment in maze environment and mountain car problem, we can achieve classified knowledge and extract easily rules from Q-Map
Keywords
reinforcement learning; Q-Learning; clustering; function approximation; rule extraction; self-organizing features map;
Citations & Related Records
연도 인용수 순위
  • Reference
1 Richard S. Sutton and Andrew G. Barto. Reinforcement Learning: An Introduction. The MIT Press, Cambridge, MA., 1998
2 R. Matthew Kretchmar, Charles W. Anderson, Using Temporal Neighborhoods to Adapt Function Approximators in Reinforcement Learning, IWANN99: International Work Conference on Artificial and Natural Neural Networks : Alicante, Spain. June 1999
3 Haixun Wang, Wei Wang, Jiong Yang, Philip S. Yu, Clustering by Pattern Similarity in Large Data Sets, ACM SIGMOD Conference 2002 Madison, Wisconsin, USA   DOI
4 Ron Sun, knowledge Extraction from Reinforcement Learning, Proceedings of International Joint Conference on Neural Networks, Washington, DC. July 10-15, 1999. IEEE Press, Piscataway, NJ
5 R.Matthew Kretchmar, Charles W. Anderson, Comparison of CMACs and Radial Basis Functions for Local Function Approximators in Reinforcement Learning, ICNN'97. International Conference on Neural Networks. 1997
6 Edward Keedwell, Ajit Narayanan and Dragon Savic, Using Genetic algorithms to extract rules from trained neural networks, Proceedings of the Genetic and Evolutionary Computing Conference, Volume 1, Morgan Kaufmann Publishers, San Francisco, California, USA, 1999: 793
7 Ron Sun, Supplementing Neural Reinforcement Learning with Symbolic Methods: Possibilities and Challenges, Proceedings of International Joint Conference on Neural Networks, Washington, DC. July 10-15, 1999. IEEE Press, Piscataway, NJ   DOI
8 Richard S. Sutton, Generalization in Reinforcement Learning: Successful Examples Using sparse Coarse Coding, Advances in Neural Information Processing Systems, pp.1038-1044, MIT Press, 1996
9 Rudy Setiono and Huan Liu, Symbolic Representation of Neural Networks, IEEE Computer March 1996 (Vol. 29, No. 3) pp. 71-77   DOI   ScienceOn
10 Michael Herrmann, Ralf Der, Efficient Q-Learning by Division of Labor, in Proc. International Conference on Artificial Neural Networks-ICANN'95, Vol. II, S.129-134
11 Stuart I. Reynolds, Adaptive Resolution Model-Free Reinforcement Learning: Decision Boundary Partition, Advances in Artificial Intelligence, 14th Biennial Conference of the Canadian Society for Computational Studies of Intelligence(AI-2001), Ottawa, Canada, June 2001, Proceedings