Browse > Article

A Function Approximation Method for Q-learning of Reinforcement Learning  

이영아 (경희대학교 컴퓨터공학과)
정태충 (경희대학교 컴퓨터공학과)
Publication Information
Journal of KIISE:Software and Applications / v.31, no.11, 2004 , pp. 1431-1438 More about this Journal
Abstract
Reinforcement learning learns policies for accomplishing a task's goal by experience through interaction between agent and environment. Q-learning, basis algorithm of reinforcement learning, has the problem of curse of dimensionality and slow learning speed in the incipient stage of learning. In order to solve the problems of Q-learning, new function approximation methods suitable for reinforcement learning should be studied. In this paper, to improve these problems, we suggest Fuzzy Q-Map algorithm that is based on online fuzzy clustering. Fuzzy Q-Map is a function approximation method suitable to reinforcement learning that can do on-line teaming and express uncertainty of environment. We made an experiment on the mountain car problem with fuzzy Q-Map, and its results show that learning speed is accelerated in the incipient stage of learning.
Keywords
Q-teaming; Q-teaming; function approximation; online fuzzy clustering;
Citations & Related Records
Times Cited By KSCI : 2  (Citation Analysis)
연도 인용수 순위
1 정석일, 이연정, '분포기여도를 이용한 퍼지 Q-Learning', 퍼지 및 지능시스템 학회 논문지, vol. 11, no. 5, pp. 388-394, 2001   과학기술학회마을
2 Richard S. Sutton, 'Generalization in Reinforcement Learning: Successful Examples Using Sparse Coarse Coding,' Advances in Neural Information Processing Systems 8, pp. 1038-1044, MIT Press, 1996
3 R. Matthew Kretchmar, Charles W. Anderson, 'Comparison of CMACs and Radial Basis Functions for Local Function Approximators in Reinforcement Learning,' Proceedings of International Conference on Neural Networks, 1997   DOI
4 Juan Carlos Santamaria, Richard S. Sutton, Ashwin Ram, 'Experiments with Reinforcement Learning in Problems with Continuous State and Action Spaces,' COINS Technical Report 96-88, 1996
5 William D. Smart, Leslie Pack Kaelbling, 'Practical Reinforcement Learning in Continuous Spaces,' Proceedings of International Conference on Machine Learning, 2000
6 William D. Smart, Leslie Pack Kaelbling, 'Reinforcement Learning for Robot Control,' In Mobile Robots XVI, 2001
7 Nicolas B. Karayiannis, James C. Bezdek, 'An Integrated Approach to Fuzzy Learning Vector Quantization and Fuzzy c-Means Clstering,' IEEE Transactions of Fuzzy systems, vol. 5, no. 4, 1997   DOI   ScienceOn
8 전종원, 민준영, 'GLVQ클러스터링을 위한 필기체 숫자의 효율적인 특징추출 방법', 한국정보처리학회 논문지, vol. 2, no. 6, 1995   과학기술학회마을
9 Barbara Hammer, Thomas Villmann, 'Generalized Relevance Learning Vector Quantization,' Neural Networks, vol. 15 no. 8-9, pp. 1059-1068, 2002   DOI   ScienceOn
10 Shyn Jong Hu, 'Pattern Recognition by LVQ and GLVQ Networks,' http://neuron.et.ntust.edu.tw/homework/87/NN/87Homework%232/M8702043
11 Michael Herrmann, Ralf Der, 'Efficient Q- Learning by Division of Labor,' Proceedings of International Conference on Artificial Neural Networks, 1995
12 K. Yamada, M. Svinin, K. Ueda, 'Reinforcement Learning with Autonomous State Space Construction using Unsupervised Clustering Method,' Proceedings of the 5th International Symposium on Artificial Life and Robotics, 2000
13 Lionel Jouffe, 'Fuzzy Inference System Learning by Reinforcement Methods,' IEEE Transactions on Systems, Man and Cybernetics pp. 338-355, 1998.   DOI
14 Andrea Bonarini, 'Delayed Reinforcement, Fuzzy Q-Learning and Fuzzy Logic Controllers,' In Herrera, F., Verdegay, J. L. (Eds.) Genetic Algorithms and Soft Computing, pp. 447-466, 1996
15 Pierre Yves Glorennce, 'Reinforcement Learning : an Overview,' Proceedings of the European Symposium on Intelligent Techniques, 2000
16 Aristidis Likas, 'A Reinforcement Learning Approach to On-line Clustering,' Neural computation 11 (8): 1915-1932, 1999   DOI   ScienceOn
17 William Donald Smart, 'Making Reinforcement Learning Work on Real Robots,' Ph. D. Thesis, Brown University, 2002
18 A.K. Jain, M.N, Murty, P.J. Flynn, 'Data Clustering: A Review,' ACM Computing Surveys, vol. 31, no. 3, 1999   DOI   ScienceOn
19 Baraldi, A. and Blonda, P., 1999, 'A Survey of Fuzzy Clustering Algorithms for Pattern Recognition - Part I,' IEEE Transactions on Systems, Man, and Cybernetics, Part B, Vol. 29, No.6, pp. 778-786   DOI   ScienceOn
20 Richard Sutton, Andrew G. Barto, 'Reinforcement Learning :An Introduction,' MIT Press, 1998
21 Leslie Pack Kaelbling, Michael L. Littman, Andrew W. Moor, 'Reinforcement Learning: A Survey,' Journal of Artificial Intelligence Research, vol. 4, pp. 237-285, 1996
22 Pierre Yves Glorennec, Lionel Jouffe, 'Fuzzy Q-Learning,' Proceedings of Sixth IEEE International Conference on Fuzzy Systems, pp. 719-724, 1997