Browse > Article
http://dx.doi.org/10.3745/KIPSTB.2003.10B.6.587

Function Approximation for Reinforcement Learning using Fuzzy Clustering  

Lee, Young-Ah (경희대학교 대학원 컴퓨터공학과)
Jung, Kyoung-Sook (경희대학교 대학원 컴퓨터공학과)
Chung, Tae-Choong (경희대학교 컴퓨터공학과)
Publication Information
The KIPS Transactions:PartB / v.10B, no.6, 2003 , pp. 587-592 More about this Journal
Abstract
Many real world control problems have continuous states and actions. When the state space is continuous, the reinforcement learning problems involve very large state space and suffer from memory and time for learning all individual state-action values. These problems need function approximators that reason action about new state from previously experienced states. We introduce Fuzzy Q-Map that is a function approximators for 1 - step Q-learning and is based on fuzzy clustering. Fuzzy Q-Map groups similar states and chooses an action and refers Q value according to membership degree. The centroid and Q value of winner cluster is updated using membership degree and TD(Temporal Difference) error. We applied Fuzzy Q-Map to the mountain car problem and acquired accelerated learning speed.
Keywords
Reinforcement Learning; Function Approximation; fuzzy clustering; Q-learning; membership degree; Fuzzy Q-Learning;
Citations & Related Records
Times Cited By KSCI : 1  (Citation Analysis)
연도 인용수 순위
1 Richard S. Sutton and Andrew G. Barto, 'Reinforcement Learning : An Introduction,' The MIT Press, Cambridge, MA., 1998
2 Stephan ten Hagen and Ben Krose, 'Q-learning for System with continuous state and action spaces,' BENELEARN 2000, 10th Belgian-Dutch conference on Machine Learning
3 Chris Gaskett, David Wettergreen, and Alexander Zelinsky, 'Q-learning in continuous state and action spaces,' Australian Joint Conference on Artificial Intelligence, 1999
4 Jan Jantzen, 'Neurofuzzy Modelling,' Technical Report 98-H-869 (soc), Technical University of Denmark : Dept. of Automation, http://fuzzy.iau.dtu/download/soc.pdf, 1998. Lecture notes, pp. 14
5 전효병, 이동욱, 김대준, 심귀보, '퍼지추론에 위한 리커런트 뉴럴 네트워크 강화학습,' 한국퍼지및지능시스템학회 '97년도 춘계학술대회논문집, 1997   과학기술학회마을
6 정석일, 이연정, '분포 기여도를 이용한 퍼지 Q-learning,' 퍼지및지능시스템학회논문지, Vol.11, No.5, pp.388-394, 2001   과학기술학회마을
7 Pierre Yves Glorennec, Lionel Jouffe, 'Fuzzy Q-learning,' Proceedings of Fuzz-Ieee'97, Sixth Internationl Conference on Fuzzy Systems, Barcelona, pp.719-724, July, 1997
8 Andrea Bonarini. 'Delayed Reinforcement, Fuzzy Q-learning and Fuzzy Logic Controllers,' In Herrera, F. Verdegay, J. L. (Eds.) Genetic Algorithns and Soft Computing, (Studies in Fuzziness, 8.), Physica-Verlag, Berlin, D., pp.447-466
9 Lionel Jouffe, 'Fuzzy Inference System Learning by Reinforcement Methods,' Ieee Transactions on System,Man and Cybernetics, Vol.98, No.3, August, 1998   DOI
10 Artistidis Likas, 'A Reinforcement Learning Approach to On-Line Clustering,' Neural Computation, Vol.11, No.8, pp.1915-1932, 1999   DOI   ScienceOn