Browse > Article

A Motivation-Based Action-Selection-Mechanism Involving Reinforcement Learning  

Lee, Sang-Hoon (College of Information and Communications, Hanyang University)
Suh, Il-Hong (College of Information and Communications, Hanyang University)
Kwon, Woo-Young (College of Information and Communications, Hanyang University)
Publication Information
International Journal of Control, Automation, and Systems / v.6, no.6, 2008 , pp. 904-914 More about this Journal
Abstract
An action-selection-mechanism(ASM) has been proposed to work as a fully connected finite state machine to deal with sequential behaviors as well as to allow a state in the task program to migrate to any state in the task, in which a primitive node in association with a state and its transitional conditions can be easily inserted/deleted. Also, such a primitive node can be learned by a shortest path-finding-based reinforcement learning technique. Specifically, we define a behavioral motivation as having state-dependent value as a primitive node for action selection, and then sequentially construct a network of behavioral motivations in such a way that the value of a parent node is allowed to flow into a child node by a releasing mechanism. A vertical path in a network represents a behavioral sequence. Here, such a tree for our proposed ASM can be newly generated and/or updated whenever a new behavior sequence is learned. To show the validity of our proposed ASM, experimental results of a mobile robot performing the task of pushing- a- box-in to- a-goal(PBIG) will be illustrated.
Keywords
Action-selection-mechanism; behavior-based control; reinforcement learning; robot learning;
Citations & Related Records

Times Cited By Web Of Science : 2  (Related Records In Web of Science)
Times Cited By SCOPUS : 3
연도 인용수 순위
1 T. Tyrrell, Computational Mechanisms for Action Selection, Ph.D. Dissertation, University of Edinburg, 1993
2 B. M. Blumberg, P. M. Todd, and P. Maes, "No bad dogs: Ethological lessons for learning in hamsterdam," Proc. of the Fourth International Conference on the Simulation of Adaptive Behavior, From Animals to Animats, 1996
3 R. Brooks, "A robust layered control system for a mobile robot," IEEE Trans. on Robotics and Automation, vol. 2, no. 1, pp. 14-23, March 1986   DOI
4 E. Hogewoning, J. Broekens, J. Eggermont, and E. G. P. Bovenkamp, "Strategies for affectcontrolled action-selection in Soar-RL," Lecture Notes in Computer Science, vol. 4528, pp. 501-510, 2007
5 R. Ameur and J.-C. Heudin, "Interactive intelligent agent architecture," Proc. WI-IAT Workshops Web Intelligence and International Agent Technology Workshops IEEE/WIC/ACM International Conference on, pp. 331-334, Dec. 2006
6 M. Azar, M. Ahmadabadi, A. Farahmand, and B. Araabi, "Learning to coordinate behaviors in soft behavior-based systems using reinforcement learning," Proc. of International Joint Conference on Neural Networks, pp. 241-248, 2006
7 N. Goerke and T. Henne, "Learning hierarchical action selection for an autonomous robot," Proc. of International Joint Conference on Neural Network, pp. 4958-4965, 16-21 July 2006
8 J. Qiao, Z. Hou, and X. Ruan, "Q-learning based on neural network in learning action selection of mobile robot," Proc. of IEEE International Conference on Automation and Logistics, pp. 263-267, 18-21 Aug. 2007
9 H. S. Ahn and J. Y. Choi, "Emotional behavior decision model based on linear dynamic systems for intelligent service robots," Proc. of the 16th IEEE International Symposium on Robot and Human interactive Communication, pp. 786-791, 26-29 Aug. 2007
10 M. Kim and H. Kim, "Behavior coordination mechanism for intelligent home," Proc. of the 5th IEEE/ACIS International Conference on Computer and Information Science, pp. 452-457, 10-12 July 2006
11 R. Arkin, M. Fujita, T. Takagi, and R. Hasegawa, "Ethological modeling and architecture for an entertainment robot," Proc. of IEEE International Conference on Robotics and Automation, vol. 1, pp. 453-458, 2001
12 A. Doucet, N. de Freitas and N. Gordon, Sequential Monte Carlo Methods in Practice, N. Gordon, Ed. Springer-Verlag, 2001
13 M. Scheutz and V. Andronache, "Architectural mechanisms for dynamic changes of behavior selection strategies in behavior-based systems," IEEE Trans. on Systems, Man and Cybernetics, Part B, vol. 34, no. 6, pp. 2377-2395, Dec. 2004   DOI   ScienceOn
14 E. Spier and D. McFarl, "A finer-grained motivational model of behavior sequencing," Proc. of the Fourth International Conference on Simulation of Adaptive Behavior, From Animals to Animats 4, 1997
15 M. Humphrys, Action Selection Methods Using Reinforcement Learning, Ph.D. Dissertation, University of Cambridge, 1996
16 J. Bryson, "Hierarchy and sequence vs. full parallelism in action selection," Proc. of the Sixth International Conference From Animals to Animats 6, R. P. Jean-Arcady Meyer, Ed. MIT Press, 2000
17 M. J. Huber, S. Kumar, and D. McGee, "Toward a suite of performatives based upon joint intention theory," Lecture Notes in Computer Science, vol. 3396, pp. 226-241, 2005
18 J. Jaafar, E. McKenzie, and A. Smaill, "A fuzzy action selection method for virtual agent navigation in unknown virtual environments," Proc. of IEEE International Fuzzy Systems Conference, pp. 1-6, 23-26 July 2007
19 E. de Sevin and D. Thalmann, "A motivational model of action selection for virtual humans," Proc. of the Computer Graphics International, Washington, DC, USA, IEEE Computer Society, pp. 213-220, 2005
20 A. Guillot and J.-A. Meyer, "Computer simulations of adaptive behavior in animals," Proc. Computer Animation '94, pp. 122-131, May 25-28, 1994
21 S. Lee and I. H. Suh, "A programming framework supporting an ethology-based behavior control architecture," Proc. of IEEE/RSJ International Conference on Intelligent Robots and Systems, pp. 4138-4144, Oct. 2006
22 P. Maes, "A bottom-up mechanism for behavior selection in an artificial creature," Proc. of the First International Conference on Simulation of Adaptive Behavior on From Animals To Animats, Cambridge, MA, USA, MIT Press, pp. 238-246, 1990
23 C. M. Witkowski, Schemes for Learning and Behavior: A New Expectancy Model, Ph.D. Dissertation, University of London, 1997
24 P. Pirjanian, "Behavior coordination mechanisms-state-of-the-art," Tech-report IRIS-99-375, Institute for Robotics and Intelligent Systems, University of Southern California, Tech. Rep., 1999
25 J. H. Connell, "A behavior-based arm controller," IEEE Trans. on Robotics and Automation, vol. 5, no. 6, pp. 784-791, Dec. 1989   DOI
26 P. Maes, "Modeling adaptive autonomous agents," Artificial Life, vol. 1, no. 1-2, pp. 135-162, 1994   DOI
27 S. W. Wilson, "The animat path to AI," Proc. of the First International Conference on Simulation of Adaptive Behavior on From Animals To Animats, Cambridge, MA, USA, MIT Press, 1990
28 T. Sawada, T. Takagi, and M. Fujita, "Behavior selection and motion modulation in emotionally grounded architecture for qrio sdr-4xii," Proc. IEEE/RSJ International Conference on Intelligent Robots and Systems, vol. 3, pp. 2514-2519, Sept. 28-Oct. 2, 2004
29 C. L. Hull, Principle of Behavior, Appleton-Century-Crofts, New York, 1943
30 W. Kwon, I. H. Suh, S. Lee, and Y.-J. Cho, "Fast reinforcement learning using stochastic shortest paths for a mobile robot," Proc. of IEEE/RSJ International Conference on Intelligent Robots and Systems, pp. 82-87, Oct. 29-Nov. 2 2007
31 Y. Hoshino, T. Takagi, U. Di Profio, and M. Fujita, "Behavior description and control using behavior module for personal robot," Proc. of IEEE International Conference on Robotics and Automation, vol. 4, pp. 4165-4171, Apr 26-May 1, 2004
32 I. H. Suh, S. Lee, W. Y. Kwon, and Y.-J. Cho, "Learning of action patterns and reactive behavior plans via a novel two-layered ethology-based action selection mechanism," Proc. of IEEE/RSJ International Conference on Intelligent Robots and Systems, pp. 1799-1805, 2-6 Aug. 2005
33 J. Rosenblatt and D. Payton, "A fine-grained alternative to the subsumption architecture for mobile robot control," Proc. of International Joint Conference on Neural Networks IJCNN, pp. 317-323, 18-22 June 1989
34 Z. Shen, C. Miao, Y. Miao, X. Tao, and R. Gay, "A goal-oriented approach to goal selection and action selection," Proc. of IEEE International Conference on Fuzzy Systems, pp. 114-121, 2006
35 M. N. Nicolescu and M. J. Matari'c, "A hierarchical architecture for behavior-based robots," Proc. of the First International Joint Conference on Autonomous Agents and Multiagent Systems, ACM, New York, NY, USA, pp. 227-233, 2002