Browse > Article

Learning Relational Instance-Based Policies from User Demonstrations  

Park, Chan-Young (경기대학교 컴퓨터과학과)
Kim, Hyun-Sik (경기대학교 전자계산학과)
Kim, In-Cheol (경기대학교 컴퓨터과학과)
Publication Information
Journal of KIISE:Software and Applications / v.37, no.5, 2010 , pp. 363-369 More about this Journal
Abstract
Demonstration-based learning has the advantage that a user can easily teach his/her robot new task knowledge just by demonstrating directly how to perform the task. However, many previous demonstration-based learning techniques used a kind of attribute-value vector model to represent their state spaces and policies. Due to the limitation of this model, they suffered from both low efficiency of the learning process and low reusability of the learned policy. In this paper, we present a new demonstration-based learning method, in which the relational model is adopted in place of the attribute-value model. Applying the relational instance-based learning to the training examples extracted from the records of the user demonstrations, the method derives a relational instance-based policy which can be easily utilized for other similar tasks in the same domain. A relational policy maps a context, represented as a pair of (state, goal), to a corresponding action to be executed. In this paper, we give a detail explanation of our demonstration-based relational policy learning method, and then analyze the effectiveness of our learning method through some experiments using a robot simulator.
Keywords
Demonstration-Based Learning; Relational Model; Instance-Based Policy; Transfer Learning;
Citations & Related Records
연도 인용수 순위
  • Reference
1 P. Tadepalli, R. Givan, and K. Driessens, "Relational Reinforcement Learning: An Overview," Proc. of the 21th International Conference on Machine Learning, ICML'04, Workshop on Relational Reinforcement Learning, 2004.
2 J. Saunders, C. L. Nehaniv, and K. Dautenhahn, "Teaching Robots by Moulding Behavior and Scaffolding the Environment," Proc. of the 1st International Conference on Human-Robot Interaction, HRI'06, pp.118-125, 2006.
3 R. Garcia-Duran, F. Fernandez, and D. Borrajo, "Nearest Prototype Classification for Relational Learning," Proc. of the 16th International Conference on Inductive Logic Programming(ILP-2006), pp.89-91, 2006.
4 K. Johns and T. Taylor, Professional Microsoft Robotics Developer Studio, Wiley, 2008.
5 S. Chernova and M. Veloso, "Confidence-Based Policy Learning from Demonstration Using Gaussian Mixture Models," Proc. of the 6th International Joint Conference on Autonomous Agents and Multi-Agent Systems, AAMAS'07, pp.1315-1322, 2007.
6 H. Veeraraghavan and M. Veloso, "Teaching Sequential Tasks with Repetition through Voice and Vision," Proc. of the 7th International Joint Conference on Autonomous Agents and Multi- Agent Systems, AAMAS'08, pp.518-527, 2008.
7 E. Winner and M. Veloso, "LoopDISTILL: Learning Domain-Specific Planners from Example Plans," Proc. of the International Conference on Automated Planning and Scheduling, ICAPS'07 Workshop on AI Planning and Learning, 2007.
8 B. D. Argall, S. Chernova, M. Veloso, and B. Browning, "A Survey of Robot Learning from Demonstration," Robotics and Autonomous Systems, vol.57, pp.469-483, 2009.   DOI   ScienceOn
9 M. Nicolescu and M. Mataric, "Methods for Robot Task Learning: Demonstrations, Generalization and Practice," Proc. of the 2nd International Joint Conference on Autonomous Agents and Multi-Agent Systems, AAMAS'03, pp.241-248, 2003.
10 E.F. Morales and C. Sammut, "Learning to Fly by Combining Reinforcement Learning with Behavioral Cloning," Proc. of the 21th International Conference on Machine Learning, ICML'04, pp.76-81, 2004.
11 L. D. Raedt, Logical and Relational Learning, Cognitive Technologies, Springer, Berlin, 2008.