Browse > Article
http://dx.doi.org/10.3745/KTSDE.2022.11.3.141

Effective Utilization of Domain Knowledge for Relational Reinforcement Learning  

Kang, MinKyo (경기대학교 컴퓨터과학과)
Kim, InCheol (경기대학교 컴퓨터과학과)
Publication Information
KIPS Transactions on Software and Data Engineering / v.11, no.3, 2022 , pp. 141-148 More about this Journal
Abstract
Recently, reinforcement learning combined with deep neural network technology has achieved remarkable success in various fields such as board games such as Go and chess, computer games such as Atari and StartCraft, and robot object manipulation tasks. However, such deep reinforcement learning describes states, actions, and policies in vector representation. Therefore, the existing deep reinforcement learning has some limitations in generality and interpretability of the learned policy, and it is difficult to effectively incorporate domain knowledge into policy learning. On the other hand, dNL-RRL, a new relational reinforcement learning framework proposed to solve these problems, uses a kind of vector representation for sensor input data and lower-level motion control as in the existing deep reinforcement learning. However, for states, actions, and learned policies, It uses a relational representation with logic predicates and rules. In this paper, we present dNL-RRL-based policy learning for transportation mobile robots in a manufacturing environment. In particular, this study proposes a effective method to utilize the prior domain knowledge of human experts to improve the efficiency of relational reinforcement learning. Through various experiments, we demonstrate the performance improvement of the relational reinforcement learning by using domain knowledge as proposed in this paper.
Keywords
Relational Reinforcement Learning; Domain Knowledge; Policy; Logic Predicate; Generality; Interpretability;
Citations & Related Records
연도 인용수 순위
  • Reference
1 A. Payani and F. Fekri, "Inductive logic programming via differentiable deep neural logic networks," arXiv preprint arXiv:1906.03523, 2019.
2 H. Dong, J. Mao, T. Lin, C. Wang, L. Li, and D. Zhou, "Neural logic machines," Proceedings of International Conference on Learning Representations (ICLR), 2019.
3 V. Zambaldi, et al., "Relational deep reinforcement learning," arXiv preprint arXiv:1806.01830, 2018.
4 A. Payani and F. Fekri, "Incorporating relational background knowledge into reinforcement learning via differentiable inductive logic programming," arXiv preprint arXiv:2003.10386, 2020.
5 O. Rivlin, T. Hazan, and E. Karpas, "Generalized planning with deep reinforcement learning," arXiv preprint arXiv: 2005.02305, 2020.
6 Z. Jiang and S. Luo, "Neural logic reinforcement learning," Proceedings of International Conference on Machine Learning (ICML), 2019.
7 J. Janisch, T. Pevny, and V. Lisy, "Symbolic relational deep reinforcement learning based on graph neural networks," arXiv preprint arXiv:2009.12462, 2020.
8 S. Das, S. Natarajan, K. Roy, R. Parr, and K. Kersting, "Fitted Q-learning for relational domains," arXiv preprint arXiv: 2006.05595, 2020.
9 S. Garg and A. Bajpai, "Symbolic network: Generalized neural policies for relational MDPs," Proceedings of International Conference on Machine Learning (ICML), pp.3397-3407, Nov. 2020.
10 D. Adjodah, T. Klinger, and J. Joseph, "Symbolic relation networks for reinforcement learning," Proceedings of the Workshop on Relational Representation Learning in Conference on Neural Information Processing Systems (NeurIPS), 2018.
11 T. Gokhale, S. Sampat, Z. Fang, Y. Yang, and C. Baral, "Blocksworld revisited: Learning and reasoning to generate event-sequences from image pairs," arXiv preprint arXiv: 1905.12042, 2019.
12 M. A. Skinner, L. Raman, N, Shah, A. Farhat, and S. Natarajan, "A preliminary approach for learning relational policies for the management of critically ill children," arXiv preprint arXiv:2001.04432, 2020.
13 Y. Zhang and A. Ramesh, "Learning interpretable relational structures of hinge-loss markov random fields," Proceedings of International Joint Conference on Artificial Intelligence (IJCAI), pp.6050-6056, Aug. 2019.
14 Y. Zhang and A. Ramesh, "Learning fairness-aware relational structures," Proceedings of European Conference on Artificial Intelligence (ECAI), 2020.