Browse > Article
http://dx.doi.org/10.7471/ikeee.2019.23.4.1187

An Artificial Intelligence Game Agent Using CNN Based Records Learning and Reinforcement Learning  

Jeon, Youngjin (Dept. of Computer Engineering, Seokyeong University)
Cho, Youngwan (Dept. of Computer Engineering, Seokyeong University)
Publication Information
Journal of IKEEE / v.23, no.4, 2019 , pp. 1187-1194 More about this Journal
Abstract
This paper proposes a CNN architecture as value function network of an artificial intelligence Othello game agent and its learning scheme using reinforcement learning algorithm. We propose an approach to construct the value function network by using CNN to learn the records of professional players' real game and an approach to enhance the network parameter by learning from self-play using reinforcement learning algorithm. The performance of value function network CNN was compared with existing ANN by letting two agents using each network to play games each other. As a result, the winning rate of the CNN agent was 69.7% and 72.1% as black and white, respectively. In addition, as a result of applying the reinforcement learning, the performance of the agent was improved by showing 100% and 78% winning rate, respectively, compared with the network-based agent without the reinforcement learning.
Keywords
Reinforcement Learning; Othello game agent; Value function network; CNN; Records learning;
Citations & Related Records
연도 인용수 순위
  • Reference
1 Y. J. Jeon, Y. W. Cho, "An Implementation of Othello Game Player Using ANN based Records Learning and Minimax Search Algorithm," The Transactions of the Korean Institute of Electrical Engineers, Vol.67, No.12, pp.1657-1664, 2018. DOI: 10.5370/KIEE.2018.67.12.1657   DOI
2 D. Silver et al., "Mastering the game of Go with deep neural networks and tree search," Nature 529, pp.484-489, 2016. DOI:   DOI
3 D. Silver et al., "Mastering the game of Go with-out human knowledge," Nature 550, pp.354- 359, 2017.   DOI
4 M. Campbell, A. J. Hoane, F. Hsu, "Deep Blue," Artificial Intelligence, Vol.134, Issues 1-2, pp.57-83, 2002. DOI: 10.1016/S0004-3702(01)00129-1   DOI
5 M. Buro, "LOGISTELLO-A Strong Learning Othello Program," NEC Research Institute, Princeton, NJ, 1997.
6 P. S. Rosenbloom, "A World-Championship- Level Othello Program," Artificial Intelligence, Vol.19, Issue.3 pp.279-320, 1982. DOI: 10.1016/0004-3702(82)90003-0   DOI
7 K.-F. Lee, S. Mahajan, "The Development of a World Class Othello Program," Artificial Intelligence, Vol.43, Issue1, pp.21-36, 1990. DOI: 10.1016/0004-3702(90)90068-B   DOI
8 J. Schaeffer, H. J. Herik, "Chips Challenging Champions: Games, Computers and Artificial Intelligence," North Holland; 1 edition, pp.135, 2002.
9 Gunawan et al., "Evolutionary Neural Network for Othello Game," Procedia-Social and Behavioral Sciences, Vol.57, pp.419-425, 2012. DOI: 10.1016/j.sbspro.2012.09.1206   DOI
10 P. Liskowski, W. M. Jaskowski and K. Krawiec, "Learning to Play Othello with Deep Neural Networks," in IEEE Transactions on Games, 2018. DOI: 10.1109/TG.2018.2799997
11 N. J. van Eck and M. van Wezel, "Reinforcement learning and its application to othello," Technical Report EI 2005-47, Econometric Institute Report, 2005.
12 M. van der Ree and M. Wiering, "Reinforcement learning in the game of Othello: Learning against a fixed opponent and learning from self-play," 2013 IEEE Symposium on Adaptive Dynamic Programming and Reinforcement Learning (ADPRL), pp.108-115, 2013. DOI: 10.1109/ADPRL.2013.6614996
13 R. S. Sutton, A. G. Barto, "Reinforcement Learning: An Introduction," MIT Press, Cambridge, MA, 1998.
14 R Hahnloser, R. Sarpeshkar, M A Mahowald, R. J. Douglas, H.S. Seung, "Digital selection and analogue amplification coesist in a cortex-inspired silicon circuit," Nature. 405. pp.947-951, 2000.   DOI
15 Y. J. Jeon, "Implementation of an artificial intelligence game agent using deep neural network and reinforcement learning," Thesis of master's degree, Seokyeong University, 2019.