Browse > Article
http://dx.doi.org/10.7583/JKGS.2015.15.5.109

The UCT algorithm applied to find the best first move in the game of Tic-Tac-Toe  

Lee, Byung-Doo (Division of Sports Science, Sehan University)
Park, Dong-Soo (Division of Sports Science, Sehan University)
Choi, Young-Wook (Division of Sports Science, Sehan University)
Publication Information
Journal of Korea Game Society / v.15, no.5, 2015 , pp. 109-118 More about this Journal
Abstract
The game of Go originated from ancient China is regarded as one of the most difficult challenges in the filed of AI. Over the past few years, the top computer Go programs based on MCTS have surprisingly beaten professional players with handicap. MCTS is an approach that simulates a random sequence of legal moves until the game is ended, and replaced the traditional knowledge-based approach. We applied the UCT algorithm which is a MCTS variant to the game of Tic-Tac-Toe for finding the best first move, and compared it with the result generated by a pure MCTS. Furthermore, we introduced and compared the performances of epsilon-Greedy algorithm and UCB algorithm for solving the Multi-Armed Bandit problem to understand the UCB.
Keywords
Go; Tic-Tac-Toe; MCTS; UCT; Multi-Armed Bandit; epsilon-Greedy; UCB;
Citations & Related Records
Times Cited By KSCI : 8  (Citation Analysis)
연도 인용수 순위
1 B.D. Lee, "Analysis of Tic-Tac-Toe Game Strategies using Genetic Algorithm", Journal of Korea Game Society, Vol. 14, No. 6, pp. 39-48, 2014.   DOI
2 Wikipedia, "Tic-Tac-Toe", from http://en.wikipedia.org/wiki/Tic-Tac-Toe, 2015.
3 A.A.J van der Kleij, "Monte Carlo Tree Search and Opponent Modeling through Player Clustering in no-limit Texas Hold'en Poker", Master thesis, University of Groningen, 2010.
4 H. Baier and M.H.M. Winands, "Monte-Carlo Tree Search and Minimax Hybrids", Computer Games, Vol. 504, pp. 45-63, 2014.   DOI
5 G. Hochmuth, "On the Genetic Evolution of a Perfect Tic-Tac-Toe Strategy", from http://www.genetic-programming.org/sp2003/Hochmuth.pdf, 2015.
6 N. Sephton, P.I. Cowling, E. Powley and N.H. Slaven, "Heuristic Move Pruning in Monte Carlo Tree Search for the Strategic Card Game Lords of War", In Computational Intelligence and Games (CIG) of IEEE, pp. 1-7, 2014.
7 T. Pepels, "Novel Selection Methods for Monte-Carlo Tree Search", Master thesis, University of Masstricht, 2014.
8 D. Brand and S. Kroon, "Sample Evaluation for Action Selection in Monte Carlo Tree Search", from http://dl.acm.org/citation.cfm?doid=2664591.2664612, 2015.
9 Y. Wang and S. Gelly, "Modification of UCT and sequence-like simulations for Monte-Carlo Go", from http://dept.stat.lsa.umich.edu/-yizwang/publications/wang07modifications.pdf, 2015.
10 J.M. White, "Bandit Algorithms for Website Optimization", O'Relly, 2013.
11 L. Lew, "Modeling Go Game as a Large Decomposable Decision Process", Ph.D. thesis, Warsaw University, 2011.
12 P. Auer, N. Cesa-Bianchi and P. Fisher, "Finite-time Analysis of the Multiarmed Bandit Problem", Kluwer Academic Publishers, 2002.
13 S. Takeuchi, T. Kanoke and K. Yamaguchi, "Evaluation of Monte Carlo Tree Search and the Application of Go", from http://www.csse.uwa.edu.au/cig08/Proceedings/papers/8046.pdf, 2015.
14 I.J. Ahn and I.K. Park, "Design of Omok AI using Genetic Algorithm and Game Trees and Their Parallel Processing on the CPU", Journal of the Korea Information Science Society, Vol. 37, No. 2, pp. 66-75, 2010.
15 A. Bhatt, P. Varshney and K. Deb, "In Search of No-loss Strategies for the Game of Tic-Tac-Toe using a Customized Genetic Algorithm", GECCO'08(Genetic and Evolutionary Computation Conference 2008, pp. 889-896, 2008.
16 B.D. Lee, "Comparison of LDA and PCA for Korean Pro Go Player's Opening Recognition", Journal of Korea Game Society, Vol. 13, No. 4, pp. 15-24, 2013.   DOI
17 B.D Lee, "Monte-Carlo Tree Search Applied to the game of Tic-Tac-Toe", Journal of Korea Game Society, Vol. 14, No. 3, pp. 47-54, 2014.   DOI
18 B.D. Lee and J.W. Park, "Applying Principal Component Analysis to Go Openings", Journal of Korea Game Society, Vol. 13, No. 2, pp. 59-70, 2013.   DOI
19 B.D. Lee, "Evolutionary neural network model for recognizing strategic fitness of a finished Tic-Tac-Toe game", Journal of Korean Society for Computer Game, Vol. 28, No. 2, pp. 95-101, 2015.
20 B.D. Lee and Y.W. Choi, "The best move sequence in playing Tic-Tac-Toe game", Journal of The Korean Society for Computer Game, Vol. 27, No. 3, pp. 11-16, 2014.
21 B.D. Lee, "Analysis of Korean, Chinese and Japanese Pro Go Player's Openings", Journal of Korean Society for Computer Game, Vol. 26, No. 4, pp. 17-26, 2013.
22 B.D. Lee, "Korean Pro Go Player's Opening Recognition Using PCA", Journal of Korean Society for Computer Game, Vol. 26, No. 2, pp. 228-233, 2013.
23 S. Gelly, M. Schoenauer, M. Sebag, O. Teytaud, L. Kocsis, D. Silver and C. Szepesvari, "The Grand Challenge of Computer Go: Monte Carlo Tree Search and Extensions", Communications of the ACM, Vol. 55, No. 3, pp. 106-113, 2012.   DOI
24 S. Gelly and D. Silver, "Monte-Carlo Tree Search and Rapid Action Value Estimation in Computer Go", Artificial Intelligence, Vol. 75, Issue 11, pp. 1856-1875, 2011.
25 Wikipedia, "Computer Go", from http://en.wikipedia.org/wiki/Computer_Go, 2015.
26 G. Chaslot, "Monte-Carlo Tree Search", Ph.D. dissertation, University of Masstricht, 2010.