Journal of Korea Game Society (한국게임학회 논문지)

Korea Game Society (한국게임학회)
권호 표지
DOI QR코드

DOI QR Code

Enhanced strategic Monte-Carlo Tree Search algorithm to play the game of Tic-Tac-Toe

삼목 게임을 위해 개선된 몬테카를로 트리탐색 알고리즘

  • Lee, Byung-Doo (Dept. of Baduk Studies, Division of Sports Science, Sehan University)
  • 이병두 (세한대학교 체육학부 바둑학과)
  • Received : 2016.05.09
  • Accepted : 2016.07.07
  • Published : 2016.08.20
Download PDF
⟨ Previous Next ⟩

Abstract

Monte-Carlo Tree Search(MCTS) is a best-first tree search algorithm and has been successfully applied to various games, especially to the game of Go. We evaluate the performance of MCTS playing against each other in the game of Tic-Tac-Toe. It reveals that the first player always has an overwhelming advantage to the second player; and we try to find out the reason why the first player is superior to the second player in spite of the fact that the best game result should be a draw. Since MCTS is a statistical algorithm based on the repeated random sampling, it cannot adequately tackle an urgent problem that needs a strategy, especially for the second player. For this, we propose a strategic MCTS(S-MCTS) and show that the S-MCTS player never loses a Tic-Tac-Toe game.

몬테카를로 트리탐색은 최대우선탐색 알고리즘이며, 많은 게임 특히 바둑 게임에 성공적으로 적용되어 왔다. 삼목 게임에서 MCTS 간의 대국을 통해 성능을 평가하고자 했다. 첫 번째 대국자는 항상 두 번째 대국자에 비해 압도적인 우위를 보였으며, 최선의 게임 결과가 무승부가 됨에도 불구하고 첫 번째 대국자가 두 번째 대국자에 비해 우월한 이유를 찾고자 했다. MCTS는 반복적인 무작위 샘플링을 기반으로 하는 통계적 알고리즘이기 때문에, 특히 두 번째 대국자를 위해 전략을 요하는 시급한 문제를 적절히 대처하지 못한다. 이를 위해 전략적 MCTS(S-MCTS)를 제안하며, S-MCTS는 결코 삼목 게임에서 지지 않는다는 것을 보였다.

Keywords

References

  1. B.D. Lee, D.S. Park and Y.W. Choi, "The UCT algorithm applied to find the best first move in the game of Tic-Tac-Toe", Journal of Korea Game Society, Vol. 15, No. 5, pp. 109-118, 2015. https://doi.org/10.7583/JKGS.2015.15.5.109
  2. B.D. Lee, "Competition between MCTS and UCT in the game of Tic-Tac-Toe", Journal of Korean Society for Computer Game, Vol. 29, No. 1, pp. 1-6, 2016. https://doi.org/10.21493/kscg.2016.29.2.1
  3. B.D. Lee, "Analysis of Tic-Tac-Toe Game Strategies using Genetic Algorithm", Journal of Korea Game Society, Vol. 14, No. 6, pp. 39-48, 2014.
  4. B.D. Lee, "Monte-Carlo Tree Search Applied to the Game of Tic-Tac-Toe", Journal of Korea Game Society, Vol. 14, No. 3, pp. 47-54, 2014. https://doi.org/10.7583/JKGS.2014.14.3.47
  5. D. Silver et al., "Mastering the game of Go with deep neural networks and tree search", Journal of Nature, Vol. 529, Issue 7587, pp. 484-489, 2016. https://doi.org/10.1038/nature16961
  6. P.S. Jang "Overseas Innovation Trend", from http://www.stepi.re.kr, 2016.
  7. B.D. Lee and Y.W. Choi, "The best move sequence in playing Tic-Tac-Toe game", Journal of The Korean Society for Computer Game, Vol. 27, No. 3, pp. 11-16, 2014.
  8. B.D. Lee, "Evolutionary neural network model for recognizing strategic fitness of a finished Tic-Tac-Toe game", Journal of Korean Society for Computer Game, Vol. 28, No. 2, pp. 95-101, 2015.
  9. S. Gelly, M. Schoenauer, M. Sebag, O. Teytaud, L. Kocsis, D. Silver and C. Szepesvari, "The Grand Challenge of Computer Go: Monte Carlo Tree Search and Extensions", Communications of the ACM, Vol. 55, No. 3, pp. 106-113, 2012.
  10. G. Chaslot, "Monte-Carlo Tree Search", Ph.D. dissertation, University of Masstricht, 2010.
  11. Wikipedia, "Computer Go", from http://en.wikipedia.org/wiki/Computer_Go, 2016.
  12. Wikipedia, "Tic-Tac-Toe", from http://en.wikipedia.org/wiki/Tic-Tac-Toe, 2016.
  13. A.A.J van der Kleij, "Monte Carlo Tree Search and Opponent Modeling through Player Clustering in no-limit Texas Hold'en Poker", Master thesis, University of Groningen, 2010. http://en.wikipedia.org/wiki/Tic-Tac-Toe, 2016.
  14. N. Sephton, P.I. Cowling, E. Powley and N.H. Slaven, "Heuristic Move Pruning in Monte Carlo Tree Search for the Strategic Card Game Lords of War", In Computational Intelligence and Games (CIG) of IEEE, pp. 1-7, 2014.
  15. T. Pepels, "Novel Selection Methods for Monte-Carlo Tree Search", Master thesis, University of Masstricht, 2014.
  16. D. Brand and S. Kroon, "Sample Evaluation for Action Selection in Monte Carlo Tree Search", from http://dl.acm.org/citation.cfm?doid=2664591.2664612, 2016.
  17. Y. Wang and S. Gelly, "Modification of UCT and sequence-like simulations for Monte-Carlo Go", from http://dept.stat.lsa.umich.edu/-yizwang/publications/wang07modifications.pdf, 2016.
  18. B.D. Lee, "Implementation of robust Tic-Tac-Toe game player, using enhanced Monte-Carlo algorithm", Journal of Korean Society for Computer Game, Vol. 28, No. 3, pp. 135-141, 2015.