Browse > Article
http://dx.doi.org/10.30693/SMJ.2020.9.4.36

Hyper-parameter Optimization for Monte Carlo Tree Search using Self-play  

Lee, Jin-Seon (Department of Information and Security, Woosuk University)
Oh, Il-Seok (Division of Computer Science and Engineering, Jeonbuk National University)
Publication Information
Smart Media Journal / v.9, no.4, 2020 , pp. 36-43 More about this Journal
Abstract
The Monte Carlo tree search (MCTS) is a popular method for implementing an intelligent game program. It has several hyper-parameters that require an optimization for showing the best performance. Due to the stochastic nature of the MCTS, the hyper-parameter optimization is difficult to solve. This paper uses the self-playing capability of the MCTS-based game program for optimizing the hyper-parameters. It seeks a winner path over the hyper-parameter space while performing the self-play. The top-q longest winners in the winner path compete for the final winner. The experiment using the 15-15-5 game (Omok in Korean name) showed a promising result.
Keywords
Machine learning; Hyper-parameter optimization; Monte Carlo tree search; m-n-k game;
Citations & Related Records
Times Cited By KSCI : 4  (Citation Analysis)
연도 인용수 순위
1 GyuHyeok Choi and Mijin Kim, "Analysis of Players' Eye-Movement Patterns by Playing Experience in FPS Game," Smart Media Journal, vol. 5, no. 2, pp.33-41, 2016
2 Browne, c., et al., "A survey of Monte Carlo tree search methods," IEEE Transactions on Computational Intelligence and AI in Games, vol. 4 Issue: 1, pp. 1-43, March 2012   DOI
3 Fu, M.C., "Monte Carlo tree search: a tutorial," Proceedings of 2018 Winter Simulation Conference, pp. 222-236, 2018
4 Brugmann, B., "Monte Carlo Go, Technical report," Department of Physics, Syracuse University, 1993
5 Coulom, R., "Efficient selectivity and backup operators in Monte-Carlo tree search," 5th International Conference on Computers and Games, pp. 72-83, 2006
6 Silver, D., et al., "Mastering the game of Go with deep neural networks and tree search," Nature 529, pp. 484-489, January 2016   DOI
7 Silver, D., et al., "Mastering the game of Go without human knowledge," Nature 550, pp. 354-359, 2017   DOI
8 Poole, D.L. and Mackworth, A.K., "Artificial Intelligence: Foundations of Computational Agents," Cambridge University Press, 2017
9 Goodfellow, I., Bengio, Y. and Courville, A., "Deep Learning", The MIT Press, 2016
10 Bergstra, J.S., Bardenet, R., "Bengio, Y. and Kegl, B., Algorithms for hyper-parameter optimization," Advances in neural information processing systems, pp. 2546-2554, 2011
11 Wang, H, Emmerich, M., Preuss and M., Plaat, A., "Analysis of hyper-parameter for small games: iterations or epochs in self-play?," arXiv:2003.05988v1, 2020
12 Ruijl, B., Vermaseren, J., Plaat, A. and Herik, J., "Combining simulated annealing and Monte Carlo Tree Search for expression simplification," Proceedings of the 6th International Conference on Agents and Artificial Intelligence, pp. 724-731, 2014
13 Walter, J.C. and Barkema, G.T., "An introduction to Monte Carlo methods," Physica A:Statistical Mechanics and its Applications, vol. 418, pp. 78-87, January 2015   DOI
14 Woo-Jin Joe, Hyo-Jeong Shin and Hyong-Shik Kim, "A log visualization method for network security monitoring," Smart Media Journal, vol. 7, no. 4, pp. 70-78, 2018
15 Dasom Seo, KangHan Oh, Il-Seok Oh and Tae-Woong Yoo, "Superpixel Exclusion-Inclusion Multiscale Approach for Explanations of Deep Learning," Smart Media Journal, vol. 8, no. 2, pp. 39-45, 2019
16 Li, L., et al., "Hyperband: a novel bandit-based approach to hyperparameter optimization," The Journal of Machine Learning Research, vol. 18, no. 1, pp. 1-52, January 2017
17 Rakotoarison, H., Schoenauer, M. and Sebag, M., "Automated machine learning with Monte Carlo tree search," IJCAI-19 28th International Joint Conference on Artificial Intelligence, pp. 3296-3303, Macau, China, Aug. 2019