DOI QR코드

DOI QR Code

Q-learning to improve learning speed using Minimax algorithm

미니맥스 알고리즘을 이용한 학습속도 개선을 위한 Q러닝

  • Shin, YongWoo (Division of Creative Convergence Education, Dong-Ah Institute of Media and Arts)
  • 신용우 (동아방송예술대학교 창의융합교양학부)
  • Received : 2018.07.18
  • Accepted : 2018.08.20
  • Published : 2018.08.20

Abstract

Board games have many game characters and many state spaces. Therefore, games must be long learning. This paper used reinforcement learning algorithm. But, there is weakness with reinforcement learning. At the beginning of learning, reinforcement learning has the drawback of slow learning speed. Therefore, we tried to improve the learning speed by using the heuristic using the knowledge of the problem domain considering the game tree when there is the same best value during learning. In order to compare the existing character the improved one. I produced a board game. So I compete with one-sided attacking character. Improved character attacked the opponent's one considering the game tree. As a result of experiment, improved character's capability was improved on learning speed.

보드게임에서는 많은 경우의 수의 말들과 많은 상태공간들을 가지고 있다. 그러므로 게임은 학습을 오래 하여야 한다. 본 논문에서는 Q러닝 알고리즘을 이용했다. 그러나 강화학습은 학습초기에 학습속도가 느려지는 단점이 있다. 그러므로 학습을 하는 동안에 같은 최선의 값이 있을 때, 게임트리를 고려한 문제영역의 지식을 활용한 휴리스틱을 사용하여 학습의 속도향상을 시도하였다. 기존 구현된 말과 개선하여 구현된 말을 비교하기 위하여 보드게임을 제작했다. 그래서 일방적으로 공격하는 말과 승부를 겨루게 하였다. 개선된 말은 게임트리를 고려하여 상대방 말을 공격하였다. 실험결과 개선하여 구현된 말이 학습속도적인 면에서 향상됨 것을 알 수 있었다.

Keywords

References

  1. Korea Creative Content Agency, "Content Industry Trend Analysis Report for 4Q 2016 (Game Industry)", 2017
  2. Richard Sutton, Andrew G. Barto, "Reinforcement Learning :An Introduction", MIT Press, Cambridge, MA, 1998.
  3. Imran Ghory, "Reinforcement learning in board games.", available at http://www.cs.bris.ac.uk/Publications/Papers/2000100.pdf, 2004.
  4. Nee Jan van Eck, Michiel van Wezel., "Reinforcement Learning and its Application to Othello", available at http://www.few.eur.nl/few/people/mvanwezel/rl.othello.ejor.pdf, 2004
  5. Yongwoo Shin, "Artificial Engine Development through Reinforcement Learning on Jul-Gonu Game ", Journal of Internet Computing and Services, Vol 10, No 1, pp93-99, 2009
  6. Yongwoo Shin, "An improvement of the learning speed through Influence Map on Reinforcement Learning", Journal of Korea Game Society, Vol 17, No 4, pp109-116, 2017 https://doi.org/10.7583/JKGS.2017.17.4.109
  7. Woosung Sim, "50 traditional games Korean folk play", Nonghyup, 1996
  8. Woosung Sim, "Korean folk play", Dongmoonsun, 1996
  9. Patrick Henry Winston, "Artificial Intelligence", Addison Wesley, 1993
  10. Sukin You, "Artificial Intelligence Fundamentals", Kyohaksa, 1988
  11. Tozour, Paul, "Influence Mapping", Game Programming Gems 2, Charles River, 2001
  12. Laramee, Francois Dominic, "A Rule-based Architecture Using Dempster-Shafer Theory", AI Game Programming Wisdom, Charles River Media, 2002.
  13. Mommersteeg, Fri, "Pattern Recognition with Sequential Prediction", AI Game Programming Wisdom, Charles River Media, 2002.
  14. Steve Woodcock, "Game AI : The State of the Industry", Game Developer Magazine, 2000.
  15. Steve Rabin, AI Game Programming Wisdom 2, Charles River Media, 2003
  16. Steve Rabin, AI Game Programming Wisdom, Charles River Media, 2002
  17. Laramee, Francois Dominic, "Using N-Gram Statistical Models to Predict Player Behavior", AI Game Programming Wisdom, Charles River Media, 2002.
  18. Andrew Kirmse, Game Programming Gems 4, Delmar Thomson Learning, 2004.
  19. Mark Deloura, Game Programming Gems 3, Charles River Media, 2002.
  20. Mark Deloura, Game Programming Gems 2, Charles River Media, 2001.