Challenges for implementing Monte Carlo Tree Search in commercial games
-
Upload
matthew-bedder -
Category
Software
-
view
35 -
download
2
Transcript of Challenges for implementing Monte Carlo Tree Search in commercial games
![Page 1: Challenges for implementing Monte Carlo Tree Search in commercial games](https://reader036.fdocuments.us/reader036/viewer/2022070513/5882c1261a28abb2478b6021/html5/thumbnails/1.jpg)
1
Click icon to add picture
Challenges for implementing Monte Carlo Tree Search in commercial games
Matthew Bedder
16/Sept/2016
![Page 2: Challenges for implementing Monte Carlo Tree Search in commercial games](https://reader036.fdocuments.us/reader036/viewer/2022070513/5882c1261a28abb2478b6021/html5/thumbnails/2.jpg)
216/Sept/2016
Disclaimer
This is pitched at people with little/no knowledge of the area
(Sorry, AI people)
![Page 3: Challenges for implementing Monte Carlo Tree Search in commercial games](https://reader036.fdocuments.us/reader036/viewer/2022070513/5882c1261a28abb2478b6021/html5/thumbnails/3.jpg)
3
“A game is a series of interesting choices”
16/Sept/2016
![Page 4: Challenges for implementing Monte Carlo Tree Search in commercial games](https://reader036.fdocuments.us/reader036/viewer/2022070513/5882c1261a28abb2478b6021/html5/thumbnails/4.jpg)
416/Sept/2016
![Page 5: Challenges for implementing Monte Carlo Tree Search in commercial games](https://reader036.fdocuments.us/reader036/viewer/2022070513/5882c1261a28abb2478b6021/html5/thumbnails/5.jpg)
5
Monte Carlo Tree Search
16/Sept/2016
![Page 6: Challenges for implementing Monte Carlo Tree Search in commercial games](https://reader036.fdocuments.us/reader036/viewer/2022070513/5882c1261a28abb2478b6021/html5/thumbnails/6.jpg)
616/Sept/2016
Monte Carlo Tree SearchOnly generate the bits of the game tree we need
![Page 7: Challenges for implementing Monte Carlo Tree Search in commercial games](https://reader036.fdocuments.us/reader036/viewer/2022070513/5882c1261a28abb2478b6021/html5/thumbnails/7.jpg)
716/Sept/2016
MCTS – the basics
Iteratively builds a decision tree
![Page 8: Challenges for implementing Monte Carlo Tree Search in commercial games](https://reader036.fdocuments.us/reader036/viewer/2022070513/5882c1261a28abb2478b6021/html5/thumbnails/8.jpg)
816/Sept/2016
MCTS Selection
![Page 9: Challenges for implementing Monte Carlo Tree Search in commercial games](https://reader036.fdocuments.us/reader036/viewer/2022070513/5882c1261a28abb2478b6021/html5/thumbnails/9.jpg)
916/Sept/2016
MCTS Expansion
![Page 10: Challenges for implementing Monte Carlo Tree Search in commercial games](https://reader036.fdocuments.us/reader036/viewer/2022070513/5882c1261a28abb2478b6021/html5/thumbnails/10.jpg)
1016/Sept/2016
MCTS Simulation
![Page 11: Challenges for implementing Monte Carlo Tree Search in commercial games](https://reader036.fdocuments.us/reader036/viewer/2022070513/5882c1261a28abb2478b6021/html5/thumbnails/11.jpg)
1116/Sept/2016
Monte Carlo evaluationsPerforming random actions gives us a (weak) approximation of state values
X O X OX
OX
O
X
![Page 12: Challenges for implementing Monte Carlo Tree Search in commercial games](https://reader036.fdocuments.us/reader036/viewer/2022070513/5882c1261a28abb2478b6021/html5/thumbnails/12.jpg)
1216/Sept/2016
MCTS Simulation
![Page 13: Challenges for implementing Monte Carlo Tree Search in commercial games](https://reader036.fdocuments.us/reader036/viewer/2022070513/5882c1261a28abb2478b6021/html5/thumbnails/13.jpg)
1316/Sept/2016
MCTS Backpropagation
![Page 14: Challenges for implementing Monte Carlo Tree Search in commercial games](https://reader036.fdocuments.us/reader036/viewer/2022070513/5882c1261a28abb2478b6021/html5/thumbnails/14.jpg)
1416/Sept/2016
MCTS – the basics
![Page 15: Challenges for implementing Monte Carlo Tree Search in commercial games](https://reader036.fdocuments.us/reader036/viewer/2022070513/5882c1261a28abb2478b6021/html5/thumbnails/15.jpg)
15
Applications of MCTS
16/Sept/2016
![Page 16: Challenges for implementing Monte Carlo Tree Search in commercial games](https://reader036.fdocuments.us/reader036/viewer/2022070513/5882c1261a28abb2478b6021/html5/thumbnails/16.jpg)
1616/Sept/2016
MCTS applications
• Amazons• Arimaa• Connect 4• Dou Di Zhu• Go variants• Havannah
• Hex• Morpion Solitaire• Othello• Settlers of Catan• Spades• ...
Many board and card games, including:
![Page 17: Challenges for implementing Monte Carlo Tree Search in commercial games](https://reader036.fdocuments.us/reader036/viewer/2022070513/5882c1261a28abb2478b6021/html5/thumbnails/17.jpg)
1716/Sept/2016
![Page 18: Challenges for implementing Monte Carlo Tree Search in commercial games](https://reader036.fdocuments.us/reader036/viewer/2022070513/5882c1261a28abb2478b6021/html5/thumbnails/18.jpg)
1816/Sept/2016
MCTS applicationsAll of the strongest AI players of Go use variants of MCTS
![Page 19: Challenges for implementing Monte Carlo Tree Search in commercial games](https://reader036.fdocuments.us/reader036/viewer/2022070513/5882c1261a28abb2478b6021/html5/thumbnails/19.jpg)
1916/Sept/2016
MCTS applicationsCurrently there are very limited applications to commercial video games or other domains
![Page 20: Challenges for implementing Monte Carlo Tree Search in commercial games](https://reader036.fdocuments.us/reader036/viewer/2022070513/5882c1261a28abb2478b6021/html5/thumbnails/20.jpg)
2016/Sept/2016
http://www.aifactory.co.uk/AIF_Games_Spades.htm
![Page 21: Challenges for implementing Monte Carlo Tree Search in commercial games](https://reader036.fdocuments.us/reader036/viewer/2022070513/5882c1261a28abb2478b6021/html5/thumbnails/21.jpg)
2116/Sept/2016
MCTS Successes
http://aigamedev.com/open/coverage/mcts-rome-ii/
![Page 22: Challenges for implementing Monte Carlo Tree Search in commercial games](https://reader036.fdocuments.us/reader036/viewer/2022070513/5882c1261a28abb2478b6021/html5/thumbnails/22.jpg)
2216/Sept/2016
http://gwaredd.github.io/nuclai_mcts
![Page 23: Challenges for implementing Monte Carlo Tree Search in commercial games](https://reader036.fdocuments.us/reader036/viewer/2022070513/5882c1261a28abb2478b6021/html5/thumbnails/23.jpg)
23
Challenges for MCTS
16/Sept/2016
![Page 24: Challenges for implementing Monte Carlo Tree Search in commercial games](https://reader036.fdocuments.us/reader036/viewer/2022070513/5882c1261a28abb2478b6021/html5/thumbnails/24.jpg)
2416/Sept/2016
Games are hard
There are as many states of Civilization II as there are atoms in the
observable universe 10,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,
000,000,000,000,000,000,000,000,000,000 times over
![Page 25: Challenges for implementing Monte Carlo Tree Search in commercial games](https://reader036.fdocuments.us/reader036/viewer/2022070513/5882c1261a28abb2478b6021/html5/thumbnails/25.jpg)
2516/Sept/2016
Computational costThis is actually important
Game AIs (usually) are limited to time sliced CPU
AlphaGo used• 1,920 CPUs and 280 GPUs• ~1MW
![Page 26: Challenges for implementing Monte Carlo Tree Search in commercial games](https://reader036.fdocuments.us/reader036/viewer/2022070513/5882c1261a28abb2478b6021/html5/thumbnails/26.jpg)
2616/Sept/2016
Monte Carlo Tree Search optimisationsGenerally either:
Domain specificor
Overly generic
![Page 27: Challenges for implementing Monte Carlo Tree Search in commercial games](https://reader036.fdocuments.us/reader036/viewer/2022070513/5882c1261a28abb2478b6021/html5/thumbnails/27.jpg)
2716/Sept/2016
Monte Carlo Tree Search optimisationsReplacing random simulations helps… usually
Adding prior knowledge helps… usually
![Page 28: Challenges for implementing Monte Carlo Tree Search in commercial games](https://reader036.fdocuments.us/reader036/viewer/2022070513/5882c1261a28abb2478b6021/html5/thumbnails/28.jpg)
2816/Sept/2016
Our approachDomain-specific optimisations for arbitrary domains
I’d love to talk about our work during/after coffee