Teaching AI to Play Chess - GitHub Pages · Teaching AI to Play Chess Like People Austin A....
Transcript of Teaching AI to Play Chess - GitHub Pages · Teaching AI to Play Chess Like People Austin A....
![Page 1: Teaching AI to Play Chess - GitHub Pages · Teaching AI to Play Chess Like People Austin A. Robinson University of Minnesota - Morris ... Development The Goal of Maia is to play the](https://reader035.fdocuments.us/reader035/viewer/2022070300/6147dc12a830d0442101b587/html5/thumbnails/1.jpg)
Teaching AI to Play Chess Like People
Austin A. RobinsonUniversity of Minnesota - Morris
Computer Science Senior SeminarSpring 2021
![Page 2: Teaching AI to Play Chess - GitHub Pages · Teaching AI to Play Chess Like People Austin A. Robinson University of Minnesota - Morris ... Development The Goal of Maia is to play the](https://reader035.fdocuments.us/reader035/viewer/2022070300/6147dc12a830d0442101b587/html5/thumbnails/2.jpg)
“If you program a machine, you know what it’s capable of. If the machine is programming itself, who knows what it might do?”
― Garry Kasparov, Deep Thinking: Where Machine Intelligence Ends and Human Creativity Begins
● Computer’s have fundamentallychanged chess
● Computers can serve a different purpose
2https://www.pri.org/stories/2018-01-05/garry-kasparov-and-game-artificial-intelligence
![Page 3: Teaching AI to Play Chess - GitHub Pages · Teaching AI to Play Chess Like People Austin A. Robinson University of Minnesota - Morris ... Development The Goal of Maia is to play the](https://reader035.fdocuments.us/reader035/viewer/2022070300/6147dc12a830d0442101b587/html5/thumbnails/3.jpg)
Outline
● Background● AlphaZero● Maia
○ Development○ Results○ Comparison
● Application● Conclusion
3
![Page 4: Teaching AI to Play Chess - GitHub Pages · Teaching AI to Play Chess Like People Austin A. Robinson University of Minnesota - Morris ... Development The Goal of Maia is to play the](https://reader035.fdocuments.us/reader035/viewer/2022070300/6147dc12a830d0442101b587/html5/thumbnails/4.jpg)
Background: Chess Terminology
● Elo rating● Blunder● Time Control
4https://ratings.fide.com/profile/1503014
![Page 5: Teaching AI to Play Chess - GitHub Pages · Teaching AI to Play Chess Like People Austin A. Robinson University of Minnesota - Morris ... Development The Goal of Maia is to play the](https://reader035.fdocuments.us/reader035/viewer/2022070300/6147dc12a830d0442101b587/html5/thumbnails/5.jpg)
Background: Chess Terminology
● Elo rating● Blunder● Time Control
4
![Page 6: Teaching AI to Play Chess - GitHub Pages · Teaching AI to Play Chess Like People Austin A. Robinson University of Minnesota - Morris ... Development The Goal of Maia is to play the](https://reader035.fdocuments.us/reader035/viewer/2022070300/6147dc12a830d0442101b587/html5/thumbnails/6.jpg)
Background: Chess Terminology
● Elo rating● Blunder● Time Control
4https://upload.wikimedia.org/wikipedia/commons/d/d3/Schachuhr_mechanisch.jpg
![Page 7: Teaching AI to Play Chess - GitHub Pages · Teaching AI to Play Chess Like People Austin A. Robinson University of Minnesota - Morris ... Development The Goal of Maia is to play the](https://reader035.fdocuments.us/reader035/viewer/2022070300/6147dc12a830d0442101b587/html5/thumbnails/7.jpg)
Background: Deep Learning
● Machine Learning○ “The hierarchy of concepts enables the computer to learn complicated concepts by building them out of
simpler ones. If we draw a graph showing how these concepts are built on top of each other, the graph is deep, with many layers. For this reason,we call this approach to AI deep learning." - Deep Learning, by Ian Goodfellow, Yoshua Bengio and Aaron Courville
● Deep Learning is allowing computers to learn from experience● Reinforcement learning
5https://insidebigdata.com/wp-content/uploads/2018/10/Neural-Network-diagram.jpg
![Page 8: Teaching AI to Play Chess - GitHub Pages · Teaching AI to Play Chess Like People Austin A. Robinson University of Minnesota - Morris ... Development The Goal of Maia is to play the](https://reader035.fdocuments.us/reader035/viewer/2022070300/6147dc12a830d0442101b587/html5/thumbnails/8.jpg)
Background: Deep Learning
● Machine Learning○ “The hierarchy of concepts enables the computer to learn complicated concepts by building them out of
simpler ones. If we draw a graph showing how these concepts are built on top of each other, the graph is deep, with many layers. For this reason,we call this approach to AI deep learning."- Deep Learning, by Ian Goodfellow, Yoshua Bengio and Aaron Courville
● Deep Learning is allowing computers to learn from experience● Reinforcement learning
5https://insidebigdata.com/wp-content/uploads/2018/10/Neural-Network-diagram.jpg
![Page 9: Teaching AI to Play Chess - GitHub Pages · Teaching AI to Play Chess Like People Austin A. Robinson University of Minnesota - Morris ... Development The Goal of Maia is to play the](https://reader035.fdocuments.us/reader035/viewer/2022070300/6147dc12a830d0442101b587/html5/thumbnails/9.jpg)
Background: Deep Learning
● Machine Learning○ “The hierarchy of concepts enables the computer to learn complicated concepts by building them out of
simpler ones. If we draw a graph showing how these concepts are built on top of each other, the graph is deep, with many layers. For this reason,we call this approach to AI deep learning."- Deep Learning, by Ian Goodfellow, Yoshua Bengio and Aaron Courville
● Deep Learning is allowing computers to learn from experience● Reinforcement learning
5https://miro.medium.com/max/478/1*QVsnwatDVz8wcqJUsLJejw.png
![Page 10: Teaching AI to Play Chess - GitHub Pages · Teaching AI to Play Chess Like People Austin A. Robinson University of Minnesota - Morris ... Development The Goal of Maia is to play the](https://reader035.fdocuments.us/reader035/viewer/2022070300/6147dc12a830d0442101b587/html5/thumbnails/10.jpg)
Background: Monte Carlo Tree Search(MCTS)
● MCTS is a tree search that also implements machine learning principles of reinforcement learning
● 4 primary steps○ Simulation
6[3]
![Page 11: Teaching AI to Play Chess - GitHub Pages · Teaching AI to Play Chess Like People Austin A. Robinson University of Minnesota - Morris ... Development The Goal of Maia is to play the](https://reader035.fdocuments.us/reader035/viewer/2022070300/6147dc12a830d0442101b587/html5/thumbnails/11.jpg)
Background: Monte Carlo Tree Search(MCTS)
● MCTS is a tree search that also implements machine learning principles of reinforcement learning
● 4 primary steps○ Simulation
6[3]
![Page 12: Teaching AI to Play Chess - GitHub Pages · Teaching AI to Play Chess Like People Austin A. Robinson University of Minnesota - Morris ... Development The Goal of Maia is to play the](https://reader035.fdocuments.us/reader035/viewer/2022070300/6147dc12a830d0442101b587/html5/thumbnails/12.jpg)
Background: Chess Engines
● Software that is used to generate and analyse positions● Stockfish
○ More traditional chess engine○ 3564
● AlphaZero and Leela○ Deep Learning○ Monte Carlo Tree Search○ 3463
7
![Page 13: Teaching AI to Play Chess - GitHub Pages · Teaching AI to Play Chess Like People Austin A. Robinson University of Minnesota - Morris ... Development The Goal of Maia is to play the](https://reader035.fdocuments.us/reader035/viewer/2022070300/6147dc12a830d0442101b587/html5/thumbnails/13.jpg)
Background: Chess Engines
● Software that is used to generate and analyse positions● Stockfish
○ More traditional chess engine○ 3564
● AlphaZero and Leela○ Deep Learning○ Monte Carlo Tree Search○ 3463
7https://stockfishchess.org/
![Page 14: Teaching AI to Play Chess - GitHub Pages · Teaching AI to Play Chess Like People Austin A. Robinson University of Minnesota - Morris ... Development The Goal of Maia is to play the](https://reader035.fdocuments.us/reader035/viewer/2022070300/6147dc12a830d0442101b587/html5/thumbnails/14.jpg)
Background: Chess Engines
● Software that is used to generate and analyse positions● Stockfish
○ More traditional chess engine○ 3564
● AlphaZero and Leela○ Deep Learning○ Monte Carlo Tree Search○ 3463
7https://lczero.org/blog/
![Page 15: Teaching AI to Play Chess - GitHub Pages · Teaching AI to Play Chess Like People Austin A. Robinson University of Minnesota - Morris ... Development The Goal of Maia is to play the](https://reader035.fdocuments.us/reader035/viewer/2022070300/6147dc12a830d0442101b587/html5/thumbnails/15.jpg)
AlphaZero
8
![Page 16: Teaching AI to Play Chess - GitHub Pages · Teaching AI to Play Chess Like People Austin A. Robinson University of Minnesota - Morris ... Development The Goal of Maia is to play the](https://reader035.fdocuments.us/reader035/viewer/2022070300/6147dc12a830d0442101b587/html5/thumbnails/16.jpg)
AlphaZero: Architecture
● Inputs● Output
○ Probabilities○ Expected outcome○ Values
● Parameters● Obtaining Outputs
○ Reinforcement through self play
9
[2]
![Page 17: Teaching AI to Play Chess - GitHub Pages · Teaching AI to Play Chess Like People Austin A. Robinson University of Minnesota - Morris ... Development The Goal of Maia is to play the](https://reader035.fdocuments.us/reader035/viewer/2022070300/6147dc12a830d0442101b587/html5/thumbnails/17.jpg)
AlphaZero: Architecture
● Inputs● Output
○ Probabilities○ Expected outcomes○ Values
● Parameters● Obtaining Outputs
○ Reinforcement through self play
9
[2]
![Page 18: Teaching AI to Play Chess - GitHub Pages · Teaching AI to Play Chess Like People Austin A. Robinson University of Minnesota - Morris ... Development The Goal of Maia is to play the](https://reader035.fdocuments.us/reader035/viewer/2022070300/6147dc12a830d0442101b587/html5/thumbnails/18.jpg)
AlphaZero: Architecture
● Inputs● Output
○ Probabilities○ Expected outcomes○ Values
● Parameters● Obtaining Outputs
○ Reinforcement through self play
9
[2]
![Page 19: Teaching AI to Play Chess - GitHub Pages · Teaching AI to Play Chess Like People Austin A. Robinson University of Minnesota - Morris ... Development The Goal of Maia is to play the](https://reader035.fdocuments.us/reader035/viewer/2022070300/6147dc12a830d0442101b587/html5/thumbnails/19.jpg)
AlphaZero: Architecture
● Inputs● Output
○ Probabilities○ Expected outcomes○ Values
● Parameters● Obtaining Outputs
○ Reinforcement through self play
9
[2]
![Page 20: Teaching AI to Play Chess - GitHub Pages · Teaching AI to Play Chess Like People Austin A. Robinson University of Minnesota - Morris ... Development The Goal of Maia is to play the](https://reader035.fdocuments.us/reader035/viewer/2022070300/6147dc12a830d0442101b587/html5/thumbnails/20.jpg)
AlphaZero: Architecture
● Inputs● Output
○ Probabilities○ Expected outcomes○ Values
● Parameters● Obtaining Outputs
○ Reinforcement through self play
9
[2]
![Page 21: Teaching AI to Play Chess - GitHub Pages · Teaching AI to Play Chess Like People Austin A. Robinson University of Minnesota - Morris ... Development The Goal of Maia is to play the](https://reader035.fdocuments.us/reader035/viewer/2022070300/6147dc12a830d0442101b587/html5/thumbnails/21.jpg)
AlphaZero: Architecture
● Inputs● Output
○ Probabilities○ Expected outcomes○ Values
● Parameters● Obtaining Outputs
○ Reinforcement through self play
9
[2]
![Page 22: Teaching AI to Play Chess - GitHub Pages · Teaching AI to Play Chess Like People Austin A. Robinson University of Minnesota - Morris ... Development The Goal of Maia is to play the](https://reader035.fdocuments.us/reader035/viewer/2022070300/6147dc12a830d0442101b587/html5/thumbnails/22.jpg)
AlphaZero: Training
● 9 hours of training time○ Reached an Elo rating of ~3200
● Completely random at beginning of the training● Learned from win, losses and draws
10https://deepmind.com/blog/article/alphazero-shedding-new-light-grand-games-chess-shogi-and-go
![Page 23: Teaching AI to Play Chess - GitHub Pages · Teaching AI to Play Chess Like People Austin A. Robinson University of Minnesota - Morris ... Development The Goal of Maia is to play the](https://reader035.fdocuments.us/reader035/viewer/2022070300/6147dc12a830d0442101b587/html5/thumbnails/23.jpg)
AlphaZero: Training
● 9 hours of training time○ Reached an Elo rating of ~3200
● Completely random at beginning of the training● Learned from win, losses and draws
10https://deepmind.com/blog/article/alphazero-shedding-new-light-grand-games-chess-shogi-and-go
![Page 24: Teaching AI to Play Chess - GitHub Pages · Teaching AI to Play Chess Like People Austin A. Robinson University of Minnesota - Morris ... Development The Goal of Maia is to play the](https://reader035.fdocuments.us/reader035/viewer/2022070300/6147dc12a830d0442101b587/html5/thumbnails/24.jpg)
AlphaZero: Training
● 9 hours of training time○ Reached an Elo rating of ~3200
● Completely random at beginning of the training● Learned from win, losses and draws
10https://deepmind.com/blog/article/alphazero-shedding-new-light-grand-games-chess-shogi-and-go
![Page 25: Teaching AI to Play Chess - GitHub Pages · Teaching AI to Play Chess Like People Austin A. Robinson University of Minnesota - Morris ... Development The Goal of Maia is to play the](https://reader035.fdocuments.us/reader035/viewer/2022070300/6147dc12a830d0442101b587/html5/thumbnails/25.jpg)
AlphaZero: Results
● Defeated Stockfish (won 155 games, drew 839 games and lost 6 games)○ Leela was also able to defeat the same version of StockFish
● Strategies Learned by AlphaZero○ Common human strategies○ Unique to AlphaZero
11[3]
![Page 26: Teaching AI to Play Chess - GitHub Pages · Teaching AI to Play Chess Like People Austin A. Robinson University of Minnesota - Morris ... Development The Goal of Maia is to play the](https://reader035.fdocuments.us/reader035/viewer/2022070300/6147dc12a830d0442101b587/html5/thumbnails/26.jpg)
AlphaZero: Results
● Defeated Stockfish ( won 155 games, drew 839 games and lost 6 games)○ Leela was also able to defeat the same version of StockFish
● Strategies Learned by AlphaZero○ Common human strategies○ Unique to AlphaZero
11ichess.net
![Page 27: Teaching AI to Play Chess - GitHub Pages · Teaching AI to Play Chess Like People Austin A. Robinson University of Minnesota - Morris ... Development The Goal of Maia is to play the](https://reader035.fdocuments.us/reader035/viewer/2022070300/6147dc12a830d0442101b587/html5/thumbnails/27.jpg)
Maia
12
![Page 28: Teaching AI to Play Chess - GitHub Pages · Teaching AI to Play Chess Like People Austin A. Robinson University of Minnesota - Morris ... Development The Goal of Maia is to play the](https://reader035.fdocuments.us/reader035/viewer/2022070300/6147dc12a830d0442101b587/html5/thumbnails/28.jpg)
Maia: Development
● The Goal of Maia is to play the most like a human● Maia utilizes a large amount of code from Leela● Data Sets● Move Prediction● Models of Maia
13[4]
![Page 29: Teaching AI to Play Chess - GitHub Pages · Teaching AI to Play Chess Like People Austin A. Robinson University of Minnesota - Morris ... Development The Goal of Maia is to play the](https://reader035.fdocuments.us/reader035/viewer/2022070300/6147dc12a830d0442101b587/html5/thumbnails/29.jpg)
Maia: Development
● The Goal of Maia is to play the most like a human● Maia utilizes a large amount of code from Leela● Data Sets● Move Prediction● Models of Maia
13[4]
![Page 30: Teaching AI to Play Chess - GitHub Pages · Teaching AI to Play Chess Like People Austin A. Robinson University of Minnesota - Morris ... Development The Goal of Maia is to play the](https://reader035.fdocuments.us/reader035/viewer/2022070300/6147dc12a830d0442101b587/html5/thumbnails/30.jpg)
Maia: Development
● The Goal of Maia is to play the most like a human● Maia utilizes a large amount of code from Leela● Data Sets● Move Prediction● Models of Maia
13[4]
![Page 31: Teaching AI to Play Chess - GitHub Pages · Teaching AI to Play Chess Like People Austin A. Robinson University of Minnesota - Morris ... Development The Goal of Maia is to play the](https://reader035.fdocuments.us/reader035/viewer/2022070300/6147dc12a830d0442101b587/html5/thumbnails/31.jpg)
Maia: Development
● The Goal of Maia is to play the most like a human● Maia utilizes a large amount of code from Leela● Data Sets● Move Prediction● Models of Maia
13[4]
![Page 32: Teaching AI to Play Chess - GitHub Pages · Teaching AI to Play Chess Like People Austin A. Robinson University of Minnesota - Morris ... Development The Goal of Maia is to play the](https://reader035.fdocuments.us/reader035/viewer/2022070300/6147dc12a830d0442101b587/html5/thumbnails/32.jpg)
Maia: Development
● The Goal of Maia is to play the most like a human● Maia utilizes a large amount of code from Leela● Data Sets● Move Prediction● Models of Maia
13[4]
![Page 33: Teaching AI to Play Chess - GitHub Pages · Teaching AI to Play Chess Like People Austin A. Robinson University of Minnesota - Morris ... Development The Goal of Maia is to play the](https://reader035.fdocuments.us/reader035/viewer/2022070300/6147dc12a830d0442101b587/html5/thumbnails/33.jpg)
Maia: Results
● Overall results should an accuracy around 50%● Maia was able to show that different ELO ranges have unique play styles● Everyone plays Chess differently; Maia is the average player for an ELO
range● Stockfish● Leela
14
[1]
![Page 34: Teaching AI to Play Chess - GitHub Pages · Teaching AI to Play Chess Like People Austin A. Robinson University of Minnesota - Morris ... Development The Goal of Maia is to play the](https://reader035.fdocuments.us/reader035/viewer/2022070300/6147dc12a830d0442101b587/html5/thumbnails/34.jpg)
Maia: Results
● Overall results should an accuracy around 50%● Maia was able to show that different ELO ranges have unique play styles● Everyone plays Chess differently; Maia is the average player for an ELO
range● Stockfish● Leela
14
[1]
![Page 35: Teaching AI to Play Chess - GitHub Pages · Teaching AI to Play Chess Like People Austin A. Robinson University of Minnesota - Morris ... Development The Goal of Maia is to play the](https://reader035.fdocuments.us/reader035/viewer/2022070300/6147dc12a830d0442101b587/html5/thumbnails/35.jpg)
Maia: Results
● Overall results should an accuracy around 50%● Maia was able to show that different ELO ranges have unique play styles● Everyone plays Chess differently; Maia is the average player for an ELO
range● Stockfish● Leela
14
[1]
![Page 36: Teaching AI to Play Chess - GitHub Pages · Teaching AI to Play Chess Like People Austin A. Robinson University of Minnesota - Morris ... Development The Goal of Maia is to play the](https://reader035.fdocuments.us/reader035/viewer/2022070300/6147dc12a830d0442101b587/html5/thumbnails/36.jpg)
Maia: Results
● Overall results should an accuracy around 50%● Maia was able to show that different ELO ranges have unique play styles● Everyone plays Chess differently; Maia is the average player for an ELO
range● Stockfish● Leela
14[1]
![Page 37: Teaching AI to Play Chess - GitHub Pages · Teaching AI to Play Chess Like People Austin A. Robinson University of Minnesota - Morris ... Development The Goal of Maia is to play the](https://reader035.fdocuments.us/reader035/viewer/2022070300/6147dc12a830d0442101b587/html5/thumbnails/37.jpg)
Maia: Results
● Overall results should an accuracy around 50%● Maia was able to show that different ELO ranges have unique play styles● Everyone plays Chess differently; Maia is the average player for an ELO
range● Stockfish● Leela
14[1]
![Page 38: Teaching AI to Play Chess - GitHub Pages · Teaching AI to Play Chess Like People Austin A. Robinson University of Minnesota - Morris ... Development The Goal of Maia is to play the](https://reader035.fdocuments.us/reader035/viewer/2022070300/6147dc12a830d0442101b587/html5/thumbnails/38.jpg)
Applications
● In Chess○ Hundreds of Thousand of daily online chess players○ Median Elo Rating 1500○ Training
● Human and AI interactions○ Harvard two-player Atari study○ Through Deep Learning were able to produce AI that helped improve human performance
15
![Page 39: Teaching AI to Play Chess - GitHub Pages · Teaching AI to Play Chess Like People Austin A. Robinson University of Minnesota - Morris ... Development The Goal of Maia is to play the](https://reader035.fdocuments.us/reader035/viewer/2022070300/6147dc12a830d0442101b587/html5/thumbnails/39.jpg)
Applications
● In Chess○ Hundreds of Thousand of daily online chess players○ Median Elo Rating(Lichess) 1500○ Training
● Human and AI interactions○ Harvard two-player Atari study○ Through Deep Learning were able to produce AI that helped improve human performance
15
![Page 40: Teaching AI to Play Chess - GitHub Pages · Teaching AI to Play Chess Like People Austin A. Robinson University of Minnesota - Morris ... Development The Goal of Maia is to play the](https://reader035.fdocuments.us/reader035/viewer/2022070300/6147dc12a830d0442101b587/html5/thumbnails/40.jpg)
Applications
● In Chess○ Hundreds of Thousand of daily online chess players○ Median Elo Rating 1500○ Training
● Human and AI interactions○ Harvard two-player Atari study○ Through Deep Learning were able to produce AI that helped improve human performance
15
![Page 41: Teaching AI to Play Chess - GitHub Pages · Teaching AI to Play Chess Like People Austin A. Robinson University of Minnesota - Morris ... Development The Goal of Maia is to play the](https://reader035.fdocuments.us/reader035/viewer/2022070300/6147dc12a830d0442101b587/html5/thumbnails/41.jpg)
Applications
● In Chess○ Hundreds of Thousand of daily online chess players○ Median Elo Rating 1500○ Training
● Human and AI interactions○ Harvard two-player Atari study○ Through Deep Learning were able to produce AI that helped improve human
performance
15
![Page 42: Teaching AI to Play Chess - GitHub Pages · Teaching AI to Play Chess Like People Austin A. Robinson University of Minnesota - Morris ... Development The Goal of Maia is to play the](https://reader035.fdocuments.us/reader035/viewer/2022070300/6147dc12a830d0442101b587/html5/thumbnails/42.jpg)
Conclusion
● Maia shows it possible to capture the play style of people● Development of more human like chess engines could lead to better training
for people
17
![Page 43: Teaching AI to Play Chess - GitHub Pages · Teaching AI to Play Chess Like People Austin A. Robinson University of Minnesota - Morris ... Development The Goal of Maia is to play the](https://reader035.fdocuments.us/reader035/viewer/2022070300/6147dc12a830d0442101b587/html5/thumbnails/43.jpg)
Questions
18
![Page 44: Teaching AI to Play Chess - GitHub Pages · Teaching AI to Play Chess Like People Austin A. Robinson University of Minnesota - Morris ... Development The Goal of Maia is to play the](https://reader035.fdocuments.us/reader035/viewer/2022070300/6147dc12a830d0442101b587/html5/thumbnails/44.jpg)
Sources
[1] Aligning Superhuman AI with Human Behavior: Chess as a Model System-Young, Sen, Kleinberg, Anderson
[2] Mastering Chess and Shogi by Self-Play with a General Reinforcement Learning Algorithm - Silver
[3] https://www.geeksforgeeks.org/ml-monte-carlo-tree-search-mcts/
[4] https://www.microsoft.com/en-us/research/blog/the-human-side-of-ai-for-chess/?OCID=msr_blog_MaiaChess_tw
19