General Game Playing (GGP)
description
Transcript of General Game Playing (GGP)
![Page 1: General Game Playing (GGP)](https://reader036.fdocuments.us/reader036/viewer/2022081502/56815b83550346895dc986c2/html5/thumbnails/1.jpg)
Marco AdelfioCMSC 828N – Spring 2009
General Game Playing (GGP)
![Page 2: General Game Playing (GGP)](https://reader036.fdocuments.us/reader036/viewer/2022081502/56815b83550346895dc986c2/html5/thumbnails/2.jpg)
Classic Game Playing AI
Deep Blue
TD-Gammon
Poki
![Page 3: General Game Playing (GGP)](https://reader036.fdocuments.us/reader036/viewer/2022081502/56815b83550346895dc986c2/html5/thumbnails/3.jpg)
General Game Playing AI
GGP Agent
![Page 4: General Game Playing (GGP)](https://reader036.fdocuments.us/reader036/viewer/2022081502/56815b83550346895dc986c2/html5/thumbnails/4.jpg)
General Game PlayingGGP Goals:
Create systems to play arbitrary games (given formal game definitions)
Eliminate game-specific strategiesEmphasize generic strategy formulation
Competition created by Stanford Logic GroupHosted during AAAI conference since 2005$10,000 Grand Prize
![Page 5: General Game Playing (GGP)](https://reader036.fdocuments.us/reader036/viewer/2022081502/56815b83550346895dc986c2/html5/thumbnails/5.jpg)
General Game PlayingQuestions:
What additional challenges arise for GGP agents?
How should a GGP agent evaluate game states?
Can a GGP agent transfer knowledge between games?
![Page 6: General Game Playing (GGP)](https://reader036.fdocuments.us/reader036/viewer/2022081502/56815b83550346895dc986c2/html5/thumbnails/6.jpg)
General Game PlayingFinitely many players, statesGame play controlled by Game Manager
over networkPlayers act synchronously (noops allowed)Time limits enforcedBasic agent must:
Understand rule specificationRespond to game states with legal actionsRecognize a terminal state and its payoffs
![Page 7: General Game Playing (GGP)](https://reader036.fdocuments.us/reader036/viewer/2022081502/56815b83550346895dc986c2/html5/thumbnails/7.jpg)
Game Definition LanguageA game definition must logically define:
Set of states in the gameLegal actions for each player from a given
game stateTransition functionInitial stateTerminal states and their payoffs
![Page 8: General Game Playing (GGP)](https://reader036.fdocuments.us/reader036/viewer/2022081502/56815b83550346895dc986c2/html5/thumbnails/8.jpg)
Game Definition Language - Example(role p1)
(role p2)
(init (cell 1 1 b))
(init (cell 1 2 b))
…
(init (control p1)
…
(<= (legal ?w (mark ?x ?y))
(true (cell ?x ?y b))
(true (control ?w)))
…
(<= (next (cell ?m ?n x))
(does p1 (mark ?m ?n))
(true (cell ?m ?n b)))
…
(<= (row ?m ?x)
(true (cell ?m 1 ?x))
(true (cell ?m 2 ?x))
(true (cell ?m 3 ?x)))
…
(<= (line ?x) (row ?m ?x))
(<= (line ?x) (column ?m ?x))
(<= (line ?x) (diagonal ?x))
…
(<= (goal p1 100)
(line x))
(<= (goal p1 0)
(line o)
…
(<= terminal
(line x))
![Page 9: General Game Playing (GGP)](https://reader036.fdocuments.us/reader036/viewer/2022081502/56815b83550346895dc986c2/html5/thumbnails/9.jpg)
Game CommunicationGame Manager Message Game Player Response(START MATCH.435 WHITE description 90 30)
READY
(PLAY MATCH.435 (NIL NIL)) (MARK 2 2)(PLAY MATCH.435 ((MARK 2 2) NOOP)))
NOOP
(PLAY MATCH.435 (NOOP (MARK 1 3))
(MARK 1 2)
(PLAY MATCH.435 ((MARK 1 2) NOOP))
NOOP
... ...(STOP MATCH.435 ((MARK 3 3) NOOP)
DONE
![Page 10: General Game Playing (GGP)](https://reader036.fdocuments.us/reader036/viewer/2022081502/56815b83550346895dc986c2/html5/thumbnails/10.jpg)
General Game PlayingDesign Challenges:
IndeterminacySizeMulti-game CommonalitiesOpponent Recognition
![Page 11: General Game Playing (GGP)](https://reader036.fdocuments.us/reader036/viewer/2022081502/56815b83550346895dc986c2/html5/thumbnails/11.jpg)
AAAI Competition – Past Winners2005 - ClunePlayer (UCLA)2006 - FluxPlayer (Technical University of
Dresden)2007 - CADIA (Reykjavik University)2008 - CADIA (Reykjavik University)
![Page 12: General Game Playing (GGP)](https://reader036.fdocuments.us/reader036/viewer/2022081502/56815b83550346895dc986c2/html5/thumbnails/12.jpg)
Agent 1: ClunePlayerApproach: MinimaxProblem: Needs to assign values to
intermediate game states in arbitrary games.Solution:
1. Calculate a vector of generic features at each node
2. Simulate games to determine which features are “stable” and correlated with either payoff or control
3. When running minimax, use a combination of those scores as the evaluation heuristic
![Page 13: General Game Playing (GGP)](https://reader036.fdocuments.us/reader036/viewer/2022081502/56815b83550346895dc986c2/html5/thumbnails/13.jpg)
Agent 2: CADIA-PlayerApproach: UCT (Variant of Monte Carlo
simulation)Monte Carlo:
Pick random actions for each player to descend the tree
After reaching a terminal state, update expected payoff Q(s,a) for each visited state s and action a
Introduces explore/exploit tradeoff
![Page 14: General Game Playing (GGP)](https://reader036.fdocuments.us/reader036/viewer/2022081502/56815b83550346895dc986c2/html5/thumbnails/14.jpg)
Agent 2: CADIA-PlayerUCT (Upper Confidence
bound for Trees)Balance exploration
and exploitationGive “bonus” to less
travelled paths
![Page 15: General Game Playing (GGP)](https://reader036.fdocuments.us/reader036/viewer/2022081502/56815b83550346895dc986c2/html5/thumbnails/15.jpg)
Agent 3: UTexas LARGApproach: Knowledge TransferUses lessons from past games to improve
play in new gamesWar Games!
Determines whether a new game is isomorphic or similar to a previous game. If so, transfer estimated rewards
![Page 16: General Game Playing (GGP)](https://reader036.fdocuments.us/reader036/viewer/2022081502/56815b83550346895dc986c2/html5/thumbnails/16.jpg)
SummaryGeneral Game Playing introduces a
different set of challenges than designing game-specific AI
Biggest challenge is evaluating states in a novel game
Better understanding of general strategy formation has many applications
![Page 17: General Game Playing (GGP)](https://reader036.fdocuments.us/reader036/viewer/2022081502/56815b83550346895dc986c2/html5/thumbnails/17.jpg)
ReferencesGGP Website: http://games.stanford.edu/Hilmar Finnson.
CADIA-Player: A General Game Playing Agent. MSc Thesis, School of Computer Science, Reykjavik University. 2007.
Kuhlmann, Gregory and Peter Stone. Graph-Based Domain Mapping for Transfer Learning in General Games. Lecture Notes in Computer Science, Volume 4701/2007.