Games and Adversarial Search Problems. - Jarrar · Mustafa Jarrar: Lecture Notes on Games and...

76
Jarrar © 2018 1 Mustafa Jarrar: Lecture Notes on Games and Adversarial Search Problems. Birzeit University, Palestine. 2018 Games and Adversarial Search Problems Version 4 (Chapter 5) Mustafa Jarrar Birzeit University [email protected] www.jarrar.info

Transcript of Games and Adversarial Search Problems. - Jarrar · Mustafa Jarrar: Lecture Notes on Games and...

Page 1: Games and Adversarial Search Problems. - Jarrar · Mustafa Jarrar: Lecture Notes on Games and Adversarial Search Problems. Birzeit University, Palestine. 2018 Games and Adversarial

Jarrar © 2018 1

Mustafa Jarrar: Lecture Notes on Games and Adversarial Search Problems.Birzeit University, Palestine. 2018

Gamesand Adversarial Search Problems

Version 4

(Chapter 5)

Mustafa JarrarBirzeit [email protected]

Page 2: Games and Adversarial Search Problems. - Jarrar · Mustafa Jarrar: Lecture Notes on Games and Adversarial Search Problems. Birzeit University, Palestine. 2018 Games and Adversarial

Jarrar © 2018 2

Watch this lecture and download the slides

Acknowledgement: This lecture is based on (but not limited to) chapter 5 in “S. Russell and P. Norvig: Artificial Intelligence: A Modern Approach”.

Download: http://www.jarrar.info/courses/AI/Jarrar.LectureNotes.Ch3.Games.pdfMore Courses: http://www.jarrar.info/courses/

Page 3: Games and Adversarial Search Problems. - Jarrar · Mustafa Jarrar: Lecture Notes on Games and Adversarial Search Problems. Birzeit University, Palestine. 2018 Games and Adversarial

Jarrar © 2018 3

Can you plan ahead with these games

Page 4: Games and Adversarial Search Problems. - Jarrar · Mustafa Jarrar: Lecture Notes on Games and Adversarial Search Problems. Birzeit University, Palestine. 2018 Games and Adversarial

Jarrar © 2018 4

Game Tree (2-player, deterministic, turns)

Calculated by utility function,depends on the game.

Last state,game is over

How to see the game as a tree

Image from [2]

Page 5: Games and Adversarial Search Problems. - Jarrar · Mustafa Jarrar: Lecture Notes on Games and Adversarial Search Problems. Birzeit University, Palestine. 2018 Games and Adversarial

Jarrar © 2018 5

Your Moves

My Moves

My Moves

Your Moves Your Moves

My Moves My MovesMy Moves

• Two players take turns making moves

• Board state fully known, deterministic evaluation of moves

• One player wins by defeating the other (or else there is a tie)

• Want a strategy to win, assuming the other person plays as well as possible

Two-Person Perfect Information Deterministic Game

Page 6: Games and Adversarial Search Problems. - Jarrar · Mustafa Jarrar: Lecture Notes on Games and Adversarial Search Problems. Birzeit University, Palestine. 2018 Games and Adversarial

Jarrar © 2018 6

Computer Games

Playing games can be seen as a Search Problem

Multiplayer games as multi-agent environments.

Agents' goals are in conflict.

Mostly deterministic and fully observable environments.

Some games are not trivial search problems, thus needs AI techniques, e.g. Chess has an average branching factor of 35, and games often go to 50 moves by each player, so the search tree has about 35100or 10154

nodes.

Finding optimal move: choosing a good move with time limits.

Heuristic evaluation functions allow us to approximate the true utility of a state without doing a complete search.

Page 7: Games and Adversarial Search Problems. - Jarrar · Mustafa Jarrar: Lecture Notes on Games and Adversarial Search Problems. Birzeit University, Palestine. 2018 Games and Adversarial

Jarrar © 2018 7

Minimax

Create a utility function– Evaluation of board/game state to determine how strong the

position of player 1 is.– Player 1 wants to maximize the utility function– Player 2 wants to minimize the utility function

Minimax Tree– Generate a new level for each move– Levels alternate between �max� (player 1 moves) and �min�

(player 2 moves)

Page 8: Games and Adversarial Search Problems. - Jarrar · Mustafa Jarrar: Lecture Notes on Games and Adversarial Search Problems. Birzeit University, Palestine. 2018 Games and Adversarial

Jarrar © 2018 8

Minimax Tree

Max

Min

Max

Min

You are Max and your enemy is Min.You play with your enemy in this way.

Page 9: Games and Adversarial Search Problems. - Jarrar · Mustafa Jarrar: Lecture Notes on Games and Adversarial Search Problems. Birzeit University, Palestine. 2018 Games and Adversarial

Jarrar © 2018 9

Minimax Tree Evaluation

Assign utility values to leaves

– Sometimes called �board evaluation function�

– If leaf is a �final� state, assign the maximum or minimum possible utility value (depending on who would win).

– If leaf is not a �final� state, must use some other heuristic, specific to the game, to evaluate how good/bad the state is at that point

Page 10: Games and Adversarial Search Problems. - Jarrar · Mustafa Jarrar: Lecture Notes on Games and Adversarial Search Problems. Birzeit University, Palestine. 2018 Games and Adversarial

Jarrar © 2018 10

Minimax Tree

Max

Min

Max

Min

100

-24-8-14-73-100-5-70-12-470412-3212823

Terminal nodes: values calculated from the utility function, evaluates how good/bad the state is at this point

Page 11: Games and Adversarial Search Problems. - Jarrar · Mustafa Jarrar: Lecture Notes on Games and Adversarial Search Problems. Birzeit University, Palestine. 2018 Games and Adversarial

Jarrar © 2018 11

Minimax Tree Evaluation

For the MAX player

1. Generate the game as deep as time permits2. Apply the evaluation function to the leaf states

3. Back-up values• At MIN assign minimum payoff move

• At MAX assign maximum payoff move

4. At root, MAX chooses the operator that led to the highest payoff

Page 12: Games and Adversarial Search Problems. - Jarrar · Mustafa Jarrar: Lecture Notes on Games and Adversarial Search Problems. Birzeit University, Palestine. 2018 Games and Adversarial

Jarrar © 2018 12

Minimax Tree

-24-8-14-73-100-5-70-12-470412-3212823

Max

Min

Max

Min

Terminal nodes: values calculated from the utility function

Page 13: Games and Adversarial Search Problems. - Jarrar · Mustafa Jarrar: Lecture Notes on Games and Adversarial Search Problems. Birzeit University, Palestine. 2018 Games and Adversarial

Jarrar © 2018 13

Minimax Tree

100

-24-8-14-73-100-5-70-12-470412-3212823

28 -3 12 70 -4 -73 -14 -8

Max

Min

Max

Min

Other nodes: values calculated via minimax algorithm

Page 14: Games and Adversarial Search Problems. - Jarrar · Mustafa Jarrar: Lecture Notes on Games and Adversarial Search Problems. Birzeit University, Palestine. 2018 Games and Adversarial

Jarrar © 2018 14

Minimax Tree

100

-24-8-14-73-100-5-70-12-470412-3212823

28 -3 12 70 -4 -73 -14 -8

-3 -4 -73

Max

Min

Max

Min

Page 15: Games and Adversarial Search Problems. - Jarrar · Mustafa Jarrar: Lecture Notes on Games and Adversarial Search Problems. Birzeit University, Palestine. 2018 Games and Adversarial

Jarrar © 2018 15

Minimax Tree

100

-24-8-14-73-100-5-70-12-470412-3212823

21 -3 12 70 -4 -73 -14 -8

-3 -4 -73

-3Max

Min

Max

Min

Page 16: Games and Adversarial Search Problems. - Jarrar · Mustafa Jarrar: Lecture Notes on Games and Adversarial Search Problems. Birzeit University, Palestine. 2018 Games and Adversarial

Jarrar © 2018 16

Minimax Tree

100

-24-8-14-73-100-5-70-12-470412-3212823

21 -3 12 70 -4 -73 -14 -8

-3 -4 -73

-3Max

Min

Max

Min

The best next move for Max

Page 17: Games and Adversarial Search Problems. - Jarrar · Mustafa Jarrar: Lecture Notes on Games and Adversarial Search Problems. Birzeit University, Palestine. 2018 Games and Adversarial

Jarrar © 2018 17

MiniMax Example-2

4 7 9 6 9 8 8 5 6 7 5 2 3 2 5 4 9 3

Terminal nodes: values calculated from the utility function

Max

Max

Min

Min

Based on [3]

Page 18: Games and Adversarial Search Problems. - Jarrar · Mustafa Jarrar: Lecture Notes on Games and Adversarial Search Problems. Birzeit University, Palestine. 2018 Games and Adversarial

Jarrar © 2018 18

MiniMax Example-2

4 7 9 6 9 8 8 5 6 7 5 2 3 2 5 4 9 3

4 7 6 2 6 3 4 5 1 2 5 4 1 2 6 3 4 3

Other nodes: values calculated via minimax algorithm

Max

Max

Min

Min

Page 19: Games and Adversarial Search Problems. - Jarrar · Mustafa Jarrar: Lecture Notes on Games and Adversarial Search Problems. Birzeit University, Palestine. 2018 Games and Adversarial

Jarrar © 2018 19

MiniMax Example-2

4 7 9 6 9 8 8 5 6 7 5 2 3 2 5 4 9 3

4 7 6 2 6 3 4 5 1 2 5 4 1 2 6 3 4 3

7 6 5 5 6 4

Max

Max

Min

Min

Page 20: Games and Adversarial Search Problems. - Jarrar · Mustafa Jarrar: Lecture Notes on Games and Adversarial Search Problems. Birzeit University, Palestine. 2018 Games and Adversarial

Jarrar © 2018 20

MiniMax Example-2

4 7 9 6 9 8 8 5 6 7 5 2 3 2 5 4 9 3

4 7 6 2 6 3 4 5 1 2 5 4 1 2 6 3 4 3

7 6 5 5 6 4

5 3 4

Max

Max

Min

Min

Page 21: Games and Adversarial Search Problems. - Jarrar · Mustafa Jarrar: Lecture Notes on Games and Adversarial Search Problems. Birzeit University, Palestine. 2018 Games and Adversarial

Jarrar © 2018 21

MiniMax Example-2

4 7 9 6 9 8 8 5 6 7 5 2 3 2 5 4 9 3

4 7 6 2 6 3 4 5 1 2 5 4 1 2 6 3 4 3

7 6 5 5 6 4

5 3 4

5 Max

Max

Min

Min

Page 22: Games and Adversarial Search Problems. - Jarrar · Mustafa Jarrar: Lecture Notes on Games and Adversarial Search Problems. Birzeit University, Palestine. 2018 Games and Adversarial

Jarrar © 2018 22

MiniMax Example-2

Max

Max

Min

Min

4 7 9 6 9 8 8 5 6 7 5 2 3 2 5 4 9 3

4 7 6 2 6 3 4 5 1 2 5 4 1 2 6 3 4 3

7 6 5 5 6 4

5 3 4

5

moves by Max and countermoves by Min

Page 23: Games and Adversarial Search Problems. - Jarrar · Mustafa Jarrar: Lecture Notes on Games and Adversarial Search Problems. Birzeit University, Palestine. 2018 Games and Adversarial

Jarrar © 201823

Properties of MiniMax

Complete: Yes (if tree is finite)

Optimal: Yes (against an optimal opponent)

Time complexity: A complete evaluation takes time bm

Space complexity: A complete evaluation takes space bm

(depth-first exploration)

For chess, b ≈ 35, m ≈100 for "reasonable" games

à exact solution completely infeasible, since it�s too big

Instead, we limit the depth based on various factors, including time

available.

Page 24: Games and Adversarial Search Problems. - Jarrar · Mustafa Jarrar: Lecture Notes on Games and Adversarial Search Problems. Birzeit University, Palestine. 2018 Games and Adversarial

Jarrar © 2018 24

Alpha-Beta Pruning Algorithm

Page 25: Games and Adversarial Search Problems. - Jarrar · Mustafa Jarrar: Lecture Notes on Games and Adversarial Search Problems. Birzeit University, Palestine. 2018 Games and Adversarial

Jarrar © 2018 25

Pruning the Minimax Tree

Since we have limited time available, we want to avoid unnecessary computation in the minimax tree.

Pruning: ways of determining that certain branches will not be useful.

a Cuts

If the current max value is greater than the successors min value, don’t explore that min subtree any more.

Page 26: Games and Adversarial Search Problems. - Jarrar · Mustafa Jarrar: Lecture Notes on Games and Adversarial Search Problems. Birzeit University, Palestine. 2018 Games and Adversarial

Jarrar © 2018 26

a Cut Example

10021 -3 12 70 -4 -73 -14

-3 -4

-3

-73

Max

Max

Min

Page 27: Games and Adversarial Search Problems. - Jarrar · Mustafa Jarrar: Lecture Notes on Games and Adversarial Search Problems. Birzeit University, Palestine. 2018 Games and Adversarial

Jarrar © 2018 27

a Cut Example

Depth first search along path 1

21

Max

Max

Min

Page 28: Games and Adversarial Search Problems. - Jarrar · Mustafa Jarrar: Lecture Notes on Games and Adversarial Search Problems. Birzeit University, Palestine. 2018 Games and Adversarial

Jarrar © 2018 28

a Cut Example

21 is minimum so far (second level)Can�t evaluate yet at top level

21

21

Max

Max

Min

Page 29: Games and Adversarial Search Problems. - Jarrar · Mustafa Jarrar: Lecture Notes on Games and Adversarial Search Problems. Birzeit University, Palestine. 2018 Games and Adversarial

Jarrar © 2018 29

a Cut Example

-3 is minimum so far (second level)-3 is maximum so far (top level)

21 -3

-3

-3Max

Max

Min

Page 30: Games and Adversarial Search Problems. - Jarrar · Mustafa Jarrar: Lecture Notes on Games and Adversarial Search Problems. Birzeit University, Palestine. 2018 Games and Adversarial

Jarrar © 2018 30

a Cut Example

12 is minimum so far (second level)-3 is still maximum (can’t use second node yet)

21 -3 12

-3

-3

12

Max

Max

Min

Page 31: Games and Adversarial Search Problems. - Jarrar · Mustafa Jarrar: Lecture Notes on Games and Adversarial Search Problems. Birzeit University, Palestine. 2018 Games and Adversarial

Jarrar © 2018 31

a Cut Example

-70 is now minimum so far (second level)-3 is still maximum (can�t use second node yet)

21 -3 12 -70

-3

-3

-70

Max

Max

Min

Page 32: Games and Adversarial Search Problems. - Jarrar · Mustafa Jarrar: Lecture Notes on Games and Adversarial Search Problems. Birzeit University, Palestine. 2018 Games and Adversarial

Jarrar © 2018 32

a Cut Example

Since second level node will never be > -70, it will never be chosen by the previous levelWe can stop exploring that node

21 -3 12 -70

-3

-3

-70

Max

Max

Min

Page 33: Games and Adversarial Search Problems. - Jarrar · Mustafa Jarrar: Lecture Notes on Games and Adversarial Search Problems. Birzeit University, Palestine. 2018 Games and Adversarial

Jarrar © 2018 33

a Cut Example

Evaluation at second level is again -73

10021 -3 12 -70 -4 -73

-3

-3

-70 -73

Max

Max

Min

Page 34: Games and Adversarial Search Problems. - Jarrar · Mustafa Jarrar: Lecture Notes on Games and Adversarial Search Problems. Birzeit University, Palestine. 2018 Games and Adversarial

Jarrar © 2018 34

a Cut Example

Again, can apply a cut since the second level node will never be > -73, and thus will never be chosen by the previous level

10021 -3 12 -70 -4 -73

-3

-3

-70 -73

Max

Max

Min

Page 35: Games and Adversarial Search Problems. - Jarrar · Mustafa Jarrar: Lecture Notes on Games and Adversarial Search Problems. Birzeit University, Palestine. 2018 Games and Adversarial

Jarrar © 2018 35

a Cut Example

As a result, we evaluated the Max node without evaluating several of the possible paths

10021 -3 12 -70 -4 -73 -14

-3

-3

-70 -73

Max

Max

Min

Page 36: Games and Adversarial Search Problems. - Jarrar · Mustafa Jarrar: Lecture Notes on Games and Adversarial Search Problems. Birzeit University, Palestine. 2018 Games and Adversarial

Jarrar © 2018 36

b Cuts

Similar idea to a cuts, but the other way aroundIf the current minimum is less than the successor’s max value, don’t look down that max tree any more

Page 37: Games and Adversarial Search Problems. - Jarrar · Mustafa Jarrar: Lecture Notes on Games and Adversarial Search Problems. Birzeit University, Palestine. 2018 Games and Adversarial

Jarrar © 2018 37

b Cut Example

Some subtrees at second level already have values > min from previous, so we can stop evaluating them.

10021 -3 12 70 -4 73 -14

Min

Min

Max 21

21

70 73

Page 38: Games and Adversarial Search Problems. - Jarrar · Mustafa Jarrar: Lecture Notes on Games and Adversarial Search Problems. Birzeit University, Palestine. 2018 Games and Adversarial

Jarrar © 2018 38

Alpha-Beta Example 2

[-∞, +∞]

– we assume a depth-first, left-to-right search as basic strategy– the range of the possible values for each node are indicated

• initially [-∞, +∞]• from Max�s or Min�s perspective• these local values reflect the values of the sub-trees in that node;

the global values a and b are the best overall choices so far for Max or Min

[-∞, +∞]

a best choice for Max ?b best choice for Min ?

Max

Min

Page 39: Games and Adversarial Search Problems. - Jarrar · Mustafa Jarrar: Lecture Notes on Games and Adversarial Search Problems. Birzeit University, Palestine. 2018 Games and Adversarial

Jarrar © 2018 39

Alpha-Beta Example 2

Max

Min[-∞, 7]

[-∞, +∞]

a best choice for Max ?b best choice for Min 7

7

Page 40: Games and Adversarial Search Problems. - Jarrar · Mustafa Jarrar: Lecture Notes on Games and Adversarial Search Problems. Birzeit University, Palestine. 2018 Games and Adversarial

Jarrar © 2018 40

Alpha-Beta Example 2

Max

Min[-∞, 6]

[-∞, +∞]

a best choice for Max ?b best choice for Min 6

7 6

Page 41: Games and Adversarial Search Problems. - Jarrar · Mustafa Jarrar: Lecture Notes on Games and Adversarial Search Problems. Birzeit University, Palestine. 2018 Games and Adversarial

Jarrar © 2018 41

Alpha-Beta Example 2

Max

Min5

[5, +∞]

a best choice for Max 5b best choice for Min 5

7 6 5

– Min obtains the third value from a successor node– this is the last value from this sub-tree, and the exact value is known– Max now has a value for its first successor node, but hopes that something

better might still come

Page 42: Games and Adversarial Search Problems. - Jarrar · Mustafa Jarrar: Lecture Notes on Games and Adversarial Search Problems. Birzeit University, Palestine. 2018 Games and Adversarial

Jarrar © 2018 42

Alpha-Beta Example 2

Max

Min[-∞, 5]

[5, +∞]

a best choice for Max 5b best choice for Min 3

7 6 5

– Min continues with the next sub-tree, and gets a better value – Max has a better choice from its perspective, however, and will not consider a

move in the sub-tree currently explored by min– Initially [-∞, +∞]

[-∞,3]

3

Page 43: Games and Adversarial Search Problems. - Jarrar · Mustafa Jarrar: Lecture Notes on Games and Adversarial Search Problems. Birzeit University, Palestine. 2018 Games and Adversarial

Jarrar © 2018 43

Alpha-Beta Example 2

Max

Min[-∞, 5]

[5, +∞]

a best choice for Max 5b best choice for Min 3

7 6 5

– Min knows that Max won�t consider a move to this sub-tree, and abandons it– this is a case of pruning, indicated by

[-∞,3]

3

Page 44: Games and Adversarial Search Problems. - Jarrar · Mustafa Jarrar: Lecture Notes on Games and Adversarial Search Problems. Birzeit University, Palestine. 2018 Games and Adversarial

Jarrar © 2018 44

Alpha-Beta Example 2

Max

Min[-∞, 5]

[5, +∞]

a best choice for Max 5b best choice for Min 3

7 6 5

– Min explores the next sub-tree, and finds a value that is worse than the other nodes at this level

– if Min is not able to find something lower, then Max will choose this branch, so Min must explore more successor nodes

[-∞,3]

3

[-∞,6]

6

Page 45: Games and Adversarial Search Problems. - Jarrar · Mustafa Jarrar: Lecture Notes on Games and Adversarial Search Problems. Birzeit University, Palestine. 2018 Games and Adversarial

Jarrar © 2018 45

Alpha-Beta Example 2

Max

Min[-∞, 5]

[5, +∞]

a best choice for Max 5b best choice for Min 3

7 6 5

– Min is lucky, and finds a value that is the same as the current worst value at this level

– Max can choose this branch, or the other branch with the same value

[-∞,3]

3

[-∞,5]

6 5

Page 46: Games and Adversarial Search Problems. - Jarrar · Mustafa Jarrar: Lecture Notes on Games and Adversarial Search Problems. Birzeit University, Palestine. 2018 Games and Adversarial

Jarrar © 2018 46

Alpha-Beta Example 2

Max

Min[-∞, 5]

5

a best choice for Max 5b best choice for Min 3

7 6 5

– Min could continue searching this sub-tree to see if there is a value that is less than the current worst alternative in order to give Max as few choices as possible

– this depends on the specific implementation– Max knows the best value for its sub-tree

[-∞,3]

3

[-∞,5]

6 5

Page 47: Games and Adversarial Search Problems. - Jarrar · Mustafa Jarrar: Lecture Notes on Games and Adversarial Search Problems. Birzeit University, Palestine. 2018 Games and Adversarial

Jarrar © 2018 47

max

min

max

min

Exercise

Page 48: Games and Adversarial Search Problems. - Jarrar · Mustafa Jarrar: Lecture Notes on Games and Adversarial Search Problems. Birzeit University, Palestine. 2018 Games and Adversarial

Jarrar © 2018 48

max

min

max

min10 9 14 2 4

10 14 4

10 4

10

Exercise (Solution)

Page 49: Games and Adversarial Search Problems. - Jarrar · Mustafa Jarrar: Lecture Notes on Games and Adversarial Search Problems. Birzeit University, Palestine. 2018 Games and Adversarial

Jarrar © 2018 49

a-b Pruning

Pruning by these cuts does not affect final result– May allow you to go much deeper in tree

�Good� ordering of moves can make this pruning much more efficient

– Evaluating �best� branch first yields better likelihood of pruning later branches

– Perfect ordering reduces time to bm/2 instead of O(bd)

– i.e. doubles the depth you can search to!

Page 50: Games and Adversarial Search Problems. - Jarrar · Mustafa Jarrar: Lecture Notes on Games and Adversarial Search Problems. Birzeit University, Palestine. 2018 Games and Adversarial

Jarrar © 2018 50

a-b Pruning

Can store information along an entire path, not just at most recent levels!Keep along the path:

a: best MAX value found on this path

(initialize to most negative utility value)

b: best MIN value found on this path

(initialize to most positive utility value)

Page 51: Games and Adversarial Search Problems. - Jarrar · Mustafa Jarrar: Lecture Notes on Games and Adversarial Search Problems. Birzeit University, Palestine. 2018 Games and Adversarial

Jarrar © 2018 51

Pruning at MAX node

a is possibly updated by the MAX of successors evaluated so far

If the value that would be returned is ever > b, then stop work on this branch

If all children are evaluated without pruning, return the MAX of their values

Page 52: Games and Adversarial Search Problems. - Jarrar · Mustafa Jarrar: Lecture Notes on Games and Adversarial Search Problems. Birzeit University, Palestine. 2018 Games and Adversarial

Jarrar © 2018 52

Pruning at MIN node

b is possibly updated by the MIN of successors evaluated so far

If the value that would be returned is ever < a, then stop work on this branch

If all children are evaluated without pruning, return the MIN of their values

Page 53: Games and Adversarial Search Problems. - Jarrar · Mustafa Jarrar: Lecture Notes on Games and Adversarial Search Problems. Birzeit University, Palestine. 2018 Games and Adversarial

Jarrar © 2018 53

Idea of a-b Pruning

We know b on this path is 21So, when we get max=70, we know this will never be used, so we can stop here 100

21 -3

12 70 -4

21

21

70

Page 54: Games and Adversarial Search Problems. - Jarrar · Mustafa Jarrar: Lecture Notes on Games and Adversarial Search Problems. Birzeit University, Palestine. 2018 Games and Adversarial

Jarrar © 2018 54

Why is it called α-β?

• α is the value of the best (i.e., highest-value) choice found so far at any choice point along the path for max

• If v is worse than α, max will avoid it

à prune that branch

• Define β similarly for min

Page 55: Games and Adversarial Search Problems. - Jarrar · Mustafa Jarrar: Lecture Notes on Games and Adversarial Search Problems. Birzeit University, Palestine. 2018 Games and Adversarial

Jarrar © 201855

Imperfect Decisions

Complete search is impractical for most games

Alternative: search the tree only to a certain depth

– Requires a cutoff-test to determine where to stop

• Replaces the terminal test

• The nodes at that level effectively become terminal leave nodes

– Uses a heuristics-based evaluation function to estimate

the expected utility of the game from those leave nodes.

Page 56: Games and Adversarial Search Problems. - Jarrar · Mustafa Jarrar: Lecture Notes on Games and Adversarial Search Problems. Birzeit University, Palestine. 2018 Games and Adversarial

Jarrar © 2018 56

Utility Evaluation Function

Very game-specificTake into account knowledge about game�Stupid� utility

– 1 if player 1 wins– -1 if player 0 wins

– 0 if tie (or unknown)

– Only works if we can evaluate complete tree– But, should form a basis for other evaluations

Page 57: Games and Adversarial Search Problems. - Jarrar · Mustafa Jarrar: Lecture Notes on Games and Adversarial Search Problems. Birzeit University, Palestine. 2018 Games and Adversarial

Jarrar © 201857

Utility Evaluation

Need to assign a numerical value to the state

– Could assign a more complex utility value, but then the

min/max determination becomes trickier.

Typically assign numerical values to lots of individual

factors:

– a = # player 1�s pieces - # player 2�s pieces

– b = 1 if player 1 has queen and player 2 does not, -1 if

the opposite, or 0 if the same

– c = 2 if player 1 has 2-rook advantage, 1 if a 1-rook

advantage, etc.

Page 58: Games and Adversarial Search Problems. - Jarrar · Mustafa Jarrar: Lecture Notes on Games and Adversarial Search Problems. Birzeit University, Palestine. 2018 Games and Adversarial

Jarrar © 2018 58

Utility Evaluation

The individual factors are combined by some function

Usually a linear weighted combination is used:– u = aa + bb + cc

– Different ways to combine are also possible

Notice: quality of utility function is based on:– What features are evaluated

– How those features are scored

– How the scores are weighted/combined

Absolute utility value doesn�t matter – relative value does.

Page 59: Games and Adversarial Search Problems. - Jarrar · Mustafa Jarrar: Lecture Notes on Games and Adversarial Search Problems. Birzeit University, Palestine. 2018 Games and Adversarial

Jarrar © 2018 59

Evaluation Functions

If you had a perfect utility evaluation function, what would it mean about the minimax tree?

You would never have to evaluate more than one level deep!

Typically, you can�t create such perfect utility evaluations, though.

Page 60: Games and Adversarial Search Problems. - Jarrar · Mustafa Jarrar: Lecture Notes on Games and Adversarial Search Problems. Birzeit University, Palestine. 2018 Games and Adversarial

Jarrar © 2018 60

Evaluation Functions for Ordering

As mentioned earlier, order of branch evaluation can make a big difference in how well you can prune

A good evaluation function might help you order your available moves:

– Perform one move only

– Evaluate board at that level

– Recursively evaluate branches in order from best first move to worst first move (or vice-versa if at a MIN node)

Page 61: Games and Adversarial Search Problems. - Jarrar · Mustafa Jarrar: Lecture Notes on Games and Adversarial Search Problems. Birzeit University, Palestine. 2018 Games and Adversarial

Jarrar © 2018 61

The following are extra Examples(Self Study)

Page 62: Games and Adversarial Search Problems. - Jarrar · Mustafa Jarrar: Lecture Notes on Games and Adversarial Search Problems. Birzeit University, Palestine. 2018 Games and Adversarial

Jarrar © 2018 62

Example: Tic-Tac-Toe (evaluation function)

Simple evaluation functionE(s) = (rx + cx + dx) - (ro + co + do)

where r,c,d are the numbers of row, column and diagonal lines still available; x and o are the pieces of the two players.

1-ply lookahead– start at the top of the tree

– evaluate all 9 choices for player 1

– pick the maximum E-value

2-ply lookahead– also looks at the opponents possible move

• assuming that the opponents picks the minimum E-value

Page 63: Games and Adversarial Search Problems. - Jarrar · Mustafa Jarrar: Lecture Notes on Games and Adversarial Search Problems. Birzeit University, Palestine. 2018 Games and Adversarial

Jarrar © 2018 63

E(s12)8

- 6= 2

E(s13)8

- 5= 3

E(s14)8

- 6= 2

E(s15)8

- 4= 4

E(s16)8

- 6= 2

E(s17)8

- 5= 3

E(s18)8

- 6= 2

E(s19)8

- 5= 3

Tic-Tac-Toe 1-Ply

X X XX X X

X X X

E(s11)8

- 5= 3

E(s0) = Max{E(s11), E(s1n)} = Max{2,3,4} = 4

Based on [3]

Page 64: Games and Adversarial Search Problems. - Jarrar · Mustafa Jarrar: Lecture Notes on Games and Adversarial Search Problems. Birzeit University, Palestine. 2018 Games and Adversarial

Jarrar © 2018 64

E(s2:16)5

- 6= -1

E(s2:15)5

-6= -1

E(s28)5

- 5= 0

E(s27)6

- 5= 1

E(s2:48)5

- 4= 1

E(s2:47)6

- 4= 2

E(s2:13)5

- 6= -1

E(s2:9)5

- 6= -1

E(s2:10)5

-6= -1

E(s2:11)5

- 6= -1

E(s2:12)5

- 6= -1

E(s2:14)5

- 6= -1

E(s25)6

- 5= 1

E(s21)6

- 5= 1

E(s22)5

- 5= 0

E(s23)6

- 5= 1

E(s24)4

- 5= -1

E(s26)5

- 5= 0

E(s1:6)8

- 6= 2

E(s1:7)8

- 5= 3

E(s1:8)8

- 6= 2

E(s1:9)8

- 5= 3

E(s1:5)8

- 4= 4

E(s1:3)8

- 5= 3

E(s1:2)8

- 6= 2

E(s1:1)8

- 5= 3

E(s2:45)6

- 4= 2

Tic-Tac-Toe 2-Ply

X X XX X X

X X X

E(s1:4)8

- 6= 2

X O XO

XO

E(s2:41)5

- 4= 1

E(s2:42)6

- 4= 2

E(s2:43)5

- 4= 1

E(s2:44)6

- 4= 2

E(s2:46)5

- 4= 1

O XO

XOX

OX X

O

XO

X

O

X

O

XXO

X OO X X

O

XO

X

O

X

O

XO

XO

X OX O X

O

O

E(s0) = Max{E(s11), E(s1n)} = Max{2,3,4} = 4

Page 65: Games and Adversarial Search Problems. - Jarrar · Mustafa Jarrar: Lecture Notes on Games and Adversarial Search Problems. Birzeit University, Palestine. 2018 Games and Adversarial

Jarrar © 2018 65

31

Checkers Case Study

• Initial board configuration– Black single on 20

single on 21king on 31

– Red single on 23king on 22

– Evaluation functionE(s) = (5 x1 + x2) - (5r1 + r2)

where x1 = black king advantage, x2 = black single advantage,r1 = red king advantage, r2 = red single advantage

1 2 3 4

865

9 10 11 12

161413

17 18 19 20

242221

25 26 27 28

323029

7

15

23

Based on [4]

Page 66: Games and Adversarial Search Problems. - Jarrar · Mustafa Jarrar: Lecture Notes on Games and Adversarial Search Problems. Birzeit University, Palestine. 2018 Games and Adversarial

Jarrar © 2018 66

1

1

1 1 1 2

2

6

6

1

1

1 1 1 1 1

1

1 1 1 1 6

6

0

0

0 0 -4

-4

-4 -8

-8

-8 -8

-8

-8

1 0 -8 -8

1

20 -> 16 21 -> 17

31 -> 26

31 -> 27

22 -> 17

22 -> 18

22 ->

25

22 ->

26 23 -> 26

23 -> 27

21 -> 14

16 -> 11

31 ->

27

16 ->

11

31 -> 27 31 ->

27

16 ->

11 31 -> 27

31 -> 24

22 -> 13

22 -> 3123 -> 30

23 -> 32

20 ->

16 31 -> 27

31 -> 26

21 -> 1720 ->

16 21 -> 17

20 -> 16

20 -> 16

21 -> 17

31

1 2 3 4

865

9 10 11 12

161413

17 18 19 20

242221

25 26 27 28

323029

7

15

23

MAX

MAX

MIN

Checkers MiniMax Example

Page 67: Games and Adversarial Search Problems. - Jarrar · Mustafa Jarrar: Lecture Notes on Games and Adversarial Search Problems. Birzeit University, Palestine. 2018 Games and Adversarial

Jarrar © 2018 67

1

1

1 1 1 2

2

6

6

1

1

1 1 1 1 1

1

1 1 1 1 6

6

0

0

0 0 -4

-4

-4 -8

-8

-8 -8

-8

-8

1 0 -4 -8

1

20 -> 16 21 -> 17

31 -> 26

31 -> 27

22 -> 17

22 -> 18

22 ->

25

22 ->

26 23 -> 26

23 -> 27

21 -> 14

16 -> 11

31 ->

27

16 ->

11

31 -> 27 31 ->

27

16 ->

11 31 -> 27

31 -> 24

22 -> 18

22 -> 3123 -> 30

23 -> 32

20 ->

16 31 -> 27

31 -> 26

21 -> 1720 ->

16 21 -> 17

20 -> 16

20 -> 16

21 -> 17

31

1 2 3 4

865

9 10 11 12

161413

17 18 19 20

242221

25 26 27 28

323029

7

15

23

a 1 b 6

MAX

MAX

MIN

Checkers Alpha-Beta Example

Page 68: Games and Adversarial Search Problems. - Jarrar · Mustafa Jarrar: Lecture Notes on Games and Adversarial Search Problems. Birzeit University, Palestine. 2018 Games and Adversarial

Jarrar © 2018 68

1

1

1 1 1 2

2

6

6

1

1

1 1 1 1 1

1

1 1 1 1 6

6

0

0

0 0 -4

-4

-4 -8

-8

-8 -8

-8

-8

1 0 -4 -8

1

20 -> 16 21 -> 17

31 -> 26

31 -> 27

22 -> 17

22 -> 18

22 ->

25

22 ->

26 23 -> 26

23 -> 27

21 -> 14

16 -> 11

31 ->

27

16 ->

11

31 -> 27 31 ->

27

16 ->

11 31 -> 27

31 -> 24

22 -> 18

22 -> 31

23 -> 30

23 -> 32

20 ->

16 31 -> 27

31 -> 26

21 -> 1720 ->

16 21 -> 17

20 -> 16

20 -> 16

21 -> 17

31

1 2 3 4

865

9 10 11 12

161413

17 18 19 20

242221

25 26 27 28

323029

7

15

23

a 1 b 1

MAX

MAX

MIN

Checkers Alpha-Beta Example

Page 69: Games and Adversarial Search Problems. - Jarrar · Mustafa Jarrar: Lecture Notes on Games and Adversarial Search Problems. Birzeit University, Palestine. 2018 Games and Adversarial

Jarrar © 2018 69

1

1

1 1 1 2

2

6

6

1

1

1 1 1 1 1

1

1 1 1 1 6

6

0

0

0 0 -4

-4

-4 -8

-8

-8 -8

-8

-8

1 0 -4 -8

1

20 -> 16 21 -> 17

31 -> 26

31 -> 27

22 -> 17

22 -> 18

22 ->

25

22 ->

26 23 -> 26

23 -> 27

21 -> 14

16 -> 11

31 ->

27

16 ->

11

31 -> 27 31 ->

22

16 ->

11 31 -> 27

31 -> 24

22 -> 18

22 -> 31

23 -> 30

23 -> 32

20 ->

16 31 -> 27

31 -> 26

21 -> 1720 ->

16 21 -> 17

20 -> 16

20 -> 16

21 -> 17

31

1 2 3 4

865

9 10 11 12

161413

17 18 19 20

242221

25 26 27 28

323029

7

15

23

a 1 b 1

b- cutoff: no need toexamine further branches

MAX

MAX

MIN

Checkers Alpha-Beta Example

Page 70: Games and Adversarial Search Problems. - Jarrar · Mustafa Jarrar: Lecture Notes on Games and Adversarial Search Problems. Birzeit University, Palestine. 2018 Games and Adversarial

Jarrar © 2018 70

1

1

1 1 1 2

2

6

6

1

1

1 1 1 1 1

1

1 1 1 1 6

6

0

0

0 0 -4

-4

-4 -8

-8

-8 -8

-8

-8

1 0 -4 -8

1

20 -> 16 21 -> 17

31 -> 26

31 -> 27

22 -> 17

22 -> 18

22 ->

25

22 ->

26 23 -> 26

23 -> 27

21 -> 14

16 -> 11

31 ->

27

16 ->

11

31 -> 27 31 ->

22

16 ->

11 31 -> 27

31 -> 24

22 -> 18

22 -> 31

23 -> 30

23 -> 32

20 ->

16 31 -> 27

31 -> 26

21 -> 1720 ->

16 21 -> 17

20 -> 16

20 -> 16

21 -> 17

31

1 2 3 4

865

9 10 11 12

161413

17 18 19 20

242221

25 26 27 28

323029

7

15

23

a 1 b 1

MAX

MAX

MIN

Checkers Alpha-Beta Example

Page 71: Games and Adversarial Search Problems. - Jarrar · Mustafa Jarrar: Lecture Notes on Games and Adversarial Search Problems. Birzeit University, Palestine. 2018 Games and Adversarial

Jarrar © 2018 71

1

1

1 1 1 2

2

6

6

1

1

1 1 1 1 1

1

1 1 1 1 6

6

0

0

0 0 -4

-4

-4 -8

-8

-8 -8

-8

-8

1 0 -4 -8

1

20 -> 16 21 -> 17

31 -> 26

31 -> 27

22 -> 17

22 -> 18

22 ->

25

22 ->

26 23 -> 26

23 -> 27

21 -> 14

16 -> 11

31 ->

27

16 ->

11

31 -> 27 31 ->

22

16 ->

11 31 -> 27

31 -> 24

22 -> 18

22 -> 31

23 -> 30

23 -> 32

20 ->

16 31 -> 27

31 -> 26

21 -> 1720 ->

16 21 -> 17

20 -> 16

20 -> 16

21 -> 17

31

1 2 3 4

865

9 10 11 12

161413

17 18 19 20

242221

25 26 27 28

323029

7

15

23

a 1 b 1

b- cutoff: no need toexamine further branches

MAX

MAX

MIN

Checkers Alpha-Beta Example

Page 72: Games and Adversarial Search Problems. - Jarrar · Mustafa Jarrar: Lecture Notes on Games and Adversarial Search Problems. Birzeit University, Palestine. 2018 Games and Adversarial

Jarrar © 2018 72

1

1

1 1 1 2

2

6

6

1

1

1 1 1 1 1

1

1 1 1 1 6

6

0

0

0 0 -4

-4

-4 -8

-8

-8 -8

-8

-8

1 0 -4 -8

1

20 -> 16 21 -> 17

31 -> 26

31 -> 27

22 -> 17

22 -> 18

22 ->

25

22 ->

26 23 -> 26

23 -> 27

21 -> 14

16 -> 11

31 ->

27

16 ->

11

31 -> 27 31 ->

22

16 ->

11 31 -> 27

31 -> 24

22 -> 18

22 -> 31

23 -> 30

23 -> 32

20 ->

16 31 -> 27

31 -> 26

21 -> 1720 ->

16 21 -> 17

20 -> 16

20 -> 16

21 -> 17

31

1 2 3 4

865

9 10 11 12

161413

17 18 19 20

242221

25 26 27 28

323029

7

15

23

a 1 b 1

MAX

MAX

MIN

Checkers Alpha-Beta Example

Page 73: Games and Adversarial Search Problems. - Jarrar · Mustafa Jarrar: Lecture Notes on Games and Adversarial Search Problems. Birzeit University, Palestine. 2018 Games and Adversarial

Jarrar © 2018 73

1

1

1 1 1 2

2

6

6

1

1

1 1 1 1 1

1

1 1 1 1 6

6

0

0

0 0 -4

-4

-4 -8

-8

-8 -8

-8

-8

1 0 -4 -8

1

20 -> 16 21 -> 17

31 -> 26

31 -> 27

22 -> 17

22 -> 18

22 ->

25

22 ->

26 23 -> 26

23 -> 27

21 -> 14

16 -> 11

31 ->

27

16 ->

11

31 -> 27 31 ->

22

16 ->

11 31 -> 27

31 -> 24

22 -> 13

22 -> 31

23 -> 30

23 -> 32

20 ->

16 31 -> 27

31 -> 26

21 -> 1720 ->

16 21 -> 17

20 -> 16

20 -> 16

21 -> 17

31

1 2 3 4

865

9 10 11 12

161413

17 18 19 20

242221

25 26 27 28

323029

7

15

23

a 1 b 0

MAX

MAX

MIN

Checkers Alpha-Beta Example

Page 74: Games and Adversarial Search Problems. - Jarrar · Mustafa Jarrar: Lecture Notes on Games and Adversarial Search Problems. Birzeit University, Palestine. 2018 Games and Adversarial

Jarrar © 2018 74

Checkers Alpha-Beta Example

1

1

1 1 1 2

2

6

6

1

1

1 1 1 1 1

1

1 1 1 1 6

6

0

0

0 0 -4

-4

-4 -8

-8

-8 -8

-8

-8

1 0 -4 -8

1

20 -> 16 21 -> 17

31 -> 26

31 -> 27

22 -> 17

22 -> 18

22 ->

25

22 ->

26 23 -> 26

23 -> 27

21 -> 14

16 -> 11

31 ->

27

16 ->

11

31 -> 27 31 ->

22

16 ->

11 31 -> 27

31 -> 24

22 -> 18

22 -> 31

23 -> 30

23 -> 32

20 ->

16 31 -> 27

31 -> 26

21 -> 1720 ->

16 21 -> 17

20 -> 16

20 -> 16

21 -> 17

31

1 2 3 4

865

9 10 11 12

161413

17 18 19 20

242221

25 26 27 28

323029

7

15

23

a 1 b -4

a- cutoff: no need toexamine further branches

MAX

MAX

MIN

Page 75: Games and Adversarial Search Problems. - Jarrar · Mustafa Jarrar: Lecture Notes on Games and Adversarial Search Problems. Birzeit University, Palestine. 2018 Games and Adversarial

Jarrar © 2018 75

22 -> 31

1

1

1 1 1 2

2

6

6

1

1

1 1 1 1 1

1

1 1 1 1 6

6

0

0

0 0 -4

-4

-4 -8

-8

-8 -8

-8

-8

1 0 -4 -8

1

20 -> 16 21 -> 17

31 -> 26

31 -> 27

22 -> 17

22 -> 18

22 ->

25

22 ->

26 23 -> 26

23 -> 27

21 -> 14

16 -> 11

31 ->

27

16 ->

11

31 -> 27 31 ->

22

16 ->

11 31 -> 27

31 -> 24

22 -> 18

23 -> 30

23 -> 32

20 ->

16 31 -> 27

31 -> 26

21 -> 1720 ->

16 21 -> 17

20 -> 16

20 -> 16

21 -> 17

31

1 2 3 4

865

9 10 11 12

161413

17 18 19 20

242221

25 26 27 28

323029

7

15

23

a 1 b -8 MAX

MAX

MIN

Checkers Alpha-Beta Example

Page 76: Games and Adversarial Search Problems. - Jarrar · Mustafa Jarrar: Lecture Notes on Games and Adversarial Search Problems. Birzeit University, Palestine. 2018 Games and Adversarial

Jarrar © 2018 76

References

[1] S. Russell and P. Norvig: Artificial Intelligence: A Modern Approach Prentice Hall, 2003, Second Edition

[2] Nilufer Onden: Lecture Notes on Artificial Intelligencehttp://www.cs.mtu.edu/~nilufer/classes/cs4811/2014-spring/lecture-slides/cs4811-ch05-adversarial-search.pdf

[3] Samy Abu Nasser: Lecture Notes on Artificial Intelligencehttp://up.edu.ps/ocw/repositories/academic/up/bs/it/ITLS4213/022009/data/ITLS4213.101_11042009.ppt

[4] Franz Kurfess: Lecture Notes on Artificial Intelligencehttp://users.csc.calpoly.edu/~fkurfess/Courses/Artificial-Intelligence/F09/Slides/3-Search.ppt