On Von Neumann Pokerwith Community Cards

Reto Spöhel

Joint work with Nicla Bernasconi and Julian Lorenz

Poker

Poker is a popular type of card game in which players gamble on the superior value of the card combination ("hand") in their possession, by placing a bet into a central pot. The winner is the one who holds the hand with the highest value according to an established hand rankings hierarchy, or otherwise the player who remains "in the hand" after all others have folded.

According to Wikipedia,

Outline

Introduction von Neumann Poker von Neumann Poker with community cards Outlook: the Newman model Conclusion

Research on poker

The game of poker has been studied from many different perspectives:

game-theory [this talk] artifical intelligence (heuristics) machine learning (opponent modeling) behavioural psychology etc.

In the game-theoretic approach, one assumes best play for all players involved.

allows development and application of mathematical theories

neglects many other factors

Game-theoretic research on poker

almost exclusively for two-player game two main lines of attack:

simplified models of „real“ poker e.g., 2 suits with 5 cards each solution by brute-force calculation; lots of

computational power needed more abstract models, which hopefully capture

essential features of poker [this talk] e.g., hands are numbers u.a.r. from [0,1] hopefully analytically solvable most important model: von Neumann poker

Von Neumann Poker

P chips are in the pot at the beginning. X and Y are dealt independent hands x,y 2 [0,1] u.a.r. X may make a bet of a or pass („check“).

If X checks, both hands are revealed („showdown“), and the player with the higher hand wins the pot.

If X makes a bet, Y can either match the bet („call“) or concede the pot to X („fold“).

If Y folds, X wins the pot (and gets his bet back). If Y calls X‘s bet, both hands are revealed and the

player with the higher hand wins the pot and the two bets.

In the following we assume for simplicity P = a = 1.

Von Neumann Poker

It seems that X has an advantage, since Y can only react. So how should X play to maximize his expected payoff?

Clearly, always checking guarantees him an expected payoff of P/2 = 1/2.

Similarly, X cannot hope for an expected payoff of more than P=1, since Y can always fold.

Von Neumann Poker

At first sight, one might guess that X should bet the better half of his hands, i.e., iff x ¸ 1/2.

However, once Y realizes that this is X‘s strategy, he will only call with hands y ¸ 2/3, since then

he wins P+a=2 chips with probability at least 1/3 he loses a=1 chips with probability at most 2/3 i.e., „the pot odds are in his favor“.

2/3

y

1/2

0

1

x

check

bet

fold

call

Von Neumann Poker

X‘s expected payoff can be found by integrating over the hands of x and y and is:

P ¢ 1/8 + P ¢ 1/2 ¢ 2/3 + (P+a) ¢ 1/18 – a ¢ 1/9

= 1/8+1/3 = 11/24 < 1/2 X loses money!

check

0

bet-call

P

bet-fold

P

-a

P+a2/3

0x

y

1/2

1/2

2/3

0

1

check

bet

fold

call

x y

So X has no advantage over Y and should never bet? NO!

With the previous strategy, most of X‘s good hands go to waste because Y just folds.

However, X can induce Y to call more often by including bluffs in his strategy!

von Neumann gave an equilibrium pair of strategies Y‘s strategy is best response to X‘s strategy X‘s strategy is best response to Y‘s strategy

X can achieve an expected payoff of 5/9, which is optimal.

von Neumann, 1928

Von Neumann Poker

von Neumann‘s solution

Heuristic: we make the following ansatz: With a hand of y1, calling and folding should have the

same expected payoff for Y:

x0 ¢ (P + a) – (1 – x1) ¢ a = 0 With a hand of x0 or x1, betting and checking should

have the same expected payoff for X:

y1 ¢ P + (x1 – y1) ¢ (P + a) – (1 – y1) ¢ a = x0 ¢ P

y1 ¢ P + (x1 – y1) ¢ (P + a) – (1 – x1) ¢ a = x1 ¢ P

0

1

check

value-bet

fold

call

x y

bluff-bet

y1

x1

x0

solution (P=a=1): x0 = 1/9 y1 = 5/9 x1 = 7/9

von Neumann‘s solution

The two resulting strategies are indeed in equilibrium. The expected payoff for X turns out to be 5/9

the value of the game is 5/9 (in zero-sum games, all equilibria have the same value!)

Insights: Bluffing is a game-theoretic necessity! You should bluff-bet your worst hands!

solution (P=a=1): x0 = 1/9 y1 = 5/9 x1 = 7/90

1

check

value-bet

fold

call

x y

bluff-bet

y1

x1

x0

Von Neumann Poker

Many extensions of the von Neumann model have been studied

allow multiple betting rounds, raises, reraises, etc. hands may depend on each other… etc.

by scientists and professional poker player alike.

Chris Ferguson, PhD 2000 World Series

of Poker champion co-author of

several papers on von Neumann poker

The mathematics of Poker, Bill Chen and Jerrod Ankenman, 2006

Our contribution

In the classical von Neumann model (and its extensions), no further random influences are present once both players have received their hands.

In real poker, community cards are drawn between betting rounds.

players may bet with a bad hand, hoping that the right card will show up and turn it into a good hand.

players with a good hand tend to bet more aggressively to force other players to fold.

We propose the following extension of the von Neumann model that accounts for these features:

Before the showdown, throw an unfair coin. With probability q, the lower hand wins!

Introducing the flip

Introducing the flip: With a hand of y1, calling and folding should have the

same expected payoff for Y: before (q=0):

x0 ¢ (P + a) – (1 – x1) ¢ a = 0 now:

[x0 ¢ (1 – q) + (1 – x1) ¢ q] ¢ (P + a)

– [(1 – x1) ¢ (1 – q) + x0 ¢ q] ¢ a = 0

solution (P=a=1): x0 = x0(q) y1 = y1(q) x1 = x1(q)

0

1

check

value-bet

fold

call

x y

bluff-bet

y1

x1

x0

Introducing the flip

y1

x1

x0

?

1/3 =: q0

5/9

1/9

7/9

We obtain

Beyond the critical q

What happens for q > q0 = 1/3? Y will call every bet since even with the worst hand y = 0

he wins P+a=2 chips with probability at least q = 1/3 he loses a=1 chips with probability at most 1 – q = 2/3

Knowing this, X will bet the better half of his hands.

1/2

0

1

check

bet

call

x y

for general P and a:

q : (1-q) = a : (P+a)

q0 = a/(P+2a)

The full picture

y1

x1

x0

q0 = 1/3

As q increases, X makes more value bets and bluffs less.

Y is induced by the q to call! For q ¸ q0, there‘s no point in

bluffing, since Y will always call anyway.

X value-bets more often to protect his good hands from being flipped into bad hands.

The value of the game

1/2 = 0.5

7/12 = 0.583…

5/9 = 0.555…

q0 = 1/3

value of the game

The expected payoff of X is maximal at q = q0.

i.e., when Y has just enough incentive to call every bet of X without X wasting money on bluffs.

An improved model

Where does the discontinuity at q0 come from? We fixed the bet size a (arbitrarily) before the game

started. What if we allow X to look at his hand and then bet any

amount a ¸ 0 he likes?

…and with flip probability q, 0 · q < 1/2

X can achieve an expected payoff of 4/7, which is optimal.

Newman, 1959

X can achieve an expected payoff of (16-q)/(28-8q), which is optimal.

Bernasconi, Lorenz, S., 2007+

Newman‘s solution

1/7 4/7

„value-bet“„bluff-bet“

bet a

hand x, resp. y

The red line is X‘s bet a(x) X checks for 1/7 · x · 4/7

The green line is Y‘s calling threshold

Y calls iff (y,a) is below the green line

obtained with similar, but more involved heuristics than before( differential equations)

(q = 0)

Introducing the flip

a0 =

1/2

Similarly to before:

q : (1-q) = a : (P+a)

a0 = q/(1-2q) ¢ P even with his worst hand, Y calls

every bet of at most a0 (pot-odds!) knowing this, X never bets an

amount between 0 and a0

neither as a value bet (if the odds are in his favour [x ¸ 1/2], he bets at least a0),

nor as a bluff bet (there‘s no value bet for which to induce more calls).

hand x, resp. y

(q = 1/3)

bet a

Introducing the flip

(q = 1/3)

(q = 0.4)

bet a

As q increases, X bets his value bets more aggressively.

hand x

The value of the game

value of the game

For 0 < q · 1/2, we have

value(q) = (16-q)/(28-8q)

the value is strictly increasing in q but discontinuous at q=1/2, since trivially

value(1/2) = 1/2.

What is going on?

We allowed arbitrarily high bets a. Moreover, a0 diverges at q = 1/2.

If we limit X‘s bankroll, the singularities vanish.

4/7 = 0.571…

31/48 = 0.645…

Conclusion

We proposed a way of including „community cards“ (random effects after betting) into both the von Neumann and the Newman model…

… and gave complete analytical solutions. As expected, we observed increasingly aggressive betting

for larger q. In both models, we observe a „phase transition“ when

q : (1-q) = a : (P+a) (“flip-odds = pot-odds”)

Questions?