Robin Hanson PM Summit NYC 2009

Combinatorial Prediction Markets

Robin HansonEconomics, George Mason UniversityChief Scientist, Consensus Point

SpeculativeSpeculative

FutureFuture Currency

Currency

StockStockBetBet

Insurance

Insurance

HedgingHedging Gambling

Gambling

Decision

Decision

BondBond

SpotSpot

Market

ContextContext DirectDirect

SearchSearch MatchingMatching

“Prediction Markets”

To Evaluate Institutions

Institution A

When They UseSimilar Inputs

Compare QualityOf Outputs

Institution B

Collective Forecasting

ForecastsOn Requested

TopicsUser Contributions User Scores

Engagement

IssuesInput: Contributions

What info can express?How account for costs?Who let in where?Enough Incentives

T-shirts enough?Zero-sum scoring?

Limit CostsAwkward InterfaceWait for offer acceptRetribution

Output: Forecasts, Scores

What questions can ask?How account for value?Use or validate system?Should adjust outputs?Who let see outputs?Sabotage & manipulationLegal, P.R. risks?

Collective Forecasting Questions

ConsensusConsensus

My ForecastMy Forecast My ScoreMy Score

TruthTruth

What exactly is my influence?

What exactly are my

incentives?

How exactly do I express my

opinion?

Editing Interface Is Transparent

ConsensusConsensus

My EditsMy Edits My ScoreMy Score

TruthTruth

I directly change theconsensus

If my edit increasesthe consensus chance

of true state, I win. If decreases, I lose.






What questions can ask?How account for value?Use or validate system?Should adjust outputs?Who let see outputs?Sabotage & manipulationLegal, P.R. risks?

Factors Might Influence Sales

Economy recovers fast?Competitors introduce new version?We do big promotion?We lower prices? They lower prices?We add distribution channel?We add feature F? They add feature F?Our defect rate very low?

E[Sales|Factor]P[Factor]

Combo Betting

Yoopick Facebook Application

WinPlace

Show

All outcomes

Win Place Show

Not Not Not

Sport Finals TicketsUEFA EURO 2008

Austria

Croatia

Czech

Germany

Poland

Portugal

Switzerl.

Turkey

France

Greece

Italy

Netherl.

Romania

Russia

Spain ActualGame

Sweden

Ticket if Greece in FinalsGreece v.Croatia

US President Decision Markets

Politimetrics.com

PAM Scenario

Saudi Arabian

100

75

125

103

203

303

403

104

204

65

35

70

30

30

35

25

10

15

40

35

10

10094

Economic Health

New Price <IQcs403 85

Return to Focus ?TradeSAum3 03105-125

No Trade

Exit Issue

Max Up

Max Dn

10% Dn

10% Up

You Pick

62.47%

Bu

y S

ell

>IQcs403 85

Payoffs:

68.72%

56.79%

48.54%

95.13%

22.98%

+$2.74

-$15.34

+$34.74

$0.00 $0.00

-$2.61

-2.04

-$3.28

-$85.18

-$120.74

+$2.74

65

+$26.02

+1.43

+$96.61

& If

65%

Ave. pay<85 Select

$0.00

+0.34

-$1.07

-$19.72

-$1.12

-$6.31

-$22.22

%

Update

Execute a Trade ?Return to Form

ExecuteAbort trade if price has changed?

If US military involvement in Saudi Arabia in 3rd Quarter 2003 is not between 105 and 125, this trade is null and void. Otherwise, if Iraq civil stability in 4th Quarter 2003 is below 85, then I will receive $1.43, but if it is not below 85, I will pay $2.04.

Imagine A DashboardUs Them A Them B

Base Price $240 $187 $320

Ship Date May ’09 Mar ’09 July ’09

Features Autozoop 38% 69% 15%

Fizzywoo 59% 8% 43%

Unit Sales Total 120K 148K 97K

Base model 82K 65K 88K

Via internet 43K 12K 73K

Promotion Magazine $30K $50K $3K

Circulars $45 $34K $39K

Ave. Worth:$12,459

Ask For DetailUs Them A Them B

Base Price $240 $187 $320



Fizzywoo 59% 8% 43%






J F M A M J J A S O N D

2009 2010

Them B Ship Date

Ave.Worth:$12,459

Make An EditUs Them A Them B

Base Price $240 $187 $320



Fizzywoo 59% 8% 43%






Ave. Worth:$12,459

If We Have Autozoop, you gain $53

But if We Don’t Have ItYou lose $78. OK?

Make an AssumptionUs Them A Them B

Base Price $240 $187 $253

Ship Date Apr ’09 Mar ’09 Assume Mar


Fizzywoo 59% 8% 13%






Ave. Worth:$10,724

Scenario:15%

Add 2nd AssumptionUs Them A Them B

Base Price $240 $187 $253



Fizzywoo 59% 8% 13%




Promotion Magazine Assume $40K

$50K $17K


Ave. Worth:$10,982

Scenario:2.3%

Edit As BeforeUs Them A Them B

Base Price $240 $187 $253



Fizzywoo 59% 8% 13%




Promotion Magazine Assume $40K

$50K $17K


Ave. Worth:$10,724

Scenario:2.3%

If we have Autozoop, you gain $53

But if we don’t have itYou lose $78. OK?

Combo Market Maker Best of 5 Mechs

3 Variables = 8 States

0.000

0.050

0.100

0.150

0.200

0.250

0.300

KL

Dis

tanc

e

8 Variables = 256 States

0.600

0.800

1.000

1.200

1.400

1.600

KL

Dis

tanc

e

3 subjects, 7 prices, 5 minutes 6 subjects, 256 prices, 5 minutes

-1

-0.5

0

0.5

1

0 5 10 15

MSR Info vs. Time – 7 Prices

0 5 10 15

Minutes

0

-1

1

% Info Agg. =

KL(prices,group)KL(uniform,group)

1-

MSR Info vs. Time – 255 prices

-1

-0.5

0

0.5

1

0 5 10 15

0 5 10 15

Minutes

0

-1

1

% Info Agg. =

KL(prices,group)KL(uniform,group)

1-






What questions can ask?How account for value?Use or validate system?Should adjust outputs?Sabotage & manipulationCan keep results secret?Legal, P.R. risks?

States

Pri

ces

Use

r A

sset

s

$1 if A&B

$1 if B

+ q1

- q2

f1>1 f2<1

A B

+

-

A Simple Implementation

LISP: http://hanson.gmu.edu/mktscore-prototype.html

A Simple Implementation

States

A&B A&B

Pri

ces

Use

r A

sset

s

Environments: Goals, TrainingWant in Environment:

Many variables, few directly related Few people, each not see all variables Can compute rational group estimatesExplainable, fast, neutral

Training Environment: 3 binary variables X,Y,Z, 23 = 8 combosP(X=0) = .3, P(X=Y) = .2, P(Z=1)= .5 3 people, see 10 cases of: AB, BC, AC Random map XYZ to ABC

Case A B C

1 1 -

1 2 1 -

0

3 1 -

0 4 1 -

0

5 1 -

0 6 1 -

1

7 1 -

1 8 1 -

0

9 1 -

0 10 0 -

0

Sum: 9 -

3

Same A B C

A -- --

4 B -- --

-- C -- --

--

(Actually: X Z Y )

Experiment Environment8 binary vars: STUVWXYZ28 = 256 combinations20% = P(S=0) = P(S=T) = P(T=U) = P(U=V) = … = P(X=Y) = P(Y=Z)6 people, each see 10 cases: ABCD, EFGH, ABEF, CDGH, ACEG, BDFHrandom map STUVWXYZ to ABCDEFGH

Case A B C D E F G H 1 0 1 0 1 - - - - 2 1 0 0 1 - - - - 3 0 0 1 1 - - - - 4 1 0 1 1 - - - - 5 0 1 1 1 - - - - 6 1 0 0 1 - - - - 7 0 1 1 1 - - - - 8 1 0 0 1 - - - - 9 1 0 0 1 - - - - 10 1 0 0 1 - - - -

Sum 6 3 4 10 - - - -

Same A B C D E F G H A -- 1 2 6 -- -- -- -- B -- -- 7 3 -- -- -- -- C -- -- -- 4 -- -- -- -- D -- -- -- -- -- -- -- -- …

(Really: W V X S U Z Y T )

Ad Agency Decision Markets

$ Revenue if not Switch

E(R | not S)Compare!

$1 if not Switch

$ Revenue if Switch

E(R | S)

$1

P(S)

$1 if Switch

$ RevenueE(R)

Robin Hanson PM Summit NYC 2009

Business

Transcript of Robin Hanson PM Summit NYC 2009