Research Virtual Currency Case Study: Prediction Markets David M. Pennock Yahoo! Research.

ResearchResearch

Virtual CurrencyCase Study: Prediction Markets

David M. Pennock

Yahoo! Research

ResearchResearch

Virtual Currency

• What actions can be motivated with virtual currency?

• To what degree of competence?• Case study: Prediction markets• Epilogue: Designing a better virtual

currency

ResearchResearch

QuickTime™ and aTIFF (LZW) decompressor

are needed to see this picture.



ResearchResearch















ResearchResearch












are needed to see this picture.worse... i would not payit would make thingsworse...I would not pay

Not many people wouldbuy it... less answers...

I would die immediately.Nobody will pay...

Keep... the Internet free



ResearchResearch


are needed to see this picture.QuickTime™ and a

TIFF (LZW) decompressorare needed to see this picture.










are needed to see this picture.worse... i would not payit would make thingsworse...I would not pay

Not many people wouldbuy it... less answers...

I would die immediately.Nobody will pay...

Keep... the Internet free Better (if done right)



ResearchResearch









ResearchResearch

Things People Do “For Free”

• Provide information• Write software• Label images• Answer questions QuickTime™ and a




ResearchResearch


• Provide information• Write software• Label images• Answer questions

WIRED MAGAZINE: ISSUE 16.03

ResearchResearch


are needed to see this picture.QuickTime™ and a





• Provide information• Write software• Label images• Answer questions

• Provide predictionsQuickTime™ and a


WIRED MAGAZINE: ISSUE 16.03

ResearchResearch

Ask The Crowd

• Choose what you’d like to predict:

• Let people bet on iti.e., Buy or sell share of

$1 if $0 if

I am entitled to:

Bird Flu Outbreak US 2008?(Y/N)

Bird FluUS ’08

Bird FluUS ’08

ResearchResearch



http://intrade.com

Screen capture 2008/03/17



ResearchResearch

The Wisdom of CrowdsBacked in dollars• What you can say/learn

% chance that• Hillary wins• GOP wins Texas• YHOO stock > 30• Duke wins tourney• Oil prices fall• Heat index rises• Hurricane hits Florida• Rains at place/time

• Where

• IEM, Intrade.com• Intrade.com• Stock options market• Las Vegas, Betfair• Futures market• Weather derivatives• Insurance company• Weatherbill.com

ResearchResearch



The Wisdom of CrowdsWith Money Without





















ResearchResearch

The Widsom of CrowdsBacked in “Points”• HSX.com• Newsfutures.com• InklingMarkets.com• Foresight Exchange• CasualObserver.net• FTPredict.com• Yahoo!/O’Reilly Tech Buzz• ProTrade.com• StorageMarkets.com• TheSimExchange.com• TheWSX.com• Alexadex, Celebdaq, Cenimar, BetBubble, Betocracy, CrowdIQ,

MediaMammon,Owise, PublicGyan, RIMDEX, Smarkets, Trendio, TwoCrowds

• http://www.chrisfmasse.com/3/3/markets/#Play-Money_Prediction_Markets

ResearchResearch


IEM: 237 Candidates HSX: 489 Movies

1 2 5 10 20 50 100estimate

1

2

5

10

20

50

100

actual

ResearchResearch


ResearchResearch

Real markets vs. market gamesHSX FX, F1P6

probabilisticforecasts

forecast source avg log scoreF1P6 linear scoring -1.84F1P6 F1-style scoring -1.82betting odds -1.86F1P6 flat scoring -2.03F1P6 winner scoring -2.32

ResearchResearch

Does money matter? Play vs real, head to headExperiment• 2003 NFL Season• ProbabilitySports.com

Online football forecasting competition

• Contestants assess probabilities for each game

• Quadratic scoring rule• ~2,000 “experts”, plus:• NewsFutures (play $)• Tradesports (real $)

• Used “last trade” prices

Results:• Play money and real

money performed similarly• 6th and 8th respectively

• Markets beat most of the ~2,000 contestants• Average of experts

came 39th (caveat)

Electronic Markets, Emile Servan-Schreiber, Justin Wolfers, David Pennock and Brian Galebach

ResearchResearch

0

25

50

75

100

TradeSports Prices

0 20 40 60 80 100NewsFutures Prices

Fitted Value: Linear regression

45 degree line

n=416 over 208 NFL games.Correlation between TradeSports and NewsFutures prices = 0.97

Prices: TradeSports and NewsFutures

Prediction Performance of MarketsRelative to Individual Experts

020406080

100120140160180200220240260280300

1 2 3 4 5 6 7 8 9 10 11 12 13 14

Week into the NFL season

Rank

NewsFutures

Tradesports

0

10

20

30

40

50

60

70

80

90

100

Observed Frequency of Victory

0 10 20 30 40 50 60 70 80 90 100Trading Price Prior to Game

TradeSports: Correlation=0.96NewsFutures: Correlation=0.94

Data are grouped so that prices are rounded to the nearest ten percentage points; n=416 teams in 208 games

Market Forecast Winning Probability and Actual Winning ProbabilityPrediction Accuracy

ResearchResearch

Does money matter? Play vs real, head to head

Probability-Football Avg

TradeSports(real-money)

NewsFutures(play-money)

DifferenceTS - NF

Mean Absolute Error

= lose_price

[lower is better]

0.443

(0.012)

0.439

(0.011)

0.436

(0.012)

0.003

(0.016)

Root Mean Squared Error

= ?Average( lose_price2 )

[lower is better]

0.476

(0.025)

0.468

(0.023)

0.467

(0.024)

0.001

(0.033)

Average Quadratic Score

= 100 - 400*( lose_price2 )

[higher is better]

9.323

(4.75)

12.410

(4.37)

12.427

(4.57)

-0.017

(6.32)

Average Logarithmic Score

= Log(win_price)

[higher (less negative) is better]

-0.649

(0.027)

-0.631

(0.024)

-0.631

(0.025)

0.000

(0.035)

Statistically:TS ~ NFNF >> AvgTS > Avg

ResearchResearch

Discussion

• Are incentives for virtual currency strong enough?• Yes (to a degree)• Conjecture: Enough to get what people already know;

not enough to motivate independent research• Reduced incentive for information discovery possibly

balanced by better interpersonal weighting

• Statistical validations show HSX, FX, NF are reliable sources for forecasts• HSX predictions >= expert predictions• Combining sources can help

ResearchResearch

A Problem w/ Virtual CurrencyPrinting Money

Alice1000

Betty1000

Carol1000

ResearchResearch

A Problem w/ Virtual CurrencyPrinting Money

Alice5000

Betty1000

Carol1000

ResearchResearch

YootlesA Social Currency

Alice0

Betty0

Carol0

ResearchResearch


I owe you 5

Alice-5

Betty0

Carol5

ResearchResearch


credit: 5 credit: 10

I owe you 5

Alice-5

Betty0

Carol5

ResearchResearch



I owe you 5 I owe you 5

Alice-5

Betty0

Carol5

ResearchResearch



I owe you 5 I owe you 5

Alice3995

Betty0

Carol5

ResearchResearch



YootlesA Social Currency• For tracking gratitude among friends• A yootle says “thanks, I owe you one”

ResearchResearch

New Prediction Game

ResearchResearch

• A. Ghosh, M. Mahdian, D. Reeves, D. Pennock, R. Fugger (2007). Mechanism design on trust networks, Workshop on Internet and Network Economics

• E. Servan-Schreiber, J. Wolfers, D.M. Pennock, B. Galebach (2004). Does money matter?, Electronic Markets

• D.M. Pennock, S. Lawrence, C.L. Giles, F.Å. Nielsen (2001). The real power of artificial markets. Science, 291(5506), pp. 987-988

• D.M. Pennock, S. Lawrence, F.Å. Nielsen, C. Lee Giles (2001). Extracting collective probabilistic forecasts from web games, Conference on Knowledge Discovery and Data Mining, pp. 174-183

Talk Bibliography

Research Virtual Currency Case Study: Prediction Markets David M. Pennock Yahoo! Research.

Documents

Transcript of Research Virtual Currency Case Study: Prediction Markets David M. Pennock Yahoo! Research.