MITMITUCLAUCLA

Securities Trading of Concepts (Securities Trading of Concepts (STOCSTOC))

Professors: Ely Dahan, Andrew Lo, Tomaso PoggioProfessors: Ely Dahan, Andrew Lo, Tomaso PoggioStudents: Adlar Kim, Nicholas ChanStudents: Adlar Kim, Nicholas Chan

Presented at the DARPA workshop onMarkets and Decisions

Arlington, Virginia

June 12, 2002June 12, 2002

Today’s Agenda - Questions about STOC

• What is STOC? How does it work?

• Does it perform well?

• Is it accurate at identifying winners?

• What does it actually measure?

• How do traders behave? Do they learn?

• For what categories does it work?

• Does it work with fully integrated concepts? Attributes?

• Are real outcomes absolutely necessary?

The Virtual Customer:

Ely DahanRob Hardy

Limor Weisberg

Ely DahanRob Hardy

Limor Weisberg

Ely Dahan John R. Hauser

Duncan SimesterOlivier Toubia

Ely DahanV. Seenu Srinivasan

Leonard Lee

N. Chan, Ely Dahan Andrew Lo, Tomaso Poggio,

Adlar Kim

Computat

ion

FixedDesign

Adaptive

Slow FastCommunication

Con

cept

ualiz

atio

n

Verbal

MediaRich

TraditionalMarket

Research

VirtualCustomerResearch

Impact of the WEB on Market Research

Are Web Respondents Representative?

Pre-recruited Web Panels• NFO Interactive Balanced 500,000 respondents• DMS, Inc. (AOL) “Opinion Place” 1,000,000• Knowledge Netwk. Rand. Digit Dialing 100,000• Greenfield Online 3,000,000 online panel• Harris Interactive 6,500,000 online panel

• Representativeness looks promisinge.g. Willkie, Adams, and Girnius (1999)

Game Markets•Iowa Electronics Markets (http://www.biz.uiowa.edu/IEM)•Foresight Exchange (http://www.ideosphere.com/FX)•Hollywood Stock Exchange (http://www.HSX.com)

UD Allows Complex Interactions

e.g. Slots, weight, battery, size, pricinge.g. Slots, weight, battery, size, pricing

Users Design High Utility Attribute Bundles

0%

5%

10%

15%

20%

<45%

45% 48% 52% 55% 58% 61% 64% 67% 70% 73% 77% 80% 83% 86% 89% 92% 95% 98%

Percentile of Designs

% o

f Res

pond

ents Participants

design near-ideal configurations

Participants design near-ideal

configurations

CEB 1/16/01 and MBA’s 3/20/01 (n=129)

User Design as Conjoint ValidationAttribute-by-Attribute “Hit Rates”

85%72% 68%

75% 70% 69%

0%

25%

50%

75%

100%

Seats Cargo MPG Horsepower 0-60 Seconds Towing

Hit Rates for Each Feature (WCA vs. UD)(By Individual by Feature, n=130)

CEB 1/16/01 and MBA’s 3/20/01 (n=130)CEB 1/16/01 and MBA’s 3/20/01 (n=130)

User Interface for Securities trading of Concepts (STOC)

Share of Vote Probability of Winning

Source: Iowa Electronics Market (IEM), 11/7/00 at 2pm

Hooray for Hollywood Hooray for Hollywood

Source: HSX.com, December 7, 2001

Bragging rights to predicting the future …Bragging rights to predicting the future …

Source: Source: http://www.ideosphere.com/fx, December 10, 2001, December 10, 2001

Current game markets share some traitsCurrent game markets share some traits

FXFXHSXHSXIEMIEM

•All three predict actual future outcomes

•Underlying reasons are not made explicit

STOC Outcomes vs. Virtual Concept Testing: Bicycle Pumps

Skitz

o Si

lverB

ullet

Epic

Ri

mGr

ipGe

arhe

adTR

S

2wist

erGe

cko

Cyclo

ne

Physical

Web StaticSTOC Test 1STOC Test 2

0%

10%

20%

30%

40%

Mar

ket S

hare

Po

tent

ial

r2 = 0.87 Physical, 0.92 Web

r2 = 0.78 Physical, 0.85 Web

Median Stock Prices

Choice Out of a Set of Eight (Rank order): Choice Out of a Set of Eight (Rank order): Crossover VehiclesCrossover Vehicles

$24,000$39,000 $49,000

$42,000$29,000

$37,000

$36,000$30,000

Securities Trading of Concepts (STOC): Securities Trading of Concepts (STOC): Crossover VehiclesCrossover Vehicles

STOC Vol.-Weighted Avg. Price vs. 1st Choice (8) Crossover Vehicles - SDM's Oct. 20, 2000

LexusAudi

BMW

Mercedes

Acura

Toyota

PontiacBuick

0%

10%

20%

30%

40%

0% 10% 20% 30% 40%

1st Choice

STO

C V

WA

P

r2 = 0.87

Securities Trading of Concepts (STOC): Securities Trading of Concepts (STOC): Crossover VehiclesCrossover Vehicles

STOC Median Price vs. Top 3 Choices(8) Crossover Vehciles - SDM October 20, 2000)

BuickPontiac

ToyotaAcura

Mercedes

BMW

Audi Lexus0%

10%

20%

30%

40%

0% 10% 20% 30% 40%

Top 3 Choice

STO

C M

edian

Pric

e

r2 = 0.80

Securities Trading of Concepts (STOC): Securities Trading of Concepts (STOC): Crossover VehiclesCrossover Vehicles

Acura

Toyota

Merced

esLex

usAud

iBMW

Pontia

cBuic

k

MBA Test 1 Top 3

VWAP %

Median Price %0%

5%

10%

15%

20%

25%

% "S

hare

"

(8) Crossover Vehicles

STOC Price vs. Top-3 ChoicesMarch 20, 2001 MBA Test 1

r2 = 0.63r2 = 0.63

r2 = 0.60r2 = 0.60

Students valued BMW and Mercedes STOC highly,

even though they “bought”them less frequently

Students valued BMW and Mercedes STOC highly,

even though they “bought”them less frequently

Which STOC metric is best?

Correlations Between STOC and 1st Choice(Max - Average - Min for four independent trials)

0.550.45

0.710.680.69

0.69

0.000.100.200.300.400.500.600.700.800.901.00

Median VWAP Mean Close Min Max

STOC Metric

Cor

rela

tion

STOC Games for STOC Games for Laptop BagsLaptop Bags: Tabular vs. Image Stimuli: Tabular vs. Image StimuliBag 8

$99

Bag 16

$87

Bag 10

$95

Bag 13

$79

Prediction was very bad with the table Prediction was very bad with the table -- better with the larger imagesbetter with the larger images

Table Formatr2 = 0.00

0%

5%

10%

15%

20%

25%

0% 5% 10% 15% 20% 25% 30% 35%

(8) Bag Imagesr2 = 0.58

0%

5%

10%

15%

20%

25%

0% 5% 10% 15% 20% 25% 30% 35%

Bag 8

Bag 8

Actual Share Actual Share

STO

C M

edia

n

STO

C M

edia

n

Bag 9

Bag 9

Securities Trading Of Attributes (STOA): Securities Trading Of Attributes (STOA): Laptop BagLaptop Bag

Red Handle

PDACell

Logo

MeshFlap

Boot Large

Will prediction improve if traders consider each attribute indepWill prediction improve if traders consider each attribute independently?endently?

Nine Nine Laptop BagLaptop Bag Attribute StocksAttribute Stocks

Prior Individual Estimates of Attributes: Prior Individual Estimates of Attributes: Laptop BagsLaptop Bags109 SurveyResponses

54% (±19%)

22% (±20%)

66% (±22%)

68% (±19%)

26% (±16%)

33% (±18%)

51% (±22%)

43% (±22%)

49% (±22%)

330 ActualChoices

61%61%

25%25%

71%71%

83%83%

25%25%

34%34%

58%58%

59%59%

75%75%

Trader Pre-Survey vs. Actual Choice of Each Attribute

(group r2 = 0.72, � = 0.85)(mean indiv. r2 = 0.33, � = 0.49)-------

0%

20%

40%

60%

80%

0% 20% 40% 60% 80%Survey

Act

ual

Aggregate OpinionsImprove prediction

What % of 330 first year Sloanies (in 2000) bought each of the nWhat % of 330 first year Sloanies (in 2000) bought each of the nine upgrades?ine upgrades?

Securities Trading Of Attributes (STOA): Securities Trading Of Attributes (STOA): Laptop BagsLaptop Bags

15.831 Laptop Bag Attributesr2 = 0.51 (0.40 rank order)

BootRed

Flap

0%

25%

50%

75%

100%

0% 25% 50% 75% 100%

Actual Choice

STO

C V

WA

P

15.828 Laptop Bag Atributesr2 = 0.33 (0.25 rank order)

Boot

Red

Flap

0%

25%

50%

75%

100%

0% 25% 50% 75% 100%

Actual Choice

STO

C V

WA

P

Vermont Laptop AttributeSTOC Median vs. "Truth"

r2 = 0.25

Flap

BootRed

20%

30%

40%

50%

60%

70%

80%

20% 30% 40% 50% 60% 70% 80%TRUTH

STO

C M

EDIA

N

Large Red Logo Handle PDA Cell Mesh Flap Boot

Can traders predict each attribute independently? Can traders predict each attribute independently? YES!YES!

STOC for Vehicle Attributes: Crossover VehiclesCrossover Vehicles

7 Seats

Tow 5000

Fast 0-60

Large Cargo

240 hp

MPG

20.0%

40.0%

60.0%

80.0%

20% 40% 60% 80%

Actual User Design

STO

C C

lose

r2 = 0.85

0.0

25.0

50.0

75.0

100.0

Powder 10

0No C

rowd 1

00Nearb

y 100

Exper

t 100

Night Life

100

Powder 500

Expert

500

No Cro

wd 500

Nearby 5

00Night

Life 50

02001 ActualMean Prior

r2 =0.67 group, 0.53 indiv.STOC r2 =0.58

STOC for Ski Resort attributes

We let 241 people design their ideal PDA/Cell phone device

Each feature was traded off against price, weight, & batteryEach feature was traded off against price, weight, & battery

The (14) Features were then traded as (14) “Stocks” in a game

COLOR

SIZE

Price Weight Battery

OS

MEMORY

HANDSFREE

CELL

WIRELESS

BATTERY

KEYBOARD

BLUETOOTH

SD SLOT

CF SLOT

GPS

MEMORY

%Upgrade

$99/149

$10

$40

$25

$50

$99

$99/149

$99

$25

$49

$15

$15

$129

$50

-

3.0 oz

-

-

-

1.0 oz

0.5 oz

0.5 oz

$

$

$

0.5 oz

1.0 oz

-

-20%

-

-5%

-

-5%

-

-10%

+300%

-10%

-5%

-

-5%

-10%

-

Stock price = % of respondents who bought each featureStock price = % of respondents who bought each feature

241 241 PDA/CellPDA/Cell Designs vs. STOC trading by MIT StudentsDesigns vs. STOC trading by MIT Students

14 PDA AttributesSTOC (Close) versus User Design Survey

r2 = 0.59

CellPhone

LongBattery

WirelessNW

ColorDisplay

CF SlotPocketPC

64MRAM32MB RAM

BluetoothHandsfree

Large SizeKeyboardSD Slot

GPS

0%

10%

20%

30%

40%

50%

60%

70%

0% 10% 20% 30% 40% 50% 60% 70%STOC Closing Prices

Use

r Des

ign

Surv

ey (%

Upg

radi

ng)

Source: Ely Dahan, MIT, January 9, 2002

Student STOC trading agreed quite well with the original surveyStudent STOC trading agreed quite well with the original survey

How did Executives do with How did Executives do with PDA/CellPDA/Cell attributes?attributes?

Source: Ely Dahan, MIT, April 18, 2002

STOC Trading of 14 PDA AttributesCEB Traders on April 18, 2002

r2 = 0.87

KeyboardSDGPS

LargeHands free

32MB

BlueTooth64MB

PocketPC

CF

Battery

ColorCell

Wireless

0.0%

10.0%

20.0%

30.0%

40.0%

50.0%

60.0%

70.0%

80.0%

0.0% 10.0% 20.0% 30.0% 40.0% 50.0% 60.0% 70.0% 80.0%

STOC Closing Prices

Use

r D

esig

n S

urve

y (%

Upg

radi

ng)

CEB Executives did even CEB Executives did even better than students!better than students!..

How do individual traders rank? How do individual traders rank? (laptop bag attributes)

15.828 Sessionr2 = 0.31

0

10

20

30

40

50

60

0 10 20 30 40 50 60

Trader Rank Using Market

Trad

er R

ank

Usi

ng A

ctua

ls

Accurate WinnersAccurate Winners

15.831 Sessionr2 = 0.10

0

10

20

30

40

50

60

0 10 20 30 40 50 60

Trader Rank Using Market

Trad

er R

ank

Usi

ng A

ctua

ls

InaccurateInaccurateWinnersWinners

AccurateAccurateLoserLoser

High marketHigh market--priced portfolios don’t always identify “accurate” traderspriced portfolios don’t always identify “accurate” traders

Order Effect: Bottom stocks get traded less, at lower pricesOrder Effect: Bottom stocks get traded less, at lower prices

Vol.-Weighted Avg. PricePosition v. Price r2 = 0.41

� = -0.64

0% 50% 100%

Boot:

Flap:

Mesh:

Cell:

PDA:

Handle:

Logo:

Red:

Large:

VOLUMEPosition v. Volume r2 = 0.14

� = -0.38

0 1000 2000

Boot:

Flap:

Mesh:

Cell:

PDA:

Handle:

Logo:

Red:

Large:

ACTUALPosition v. Actual r2 = 0.03

� = +0.18

0% 50% 100%

Boot:

Flap:

Mesh:

Cell:

PDA:

Handle:

Logo:

Red:

Large:

October 15, 2001, 1pm (n=52)

This suggests a need to have stocks randomly ordered by traderThis suggests a need to have stocks randomly ordered by trader

Do STOC sellers rank concepts lower than buyers?

-7 -6 -5 -4 -3 -2 -1 0 1 2 3 4 5 6 7

STOC

Random0%5%

10%15%20%25%30%

% o

f T

rade

s

Rank Difference (Buyer - Seller)

STOC Traded Vehicles Are Close in Rank(n = 165 transaction pairings)

62% of trades are in the "right"

direction

Conclusions about STOC

• Functions well with informed traders

• Is accurate at identifying winners

• Measures preferences in the aggregate

• Traders behave heterogeneously, learn

• Predicts well for many product categories

• Can be effective with concepts AND attributes

• Real outcomes are not absolutely necessary

Thank you.

UCLA Andersonely.dahan@anderson.ucla.edu

MIT Sloanedahan@mit.edu