MITMITUCLAUCLA
Securities Trading of Concepts (Securities Trading of Concepts (STOCSTOC))
Professors: Ely Dahan, Andrew Lo, Tomaso PoggioProfessors: Ely Dahan, Andrew Lo, Tomaso PoggioStudents: Adlar Kim, Nicholas ChanStudents: Adlar Kim, Nicholas Chan
Presented at the DARPA workshop onMarkets and Decisions
Arlington, Virginia
June 12, 2002June 12, 2002
Today’s Agenda - Questions about STOC
• What is STOC? How does it work?
• Does it perform well?
• Is it accurate at identifying winners?
• What does it actually measure?
• How do traders behave? Do they learn?
• For what categories does it work?
• Does it work with fully integrated concepts? Attributes?
• Are real outcomes absolutely necessary?
The Virtual Customer:
Ely DahanRob Hardy
Limor Weisberg
Ely DahanRob Hardy
Limor Weisberg
Ely Dahan John R. Hauser
Duncan SimesterOlivier Toubia
Ely DahanV. Seenu Srinivasan
Leonard Lee
N. Chan, Ely Dahan Andrew Lo, Tomaso Poggio,
Adlar Kim
Computat
ion
FixedDesign
Adaptive
Slow FastCommunication
Con
cept
ualiz
atio
n
Verbal
MediaRich
TraditionalMarket
Research
VirtualCustomerResearch
Impact of the WEB on Market Research
Are Web Respondents Representative?
Pre-recruited Web Panels• NFO Interactive Balanced 500,000 respondents• DMS, Inc. (AOL) “Opinion Place” 1,000,000• Knowledge Netwk. Rand. Digit Dialing 100,000• Greenfield Online 3,000,000 online panel• Harris Interactive 6,500,000 online panel
• Representativeness looks promisinge.g. Willkie, Adams, and Girnius (1999)
Game Markets•Iowa Electronics Markets (http://www.biz.uiowa.edu/IEM)•Foresight Exchange (http://www.ideosphere.com/FX)•Hollywood Stock Exchange (http://www.HSX.com)
UD Allows Complex Interactions
e.g. Slots, weight, battery, size, pricinge.g. Slots, weight, battery, size, pricing
Users Design High Utility Attribute Bundles
0%
5%
10%
15%
20%
<45%
45% 48% 52% 55% 58% 61% 64% 67% 70% 73% 77% 80% 83% 86% 89% 92% 95% 98%
Percentile of Designs
% o
f Res
pond
ents Participants
design near-ideal configurations
Participants design near-ideal
configurations
CEB 1/16/01 and MBA’s 3/20/01 (n=129)
User Design as Conjoint ValidationAttribute-by-Attribute “Hit Rates”
85%72% 68%
75% 70% 69%
0%
25%
50%
75%
100%
Seats Cargo MPG Horsepower 0-60 Seconds Towing
Hit Rates for Each Feature (WCA vs. UD)(By Individual by Feature, n=130)
CEB 1/16/01 and MBA’s 3/20/01 (n=130)CEB 1/16/01 and MBA’s 3/20/01 (n=130)
Bragging rights to predicting the future …Bragging rights to predicting the future …
Source: Source: http://www.ideosphere.com/fx, December 10, 2001, December 10, 2001
Current game markets share some traitsCurrent game markets share some traits
FXFXHSXHSXIEMIEM
•All three predict actual future outcomes
•Underlying reasons are not made explicit
STOC Outcomes vs. Virtual Concept Testing: Bicycle Pumps
Skitz
o Si
lverB
ullet
Epic
Ri
mGr
ipGe
arhe
adTR
S
2wist
erGe
cko
Cyclo
ne
Physical
Web StaticSTOC Test 1STOC Test 2
0%
10%
20%
30%
40%
Mar
ket S
hare
Po
tent
ial
r2 = 0.87 Physical, 0.92 Web
r2 = 0.78 Physical, 0.85 Web
Median Stock Prices
Choice Out of a Set of Eight (Rank order): Choice Out of a Set of Eight (Rank order): Crossover VehiclesCrossover Vehicles
$24,000$39,000 $49,000
$42,000$29,000
$37,000
$36,000$30,000
Securities Trading of Concepts (STOC): Securities Trading of Concepts (STOC): Crossover VehiclesCrossover Vehicles
STOC Vol.-Weighted Avg. Price vs. 1st Choice (8) Crossover Vehicles - SDM's Oct. 20, 2000
LexusAudi
BMW
Mercedes
Acura
Toyota
PontiacBuick
0%
10%
20%
30%
40%
0% 10% 20% 30% 40%
1st Choice
STO
C V
WA
P
r2 = 0.87
Securities Trading of Concepts (STOC): Securities Trading of Concepts (STOC): Crossover VehiclesCrossover Vehicles
STOC Median Price vs. Top 3 Choices(8) Crossover Vehciles - SDM October 20, 2000)
BuickPontiac
ToyotaAcura
Mercedes
BMW
Audi Lexus0%
10%
20%
30%
40%
0% 10% 20% 30% 40%
Top 3 Choice
STO
C M
edian
Pric
e
r2 = 0.80
Securities Trading of Concepts (STOC): Securities Trading of Concepts (STOC): Crossover VehiclesCrossover Vehicles
Acura
Toyota
Merced
esLex
usAud
iBMW
Pontia
cBuic
k
MBA Test 1 Top 3
VWAP %
Median Price %0%
5%
10%
15%
20%
25%
% "S
hare
"
(8) Crossover Vehicles
STOC Price vs. Top-3 ChoicesMarch 20, 2001 MBA Test 1
r2 = 0.63r2 = 0.63
r2 = 0.60r2 = 0.60
Students valued BMW and Mercedes STOC highly,
even though they “bought”them less frequently
Students valued BMW and Mercedes STOC highly,
even though they “bought”them less frequently
Which STOC metric is best?
Correlations Between STOC and 1st Choice(Max - Average - Min for four independent trials)
0.550.45
0.710.680.69
0.69
0.000.100.200.300.400.500.600.700.800.901.00
Median VWAP Mean Close Min Max
STOC Metric
Cor
rela
tion
STOC Games for STOC Games for Laptop BagsLaptop Bags: Tabular vs. Image Stimuli: Tabular vs. Image StimuliBag 8
$99
Bag 16
$87
Bag 10
$95
Bag 13
$79
Prediction was very bad with the table Prediction was very bad with the table -- better with the larger imagesbetter with the larger images
Table Formatr2 = 0.00
0%
5%
10%
15%
20%
25%
0% 5% 10% 15% 20% 25% 30% 35%
(8) Bag Imagesr2 = 0.58
0%
5%
10%
15%
20%
25%
0% 5% 10% 15% 20% 25% 30% 35%
Bag 8
Bag 8
Actual Share Actual Share
STO
C M
edia
n
STO
C M
edia
n
Bag 9
Bag 9
Securities Trading Of Attributes (STOA): Securities Trading Of Attributes (STOA): Laptop BagLaptop Bag
Red Handle
PDACell
Logo
MeshFlap
Boot Large
Will prediction improve if traders consider each attribute indepWill prediction improve if traders consider each attribute independently?endently?
Prior Individual Estimates of Attributes: Prior Individual Estimates of Attributes: Laptop BagsLaptop Bags109 SurveyResponses
54% (±19%)
22% (±20%)
66% (±22%)
68% (±19%)
26% (±16%)
33% (±18%)
51% (±22%)
43% (±22%)
49% (±22%)
330 ActualChoices
61%61%
25%25%
71%71%
83%83%
25%25%
34%34%
58%58%
59%59%
75%75%
Trader Pre-Survey vs. Actual Choice of Each Attribute
(group r2 = 0.72, � = 0.85)(mean indiv. r2 = 0.33, � = 0.49)-------
0%
20%
40%
60%
80%
0% 20% 40% 60% 80%Survey
Act
ual
Aggregate OpinionsImprove prediction
What % of 330 first year Sloanies (in 2000) bought each of the nWhat % of 330 first year Sloanies (in 2000) bought each of the nine upgrades?ine upgrades?
Securities Trading Of Attributes (STOA): Securities Trading Of Attributes (STOA): Laptop BagsLaptop Bags
15.831 Laptop Bag Attributesr2 = 0.51 (0.40 rank order)
BootRed
Flap
0%
25%
50%
75%
100%
0% 25% 50% 75% 100%
Actual Choice
STO
C V
WA
P
15.828 Laptop Bag Atributesr2 = 0.33 (0.25 rank order)
Boot
Red
Flap
0%
25%
50%
75%
100%
0% 25% 50% 75% 100%
Actual Choice
STO
C V
WA
P
Vermont Laptop AttributeSTOC Median vs. "Truth"
r2 = 0.25
Flap
BootRed
20%
30%
40%
50%
60%
70%
80%
20% 30% 40% 50% 60% 70% 80%TRUTH
STO
C M
EDIA
N
Large Red Logo Handle PDA Cell Mesh Flap Boot
Can traders predict each attribute independently? Can traders predict each attribute independently? YES!YES!
STOC for Vehicle Attributes: Crossover VehiclesCrossover Vehicles
7 Seats
Tow 5000
Fast 0-60
Large Cargo
240 hp
MPG
20.0%
40.0%
60.0%
80.0%
20% 40% 60% 80%
Actual User Design
STO
C C
lose
r2 = 0.85
0.0
25.0
50.0
75.0
100.0
Powder 10
0No C
rowd 1
00Nearb
y 100
Exper
t 100
Night Life
100
Powder 500
Expert
500
No Cro
wd 500
Nearby 5
00Night
Life 50
02001 ActualMean Prior
r2 =0.67 group, 0.53 indiv.STOC r2 =0.58
STOC for Ski Resort attributes
We let 241 people design their ideal PDA/Cell phone device
Each feature was traded off against price, weight, & batteryEach feature was traded off against price, weight, & battery
The (14) Features were then traded as (14) “Stocks” in a game
COLOR
SIZE
Price Weight Battery
OS
MEMORY
HANDSFREE
CELL
WIRELESS
BATTERY
KEYBOARD
BLUETOOTH
SD SLOT
CF SLOT
GPS
MEMORY
%Upgrade
$99/149
$10
$40
$25
$50
$99
$99/149
$99
$25
$49
$15
$15
$129
$50
-
3.0 oz
-
-
-
1.0 oz
0.5 oz
0.5 oz
$
$
$
0.5 oz
1.0 oz
-
-20%
-
-5%
-
-5%
-
-10%
+300%
-10%
-5%
-
-5%
-10%
-
Stock price = % of respondents who bought each featureStock price = % of respondents who bought each feature
241 241 PDA/CellPDA/Cell Designs vs. STOC trading by MIT StudentsDesigns vs. STOC trading by MIT Students
14 PDA AttributesSTOC (Close) versus User Design Survey
r2 = 0.59
CellPhone
LongBattery
WirelessNW
ColorDisplay
CF SlotPocketPC
64MRAM32MB RAM
BluetoothHandsfree
Large SizeKeyboardSD Slot
GPS
0%
10%
20%
30%
40%
50%
60%
70%
0% 10% 20% 30% 40% 50% 60% 70%STOC Closing Prices
Use
r Des
ign
Surv
ey (%
Upg
radi
ng)
Source: Ely Dahan, MIT, January 9, 2002
Student STOC trading agreed quite well with the original surveyStudent STOC trading agreed quite well with the original survey
How did Executives do with How did Executives do with PDA/CellPDA/Cell attributes?attributes?
Source: Ely Dahan, MIT, April 18, 2002
STOC Trading of 14 PDA AttributesCEB Traders on April 18, 2002
r2 = 0.87
KeyboardSDGPS
LargeHands free
32MB
BlueTooth64MB
PocketPC
CF
Battery
ColorCell
Wireless
0.0%
10.0%
20.0%
30.0%
40.0%
50.0%
60.0%
70.0%
80.0%
0.0% 10.0% 20.0% 30.0% 40.0% 50.0% 60.0% 70.0% 80.0%
STOC Closing Prices
Use
r D
esig
n S
urve
y (%
Upg
radi
ng)
CEB Executives did even CEB Executives did even better than students!better than students!..
How do individual traders rank? How do individual traders rank? (laptop bag attributes)
15.828 Sessionr2 = 0.31
0
10
20
30
40
50
60
0 10 20 30 40 50 60
Trader Rank Using Market
Trad
er R
ank
Usi
ng A
ctua
ls
Accurate WinnersAccurate Winners
15.831 Sessionr2 = 0.10
0
10
20
30
40
50
60
0 10 20 30 40 50 60
Trader Rank Using Market
Trad
er R
ank
Usi
ng A
ctua
ls
InaccurateInaccurateWinnersWinners
AccurateAccurateLoserLoser
High marketHigh market--priced portfolios don’t always identify “accurate” traderspriced portfolios don’t always identify “accurate” traders
Order Effect: Bottom stocks get traded less, at lower pricesOrder Effect: Bottom stocks get traded less, at lower prices
Vol.-Weighted Avg. PricePosition v. Price r2 = 0.41
� = -0.64
0% 50% 100%
Boot:
Flap:
Mesh:
Cell:
PDA:
Handle:
Logo:
Red:
Large:
VOLUMEPosition v. Volume r2 = 0.14
� = -0.38
0 1000 2000
Boot:
Flap:
Mesh:
Cell:
PDA:
Handle:
Logo:
Red:
Large:
ACTUALPosition v. Actual r2 = 0.03
� = +0.18
0% 50% 100%
Boot:
Flap:
Mesh:
Cell:
PDA:
Handle:
Logo:
Red:
Large:
October 15, 2001, 1pm (n=52)
This suggests a need to have stocks randomly ordered by traderThis suggests a need to have stocks randomly ordered by trader
Do STOC sellers rank concepts lower than buyers?
-7 -6 -5 -4 -3 -2 -1 0 1 2 3 4 5 6 7
STOC
Random0%5%
10%15%20%25%30%
% o
f T
rade
s
Rank Difference (Buyer - Seller)
STOC Traded Vehicles Are Close in Rank(n = 165 transaction pairings)
62% of trades are in the "right"
direction
Conclusions about STOC
• Functions well with informed traders
• Is accurate at identifying winners
• Measures preferences in the aggregate
• Traders behave heterogeneously, learn
• Predicts well for many product categories
• Can be effective with concepts AND attributes
• Real outcomes are not absolutely necessary
Top Related