On the Heterogeneity, Stability and Validity of Risk Preference Measures

On the Heterogeneity, Stability and Validity of Risk Preference Measures

Chetan

Dave

Catherine

Eckel

Cathleen

Johnson

Christian

RojasUniversity of Texas

at DallasUniversity of Texas

at DallasUniversity of

ArizonaUniversity of

Massachusetts (Amherst)

ESA, July, 2007

Measuring Preferences: Key Questions

Are risk preferences stable across elicitation measures? Since Slovic (1964), considerable evidence they are not

Do risk preferences vary systematically by subject characteristics in the same way across measures? (Gender, age, income)

Does math ability affect elicited preferences? Ability: Low/Medium/High Math Literacy

Are risk preferences stable across time? Two Datasets: Test Data vs. Re-Test Data

Motivation

Lots of folks are working on heterogeneity in risk preferences (many of them are here)

There are lots of elicitation tasks, depending on the purpose of the study

Our purpose: to develop a simple, easy to administer measure of individual risk preferences that can be used to “predict” or better understand individual choices. Household decisions, human capital choices, technology

adoption

Motivation, cont’d

We compare two lottery choice methods (more on these soon)

Non-student data set (collected for another purpose)

Subjects complete both risk elicitation measures

Survey, math literacy measure Subset of pre-selected subjects participated

in a retest

The CSL Study

Purpose: study barriers to post-secondary acquisition of human capital

Data: lab experiments with non-traditional subject pool ~900 adult participants in Canada (recruited, volunteers) 102 sessions Ages 18-54 years Urban and Non-Urban samples 156 participate in retest 6 months later

Average earnings: CAD$165 Subjects paid for one decision chosen at random 100 decisions: 40 time pref; 32 risk and ambiguity; 28

financing for post-secondary education, max = $1000 grant, $5000 guaranteed loan

The CLS Study (cont’d.)

Two experiments to elicit risk preferences: Holt-Laury (2002)

10 binary choices between more and less risky gambles

Eckel-Grossman (2002) Choice of 1 from set of 6 50/50 gambles

Discrete binary response data 0 = less risky choice 1 = more risky choice

Holt-Laury Task

OPTION A Your Choice A or B

OPTION B

Decision 42 1/10 of $40 9/10 of $32 A 1/10 of $77 9/10 of $2




Decision 46 5/10 of $40 5/10 of $32 B 5/10 of $77 5/10 of $2






Eckel-Grossman Task

x

Short Detour

FAQ: How do you code Eckel-Grossman as a set of binary choices?

Notice the gambles are ordered Increase linearly in risk and return as you

go down the table One choice implies binary ordering

Figure 2: Risk and Return of Gamble Choices

25

27

29

31

33

35

37

0 10 20 30 40

Risk (Standard Deviation)

Exp

ecte

d R

etu

rn

1

65

4

3

2This person chooses Gamble 3This implies 5 binary choices:

2 is preferred to 1

3 is preferred to 2

3 is preferred to 4

4 is preferred to 5

5 is preferred to 6

Comparison of Eckel-Grossman and Holt-Laury

H-L: varying probabilities, constant payoffs

E-G: constant probability, varying payoffs, all 50-50 gambles

Cumulative Distribution of r values

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

-2 -1 0 1 2 3 4

r value

Per

cen

t

Holt/Laury

Eckel/Grossman

The Numeracy Measure

Subcomponent of the Educational Testing Service’s Adult Literacy and Lifeskills Survey (ALLS)

31 problems involving the use of mathematics in real-life situations.

Low numeracy ≤ 1 standard dev. below average

High numeracy ≥ 1 standard dev. above average

Model and Estimation

Given a CRRA utility function for money (M) parameterized by r

U(M|r) = M1-r/(1-r) Can construct a likelihood function

L(r,,|Y) where measures ‘noise’ in binary risk

choices (Y) r and can also be made a function of

observed characteristics of subjects

Results 1: Does the Elicitation Instrument Affect Estimates?

Estimates of CRRA Utility Function

Estimate Std. Err. p-value Estimate Std. Err. p-value

r 0.449 0.024 0.000 0.626 0.020 0.000 0.044 0.002 0.000 0.122 0.007 0.000

Eckel-Grossman Task Holt-Laury Task

r = Coefficient of relative risk aversion

= Logit error parameter

Differences significant at p = 0.000, 2(1) = 52.62

Estimates of CRRA Utility (cont’d.)

Eckel-Grossman and Holt-Laury give significantly different estimates of r

“Ruler” affects measure! H-L has higher error parameter than E-G

Why?

Results II: How do demographics and ability affect estimates ?

IIA: Effect on preference parameters

IIB: Effect on parameters and errors

IIC: Are parameters stable across instruments?

Effect on preference parametersEstimateStd. Err.p-value

r Instrument (H/ L) 0.219 0.038 0.000 Consistent (44% are) 0.026 0.023 0.265 Female 0.007 0.025 0.782 Low Income (<$CA30K) 0.037 0.029 0.202 Medium Income ($CA30-65K) 0.045 0.027 0.098 Young (<25) -0.100 0.031 0.001 Middle (25-44) -0.055 0.025 0.025 Low Math Score (-1 std dev) 0.199 0.041 0.000 Medium Math Score 0.041 0.040 0.305 Constant 0.461 0.043 0.000 0.080 0.007 0.000

Effect on Preference Parameters (cont’d.)

Income, Age and Math Literacy are significant determinants (correlates) of preferences

Low Income: more risk averse Young: less risk averse Low math score: appear more risk averse

Could this be due to greater errors?

Effect on Parameters and Errors

EstimateStd. Err.p-valuer Instrument 0.107 0.053 0.041 Consistent 0.093 0.038 0.015 Female 0.128 0.034 0.000 Low Income 0.011 0.039 0.767 Medium Income 0.046 0.052 0.380 Young -0.030 0.039 0.445 Middle -0.047 0.045 0.298 Low Math Score 0.133 0.053 0.012 Medium Math Score 0.035 0.037 0.336 Constant 0.388 0.055 0.000

Effect on Parameters and Errors (cont’d)

Estimate Std. Err. p-value

Instrument 0.058 0.008 0.000 Consistent -0.017 0.006 0.008 Female -0.028 0.009 0.002 Low Income 0.006 0.007 0.377 Medium Income -0.001 0.013 0.937 Young -0.034 0.011 0.002 Middle -0.024 0.015 0.123 Low Math Score 0.039 0.022 0.076 Medium Math Score -0.005 0.008 0.520 Constant 0.090 0.015 0.000

IIB. Parameters and Errors (cont’d.)

People who are consistent are more risk averse and have less noise (randomness)

Young have less noise Lower numerical literacy increases

randomness in risk parameters; appear more risk averse

Females are more risk averse and have lower noise

IIIC: Parameter stability across measures

Is the impact of gender, age, income similar across measures?

Is the impact of math literacy similar across measures?

Separate regressions for EG and HL instruments E/ G H/ L Estimate p-value Estimate p-value r Consistent -0.061 0.543 0.175 0.000 Female 0.364 0.000 0.099 0.009 Low Income -0.015 0.864 0.018 0.670 Medium Income -0.014 0.870 0.049 0.187 Young (<25) 0.072 0.367 -0.017 0.752 Middle (25-44) 0.054 0.655 -0.066 0.233 Low Math Score -0.065 0.668 -0.049 0.500 Medium Math Score -0.188 0.190 -0.019 0.642 Constant 0.481 0.000 0.529 0.000 Consistent -0.009 0.327 -0.031 0.003 Female -0.025 0.000 -0.033 0.008 Low Income 0.008 0.380 0.005 0.690 Medium Income 0.011 0.157 -0.006 0.614 Young (<25) -0.026 0.015 -0.045 0.031 Middle (25-44) -0.027 0.001 -0.015 0.489 Low Math Score 0.022 0.271 0.126 0.001 Medium Math Score -0.006 0.594 0.020 0.084 Constant 0.072 0.000 0.136 0.000

Distribution of noise parameter0

.2.4

.6.8

1

0 .1 .2 .3Linear prediction

cum_mu_HL cum_mu_HL_lowcum_mu_HL_med cum_mu_HL_highcum_mu_EG

How good are predictions?

Calculate index:

n is the number of decisions for the task (5 for Eckel-Grossman and 10 for Holt-Laury)

‘fraction’ variables represent the true and the predicted fraction of safe choices for decision i across all individuals.

Holt-Laury task has indices of 0.79, 0.88 and 0.92 for low, medium and high math literacy subjects

Eckel-Grossman instrument has a precision index of 0.81.

1

Precision 1 1/ Actual Fraction Predicted Fractionn

i iin

IIC: Comparison

Women are substantially more risk averse in E/G; lower noise in both

Young have lower noise in both Low math literacy affects noise in H/L only Noise parameter is similar to E/G for H/L

high literacy, but higher for med and especially low literacy

“Precision” is equivalent for high literacy, not for Medium or Low.

Results III: Are Preferences Stable Across Time?

IIIA. Eckel-Grossman

IIIB. Holt-Laury

IIIA. Stability of Eckel-Grossman

EG InstrumentEstimat

eStd. Err.

p-value

r Re-test 0.080 0.090 0.377Constant 0.432 0.061 0.000 Re-test -0.032 0.008 0.687Constant 0.052 0.005 0.000

LogL: 920.631# obs 1560

IIIB. Stability of Holt-Laury

HL Instrument

EstimateStd. Err.

p-valu

er Re-test 0.118 0.080 0.141Consta

nt 0.707 0.055 0.000 Re-test -0.044 0.028 0.122Consta

nt 0.112 0.020 0.000

LogL: 1182.848

# obs 3,120

Concluding Remarks

Question # 1: Are preferences stable across measures used to elicit them? Answer: A qualified No. Elicitation

instruments do matter to an extent Question # 2: Do preference

estimates by subject characteristics vary across measures? Answer for Demographics: Yes

Concluding Remarks (cont’d.)

Question #3: Are preference measures sensitive to math literacy: Answer: Yes, H/L only

Question # 4: Are preference measures stable across time? Answer: Yes. But E-G does “better”

than H-L Question #A1: Internal v. external

validity?

Validity: Correlations of risk measures with “real” decisions.

Hs : 80 obs Eckel/ Grossman Holt/Laury

Preference for $1000 grant - full time 0.21 0.14

Preference for $1000 grant - Part time 0.18 0.14

At least one loan preferred over cash 0.21 0.12At least one grant preferred over cash 0.08 0.15Not HS : 801 obs

Preference for $1000 grant - full time 0.04 0.01

Preference for $1000 grant - Part time 0.10 0.00

At least one loan preferred over cash 0.05 0.06

At least one grant preferred over cash 0.08 -0.02

Measuring preferences

Need to develop instruments that are appropriate for population, purpose

Finer screen v. accessibility tradeoff For literate population, no tradeoff For medium and low literate population, there

may be a serious tradeoff! Considerable precision is lost with more

complex instrument Important to take care with design for low-

literacy populations

Ex: Next gen of instruments (risk)

On the Heterogeneity, Stability and Validity of Risk Preference Measures

Documents

Transcript of On the Heterogeneity, Stability and Validity of Risk Preference Measures