Analysis of Experimental Data IV

Christoph Engel

non-linear dpg

I. binary dvII. dv with ordered discrete stepsIII. censored dvIV. dv with unordered discrete

choices

illustrations

binary responder in ultimatum game

ordered vote for contribution level

0 / 10 / 20 censored

contributions to public good unordered

public official in bribery experiment reject accept, but cheat accept and grant favor

reason to go non-linear?

outside the lab sample as proxy for true dgp

in the lab dgp follows from design e.g.: ? is action space constrained

public good contributions in [0, 20]

is problem small enough to be ignored?

I. binary

dgp hdv = 5 + .5*level + error dv = 1 if hdv > 30

020

4060

hdv

0 20 40 60 80 100level

latent

0.2

.4.6

.81

dv

0 20 40 60 80 100level

observed

linear probability model0

2040

60

0 20 40 60 80 100level

hdv Fitted values

latent

-.50

.51

1.5

0 20 40 60 80 100level

dv Fitted values

observed

interpretation of prediction as probabilitybut: some predicted values out of range(< 0 or > 1)

additional problem0

2040

60

0 20 40 60 80 100level

hdv Fitted values

latent

0.5

11.

5

0 20 40 60 80 100level

dv Fitted values

observed

bias if a lot of mass on one end use non-linear model

non-linear model-.5

0.5

11.

5

0 20 40 60 80 100level

data OLS Logit

mass on one end0

.51

1.5

0 20 40 60 80 100level

data OLS Logit

which model?

logit or probit a matter of taste different distributional assumption

probit normality

logit logistic distribution

logit more robust faster coefficients can be directly interpreted

statistical model

nice mathematical properties exp(.) is positive exp(.)/(1+exp(.)) goes to

1 if (.) is positive and large 0 if (.) is negative and large

ii

iii x

xdv

1

1

exp(1exp(

standard output

_cons -10.32078 1.947648 -5.30 0.000 -14.1381 -6.503457 level 1.134157 .2107922 5.38 0.000 .7210122 1.547302 dv Coef. Std. Err. z P>|z| [95% Conf. Interval]

Log likelihood = -29.001404 Pseudo R2 = 0.9011 Prob > chi2 = 0.0000 LR chi2(1) = 528.37Logistic regression Number of obs = 1000

Iteration 8: log likelihood = -29.001404 Iteration 7: log likelihood = -29.001404 Iteration 6: log likelihood = -29.001452 Iteration 5: log likelihood = -29.052485 Iteration 4: log likelihood = -30.739153 Iteration 3: log likelihood = -57.215141 Iteration 2: log likelihood = -111.90476 Iteration 1: log likelihood = -196.48485 Iteration 0: log likelihood = -293.18427

. logit dv level

odds ratio

level 3.108553 .6552586 5.38 0.000 2.056514 4.698777 dv Odds Ratio Std. Err. z P>|z| [95% Conf. Interval]

Log likelihood = -29.001404 Pseudo R2 = 0.9011 Prob > chi2 = 0.0000 LR chi2(1) = 528.37Logistic regression Number of obs = 1000

. logit, or

rewrite

interpretation

marginal effect of 1 unit change in iv

on odds ratio

ii

i xpp

111ln

log-linear

ln is good approximation of % change

020

4060

8010

0

0 10 20 30 40id

percent change ln(level)

example

predicted prob at level = 10 .735 odds 735/265 = 2.775

predicted prob at level = 11 .896 odds 896/104 = 8.627

odds ratio of 1 unit change 8.627/2.775 = 3.1085

holds for all comparisons!

why so complicated?

change in probability not the same all over 10 11

.8961308-.7351278 = .1610030

19 20 .9999957-.999986

7 = .00000900

.51

1.5

0 20 40 60 80 100level

data OLS Logit

drawbacks

change in odds ratio not overly intuitive only works for logit

not for any other non-linear model different questions

marginal effect at average of all ivs

average marginal effect ~ conditional on one iv

OLS coef = answer to all of them not true for non-linear model

mathematics: OLS

model

marginal effect of 1 unit change in x1 = partial first derivative wrt x1

= beta1

iii xy 1

logit

model

marginal effect of 1 unit change in x1

illustration

dgp hdv = 40 - 3*treat + .5*level + error dv = (hdv > 50)

graph0

.2.4

.6.8

1

0 20 40 60 80 100level

dv Pr(dv) prediction from logit

marginal effect at means

level .0218054 .0072921 2.99 0.003 .0075132 .0360976 dy/dx Std. Err. z P>|z| [95% Conf. Interval] Delta-method

level = 50.5 (mean)at : treat = 4.509 (mean)dy/dx w.r.t. : levelExpression : Pr(dv), predict()

Model VCE : OIMConditional marginal effects Number of obs = 1000

. margins, dydx(level) atmeans

average marginal effect

. margins, dydx(level) atmeansConditional marginal effects Number of obs = 1000Model VCE : OIMExpression : Pr(dv), predict()dy/dx w.r.t. : levelat : treat = 4.509 (mean) level = 50.5 (mean)------------------------------------------------------------------------------ | Delta-method | dy/dx Std. Err. z P>|z| [95% Conf. Interval]-------------+---------------------------------------------------------------- level | .0218054 .0072921 2.99 0.003 .0075132 .0360976------------------------------------------------------------------------------. margins, dydx(level)Average marginal effects Number of obs = 1000Model VCE : OIMExpression : Pr(dv), predict()dy/dx w.r.t. : level------------------------------------------------------------------------------ | Delta-method | dy/dx Std. Err. z P>|z| [95% Conf. Interval]-------------+---------------------------------------------------------------- level | .0239235 .001772 13.50 0.000 .0204504 .0273965------------------------------------------------------------------------------

ME level,conditional on treat

0.0

2.0

4.0

6.0

8.1

mar

gina

l effe

ct o

f 1 u

nit i

ncre

ase

in le

vel

0 2 4 6 8 10treatment

marginal effect confidence interval

explanation

recall dgp hdv = 40 - 3*treat + .5*level + error dv = 1 if hdv > 50

if treat is large hdv always < 50

if treat is very small hdv always close to 50

ME with interactions

dgp hdv = 40 +2*treat + .25*level

- .05*treat*level + error dv = (hdv > 47)

predicted from logit0

.2.4

.6.8

1P

r(dv

)

0 20 40 60 80 100level

statistical model

_cons -13.40162 1.269447 -10.56 0.000 -15.88969 -10.91355 c.level -.0977411 .00795 -12.29 0.000 -.1133227 -.0821594 c.treat# level .4938895 .07228 6.83 0.000 .3522233 .6355556 treat 3.793513 .6003316 6.32 0.000 2.616884 4.970141 dv Coef. Std. Err. z P>|z| [95% Conf. Interval]

Log likelihood = -136.00823 Pseudo R2 = 0.8038 Prob > chi2 = 0.0000 LR chi2(3) = 1114.08Logistic regression Number of obs = 1000

Iteration 6: log likelihood = -136.00823 Iteration 5: log likelihood = -136.00823 Iteration 4: log likelihood = -136.00827 Iteration 3: log likelihood = -136.04407 Iteration 2: log likelihood = -141.21212 Iteration 1: log likelihood = -188.96252 Iteration 0: log likelihood = -693.04918

. logit dv c.treat##c.level

ME must take into account

mathematics using chain rule partial derivative wrt one main effect

Stata does if properly informed about interaction

average MEs

level .0021692 .0023519 0.92 0.356 -.0024405 .0067788 treat -.0431288 .0223182 -1.93 0.053 -.0868717 .0006141 dy/dx Std. Err. z P>|z| [95% Conf. Interval] Delta-method

dy/dx w.r.t. : treat levelExpression : Pr(dv), predict()

Model VCE : OIMAverage marginal effects Number of obs = 1000

. margins, dydx(*)

on average marginal change in level immaterial

hides more complex story

-.05

0.0

5

0 2 4 6 8 10treatment

marginal effect confidence interval

marginal effect of 1 unit increase in level

opposite story for treat

-.001

0.0

01.0

02.0

03.0

04

0 2 4 6 8 10level

marginal effect confidence interval

marginal effect of 1 unit increase in treat

on average significantly different from 0but not conditional on specific levels

II. ordered

dgp hdv = 5 + .5*level + error dv = 0 if hdv < 20 dv = 1 if hdv in [20,40] dv = 2 if hdv > 40

linear model0

2040

60

0 20 40 60 80 100level

hdv Fitted values

latent

0.5

11.

52

2.5

0 20 40 60 80 100level

dv Fitted values

observed

ordered logit0

.2.4

.6.8

1

0 20 40 60 80 100level

Pr(dv==0) Pr(dv==1) Pr(dv==2)

too conservative

if steps in dv have cardinal interpretation 1-0 = 2-1

two options count model interval regression

differences distributional assumptions

count model: Poisson intreg: normal

intreg is linear coefficients can be directly interpreted

intreg

sigma .0488678 .0052795 .0395423 .0603926 /lnsigma -3.018636 .1080365 -27.94 0.000 -3.230383 -2.806888 _cons -.2177289 .0149256 -14.59 0.000 -.2469825 -.1884752 level .0245559 .0002828 86.83 0.000 .0240016 .0251102 Coef. Std. Err. z P>|z| [95% Conf. Interval]

Log likelihood = -72.312708 Prob > chi2 = 0.0000 LR chi2(1) = 2167.01Interval regression Number of obs = 1000

much more statistical power

. intreg ldv udv level

III. censored

dgp hdv = -20 + .5*level + error dv = hdv if hdv > 0 else dv = 0

censored-2

0-1

00

1020

30

0 20 40 60 80 100level

hdv Fitted values

latent

-20

-10

010

2030

0 20 40 60 80 100level

dv Fitted values

observed

linear model biased

solution: Tobit

0 right-censored observations 611 uncensored observations Obs. summary: 389 left-censored observations at dv<=0 /sigma .9402592 .0267063 .8878523 .9926662 _cons -19.77266 .1487295 -132.94 0.000 -20.06452 -19.48081 level .4971101 .002079 239.11 0.000 .4930305 .5011897 dv Coef. Std. Err. t P>|t| [95% Conf. Interval]

Log likelihood = -843.49549 Pseudo R2 = 0.7035 Prob > chi2 = 0.0000 LR chi2(1) = 4002.16Tobit regression Number of obs = 1000

. tobit dv level, ll(0)

prediction-2

0-1

00

1020

30

0 20 40 60 80 100level

data linear prediction Tobit prediction

Tobit

procedure

maps zeros into negatives assumes

latent variable incompletely observed

all data points are observed but some are only observed to be

censored

assumption appropriate?

dictator game dictators would even want to take warranted

Bardsley ExpEc 2008 punishment in public good

non-punishers would even want to reward

warranted?

what if not?

dgp hurdle

hdv = pers + error1 dv = 0 if hdv < 0

conditional on hurdle being passed dv = 5 + .5*level + error2 if hdv > 0

Tobit biased0

2040

60

0 20 40 60 80 100level

data predicted

Tobit

single hurdle model

/sigma .9963187 .0256565 38.83 0.000 .9460329 1.046605 _cons 5.06777 .0733597 69.08 0.000 4.923988 5.211552 level .4993736 .0012627 395.47 0.000 .4968987 .5018486 dv Coef. Std. Err. z P>|z| [95% Conf. Interval]

Log likelihood = -1067.0988 Prob > chi2 = 0.0000 upper = +inf Wald chi2(1) = 1.6e+05Limit: lower = 0 Number of obs = 754Truncated regression

Iteration 2: log likelihood = -1067.0988 Iteration 1: log likelihood = -1067.0988 Iteration 0: log likelihood = -1067.1001

Fitting full model:

(note: 246 obs. truncated). truncreg dv level, ll(0)

_cons .0343676 .1172594 0.29 0.769 -.1954565 .2641918 pers -1.864497 .1599703 -11.66 0.000 -2.178033 -1.550961 zero Coef. Std. Err. z P>|z| [95% Conf. Interval]

Log likelihood = -486.48539 Pseudo R2 = 0.1280 Prob > chi2 = 0.0000 LR chi2(1) = 142.82Logistic regression Number of obs = 1000

double hurdle model

dgp first hurdle

hhd = -.5 + 2*pers + error1 hd = (hhd > 0)

second hurdle and above hhdv = -10 + .5*level + error2 hdv = hhdv*(hhdv > 0) dv = hdv*hd

second process also generates zeros

0.1

.2.3

Den

sity

0 10 20 30 40hdv

first hurdle

0.1

.2.3

Den

sity

0 10 20 30 40dv

second hurdle

estimation

. dhreg dv level, hd(pers)

_cons -.5181453 .0876164 -5.91 0.000 -.6898703 -.3464203 pers 2.134447 .1254733 17.01 0.000 1.888524 2.380371hurdle _cons 1.977432 .0565331 34.98 0.000 1.866629 2.088235sigma _cons -9.984145 .2095 -47.66 0.000 -10.39476 -9.573533 level .4999016 .0032727 152.75 0.000 .4934872 .506316above Coef. Std. Err. z P>|z| [95% Conf. Interval]

Log likelihood = -1574.3771 Prob > chi2 = 0.0000 Wald chi2(1) = 23332.16 Number of obs = 1000

prediction0

1020

3040

0 20 40 60 80 100level

data predicted

Double Hurdle Model

IV. unordered discrete

dgp latent

hdv0 = - 10 + 1.2*level - 8*type + error1 hdv1 = 5 - .2*level + 5*type + error2 hdv2 = .8*level - .7*type + error3

observed dv = 0 if hdv0 > hdv1 & hdv2 dv = 1 if hdv1 > hdv0 & hdv2 dv = 2 if hdv2 > hdv0 & hdv1

estimation

_cons 11.2145 1.398386 8.02 0.000 8.473716 13.95529 type 8.328941 1.025337 8.12 0.000 6.319317 10.33857 level -.4560097 .0551126 -8.27 0.000 -.5640284 -.34799112 _cons 16.56466 1.760686 9.41 0.000 13.11378 20.01554 type 14.60946 1.57338 9.29 0.000 11.52569 17.69322 level -1.517425 .1997494 -7.60 0.000 -1.908926 -1.1259231 0 (base outcome) dv Coef. Std. Err. z P>|z| [95% Conf. Interval]

Log likelihood = -100.83695 Pseudo R2 = 0.8958 Prob > chi2 = 0.0000 LR chi2(4) = 1734.50Multinomial logistic regression Number of obs = 1000

prediction0

.2.4

.6.8

1

0 20 40 60 80 100level

0 1 2

type 1

0.2

.4.6

.81

0 20 40 60 80 100level

0 1 2

type 2

0.2

.4.6

.81

0 20 40 60 80 100level

0 1 2

type 3

example0

.2.4

.6.8

1

0 20 40 60 80 100level

0 1 2

type 1

The end