NTTS conference, February 18 – 20 2009 New Developments in Nonresponse Adjustment Methods Fannie...

NTTS conference, February 18 – 20 2009

New Developments in Nonresponse Adjustment Methods

Fannie Cobben

Statistics Netherlands

Department of Methodology and Quality


In this presentation…

Response selection model Use of response propensities Application to POLS 2002 Discussion


The response selection model

Consists of two equations: Response equation (dichotomous) Survey item equation (continuous)

The error terms are allowed to be correlated The outcome is adjusted for a possible selection bias


Extensions to response selection model

Multiple selection equations, i.e. different response types

Contact equation Participation equation Survey item equation

Contact and participation are dependent and can both introduce a bias for the survey item If desirable, equations for other response types

Categorical survey items


Response selection model – contact and participation

Yi

YYii

Pi

PPii

Ci

CCii

Y

X

X

X

*

*

*

ςi*

γi*

Pi = 1

Ci = 1

Pi = 0

Ci = 0

Sample

Non-contact Contact

Refusal Participation

Yi* missing

Yi* = Yi Yi

* missing

else missing,

11 if ,*iii

i

CPYY


Advantages

Model relationship with both R and Y Efficient use of auxiliary variables; paradata Closely follow fieldwork process

Disadvantages

Model based; dependent on distributional assumptions

Issues of identification; exclusion restriction


Response propensities (1)

Assume: Sample is selected from a sampling frame by some random selection procedureTwo groups:R=1 responseR=0 non-response

X auxiliary information, available for all elements. For instance from the sampling frame.

ρ(X) = P(R=1| X) is the propensity score, for instance determined by a logistic regression, i.e. )(1

1)(

XfeX

)|1()(ˆ R of Propensity

)1( R ofy Probabilit

iiii

ii

XRPX

RP


Response propensities (2)

We can use the response propensities to adjust for nonresponse bias:

Directly Response propensity weighting Response propensity stratification

Indirectly In combination with linear weighting


Direct use of response propensities

Response propensity weighting, Särndal (1981)

rr n

i ii

in

iiiiwi

Y

NYgd

Nyg

11

ˆ

ˆ1

1

ˆ1

Response propensity stratification

F

f

fhtfr

n

iiiips

fr

fi

fr

f

i

ynN

YgdN

yn

ng

fn

fn

F ,,f

r

1

)(,

1

ˆ

,

,

11

stratumin srespondent ofnumber

stratum size

ˆon based strata ,21Construct


Indirect use of response propensities

GREG – estimator with adopted inclusion probabilities

1

)ˆ

()(1

1

ˆ

1-

1

'

r

r

n

iiiigr

i'ii

n

i i

irhti

YgdN

y

XXXρ

dxXg


POLS 2002

Integrated Survey on Household Living Conditions (in Dutch: Permanent Onderzoek LeefSituatie) Monthly 3.000 persons are selected Questions on living conditions, safety, health Basic module for persons > 12 years

Datafile aggregated over 2002: n = 35.594 and nr = 20.168 (57%)

Survey variables: Employment, Education and Religion Numerous auxiliary variables, such as age, region, house value, social insurance, ethnicity, etc.


Analysis POLS 2002

Two models:Weighting model (relation Y and R; Schouten, 2004)

Age15 + Houseval14 + % Non-natives8 + Ethnicity7 + Region15

+ Type hh4 + Telephone2

Response model (relation R; psuedo R2 = 2,2%)

Age15 + Houseval14 + Urbanicity5 + Mar_staat4 + Ethnicity7 + Region15

+ Type hh4 + Telephone2

(1)

(2)

Aim: Compare different response propensity methods Compare regular GREG-estimator and other methods


Propensity stratification

Propensity weighting

Survey item Response average

GREG

(1)

(1) (2) (1) (2) Propensity GREG

Employed

>= 12 hours 52,4

(0,35)

53,7

(0,18)

53,5

(0,38)

53,7

(0,38)

53,7

(0,34)

53,8

(0,34)

53,8

unemployed 6,7

(0,18)

6,4

(0,11)

6,4

(0,17)

6,4

(0,17)

6,4

(0,17)

6,4

(0,18)

6,4

< 12 hours 40,9

(0,35)

39,9

(0,17)

40,0

(0,35)

39,9

(0,37)

39,9

(0,34)

39,8

(0,33)

39,9

Results - Employment

No significant differences between method; clear difference with response average

GREG and propensity weighting with same model: same results

Propensity weighting and propensity GREG highest estimate employed labour force


Results - EducationPropensity

stratificationPropensity weighting


GREG

(1)


Education

Primary 7,2

(0,18)

6,3

(0,09)

6,3

(0,20)

6,3

(0,18)

6,3

(0,18)

6,3

(0,20)

6,3

MAVO 12,1

(0,23)

12,0

(0,15)

12,0

(0,23)

12,0

(0,22)

12,0

(0,25)

12,0

(0,24)

12,0

HAVO 19,7

(0,28)

19,9

(0,18)

19,9

(0,27)

19,9

(0,26)

19,9

(0,27)

19,9

(0,29)

19,9

VWO 7,1

(0,18)

7,2

(0,12)

7,2

(0,17)

7,2

(0,19)

7,2

(0,18)

7,2

(0,21)

7,2

MBO 30,8

(0,32)

31,1

(0,21)

31,0

(0,31)

31,1

(0,32)

31,1

(0,32)

31,0

(0,32)

31,0

HBO 16,7

(0,26)

16,9

(0,17)

16,8

(0,26)

16,9

(0,25)

16,9

(0,25)

16,9

(0,26)

16,9

WO 6,3

(0,17)

6,4

(0,11)

6,4

(0,16)

6,5

(0,18)

6,4

(0,19)

6,5

(0,18)

6,5

else 0,2

(0,03)

0,2

(0,02)

0,2

(0,03)

0,2

(0,03)

0,2

(0,03)

0,2

(0,03)

0,2


Propensity stratificiation

Propensity weighting


GREG

(1)


Religion

none 37,7

(0,34)

38,5

(0,21)

38,3

(0,35)

38,4

(0,34)

38,4

(0,37)

38,5

(0,37)

38,5

R.K. 33,5

(0,33)

32,4

(0,19)

32,7

(0,34)

32,6

(0,35)

32,5

(0,35)

32,5

(0,35)

32,5

Protestant 21,0

(0,29)

20,4

(0,17)

20,6

(0,28)

20,5

(0,30)

20,5

(0,35)

20,4

(0,29)

20,4

Islam 2,5

(0,11)

3,3

(0,06)

3,1

(0,18)

3,1

(0,11)

3,2

(0,12)

3,2

(0,12)

3,2

else 5,2

(0,16)

5,4

(0,01)

5,4

(0,16)

5,3

(0,16)

5,4

(0,15)

5,4

(0,15)

5,4

Results - Religion

More differences between methods

Propensity weighting and propensity GREG highest estimate no religion


Conclusions

Remarks: Almost the same variables in weighting model and

response model Low pseudo R2 response model

Conclusions:1. Small difference between methods2. No reduction of variance direct response propensity

methods; reduction of variance GREG-estimator3. Propensity weighting and GREG-estimator with

same model gives same results for employed and education


Thank you for your attention!


Total Survey Error

Total error

Sampling error

Estimation error

Selection error

Non-sampling error

Observation error

Over-coverage error

Measurement error

Processing error

Non-observation error

Under-coverage error

Nonresponse error


Probability based surveys (2)

Under-coverage in telephone surveys

Two groups: with telephone (C=1) without telephone (C=0)

Non-response

Two groups: Respondents (C=1) Non-respondents (C=0)

)|1()(ˆ propensity Telephone iiii XCPX

)|1()(ˆ propensity Response iiii XCPX


Discussion

Which variables should be inserted in the different equations, and how should these variables be selected? How to construct the hierarchical structure of the model. For instance, which response types should be distinguished? Is it profitable to construct a model for every survey item? If so, how should we deal with this in practice? The response selection assumes a correlation structure between response types and survey items. This correlation has a model-based interpretation, but how should it be interpreted in practice?


Example relationship R and Y in the LFS

Relation between the number of contact attempts and estimated size of the labour force

5800000

6000000

6200000

6400000

6600000

6800000

7000000

7200000

7400000

2002-01 2002-02 2003-03 2002-04 2003-01 2003-02 2003-03 2003-04 2004-01 2004-02 2004-03 2004-04

Kwartalen

We

rkza

me

be

roep

sb

ev

olk

ing

CAPI-A-max 1 CAPI-A-max 2 CAPI-A-max 4 CAPI-A

1 attempt

2 attempts

4 attempts

all attempts

Quarter

size of the labour force

NTTS conference, February 18 – 20 2009 New Developments in Nonresponse Adjustment Methods Fannie...

Documents

Transcript of NTTS conference, February 18 – 20 2009 New Developments in Nonresponse Adjustment Methods Fannie...