Quantitative Methods in the Social Sciences (QMSS) Lugano 25-26 August 2005 Increasing response...

Quantitative Methods in the Social Sciences (QMSS) Lugano 25-26 August 2005

Increasing response rates

Ineke StoopSCP


Is it possible?


and is it effective?


Nonresponse research

• How to increase response rates?• How to measure and correct for

nonresponse bias?


SCP-survey

• Face-to-face questionnaire + long drop off for every 6+ years household member

• Use of social and cultural services, amenities, facilities (no values, opinions)

• Very low non-contact rate• High rate of refusal conversion• Final response rate 1999 66%


Response rates AVO

0

10

20

30

40

50

60

70

1979 1983 1987 1991 1995 1999 2003


Cumulative response rates

0

10

20

30

40

50

60

70

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20

1995 1999


Increase response ratesDescribe response process

ContactabilityReluctance


Paradata

• Call records (nr, timing, outcome)• Characteristics interaction• Interviewer observations• Sample frame data• Low-level geographic data bases• Interviewer characteristics


ContactabilityTNC/NAH/CFC

Call (nr, time, mode)First

contact


At home, interviewer calls, contact rate

0

10

20

30

40

50

60

70

80

90

100

6 12 18 24

morning afternoon evening

0

10

20

30

40

50

60

70


Timing contacts

• Interviewers prefer working hours• Contact rates higher in the evening• Cooperation independent of timing

• Fewer calls if evening calls only• Length fieldwork period/costs?


Contactability

• Easy– Evening calls– Phone number

available– Detached dwelling– Child at home

• Difficult– Big city dweller– Poor maintenance

neighbourhood– Young– Single– Employed, student– High cultural

participation


Continuum of resistance

• Extrapolate from hard to reach respondents to noncontacts?– Socio-demographics– Core variables– Two types of hard to contact (short

term/long term)


Reluctance

Respondent has been contacted


Reluctance

Request to cooperate

Inter-view

Cooperative respondent

Refusal

Establishfirst contact

Follow-up

Con-tact

Inter-view

Refusal


Soft convertedrefusal

Inter-view

Follow-up

Con-tact

Refusal

Refusal


Refusal

Hard converted refusal

NoYes

Yes

Yes

Yes

Yes

Yes

Yes

NoNo

No

NoNo

No


What happens in the AVO after a first refusal?

Refusal at first contact

Finalrefusal

Re-approached

Inter-view

Finalrefusal


Reluctant respondents

• Many among– Big city dwellers

• Few among– Men– Singles– Higher educated


Cooperative and reluctant respondents

0 10 20 30 40 50 60 70 80 90 100

total

couple

single male

Mid east

Big cities

immediate converted


Refusers, reluctant and cooperative respondents

0 10 20 30 40 50 60 70 80 90 100

total

couple

single male

Mid east

Big cities

total

couple

single male

Mid east

Big cities

immediate converted refuse


Region and refusal conversion

14

25

51

45

Amsterdam, Rotterdam, The Hague

Mid east

48

-34

7

15

-40 -20 0 20 40 60

still in the field immediate nonresponse re-issue, 2nd refusal

re-issue, no 2nd refusal converted nonrespondents

immediate cooperation


Results refusal conversion

• Males, singles and higher educated underrepresented among converted refusals– Do interviewer strategies work for

everybody?• Are only promising cases re-

contacted? – May worsen final sample composition

• Effect refusal conversion– Works only for honest burghers, Mr.

and Mrs. Average?


Conclusions regular AVO

• Contactability related to socio-demographics, being out of the house and fieldwork strategy

• Reluctance related to family composition, gender and education and persuasiveness


Do nonrespondents differ?

Are final refusals similar to converted refusals

• Sample frame information• Neighbourhood information• Fieldwork information

– Late respondents (similar to non-contacts?)

– Reluctant respondents (similar to refusers?)

• Socio-demographic information


Response and nonresponse

Frame

Observation

EasyFieldwork

Hard to contact

Reluctant

Follow-up


Follow-up survey

• Small subsample persistent refusers

• 1 person, 20 minutes, multi-mode• Experienced, motivated, well paid

interviewers• Wide range of incentives• 235 successful interviews• 70% cooperation rate


Why did refusers cooperate?

• High quality interviewers (and telling them they are the best)

• Extensive briefing• Trust (money for incentives) and

support (newsletter)• Importance (newspaper article)• Wide range of incentives• Better payment• Commitment sponsor and fieldwork

organisation


Idiosyncratic choice of incentives


Final analysis

Hard to contact respondentsReluctant respondents

Refusers who cooperated in follow-up survey


Follow-up survey• Sample of persistent refusals• Socio-demographic differences (made

up for composition regular respondents)

• Small differences in survey variables (mostly related to socio-demographics)

• Lower participation in classical culture• Less PC ownership• Fewer sports activities• Slightly less active in many aspects• Not at all similar to converted refusals


refuser reluctant hard to contact

Intercept -2,42 -0,04 -0,38

Age (years) -0,01 0,00 -0,01

Age: absolute deviation mean (years)

-0,01 -0,01 -0,01

Listed phone number 0,50 -0,10 -0,26

Amsterdam, Rotterdam, The Hague -0,22 0,69 0,63

Good maintenance -0,21 -0,26 -0,26

Detached dwelling 0,00 -0,42 -0,28

Male -0,16 -0,76 -0,05

Single 0,14 -0,37 0,29

Child at home -0,30 -0,11 -0,37

Employed, student 0,17 -0,13 0,31

Higher education -0,15 -0,51 -0,03

Cultural participation

Both classical and popular -0,21 -0,03 0,13

Classical -0,56 0,23 0,17

Popular 0,17 -0,07 0,15

No sports activities 0,41 -0,03 0,00

PC in household -0,87 0,18 0,00

Use of internet, e-mail 0,94 0,07 0,14

No religious affiliation 0,38 -0,11 0,00


single female

single malesingle parent

couplechildren

couple

no vcr

ethnic minority

25+% ethnic

no ethnic minorities

Randstad

South

Mid east

North east

AmsterdamRotterdamThe Hague

detached

multi-unit

terraced

no phone

75+

16-24

no sports

many sports

no culture

classical

popularculturalomnivore

no pc

pc not used

pc off line

internet

EASY

RELUCTANTREFUSER

HARD TO CONTACT

poor maintenance

excellent

not employed, student

employed, student

atheist

religious

no naturenature

avid reader

no booksprimary ?education?

academic

?income?

veryhigh

income

vey low income

-1,2

-0,8

-0,4

0

0,4

0,8

1,2

1,6

2

-1,6 -0,8 0 0,8

family composition video ethnic minority ethnic neighbourhood region

dwelling telephone age sports culture

PC response maintenance employed, student religion

nature conservancy books education income


Reducing bias instead of increasing response

rates• Increasing response rates MAY not lead

to better survey estimates• Difficult respondents may not be similar

to final nonrespondents– Obtain information on the process (cause

of/reason for nonparticipation)

• Collect independent information on nonrespondents– Frame information + registers– Central question procedure– Follow-up survey among refusers

Quantitative Methods in the Social Sciences (QMSS) Lugano 25-26 August 2005 Increasing response...

Documents

Transcript of Quantitative Methods in the Social Sciences (QMSS) Lugano 25-26 August 2005 Increasing response...