Does mixing F2F and web lead to cost savings?
description
Transcript of Does mixing F2F and web lead to cost savings?
Does mixing F2F and web lead to cost savings?Mari Toomse-Smith
@MariToomseSmith
18 July 2013 ESRA2013
2
Contents Background
About the study
Response
Potential for cost savings
Actual cost savings
Cost model
Conclusions
Background
1.
4
Context Increasing attention on mixed modes
Potential for cost savings
Potential for increasing response
Unclear to what extent this potential realised in reality Especially if face-to-face involved
5
Survey I
NatCen ran for ISER at University of Essex
Main survey largest household panel in the world
IPs test innovations and carry experiments
IP5 – first test of F2F/Web in a large F2F panel survey
6
Survey II
Household survey: Household questionnaire Individual questionnaire Adult self-completion Youth self-completion
Split off households
Sample: original W1, W4 refresh, W4 non-respondents
15 experiments
Interview length 60 minutes per household
Incentives: £5, £10, £20, £30
7
Design
Experimental group
F2F phase, web open
Web only phase
Control group F2F only phaseNo web
Response
2.
9
Web take-up
23% Households completed fully onlineAnother 13% completed partially
10
Household response by sample type, original responding sample
Base: All issued households in original sample (responding)
Source: Jäckle, Lynn, Burton (2013)
78
20
9
74
25
14
Household responserate
Parital household Refusals
F2F Mixed mode
11
Household response by sample type, refreshment sample
Base: All issued households in refreshment sample
Source: Jäckle, Lynn, Burton (2013)
85
25
8
82
1610
Household responserate
Partial household Refusal
F2F Mixed mode
12
Web response by incentive amount
Base: All households in mixed mode sample
Source: Jäckle, Lynn, Burton (2013)
17 21
3742
7176
8292
£5 £10 £20 £30
Full household web completionTotal household response rate
Cost
3.
14
Expected effect on cost
Decrease No interviewing fees for web completes No trips required to web completes
Increase More programming: web and F2F questionnaires Set up and manage emails Man a helpline for the web group
15
One-off costs Research time to develop new procedures and
support the mixed modes
Develop sample management system
Update response monitoring tools
Field management cost to support interviewers
16
Effect on costs
Total costs up by third
Fielwork costs down by tenth
Programming cost up three times
Research cost up by two-thirds
17
Fieldwork costs Main source of savings:
Fees for productive cases Marginal savings on:
Travel time and mileage – 3% Why travel less affected:
Main source of cost – travel to and from PSUs Smaller points – more travel Web cases in F2F more difficult – require more calls
18
Cost of achieved interview by incentive amount
Base: All issued households
£5 £10 £20 £30
F2F Mixed mode
19
Simulated costs per achieved sample size
500 1000 1500 2000 2500 3000 3500 4000 4500 5000 5500 6000 6500Achieved sample size
F2F
MM
Conclusion
4.
21
Conclusions Mixed modes take up is higher than expected No evidence that response in mixed modes is higher Risk of more refusals and partial households Costs increase in the first year Fieldwork costs do not decrease as much as might be
expected Increase in web response with higher incentives is not
enough to offset the incentive value Mixed modes design becomes cost effective at higher
achieved sample sizes
If you want further information or would like to contact the author,
Mari Toomse-Smith
Senior Research Director
T. +44 020 7549 9580
Visit us online, natcen.ac.uk
Thank you