Does mixing F2F and web lead to cost savings?Mari Toomse-Smith

@MariToomseSmith

18 July 2013 ESRA2013

2

Contents Background

About the study

Response

Potential for cost savings

Actual cost savings

Cost model

Conclusions

Background

1.

4

Context Increasing attention on mixed modes

Potential for cost savings

Potential for increasing response

Unclear to what extent this potential realised in reality Especially if face-to-face involved

5

Survey I

NatCen ran for ISER at University of Essex

Main survey largest household panel in the world

IPs test innovations and carry experiments

IP5 – first test of F2F/Web in a large F2F panel survey

6

Survey II

Household survey: Household questionnaire Individual questionnaire Adult self-completion Youth self-completion

Split off households

Sample: original W1, W4 refresh, W4 non-respondents

15 experiments

Interview length 60 minutes per household

Incentives: £5, £10, £20, £30

7

Design

Experimental group

F2F phase, web open

Web only phase

Control group F2F only phaseNo web

Response

2.

9

Web take-up

23% Households completed fully onlineAnother 13% completed partially

10

Household response by sample type, original responding sample

Base: All issued households in original sample (responding)

Source: Jäckle, Lynn, Burton (2013)

78

20

9

74

25

14

Household responserate

Parital household Refusals

F2F Mixed mode

11

Household response by sample type, refreshment sample

Base: All issued households in refreshment sample

Source: Jäckle, Lynn, Burton (2013)

85

25

8

82

1610

Household responserate

Partial household Refusal

F2F Mixed mode

12

Web response by incentive amount

Base: All households in mixed mode sample

Source: Jäckle, Lynn, Burton (2013)

17 21

3742

7176

8292

£5 £10 £20 £30

Full household web completionTotal household response rate

Cost

3.

14

Expected effect on cost

Decrease No interviewing fees for web completes No trips required to web completes

Increase More programming: web and F2F questionnaires Set up and manage emails Man a helpline for the web group

15

One-off costs Research time to develop new procedures and

support the mixed modes

Develop sample management system

Update response monitoring tools

Field management cost to support interviewers

16

Effect on costs

Total costs up by third

Fielwork costs down by tenth

Programming cost up three times

Research cost up by two-thirds

17

Fieldwork costs Main source of savings:

Fees for productive cases Marginal savings on:

Travel time and mileage – 3% Why travel less affected:

Main source of cost – travel to and from PSUs Smaller points – more travel Web cases in F2F more difficult – require more calls

18

Cost of achieved interview by incentive amount

Base: All issued households

£5 £10 £20 £30

F2F Mixed mode

19

Simulated costs per achieved sample size

500 1000 1500 2000 2500 3000 3500 4000 4500 5000 5500 6000 6500Achieved sample size

F2F

MM

Conclusion

4.

21

Conclusions Mixed modes take up is higher than expected No evidence that response in mixed modes is higher Risk of more refusals and partial households Costs increase in the first year Fieldwork costs do not decrease as much as might be

expected Increase in web response with higher incentives is not

enough to offset the incentive value Mixed modes design becomes cost effective at higher

achieved sample sizes

If you want further information or would like to contact the author,

Mari Toomse-Smith

Senior Research Director

T. +44 020 7549 9580

E. Mari.toomse-smith@natcen.ac.uk

Visit us online, natcen.ac.uk

Thank you