Knut Kalgraff SkjåkKirstine Kolsrud

2nd WEBDATANET Meeting, Amsterdam,

30 November 2011

Norwegian Social Science Data Services

Experiences with mixed mode mail & web-enquêtes

in probability samples with known individuals

Presented by Knut Kalgraff Skjåk

Background

International Social Survey Programme (ISSP)• Annual social survey• NSD, Norwegian member and responsible for ISSP Norway

since 1989• The ISSP surveys

– Topics of central importance for cross national social science (Role of Government, Social Inequality, Religion, Family and Gender Roles, Work orientation, Environment etc)

– 11 topics fielded to date, repeated at intervals

– 60 topic items + 40 background variables

– Mode of administration: self-administration and face to face

– National representative random samples aged 18 and older.

Background

National setting

• Sample frame: National Population Register (named individuals)

• Restricted number of reminders

• Restricted budgets

• Mode: self-administration• Information, deliveries and social exchange by post

Unit nonresponse in ISSP Norway 2000-2009Total and selected age/gender groups

0

10

20

30

40

50

60

70

80

2000 2001 2002 2003 2004 2006 2007 2008 2009

Total Male 18-24 Female 18-24 Male 65-79 Female 65-79

Other bias

Nonresponse among younger men, 2011, by place of living

Representing men 18-24 years in Oslo

18-24 yrs 95%

25-34 yrs 76%

Oslo

18-24 yrs 71%

25-34 yrs 64%

Next three largest cities

18-24 yrs 72%

25-34 yrs 72%

Rest of the country

Why introduce websurvey?

• Mixed mode design, freedom of choice, reduce unit nonresponse and increase data quality

• More attractive for certain groups• No “mailbox effect”• Administrative advantages

– Low cost– Collected electronically, no transfer costs – Data file created on the spot– Administrative records continuously updated– Paradata

• Quality– Control of skip patterns– More detailed response to open ended question (Schaefer &

Dillman 1989)– Less item non-response ?

Implementation of experiment 2011

•Gross sample 3 600 persons aged 18-79

•2 modes of self-administration, Web and PaP

•Aims– decrease in unit nonresponse– maximised share of Web-respondents, (reduce bias)

• Contact design– Split ballot (1 800 + 1 800)

Group 1. “This is a Websurvey” Group 2. “This is a mixed mode survey”

Timeline

Results Response rates1 by response mode, and 1st

response mode offered

Item nonresponse 1 – topic module

Item nonresponse 2 – verbatim recorded answers

Occupation of respondent:

Web: 8,2 % missing

Paper: 11,5 % missing

What is next?

• Keep mixed mode with Web and PaP

• Target groups for tailored contact design?

• Possibilities for obtaining e-mail addresses– By phone?

• Reminders by phone?

• Split ballot with a clean post-enquête sample

• Sample design: stratify by gender, age groups and region

• Analyses of mode effects