1 / 18

Nonresponse analyse/veldwerk

monitoring in ESS

Koen Beullens -

Centrum voor Sociologisch Onderzoek, KU Leuven

Den Haag, 24 maart 2015

Harmonisering in cross-nationale surveys

2 / 18

■ Input of output harmonisering?

■ Alle landen even hoge/lage kwaliteit?

■ 5 strategien (Lynn, 2003)

◆ “Maximum quality”

◆ “Consistent quality”

◆ “Constrained maximum quality”

◆ “Target quality”

◆ “Constrained target quality”

European Social Survey

3 / 18

■ Social survey: over het leven van alledag (religie,

arbeidsmarkt, vertrouwen in instituties, . . . )

■ Elke twee jaar sedert 2001/02, nu in ronde 7

■ Meer dan 20 deelnemende landen per ronde

■ 1500-2000 respondenten per land per ronde

■ Alleen face-to-face

Gedecentraliseerd veldwerk

4 / 18

■ Elke nationaal coordinator kiest zelf nationale partner

voor uitvoering veldwerk

◆ Universiteiten

◆ Nationale statistische instituten

◆ Profit bedrijven

■ Internationaal (bv. TNS, IPSOS of GFK)

■ Nationaal

■ Financiering ook nationaal

Nonresponse specificaties in ESS

5 / 18

■ 70% response rate

■ 3% noncontact rate

◆ Minimaal 4 pogingen

◆ Verspreid over twee weken

◆ Minstens een avond en een weekend poging

ESS wellicht voorbeeld van input harmonisering en

“Constrained target quality”

Response rates in de European Social Survey

6 / 18

●

●●

● ● ● ● ● ● ●●●

● ● ● ●●

● ● ●●

●●

●● ● ● ●

●

● ●●

●●

● ● ● ●

●

● ● ●● ●

●●

● ● ● ●●

●

●●

●●

● ● ● ●

●

●

● ● ●

●●

● ● ● ●

● ● ●

●

●●

●●

● ● ● ●

●●

●● ● ●

●●

● ● ● ●

●

●

● ●

●●

●● ● ● ●

●●

● ● ● ●

●●

● ● ● ●

●●

●●

●

●●

●● ● ● ●

● ●●

●● ●

●●

● ● ● ●

●● ●

● ●●

●●

● ● ● ●

● ●●

●●

●● ● ● ●

●

●

●●

● ● ● ●

●● ●

● ●●

●●

● ● ● ● ●●

● ●

●●

●●

● ● ● ●

●●

●●

●●

● ● ● ●

●

●

●

●●

● ● ● ●

●

●

●

●●

● ● ● ●

●●

●● ● ● ●

●

●

●●

● ● ● ●●

●●

●

●●

●●

● ● ● ●● ● ●

● ●●

●●

● ● ● ●

● ●● ● ●

●

●●

● ● ● ●

●● ●

●

●

●

●●

● ● ● ●●

●●

● ● ● ●

● ● ● ●●

●● ● ● ●

●● ●

●

● ●

●●

● ● ● ●

● ●●

●●

●●

●● ● ● ● ●

● ● ● ●

●●

● ● ● ● ●●●

● ● ● ●● ●

● ●●

●●

● ● ● ●

●

●●

● ● ● ●

Albania Aust ria Belgium Bulgaria Switzerland Cyprus Czech Republic

Germany Denmark Estonia Spain Finland France United Kingdom

Greece Croat ia Hungary Ireland Israel Italy Luthania

Latvia Luxemburg the Netherlands Norway Poland Portugal Romania

Russian Federat ion Sweden Slovenia Slovakia Turkey Ukraine Kosovo

0.30.40.50.60.70.8

0.30.40.50.60.70.8

0.30.40.50.60.70.8

0.30.40.50.60.70.8

0.30.40.50.60.70.8

2 4 6 2 4 6 2 4 6 2 4 6 2 4 6 2 4 6 2 4 6ESS Round

Res

pon

se r

ate

Nonresponse bias in Duitsland

7 / 18

respon- nonres- total

TYPE? dents pondents sample

No apartment 1619 54% 2333 40% 3952 45%

apartment 1248 42% 2873 49% 4121 47%

na 126 4% 613 11% 739 8%

2993 5819 8812

Nonrespondentsrespondents

ap

artm

en

tn

aN

oa

pa

rtme

nt

ap

art

me

nt

na

No

ap

art

me

nt

Nonresponse bias in Europa

8 / 18

BE BG CH CY CZ DE EE ES FI GB HU

AGE

GNDR

FLAT

PHONE

GATE/

PHYSA

LITTER

VANDA

Fieldwork monitoring

9 / 18

0

500

AL

0

500

BE

0

500

BG

0

500

CH

0

500

CY

0

500

CZ

0

500

DE

0

500

DK

0

500

EE

0

500

ES

0

500

FI

0

500

FR

0

500

GB

0

500

HU

0

500

IE

0

500

IL

03/08/2012 16/12/2013Fieldwork days

Interviewer monitoring - Belgie

10 / 18

Interviewer monitoring - Nederland

11 / 18

Interviewer monitoring Israel

12 / 18

Assigning areas to interviewers - Belgium

13 / 18

0

50

100

150

0 50 100 150 200

221 Municipalities

155 Inte

rvie

wers

Assigning areas to interviewers - Slovakia

14 / 18

0

50

100

0 50 100 150 200 250

249 Municipalities

132 Inte

rvie

wers

Assigning interviewers to areas - Russia

15 / 18

0

50

100

0 100 200 300

295 Interviewers

13

1 M

un

icip

alit

ies

Inspanningen per case

16 / 18

■�✁✂✄☎

▲✆✝✞✂✟✠✂

❯✡✁✂✠✟✄

❑♦�♦✈♦

❈☛✄☞✞ ✌✄✍✆✎☎✠☞

❈✏✍✁✆�

❇✆☎✑✂✁✠✂

✌✆��✠✂✟ ❘✄❡✄✁✂✝✠♦✟

❆☎✎✂✟✠✂

P♦✁✝✆✑✂☎

◆♦✁✒✂✏

❙☎♦✈✂✡✠✂

P♦☎✂✟❡

■✁✄☎✂✟❡

❊�✝♦✟✠✂

❙☎♦✈✄✟✠✂

■✝✂☎✏

❍✆✟✑✂✁✏

❘✁✂✟☞✄

❇✄☎✑✠✆✓

❉✄✟✓✂✁✡

●✄✁✓✂✟✏

✝✞✄ ◆✄✝✞✄✁☎✂✟❡�

❙✒✠✝☛✄✁☎✂✟❡

❙✍✂✠✟

❯✟✠✝✄❡ ❑✠✟✑❡♦✓

❘✠✟☎✂✟❡

❙✒✄❡✄✟

✵ ✸ ✻ ✾

✔✕✕✖✗✘✙ ✚✛✗ ✜✢✙✛ ✣✢✤✛✗✢✥✛✦

Refusal conversion inspanningen

17 / 18

PT

LT

CY

IL

AL

SK

BG

PL

XK

CZ

RU

IE

EE

ES

FI

UA

HU

DK

SI

BE

NO

FR

NL

GB

CH

SE

IT

DE

0% 20% 40% 60% 70% 80%

Response rate

Discussie

18 / 18

■ Sterke verschillen tussen landen

◆ Deels gerechtvaardigd (bv. surveyklimaat, privacy

wetgeving)

◆ Deels niet gerechtvaardigd (bv. gebrekkige

(interviewer-)capaciteit)

■ Groot gebrek aan kennis over effecten van

landenverschillen op kwaliteit van statistieken

■ Meer gecentraliseerde monitoring/samenwerking?

koen.beullens@soc.kuleuven.be