LS attrition: best practice and lessons learned

Louisa Blackwell, ONS

Introduction

• Structure of the ONS Longitudinal Study• Samples available for analysis• LS losses to follow-up; change over time• Impact of loss to follow-up on analysis• Explaining LS loss to follow-up • Minimising attrition through design• Minimising attrition through best practice in data

processing• Treatment of attrition in LS analysis

LS Structure

1971

Original sample: 530,000

members; selected from 1971 Census

1971

Original sample: 530,000

members; selected from 1971 Census

1981

534,000 sample

members found at 1981

Census

1981

534,000 sample

members found at 1981

Census

1991

543,000 sample

members found at 1991

Census

1991

543,000 sample

members found at 1991

Census

2001

540,000sample

members found at 2001

Census

2001

540,000sample

members found at 2001

Census

Entries 1971-2001New Births 228,000Immigrations 122,000

Entries 1971-2001New Births 228,000Immigrations 122,000

Exits 1971-2001Deaths 201,000Embarkations 32,000

Exits 1971-2001Deaths 201,000Embarkations 32,000

Events: April 1971 to April 2002Births to sample women 215,000Births to sample men 49,500Infant Deaths 2,500Widow(er)hoods 70,000Cancer registrations 78,000

173,000 Deaths 11,000 Emigrations

Total traced LS members: 944,000

536,000528,000 535,000513,000

1971 20011981 1991

1 census

408,000

420,000

418,000

2 censuses

327,000

331,000

3 censuses

256,000 4 censuses

Samples available for analysis

10-year and 30-year losses to follow-up

Losses over 30 years

History Total traced LS members

Deaths and embarkations 1971-2001

Matched at 2001 Census

Intercensal loss (per cent)

Present 1971 Census

513,000 184,000 256,000 14

Losses over 10 years

History Total traced LS members

Deaths and embarkations 1991-2001

Matched at 2001 Census

Intercensal loss (per cent)

Present 1991 Census

537,000 60,000 419,000 11

1990s births 71,000 1,000 60,000 14

1990s immigrants

46,000 2,000 15,000 63

Loss to follow-up in each decade, 1971-81, 1981-91 and 1991-2001

Census sample

0

10

20

30

40

50

60

70

1971-1981 1981-1991 1991-2001

Census

per

cen

t

Loss to follow-up in each decade, 1971-81, 1981-91 and 1991-2001

Census sample Intercensal births

0

10

20

30

40

50

60

70

1971-1981 1981-1991 1991-2001

Census

per

cen

t

Loss to follow-up in each decade, 1971-81, 1981-91 and 1991-2001

Census sample Intercensal births Intecensal immigrants

0

10

20

30

40

50

60

70

1971-1981 1981-1991 1991-2001

Census

per

cen

t

Impact of loss to follow-up on analysis

• Introduces significant error in all survival analysis • Biases all analyses of sub-groups; those lost to follow-up

are disproportionately:

-young and male

-immigrants

-living in London, particularly Inner London

-in a minority ethnic group

- not married

- born to a young mother in the 1990s

- sole registered at birthSee Blackwell, L., Lynch, K., Smith, J. and Goldblatt, P. (2003) Longitudinal Study

1971-2001: Completeness of Census Linkage, Series LS No. 10, London: ONS

Explaining LS loss to follow-up

Census underenumeration• LS members resident in England and Wales but

not enumerated by Census

Unobserved embarkation• LS members no longer resident in England and

Wales at Census but their emigration was not previously recorded in the LS

LS non-linkage• LS linkage failure, through mis-recording of date

of birth or poor quality person identifiers

Census underenumeration

Census Estimated under-enumeration (per cent)

1971 0.40-1.00

1981 0.45

1991 1.90

2001 6.00

Unobserved embarkation

0.00

0.20

0.40

0.60

0.80

1.00

1.20

1.40

1971

1973

1975

1977

1979

1981

1983

1985

1987

1989

1991

1993

1995

1997

1999

Year

Rat

io -

LS

/IPS

Males Females

Ratio of LS emigrants to International Passenger Survey outflows from England and Wales, 1971-2000

LS attrition through non-linkage: LS tracing at NHSCR

Tracing rates by census year and sex

Census Males(%)

Females(%)

Persons(%)

1971 97.2 96.5 96.81981 98.8 98.9 98.81991 98.3 98.5 98.42001 99.1 99.5 99.3

1. Source: Office for National Statistics Longitudinal Study (LS)

Minimising attrition through design

• No respondent burden as a result of the LS• Comprehensive recording of deaths (deaths

abroad are under-recorded)• Use of NHSCR for tracing allows cross-validation

and avoids double counting • Entry to the LS is only permitted through birth,

immigration and Census- example of quality management is provided by CCC retry routines

• Intelligent matching

Minimising attrition through processing

• Electronic data capture for 2001• Use of Census images to aid tracing in 2001• Combining automatic with operator matching:

- 2001 Link had 70 per cent automatch rate

- Claimant count linkage had 88-90 per cent automatch rate

• Query resolution between processing and tracing teams

• Dedicated teams drawing on 35 years’ of experience in matching LS data

Treatment of attrition in analysis

• Restrict analysis to LS members traced at NHSCR with matched Census records

• Investigate characteristics of LS members lost to follow-up to understand bias in analysis

• Validate cross-sectional distributions with 100 per cent census data

• For survival analysis:

- Make assumptions about timing and characteristics of those lost, based on available evidence