1 Weighting on National Statistics Household Surveys Jeremy Barton Office for National Statistics.
-
Upload
alyssa-mcdonough -
Category
Documents
-
view
219 -
download
0
Transcript of 1 Weighting on National Statistics Household Surveys Jeremy Barton Office for National Statistics.
2
Outline of talk
• The ONS surveys
• Why should we weight?
• The weighting process
• When should we use weights?
3
ONS social surveys
• Labour Force Survey (LFS)
• General Household Survey (GHS)
• Expenditure & Food Survey (EFS)
• Family Resources Survey (FRS)
• Omnibus Survey (OMN)
4
Labour Force Survey
• Quarterly panel survey (c. 56K hh per qtr)
• HHs stay in survey for 5 qtrs
• require estimates of:
– totals (e.g. employment)
– rates (e.g. unemployment)
• interview: all hh members
• Local boosts for annual estimates
5
General Household Survey
• 9,000 hhs per annum
• housing, consumer durables, employment, health, family structure, pensions, education
• also ad hoc trailers, e.g. drinking
• interview: all hh members
6
Expenditure & Food Survey
• Merger of Food and Expenditure surveys
• 7,000 HHs in UK
• 14 day expenditure diary
• Expenditure and income
• Food consumption and nutrient intake
• Interview: all hh members
7
Family Resources Survey
• For DWP
• 25,000 HHs per year
• Income, benefits, pensions, savings
• Fieldwork shared by ONS and NatCen
• HHs, individuals, benefit units
• Interview: all hh members
8
Omnibus Survey
• Interview: 1 adult per hh
• 1,800 adults per month
• Core questionnaire and modules
• Covers a great range of different topics
9
Why should we weight?
• Adjust for unequal selection probabilities
• Adjust for nonresponse
• Adjust our sample to match known population totals
10
Probability weights
• Weight 1/(prob of selection)
• Boost samples
– EFS in NI, weight = GB weight /4
– more common in ad hocs
11
Probability weights
• Subsampling of units
• Omnibus (1 adult per hh)
– weight = # adults in hh
• FRS (Multi-household addresses)
13
Nonresponse weights
• Sample-based nonresponse methods
• Split set sample into weighting classes
• Estimate weighted response rates in each class
• New weight is 1/RR
14
Nonresponse weights
• Response rates different in each weighting class
• Means for major survey variables must differ between each class
• Means for major survey variables must be same for R and NR within each class
15
Nonresponse weights
• GHS and EFS
• Based on Census-link studies 1991
• Target nonresponse in specific demographic groups
• Sampling frame information
• Interviewer observations
16
EFS NR weighting classesGroup Characteristics Response
rateWeight
1 or 2 adults No children
1 London or Met. area 67% 1.49 Non-met. area
2 Scotland /North 78% 1.283 South East 64% 1.564 Other non-met 73% 1.375 With children 77% 1.30
3 or more adults No children
6 London 42% 2.387 Not London 58% 1.728 With children 68% 1.47
17
Population weights
Sample % Population %MalesUnder 16 14456 11% 5,987,000 10%16-64 39107 30% 18,538,000 32%65+ 9332 7% 3,902,000 7%
FemalesUnder 16 13777 11% 5,705,000 10%16-59 38733 30% 17,589,000 30%65+ 15012 12% 6,617,000 11%
All 130417 100% 58,337,000 100%
LFS Mar-May 2003
18
Population weights
• Produce population totals of estimates
• Reduce nonresponse bias further
• Improve precision (reduce SEs)
• Comparability across surveys
• a.k.a. calibration, post-stratification
19
LFS Population weights
• LFS - Individual level weights
– raking to 3 controls:
• 5 yr age group by sex within region
• Local Authority
• Single years 16-24 by sex
– population projections
20
LFS Population weights
• LFS HH level weights
– Same weight each hh member (Lemaitre/Dufour)
– software: Calmar
– bounded weights
– Age group 5 yrs and single years 16-24 by sex and region
21
GHS/EFS Pop. weights
• HH-level weights
• Pre-weighted by NR/prob weights
• Calibrate to 5-year age groups by sex and to region
• Pop estimates excl. communal establishments
22
FRS Population weights
• Calibration to:
– Age group, sex, marital status
– Lone Parents
– Families
– Tenure Type
– Council Tax band
– Region
23
The weighting process
Unweighted Probabilityweight (NI)
NR weight Calibrationweight
COUNTRY % % % %England 79.8 84.3 84.9 83.8Scotland 4.8 5.0 4.9 5.0Wales 8.3 8.8 8.3 8.7NorthernIreland
7.1 1.9 1.9 2.5
Total 100 100 100 100
EFS 2001/02
24
When to use weights
• Always (whenever you can)
• Problems with presentation /interpretation
– estimates / sample sizes / SEs
• NR & probability weights tend to increase variances
25
When to use weights
• Stat packages (e.g. SPSS) don’t always deal with weighting correctly
– Scale weights to average 1
• Stata/SAS survey estimation procedures
• Calibration tends to reduce variances
26
Conclusions
• Weights combination of probability, NR, calibration
• Required for unbiased estimation
• May require specialist software for correct hypothesis testing
27
Current Issues
• Use of 2001 Census data– Census-linked NR studies– Change in Pop. Controls (back-weighting)
• Integrated survey (CPS)
• LFS: – Attrition weighting– Local LFS– Number of controls
28
References
• Weighting for non-response, Elliot, D. NM17• Grossing Up - when and how, Butcher, B. SMB 14• The presentation of weighted data in survey report tables, Elliot, D.
SMB 38• Using weights in regression analysis: A comparison between SPSS
and STATA packages, Insalaco, F. SMB 45• Developing a weighting and grossing system for the GHS, Barton, J.
SMB 49
• Evaluation nonresponse on household surveys, Foster, K. GSS
Methodology Series 8. • Report of the Task Force on Weighting and Estimation, Elliot, D. GSS
Methodology Series 16.