Nonresponse Bias Correction in Telephone Surveys Using Census Geocoding: An Evaluation of Error...
-
Upload
alisa-linscomb -
Category
Documents
-
view
219 -
download
2
Transcript of Nonresponse Bias Correction in Telephone Surveys Using Census Geocoding: An Evaluation of Error...
![Page 1: Nonresponse Bias Correction in Telephone Surveys Using Census Geocoding: An Evaluation of Error Properties Paul Biemer RTI International and University.](https://reader036.fdocuments.us/reader036/viewer/2022062417/551c0dce550346a34f8b524c/html5/thumbnails/1.jpg)
Nonresponse Bias Correction in Telephone Surveys Using Census Geocoding:
An Evaluation of Error PropertiesPaul Biemer
RTI International and
University of North Carolina
Andy PeytchevRTI International
![Page 2: Nonresponse Bias Correction in Telephone Surveys Using Census Geocoding: An Evaluation of Error Properties Paul Biemer RTI International and University.](https://reader036.fdocuments.us/reader036/viewer/2022062417/551c0dce550346a34f8b524c/html5/thumbnails/2.jpg)
2
Estimating the Population Mean in an RDD Survey
Respondents {R} Nonrespondents
TOTAL SAMPLE
Nonrespondents {NR}
{ } { } { } { }R R NR NRy p y p y
mean of NRs is unknown
![Page 3: Nonresponse Bias Correction in Telephone Surveys Using Census Geocoding: An Evaluation of Error Properties Paul Biemer RTI International and University.](https://reader036.fdocuments.us/reader036/viewer/2022062417/551c0dce550346a34f8b524c/html5/thumbnails/3.jpg)
3
Methods for Adjusting and Evaluating for RDD Nonresponse
Very limited information on NRs in RDD surveys Post-stratification adjustments are the norm
Effectiveness at reducing bias is questionable at best Bias is sometimes evaluated using
Maximum followup effort approaches Only evaluates reduction in bias due to slight elevations in response rates
Comparison to external gold standard estimates Limited in scope to a few characteristics
Census block group geocoding Error properties are largely unknown
The focus of this research
![Page 4: Nonresponse Bias Correction in Telephone Surveys Using Census Geocoding: An Evaluation of Error Properties Paul Biemer RTI International and University.](https://reader036.fdocuments.us/reader036/viewer/2022062417/551c0dce550346a34f8b524c/html5/thumbnails/4.jpg)
4
Census Geocoding (CG) Method
Obtain the addresses for nonrespondents (50-60% “success” rate)
Geocode addresses Link to census aggregate data
Address matched: link unit to census block group (CBG) via geocode
Exchange matched: link to census tract (CT) via telephone number
Substitute the corresponding CBG or CT mean for the nonrespondent’s characteristic
![Page 5: Nonresponse Bias Correction in Telephone Surveys Using Census Geocoding: An Evaluation of Error Properties Paul Biemer RTI International and University.](https://reader036.fdocuments.us/reader036/viewer/2022062417/551c0dce550346a34f8b524c/html5/thumbnails/5.jpg)
5
Estimating the Population Mean in an RDD Survey
Respondents {R} Nonrespondents
TOTAL SAMPLE
Nonrespondents {NR}
{ } { } { } { }R R NR NRy p y p y
mean of NRs is unknown
![Page 6: Nonresponse Bias Correction in Telephone Surveys Using Census Geocoding: An Evaluation of Error Properties Paul Biemer RTI International and University.](https://reader036.fdocuments.us/reader036/viewer/2022062417/551c0dce550346a34f8b524c/html5/thumbnails/6.jpg)
6
Impute Nonrespondent Characteristics from Census Aggregate Data
Respondents {R} Nonrespondents
TOTAL SAMPLE
Nonrespondents {NR}
{ } { } { } { }NR RR NRp y py y
Obtained from NRs CBG or CT
![Page 7: Nonresponse Bias Correction in Telephone Surveys Using Census Geocoding: An Evaluation of Error Properties Paul Biemer RTI International and University.](https://reader036.fdocuments.us/reader036/viewer/2022062417/551c0dce550346a34f8b524c/html5/thumbnails/7.jpg)
7
Questions Addressed by this Research
What is the bias in ?
Is a valid estimate of the bias in the
unadjusted (or post-stratified) estimator of the mean?
Does the CG method provide useful data for modeling
response propensity?
The first two questions will be addressed in today’s
presentation.
y
{ }ˆ
RB y y
y
{ }ˆ
RB y y
![Page 8: Nonresponse Bias Correction in Telephone Surveys Using Census Geocoding: An Evaluation of Error Properties Paul Biemer RTI International and University.](https://reader036.fdocuments.us/reader036/viewer/2022062417/551c0dce550346a34f8b524c/html5/thumbnails/8.jpg)
8
Decomposition of the Bias in
Respondents Nonrespondents
Correctly matched
addresses
Incorrectly matched
addresses
Correctly matched
exchanges
Incorrectly matched
exchanges
TOTAL SAMPLE
{ }CA { }IA { }CE { }IE
{ }CA { }IA { }CE { }IE
y
Size
Expected Difference
![Page 9: Nonresponse Bias Correction in Telephone Surveys Using Census Geocoding: An Evaluation of Error Properties Paul Biemer RTI International and University.](https://reader036.fdocuments.us/reader036/viewer/2022062417/551c0dce550346a34f8b524c/html5/thumbnails/9.jpg)
9
{ } { } { }
{ } { } { }
{ } { } { }
{ } { } { }
CA CA CA
IA IA IA
CE CE CE
IE IE IE
y y
y y
y y
y y
{ } { }
{ } { }
{ } { }
{ } { }
( )
( )
( )
( )
CA CA
IA IA
CE CE
IE IE
E p
E p
E p
E p
![Page 10: Nonresponse Bias Correction in Telephone Surveys Using Census Geocoding: An Evaluation of Error Properties Paul Biemer RTI International and University.](https://reader036.fdocuments.us/reader036/viewer/2022062417/551c0dce550346a34f8b524c/html5/thumbnails/10.jpg)
10
Components of the Bias in
{ } { } { } { } { } { }
{ } { } { } { }
NR NR CA CA IA IA
CE CE IE IE
E y y
{ } { } { }
Bias( ) ( )
NR NR NR
y E y y
E y y
y
where
![Page 11: Nonresponse Bias Correction in Telephone Surveys Using Census Geocoding: An Evaluation of Error Properties Paul Biemer RTI International and University.](https://reader036.fdocuments.us/reader036/viewer/2022062417/551c0dce550346a34f8b524c/html5/thumbnails/11.jpg)
11
Components of the Bias in
{ } { } { } { } { } { }
{ } { } { } { }
NR NR CA CA IA IA
CE CE IE IE
E y y
{ } { } { }
Bias( ) ( )
NR NR NR
y E y y
E y y
y
wherecorrect CBG match incorrect CBG match
correct CT match incorrect CT match
![Page 12: Nonresponse Bias Correction in Telephone Surveys Using Census Geocoding: An Evaluation of Error Properties Paul Biemer RTI International and University.](https://reader036.fdocuments.us/reader036/viewer/2022062417/551c0dce550346a34f8b524c/html5/thumbnails/12.jpg)
12
Estimation of the Bias Components
National Comorbidity Survey Replication (NCS-R) National probability sample of 18+ in households Face to face survey with 71% response rate All addresses were geocoded
CG was applied to 8,178 responding hh’s that provided a telephone number (88% of NCS sample)
CG bias components estimated based on 41% response rate (response after 3 callbacks)
Sensitivity analysis based on three response rates: 2 callbacks 26% response rate 3 callbacks 40% response rate 5 callbacks 60% response rate
![Page 13: Nonresponse Bias Correction in Telephone Surveys Using Census Geocoding: An Evaluation of Error Properties Paul Biemer RTI International and University.](https://reader036.fdocuments.us/reader036/viewer/2022062417/551c0dce550346a34f8b524c/html5/thumbnails/13.jpg)
13
Why is it reasonable to use a face to face survey to evaluate the CG bias in an RDD survey?
The nonresponse mechanism is not a critical factor in the assessment of the CG bias.
A survey with a relatively high response rate is needed to evaluate the bias.
Addresses are known for all sample members and can therefore be geocoded to their correct CGs.
Sensitivity analysis can be performed to assess the effect on CG bias of increasing response rates.
![Page 14: Nonresponse Bias Correction in Telephone Surveys Using Census Geocoding: An Evaluation of Error Properties Paul Biemer RTI International and University.](https://reader036.fdocuments.us/reader036/viewer/2022062417/551c0dce550346a34f8b524c/html5/thumbnails/14.jpg)
14
Weighted Respondent Mean, True Mean, and CG Imputed Mean for Available Characteristics
Whit
eBlac
kAsia
n
Oth
erHisp
Male
Female
1 pe
rs H
H
2+ p
ers
HH
< 18
HH
Oth
er H
H
Age 1
8–24
Age 2
5–34
Age 3
5–49
Age 5
0–59
Age 6
0+
< $15
,000
$15K
- $3
0K
$30K
–$50
K
$50K
–$75
K
≥ $
75K
0
10
20
30
40
50
60
70
80
Respondent Mean True Mean Imputed Mean
![Page 15: Nonresponse Bias Correction in Telephone Surveys Using Census Geocoding: An Evaluation of Error Properties Paul Biemer RTI International and University.](https://reader036.fdocuments.us/reader036/viewer/2022062417/551c0dce550346a34f8b524c/html5/thumbnails/15.jpg)
15
Weighted Respondent Mean, True Mean, and CG Imputed Mean for Available Characteristics
Whit
eBlac
kAsia
n
Oth
erHisp
Male
Female
1 pe
rs H
H
2+ p
ers
HH
< 18
HH
Oth
er H
H
Age 1
8–24
Age 2
5–34
Age 3
5–49
Age 5
0–59
Age 6
0+
< $15
,000
$15K
- $3
0K
$30K
–$50
K
$50K
–$75
K
≥ $
75K
0
10
20
30
40
50
60
70
80
Respondent Mean True Mean Imputed Mean
![Page 16: Nonresponse Bias Correction in Telephone Surveys Using Census Geocoding: An Evaluation of Error Properties Paul Biemer RTI International and University.](https://reader036.fdocuments.us/reader036/viewer/2022062417/551c0dce550346a34f8b524c/html5/thumbnails/16.jpg)
16
Whit
eBlac
kAsia
nOth
erHisp
Male
Female
1 HH
1+ H
H
1+ c
hild
No ch
ild18
-24
25-3
435
-49
50-5
960
+<15
k
15K-3
0K
30k-
50k
50k-
75k
75k+
-50.00%
-40.00%
-30.00%
-20.00%
-10.00%
0.00%
10.00%
20.00%
26% RR
41% RR
60% RR
Demographic Characteristics
by Response RateRelbias( | ) Relbias( | )R Ry y y y
![Page 17: Nonresponse Bias Correction in Telephone Surveys Using Census Geocoding: An Evaluation of Error Properties Paul Biemer RTI International and University.](https://reader036.fdocuments.us/reader036/viewer/2022062417/551c0dce550346a34f8b524c/html5/thumbnails/17.jpg)
RTI International
17
Average Estimates of for {s} = {CA}, {IA}, {CE}, and {IE}
{ } { }s s
{ } { }s s
Bias Component
{CA} {IA} {CE} {IE}0.00%
0.10%
0.20%
0.30%
0.40%
0.50%
0.60%
0.70%
0.80%
0.90%
1.00%
26% RR
41% RR
60% RR
(Percentage points)
![Page 18: Nonresponse Bias Correction in Telephone Surveys Using Census Geocoding: An Evaluation of Error Properties Paul Biemer RTI International and University.](https://reader036.fdocuments.us/reader036/viewer/2022062417/551c0dce550346a34f8b524c/html5/thumbnails/18.jpg)
RTI International
18
Average Relative Size of the Bias Components
{ }s { }| |s26% RR 41% RR 60% RR 26% RR 41% RR 60% RR
0.00%
5.00%
10.00%
15.00%
20.00%
25.00%
30.00%
35.00%
40.00%
45.00%
50.00%
{CA} {IA} {CE}
{IE}
![Page 19: Nonresponse Bias Correction in Telephone Surveys Using Census Geocoding: An Evaluation of Error Properties Paul Biemer RTI International and University.](https://reader036.fdocuments.us/reader036/viewer/2022062417/551c0dce550346a34f8b524c/html5/thumbnails/19.jpg)
19
Conclusions
Bias in the CG estimates of NR bias is unacceptably large race, age, and income were the most biased
Major source of bias {IE} followed by {CA} (surprisingly) Approximately 75% of the cases fall into these subsets
Correctly matching to CBGs reduces the bias, but minimally Biases tend to build across components rather than netting
out. Increasing the survey response rate reduces bias in the CG
approach; relative importance of each component is stable
![Page 20: Nonresponse Bias Correction in Telephone Surveys Using Census Geocoding: An Evaluation of Error Properties Paul Biemer RTI International and University.](https://reader036.fdocuments.us/reader036/viewer/2022062417/551c0dce550346a34f8b524c/html5/thumbnails/20.jpg)
20
Next Steps
Further characterize the CG bias by its components Consider the use of CBG and CT information obtain from
the CG method for: modeling of response propensities adjusting for nonresponse bias