Improving Reliability of Small-Area ACS Data · Software: EMME/2 Modeled area: ... 2014 ACS Users...
Transcript of Improving Reliability of Small-Area ACS Data · Software: EMME/2 Modeled area: ... 2014 ACS Users...
![Page 1: Improving Reliability of Small-Area ACS Data · Software: EMME/2 Modeled area: ... 2014 ACS Users Conference Presentations by: ... • Increased sample size beginning in 2011 has](https://reader033.fdocuments.us/reader033/viewer/2022060218/5f067dca7e708231d41841e7/html5/thumbnails/1.jpg)
Boston Region Metropolitan Planning Organization
Improving Reliability of
Small-Area ACS Data
January 14, 2015
Paul Reim
Transportation Research Board 94th Annual Meeting
![Page 2: Improving Reliability of Small-Area ACS Data · Software: EMME/2 Modeled area: ... 2014 ACS Users Conference Presentations by: ... • Increased sample size beginning in 2011 has](https://reader033.fdocuments.us/reader033/viewer/2022060218/5f067dca7e708231d41841e7/html5/thumbnails/2.jpg)
Agenda
1. The need for, and problems with, small-area
ACS data
2. A solution: the ‘touch method’
3. Future trends
![Page 3: Improving Reliability of Small-Area ACS Data · Software: EMME/2 Modeled area: ... 2014 ACS Users Conference Presentations by: ... • Increased sample size beginning in 2011 has](https://reader033.fdocuments.us/reader033/viewer/2022060218/5f067dca7e708231d41841e7/html5/thumbnails/3.jpg)
The Need for Small-Area Data
• Equity analyses: Title VI, EJ
• Corridor studies
• Development reviews
• Travel demand model inputs
![Page 4: Improving Reliability of Small-Area ACS Data · Software: EMME/2 Modeled area: ... 2014 ACS Users Conference Presentations by: ... • Increased sample size beginning in 2011 has](https://reader033.fdocuments.us/reader033/viewer/2022060218/5f067dca7e708231d41841e7/html5/thumbnails/4.jpg)
MPO Model in 1990
Software: UTPS
Modeled area:
• 986 TAZs
• 887 Census tracts
• 3,623 Census block groups
![Page 5: Improving Reliability of Small-Area ACS Data · Software: EMME/2 Modeled area: ... 2014 ACS Users Conference Presentations by: ... • Increased sample size beginning in 2011 has](https://reader033.fdocuments.us/reader033/viewer/2022060218/5f067dca7e708231d41841e7/html5/thumbnails/5.jpg)
MPO Model in 2002
Software: EMME/2
Modeled area:
• 2,727 TAZs
• 894 Census tracts
• 3,324 Census block groups
![Page 6: Improving Reliability of Small-Area ACS Data · Software: EMME/2 Modeled area: ... 2014 ACS Users Conference Presentations by: ... • Increased sample size beginning in 2011 has](https://reader033.fdocuments.us/reader033/viewer/2022060218/5f067dca7e708231d41841e7/html5/thumbnails/6.jpg)
MPO Model in 2014
Software: TransCAD
Modeled area:
• 2,727 TAZs
• 976 Census tracts
• 3,341 Census block groups
![Page 7: Improving Reliability of Small-Area ACS Data · Software: EMME/2 Modeled area: ... 2014 ACS Users Conference Presentations by: ... • Increased sample size beginning in 2011 has](https://reader033.fdocuments.us/reader033/viewer/2022060218/5f067dca7e708231d41841e7/html5/thumbnails/7.jpg)
Statistical Testing and the ACS
Two useful publications:
• Multiyear Accuracy of the Data
• Instructions for Applying Statistical Testing
![Page 8: Improving Reliability of Small-Area ACS Data · Software: EMME/2 Modeled area: ... 2014 ACS Users Conference Presentations by: ... • Increased sample size beginning in 2011 has](https://reader033.fdocuments.us/reader033/viewer/2022060218/5f067dca7e708231d41841e7/html5/thumbnails/8.jpg)
Statistical Testing and the ACS
Important statistics:
• Margin of error (MOE): published with data
• Standard error (SE)
• Coefficient of variation (CV)
• Z statistic
![Page 9: Improving Reliability of Small-Area ACS Data · Software: EMME/2 Modeled area: ... 2014 ACS Users Conference Presentations by: ... • Increased sample size beginning in 2011 has](https://reader033.fdocuments.us/reader033/viewer/2022060218/5f067dca7e708231d41841e7/html5/thumbnails/9.jpg)
Example: HH by Income Quartile
• Re-state the ACS estimates as proportions
• Apply the proportions to 2010 Census
counts
• Allocate the resulting estimates to TAZ
using factors derived from 2010 Census
block statistics
![Page 10: Improving Reliability of Small-Area ACS Data · Software: EMME/2 Modeled area: ... 2014 ACS Users Conference Presentations by: ... • Increased sample size beginning in 2011 has](https://reader033.fdocuments.us/reader033/viewer/2022060218/5f067dca7e708231d41841e7/html5/thumbnails/10.jpg)
Example: Using Block Group Data
Percentage of table cells with CV <= 0.3:
• 2000 Census SF3: 85%
• 2006-2010 ACS: 41%
![Page 11: Improving Reliability of Small-Area ACS Data · Software: EMME/2 Modeled area: ... 2014 ACS Users Conference Presentations by: ... • Increased sample size beginning in 2011 has](https://reader033.fdocuments.us/reader033/viewer/2022060218/5f067dca7e708231d41841e7/html5/thumbnails/11.jpg)
Example: Using Tract Data
Percentage of table cells with CV <= 0.3:
• 2000 Census SF3: 92%
• 2006-2010 ACS: 87%
![Page 12: Improving Reliability of Small-Area ACS Data · Software: EMME/2 Modeled area: ... 2014 ACS Users Conference Presentations by: ... • Increased sample size beginning in 2011 has](https://reader033.fdocuments.us/reader033/viewer/2022060218/5f067dca7e708231d41841e7/html5/thumbnails/12.jpg)
Solution: the Touch Method
2014 ACS Users Conference Presentations by:
• Ken Hodges, Nielsen Company
• Ben Horwitz: Greater New Orleans
Community Data Center
![Page 13: Improving Reliability of Small-Area ACS Data · Software: EMME/2 Modeled area: ... 2014 ACS Users Conference Presentations by: ... • Increased sample size beginning in 2011 has](https://reader033.fdocuments.us/reader033/viewer/2022060218/5f067dca7e708231d41841e7/html5/thumbnails/13.jpg)
Solution: the Touch Method
Combine each block group’s estimates with
those of its neighbors.
![Page 14: Improving Reliability of Small-Area ACS Data · Software: EMME/2 Modeled area: ... 2014 ACS Users Conference Presentations by: ... • Increased sample size beginning in 2011 has](https://reader033.fdocuments.us/reader033/viewer/2022060218/5f067dca7e708231d41841e7/html5/thumbnails/14.jpg)
Example: Touch Method
Percentage of table cells with CV <= 0.3:
• 2010 ACS published: 41%
• 2010 ACS touch method: 76%
![Page 15: Improving Reliability of Small-Area ACS Data · Software: EMME/2 Modeled area: ... 2014 ACS Users Conference Presentations by: ... • Increased sample size beginning in 2011 has](https://reader033.fdocuments.us/reader033/viewer/2022060218/5f067dca7e708231d41841e7/html5/thumbnails/15.jpg)
Example: Compare Estimates
• The touch method has produced block
group estimates with better margins of error
• The next step: evaluate whether the
differences between estimates are
statistically significant
• How? The Z statistic
![Page 16: Improving Reliability of Small-Area ACS Data · Software: EMME/2 Modeled area: ... 2014 ACS Users Conference Presentations by: ... • Increased sample size beginning in 2011 has](https://reader033.fdocuments.us/reader033/viewer/2022060218/5f067dca7e708231d41841e7/html5/thumbnails/16.jpg)
Example: Compare Estimates
• Difference is statistically significant if Z is
less than -1.645 or greater than +1.645
• We can use the touch method if the vast
majority of its estimates do not differ
significantly from published estimates
• In this example, the difference is
insignificant for 89% of cells in the table
![Page 17: Improving Reliability of Small-Area ACS Data · Software: EMME/2 Modeled area: ... 2014 ACS Users Conference Presentations by: ... • Increased sample size beginning in 2011 has](https://reader033.fdocuments.us/reader033/viewer/2022060218/5f067dca7e708231d41841e7/html5/thumbnails/17.jpg)
Future Trends
• ACS Sample size was increased beginning
in 2011
• Improvements in margins of error and other
statistics are already showing up
![Page 18: Improving Reliability of Small-Area ACS Data · Software: EMME/2 Modeled area: ... 2014 ACS Users Conference Presentations by: ... • Increased sample size beginning in 2011 has](https://reader033.fdocuments.us/reader033/viewer/2022060218/5f067dca7e708231d41841e7/html5/thumbnails/18.jpg)
Example: 2012 ACS Block Group
Percentage of table cells with CV <= 0.3:
• 2006-2010 ACS published: 41%
• 2008-2012 ACS published: 68%
![Page 19: Improving Reliability of Small-Area ACS Data · Software: EMME/2 Modeled area: ... 2014 ACS Users Conference Presentations by: ... • Increased sample size beginning in 2011 has](https://reader033.fdocuments.us/reader033/viewer/2022060218/5f067dca7e708231d41841e7/html5/thumbnails/19.jpg)
Conclusions
• Small-area 2010 ACS estimates have been
shown to be less reliable than
corresponding 2000 Census estimates
• The touch method may provide more usable
ACS estimates at the block group level
• Increased sample size beginning in 2011 has
had positive results