Measuring Accuracy of Street Centerline Datasets

Donald Cooke

Founder, GDT

CLEM2001

August 6-7, 2001

Accuracy of Centerline Files

• History: NMAS

• History: GDT involvement

• NSSDA, July 1998

• GDT procedure

• Results

• 3 meter accuracy

History, NMAS

• NMAS: National Map Accuracy Standard

• Promulgated in 1940’s

• Production Standard: “Build to this Spec.”

• Binary (pass-fail) standard

• Tied to scale/paper/”Gutenberg Disease”

• Did they ever test the quads? How?

History, GDT Involvement

• Geocoding accuracy

Tampa, Florida, April 1997Red: DGPS positions of addressesBlack: Interpolated geocoded addressesMost of error budget comes from storing potential address ranges…….

History, GDT Involvement

• Geocoding accuracy

• GDT inherited spatial accuracy of TIGER

• Need to realign streets and improve accuracy

• Need to qualify sources for realignment

• Need to Q/C results of realignment operation

NSSDA

• NSSDA: “National Standard for Spatial Data Accuracy”

• July 1998; draft available well in advance

NSSDA Procedure

• “accuracy testing by an independent source of higher accuracy is the preferred test”

• Compile a test suite of coordinate measurements for well-defined points in the study area

• Extract corresponding points from the dataset being evaluated

• Statistically compare the two samples by computing RMSE and "accuracy" measures

• ("accuracy" = RMSE * 1.7308)

GDT Procedure

• Well-defined points: “T” or “Cross” intersections

• Park in center of intersection; average for 1 minute

• Post-processed code-phase differential

• ~30, and later ~45 points collected per sample

• Sample skewed to exclude major road intersections

• Sample sometimes adjusted to fall within DOQQ or 7.5 minute quad tile.

Sample point spacing; Deerfield Illinois

Results

• 56 areas surveyed; more in process

• TIGER: 16-100+ meter RMSE

• Dynamap aligned to DOQQ: 4-6 meter RMSE

• See some results...

Test Area Number TIGER Dynamapof Points RMSE RMSE

(meters) (meters)Sarasota, FL 39 26.5 4.3Ann Arbor, MI 29 46.7 4.5Deerfield, IL 29 27.0 4.2Manchester, NH 36 25.5 4.6Morristown, NJ 48 48.2 6.1Greenfield, MA 70 21.3 6.0Colma/Pacifica, CA 51 57.1 6.4W Palm Beach, FL 38 44.2 3.0* Utility GISDenver, CO 37 16.7 3.4 DatasetWarwick, RI 35 37.6 5.2San Diego, CA 31 20.9 4.1

0

50

100

150

200

1 3 5 7 9 11 13 15 17 19 21 23 25 27 29

Check point #

Error, Meters

Absolute error for 29 check points, Ann Arbor, MI

0

50

100

150

200

1 3 5 7 9 11 13 15 17 19 21 23 25 27 29

Check point #

Error, Meters

Absolute error for 29 check points, Ann Arbor, MI

RMSE: 46 meters RMSE of “good” pts: 20 meters; bad: 102 meters

Results

• TIGER spatial errors are not normally distributed

• Large population from 1:100K DLGs

• Many from 1980 GBF/DIME files

• Some streets “cartooned” freehand

• Picking sample is crucial

• Cannot make a blanket statistical statement that describes TIGER spatial error

Results, continued

• 1:24,000 DLGs are spotty; South Carolina exp.

• Some (25%?) 7.5 minute quads don’t pass NMAS

• Impossible to test accuracy until recently

• 1 meter DOQs appear to be boringly reliable

Conclusions on NMAS vs NSSDA

• NSSDA is a workmanlike improvement

• No longer a binary test

• No longer tied to scale of analogue map

• Procedure extensible to testing GPS accuracy

• But still considers “map” to be a monolithic, one-time compilation to a single standard

• Need to carry accuracy metadata on each object

Census TIGER/2010 Accuracy

• 3-meter or better accuracy (=1.7 meter RMSE)

• Cannot achieve this from off-the-shelf DOQs

• Must use new imagery or drive DGPS

• Code-phase differential GPS is marginal for test suite

• We use carrier-phase for test points