Estimating the Effectiveness of Speed Cameras

Mike MaherInstitute for Transport Studies

University of Leeds

Hong Kong Poly U, 22 Oct 2013

Background to the talk

• Speed cameras widely-used in UK• Do they save lives? Or simply make money?• Unpopular with many motorists• Long-running controversy• DfT under pressure to establish their effect

A brief history of speed cameras (1)

• First introduced in UK in early 1990s• Policy decision in December 1998

– “hypothecation”, local camera partnerships set up• In 2002, cameras made more conspicuous• In 2004, 3-year evaluation report criticised

– no allowance for regression to mean• In 2005, 4 year evaluation report

– Appendix H allows for RTM on subset of data– shows camera effectiveness reduced

A brief history of speed cameras (2)

• In 2011, Minister’s letter to English local authorities– requiring them to publish camera data– FSCs, PICS for all years 1990 – 2010

• 2012, Scottish safety camera bulletin criticised– report advising how to analyse and present data

• June 2013 RAC Foundation report: R Allsop– guidance on use of transparency data– proposing a method of analysis of such data– allows for trend, RTM and estimates camera effect

• Still criticised by anti-camera lobby

Outline of the talk

• Remedial safety treatments (eg cameras)– applied to “problem” sites– to reduce accidents at the site

• Identification of “problem” sites– those with high number of accidents in last 3 years

• Evaluation of effect of remedial treatment– compare after accidents with before accidents– allow for trend (compare with regional numbers)

• But - not as straightforward as it may seem!

North Lanarkshire data

Number of sites Nk with k accidents in 3-year period

k 0 1 2 3 4 5 6 7 8 9 11 13

Nk 7411 1645 341 117 38 26 13 7 2 1 1 1

Sites with at least 4 accidents called “cluster sites”

Earmarked for remedial treatment

North Lanarkshire data

Number of sites Nk with k accidents in 3-year period

k 0 1 2 3 4 5 6 7 8 9 11 13

Nk 7411 1645 341 117 38 26 13 7 2 1 1 1

before after change

Whole network 3136 2799 -11%

Cluster sites 458 233 -49%

North Lanarkshire data

Number of sites Nk with k accidents in 3-year period

k 0 1 2 3 4 5 6 7 8 9 11 13

Nk 7411 1645 341 117 38 26 13 7 2 1 1 1

before after change

Whole network 3136 2799 -11%

Cluster sites 458 233 -49%

BUT – NO TREATMENT APPLIED!!

Regression to the mean

• Bias by selection• Sites chosen on basis of high Y = m + • Top sites tend to have both:

– high systematic component (mean m)– high positive random component (error )– systematic component persists …– … but random component does not

• Exaggerated estimate of treatment effectiveness, unless corrected for

Why “regression to the mean”?

• Sir Francis Galton, (1822 – 1911), eugenicist, biometrician, statistician, observed:

Tall fathers tend to have sons who are also tall – but who are not as tall

as themselves

Galton height data

58

60

62

64

66

68

70

72

74

76

78

58 60 62 64 66 68 70 72 74 76 78

father's height

so

n's

he

igh

t

Galton height data

58

60

62

64

66

68

70

72

74

76

78

58 60 62 64 66 68 70 72 74 76 78

father's height

so

n's

he

igh

t

tall fathers

RTM appears in other places, too

• Golf tournaments:– the players who score well in the first round

tend, on average, to score well in the second round too - but not as well as they did in the first

2013 British Open Golf Tournament

Pos Name Rounds 1-2 Rounds 3-41 Miguel Angel Jimenez 1392 Henrik Stenson 1403 Lee Westwood 1404 Tiger Woods 1405 Dustin Johnson 1406 Zach Johnson 1417 Angel Cabrera 1418 Rafael Cabrera-Bello 1419 Martin Laird 141

10 Ryan Moore 142

2013 British Open Golf Tournament

Pos Name Rounds 1-2 Rounds 3-41 Miguel Angel Jimenez 139 1502 Henrik Stenson 140 1443 Lee Westwood 140 1454 Tiger Woods 140 1465 Dustin Johnson 140 1536 Zach Johnson 141 1457 Angel Cabrera 141 1478 Rafael Cabrera-Bello 141 1509 Martin Laird 141 153

10 Ryan Moore 142 151

2013 British Open Golf Tournament

Pos Name Rounds 1-2 Rounds 3-41 Miguel Angel Jimenez 139 1502 Henrik Stenson 140 1443 Lee Westwood 140 1454 Tiger Woods 140 1465 Dustin Johnson 140 1536 Zach Johnson 141 1457 Angel Cabrera 141 1478 Rafael Cabrera-Bello 141 1509 Martin Laird 141 153

10 Ryan Moore 142 151Average increase = 7.9

2012 British Open Golf Tournament

Pos Name Rounds 1-2 Rounds 3-41 Brandt Snedeker 130 1472 Adam Scott 131 1433 Tiger Woods 134 1434 Thorbjorn Olesen 135 1455 Graeme McDowell 136 1426 Thomas Aiken 136 1437 Matt Kuchar 136 1448 Jason Duffner 136 1479 Paul Lawrie 136 148

10 Ernie Els 137 136Average increase = 9.1

2011 British Open Golf Tournament

Pos Name Rounds 1-2 Rounds 3-41 Darren Clarke 136 1392 Lucas Glover 136 1473 Thomas Bjorn 137 1424 Chad Campbell 137 1435 Martin Kaymer 137 1466 Miguel A Jimenez 137 1507 Dustin Johnson 138 1408 Davis Love III 138 1449 George Coetzee 138 146

10 Charl Schwartzel 138 147Average increase = 7.2

2010 British Open Golf Tournament

Pos Name Rounds 1-2 Rounds 3-41 Louis Oosthuizen 132 1402 Mark Calcavecchia 137 1573 Lee Westwood 138 1414 Paul Casey 138 1425 Jin Jeong 138 1466 Alejandro Canizares 138 1487 Retief Goosen 139 1428 Sean O’Hair 139 1439 Tom Lehman 139 145

10 Graeme McDowell 139 146Average increase = 7.3

The problem

• Treatment applied at sites with high number of accidents – eg k 4 accidents in before period– speed cameras: 8 PICs/km in three years

• Accidents reduce even if nothing done– bias produced by selection criterion (RTM)– so need to allow for (or avoid) that in the analysis – and also allow for other effects: eg trend

Problem and possible approaches

• observed before frequency is not a reliable measure of true frequency

• Empirical Bayes Method (EBM)– use predictive accident model to estimate is a function of site variables: flow, length ..– combine observed accidents y with to give m

• Use time series data for each camera site– as for “transparency” data– before period, selection period, after installation

Empirical Bayes Method

• What is the expected value of true mean m– given the observed value of no. accidents y?

• Bayes’ Theorem– prior distribution for m from predictive accident model– combine with observed y– to give posterior estimate of m

• Depends on the model and its precision

ym )1(ˆ 1

1:where

K

RTM in camera partnerships data

• Asked by DfT to work with UCL and PA on four year report– previous Napier / Liverpool EPSRC research– carry out our analysis on subset of data– allow for trend and RTM– see how much apparent effect of cameras is

due to RTM, and how much is real

Overall reduction = trend + RTM + camera effect

So ….

• Subset of 216 sites for which data available– urban sites (30 and 40 mph limits)– traffic flows and number of junctions/km

• Use this data in an existing predictive accident model to calculate number of accidents to be expected at such a site

• Best estimate of true mean number of accidents in before period at the site is then:

y)(m̂ 11

1

K

where:

Results – for FSCs

FSCs/site/year:before 1.05

after 0.48

Overall reduction = 0.57 = 0.10 + 0.36 + 0.11

trend RTM camera

54% = 10% + 34% + 10%

(-54%)

relative to what would have been: 50% allowing for trend

19% allowing for trend + RTM

RAC Foundation method

• Report written by Richard Allsop in June 2013• Not all partnerships have yet published data

– in varied formats originally– data from ten partnerships analysed in report– available on RACF website as .csv files– now in standard format: one row per camera per year– annual data for 21 years: 1990 – 2010– main interest on PICS and FSCs, but also casualties– trend given by partnership annual totals

Data periods

• For each camera, years divided into periods– before– site selection period (SSP): 3 years (not specified?)– transition (year camera installed): specified– post-installation (camera period)

• If camera installed in mid-2000– SSP assumed to be 97-99– before period is then 90-96– camera period is 01-10

Form of model

• accidents yit Poisson distributed mean mit

• mit proportional to partnership total Pt

• rate factored in SSP by (RTM effect)• rate factored in camera period by • dummy (0/1) variables to indicate period

– before, SSP or camera• Poisson regression model to estimate and

– with confidence intervals• or, equivalent but simpler, multinomial model

– split of total accidents between periods

Form of data required

Before SSP After

No. years 9 3 8

Site total 78 31 43

Partnership total 32376 11308 24148

For each camera (eg PICs at LCR C1)

Approximate camera estimate = 43/24148

78/32376= 0.739

Compare the numbers in each period relative to partnership totals

Estimates of : camera effect

Estimates of : RTM effect

Timing of the SSP?

• longer gap from end of SSP to installation?• is there an ASBiC period?

– after selection but before installation of camera– mean rate drops to the before level

• if SSP is earlier, some RTM in the before period– hence inflates camera benefit– important to get timing of assumed SSP right– assuming it is not known

time

Pre-SSP

SSP

ASBiC

Post-installation

Accs/yr (adjusted for trend)

Installation of camera

RTM

Camera effect

Line up sites by installation dateso year 1 is first post-camera yearTransition year omittedScaled and averaged

Plot from all ten partnerships

before post-cameraSSP?

Clear signs of raised levelbefore assumed SSP

So how to define SSP?Leave out 4 years instead of 3?Find 3-yrs with max accidentsto find “most likely” SSP?

553 cameras in total

EBM or RACF method?

• EBM more complex– allows for trend, using national totals– requires a predictive accident model (PAM)– and data on flows etc for each camera site– robust to uncertainty about the timing of the SSP

• RACF method simpler in many respects– allows for trend (in same way)– needs long run of annual accident data– comparison of accidents in before, SSP, camera periods– estimates obtained by statistical model fitting – eg R– but no PAM, and no flow data required– but potentially sensitive to assumption of SSP

Summary

• EBM used in DfT 4-year evaluation report– but requires reliable flow data for each site (and PAM)– seen as complex

• RACF method has some advantages– no PAM needed, no flow data needed– but does need sufficient before accident data – but arguments about what to assume about SSP– discussions between Richard Allsop and me ..– .. criticised by Idris Francis, Dave Finney and others ...– lots of letters in Local Transport Today– Allsop revising his recommended method

Thank you!

Any questions?