INDEPENDENT TECHNICAL EVALUATION (GATE 2) · PDF fileINDEPENDENT TECHNICAL EVALUATION (GATE 2)...

Leeds | 67 Albion Street Leeds LS1 5AA [email protected] | +44 113 389 6400

1 of 21

www.steerdaviesgleave.com

INDEPENDENT TECHNICAL EVALUATION (GATE 2) Greater Cambridge Greater Peterborough Enterprise Partnership Business Case Review Pro-Forma: Strategic Case

Scheme Name: Junction 20 Improvement Scheme

Date Completed: Gate 1 – 9 October 2015, Gate 2 – 9 November 2015

Completed By: Steer Davies Gleave - Steve Hunter, David Sutanto

Document Reference Comment (October) Assessment (RAG) Comment (November) Assessment (RAG)

INTRODUCTION

Headline Description

Some rewording suggested. For example, the first paragraph implies that traffic to East Anglia and the ports is entirely the heavy use. The last sentence of second paragraph (“roundabout is often subject to extensive and sporadic queuing”) (emphasis added) could support the scheme more convincingly.

G

Emphasis in October comment was added to indicate the inherent uncertainty in the statement, rather than as a suggested change to the document.

G

Geographical Area

Some rewording required. Junction 20 is no longer the point where the A47 changes from single to dual carriageway, and use of ‘free flow’ to describe the A15 (when elsewhere in the document this is used to describe conditions between midnight and 5am).

A Addressed in document. -

Suggest this section would benefit from some further context of the (limited?) strategic highway network around Peterborough, for example the constraints that the East Coast Mainline places on east-west movements and River Nene on north-south movements.

G Addressed in document. -

STRATEGIC CASE

Case for Change

Summary of Strategic Case

This is more a description of what the scheme will do, rather than (as it should be) why action needs to be taken and why this is an appropriate way to do it.


2 of 21 www.steerdaviesgleave.com


It should be possible to follow the ‘primary aims’ of the scheme (even if they shouldn’t be in this particular section) right the way through to monitoring and evaluation — to eventually show that the scheme delivered what was intended, provides value for money, and is the best option.

G Improved in document. -

Existing Problems

Rewording of first sentence suggested (“has often been subject to delays and queuing”) and evidence of these problems (“as confirmed by observation”) is not considered to be sufficiently convincing. The model results are better, but referring to the queue length surveys here would be even better.

A Improved in document. Would still benefit from including evidence from queue length surveys.

G

The description of what the scheme will do (which follows) should not be in this section, and should ideally be clear how it follows on from the problems. For example, “the proposed scheme will increase capacity on all four approaches” should follow the existing problems section explaining that capacity on all four approaches is an issue. The model results appear to show that only two arms (one in each peak) are at capacity, therefore, a justification as to why additional capacity is required on the A15 arms is needed.


The table of ARCADY results would benefit from ‘W’ and ‘E’ labels being added for the A47 approaches (similar to those for the A15). In the paragraph which follows Eastbound and Westbound appear to be wrongly/inconsistently attributed (Westbound approach first said to exceed capacity in the PM Peak and then the AM Peak).

G

Improved in document, not clear in new Table how AM peak on A47W approach has less delay than off-peak (presumed meaning of negative values) or how aspects have been selected to be drawn out in the text (for example no mention of A15S approach which appears to have the longest delays?).

G



The figure of flows does not support the conclusions drawn from it in the paragraph below. Adding a total line and making sure that horizontal axis labels match quoted time periods would perhaps help to show:

Peaks being ‘significantly higher’.

AM peak plateauing.

4,000 vehicles in the peak hours.


No results are included for the journey time analysis from TomTom, which, therefore, makes a limited contribution. Comparison of peak and off-peak journey times would also be useful. The second journey time paragraph is of limited use without overnight approach count information for context.

G Addressed in document. G

Suggest rewording of paragraph on Peterborough’s main growth areas:

Description of Junction 20 as a crossroads not very clear.

Sites not usually described as ‘residing’.

Presume should be city centre comprises (rather than compromises).

The implication in the last bullet, which suggests that each neighbourhood centre has a business park associated with it, is unclear.


The paragraph on what a lack of capacity will do needs evidence for each bullet and as to why the benefits are important to the sub-region. The paragraph on the Paston Reserve development stalling appears a key point, but no evidence is given regarding the link between this and Junction 20.

A Improved in document. -



Considered Options

Do Nothing

This section needs supporting evidence (for example forecast year junction performance). The wording states that the new development traffic would make the junction unviable so the development would not happen, which is contradictory.

A

Would benefit from explanation of whether or not the development traffic is in the table (comparative results with and without the demand would be helpful).

G

Strong statements, such as that this would be ‘highly detrimental’ (for Peterborough) needs supporting evidence.


Do Something See comments on Value for Money Report. R See comments on Value for Money Report. R

Preferred Option See comments on Value for Money Report. R See comments on Value for Money Report. R

The section discussing the fit with Peterborough’s Core Strategy and Local Transport Plan would more usually be in Strategic Fit section. The explanation of this fit could be enhanced. For example, why is possible the phased development of the junction might affect policy CS12; why do more efficient journeys reduce the need to travel and improve quality of life (policy CS14, noting that the fact that a major development ‘can’ contribute to improved public transport provision, walking and cycling is of limited veracity); how does Junction 20 assist policy CS15 (perhaps policy CS8 would be a bit clearer); and how does it contribute to the overarching vision of the Local Transport Plan 3.


Expected Benefits

There should be a clear relationship between these sections and the primary aims of the scheme (currently in Summary of Strategic Case).




Managing and Delivering Growth and Development of Sustainable Communities

This section would benefit from a clearer explanation of the link between the junction improvements (and reduced congestion etc.) and improved access routes to the two sites. The final sentence could be clearer as to whether it is that facilitating the two strategic growth sites which contributes to Peterborough’s growth strategy, or whether there is a direct link between the Junction 20 improvements and the growth strategy.

G Removed from document. -

Maintain a Thriving Economy

This seems to be the first mention of improving journey time reliability. It should be noted that in WebTAG reliability is day-to-day or seasonal variability within a time period, rather than the difference between peak and off-peak journey times. Suggest rewording to make it clear what is meant.

This section would benefit from a clearer explanation of how the scheme will generate ‘direct and indirect employment’ and also more specifically how it will ‘unlock and accelerate the delivery of new floor space within Peterborough’.

A Removed from document. -

Better Accessibility

A reduction in accidents is not a measure of accessibility. Whilst this could be a benefit, if it is described as critical to the scheme then analysis/appraisal effort would be needed (the same is true for its later inclusion under monitoring and evaluation).

A Removed from document. -



It is not immediately clear how the scheme improves ‘sustainable travel networks’ or how the road improvement will enable ‘all users to thrive equally’, therefore, an explanation is needed for why this is the case.

An explanation of the scheme’s effect on accessibility is required.

G Removed from document. -

Project Scope

Cost Reduction

Further explanation would be useful (in the absence of the technical notes which are listed as supporting the Value for Money note) of why the A15 slip roads (ARCADY results show a reasonable amount of spare capacity) need widening.


First mention of the ‘far reaching’ benefits of upgrading street lighting, if it is as critical to the scheme as this sounds then analysis/appraisal effort would be needed and its inclusion in the monitoring and evaluation strategy.

A No changes made. Not clear that it is relevant to this section – suggest is re/moved.

G

Potential for Lower Cost Option

Some discussion of why an un-signalised reconfigured junction would not be appropriate as an option would be useful here.

The second and third paragraphs seem to be more about reducing the cost to the LEP by other sources of funding rather than by spending less money. An explanation of why the scheme could not be taken forward (as planned) with Section 106 contributions (thereby reducing the cost to the LEP) would be helpful, including why it is reasonable that the LEP should pay the full cost so that Peterborough can get Section 106 contributions to spend on other schemes (final paragraph).




Impacts and Mitigations

The impact description is only correct if implementing the scheme will not result in additional trip. This contradicts the assertion that the development sites would not go ahead without the scheme.

It is also not entirely clear that removing congestion reduces traffic noise. If air quality and noise reduction are as critical to the scheme as presented in this section, then analysis/appraisal effort would be needed, and relevant measures should be included in the monitoring and evaluation strategy.

A

No changes made. Suggest wording is altered to make clearer that benefit here is more about reducing the disbenefit of additional traffic.

G

The text on tree/vegetation clearance describes the cause but needs to also explain the effect (which may well be slight).



INDEPENDENT TECHNICAL EVALUATION (GATE 2) Greater Cambridge Greater Peterborough Enterprise Partnership Business Case Review Pro-Forma: Economic Case





ECONOMIC CASE

Economic Summary The Value for Money Category of Full Signalisation option should be ‘High’ (not ‘Very High’).

G No longer relevant. -

See comments on Value for Money Report. R See comments on Value for Money Report. R

Funding Profile

Cost Estimates

Optimism bias is conventionally only included in the economic appraisal and not presented in cost estimate. The price base (nominal or outturn) should be stated.

A No longer relevant. -

Local Contribution/Third Party Funding See comment on ‘Potential for Lower Cost Option’ above.

A Addressed in document (now in financial case). -

Affordability and Financial Risk

Risk Assessment

A more specific discussion of the risk assessment for this scheme needed, rather than an explanation of what a risk assessment is.

A No changes made (now in financial case). A

Risk Allowance and Cost Overruns

More specific discussion of the value of the QRA for the scheme needed.

Optimism Bias is a different concept, which is exclusively for the Cost Benefit Appraisal.

A Addressed in document (now in financial case). -



Some rewording is required of the description of ‘additional risks’ to Peterborough Council. It should be made clear that it is not that the promoter faces additional risks (they are the same), just that it is expected to have to pay if project cost goes higher than the base+P50 cost (which is the cost overrun portion).

G Addressed in document (now in financial case). -

The inflation rate applied would more usefully be described under Cost Estimate. It also needs to specified how the 5% is made up of background inflation and real construction cost growth (the appraisal needs this information).

G

Outturn inflation rate no longer specified, would be useful information (with breakdown into assumed RPI and real construction escalation) for Project Costs section at beginning of Financial Case.

G

Project Risks

Ranking of these risks would be beneficial and perhaps only the top five provided (as they do not all appear to be material).


A delay in the development sites progressing if scheme is delayed does not appear like a project risk.


With regards to inflation risk, it should be specific how reappraisal of the costs mitigates this risk.


Value for Money

Economic Benefits See comments on Value for Money Report. R See comments on Value for Money Report. R

Long Term Road User Benefits See comments on Value for Money Report. R See comments on Value for Money Report. R



Value for Money Technical Note

3.2

Material concern that the appraisal is underpinned by the difference between two sets of traffic modelling results that come from two different traffic modelling packages. An explanation of what steps have been taken to ensure that differences between the way the two models work (for example profiling demand over the modelled period) have been accounted for is critical.

R

Approach changed such that the Linsig platform is used in the DM and DS. Specific comment no longer relevant – but note that concerns about comparability of DM and DS remain. For example an AM peak turn in 2021 from Arm A to Arm B is shown to take 63.3 seconds per vehicle with signals but 19 minutes for a roundabout. A very detailed explanation of why this is robust would be required before it could be accepted. Thus the BCR cannot currently be attached any measure of certainty.

R

Table 3.2

Have return trips from school drop-offs been accounted for (differences between TRICS arrivals and departures do not seem to reflect that many trips would immediately leave again)?

A Clarification has been added in document, explanation of materiality of assumption still required.

G

Table 3.2 Were any development trips forecast for the inter-peak? This could be significant for the schools (15:00 -16:00 presumably).

G No changes made. G

Figure 3.1 Has any sensitivity testing around the trip rate, development phasing or distribution been undertaken?


3.23

Has any sensitivity testing to alternative growth assumptions been undertaken? At a local level capping to NTEM forecasts (average across much wider area) may not be appropriate.




3.33-3.35 + Table 3.10

Traffic flow validation is not particularly relevant for a junction model (the usual application is for models where the flows through a link result from the assignment algorithm rather than being user specified). Table 3.10 shows us that the data which is used in ARCADY is similar to a separate independent count, and whilst this is useful it does not add much to the argument that the modelling is robust.

G No longer relevant. -

Table 3.11/3.38/Table 3.12

The queue length validation against DMRB criteria does not appear to be an acceptable basis for the appraisal. Justification of the alternative +/- 10 vehicles tested in Table 3.12 is required; given the scale of the modelled and observed queues (max 15) such a broad range would need this justification to be extremely compelling. The argument that this might be because the queue lengths were surveyed in mid-July is not considered sufficient.

R

Queue length validation no longer presented, strongly recommend is added back in. Validation against TomTom delay data is useful, but would benefit from more explanation of what the tables are demonstrating and how judged that the model validates.

A

3.41

Significant concern that capping growth in the Do Minimum models but not in the Do Something will distort the results of the Cost Benefit Appraisal.

The appropriateness of capping all approaches based on the overall junction RFC rather than capping individual approaches based on their own RFCs is questioned.

Note that in TUBA it appears that demand is capped for both DM and DS but not in the derivation of journey times from the LinSig analysis.

R

Discussion of capping growth no longer within document, instead sensitivity tests with different cap years are presented. VfM performance very sensitive to cap year – explanation should be included of why this is and why the 2021 cap version (negative NPV) isn’t the anticipated outcome.

A



3.53

This report would benefit from including some results from the DS/LinSig assessment (Table 3.13 gives some for the DM/ ARCADY assessment).

G No changes made. Do Minimum assessment results no longer included in the VfM report.

A

4.2

It is specified that costs have been estimated in September 2015 prices, but no information is given on how these have been converted to the 2012 price base referenced in Table 4.1.

R

No longer relevant -

4.5

A reference is requested as to where 5% inflation is specified in WebTAG (value not recognised). Not clear how Table 4.2 values (header specifies in outturn and 2012 price base—could only be one of these) derived from Table 4.1 and this inflation rate (seems to be an increase of 2.4%). On this basis not clear that the £5.00m funding requested is outturn value.

TAG Unit A1.2, Appendix A is a worked example, rather than guidance on appropriate values to use. A justified choice of locally appropriate construction cost inflation is required.

A

4.7 Real inflation rates applied to derive values in Table 4.3 not specified (they appear to be around 2%).

No changes made. Appears that assumption is of equal contribution of background inflation and real construction cost growth (that is 2.47% pa for each compounded). Justification for both assumptions is required.

A

4.10 The text specifies that maintenance is every five years, whilst Table 4.3 shows it happening every ten years. Approach to be clarified.

No changes made. A



Table 4.5 and Appraisal Cost Pro Forma (p25)

Cost totals do not match what is in TUBA, which includes a capital cost lower than £4,848 (in the bottom right cell) which should match and no maintenance or renewal costs have been included in TUBA.

Cost totals still do not match TUBA, partly because the spend profiles do not precisely match (three years shown in the VfM report and two in TUBA inputs). Not clear how the numbers input to TUBA relate to the VfM report (they do not appear to match any of the tables presented). Although TUBA has percentages for maintenance spend there is no total included so no values are included. Not clear why/what risk is included for maintenance (Table 4.4) nor why no renewals (over 60 yrs).

R

4.11

The QRA adjustment of 3.7% appears relatively low (presumably a product of the use of fairly narrow cost risk ranges in the assessment (Appendix). It is not clear what role the uncertainty assessment (also in Appendix) plays in the cost calculations.

A

No changes made. Note that no inflation appears to have been applied to QRA – which conventionally would be in the same price base as the cost estimate (2015) with risks potentially occurring throughout the construction timescale.

A

4.28 A description of the benefits and why they arise would be useful in this section

G

No changes made. Existing text would also usefully make clear that journey speed is the driver of vehicle operating cost differences (that is, even though distances do not change – time is not the only impact).

G

4.30

Not clear how the distance input to TUBA was derived and how it relates to the journey time (appears to be a notional 1,000 metres). Ensuring that the distance and journey time measures are consistent is important because TUBA calculates average vehicle speed and uses this to derive the change in vehicle operating costs.

R Appears to have been improved in analysis, explanation of approach taken needs to be in the report.

A



4.31

Treating the junction as a single Origin Destination pair in TUBA is unsatisfactory. It is suggested that a 4 x 4 matrix is used with each turning movement separated (rather than basing TUBA’s calculations on an average of all turns). Junction model results were not provided for this review; more detail of the process to derive the table on p27 is requested.

Approach has been improved in the analysis, however no further explanation of how the inputs were derived has been provided (and the comparison table has been removed.

Further explanation of the TUBA results is required – particular concern is the proportion of time savings which are more than five minutes per vehicle (for example 70% of commuter time saving benefits are from vehicles saving more than five minutes each).

R

4.33

Uniform local vehicle type split proportions are used when values which vary by turn and by peak are available (Tables 3.8 and 3.9). The data shows material differences between turns (and peaks). This should be reflected in TUBA to ensure that the delays per turn movement and time period are based on an appropriate vehicle mix (and therefore blend of values of time)

TUBA input data continues to use uniform vehicle split (now per turn rather than for aggregate flow). Local information on turn specific vehicles splits is available (was used in Arcady) and should be used.

R

4.34

Annualisation value and justification to be included. It appears to be 253 for all periods, so any impacts in the evenings and weekends are being ignored in the assessment. Note that splitting the inter-peak period into individual hours (but with the same inputs) is not necessary. Adding the shoulder peak hours does include additional information (but may not be proportionate).

Annualisation approach improved. Text would benefit from further explanation of why the factors presented are appropriate (in general terms, for example one could say that from the data an average across the 48 hr weekend demand was half the standard interpeak hour).

Not clear that interpeak hour flow input into TUBA matches the graph at page 3 of the new document – total demand input to TUBA seems to be higher than average interpeak flow might suggest.

G



4.39

The suggestion that the scheme cost estimate is the same for partial and fully signalised options does not sound plausible. On the current BCR performance of the fully signalised option it would not be Value for Money to include any passive provision which incurs cost (over the partial signalisation solution).

A No longer relevant. -

4.42/4.43

The difference between the partial and full signalisation schemes is a little surprising. The ‘full’ plan at Figure 2.1 shows two additional stop lines/sets of signals in comparison to the ‘partial’ scheme.

A

No longer relevant, but (partly from having been there previously) these results would benefit the explanation of why a full signalisation solution is now preferred.

G


INDEPENDENT TECHNICAL EVALUATION (GATE 2) Greater Cambridge Greater Peterborough Enterprise Partnership Business Case Review Pro-Forma: Financial Case




As there was no ‘Financial Case’ heading in the scheme documents, please see comments relating to the other sections.

Comments on text transferred to Financial Case in the updated scheme documents remain with the original comments in other sections.


FINANCIAL CASE

Project Costs Maintenance and renewal (see comments on VfM technical note) assumptions should be summarised here.

G


INDEPENDENT TECHNICAL EVALUATION (GATE 2) Greater Cambridge Greater Peterborough Enterprise Partnership Business Case Review Pro-Forma: Commercial Case





COMMERCIAL CASE

Risk Allocation

The current text is mainly a description of how the procurement will be undertaken, rather than specifying how risks are will be allocated between promoter and contractor.

The text describes the benefits of the target price contracting strategy that is ‘likely’ to be used. It would be useful to outline what other contracting strategies have been considered, and why they are considered to be inferior, and in what circumstances they might be used. This should also include any past experience the council has with target price contracts on similar schemes.

A Some improvement in document, further amendments to address October comments required.

G

Procurement

Except for the first paragraph, the current text is more about risks and programme than procurement.

The first paragraph outlines the procurement strategy that has been chosen. A discussion on any alternatives considered, and why they were discounted, would be useful.



INDEPENDENT TECHNICAL EVALUATION (GATE 2) Greater Cambridge Greater Peterborough Enterprise Partnership Business Case Review Pro-Forma: Management Case


Date Completed: 19 October 2015, 9 November 2015



MANAGEMENT CASE

Delivery

Project Plan

The current text does not appear to relate to the project plan. For example, the last few paragraphs are a list of tasks to be undertaken.

A No changes made, amendments to address October comments required.

A

Construction Milestones

More detailed should be provided. This should include funding approval dates and the completion date.

A Improved in document. Funding approval/ gateway (target) dates required.

G

Governance

Risk Management

The use of a risk register is discussed, but it would also be useful to specify more broadly how risks will be managed, and that a risk management strategy will be implemented.

A Addressed in document -

Monitoring, Evaluation and Benefits Realisation

Benefits

There should be a clear relationship between these sections, the primary aims of the scheme (currently in Summary of Strategic Case) and the Expected Benefits set out in the Strategic Case. Also see comments on the latter parts of the Strategic Case, for example benefit of street lighting introduced under Cost Reduction.

G

Improved in document. Would benefit from consistent naming/descriptions between sections. For example ‘accessibility’ rather than ‘capacity’ used here.

G



Monitoring and Evaluation

Again, there should be greater consistency between this section and the earlier sections of the form. The process for specifying targets for each of the measures is not described. Responsibility for benefits realisation is not specified.

G

Improved in document. Would benefit from consistent naming/descriptions between sections. For example ‘journey time reliability’ used (WebTAG interpretation is that eg AM peak journey times will be predictable day to day) rather than ‘reduced congestion used here.

G

Leeds | 67 Albion Street Leeds LS1 5AA [email protected] | +44 113 389 6400

20 of 21

www.steerdaviesgleave.com

Reviewer Summary Comments

1. Has a sensible and proportionate methodology been applied?

Broadly the methodology is sensible in the strategic, commercial, financial and management cases. The economic case and particularly the modelling requires

significant work before it can be recommended for funding. Issues include: the way in which TUBA has been used to gauge the user benefits, lack of clarity of the

cost calculations, lack of justification for capping growth across all approaches, the interaction between the ARCADY, LinSig and TUBA. It is important, in order for

us to assess the value for money of the scheme, that we have access to the detailed breakdown of the economic case.

The strategic case would also benefit from an improvement in the logical flow. It is important that the scheme sponsor makes it clear that the scheme is responding

to current and future problems and also that they are developed in line with the local, regional and national policy direction.

The majority of changes made in the most recent submission improve the narrative and flow of the case, however in some areas more evidence is required. A

modelling methodology with fewer areas of uncertainty has now been adopted for the analysis underlying the economic case, however some further explanation of

the method and of why the results are robust/appropriate is required.

2. Has the methodology been applied accurately?

Methodology has generally been applied accurately with the exception of the economic case as explained above. There are some improvements that could be

made to make a more compelling business case. In the discussion of alternative options a more evidenced based approach would help the case for discounting

schemes. The management case requires a project plan so that the project board can gauge the progress of the scheme construction. Additionally, risks are not

dealt with comprehensively enough through the business case. A full risk register with mitigation strategy would show that the project team had considered what

could go wrong and that they had a plan for remedying the situation.

It is not clear from the evidence whether the methodology has been applied accurately. The BCR for the scheme has increased dramatically and is extremely

sensitive to growth cap assumptions. Further explanation and justification of the results is required. Cost calculations require further explanation and remain

inconsistent between the Value for Money report and TUBA, the latter being the source of the BCR. In other areas further explanation and evidence remain

required.

3. Has the analysis either reduced or helped to expose/ understand uncertainty?

There is clarity in parts of the case, but the confusion caused by the economic case means that this is currently not a compelling case. The strategic case shows that

there is a need for the scheme, but would benefit from a more ,logical structure. The commercial case discusses the problems that could arise with the

procurement process, but does not explain in any depth the process itself. The financial case is not specifically signposted, being part of the value for money

section. Having a specific section for financial case would help the flow of the document and make it easier to follow. The management case shows a clear

governance structure, but is in need of a project plan.


The updated submission has reduced methodological uncertainty but requires further explanation and justification of the results. In general the presentation of the

case for the scheme is significantly improved, but there remain some areas where further evidence is justified in reducing residual areas of uncertainty.

INDEPENDENT TECHNICAL EVALUATION (GATE 2) · PDF fileINDEPENDENT TECHNICAL EVALUATION (GATE 2)...

Documents

Transcript of INDEPENDENT TECHNICAL EVALUATION (GATE 2) · PDF fileINDEPENDENT TECHNICAL EVALUATION (GATE 2)...