MScEPS EC8014: Economic Evaluation: Theory, Techniques and Applications

Infrastructure and Environment

Dr Edgar Morgenroth Associate Research Professor

Economic and Social Research Institute Adjunct Professor

Trinity College Dublin

edgar.morgenroth@esri.ie

“In the US no one can market a prescriptive medicine for male pattern baldness without evidence that it is “safe and effective”.... Few public actions, even those of tremendous importance, are ever evaluated to a standard required of even the most trivial medicine.”

Pritchett, 2002

“the value of infrastructure is not equal to its cost” Pritchett (1996)

2

Outline

Why evaluate?

Evaluation methodologies

Key parameters

Examples

‘Accompanying Measures’ – institutions, implementation, optimism bias.

3

Why Evaluate?

Resources are limited;

One can think of an almost unlimited number of desirable (at least on the face of it) projects;

Governments do not have unlimited budgets;

Governments need to prioritise;

The economic approach to prioritisation is to do so on the basis of the most efficient allocation of resource;

Evaluation of public projects is different to evaluation of private projects – correct prices, market failure, taxation.

4

Not all infrastructure is created equal

Some types of infrastructure investment have a higher return than others.

This is related to the adequacy of the existing stock – eliminating bottlenecks or other capacity constraints has the highest return.

The return to infrastructure investment is also related to the relevance of the infrastructure for the economy – presidential palaces tend to have no impact.

Core infrastructure has been found to have the highest return – energy, telecoms, transport and water.

Lower returns on education, health and public buildings. With limited resources, prioritisation is even more important. The lawnmower approach to cutting capital (and current)

expenditure is an expedient but flawed solution!

5

Environment

Just as not all infrastructure is created equal there are differences across environmental investments.

The cost of reducing carbon emissions differs across different technologies.

What is the net benefit of avoiding different types of pollution?

Habitat protection

The environment should also be considered in infrastructure projects

6

Infrastructure Investment in Ireland

In common with all countries, there has been public investment in Ireland since the foundation of the State.

This has varied in scale and in scope, reflecting economic cycles and policy focus.

As a less developed EU member Ireland received EU Structural Funds.

In recent years capital expenditure has fallen but is still significant.

7

8

0.0%0.5%1.0%1.5%2.0%2.5%3.0%3.5%4.0%

% o

f G

DP

General Government GFCF Average 1970-2013

Source: Own calculations using data from EU DG-ECFIN AMECO database.

Evaluation

In order to prioritise we must evaluate.

Typically evaluation is cheap relative to the overall costs of any policy instrument e.g. the cost of the ex-ante evaluation of the last NDP 2007-2013 was 0.0002% of the total planned expenditure!!

So why is there so little hard evaluation?

9

Who Plans and Evaluates Infrastructure Investment in Ireland?

Current situation involves promoting agencies in charge of evaluation – some oversight by the relevant government departments and the department of finance/public expenditure and reform.

It has been (is?) government policy not to publish evaluations or to withhold key aspects of the evaluation.

The public is usually the only party that does not have full knowledge of a project/programme.

In the past public investment in Ireland was significantly co-financed by the EU Structural Funds – the EU insisted on evaluation and planning.

10

Planning & Evaluation

Why did the EU insist on planning and evaluation?

1. Not all investments have the same return – evaluation helps identify those with the highest return;

2. Investments should relate to economic needs – where are the constraints?

3. Different investments may be complements or substitutes;

4. Investments typically run over several years – multi-annual plans give funding certainty.

5. Agreed plans are easier to evaluate ex-post, than ad-hoc investments.

11

Institutions and Public Investment – the impact of corruption

“Any project evaluation can be disrupted if corruption and bad regulation distort the decision making process” (Florio and Myles, Fisc.Stud. 2011)

Low-quality governance leads to higher level of public investment (Kiefer and Knack, Rev.Econ.Stat. 2007) .

An increase in public investment can raise the level of corruption (Hanousek and Kocenda, Fisc.Stud. 2011)

12

The political economy of evaluation

Pritchett (2002) constructs a simple political economy model where projects/programmes are support by “advocates” who are more committed to pursuing their project that the general public and where the latter is split into three groups according to their attitudes towards the project.

He finds that except in the case where advocates know that the project will have the desired outcome, they will prefer not to evaluate the project i.e. they will prefer ignorance!

13

Infrastructure Institutions in Ireland

Current situation involves promoting agencies (a large and very heterogeneous group) in charge of evaluation – some oversight by the relevant government departments and the department of finance/public expenditure and reform.

It has been (is?) government policy not to publish evaluations or to withhold key aspects of the evaluation (where is the CBA on water meters?).

The public is usually the only party that does not have full knowledge of a project/programme.

14

Evaluation Strategies: General Considerations

Evaluation can take place at different points in the project life cycle and at different scales:

Individual project

Programme of investment

Ex-post (after the investment has been put into place)

Ex-ante (before the investment is put in place)

Mid-term (during a programme of many individual investments)

Short-run impact

Long-run impact

Indirect and induced effects.

There are many evaluation methods. All have strengths an weaknesses related to what, when and which impact(s) is to be evaluated i.e. The purpose of the evaluation should determine the method that is to be used.

15

Evaluation Methodologies

A wide set of evaluation methodologies could be used:

Cost-benefit analysis (CBA), cost effectiveness analysis, multi-criterion decision analysis (MCDA), case studies, simple econometric modelling, input-output models (I-O), computable general equilibrium models (CGE), small macro-econometric models, large macro-econometric models, time series modelling (e.g. ARIMA or VARs).

Some methods are focused on individual projects (micro) while others are focused on all investment (macro)

Each method has advantages and disadvantages - one method on its own may net be enough!

The methodology should be able to measure the impact in relation to the objectives.

16

Evaluation Strategies

Macro-evaluation – evaluate the impact of all the investments together.

Wider coverage including spillovers.

Data is often more readily available.

Fully specified models (general equilibrium or econometric) have the advantage that they capture direct and indirect benefits.

This has the disadvantage that it will not be able to identify the poorly performing projects/measures.

This requires the use of a model.

17

Evaluation Strategies

Micro-evaluation

Detailed analysis of specific projects or programmes – often identifies why certain projects should not go ahead – not popular but very necessary!

There is also evidence that this type of analysis is sometimes badly done so that a project looks good.

It is difficult to get an overview of total impact (hence the need for macro analysis).

Micro analysis is a significant task for large programmes of investment.

Background research on particular areas is important.

18

Input-Output Models

Sets out the interrelationship between different sectors of the economy – supply and use of output.

Advantages: straightforward (I-O tables are published for most countries), usually very detailed.

Disadvantages: static – short-run effects only, no structural change, lacks behavioural aspects, constant marginal product, double counting.

Examples: Beutel, (2002), very popular with consultants!

19

I-O tables

Mining & Quarrying

Non-Metallic Minerals

Construction .... Total

Mining & Quarrying

111 286 240 ... 3,907

Non-Metallic Minerals

15

234 1,225 ... 2,022

Construction

11 11 1,708 ... 13,048

.... ... ... ... ... ...

Total 3,907 2,022 13,048

Output Multiplier

1.332 1.685 1.703

20

Computable General Equilibrium Models

Related to I-O Models but involve a detailed set of equations describing the economy (behavioural).

Advantages: Often very detailed theoretically based structure, don’t need time series data, some ready made models can be used.

Disadvantages: Usually static => best suited to evaluate short-run effects, takes some effort to set up, parameters are usually drawn from other studies or assumed and calibrated to one years data.

21

Single Equation Econometric Analysis

Advantages: Straightforward to implement, estimated using data.

Disadvantages: Partial equilibrium, often simplistic, dynamics often not well specified (no distinction between long-run and short-run impacts as channels are not modelled), usually focused on ex-post analysis.

These are better suited to provide supporting analysis. 22

Example These inputs are used to produce output, Y.

Production function is given as:

(1)

where the t subscripts denote time.

to estimate (1) we need to choose a functional form. The most common choice in the literature has been the familiar Cobb-Douglas:

(2)

where a has been added to represent the state of technology.

differentiating the production function with respect to infrastructure yields:

(3)

where MP denotes the marginal product of infrastructure

23

tttt KGLKfY ,,

ttttt KGLKAY

t

ttKG

Y

KGMP ,

Elasticities with respect to Infrastructure

-0.09

-0.48

-0.07

0.25

-0.19

0.33

0.59

0.10

0.73

-0.60

-0.40

-0.20

0.00

0.20

0.40

0.60

0.80

Output Cost TFP

Interpretation: A 1% increase in infrastructure increases output by 0.25% on average. Outer bounds correspond to two standard deviations from mean. Based on parameters taken from studies on the economic impact of infrastructure in Greece, Ireland, Italy, Spain and Portugal.

24

Time-Series Models - VARs

Advantages: Reasonably straightforward to estimate, no need to specify a theoretical model, very good on dynamics.

Disadvantages: Lacks theoretical foundations, not terribly good at identifying long-run structural impacts, simplistic aggregate relationships only (parsimony), data hungry (ideally quarterly data).

25

Small Macroeconometric Models

HERMIN (27 EU Countries plus some others e.g. Turkey)

Advantages: Theory consistent, parameters usually estimated using data, more comprehensive than single equation approaches, long-run and short run impacts.

Disadvantages: Need to simplify (aggregate), can be misleading if not properly understood (maintained assumption of infrastructure deficit implies that investment always has a positive impact).

26

HERMIN

HERMIN

[1] National

accounting

Framework

[3] Expenditure

[5] Model as

integrated system

[4] Income

[6] Behavioural and

identity equationsManufacturing (T)

Agriculture (A)

Non-market services (G)

Market services (N)Utilites

Other market services

Output (GDP)

Expenditure (GDE)

Income (GDI)

Private consumption

Public consumption

Investment

Stock changes

Exports

Imports

General Government

Health

Education

Agriculture

Forestry

Fishing

GDP

productivity

PSBR

Output = Expenditure usedto determine net trade balance

(NTS = GDP - GDA)where GDA = C + G + I + DS

Output = Incomeused to determinecorporate profits

(IYC = GDP - YWI)

Revenue

Expenditure

Borrowing requirement

Debt accumulation

Corporate sector

Household sector

Behaviouralequationsinfluencedby theory

Identities adding up,definitional & closure

Public sector

Private sector

Consumption function

Wage bargaining

Factor demands

[2] Output

Building (BC)

27

Fully Specified Macro-Models

HERMES (Ireland)

Quest (EU Model)

Advantages: Very detailed and comprehensive, theory consistent, parameters estimated from data, long-run and short-run impact.

Disadvantages: Costly to built and maintain (HERMES has over 900 equations), more difficult to identify causality, results can be difficult to interpret.

28

Ex-Ante Macro-Model Evaluation

To carry out an evaluation we do the following:

We carry out a model simulation starting in the a year previous to the start of the investment programme and continue the simulation out past the end of the programme

We then “extract” the NDP “policy shocks”, i.e., we set the NDP expenditures at zero and re-simulate the model.

The difference between the two simulations gives us the impact of the NDP.

This is quite an artificial way to identify the impact as in a normal setting investment is unlikely to be zero i.e. The counterfactual is quite extreme. An alternative would be to compare the with new NDP scenario with a ‘business as usual’ scenario.

29

Mid-Term Evaluation of the NDP 2000- 2006 (GNP) – impact of

investment for the period 2000- 2003

0

1

2

3

4

5

6

7

8

20002002

20042006

20082010

%

Demand Only Total

Crowding Out

Building and Construction bids resources away from other sectors

Higher wages because labour demand

Higher house prices affects wages

The tradable sector has to sell abroad

Badly affected by competitiveness

More construction jobs – less jobs in manufacturing & business services

Should introduce measures to take the heat out of the housing market!

More modest NDP recommended.

31

Multi Criterion Decision Analysis (MCDA)

Implemented through a scoring model, that defines criteria for different types of investments and weights them in accordance with priorities.

Advantages: Very flexible, allows for multiple conflicting objectives, allows for comparison of very heterogeneous investments, can be done where little data exists, can be used for loosely defined projects/programmes.

Disadvantages: Can be accused of being subjective, requires thorough knowledge of the investments and background analysis.

32

Multi-Criterion Decision Analysis (MCDA)

Widely applied in management science.

Multi Criterion Analysis, was first applied by Honohan (1997) and subsequently in ESRI investment priorities and mid-term evaluations, is becoming more popular (e.g. Cundric, Kern and Rajkovic, 2008, KPMG, World Bank 2010 etc.)

This is a simple method that entails the classification of measures and a judgement of the measures against a set of criteria.

33

MCDA

For each type of intervention a set of criteria is defined - importance, contribution, dead-weight, least cost, targeting, environmental impact……..

The criteria should reflect government objectives and then economic rationale for such interventions.

It is possible to have a large set of criteria covering all objectives in detail. Criteria can have different weights.

Each possible investment is judged (scored) against the criteria.

A composite score is calculated, which allows investments to be ranked.

The results are remarkably robust once the criteria are well chosen – the cream always rises to the top!!!

34

Multi-criterion Decision Analysis (MCDA)

Goal 1Criterion

2 Scoring 3 Ranking 4 Decision

5

Operationalisation Scoring

Selection of

preferred

alternativesImplementation

Monitoring and evaluation

of the results

35

MCDA Results - Examples

Priority Measure Sub-Measure Area Composite Score

Local Infrastructure

Non-National Roads

Specific Improvement Grant Scheme Transport 0.8

Environmental Infrastructure

Management and Rehabilitation of infrastructure Environment 0.8

Local Enterprise Tourism Special Interests Pursuits Tourism -0.1

Local Enterprise Forestry Harvesting Equipment Agriculture -0.1

36

Cost Effectiveness Analysis

Considers the most effective option once a specific objective is chose – popular to assess treatment options in health care or other areas where it may be difficult to monetise all benefits.

Advantages: Can be applied in situations where it is difficult to monetise benefits, clear ranking

Disadvantages: Not practicable for a large set of heterogenous projects/ programmes. 37

Social Cost Benefit Analysis (CBA)

Widely used. Involves comparing total (all) expected benefits with total (all) expected costs. In order to take account of the timing of benefits and costs a discount rate is used to convert these into a present value;

Advantages: Very detailed analysis using a common metric, comparable results across different types of investments.

Disadvantages: Dependent on parameters and counterfactual, onerous to implement for a large programme of projects, some projects are difficult to evaluate (especially ill defined projects/programmes).

Standard methodology – required in Ireland for all public projects above a threshold size (€20 million)

38

Steps Required in doing CBA

1. What policy or project is being evaluated? What alternatives are there? - often the only alternative considered is to do nothing!

2. Whose costs and benefits are to count? – often not all costs/benefits are considered!

3. Over what time horizon are costs and benefits are counted? – there could be terminal values at the end of the time horizon that may be important!

4. Costs/benefits today are not the same as costs/benefits in the future – need to decide on discounting. – discount rates can have a significant impact on results.

5. What are the risks to the costs and benefits?

6. What are the distributional impacts of costs/benefits?

39

Social Cost Benefit Analysis (CBA)

Involves comparing total expected benefits with total expected costs.

The usefulness of the analysis is critically dependent on key parameters;

In order to take account of the timing of benefits and costs a discount rate is used to convert these into a present value;

Risks;

Counterfactual.

This looks like a simple task??

40

Decision Rule

Having done the measuring one can assess whether the project should go ahead.

Pareto improvement – do the project if some people gain and nobody loses.

Few projects would ever meet this criterion.

Hicks-Kaldor criterion – a project should be carried out if the gainers could compensate the losers and still be better off.

Of course the losers are often not compensated in practice!! => implications for the planning system – objectors!!

41

CBA

What is a benefit? – anything that increases national welfare.

Benefits can be monetary (cost reduction), non-monetary (e.g. reduction in complaints), non-monetary qualitative (improved quality of life).

In many cases it is possible to find ways to monetise non-monetary benefits, but this takes some effort e.g. Environmental quality effects can be assessed with hedonic models of property prices (the benefits/disamenities should be capitalised in property prices).

What are costs? – Costs relate to actual resource use in the economy and reflect the best alternative uses that the resources could be put to (i.e. they are opportunity costs). It is important to explore what alternative opportunities may exist.

42

Market Failures

In an ideal world the market system leads to an efficient outcome. However, we do not live in an ideal world (perfect competition and perfect information are interesting academic concepts but are far removed from the ‘real world’)

Market failures/externalities are not taken into account in normal market interaction including price setting => market prices are often not appropriate for social cost benefit analysis

Taking such distortions into account distinguishes social cost benefit analysis from standard investment appraisal.

Price distortions also make cost-benefit analysis more difficult to conduct.

43

Market Failures

Monopoly – price is above marginal cost – should we use the (higher) market price or the marginal cost?

If the input used is diverted from other activity we should use the marginal cost (because other users of the input loose)

If the input used is additional to that produced then the market price should be used.

Indirect taxes such as VAT imply that the price faced by consumers are not equal to ‘free’ market prices – the same goes for subsidies and tariffs. Taxes that address distortions/externalities imply that the tax inclusive price is more efficient – that is the price to be used.

44

Market Failures

Unemployment – if a project employs unemployed people then the cost of employing them is their lost leisure rather than the wage they receive, which is typically less than the wage (involuntary unemployment). Employing unemployed people also has a consumption effect. If labour is fully employed an additional project could also have significant inflationary consequences.

Foreign exchange/trade – inputs into a project could be imported which implies that the benefits accrue abroad (a significant proportion of the cost to build the bridge of peace in Derry went on steel from Wales – local benefits were limited).

45

Market Failures

Property rights – some externalities are difficult to capture in property rights (Coase Theorem - if trade in an externality is possible and there are sufficiently low transaction costs, bargaining will lead to an efficient outcome regardless of the initial allocation of property - fails). Current controversies around wind turbines and pylons are examples – land owners are compensated if a pylon is placed on their land but neighbours who get a visual dis-amenity are not compensated!

Pricing of non-market items – anything that is not traded on a market does not have an observable market value. Nevertheless these items may account for a large proportion of benefits – travel time savings/value of time, lives saved.

46

Shadow Prices

Shadow price = Social opportunity cost;

Market prices can give a misleading signal;

We have already noted the issue regarding unemployment that the cost of employing unemployed people is the value of their lost leisure time because that is their opportunity cost. With full employment the opportunity cost is the wage earned in another job (i.e. The project would displace employment elsewhere);

Assuming labour is allocated efficiently the market wage equals the opportunity cost so that the shadow wage is equal the market rate – this simplifies a CBA – but is often not valid.

47

Shadow Wage

In practice if the shadow wage is lower than the market wage then the benefit is calculated by pre-multiplying the wage bill associated with a project by (1-v) where v is the shadow wage expressed as a fraction of the market wage.

If v=1 then the shadow wage equals the market wage and there is no benefit.

But how do we determine the shadow wage?

48

Discount Rate

Costs and benefits do not accrue immediately.

A euro to be received in 10 years time is worth less than a euro received today.

Converting future values to a present value is accomplished using a discount rate.

Setting the appropriate discount rate has been a topic of countless research papers.

Current test discount rate is 5% (4% until recently).

49

Example: €1000 received in the future

50

Discount Rate

Years 0% 4% 8% 12%

1 €1000 €962 €926 €893

10 €1000 €676 €463 €322

20 €1000 €456 €215 €104

50 €1000 €141 €21 €3

100 €1000 €20 €0 €0

Setting the Discount Rates

The choice of discount rate clearly matters.

What is the right rate? Alternative approaches:

1. Use the cost of public borrowing to proxy the discount rate – This is taken as a riskless rate, but projects are inherently risky.

2. Use the opportunity cost of funds not used for other projects (private return).

3. Use a rate that reflects the rate of time preference of individual with respect to consumption decisions (Ramsey Equation).

The different approaches yield different results.

51

Ramsey Equation

Set up a simple growth model where investor must decide how much to save (invest) and how much to consume.

Utility is a function of current and expected future consumption.

The marginal utility loss of consuming a little less today and buying a little more of the asset should equal the marginal utility gain of consuming a little more of the asset’s payoff in the future.

Solving the model gives rise to the so-called Ramsey equation:

rtf=φ+ η g

Where rtf is the risk free social discount rate, φ is the pure rate of time

preference used to discount utility, η is the elasticity of marginal utility of consumption and g is the growth rate of consumption.

Given values for each of the components it is simple to derive an estimate of the discount rate.

52

Discount Rate Estimates

53

Source: Bond Rate from OECD, Real net return calculated using CSO Institutional Sector Accounts and Capital Assets, time preference calculated using consumption from CSO National Accounts (see paper for detailed description) – see Morgenroth

(2013).

Example – Roads Scheme

Benefits:

- Changes in Travel time

- Changes in Operating cost

- Changes in Accident costs

Costs

- Capital Costs

- Changes in maintenance costs

54

Examples – Roads Scheme

55

Examples – Bus Rapid Transit

Benefits/Disbenefits

- Change in travel time for drivers, transit users

- Change in vehicle operating costs for drivers, fares for transit users

- Change in emissions of criteria pollutants and greenhouse gases

- Change in crash costs

Costs:

- Capital costs of materials, equipment, infrastructure construction, new buses

- Operations and maintenance costs

56

Risks: Cost & Benefit Estimates

Projects are subject to a range of risks: construction risk, operating risk, demand risk, financial risks and political risk.

But projects are typically promoted on an “everything goes according to plan” basis.

Given the uncertainty it would not be surprising to find that project costs/benefits are not accurately projected – but one would expect the average error to be zero.

The international literature finds a systematic optimism bias (Flyvbjerg, 2003, 2004, 2005, Bain, 2009, Pickrel, 1990).

57

Cost Overruns

Flyvbjerg et al (2003) analysed 258 projects from 20 countries covering rail, bridge, tunnel and road projects. They found that 90% of projects were subject to cost overruns. The average cost overrun for rail projects was 45%, bridges and tunnels were subject to an average 34% cost overrun and roads cost on average 20% more than initially estimated.

ICT projects in the transport area were found to suffer particularly large cost overruns averaging 200%!

58

Lower Benefit

Pickrel (1990) considered rail projects in the USA. He found that for nine projects the actual passenger numbers were 50% lower than expected while for one project the passenger numbers exceeded the predicted level by 50%.

Flyvbjerg et al (2004) analysed 210 projects from 14 countries 90% of rail projects overestimated demand (average bias 51%). For Roads the estimates on average understated traffic by 9.5%.

Parthasarathi and Levinson (2010) considered 391 road projects in Minnesota constructed in the 1960’s and compared the traffic forecasts with the traffic counts taken in 1978. They found that on average traffic on roads was underestimated by 19.5% and that the deviations of actual from projected traffic ranged from -60% to +57%. 59

A Simple Example

The implications of the findings of this literature are best illustrated by applying them to an example of a rail project with benefit to cost ration of 2:1.

For rail projects the findings are that benefit (demand) is overestimated by 50% and that costs are underestimated up by 40% at the time of project proposal.

Adjusting the benefits and costs accordingly reduces the Benefit to Cost Ratio (BCR) to less than 0.75:1.

60

Optimism Bias Implications for a Hypothetical Rail Project (BCR 2:1)

61

How can we Eliminate Bias?

One method proposed by Flyvbjerg is called reference class forecasting (see Flyvbjerg, 2007, Flyvbjerg and COWI, 2004).

What does this involve?

The conventional approach is to take an “inside view” which focuses at the project to be assessed.

The alternative is to take an “outside view” considering the outcomes achieved by other similar projects.

This approach has its origins in behavioural economics and was initially proposed to adjust for systematic cognitive biases.

62

Reference Class Forecasting

Reference class forecasting requires three steps:

1. Identifying the relevant class or type of projects with which the proposed project is to be compared;

2. Establish the probability distribution of outcomes for the relevant reference class;

3. Establish the probability of a specific outcome being achieved for the proposed project.

63

Reference Classes

What types of projects – reference classes?

They need to be reasonably homogenous.

There need to be sufficient number of comparable projects for which appropriate data is available.

64

Reference Classes Category Type of Project

Roads Motorways Trunk roads Local roads Bicycle facilities Pedestrian facilities Park and ride Bus lane schemes Bus rapid transit

Rail Metro Light rail Mainline rail High speed rail

Fixed Links Bridges Tunnels

Other building Stations Airports

IT Projects IT systems

65

Probability Distribution of Outcomes

Once the reference class is decided it is necessary to get data on the relevant information e.g. construction costs, construction time, demand etc.

This is then used to estimate the probability distribution of the target variables.

This is straightforward – cumulative distribution.

66

Cost Overruns for Roads

67

Cost Overruns for Rail

68

Probability of Outcome

Having established the probability distribution it is possible to consider the chance of a project having a particular deviation from the originally projected cost/duration/benefit.

60% of projects have a cost overrun of up to 20% => there is a 60% chance that cost overruns will not exceed 20%.

Adding 20% to the projected costs would reduce the probability of any cost overrun to 40%.

69

Limitations

The method requires access to the details of outcomes for a large number of projects (especially if one wants to be able to apply the method to different classes of projects).

The probability distributions can be biased due to outliers.

The UK Department of Transport recommends testing projects by applying the cost uplift as discussed – if this is the general rule then it is easy to ‘game’ the evaluation by reducing the projected cost to an even lower level than the possibly already biased cost estimates.

70

Readings

Bain, R. (2009). “Error and Optimism Bias in Toll Road Traffic Forecasts”, Transportation, Vol. 36, pp. 469-482.

Beutel, J. (2002). The Economic Impact of Objective 1 Interventions for the period 2000 – 2006, Final Report to DG-REGIO, May.

http://ec.europa.eu/regional_policy/sources/docgener/studies/pdf/objective1/final_report.pdf

Bradley J., T. Mitze, E. Morgenroth and G. Untiedt (2006). How can we know if EU cohesion policy is successful? Integrating micro and macro approaches to the evaluation of Structural Funds. Münster: GEFRA.

http://www.dcu.ie/education_studies/ien/iendata/Bradley-Cohesion%20Policy%20Analysis-%20Micro%20and%20Macro%20Approaches.pdf

Brent, R., (2006) Applied Cost-Benefit Analysis. Cheltenham: Edward Elgar

Burgess, D. and R. Zerbe (2011). “Appropriate Discounting for Benefit Cost Analysis”, Journal of Cost Benefit Analysis, Vol. 2, No. 2, pp. 1-18.

Cundric, A.,T. Kern and V. Rajkovic (2008). “A qualitative model for road investment appraisal”, Transport Policy, Vol. 15, pp. 225-231.

De Brucker, K., C. Macharis and A. Verbeke (2011). “Multi-criterion Analysis in Transport Project Evaluation: An Institutional Approach”, European Transport, Vol. 47, pp.3-34.

Department of Finance (2005). Guidelines for the Appraisal and Management of Capital Expenditure Proposals in the Public Sector.

http://www.finance.gov.ie/documents/publications/other/capappguide05.pdf

Department of Public Expenditure and Reform (2011). Project Discount and Inflation Rates

http://per.gov.ie/project-discount-inflation-rates/

Evans, D. (2005). “The Elasticity of Marginal Utility of Consumption: Estimates for 20 OECD Countries”, Fiscal Studies, Vol. 26, No. 2, pp. 197-224.

FitzGerald, J., C. McCarthy, E. Morgenroth and P.J. O’Connell (2003). The Mid-Term Evaluation of the National Development Plan (NDP) and Community Support Framework (CSF) for Ireland, 2000-2006. Policy Research Series No. 50, Dublin: Economic and Social Research Institute.

71

Readings

Flyvbjerg, B., in association with COWI (2004). Procedures for Dealing with Optimism Bias in Transport Planning – Guidance Document. London: Department of Transport.

Flyvbjerg, B., M. Holm, K.Skamris and S.L.Buhl, (2003). ‘How Common and How Large Are Cost Overruns in Transport Infrastructure Projects?’, Transport Reviews, Vol. 23, No. 1, pp. 71-88.

Flyvbjerg, B., M. Holm, K.Skamris and S.L.Buhl, (2004). ‘What Causes Cost Overrun in Transport Infrastructure Projects?’, Transport Reviews, Vol. 24, No. 1, pp. 3-18.

Flyvbjerg, B., M. Holm, K.Skamris and S.L.Buhl, (2005). “How (In)accurate Are Demand Forecasts in Public Works Projects? The Case of Transportation”, Journal of the American Planning Association, Vol. 71, No. 2, pp. 131–46.

Flyvbjerg, B., (2007) Eliminating Bias In Early Project Development through Reference Class Forecasting and Good Governance. Concept Report No.17.

http://www.concept.ntnu.no/Publikasjoner/Rapportserie/Rapport%2017%20kapittelvis/Concept%2017-6%20Reference%20Class%20Forecasting%20and%20Good%20Governance.pdf

Flyvbjerg, B., (2008) “Curbing Optimism Bias and Strategic Misrepresentation in Planning: Reference Class Forecasting in Practice”. European Planning Studies, Vol. 16(1), pp.

http://commonsenseatheism.com/wp-content/uploads/2011/09/Flyvbjerg-Curbing-optimism-bias-and-strategic-misrepresentation-in-planning-reference-class-forecasting-in-practice.pdf

Gillespie, G., P. McGregor, K. Swales and Y. P. Yin (2001). “The Displacement and Multiplier Effects of Regional Assistance: A Computable General Equilibrium Analysis”, Regional Studies, Vol. 35, No. 2, pp. 125-139.

Hahn, R. and P. Dudley (2007). “How Well Does the U.S. Government Do Benefit-Cost Analysis?”, Review of Environmental Economics and Policy, Vol. 1, No. 2, pp. 192-211.

Harrison, M. (2010). Valuing the Future: The Social Discount Rate in Cost-Benefit Analysis. Visiting Researcher Paper, Australian Government Productivity Commission.

http://pc.gov.au/__data/assets/pdf_file/0012/96699/cost-benefit-discount.pdf

HM Treasury (2011). The Green Book: Appraisal and Evaluation in Central Government. London: TSO.

72

Readings

Honohan, P., (1998) Key Issues of Cost-Benefit Methodology for Irish Industrial Policy. ESRI General Research Series Paper. Dublin: ESRI.

Honohan, P. (ed.) (1997). EU Structural Funds in Ireland: A Mid-Term Evaluation of the CSF 1994-1999, Policy Research Series, No. 31, Dublin: The Economic and Social Research Institute.

Juri N., R. and K. Kockelman (2006). “Evaluation of the Trans-Texas Corridor Proposal: Application and Enhancement of the Random-Utility-Based Multiregional Input-Output Model”, Journal of Transport Engineering, Vol. 132, No. 7, pp. 531-539.

Layard R. and S. Glaister (1994) Cost-Benefit Analysis. Cambridge: Cambridge University Press.

Loewenstein, G. and R. Thaler (1989). “Anomalies: Intertemporal Choice”, Journal of Economic Perspectives, Vol. 3, No. 4, pp. 181-193.

Mackett, R.L. and Edwards, M. (1998). ‘The Impact of New Urban Public Transport Systems: Will the Expectations Be Met?’, Transportation Research A, Vol. 32, No. 4, pp. 231-45.

Mishan, E., and E. Quah (2007) Cost Benefit Analysis. London and New York: Routledge

Morgenroth, E. (2013) “How Can We Improve Evaluation Methods for Public Infrastructure?” in Lunn, P., and F. Ruane (eds.) Using Evidence to Inform Policy. Dublin: Gill and Macmillan

Mulreany, M., (2002) Cost Benefit Analysis Readings. Dublin: Institute of Public Administration.

Murphy, A., Walsh, B. And F. Barry (2003) The economic appraisal system for projects seeking support from the industrial development agencies. Forfas

Oosterhavn, J., and T. Knaap (2003) “Spatial Impacts of Transport Infrastructure Investments”, in Pearman, A., Mackie, P. And J. Nellthorp (eds) Transport Projects, Programmes and Policies: Evaluation Needs and Capabilities. Aldershot: Ashgate.

Parthasarathi, P. and D. Levinson (2010). “Post-construction evaluation of traffic forecast accuracy," Transport Policy, Vol. 17, No. 6, pp. 428-443.

Pearce, D., Atkinson, G., and S. Mourato (2006) Cost-Benefit Analysis and the Environment: Recent Developments. Paris: OECD.

Pickrell, D.H. (1990). “Urban Rail Transit Projects: Forecast versus Actual Ridership and Costs”, US ‘Department of Transportation.

73

Readings

Pritchett, L. (1996) “Mind Your P’s and Q’s: The Value of Infrastructure is not Equal to its Cost”, World Bank Working Paper No. 1660.

Pritchett, L. (2002) “It Pays to be Ignorant: A Simple Political Economy of Rigorous Program Evaluation”, Policy reform, Vol. 5(4), pp. 251-269.

Roeger, W. (1996). Macroeconomic Evaluation of the Effects of Community Structural Funds with QUEST II. Mimeo, European Commission, DG-ECFIN.

http://ec.europa.eu/regional_policy/sources/docconf/eva/download/roeger.pdf

http://www.rug.nl/staff/j.oosterhaven/transtalk03_raem_zzl.pdf

Ramsey, F. (1928). “A Mathematical Theory of Saving”, Economic Journal, Vol. 38, pp. 543-559.

Sah, R. K. and J. Stiglitz (1985) "The social cost of labor and project evaluation: A general approach," Journal of Public Economics, Vol. 28(2), pp. 135-163

Stiglitz, J. (1994). “Discount Rates: The Rate of Discount for Cost-Benefit Analysis and the Theory of the Second Best” in R. Layard and S. Glaister (eds.), Cost-Benefit Analysis, Cambridge: Cambridge University Press.

Thaler, R. (1981). “Some Empirical Evidence on Dynamic Inconsistency”, Economics Letters, Vol. 8, pp. 201-207.

Törmä, H. (2008). “Do Small Town Development Projects Matter, and can CGE Help?”, Spatial Economic Analysis, Vol. 3, No. 2, pp. 247-268.

Van Ewijk, C. and P. Tang (2003). “How to Price Risk In Public Investment”, De Economist, Vol.151, No. 3, pp.317-328.

Van Wee, B. (2012). “How Suitable is CBE for the Ex-Ante Evaluation of Transport Projects: A Discussion from the Perspective of Ethics” Transport Policy, Vol. 19, pp. 1-7.

Venables, A., and Gasiorek (1997) Evaluating Regional Infrastructure: A Computable Equilibrium Approach

Vassallo, J. (2010). “The Role of the Discount Rate in Tendering Highway Concessions under the LPVR Approach”, Transportation Research Part A, Vol. 44, pp. 806-814.

Viscusi, K., J. Huber and J. Bell (2008).“Estimating Discount Rates for Environmental Quality from Utility-based Choice Experiments”, Journal of Risk and Uncertainty, Vol. 37, pp. 199-220.

74

Readings

Weitzman, M. (1998). “Why the Far-Distant Future Should be Discounted at its Lowest Possible Rate”, Journal of Environmental Economics and Management, Vol. 36, No. 3, pp. 201-208.

Weitzman, M. (2001). “Gamma Discounting”, American Economic Review, Vol. 91, No. 1, pp. 260-271.

Weitzman, M. (2010). “Risk-adjusted Gamma Discounting”, Journal of Environmental Economics and Management, Vol. 60, No. 1, pp. 1-13.

75