EUROPEAN COMMISSION Directorate-General Regional...

EUROPEAN COMMISSION Directorate-General Regional Policy

Ex post evaluation of cohesion policy interventions 2000-2006 financed by the Cohesion Fund (including former ISPA) Work Package C - Cost benefit analysis of environment projects Final report

May 2011

http://www.tecninvest.com/index_e

http://www.tecninvest.com/index_e

Ex post evaluation of cohesions policy interventions 2000-2006 - Work Package C - Cost benefit analysis of environment projects

Final report - May 2011 .

Disclaimer: The views expressed in this study are those of the authors and do not necessarily reflect the views of the European Commission or of its ser-vices. This report merely acts as a guideline document for policy mak-ers and does not seek to establish the Commission’s future policy in this area. Reproduction or translation is permitted, provided that the source is duly acknowledged and no modifications to the text are made.



Acknowledgement

COWI A/S and partners Csil and Tecninvest were commissioned by the Euro-pean Commission, DG Regio to carry out the study.

We would like to thank the national authorities, project owners, project opera-tors and other stakeholders from the projects for their kind cooperation and valuable contributions throughout the project. We would also like to thank the national desk officers for their competent assistance, as well as the scientific advisors and DG Regio for fruitful and inspiring cooperation.

The study team included the following persons.

COWI A/S: Birgitte Holt Andersen (project manager), Szabolsc Szekeres, Christina van Breugel, Peter G Madsen, Birgitte Grønbech, Niels Erik Houe, Ole Dalgaard and Tine S Skyggebjerg

Csil: Silvia Vignetti and Davide Sartori

TecInvest: Antonio Coimbra

Scientific advisors: Prof. Giles Atkinsons, The London School of Economics and Political Science, Department of Geography and Environment UK Prof. Runar Brannlund, Economics, University of Umeaa, Sweden

Prof. Sándor Kerekes, vice rector responsible for academic affairs of Corvinus University Budapest.



Table of Contents

Executive summary 7

Rapport de synthèse 17

Zusammenfassung 28

1 Introduction 40

2 CBA method considerations 42

2.1 CBAs applied to waste and water projects 42

2.2 Quantification of environmental benefits and externalities 45

2.3 The impact of regulations 46

2.4 Special considerations of Ex-Post evaluations 46

2.5 Margins of error of the CBAs performed. 48

3 Study methodology 49

3.1 Study objectives 49

3.2 Study tasks 50

3.3 Selection of projects 51

3.4 Information collection approach for case studies 54

3.5 Steps of applying the CBA tool 55

3.6 Introduction to the ten selected projects 56

4 Results of ex-post assessment 58

4.1 Results of ex ante analysis 58

4.2 Ex-post assessments 62

4.3 Comparison of ex post with ex ante results 67

4.4 Wider project outcomes 67

4.5 Typical components, benefit items per type of project and examples of calculation externalities 70

5 CBA as a method 79

5.1 Strengths and weaknesses of the CBA methodologies applied by Member States 79



5.2 Effectiveness of CBA as a tool supporting project generation and project decision of MS and EC 81

5.3 Utility of ex post CBA from the point of view of project promoters, MS and EC 85

5.4 The appropriateness of ex post CBA to evaluate impact 86

5.5 Practical treatment of intangible environmental costs and benefit and how they can be included as part of the decision making 89

5.6 Relevance and potential utility of data obtained from CBA in macro-economic modelling 90

6 Main findings and recommendations 92

6.1 What were the impacts of the projects investigated? 92

6.2 What have we learned that might improve the practice of ex ante CBA? 94

6.3 How useful is CBA for ex post impact assessment? 94

Table of Appendices

Appendix I Fact sheets 10 projects

Appendix II Literature

Ex post evaluation of cohesion interventions 2000-2006, WP C

Final report - May 2011

7

.

Executive summary A consortium consisting of COWI A/S, Tecninvest (PT) and Csil (IT) with COWI A/S as lead has completed the present study: "Ex post evaluation of co-hesion policy interventions 2000-2006 financed by the Cohesion Fund (includ-ing former ISPA) - Work Package C - Cost benefit analysis of selected envi-ronment projects" (Reference: REGIO.C.4/JS D (2009) 680291). The study was further supported by three independent, external cost benefit analysis (CBA) experts1. The study was initiated in January 2010 and concluded in April 2011.

The current study set out to investigate the following three main fundamental questions:

• What were the impacts of the examined project?

• How can ex post cost-benefit analysis contribute to the practice of ex ante cost-benefit analysis?

• What are the potential and limits of carrying out ex post cost-benefit analy-sis to identify and/or analyse the impacts of projects? Is it an appropriate tool for impact analysis?

From an initial list of 40 major environmental Cohesion Fund (CF) projects, completed or nearly completed, 20 projects were selected mainly among al-ready completed projects that represented a good mix of the three types of pro-jects (waste management, wastewater treatment and water management) and a fair distribution of beneficiary countries. Each of these twenty projects was briefly described based on available data from DG REGIO and data available on project websites or from other media. The final ten projects were selected among the 20 projects; again the selection was based on the quality of the mate-rial available combined with the requirement to secure a good mix of projects types and sufficient geographical spread.

The ten selected projects were thoroughly analysed starting with the original CBA (the ex ante CBA) to understand the assumptions behind the financial

1 Professor Giles Atkinsons, The London School of Economics and Political Science, De-partment of Geography and Environment UK. Professor Sándor Kerekes, vice rector re-sponsible for academic affairs of Corvinus University Budapest. Prof essor Runar Brannlund, Economics, University of Umeaa, Sweden

Study aim

Selection of projects to be investigated

The ten selected pro-jects



8

.

analysis, the option analysis, the demand analysis and the economic analysis. The context of the project was also investigated, and in particular the degree to which alternative solutions were considered during project formulation. Sec-ondly, the ex post CBA was conducted, including a reality check on the original assumptions, using operational data for the financial analysis, e.g. the actual operational costs and the actual revenues, and finally an assessment of the eco-nomic analysis was made, including identification of the positive as well as negative alternative. Ultimately, the ex ante (financial and economic analysis) and ex post (financial and economic analysis) were compared.

Figure 0-1 Groupings of the ten selected projects

The ten projects investigated are pictured above. Four projects concern waste management, two wastewater projects, two water management projects and two are combined wastewater and water management projects.

The basis for providing a solid analysis clearly relies on the avail-ability and accessibility of reliable data. Our ap-proach to data

Data sources and data collection ap-proach



9

.

collection is briefly summarised below.

First available data from the relevant DG REGIO Geographic desks were ob-tained. Based on this and complemented with the above-mentioned brief de-scription of the project, a first perception and understanding of the project was formed. Secondly, and very important, site visits to the projects provided in-depth familiarity with the individual projects. In addition, site visits provided an opportunity to collect additional operational data as well as to gain an under-standing of the wider impacts of the project on the local community and the region. During the subsequent analysis, the dialogue with the project owner, operator or other stakeholders was maintained if additional data or clarifica-tions were needed.

Towards the end of the study, DG REGIO hosted a workshop to discuss the results of the study. Participants included representatives from the projects in-vestigated as well as representatives from national administrations. Feedback and results of the workshop have been included in this final report.

Study results

Against the three fundamental questions, the study has revealed the following:

1. What were the impacts of the examined projects?

In the light of the results of the ex-post analysis, the following can be con-cluded:

Only two projects (wastewater treatment in Zaragoza and two components of the Portuguese case) showed net positive impacts in terms of improved quality of drinking water supplied, improved efficiency in the management of the ser-vice (savings in purification) and environmental effects (health effects). Such net positive results are shown by a positive ENPV and supported by the wide-spread perception of the project promoters, decision-makers and the citizens.

The remaining eight cases all provided more or less negative results due to dif-ferent reasons.

In four cases (Czech Republic, Barcelona, Poland, Dublin) the project imple-mented showed positive impacts in terms of better services delivered to the population and impacts on the natural environment, with wider effects in terms of recreational activities promoted, environmental awareness raised and institu-tional learning. Such positive results were obtained at high costs, however, re-sulting in negative ENPV and B/C ratios lower than 1. There is a general per-ception that some relevant impacts were not captured by the quantitative CBA, but whether these would suffice to alter the balance of the quantification ob-tained is unknown.

In four cases (Hungary, Bulgaria, Madrid, Crete), the results of the CBA and the perception of the citizens and stakeholders support the argument that the

Workshop with Member States (MS)

Positive net impacts for some but not all projects...



10

.

projects under assessment produced a stream of benefits which, however, did not outweigh costs. Reasons for that are to be found in technical problems of overcapacity (Madrid), problems with demand forecasts (Crete) or technical solutions to comply with environmental standards deemed too costly for the actual needs (Hungary and Bulgaria). Hence, the mere compliance with the en-vironmental directives do not necessarily lead to evidenced positive impacts in net welfare terms.

As just mentioned, the results of the CBAs carried out in this project were posi-tive as well as negative NPVs. The result of the NPV should be interpreted as the sum of the positive (income/revenue) and negative (cost) effects of the pro-ject over time (cash flow). When a project has a positive NPV, it means that it is a solid business case and that private companies should be willing to invest in the project as a profit is expected over time. When a negative NPV occurs, it is related to the fact that project costs exceed project income over time. This is likely to happen in projects where rather big initial investments are made and the expected incomes turn out to be much lower. When the loss of income ex-ceeds the expected profit of the project, the NPV will become negative. In this context the NPV is calculated both with and without Cohesion funding. If the result is very positive without Cohesion funding, one should consider whether funding is necessary for the project. On the other hand, a negative NPV does not mean that the project is not a good investment for DG REGIO, however, it would be advisable to consider the sustainability of the project.

The B/C ratio expresses the division between the benefits and the costs related to the project. A B/C ratio above one indicates that the benefits exceed the costs related to the project and that the projects provide positive net benefits to that society from a socio economic view. When the B/C is lower than one, which is the case with some of the projects included in this analysis, it means that the project costs more than it provides through benefits to society. Often, it can be very challenging to capture and to monetise benefits, which implies that some are left out, not accurately valuated or only qualitatively described. When a pro-ject is a part of a master plan, it can be difficult to isolate the benefits of one intervention and include this in a CBA. This can lead to underestimation or even that a benefit is left out.

Based on this insight, it is important to view the projects from a broader per-spective and not just evaluate them based on the NPV and B/C ratios.

Some of the projects in the sample were undertaken to comply with European regulatory requirements and in all cases to provide or enhance basic infrastruc-ture. However, in most cases this was achieved at costs that exceeded the bene-fits that could be quantified. It is conceivable that in some cases better evaluation methods might have re-sulted in the identification of benefits that would have contributed to attaining a positive balance, but this is unlikely to hold true in all cases.

Given that CBA was only undertaken at the very end of the project planning process, it is not surprising that some of the projects have negative ENPVs. They were not designed from the outset to attain positive ENPVs.

Interpretation of the NPV and B/C ratios

...with merely legal compliance not nec-essarily leading to positive impacts in economic welfare terms.



11

.

In the case of projects that were undertaken to meet regulatory requirements, it might be impossible to achieve a positive ENPV. In such cases the CBA result shows the net welfare loss caused by regulatory compliance. (In some cases, the regulation provides for exemptions when the cost of compliance is high, but this was not invoked in any of the cases analysed.)

Of the four waste management projects, the Bulgarian and Crete projects both constitute a first step towards creating a modern waste management handling infrastructure. In these two cases, the waste was not treated but simply disposed of into illegal dumping sites causing groundwater contamination and other in-appropriate implications such as non-pleasant odours, negative aesthetics, inap-propriate land use, bad image, etc. In the case of the Madrid and Portuguese waste projects, the situation is slightly different.

In the Madrid case, the project concerned sealing off an old illegal landfill that had accumulated around 22 million m3 of solid waste produced by Madrid's three million citizens over the last two decades covering an area of 110 hec-tares. At the same time, the project has transformed landfill gas into green elec-tricity producing a total of 1,925 million kWh worth of energy until 2017. The area of the old illegal landfill has been restored and replanted with trees, ponds have been established to the benefit of foraging birds, and by 2025 the area can be used for recreational purposes. The Portuguese waste project is yet another story as it concerns upgrading of an existing but outdated waste management system.

The wastewater management projects include the Dublin case and the Hungar-ian case. Wastewater components are also included in the Plzen and Polish cases. The Dublin case is an extraordinary story because it turned out that the project planners significantly underestimated the demand for wastewater treat-ment. With Ireland being a 'Celtic tiger', Dublin experienced a period of rapid economic growth between 1995 and 2007 - which however came to a dramatic halt in 2008. At the same time, the population of Dublin grew to unexpected heights. Hence, already from the start, the Dublin WWTS suffered from over-loading. Furthermore, due to the designation of the Liffey Estuary as sensitive water, there is a need for new infrastructure developments to comply with stricter discharge standards, and so an expansion of the wastewater treatment facilities is in process.

However, the Dublin Bay now enjoys a substantially cleaner water environ-ment. A proof that the water in the Dublin Bay has become usable for bathing and other water recreational use is that four beaches now comply with the water quality criteria of the Blue Flag Programme. Furthermore, it appears that a sub-stantial number of housing permissions has been granted as new localities have been connected to the wastewater treatment system.

The Hungarian Szeged wastewater project is an excellent example of an inter-vention that, in line with nearly all the other nine projects, was initiated to comply with the Waste Water Directive and involved the construction of a WWTP and upgrading the sewer network completely according to plan. How-ever, the project is still not delivering the full benefits of the investment in

The waste projects have replaced illegal waste disposals with modern waste man-agement…

….and turning past sins into future envi-ronmental activi-ties…

The impacts of waste water projects are very diverse.

…recreational bene-fits to the Dubliners due to a cleaner Dublin Bay

...unrealised benefits due to dilution ef-fects of the River Tisza



12

.

terms of improved water quality of the River Tisza. The reason being that the outflows upstream (coming from Romania) are not treated to the same standard as the wastewater generated from the City of Szeged.

Both the Plzen case and the Polish Szczecin case are combined water manage-ment and wastewater projects. Both projects involve upgrading of existing, but outworn and inadequate infrastructure. In the Plzen case, 1,400 households are now connected to the sewer system, and the city water supply is reliable and of good quality. For a number of citizens, tap water was simply not fit to drink before the project, and citizens were relying on bottled water for consumption of water. Thanks to the construction of two retention tanks, the number of over-flows into the River Berounka has been reduced eliminating one of the main sources of pollution of the river. A cleaner river has, in turn, resulted crayfish to return to the river, people have started swimming in the river, beaches have been created and a number of projects are being implemented to restore river banks and create bicycle and walking paths along the river, e.g. Green Ways project.

The majority of the wastewater collection system in Szczecin was more than 60 years old, and some of it still suffered from damage inflicted during the Second World War. The wastewater treatment system relied on an obsolete technology of primary wastewater treatment causing polluted water to be directly dis-charged into the River Oder. This meant that Szczecin was on the hot spot list established under the Helsinki Convention - which is no longer the case with the new infrastructure. Furthermore, the users of the water systems experienced that the services were of a low quality. These problems are now solved.

The Barcelona and the Zaragoza cases are the only non-compliance driven pro-jects. The Barcelona project is about reducing the risk of flooding while the Zaragoza project involve building a 70 km long pipeline from the mountain down to Zaragoza to ensure fresh water to benefit the 821 thousand citizens in Zaragoza which now can rely on tapped water for their water demand to replace the purchase of bottled water. The project has also made the Municipality take other measures to reduce water consumption. In the Barcelona case, the main impacts are a reduced risk of flooding and beaches that do not have stay closed for tourists for some days following flooding.

In addition to the above-mentioned positive results, almost all the projects gen-erated positive side effects in terms of environmental awareness (e.g. change of attitudes in the wider public, e.g. more efficient use of water – the Zaragoza case - or initiating waste sorting) and institutional learning (the capacity of the public administration to deal with EU projects – the EU unit in the Czech case – or to manage complex and technologically innovative infrastructure – the Barcelona case).

Embedded in the Cohesion Policy is a distributional goal seeking to bring tan-gible benefits to citizens and regions that are least well off. The projects ana-lysed in the current study do highlight the issue of distributional effects as these projects benefit low-income regions but are indirectly paid by higher income regions (hence the solidarity aim of CF policy). The CBA as a tool does not

…and political bene-fits since Szczecin is no longer on the hot spot list of Helcom

Environmental awareness and insti-tutional learning as common side effects

Distributional effects are difficult to cap-ture in a CBA



13

.

explicitly capture such distributional effects, but some sort of formal considera-tion of the implications for CBA could be considered in the future. Connected to this issue is affordability of the citizens benefitting from the projects. The current report reveals that affordability is an issue for some of the projects, as the local citizens through taxes or user-paid tariffs have to pay for the operation of the environmental service put in place.

2. How can ex post cost-benefit analyses contribute to the practice of ex ante cost-benefit analyses?

This study has contributed to our understanding of how cost-benefit analyses (CBAs) are made in practice, how they are used, and in particular, how they are not used for what they were supposed to be used for in an ante context, namely as the basis for selecting the option with the highest value for money and the lowest risk. Despite this the study shows that CBAs if conducted rightfully can improve project design and solutions.

The study also reveals that there are a few problems with either under-capacity or over-capacity in a few of the projects investigated that might have been bet-ter addressed if a proper demand analysis had been applied.

From a strict CBA point of view, only one of the projects would have deserved funding if we were to respect that the benefit to society at large is to be larger than the cost of investing and operating the infrastructure (B-C ratio >1). This circumstance points to one of the most critical findings of the current study: if the CBA was to be used in the proper and timely way during the selection phase, some re-design or re-consideration of projects which underperformed would have been possible. In some cases this would have implied a different design of projects for which legal compliance was most strict, or could have served as a basis for a derogation request.

In practice, it appears that CBAs are primarily made because it is an application requirement when seeking Cohesion Funding. The CBA is often outsourced to a specialised consulting company and rather than being an integrated part of the decision-making process, it is merely a question of 'ticking the box'.

There is a perception among practitioners that ensuring Cohesion Fund assis-tance, as an MS representative articulated, is: ‘a matter of making the financial analysis look as bad as possible in order to increase the funding need, and to make the economic analysis to look as positive as possible in order to justify the public funding’.

A clear message from the MS Workshop as expressed by several Member States was that although the CBA methodologies are interesting, they often ap-pear to be an academic exercise, while the reality is often very complex and difficult to depict with a set of Excel sheets. It has also been reported that the CBA does not always reflect the current situation as things can change rapidly; for instance new directives coming into place quickly outdate CBA studies. In response, several Member States called for a more simple approach to CBA.

CBA could support in killing the white elephant if applied correctly

The current role of ex ante CBAs is merely a 'ticking the box' exercise….

And Member States show scepticism about the usefulness of CBA…



14

.

Also the MS expressed a wish to have ‘best practice showcases’ that could in-spire the CBA process.

Generally, a lack of a common language and a shared understanding among project promoters, decision-makers at the MS level and the EC about the need and use of the CBA is reported.

Concerning the application process, a need for better 'training' of those involved in reviewing the applications was also expressed and in particular to use a more clear language (non-technical) and to make the process more transparent and to ensure more smooth communication between the beneficiary and EC. Hence, more training at all levels would improve the common understanding behind the use of CBA for project selection.

If the CBA is to have any impact on how the choice between options is made, it is firstly important to focus on the timing of CBA integration. Hence, the CBA thinking has to be an integrated part of the decision-making and it calls for a change in the CBA culture implying that the CBA becomes an integrated part of project selection and evaluation. The main issues preventing CBAs from mainstream usage seem to be rather practical ones, such as lack of data in the case of evaluating environmental impacts and lack of political will and checks in place to implement CBA evaluation at Member State level.

3. What are the potentials and limits of making an ex post cost-benefit analysis to identify and/or analyse the impact of projects? Is it an appropri-ate tool for impact analysis?

Proper ex-post CBAs are useful, because they can serve to assess if the project will generate positive net impacts to society and identify causes of deviation from the ex-ante assumptions and forecast. In this respect, the CBA can be seen as a project management tool, aiming at guiding corrective measures in case significant adjustments are needed in the implementation phase. Although it is not common to use the CBA as an ex post impact assessment tool; this study has done it and has proved that it is indeed useful.

The use of the ex post CBA is relevant as much as the quality and use of the ex-ante CBA is improved. In fact, as long as the ex ante CBA is carried out with a ‘tick the box’ kind of approach, more as an ex post rationalisation of a political decision already taken (therefore highly subject to manipulation of the results), the political willingness to carry out the ex post CBA may be poor and the use very limited.

Project proponents, politicians and so on presumably have little interest in find-ing out that their ex ante decision to implement might be contradicted in the light of ex post events. But that does not mean that it is not worthwhile finding out more about costs and benefits ex post. Given that in the context of the Co-hesion Policy (CP) projects there might be strong incentives towards 'optimism bias', ex post studies of some description surely provide a valuable tool. The report rightly implies that these are possibly sensitive issues. Ultimately, it

…but more training at all levels might add to the CBA cul-ture…

…and there is a need to change the CBA culture…

Full ex-post CBAs are useful

...but the ex-ante CBA is a matter of "ticking the box"

…and only little in-terest locally but in a greater context the knowledge could be beneficial



15

.

would be a desirable goal to be able to show how better economic appraisal more generally (both ex ante and ex post) can lead to better CP project out-comes.

The timing of the ex-post exercise is crucial to understand the use of the ex-post results and the capacity to measure actual impacts. The case studies per-formed for the environmental cases demonstrated that the ex post carried out just after the construction phase are more an update of the ex ante CBA than a true ex post (the flow of benefits have not yet materialised). Three to five years after project completion have been deemed a proper timing for an ex post exer-cise. However, it should be acknowledged that the impacts captured by such an analysis are not the long-term stabilised ones, but only the initial, more short-term ones. Still, the exercise is potentially extremely useful in order to plan cor-rective measures, in case unexpected events or structural changes in the context (e.g. the financial crises influenced some key macroeconomic parameters) have occurred in the meantime. A true ex post CBA needs a longer timescale of at least 10-15 years (depending also on the typologies of investments).

Availability of data and methodological expertise are essential requirements in order to feed into the calculations. For example, it has been stressed that some impacts may be difficult to capture with a CBA, such as for example wider im-pacts or impacts of less traditional typologies of infrastructure (such as ICT or R&D).

Policy recommendations The main policy recommendations as revealed by the current study with focus on where the EC can take lead include sharing of best practice, improved envi-ronmental data, e.g. benefit transfer protocol and primary valuation studies and finally CBA training resources. These are briefly elaborated on below.

Member States expressed an interest in having access to 'good practices of ex ante studies' as examples and learning for beneficiaries in undertaking ex ante CBA for own projects.

Such good practices could be supported by standard parameters for evaluating externalities. As there are 'standard' types of impacts across the same types of environmental infrastructure projects, e.g. methane emission, odours, etc., it might be beneficial to apply the same types of values against these parameters in order to compare across the same types of projects. An on-line catalogue of such parameters available for practitioners might ease the CBA process.

Some externalities and even direct benefits are clearly very difficult to measure, and it could be useful to have access to reliable data instead of having to make surveys each time. This is in particular relevant when trying to assess citizens 'willingness to pay' (WTP), e.g. access to updated EU data on WTP for the dif-ferent types of environmental infrastructure projects.

The study has revealed a need to encourage CBA 'thinking' among practitioners to ensure that the CBA is further integrated into the project definition and deci-sion-making. To develop further a CBA culture, more CBA training at all lev-

The timing of the ex-post exercise is cru-cial

Share good practice of ex ante CBA…

...including access to shared resources on parameters, e.g. common values on calculating external-ities

…and even CBA training resources



16

.

els might be desirable. In addition to the CBA guidelines, access to common CBA resources, e.g. the common parameters, development of CBA training courses or even eLearning resources should be considered.



17

.

Rapport de synthèse Un consortium, composé de COWI A/S, Tecninvest (PT) et Csil (IT) et cha-peauté par COWI A/S, a réalisé l’étude suivante: «Ex post evaluation of cohe-sion policy interventions 2000-2006 financed by the Cohesion Fund (including former ISPA) - Work Package C - Cost benefit analysis of selected environ-ment projects» (référence: REGIO.C.4/JS D (2009) 680291). Cette étude a en outre bénéficié du soutien de trois experts externes indépendants en analyses coûts/avantages (ACA)2. Entamée en janvier 2010, l’étude s’est terminée en avril 2011.

La présente étude cherche à répondre aux trois principales questions fondamen-tales suivantes:

• Quels sont les impacts des projets examinés?

• Comment les analyses coûts/avantages ex post peuvent-elles contribuer à la pratique des analyses coûts/avantages ex ante?

• Quels sont le potentiel et les limites liés à la réalisation d’analyses coûts/avantages ex post en vue d’identifier et/ou d’analyser l’impact des projets? Ces analyses constituent-elles un outil adéquat pour l’analyse des impacts?

Sur un total de 40 grands projets environnementaux partiellement financés par le Fonds de cohésion, terminés ou presque, 20 projets ont été sélectionnés, principalement parmi les projets terminés. Les projets sélectionnés reflètent un bon mélange des trois types de projets (gestion des déchets, traitement des eaux usées et gestion de l’eau) et une répartition adéquate des pays bénéficiaires. Chacun des projets a été brièvement décrit sur la base des données disponibles auprès de la DG REGIO et sur les sites web des projets ou d’autres médias. Dix projets ont finalement été sélectionnés parmi ceux-ci. Encore une fois, les pro-jets ont été sélectionnés sur la base de la qualité du matériel disponible et dans un souci de garantir une bonne représentation des types de projets et une distri-bution géographique suffisante.

2 Professeur Giles Atkinsons, London School of Economics and Political Science, Dépar-tement de géographie et d’environnement, Royaume-Uni. Professeur Sándor Kerekes, vice-recteur en charge des affaires académiques de l’université Corvinus de Budapest. Pro-fesseur Runar Brannlund, Économie, université d’Umeaa, Suède.

Objectif de l’étude

Sélection des projets à examiner



18

.

Nous avons analysé en profondeur les dix projets sélectionnés, en commençant par l’analyse ACA originale (ACA ex ante), afin de bien comprendre les hypo-thèses à la base de l’analyse financière, de l’analyse des options, de l’analyse de la demande et de l’analyse économique. Nous avons également examiné le contexte des projets et déterminé notamment dans quelle mesure d’autres solu-tions ont été envisagées durant la phase de formulation. Ensuite, nous avons procédé à l’analyse ACA ex post. Nous avons, entre autres, comparé les hypo-thèses originales par rapport à la réalité, au moyen des données opérationnelles de l’analyse financière (par exemple, les coûts opérationnels réels et les recettes réelles). Nous avons évalué l’analyse économique et notamment identifié les alternatives positives et négatives. Enfin, nous avons comparé les analyses ex ante (financière et économique) et ex post (financière et économique).

Figure 01

Les dix projets étudiés sont représentés ci-dessus. Quatre d’entre eux portent sur la gestion des déchets, deux sur le traitement des eaux usées et deux sur la gestion de l’eau. Les deux derniers concernent à la fois le traitement des eaux usées et la gestion de l’eau.

Une analyse bien fondée dépend de toute évidence de l’accessibilité et de la disponibilité de données fiables.

Les dix projets sélec-tionnés

Sources de données et approche de la récolte des données



19

.

Notre approche de la récolte des données est brièvement résumée ci-après.

Tout d’abord, nous avons récolté les données disponibles auprès des bureaux pertinents des délégations de la DG REGIO. Sur cette base et à partir de la brève description ci-dessus des projets, nous nous sommes fait une première idée de ceux-ci. Ensuite, nous avons réalisé des visites clés sur le site des pro-jets. Celles-ci nous ont permis de nous familiariser avec chacun d’entre eux. En outre, ces visites ont été pour nous l’occasion de récolter des données opéra-tionnelles supplémentaires et de comprendre les impacts plus vastes des projets sur les communautés locales et les régions. Durant l’analyse subséquente, nous sommes restés en dialogue avec les détenteurs des projets, les opérateurs et d’autres parties prenantes dans la mesure où nous avions besoin de données complémentaires et souhaitions clarifier certains points.

Vers la fin de l’étude, la DG REGIO a organisé un atelier en vue de débattre des résultats. Des représentants des projets étudiés et des autorités nationales faisaient partie des participants. Les avis obtenus et les résultats de l’atelier sont inclus dans ce rapport final.

Résultats de l’étude

S’agissant des trois questions fondamentales, l’étude a révélé ce qui suit:

1. Quels sont les impacts des projets examinés?

À la lumière des résultats des analyses ex post, nous pouvons tirer les conclu-sions suivantes:

Seuls deux projets (traitement des eaux usées à Saragosse et deux volets du pro-jet portugais) ont produit des impacts positifs nets en termes d’une meilleure qualité de l’eau potable fournie, d’une plus grande efficacité de la gestion du service (économies au niveau de l’épuration) et d’effets environnementaux (ef-fets sur la santé). De tels résultats positifs nets sont indiqués par une valeur ac-tuelle nette économique (VANE) positive et appuyés par les vues générales des promoteurs des projets, des décisionnaires et des citoyens.

Les huit projets restants ont tous produit des résultats plus ou moins négatifs pour différentes raisons.

Dans le cas de quatre d’entre eux (République tchèque, Barcelone, Pologne et Dublin), le projet mis en œuvre a produit des impacts positifs en termes d’amélioration des services à la population ainsi que des impacts sur l’environnement naturel, avec des effets plus larges en termes d’activités ré-créatives promues, de sensibilisation à l’environnement et d’apprentissage au sein des institutions. Toutefois, ces résultats positifs ont été obtenus à un coût élevé, entraînant une VANE négative et des rapports A/C inférieurs à 1. De manière générale, certains impacts pertinents ne semblent pas avoir été identi-fiés par l’analyse ACA quantitative. Toutefois, il est impossible de savoir si ces impacts suffiraient à modifier l’équilibre de la quantification obtenue.

Organisation d’un atelier avec les États membres

Impacts positifs nets pour certains pro-jets, mais pas pour tous…



20

.

Dans quatre des projets (Hongrie, Bulgarie, Madrid et Crête), les résultats de l’analyse ACA et les vues des citoyens et des parties prenantes corroborent la thèse selon laquelle les projets évalués ont produit une série d’avantages. Néanmoins, ceux-ci ne compensent pas les coûts pris en charge. Ce déséquili-bre est attribuable à des problèmes techniques de surcapacité (Madrid), à des problèmes liés aux prévisions de la demande (Crête) ou à la mise en œuvre de solutions techniques visant à respecter les normes environnementales mais ju-gées trop onéreuses par rapport aux besoins réels (Hongrie et Bulgarie). Par conséquent, le simple respect des directives environnementales ne débouche pas nécessairement sur des impacts positifs nets en termes de bien-être.

Comme il a été dit, les analyses ACA réalisées dans le cadre de cette étude ont révélé des valeurs actuelles nettes (VAN) aussi bien positives que négatives. Le résultat de la VAN doit être interprété comme la somme des effets positifs (re-venus/recettes) et négatifs (coûts) des projets dans le temps (cash-flow). Lors-qu’un projet affiche une VAN positive, cela indique que le projet est solide-ment fondé et que des entreprises privées devraient être disposées à investir dans celui-ci dans la perspective d’engranger des bénéfices à terme. En revan-che, une VAN négative est liée au fait que les coûts du projet dépassent les re-venus à terme. Cela est susceptible de se produire dans le cadre de projets exi-geant des investissements initiaux relativement importants et pour lesquels les revenus attendus s’avèrent largement inférieurs. Lorsque la perte de revenus est supérieure aux bénéfices attendus du projet, la VAN est alors négative. À cet égard, la VAN est calculée à la fois avec et sans le financement du Fonds de cohésion. Dans la mesure où le résultat s’avère très positif sans l’aide du Fonds de cohésion, il convient d’envisager si ce financement est bien nécessaire pour le projet. Par contre, si une VAN négative ne signifie nécessairement pas qu’un projet constitue un mauvais investissement pour la DG REGIO, il est néan-moins conseillé de s’interroger sur la viabilité du projet en question.

Le rapport A/C exprime le rapport entre les avantages et les coûts liés à un pro-jet. Un rapport A/C supérieur à 1 indique que les avantages dépassent les coûts liés au projet et que celui-ci offre à la société des avantages positifs nets d’un point de vue socio-économique. En revanche, lorsque le rapport A/C est infé-rieur à 1 (ce qui est le cas pour certains projets inclus dans la présente analyse), cela signifie que les coûts du projet sont supérieurs aux avantages procurés à la société. Il s’avère souvent difficile d’identifier les avantages et de leur attribuer une valeur monétaire: certains sont alors omis, évalués de manière imprécise ou uniquement décrits en termes qualitatifs. Lorsqu’un projet fait partie d’un plan directeur, il n’est pas toujours facile d’isoler les avantages liés à une interven-tion spécifique et de les inclure dans une analyse ACA. Cela peut entraîner une sous-estimation, voire l’omission d’un avantage.

À la lumière de ce qui précède, il convient donc d’envisager les projets sous un angle plus large, et pas seulement de les évaluer à partir de la VAN et des rap-ports A/C.

Interprétation de la valeur actuelle nette et des rapports A/C



21

.

Certains projets de l’échantillon sélectionné ont été entrepris en vue de respec-ter les exigences réglementaires européennes et, dans tous les cas, de fournir ou d’améliorer l’infrastructure de base. Toutefois, le plus souvent, ces résultats ont été obtenus moyennant des coûts supérieurs aux avantages quantifiables.

Il est concevable que, dans certains cas, de meilleures méthodes d’évaluation eussent permis d’identifier des avantages qui auraient contribués à atteindre un équilibre positif. Cependant, cela n’est pas susceptible d’être vrai dans tous les cas.

Les analyses ACA n’ayant été entreprises qu’à la fin du processus de planifica-tion des projets, il n’est pas surprenant de constater que certains d’entre eux présentent des VANE négatives: à la base, ils n’ont pas été conçus pour attein-dre des VANE positives.

Dans le cas des projets entrepris en vue de répondre aux exigences réglementai-res, une VANE positive pourrait s’avérer impossible à atteindre. Le cas échéant, les résultats de l’analyse ACA montrent une perte de bien-être nette, due à la mise en conformité avec les exigences réglementaires (dans certains cas, la réglementation prévoit des cas d’exemption lorsque le coût de mise en conformité est élevé; toutefois, cela n’a été invoqué dans aucun des projets ana-lysés).

Sur les quatre projets consacrés à la gestion des déchets, les projets bulgare et crétois constituent tous deux un premier pas vers la création d’une infrastruc-ture moderne de gestion des déchets. Dans ces deux cas, les déchets n’étaient pas traités, mais simplement éliminés auprès de décharges illégales, entraînant une contamination des eaux souterraines et d’autres répercussions telles que des odeurs désagréables, un manque d’esthétique, une utilisation inappropriée du sol et une mauvaise image. Dans le cas des projets de Madrid et du Portugal, la situation est légèrement différente.

Dans le cas de Madrid, le projet portait sur la fermeture d’une ancienne dé-charge illégale, qui avait accumulé, sur une zone de 110 hectares, quelque 22 millions m3 de déchets solides produits par les trois millions de Madrilènes ces deux dernières décennies. Parallèlement, le projet a permis de transformer les gaz de la décharge en électricité verte, produisant un total de 1 925 millions kWh d’énergie jusqu’en 2017. La zone de l’ancienne décharge illégale a été réhabilitée et des arbres ont été replantés. Des étangs ont été aménagés au profit des oiseaux. D’ici 2025, la zone pourra être utilisée à des fins récréatives. Le projet portugais porte, quant à lui, sur la mise à niveau d’un système de gestion des déchets existant devenu obsolète.

Les projets de gestion des eaux usées incluent un projet à Dublin et un autre en Hongrie. Le projet de Plzen et le projet polonais comprennent également des volets consacrés au traitement des eaux usées. Le projet de Dublin est l’histoire d’un projet extraordinaire: il s’est avéré que les planificateurs avaient considé-rablement sous-estimé la demande en traitement des eaux usées. L’Irlande étant un «tigre celtique», Dublin a connu une période de rapide croissance économi-que entre 1995 et 2007, une croissance qui s’est brutalement arrêtée en 2008.

…Le simple respect des exigences légales ne débouche pas né-cessairement sur des impacts positifs en termes de bien-être économique

Les projets entrepris dans le domaine des déchets ont remplacé des mises en dé-charge illégales par une gestion moderne des déchets…

…Conversion des erreurs passées en activités environne-mentales d’avenir…

Les impacts des pro-jets consacrés aux eaux usées sont très diversifiés.



22

.

Dans le même temps, la population de Dublin a atteint des sommets imprévus. C’est pourquoi, dès le début, la station d’épuration de Dublin a souffert d’une surcharge. Par ailleurs, l’estuaire de la Liffey ayant été désigné «eaux sensi-bles», de nouveaux développements de l’infrastructure s’imposaient afin de respecter les normes plus rigoureuses en vigueur concernant le rejet des eaux usées. C’est pourquoi une expansion de la station d’épuration est en cours.

Quoi qu’il en soit, les eaux de la baie de Dublin sont aujourd’hui beaucoup plus propres. Le fait que quatre plages répondent aux critères de qualité des eaux du programme Pavillon Bleu est la preuve qu’il est désormais possible de se bai-gner dans les eaux de la baie et d’y exercer d’autres activités récréatives. En outre, il s’avère qu’un grand nombre de permis de construire ont été octroyés à mesure que de nouvelles localités ont été rattachées au système de traitement des eaux usées.

En Hongrie, le projet Szeged consacré aux eaux usées est le parfait exemple d’une intervention qui, comme pratiquement tous les autres projets, a été mise en œuvre en vue de respecter la directive sur les eaux usées et impliquait la construction d’une station d’épuration et une mise à niveau complète du réseau d’assainissement conformément au plan défini. Toutefois, le projet n’a pas en-core produit tous les avantages liés à l’investissement en termes d’amélioration de la qualité de l’eau de la rivière Tisza. La raison en est que les écoulements en amont (provenant de Roumanie) ne sont pas traités selon les mêmes normes que les eaux usées produites par la ville de Szeged.

Les projets de Plzen et de Szczecin combinent tous deux des projets de gestion de l’eau et de traitement des eaux usées. Ils impliquent une mise à niveau de l’infrastructure existante, obsolète et inappropriée. Dans le cas de Plzen, 1 400 ménages sont désormais connectés au réseau d’assainissement. L’approvisionnement de la ville en eau est fiable et de bonne qualité. Pour un certain nombre de citoyens, l’eau du robinet n’était pas potable antérieurement au projet et ceux-ci consommaient donc de l’eau en bouteille. Grâce à la cons-truction de deux bassins de retenue, le nombre de débordements dans la rivière Berounka a été réduit, éliminant ainsi l’une des principales sources de pollution de la rivière. À son tour, la meilleure qualité des eaux de la rivière a permis le retour de l’écrevisse, les citoyens se sont mis à nager dans la rivière, des plages ont été aménagées et des projets ont été mis en œuvre en vue de restaurer les rives et de créer des pistes cyclables et des sentiers le long de la rivière (par exemple, le projet Green Ways).

En majorité, le système de collecte des eaux usées de Szczecin avait plus de 60 ans. Une certaine partie de ce système souffrait encore des dommages subis durant la Seconde Guerre mondiale. Le système reposait sur une technologie obsolète de traitement primaire des eaux usées, suite à laquelle de l’eau polluée était directement déversée dans l’Oder. Dès lors, Szczecin figurait sur la liste des points chauds établie dans le cadre de la Convention d’Helsinki. Grâce à la nouvelle infrastructure, le nom de la ville a été supprimé de cette liste. De plus, les services fournis aux utilisateurs des systèmes de traitement des eaux étaient de mauvaise qualité. Ces problèmes sont désormais résolus.

…Avantages récréa-tifs pour les Dubli-nois grâce à l’épuration des eaux de la baie de Dublin

…Avantages non ré-alisés suite aux effets de dilution de la ri-vière Tisza

…Avantages politi-ques: Szczecin ne figure plus sur la liste des points chauds de l’Helcom



23

.

Les projets de Barcelone et de Saragosse sont les seuls qui ne visent pas une mise en conformité. Le projet de Barcelone consiste à réduire le risque d’inondation tandis que celui de Saragosse porte sur la construction d’une conduite, longue de 70 km, descendant de la montagne jusqu’à la ville de Sara-gosse, en vue de garantir l’approvisionnement en eau potable des 821 000 habi-tants, qui peuvent désormais boire l’eau du robinet plutôt que de l’eau en bou-teille. Le projet a également incité les autorités municipales à prendre d’autres mesures afin de limiter la consommation d’eau. Dans le cas du projet de Barce-lone, les principaux impacts sont une réduction du risque d’inondation et la possibilité de garder les plages ouvertes aux touristes, celles-ci ne devant plus être fermées pendant plusieurs jours à la suite d’inondations.

Outre les résultats positifs indiqués ci-dessus, pratiquement tous les projets ont produit des effets secondaires favorables en termes de sensibilisation à l’environnement (par exemple, changement d’attitude du grand public, meil-leure exploitation de l’eau dans le cas de Saragosse et mise en place d’un sys-tème de triage des déchets) et d’apprentissage au sein des institutions (par exemple, capacité des pouvoirs publics à prendre en charge des projets euro-péens [unité européenne dans le cas du projet tchèque] ou à gérer une infras-tructure complexe et technologiquement novatrice dans le cas de Barcelone).

La politique de cohésion intègre un objectif distributif, qui cherche à ce que les citoyens et les régions les moins favorisés puissent profiter d’avantages tangi-bles. Les projets analysés dans le cadre de l’étude mettent en avant la dimen-sion des effets distributifs puisque ces projets profitent à des régions à faibles revenus tout en étant indirectement financés par des régions à plus hauts reve-nus (comme le veut l’objectif de solidarité de la politique du Fonds de cohé-sion). En tant qu’outil, l’analyse ACA ne permet pas d’identifier explicitement de tels effets distributifs, mais une certaine prise en compte formelle des impli-cations pour l’analyse ACA pourrait être envisagée à l’avenir. Cette dimension rejoint la question de l’abordabilité pour les citoyens profitant des projets. Le présent rapport indique que l’abordabilité constitue un problème dans le cas de certains projets, vu que les citoyens locaux doivent financer le fonctionnement du service environnemental mis en place, par le biais de taxes et de tarifs à la charge des utilisateurs.

2. Comment les analyses coûts/avantages ex post peuvent-elles contribuer à la pratique des analyses coûts/avantages ex ante?

Cette étude nous a permis de comprendre comment les analyses ACA sont ré-alisées en pratique, comment elles sont utilisées et surtout comment elles ne sont pas utilisées aux fins auxquelles elles ont été conçues dans le contexte an-térieur, c.-à-d. pour servir de base à la sélection de l’option la plus rentable et la moins risquée. Néanmoins, l’étude montre que les analyses ACA peuvent, dans la mesure où elles sont correctement effectuées, améliorer la conception des projets et les solutions.

En outre, l’étude révèle l’existence de quelques problèmes liés à la sous-capacité ou à la surcapacité dans le cadre d’un petit nombre des projets exami-

Sensibilisation à l’environnement et apprentissage au sein des institutions: des effets secondai-res courants

Les effets distributifs sont difficiles à iden-tifier dans une ana-lyse ACA



24

.

nés. Ces problèmes auraient peut-être pu être abordés plus efficacement si une analyse de la demande avait été correctement appliquée.

D’un point de vue strictement ACA, seul l’un des projets méritait d’être financé si l’on tient compte du principe selon lequel les avantages pour la société doi-vent être supérieurs aux coûts d’investissement et de fonctionnement de l’infrastructure (rapport A/C >1). Cette constatation souligne l’une des princi-pales conclusions de l’étude: si les analyses ACA avaient été appliquées de manière correcte et opportune durant la phase de sélection, il aurait été possible de reconcevoir ou de réenvisager les projets qui n’ont pas fourni les résultats attendus. Dans certains cas, cela aurait permis de concevoir différemment les projets pour lesquels les exigences de conformité légale étaient les plus rigou-reuses ou cela aurait pu servir de base pour la demande d’une dérogation.

En pratique, il s’avère que les analyses ACA sont principalement réalisées parce qu’elles sont exigées dans le cadre des demandes présentées au Fonds de cohésion. Les analyses ACA sont souvent confiées à une société de consultance externe spécialisée, au lieu d’être pleinement intégrées au processus de déci-sion: elles consistent simplement à «cocher des cases».

Aux yeux des professionnels, afin de s’assurer l’appui du Fonds de cohésion, il suffit, pour reprendre les mots d’un représentant des États membres, «de pré-senter une analyse financière la plus mauvaise possible afin d’accroître le be-soin de financement et de présenter une analyse économique la meilleure possi-ble afin de justifier le financement public».

Lors de l’atelier organisé avec les États membres, plusieurs d’entre eux ont in-diqué que bien qu’elles soient intéressantes, les méthodologies ACA ne sont souvent qu’un simple exercice théorique. La réalité est, quant à elle, souvent très complexe et difficile à décrire dans des feuilles de calcul Excel. Il est éga-lement ressorti de l’atelier que les analyses ACA ne reflètent pas toujours la situation réelle, car les choses évoluent rapidement: par exemple, les analyses ACA sont périmées lorsque de nouvelles directives entrent en vigueur. En ré-ponse, plusieurs États membres ont réclamé une approche plus simple des ana-lyses ACA. De plus, ils ont exprimé le souhait de disposer de «cas de pratiques d’excellence» susceptibles d’inspirer le processus ACA.

De manière générale, on constate l’absence d’un langage commun et d’une compréhension commune entre les promoteurs des projets et les décisionnaires à l’échelon des États membres et de l’Union européenne concernant la nécessité et l’application des analyses ACA.

S’agissant du processus de demande, on constate également la nécessité d’une meilleure «formation» des personnes impliquées dans l’examen des demandes, et notamment la nécessité d’utiliser un langage plus clair (non technique), de rendre le processus plus transparent et de faciliter la communication entre le bénéficiaire et l’Union européenne. Ainsi, une meilleure formation à tous les niveaux améliorerait la compréhension commune à la base de l’utilisation des analyses ACA pour la sélection des projets.

Correctement appli-quées, les analyses ACA pourraient contribuer à limiter les gaspillages

Les analyses ACA ex ante actuelles consis-tent simplement à «cocher des cases»…

Les États membres se montrent scepti-ques quant à l’utilité des analyses ACA…

…Une meilleure formation à tous les niveaux pourrait ren-forcer la culture des analyses ACA



25

.

Pour que les analyses ACA aient un impact sur le choix entre différentes op-tions, l’accent doit avant tout être mis sur l’intégration en temps opportun des analyses ACA dans le processus. Ainsi, la pensée ACA doit être intégrée au processus de décision. Cela nécessite un changement au niveau de la culture ACA, visant à incorporer les analyses ACA dans les phases de sélection et d’évaluation des projets. Les principaux obstacles à l’utilisation généralisée des analyses ACA semblent avant tout d’ordre pratique, comme le manque de don-nées pour l’évaluation des impacts environnementaux et le manque de volonté politique et de contrôle pour la mise en œuvre des évaluations ACA au niveau des États membres.

3. Quels sont le potentiel et les limites liés à la réalisation d’analyses coûts/avantages ex post en vue d’identifier et/ou d’analyser l’impact des projets? Ces analyses constituent-elles un outil adéquat pour l’analyse des impacts?

De bonnes analyses ACA ex post sont utiles, car elles peuvent servir à détermi-ner si le projet aura des impacts positifs nets pour la société et à identifier les causes des écarts par rapport aux hypothèses ex ante et aux prévisions. À cet égard, les analyses ACA peuvent être considérées comme un outil de gestion des projets, visant à guider des mesures correctives dans la mesure où des ajus-tements majeurs s’imposent durant la phase de mise en œuvre. Bien qu’il ne soit pas courant d’utiliser les analyses ACA comme outil d’évaluation ex post des impacts, nous les avons utilisées à cette fin dans le cadre de la présente étude et prouvé ainsi qu’elles sont bel et bien utiles.

L’utilisation des analyses ACA ex post est pertinente dans la mesure où la qua-lité et l’utilisation des analyses ACA ex ante sont améliorées. En fait, aussi longtemps que les analyses ACA ex ante sont réalisées comme un simple exer-cice de «cochage de cases» et comme une rationalisation ex post d’une décision politique déjà prise (elles sont donc largement sujettes à une manipulation des résultats), la volonté politique d’effectuer des analyses ACA ex post peut s’avérer faible et leur utilisation très limitée.

Les proposants des projets, les politiques et autres personnes impliquées n’ont probablement pas d’intérêt à ce que des événements ex post viennent contredire leur décision ex ante. Néanmoins, cela ne signifie pas qu’il ne vaut pas la peine d’en savoir davantage sur les coûts et les avantages a posteriori. Certains élé-ments étant susceptibles de favoriser largement un «excès d’optimisme» dans le cadre de projets entrepris en relation avec la politique de cohésion, certaines études ex post constituent sans aucun doute un précieux outil. Le rapport laisse entendre à juste titre qu’il s’agit d’un point délicat. Au bout du compte, l’objectif idéal serait de pouvoir démontrer comment une meilleure apprécia-tion économique peut, de manière plus générale (ex ante comme ex post), dé-boucher sur la production de meilleurs résultats par les projets mis en œuvre dans le cadre de la politique de cohésion.

…Un changement s’impose au niveau de la culture ACA…

Des analyses ACA ex post complètes sont utiles

…Les analyses ACA ex ante consistent simplement à «co-cher des cases»

…Si l’intérêt local est limité, les connaissances pour-raient, à plus grande échelle, s’avérer uti-les



26

.

Le timing de l’exercice ex post est crucial pour bien comprendre l’utilisation des résultats ex post et la capacité d’évaluer les impacts réels. Les études de cas réalisées pour les projets environnementaux ont montré que les analyses ex post réalisées juste après la phase de construction constituent davantage une mise à jour des analyses ACA ex ante que de véritables analyses ex post (le flux des avantages n’est pas encore matérialisé). Trois à cinq ans après l’achèvement du projet: telle est la période qu’il est jugé opportun d’attendre avant la réalisation d’un exercice ex post. Néanmoins, il faut reconnaître que les impacts identifiés par une telle analyse ne sont pas les impacts stabilisés à long terme, mais uni-quement les premiers impacts à plus ou moins court terme. Quoi qu’il en soit, l’exercice est probablement très utile pour la planification de mesures correcti-ves en cas d’événements imprévus ou de changements structurels au niveau du contexte (par exemple, la crise financière a influé sur certains paramètres macroéconomiques clés). Une véritable analyse ACA ex post requiert une plus longue perspective, de 10 à 15 ans minimum (selon la typologie des investis-sements).

La disponibilité des données et d’un appui méthodologique sont autant d’éléments indispensables qui jouent un rôle essentiel dans les calculs. Par exemple, il a été souligné que certains impacts pouvaient être difficiles à identi-fier à l’aide d’une analyse ACA, tels que les impacts plus généraux ou les im-pacts liés à des typologies moins traditionnelles d’infrastructure (comme les technologies de l’information et de la communication ou la recherche et le dé-veloppement).

Recommandations politiques Les principales recommandations politiques, telles qu’elles ressortent de la pré-sente étude, qui cherche à identifier les domaines dans lesquels l’Union euro-péenne pourrait prendre la tête, incluent l’échange de pratiques d’excellence, une amélioration des données environnementales (par exemple, un protocole sur le transfert des avantages), la réalisation d’études d’évaluation primaire et l’affectation de ressources pour une formation sur les analyses ACA. Ces re-commandations sont brièvement décrites ci-après.

Les États membres ont indiqué qu’ils souhaiteraient pouvoir accéder à des «pratiques d’excellence liées aux études ex ante» à titre d’exemples et afin d’apprendre aux bénéficiaires comment réaliser une analyse ACA ex ante de leurs propres projets.

Les pratiques d’excellence pourraient être appuyées par des paramètres stan-dard conçus pour l’évaluation des effets externes. Tout comme il existe des ty-pes d’impact «standard» pour de mêmes types de projets d’infrastructure envi-ronnementale (par exemple, émissions de méthane et odeurs), il pourrait être utile d’appliquer de mêmes types de valeurs aux paramètres afin de comparer des projets de même type. Un catalogue de paramètres en ligne, accessible aux professionnels, pourrait faciliter le processus ACA.

Certains effets externes, voire certains avantages directs, sont clairement très difficiles à évaluer. Il pourrait être utile d’avoir accès à des données fiables plu-tôt que de devoir réaliser des études à chaque fois. Cela est particulièrement

Le timing de l’exercice ex post est crucial

Échange de prati-ques d’excellence liées aux analyses ACA ex ante…

…Accès à des res-sources communes concernant certains paramètres, comme des valeurs commu-nes pour le calcul des effets externes



27

.

valable lorsqu’il s’agit d’évaluer le «consentement à payer» des citoyens (par exemple, accès à des données européennes actualisées sur le consentement à payer pour les différents types de projets d’infrastructure environnementale).

L’étude a révélé la nécessité de promouvoir la «pensée» ACA parmi les profes-sionnels afin de garantir l’intégration plus avant des analyses ACA dans les phases de définition des projets et de décision. Afin de renforcer une culture ACA, il pourrait être utile d’organiser une meilleure formation en la matière à tous les niveaux. Outre des directives ACA, on pourrait envisager l’accès à des ressources ACA communes, comme des paramètres communs, l’organisation de formation ACA ou encore l’accès à des ressources d’apprentissage en ligne (eLearning).

…Accès à des res-sources pour la for-mation sur les analy-ses ACA



28

.

Zusammenfassung Ein Konsortium bestehend aus COWI A/S, Tecninvest (PT) und Csil (IT), wo-bei COWI A/S als Konsortialführerin fungierte, hat die vorliegende Studie ab-geschlossen: „Nachträgliche Beurteilung von Kohäsionspolitik-Interventionen 2000-2006, finanziert durch den Kohäsionsfonds (darunter ehem. ISPA) - Work Package C – Kosten/Nutzen-Analyse bestimmter Umweltprojekte“ (Referenz: REGIO.C.4/JS D (2009) 680291). Die Studie wurde außerdem von drei unab-hängigen externen Fachleuten im Bereich Kosten/Nutzen-Analyse (CBA) un-terstützt3. Die Studie wurde im Januar 2010 begonnen und im April 2011 abge-schlossen. Ziel der vorliegenden Studie war eine Untersuchung der folgenden drei wich-tigsten Grundsatzfragen:

• Welche Auswirkungen hatte das geprüfte Projekt?

• Wie können nachträgliche Kosten/Nutzen-Analysen zur Praxis von Vorab-Kosten/Nutzen-Analysen beitragen?

• Welches Potenzial und welche Grenzen bestehen bei der Durchführung von nachträglichen Kosten/Nutzen-Analysen, um die Auswirkungen von Projekten zu identifizieren bzw. zu analysieren? Handelt es sich dabei um ein angemessenes Werkzeug zur Analyse von Auswirkungen?

Aus einer anfänglichen Liste von 40 wichtigen Umwelt-Kohäsionsfonds-Projekten, die ganz oder fast abgeschlossen waren, wurden 20 Projekte haupt-sächlich aus einer Reihe von bereits abgeschlossenen Projekten ausgewählt, die eine gute Mischung der drei Projektarten (Abfallwirtschaft, Abwasseraufberei-tung und Wasserwirtschaft) und eine angemessene Verteilung der begünstigten Länder darstellen. Jedes dieser zwanzig Projekte wurde auf der Basis der ver-fügbaren Daten aus DG REGIO und von Projekt-Webseiten bzw. sonstigen Medien kurz beschrieben. Unter den 20 Projekten wurden die endgültigen zehn ausgewählt; auch hier beruhte die Auswahl auf der Qualität des verfügbaren

3 Professor Giles Atkinsons, The London School of Economics and Political Science, De-partment of Geography and Environment UK. Professor Sándor Kerekes, Stellv. Rektor für akademische Belange an der Corvinus-Universität Budapest. Professor Runar Brannlund, Wirtschaft, Universität von Umeaa, Schweden

Zielsetzung der Stu-die

Auswahl der zu un-tersuchenden Projek-te



29

.

Materials in Verbindung mit dem Erfordernis, eine gute Mischung aus Projekt-arten und einer ausreichenden geografischen Verteilung zu erreichen.

Die zehn ausgewählten Projekte wurden gründlich analysiert, beginnend mit der Original-CBA (der Vorab-Kosten/Nutzen-Analyse), um die Annahmen hin-ter der Finanzanalyse, der Optionsanalyse, der Nachfrageanalyse und der Wirt-schaftsanalyse verstehen zu können. Auch der Kontext des Projekts wurde un-tersucht, darunter insbesondere das Ausmaß, in dem alternative Lösungen bei der Projektformulierung berücksichtigt wurden. Zweitens wurde dann die nach-trägliche CBA durchgeführt; dazu gehörte auch eine „Realitätsprüfung“ der ursprünglichen Annahmen unter Verwendung der Betriebsdaten für die Finanz-analyse, beispielsweise die tatsächlichen Betriebskosten und die tatsächlichen Erlöse, und zu guter Letzt wurde eine Bewertung der Wirtschaftsanalyse durchgeführt, darunter auch die Bestimmung einer positiven sowie einer nega-tiven Alternativlösung. Und schließlich wurde die Vorab-Analyse (Finanz- und Wirtschaftsanalyse) mit der nachträglichen (Finanz- und Wirtschaftsanalyse) verglichen.

Abb. 0-1 Gruppierungen der zehn ausgewählten Projekte

Die zehn untersuchten Projekte sind oben abgebildet. Vier Projekte befassen sich mit der Abfallwirtschaft, zwei mit Abwasserprojekten, zwei mit Projekten im Bereich Wasserwirtschaft und zwei sind kombinierte Abwasser- und Was-serwirtschaftsprojekte.

Die zehn ausgewähl-ten Projekte



30

.

Die Grundlage zur Vermittlung einer soliden Analyse beruht eindeutig auf der Verfügbarkeit und Zugänglichkeit verlässlicher Daten. Unser Ansatz zur Da-tenerfassung wird weiter unten kurz beschrieben.

Zunächst wurden verfügbare Daten von den relevanten Geographic Desks von DG REGIO beschafft. Auf die-ser Grundlage, und nach Ergänzung der obigen Kurzbeschreibung des Projekts, erhielt man eine erste Wahrneh-mung vom bzw. ein Verständnis für das Projekt. Zweitens konnte man seine Vertrautheit mit den einzelnen Projekten durch sehr wichtige Projektbesichti-gungen vor Ort noch erheblich vertiefen. Außerdem boten die Besichtigungen vor Ort eine Möglichkeit, zusätzliche Betriebsdaten sowie ein Verständnis der breiteren Auswirkungen des Projekts auf die örtliche Gemeinde und auf die Region zu erlangen. Während der nachträglichen Analyse wurde der Dialog mit dem Projektträger, dem Betreiber oder sonstigen Interessengruppen fortgeführt, falls zusätzliche Daten oder Erläuterungen erforderlich waren.

Zum Ende der Studie hin veranstaltete DG REGIO einen Workshop, um die Ergebnisse der Studie zu erörtern. Beteiligt waren u. a. Vertreter der untersuch-ten Projekte sowie Vertreter der staatlichen Verwaltungen. Dieser Abschlussbe-richt enthält das Feedback sowie die Ergebnisse des Workshops.

Ergebnisse der Studie

In Bezug auf die drei Grundsatzfragen kamen folgende Ergebnisse bei der Stu-die heraus:

1. Welche Auswirkungen hatten die geprüften Projekte?

Angesichts der Ergebnisse der nachträglichen Analyse lassen sich folgende Schlüsse ziehen:

Nur bei zwei Projekten (Abwasseraufbereitung in Zaragoza und zwei Kompo-nenten des portugiesischen Falls) konnten per Saldo positive Auswirkungen im Hinblick auf eine Verbesserung des gelieferten Trinkwassers, verbesserte Effi-zienz bei der Durchführung der Dienstleistung (Einsparungen bei der Reini-gung) und umweltbezogene (gesundheitliche) Auswirkungen verbucht werden. Solche positiven Netto-Ergebnisse zeigen sich im positiven wirtschaftlichen Nettobarwert [Economic Net Present Value - ENPV] und werden durch die ver-breitete Wahrnehmung der Projektförderer, Entscheidungsträger und Bürger gestützt.

Datenquellen und Ansatz zur Datenerfassung

Workshop mit Mit-gliedstaaten (MS)

Per Saldo positive Auswirkungen bei manchen, aber nicht bei allen Projekten...



31

.

Die restlichen acht Fälle ergaben alle mehr oder weniger negative Ergebnisse aus unterschiedlichen Gründen.

In vier Fällen (Tschechien, Barcelona, Polen, Dublin) führte das jeweils umge-setzte Projekt zu positiven Auswirkungen im Sinne einer Verbesserung der Dienstleistungen gegenüber der Bevölkerung und der Auswirkungen auf die natürliche Umwelt; dabei wurden größere Effekte im Rahmen des geförderten Freizeitangebots, eine verbesserte Wahrnehmung im Bereich Umwelt sowie institutionelles Lernen erzielt. Diese positiven Ergebnisse wurden jedoch zu Lasten hoher Kosten erzielt, was zu einem negativen ENPV und einem Nut-zen/Kosten-Verhältnis von weniger als 1 geführt hat. Es gibt eine allgemeine Auffassung, dass manche relevanten Auswirkungen nicht vom quantitativen CBA erfasst wurden; ob diese jedoch genügen würden, um die Bilanz hinsicht-lich der Quantifizierung zu verändern, ist unbekannt.

In vier Fällen (Ungarn, Bulgarien, Madrid, Kreta) unterstützen die Ergebnisse der CBA und die Wahrnehmung der Bürger und Interessengruppen das Argu-ment, dass die beurteilten Projekte zahlreiche Vorteile gebracht haben, welche die Kosten allerdings nicht ausgleichen. Zu den Gründen gehören technische Probleme von Überkapazität (Madrid), Probleme bei Nachfrageprognosen (Kreta) oder dass technische Lösungen zur Einhaltung von Umweltstandards für die tatsächlichen Bedürfnisse als zu kostspielig erachtet werden (Ungarn und Bulgarien). Daher führt die bloße Einhaltung der Umweltrichtlinien nicht unbedingt zu positiven Ergebnissen hinsichtlich des verbleibenden Nutzens.

Wie gerade erwähnt, ergaben die im Rahmen dieses Projekts durchgeführten Kosten/Nutzen-Analysen sowohl positive als auch negative NPVs. Das Ergeb-nis des NPV sollte als Summe dieser positiven (Ergebnis/Erlös) und negativen (Kosten-) Effekte des Projekts im Laufe der Zeit (Cash Flow) betrachtet wer-den. Wenn ein Projekt einen positiven NPV aufweist, bedeutet dies, dass es sich um ein solides Geschäftsmodell handelt, und dass Privatunternehmen be-reit dazu sein sollten, in das Projekt zu investieren, da im Laufe der Zeit ein Gewinn zu erwarten ist. Wenn sich ein negativer NPV einstellt, so hängt das damit zusammen, dass die Projektkosten die Projekteinkünfte im Laufe der Zeit übersteigen. Das kommt wahrscheinlich bei Projekten vor, bei denen ziemlich hohe Anfangsinvestitionen getätigt werden und die erwarteten Gewinne dann viel geringer ausfallen. Wenn der Einkommensverlust den erwarteten Gewinn des Projekts übersteigt, fällt der NPV negativ aus. In diesem Zusammenhang wird der NPV sowohl mit und ohne Finanzierung vom Kohäsionsfonds berech-net. Ist das Ergebnis ohne Finanzierung durch den Kohäsionsfonds deutlich po-sitiv, so sollte man erwägen, ob eine Finanzierung für das Projekt überhaupt notwendig ist. Andererseits bedeutet ein negativer NPV nicht unbedingt, dass das Projekt keine gute Investition für DG REGIO ist; allerdings wäre es ratsam, die Nachhaltigkeit des Projekts zu berücksichtigen.

Das Nutzen/Kosten-Verhältnis ist die Division des Nutzens durch die im Zu-sammenhang mit dem Projekt entstehenden Kosten. Ein Nutzen/Kosten-Verhältnis größer als eins lässt vermuten, dass der Nutzen die Kosten im Zu-

Interpretation der NPV- und B/C-Kennziffern



32

.

sammenhang mit einem Projekt übersteigt und dass die Projekte der entspre-chenden Gesellschaft aus sozioökonomischer Sicht per Saldo einen Nutzen vermitteln. Ist das Nutzen/Kosten-Verhältnis geringer als eins, was auf manche Projekte in dieser Analyse zutrifft, bedeutet dies, dass die Projektkosten höher sind als der Nutzen, den die Gesellschaft dadurch erhält. Oftmals kann es eine große Herausforderung darstellen, den pekuniären Nutzen zu ermitteln. Daraus folgt, dass manche Aspekte des Nutzens ausgelassen, nicht genau bewertet oder nur qualitativ beschrieben werden. Wenn ein Projekt Bestandteil eines „Master-Plans“ ist, kann es schwierig sein, die Arten von Nutzen einzeln zu ermitteln und diese in einer CBA einzuschließen. Dies kann zu Unterschätzungen führen, oder sogar dazu, dass ein Nutzen nicht berücksichtigt wird.

Aufgrund dieser Einsicht ist es wichtig, die Projekte aus einer breiteren Per-spektive zu betrachten und diese nicht nur aufgrund der NPV- und Nut-zen/Kosten-Verhältnisse zu beurteilen.

Manche Projekte in der Auswahl der Studie wurden durchgeführt, um europäi-sche Regulierungserfordernisse zu erfüllen und in allen Fällen mit der Zielset-zung, eine Basis-Infrastruktur zu erstellen oder diese zu verbessern. In den meisten Fällen wurde dies jedoch zu einem Kostenpunkt erzielt, der den quanti-fizierbaren Nutzen überstieg.

Es ist denkbar, dass in manchen Fällen bessere Beurteilungsmethoden evtl. zu einer Ermittlung von Nutzen geführt hätten, die zu einer positiven Bilanz beige-tragen hätten, aber es ist unwahrscheinlich, dass dies auf alle Fälle zutrifft.

Angesichts der Tatsache, dass eine CBA erst am Ende des Projektplanungspro-zesses durchgeführt wurde, ist es nicht verwunderlich, dass einige der Projekte negative ENPVs aufweisen. Sie waren von Anfang an nicht dazu konzipiert, positive ENPVs zu erzielen.

Bei den Projekten, die zur Einhaltung regulatorischer Bestimmungen durchge-führt wurden, kann es u. U. unmöglich sein, ein positives ENPV zu erzielen. In solchen Fällen weist das CBA den Nettoverlust am gemeinschaftlichen Nutzen aus, der durch die Einhaltung der entsprechenden Richtlinien verursacht wird. (In manchen Fällen bietet die Richtlinie auch Ausnahmen, wenn die Kosten der Einhaltung hoch sind, aber das ist in keinem der analysierten Fälle geschehen.)

Von den vier Abfallwirtschaftsprojekten bilden sowohl das bulgarische Projekt als auch das Projekt auf der Insel Kreta einen ersten Schritt zur Schaffung einer modernen Infrastruktur für die Abfallwirtschaft. In diesen beiden Fällen wurde der Abfall nicht behandelt, sondern einfach in zwei illegalen Deponien ent-sorgt; dadurch wurde das Grundwasser verseucht und es entstanden auch ande-re unangenehme Auswirkungen wie Gerüche, Verschandelung der Landschaft und ungeeignete Landnutzung, schlechtes Image, usw. Bei den Abfallprojekten in Madrid und Portugal war die Situation etwas anders.

...wobei die bloße Einhaltung nicht un-bedingt zu positiven Auswirkungen im Sinne eines wirt-schaftlichen Nutzens im Sinne des Allge-meinwohls führt.

Die Abfallprojekte haben die illegale Abfallentsorgung durch eine moderne Abfallwirtschaft er-setzt ...



33

.

Im Madrider Fall ging es bei dem Projekt um die Versiegelung einer alten, ille-galen Deponie, in der sich die rund 22 Mio. m3 an festen Abfällen angesammelt hatten, die in den letzten zwei Jahrzehnten von den drei Millionen Einwohnern von Madrid auf einer Fläche von 110 Hektar erzeugt worden waren. Gleichzei-tig wurde im Laufe des Projekts Deponiegas in grünen Strom verwandelt, was bis zum Jahr 2017 einen Energiewert in Höhe von 1.925 Mio. kWh ausmacht. Das Land auf der alten illegalen Deponie ist saniert worden; dabei wurden Bäume gepflanzt und Teiche angelegt, wovon die Vogelwelt profitiert, und ab dem Jahr 2025 kann das Land als Naherholungsgebiet genutzt werden. Bei dem portugiesischen Abfallprojekt handelt es sich um eine ganz andere Geschichte, da es sich hier um den Ausbau eines bestehenden, aber veralteten Abfallwirt-schaftssystems handelt.

Die Abwasserwirtschaftsprojekte umfassen auch den Fall Dublin sowie den Fall Ungarn. Abwasserkomponenten finden sich auch in den Fällen Pilsen und Polen wieder. Der Fall Dublin ist ein außergewöhnlicher Sachverhalt, da sich herausstellte, dass die Projektplaner die Nachfrage nach der Abwasseraufberei-tung signifikant unterschätzt hatten. Irland galt als sog. „Celtic Tiger“, wobei Dublin zwischen 1995 und dem Jahr 2007 eine Phase rapiden Wirtschafts-wachstums erlebte – die allerdings im Jahr 2008 zu einem dramatischen Ende kam. Gleichzeitig nahm das Bevölkerungswachstum von Dublin unerwartet rapide zu. Daher litt das Dubliner WWTS an Überlastung, und das bereits von Anfang an. Außerdem führt die Klassifizierung der Liffey-Mündung als ge-fährdetes Gewässer dazu, dass neue Infrastrukturentwicklungen erforderlich sind, um strengere Einleitungsstandards zu erfüllen, und deswegen findet zur-zeit ein Ausbau der Abwasseraufbereitungsanlagen statt.

Allerdings weist die Bucht von Dublin nunmehr eine wesentlich bessere Was-serumgebung auf. Dass das Wasser in der Bucht von Dublin inzwischen zum Baden und zur weiteren Nutzung im Naherholungsbereich geeignet ist, wird dadurch belegt, dass vier Strände nunmehr die Wasserqualitätskriterien des Programms „Blaue Flagge“ erfüllen. Offenbar gibt es im Zuge der Entwick-lung, dass neue Ortschaften an das Abwasseraufbereitungssystem angeschlos-sen wurden, eine beträchtliche Anzahl an neuen Baugenehmigungen.

Das ungarische Abwasserprojekt Szeged ist ein ausgezeichnetes Beispiel einer Intervention, die auch im Rahmen fast aller anderen neun Projekte eingeleitet wurde, um der Abwasserrichtlinie zu entsprechen; dabei verliefen der Bau einer Abwasseraufbereitungsanlage und der Ausbau des Abwasserkanalnetzes voll-kommen nach Plan. Allerdings liefert das Projekt noch nicht den vollen Nutzen der Investition im Hinblick auf die verbesserte Wasserqualität der Theiß. Der Grund dafür ist, dass die Einleitungen stromaufwärts (aus Rumänien) nicht nach Maßgabe der gleichen Standards behandelt werden wie das von der Stadt Szeged erzeugte Abwasser.

Sowohl beim Fall Pilsen (Plzen) als auch beim polnischen Fall Stettin (Szcze-cin) handelt es sich um kombinierte Abwasserwirtschafts- und reine Abwasser-projekte. Bei beiden Projekten geht es um den Ausbau einer bestehenden aber

….und eine Wand-lung der Sünden aus der Vergangenheit in künftige Umwelt-aktivitäten…

Die Auswirkungen der Abwasserprojek-te fallen sehr unter-schiedlich aus.

…Vorteile für die Bürger von Dublin im Naherholungsbe-reich dank einer saubereren Dubliner Bucht

...unrealisierter Nut-zen durch Verwässe-rungseffekte der Theiß (Tisza) in Un-garn



34

.

veralteten und nicht mehr adäquaten Infrastruktur. Im Falle Pilsen sind 1.400 Haushalte nunmehr an das Abwassersystem angeschlossen, und die Trinkwas-serversorgung ist zuverlässig und von guter Qualität. Vor dem Projekt war das Leitungswasser für einige Bürger einfach nicht trinkbar, so dass sie auf Wasser in Flaschen angewiesen waren. Dank dem Bau von zwei Rückhaltebecken wur-de die Anzahl an Überläufen in die Beraun reduziert; damit wurde eine der wichtigsten Verunreinigungsquellen des Flusses abgeschafft. Ein saubererer Fluss wiederum hat dazu geführt, dass sich dort wieder Flusskrebse angesiedelt haben, dass die Menschen wieder im Fluss baden, Strände angelegt wurden, und dass eine Reihe von Projekten zur Wiederherstellung der Uferböschungen umgesetzt wurde; außerdem wurden Fahrrad- und Spazierwege entlang des Flusses angelegt, beispielsweise das „Green Ways“ Projekt.

Das Abwassersammelsystem in Stettin war größtenteils über 60 Jahr alt und teilweise noch marode durch Schäden, die im zweiten Weltkrieg verursacht wurden. Das Abwasseraufbereitungssystem beruhte auf einer veralteten Tech-nologie primärer Abwasserbehandlung; dabei wurde verschmutztes Wasser di-rekt in die Oder geleitet. Dadurch war Stettin auf die Brennpunktliste nach Maßgabe der Konvention von Helsinki geraten – dank der neuen Infrastruktur ist das nicht mehr der Fall. Außerdem beklagten die Nutzer der Wassersysteme, dass die Dienstleistungen qualitativ zu wünschen übrig ließen. Diese Probleme sind inzwischen behoben worden.

Die Fälle Barcelona und Zaragoza sind die einzigen Projekte, die nicht durch mangelnde Einhaltung von Bestimmungen entstanden sind. Beim Barcelona-Projekt geht es darum, das Risiko der Überflutung zu vermindern, während das Zaragoza-Projekt den Bau einer 70-km langen Rohrleitung von den Bergen bis herunter nach Zaragoza bedeutete, damit die 821 Bürger von Zaragoza Trink-wasser bekamen und nunmehr Leitungswasser statt Wasser in Flaschen ver-wenden können. Das Projekt hat auch die Stadtverwaltung dazu gezwungen, weitere Maßnahmen zu ergreifen, um den Wasserverbrauch zu senken. Im Falle Barcelona sind die wichtigsten Auswirkungen ein verringertes Überschwem-mungsrisiko und Strände, die nach einer Überschwemmung nicht mehr tage-lang für Touristen gesperrt werden müssen.

Zusätzlich zu den oben genannten positiven Ergebnissen haben fast alle Projek-te positive Nebeneffekte im Hinblick auf das Umweltbewusstsein erzeugt (bei-spielsweise Änderung der Meinung in der allgemeinen Bevölkerung, z. B. effi-zientere Wassernutzung – im Fall Zaragoza – oder die Einführung von Müll-trennung) sowie institutionelles Lernen (die Kapazität der öffentlichen Verwal-tung, sich mit EU-Projekten zu befassen – die EU-Einheit im Fall Tschechien – oder aber eine komplexe und technologisch innovative Infrastruktur zu verwal-ten – im Falle Barcelona).

In die Kohäsionspolitik ist ein Distributionsziel integriert, um den ärmsten Bür-gern und Regionen spürbare Vorteile zu bringen. Die bei der laufenden Studie analysierten Projekte weisen auf das Thema der Distributionseffekte hin, da diese den Regionen mit geringem Einkommen zugute kommen, aber indirekt

... und politische Vorteile, da Stettin nicht mehr auf der Brennpunktliste der Helsinki-Kommission (Helcom) steht

Umweltbewusstsein und institutionelles Lernen als einherge-hende Nebeneffekte

Distributionseffekte lassen sich in einer CBA nur schwierig erfassen



35

.

durch Regionen mit höherem Einkommen getragen werden (daher auch das So-lidaritätsziel der CF-Politik). In der Kosten/Nutzen-Analyse als Instrument werden solche Distributionseffekte nicht explizit erfasst, aber künftig könnte man eine Art formale Berücksichtigung der Implikationen für die Kos-ten/Nutzen-Analyse ins Auge fassen. Im Zusammenhang mit diesem Thema ergibt sich die Frage, ob sich die Bürger als Nutznießer der Projekte diese auch leisten können. Im jüngsten Bericht steht, dass die Erschwinglichkeit für einige der Projekte ein Thema ist, da die Bürger vor Ort Steuern oder von den Nutzern übernommene Tarife für den Betrieb der eingeführten Umweltdienstleistungen zahlen müssen.

2. Wie können nachträgliche Kosten/Nutzen-Analysen zur Praxis von Vor-ab-Kosten/Nutzen-Analysen beitragen?

Diese Studie hat zu unserem Verständnis beigetragen, wie Kosten/Nutzen-Analysen (CBAs) in der Praxis durchgeführt werden, wie sie verwendet werden und insbesondere wie sie nicht für das verwendet werden, wofür sie im Vorab-Kontext gedacht waren, und zwar als Grundlage zur Auswahl der preiswertes-ten Möglichkeit mit dem geringsten Risiko. Trotzdem zeigt die Studie, dass Kosten/Nutzen-Analysen – wenn sie richtig durchgeführt werden, die Projekt-auslegung und -lösungen verbessern können.

Ferner weist die Studie darauf hin, dass es bei einigen Projekten, die untersucht wurden, teilweise Probleme mit entweder Unter- oder Überkapazität gibt, die man besser in den Griff bekommen hätte, wenn eine ordentliche Nachfrageana-lyse zur Anwendung gekommen wäre.

Aus strenger CBA-Sicht hätte sich nur eines der Projekte für eine Finanzierung qualifiziert, wenn wir respektierten, dass der Vorteil für die allgemeine Gesell-schaft höher ist als die Investitions- und Betriebskosten für die Infrastruktur (Nutzen-Kosten-Verhältnis >1). Dieser Umstand zeigt eines der kritischsten Ergebnisse der gegenwärtigen Studie auf: Wenn die Kosten/Nutzen-Analyse während der Auswahlphase korrekt und zeitnah durchgeführt worden wäre, so wäre es möglich gewesen, leistungsschwache Projekte zu ändern oder neu zu berücksichtigen. In manchen Fällen hätte dies eine andere Entwicklung der Pro-jekte impliziert, für welche die rechtliche Einhaltung am strengsten war, oder es hätte als Basis für einen Ausnahmeantrag dienen können.

In der Praxis erscheint es so, als ob CBAs hauptsächlich durchgeführt werden, weil es sich dabei um ein Antragserfordernis für die Kohäsionsfinanzierung handelt. Die Kosten/Nutzenanalyse wird oftmals an Fachberatungsunternehmen ausgelagert, statt dieses als Bestandteil des Entscheidungsfindungsprozesses zu integrieren; es geht lediglich darum, „Kästchen anzukreuzen“.

Unter Praktikern besteht die Wahrnehmung, dass die Sicherung der Kohäsions-fondshilfe, wie sich der Vertreter eines Mitgliedsstaats aussprach, folgender-maßen aussieht: „Es ist eine Frage, wie man die Finanzanalyse so schlecht wie nur möglich aussehen lässt, um die Finanzierungsnotwendigkeit zu erhöhen,

Bei sachgemäßer Anwendung könnten CBAs dazu beitra-gen, Geldverschwen-dungen auszuschal-ten

Die jetzige Rolle der Vorab-Kosten/ Nut-zenanalysen ist le-diglich eine Übung im „Kästchenan-kreuzen“...



36

.

und wie man die Wirtschaftsanalyse so positiv wie nur möglich gestaltet, um die Finanzierung mit öffentlichen Geldern zu rechtfertigen.“

Eine eindeutige Nachricht vom Workshop der Mitgliedsstaaten, wie sie von einigen Mitgliedsstaaten ausgedrückt wurde, war dass obwohl die CBA-Methodologien interessant sind, erscheinen sie oftmals als akademische Übung, während die Realität oftmals sehr komplex ist und mit ein paar Excel-Arbeitsblättern nur schwierig darzustellen ist. Es wurde zudem gemeldet, dass eine Kosten/Nutzen-Analyse nicht immer die gegenwärtige Lage widerspiegelt, da sich die Sachverhalte schnell ändern können; beispielsweise sorgen neu er-lassene Richtlinien dafür, dass die CBA-Studien schnell veraltet sind. Als Re-aktion darauf haben einige Mitgliedsländer einen vereinfachten Ansatz für Kos-ten/Nutzen-Analysen gefordert. Außerdem drückten die Mitgliedsstaaten den Wunsch aus, eine Sammlung von „Best-Practice“-Fällen zu erstellen, die eine zusätzliche Motivation für den CBA-Prozess schaffen könnten.

Generell wird ein Mangel an gemeinsamer sprachlicher und auch fachlicher Verständigung unter den Projektförderern, Entscheidungsträgern auf der Ebene der Mitgliedsstaaten und der EG über die Notwendigkeit und den Einsatz von CBAs gemeldet.

Unter Bezugnahme auf den Antragsprozess wurde auch die Notwendigkeit ei-ner verbesserten „Schulung“ derjenigen geäußert, die für die Beurteilung von Anträgen zuständig sind; insbesondere sollte eine klarere (nichttechnische) Sprache verwendet werden, um den Prozess transparenter zu gestalten und eine reibungslose Kommunikation zwischen dem Begünstigten und der EG zu ge-währleisten. Somit würde eine intensivierte Schulung auf allen Ebenen das ge-meinsame Verständnis über den Einsatz von Kosten/Nutzen-Analysen für die Projektauswahl verbessern.

Wenn die CBA irgendeine Auswirkung darauf haben soll, wie die Auswahl zwischen unterschiedlichen Optionen stattfindet, so ist es zunächst wichtig, sich mit der zeitlichen Abstimmung der CBA-Integration zu befassen. Daher muss die CBA-Denkweise ein integraler Bestandteil des Entscheidungsfindungspro-zesses sein, und das erfordert eine Veränderung der CBA-Kultur und setzt vor-aus, dass die CBA ein integraler Bestandteil der Projektauswahl und -beurteilung wird. Die wichtigsten Fragen, die CBAs von der allgemeinen Nut-zung ausschließen, scheinen eher praktischer Art zu sein, wie beispielsweise ein Mangel an Daten bei der Beurteilung von Auswirkungen auf die Umwelt sowie ein Mangel an politischem Willen und Kontrollen zur Umsetzung von CBA-Beurteilungen auf Mitgliedsstaatenebene.

Und die Mitglieds-staaten zeigen sich skeptisch über den Sinn von Kos-ten/Nutzen-Analysen...

...zusätzliche Ausbil-dung auf allen Stufen könnte die CBA-Kultur jedoch beflü-geln…

... und es besteht die Notwendigkeit, die CBA-Kultur zu ver-ändern...



37

.

3. Welches Potenzial und welche Grenzen bestehen bei der Durchführung von nachträglichen Kosten/Nutzen-Analysen, um die Auswirkungen von Projekten zu identifizieren bzw. zu analysieren? Handelt es sich dabei um ein angemessenes Werkzeug zur Analyse von Auswirkungen?

Ordentliche nachträgliche Kosten/Nutzenanalysen sind sinnvoll, weil sie dazu herangezogen werden können, um festzustellen, ob ein Projekt per Saldo posi-tive Auswirkungen für die Gesellschaft erzeugen wird und die Ursachen von Abweichungen von den Vorab-Annahmen und Prognosen feststellen kann. In dieser Hinsicht kann man die CBA als Projektmanagementinstrument betrach-ten, das dazu dienen soll, richtungsweisende korrigierende Maßnahmen einzu-leiten, falls signifikante Anpassungen während der Umsetzungsphase erforder-lich sind. Und obgleich es nicht üblich ist, die CBA als nachträgliches Instru-ment zur Ermittlung der Auswirkungen einzusetzen, hat man das im Zuge die-ser Studie vorgenommen; dabei stellte sich heraus, dass sie tatsächlich sinnvoll ist.

Der Einsatz der nachträglichen CBA ist in dem Maße relevant, in dem die Qua-lität und der Einsatz der Vorab-CBA verbessert werden. Tatsächlich ist es so, dass solange bei der Vorab-CBA ein Ansatz in Richtung „Kästchenankreuzen“ vorkommt, so wird die nachträgliche CBA als Rationalisierung einer politi-schen Entscheidung, die bereits gefällt wurde, dienen (daher besteht eine große Gefahr, dass die Ergebnisse manipuliert werden), so wird es vielleicht nur einen geringen politischen Willen geben, die nachträgliche CBA durchzuführen, und der Einsatz wird deswegen sehr beschränkt sein.

Die Projektträger, Politiker usw. haben vermutlich kaum Interesse daran, an-schließend zu erfahren, dass ihrer Vorab-Entscheidung zur Umsetzung ange-sichts späterer Ereignisse widersprochen wird. Das bedeutet aber nicht, dass es sich nicht lohnt, nachträglich mehr über die Kosten und den Nutzen zu erfah-ren. Angesichts der Tatsache, dass es im Kontext von Projekten im Rahmen der Kohäsionspolitik (CP) u. U. starke Anreize für eine „Optimismusneigung“ gibt, werden bestimmte nachträgliche Studien sicherlich ein wertvolles Instrument darstellen. Der Bericht gibt zu Recht zu bedenken, dass möglicherweise heikle Fragen auftauchen könnten. Im Endeffekt wäre es ein wünschenswertes Ziel, nachweisen zu können, dass eine bessere wirtschaftliche Einschätzung (sowohl vorab als auch nachträglich) zu besseren CP-Projektergebnissen führen kann.

Das Timing der nachträglichen Analyse ist extrem wichtig, um die Nutzung der nachträglichen Ergebnisse und die Fähigkeit zur Bewertung der tatsächlichen Auswirkungen verstehen zu können. Die Fallstudien, die für die Umweltfälle durchgeführt wurden, zeigten, dass sich die nachträgliche Analyse, die unmit-telbar nach der Bauphase durchgeführt wurde, eher als Aktualisierung einer Vorab-Analyse zu werten ist statt als eine echte nachträgliche Analyse (bei der sich der Nutzen noch gar nicht eingestellt hat). Ein Zeitraum von drei bis fünf Jahren nach der Fertigstellung eines Projekts wird für eine nachträgliche Ana-lyse als angemessen betrachtet. Es sollte jedoch zur Kenntnis genommen wer-den, dass die von einer derartigen Analyse erfassten Auswirkungen nicht die

Vollständige nach-trägliche Kos-ten/Nutzen-Analysen sind sinnvoll

... aber bei der Vor-ab-CBA geht es nur darum, „Kästchen anzukreuzen“

... und während auf lokaler Ebene nur wenig Interesse be-steht, könnte das Wissen im erweiter-ten Kontext gewinn-bringend sein

Der Zeitpunkt der nachträglichen Ana-lyse ist von entschei-dender Bedeutung



38

.

langfristig stabilen Folgen sind, sondern nur die anfänglichen, kurzfristigeren Auswirkungen. Trotzdem ist die Analyse potenziell extrem nützlich, um korri-gierende Maßnahmen einzuplanen, wie bei unerwarteten Ereignissen oder bei strukturellen Veränderungen in diesem Zusammenhang (beispielsweise hat die Finanzkrise einige wichtige makroökonomische Parameter beeinflusst), die zwischenzeitlich eingetreten sind. Eine echte nachträgliche Kosten/Nutzen-Analyse erfordert eine längere Zeitskala von mindestens 10 - 15 Jahren (je nach den jeweiligen Anlagestrukturen).

• Die Verfügbarkeit von Daten und methodologischem Fachwissen sind wichtige Erfordernisse, die bei den Berechnungen mit eine Rolle spielen. Bei-spielsweise wurde betont, dass bei einer Kosten/Nutzenanalyse einige Auswir-kungen, beispielsweise die breiteren Auswirkungen oder aber diejenigen aus weniger traditionellen Infrastrukturtypologien (beispielsweise Informations- und Kommunikationstechnologie oder Forschung und Entwicklung) u. U. nur schwer zu ermitteln sind.

Politikempfehlungen Die wichtigsten Politikempfehlungen gemäß der gegenwärtigen Studie, im Rahmen derer die EG eine führende Rolle einnehmen kann, beinhalten auch einen Austausch über gemeinsame „Best-Practice“-Lösungen, verbesserte Umweltdaten, beispielsweise ein Protokoll über den Transfer von Nutzen sowie primäre Bewertungsstudien und schließlich auch CBA-Schulungsressourcen. Diese werden kurz nachstehend erläutert.

Die Mitgliedsstaaten bekundeten ihr Interesse daran, Zugriff auf „gute Prakti-ken von Vorab-Studien“ als Beispiele sowie Lernmaterial für Begünstigte im Rahmen der Erstellung eigener Vorab-CBAs zu erhalten.

Solche guten Praktiken könnten durch Standardparameter zur Beurteilung ex-terner Faktoren verwendet werden. Da es „Standardarten“ von Auswirkungen im Rahmen der gleichen Kategorien von Umweltinfrastrukturprojekten gibt, beispielsweise beim Austritt von Methan oder schlechten Gerüchen etc., könnte es nützlich sein, die gleichen Wertekategorien gegenüber diesen Parametern einzusetzen, um eine Vergleichbarkeit mit ähnlichen Projektarten sicherzustel-len. Ein Online-Katalog dieser Parameter, auf den die Praxis-Fachleute Zugriff haben, könnte den CBA-Prozess u. U. erleichtern.

Manche externen Faktoren und sogar manch ein direkter Nutzen lassen sich nur sehr schwer messen, und es könnte sinnvoll sein, auf verlässliche Daten zugrei-fen zu können statt jedes Mal Umfragen durchzuführen. Dies ist von besonde-rer Relevanz, wenn man die Bereitschaft der Bürger ermittelt, für etwas zu zah-len „Willingness to pay“ (WTP), beispielsweise beim Zugang zu aktualisierten EU-Daten über die WTP im Hinblick auf die unterschiedlichen Arten von Um-weltinfrastrukturprojekten.

Die Studie hat erwiesen, dass es notwendig ist, eine CBA-„Denkweise“ in der Fachwelt zu fördern, um sicherzustellen, dass die CBA weiter in der Projektde-

Gemeinsame positive Praxis im Rahmen von Vorab-CBAs…

... darunter auch der Zugang zu gemein-samen Ressourcen über die Parameter, beispielsweise ge-meinsame Werte bei der Beurteilung ex-terner Faktoren

... und sogar CBA-Schulungsressourcen



39

.

finition und in den Entscheidungsfindungsprozess integriert ist. Um eine CBA-Kultur weiterentwickeln zu können, wären zusätzliche CBA-Schulungen auf allen Ebenen eventuell wünschenswert. Zusätzlich zu den CBA-Richtlinien sollte auch der Zugriff auf gemeinsame CBA-Ressourcen, beispielsweise ge-meinsame Parameter, die Entwicklung von CBA-Schulungskursen oder sogar Ressourcen im Bereich des eLearning in Betracht gezogen werden.



40

.

1 Introduction In January 2010, DG REGIO accepted COWI’s proposal in response to the call for tenders by open procedure Nº 2009CE160CAT052 launched by DG REGIO, European Commission. It regards the provision of services under con-tract "Ex post evaluation of cohesion policy interventions 2000-2006 financed by the Cohesion Fund (including former ISPA) - Work Package C - Cost bene-fit analysis of selected environment projects" (Reference: REGIO.C.4/JS D (2009) 680291).

The ex post evaluation of cohesion policy interventions 2006 financed by the Cohesion Fund (including the former Instrument for Structural Policies for Pre-accession [ISPA]) during the period 2000-2006 is an important instrument to inform national and regional authorities, the general public, the European Par-liament and other stakeholders involved about the outcome. The evaluation will provide lessons for the next programming period of cohesion policy.

The focus of this study is on cost benefit analyses (CBA) for environment in-frastructure projects within the topics water and waste. A contribution from this study to the evaluation thus comprises assessments of the impacts of 10 se-lected projects that have received financing from the Cohesion Fund or ISPA.

It must, however, be acknowledged that the 10 projects selected for assessment are not necessarily representative of all water and waste projects financed dur-ing the period 2000-2006 and so the results cannot be generalised. While pro-jects in both the Cohesion Fund countries and the ISPA countries are covered, priority in the selection was given to projects that were completed and to pro-jects where it appeared that satisfactory data on costs and benefits were acces-sible.

1 What were the impacts of the examined project?

2 How can ex post cost-benefit analysis contribute to the practice of ex ante cost-benefit analysis?

3 What are the potential and limits to carry out an ex post cost-benefit analy-sis to identify and/or analyse the impacts of projects? Is it an appropriate tool for impact analysis?

Learning lessons for current and future programming peri-ods …

… focusing on cost benefit analyses for environment projects …

And answering three fundamental ques-tions…



41

.

The study report is structured as follows:

• Chapter 2 introduces CBA methodological considerations as applied to the environmental infrastructure projects.

• Chapter 3 provides an overview of the study methodology and introduces the environment projects that were examined during the study with a focus on the 20 projects from which 10 projects were selected for CBA analysis. Apart from providing the foundation for selecting the 10 projects, the overview provides insight into the quality of the project applications in general - in particular the ex ante CBAs.

• Chapter 4 contains the ex post project analyses for each of the 10 selected projects - with a focus on in-depth CBAs. The analysis also covers revi-sions of the ex ante CBAs produced by the project applicants and compari-sons of the ex ante and the ex post CBAs.

• Chapter 5 provides an assessment of the CBA as a method. This is partly based on the results of the above analysis combined with the feedback pro-vided by Member State representatives and project representatives at a workshop held in Brussels on 3 February 2011. Further input has been provided by the external experts to the study.

• Chapter 6 finally presents the main findings and recommendations of the study.

Structure of report-ing



42

.

2 CBA method considerations This chapter will highlight CBA methodological considerations particular to the environmental infrastructure projects.

2.1 CBAs applied to waste and water projects The discussions that follow address the peculiarities of the CBA methods of analysis in relation to the specific project types under consideration, namely solid waste, water management and wastewater projects. Hence, these sections will not cover the standard aspects common to all CBAs, e.g. the requirement to make forecasts with and without the project, to use shadow prices where re-quired, or to use discounting, etc. All elements in the Commission CBA Guide are observed.

Waste is a by-product of normal life. What can we do with it? We could pile it up in our backyard, but that might have unpleasant consequences. The pile of waste would soon become an unpleasant eyesore, and a possible health hazard, by being the source of bad smells, and, in some climates, attract insects. The runoffs could threaten the soil and groundwater. For this reason, people have preferred to carry their waste to disposal lots far from their own backyard, but not so far so as to make the transport too expensive.

But this merely shifts the problem from the backyard of individuals to the by-ways of their villages or towns. The problems of aesthetics, smells, runoffs, and disease vectors remain, but at different locations. The solution to these prob-lems is offered by modern solid waste projects that seek to dispose of waste in a sanitary way and avoid the unpleasant external effects, as economically as pos-sible.

The above description guides us through the formulation of the CBA method-ology of solid waste projects. Some key observations can be made:

1 There is willingness to pay for the removal of waste. People prefer to pay to have waste removed, rather than having it in their backyards. Payment can be in cash, or through the effort of carrying the waste oneself. How-ever because considerable external costs are involved in solid waste han-dling and disposal, a focus only on willingness to pay falls short of meas-uring the full benefits of a project.

2 Properly designed and operated solid waste projects will reduce or elimi-nate external environmental costs (aesthetics, smells, soil and water con-

Solid waste projects



43

.

tamination). Also, the reduction of GHG emission from flaring or burning off the landfill gas also have to be considered in the CBA.

3 In some cases, solid waste projects may earn revenues through recycling waste or energy generation.

4 Solid waste projects have capital and operating costs, which may include costs directly borne by clients, such as packaging (typically plastic bags) and sometimes transport to collection points.

It is rare to find a solid waste project with significant health benefits, because people generally manage to remove waste far enough from their immediate sur-roundings to prevent exposure to health hazards. Two aspects are important when designing and evaluating a solid waste project:

• Cost minimization, through the comparison of alternatives in dimensions such as choice of technology, geographical area of coverage, current and future need for waste removal depending on demographic and industrial development trends.

• Quantification of the externalities associated with the project alternatives. These are aesthetics, smells, and soil and groundwater contamination. The quantification of these externalities can be achieved through the techniques of hedonic pricing or contingent valuation, or by costing remediation measures.

Data are needed for the project, for the situation without the project, and ideally for one or more project alternatives, in the following categories, which can be further subdivided as needed, depending on the degree of detail available:

• Capital and operating costs

• Revenues from recycling

• Externalities.

The analysis of wastewater projects is very similar to that of solid waste pro-jects. For wastewater originating in households, the technological alternatives are (1) connecting to a sewage system or (2) the construction of an on site sew-age facility. Industrial waste water sometimes must be treated separately from municipal sewage systems because that allows the treatment to be tailored to the characteristics of the wastewater generated.

The analysis of external costs is important for this type of project, and it needs to be conducted for all considered alternatives. Even the proposed projects themselves could have external costs, albeit, it is hoped, lesser ones than that of its alternatives. The external costs usually relate to the discharge from the pro-ject (liquid or sludge disposal), and will vary depending on the number of stages of cleaning applied (primary, secondary, or tertiary). The external costs

Wastewater projects



44

.

are incurred trough the project alternatives’ effects on bodies of water, on the land, or on the atmosphere.

In some cases, this type of project could generate direct benefits as well, if it can produce treated water for irrigation or sludge for soil treatment. These liq-uids contain nitrogen, phosphorous and potassium, which are valuable fertiliz-ers. But water and sludge must be treated to keep toxic or infectious compo-nents below threshold values. Consequently, it could happen that they have no commercial value.



• Revenues from recycling

• Externalities.

The consumption of potable water involves fewer externalities than waste dis-posal. For this reason, willingness-to-pay by users plays a larger role in the analysis of this kind of project, as in this case willingness-to-pay captures a high proportion of the benefits enjoyed by users.

Despite this, the comparison of alternatives is still the key element of analyses of this project type as well, given that projects or their alternatives seldom re-sult in great changes in the quality or quantity of water consumed (the only cases in which willingness-to-pay plays a role in the analysis).

The externalities that might have to be considered are those that would be gen-erated in providing and using the service. The construction and operation of water supply systems could have adverse environmental impacts, and the dis-posal of used water by consumers could create external effects, especially in the absence of adequate sewage services of some kind. The health consequences of potable water projects are seldom an explicit factor, for alternative methods of providing potable water nearly always exist.

Water is often a scarce resource, and its opportunity cost must be estimated.

It is usual to review tariff policies when analysing potable water projects, be-cause when tariffs fail to reflect true costs, the demand for water will be larger than it would otherwise be, which might in turn be the main driver for the pro-ject. So setting the tariffs right could indeed become an alternative to the pro-ject, and needs to be looked at.

It is also customary to review water losses in potable water systems, for stem-ming such losses might be more economical than building new adduction in-frastructures. However, donor bias often conspires against this option.

Water management projects



45

.



• Willingness-to-pay for changes in water consumption

• Externalities.

2.2 Quantification of environmental benefits and externalities

Assigning monetary values to environmental benefits and external costs to pro-jects in the fields of waste disposal, wastewater and potable water usually con-cern damage to soil, water or the atmosphere. These can be quantified directly or indirectly.

Hedonic pricing. Under this method, the external cost is quantified by its ob-servable effects on something that is affected by the externality and that has a market price. The most typical example is the reduction in the value of houses that are subject to odours near a waste treatment site. The value of the external-ity can be established econometrically.

Stated preference or Contingent valuation. Under this method members of a sample of affected people are asked about their willingness to pay for the re-moval of an external cost that affects them. The sample is stratified, and differ-ent strata are confronted with different hypothetical costs. The response rates can be used to derive a monetary value for the externality.

An indirect method is estimating the cost of remediation that removes the ex-ternal cost (avoidance costs). This method has an important weakness, how-ever, as it relies on the (untested) assumption that the cost of remediation is less than the cost of the externality.

For practical reasons and because the above-mentioned methods are relatively expensive, 'benefit transfer' methods are often used.

'Benefit transfer' is the method of attributing to a project benefits calculated for a similar project elsewhere. To apply this method care has to be taken to:

• ensure that the externality is comparable.

• make adjustments to systematic differences in willingness to pay. (This is important if the quantification of the externality was done in a different country).

• scale the results as needed: take into account the number of people ex-posed, the size of the area affected, the starting point (e.g. no waste man-agement/improvement of existing system etc.).

The direct methods are…

Indirect methods…



46

.

It will be necessary to apply this method for projects in which it and its alterna-tives differ perceptibly in the extent of external costs caused or avoided. To this end it will be necessary to:

• find studies in the literature quantifying similar externalities

• find project data with which to do the necessary adjustment and scaling.

2.3 The impact of regulations Potable water, solid waste and wastewater projects are often subject to regula-tion and standards that constrain the design of projects. These constraints re-duce the feasible set of alternatives to those that are compliant with the regula-tions in force. This must be taken into account when performing CBA of indi-vidual projects.

A legitimate question might arise about the desirability of having such regula-tions. To answer that question, of course, the cost of meeting the standards would have to be compared with the cost of alternatives that are not compliant. Non-compliant alternatives might differ significantly from the compliant ones in both costs and benefits.

But this is not the task at hand in the current study. The current study only needs to be careful not to compare complying projects with non complying al-ternatives.

2.4 Special considerations of Ex-Post evaluations CBAs are forecasts of the future. An ex-post evaluation is usually conducted after some time has passed after a project’s completion, and can serve either the purpose of ascertaining the actual performance of the project or of analysing the quality of the original analysis, or both.

It is important to note that the two purposes require different approaches to un-dertaking the ex post evaluation. Therefore, a single ex post evaluation cannot serve both objectives at once without adjustment. The evaluation done for one of the purposes cannot automatically be used for the other.

The advantage of an ex post evaluation is the benefit of (partial) hindsight. The ex ante evaluation is largely based on forecasts, which are subject to uncer-tainty (which may or may not have been acknowledged at the time). The ex post evaluation has the benefit of being able to observe the actual values of some variables, without any uncertainty (typically investment costs), and of observing the trends of other variables (typically benefits and operating costs) that reduce the uncertainty of the forecasts required for the rest of the useful life of the project.

If the purpose of the ex post evaluation is ascertaining the performance of the project, the task at hand is simply that of performing an appropriate CBA, using the new, more reliable data. This is the simpler of the two possible purposes.

Objectives



47

.

If the purpose is to assess the quality of the original analysis (beyond simply checking if it followed the right methodology), the ex post evaluation task is more difficult. In that case, it is not enough to compare the forecast to what ac-tually happened, as that is an unfair test. The question to be answered is whether the recommendation of the original study was correct or not based on the information available at the time. This distinction derives from the fact that CBAs are forecasts of uncertain events, and nobody can be required to own a crystal ball.

Not all studies acknowledge their expected margins of error, but a margin of error is there none the less. It is a very rare project indeed that has a zero prob-ability of being unfeasible. Almost no projects would ever be undertaken if they were required to guarantee a positive return under all circumstances. It does happen that events unfold in such a manner that a project with high ex ante probability of acceptable returns end up delivering an unacceptable one ex post.

For this reason, a low ex post return is not necessarily proof of a wrong ex ante decision. To show that in regards to public sector projects, which are usually evaluated under the assumption of risk neutrality, one would have to show that with the information available at the time the expected value of the return indi-cator (typically NPV) was unacceptable. This is a more difficult task, because one would have to find out what information was available at the time when the original CBA was conducted. (This comment refers more to the forecast of variables required in the analysis, rather than to the methodologies of the origi-nal analyses, which are easer to qualify.)

The current study has applied both types of analysis, but with a greater weight being placed on evaluating the actual consequence of having undertaken the project.

For all three project types analysed the comparison of the costs of alternatives has been a key element of the analyses performed. For each project, the follow-ing steps were undertaken:

1. Determination of alternatives that were considered ex ante, and establish-ment of whether they were reasonably well chosen.

2. If not, then suggestions for reasonable alternatives to be used in the analy-sis.

3. Identification of actual costs of the project. If data were unavailable or in-complete, best estimates were used.

4. Estimation of the cost of actual or supposed alternatives.

5. Acquirement of data on utilization rates (or demand for the services) of the projects; this cannot be replaced by estimates. However, the utilization rates (or demands) for the remaining useful time of the projects were esti-mated.

Methods



48

.

6. Similarly determination of operating costs of the projects, but their future values were based on an estimate.

Many of the estimates alluded to above have been based on engineering esti-mates. They have been based on reasonable alternatives, taking acceptable quantities of materials and local unit costs values. Nevertheless, they will be subject to error.

2.5 Margins of error of the CBAs performed. Even though actual project data were available in all cases, it should be clear from the foregoing discussion that the ex post CBA analyses rely heavily on estimates. To quantify the consequences of using estimated data on the conclu-sions of the analyses, the Monte Carlo simulation method has been used. This has combined the uncertainties involved leading to the establishment of confi-dence intervals for the conclusions.

To this end, the following steps have been followed:

1 Computational models were constructed for the three project types, allow-ing enough flexibility in them to accommodate variations in project struc-ture.

2 Sensitivity analyses were conducted for each project to identify the vari-ables with greatest impact on the results of the analysis.

3 Margins of error were estimated for all estimations made, and are repre-sented by probability distributions. Very careful attention was given to choosing the margin of error of variables ranking high in the results of the sensitivity analysis. Likely correlation between variables was taken into account.

4 Using the Monte Carlo method, the probability distributions of the project indicators were obtained, providing for margins of error for the conclu-sions.

5 The expected opportunity loss of each alternative was computed when more than one alternative to the situation without the project could be quantified.



49

.

3 Study methodology 3.1 Study objectives The Terms specify the objective of the project in a very clear manner; it is "to carry out ex post cost-benefit analyses for selected environment projects fi-nanced by the Cohesion Fund and ISPA, and to learn from these analyses for the current and future programming periods". This objective is elaborated on through the three fundamental questions, which are:

1 What were the impacts of the examined projects?

2 How can ex post cost-benefit analyses contribute to the practice of ex ante cost-benefit analyses?

3 What are the potentials and limits to carry out an ex post cost-benefit analysis to identify and/or analyse the impact of projects? Is it an appropri-ate tool for impact analysis?

We acknowledge that these three fundamental questions in practice imply a double objective of the project: the first objective is to deliver actual CBA re-sults for ten environmental infrastructure investment projects, while the second objective is to deliver an assessment of the usefulness of the CBA method in the given context - and in this context, to provide recommendations for its fu-ture use. The following figure illustrates this understanding of the project ob-jectives.

Two objectives



50

.

Figure 3-1 Project overview

The figure highlights the preparation of CBAs for 10 projects (the first funda-mental question) and the assessment of CBA as a method (the second and third fundamental questions). The project is proposed to consist of four tasks - Tasks 0-3 - where the two last tasks are crucial to the preparation of CBA of 10 pro-jects and also to the assessment of CBA as a method (see Chapter 5 for further details of the tasks).

3.2 Study tasks As presented above in Figure 3-1 Project Overview, the study was proposed to consist of the following four major tasks:

• Task 0: the inception phase,

• Task 1: selection of 10 major environmental projects,

• Task 2: the Ex Post project analysis

• Task 3: the assessment of the CBA method.

The following chart displays the major outputs and deliverables expected from these four tasks.



51

.

Figure 3-2 Major outputs from the four tasks

3.3 Selection of projects

3.3.1 Selection of the 20 projects The selection of 20 projects was made in collaboration with DG REGIO with additional input from geographical desk officers.

The process The selection process involving the 20 projects was initiated already in the proposal, where an initial list was suggested. This list was based on a number of objective selection criteria to ensure a representative sample and even split between geographical regions, subject of project, size of project , maturity of project as well as projects stemming from 'new' or 'old' Member States. The following table provides an overview of the spread of these characteristics on the initial 40 projects.



52

.

Table 3-1 Overview of the 40 projects characteristics

Country Topic Size of project BG-Bulgaria 2 Solid waste 9 <35 10CZ-Czech Rep 1 Water 31 35-50 12EE-Estonia 1 50-75 12GR-Greece 2 >75 6ES-Spain 15IE-Ireland 2 LV-Latvia 2 LT-Lithuania 1 Maturity Old/New MS HU-Hungary 2 Completed 7 Old 22 PL-Poland 3 Not completed 33 New 18PT-Portugal 3 RO-Romania 1SII-Slovania 2SK-Slovakia 3 Total 40

At the kick-off meeting, 20 projects were selected, and in cooperation with the Steering Group – including geographical desk officers – the list of 20 projects was revised incorporating the experiences of the projects. The final list of 20 is presented in Table 3-2.

A common template was developed and applied for the brief descriptions of the selected 20 projects.

During the inception phase, two projects, the Hungarian and Slovenian ones, were selected as test projects for developing the project profiles. The results of this work were two profiles that were submitted together with the inception re-port and were approved as the point of departure for the remaining 18 projects. Based on the development of these two profiles, the project description tem-plate was refined incorporating details requested by the Commission.

Each of the twenty projects was briefly described based on available data from DG REGIO and data available on project websites or from other media. The 20 brief descriptions constituted a separate deliverable, which is available at the DG REGIO website4.

During the work with the 20 profiles, personal contacts (email/telephone) were established to each of the projects. In some cases, contact was easier to estab-lish than in other cases. The following table presents an overview of the 20 pro-jects.

Project no. 34 from Portugal is highlighted, as it is eliminated from list of the final 10, the reason being that the project had already been the focus of a thor-ough internal evaluation.

4 D2 Project description of 20 projects, http://ec.europa.eu/regional_policy/sources/docgener/evaluation/expost2006/wpc_en.htm

Template for project descriptions

Initial contact to projects

http://ec.europa.eu/regional_policy/sources/docgener/evaluation/expost2006/wpc_en.htm




53

.

Table 3-2 Overview of the 20 projects

A brief description of each of the 20 projects (D2) is available online at the DG REGIO website5.

5 http://ec.europa.eu/regional_policy/sources/docgener/evaluation/expost2006/wpc_en.htm





54

.

3.3.2 Selection of the 10 projects In collaboration with DG REGIO and based on the outcome of the brief de-scription of the 20 projects, a final list of 10 projects was drawn up. The selec-tion was made based on availability of data, considerations regarding the split of thematic projects, a good split between relevant Member States as well as the inclusion of the largest of the projects in terms of funding. An overview of the final list of ten projects is provided below.

Table 3-3 Overview of the ten projects selected for in-depth analysis

Project no. Country Ref. Title Sector

01 Bulgaria 2000BG16PPE002 Set of 6 Regional Waste Disposal Sites (Mon-tana, Ruse, Pernik, Sevlievo, Silistra, Sozopol)

solid waste

03 Czech Rep

2004CZ16CPE003 Plzen: Expansion of the water distribution and sewer infrastructure

Combined wa-ter and waste water

06 Greece 2000GR16CPE001 Gestion de déchets-Chania solid waste

09 Spain 2000ES16CPE035 Abastecimiento con agua a Zaragoza y corre-dor del Ebro

water

13 Pilot

Spain 2001ES16CPE012 Gestion de Residuos en Madrid-2001-grupo2, solid waste

17 Spain 2000ES16CPE060 SANEAMIENTO BARCELONA 2000-2004 water

22 Pilot

Ireland 2000IE16CPE001 Dublin regio Wastewater Treatment Scheme-Stage 5

water

27 Hungary 2000HU16PPE003 Szeged - establishing waste water treatment system

water

29 Poland 2000PL16PPE016 Sewage disposal for Szczecin water

50 Portugal 2002PT16CPE002 LIPOR - SERVIÇO INTERMUNICIPALIZADO DE GESTÃO DE RESÍDUOS DO GRANDE PORTO

solid waste

The Irish project IE22 and the Spanish project ES13 were selected as pilot pro-jects. The two pilot projects were submitted to DG REGIO as the first interme-diate report.

An interview guide was drafted and tailored for each of the two pilot projects. The guide was elaborated on following the two pilot projects and applied to the eight case studies.

3.4 Information collection approach for case studies Data collection and information retrieval to support the assessment of the pro-jects in terms of background information, ex ante analysis and ex post CBA were done in different tempi and by different means.



55

.

Figure 3-3 Information collection approach

The basis for making a good judgement clearly relies on the availability and accessibility of reliable data. Our approach to data collection is briefly summa-rised below. First, available data from the relevant DG REGIO Geographic desks were obtained. Based on this and complemented with the above-mentioned brief project description, a first perception and understanding of the project was formed. Site visits to the projects provided in-depth familiarity with the project. At site visits, additional operational data were collected and an un-derstanding of the wider felt impacts of the project to the local community and the region was gained. During the subsequent analysis, dialogue with the pro-ject owner, operator or other stakeholders were continued if additional data or clarifications were needed.

3.5 Steps of applying the CBA tool In accordance with the CBA Guidelines, the following briefly introduces the steps of applying the CBA tool.

Project identification It is essential to define the new infrastructure as a self-contained unit to which capital costs and operating costs and revenues can be attributed. The cost bene-fit analysis will also concern this unit. However, possible linkages with other infrastructures may affect the operation of the new infrastructure unit, and if so, they should be addressed in e.g. the economic analysis (see below). Further-more, the analysis must ascertain whether the definition of the new infrastruc-ture is the same as the one planned at the ex ante stage. If this is not the case, the analysis should reveal why this is so. Finally, the management setup for op-erating the new infrastructure unit should be clarified.

Project feasibility and alternative options The main question is whether the utilisation of the new infrastructure is ade-quate or whether there is excess capacity or excess demand; and if inadequate, which demand assumptions made during the ex ante analysis failed. Further, it will be assessed whether the environmental requirements have been met. An-



56

.

other question to address is whether a different option would have been chosen (ex ante) if the present (ex post) demand had been known. In this context, it might be assessed whether a more flexible technical option would have been feasible - i.e. an option that would allow more timely adjustments to changes in demand.

Financial analysis The financial result might differ ex post from that expected at the ex ante stage, and so the degree of financial sustainability may have changed. This might be due to changes in the definition of the infrastructure unit, in the demand situa-tion, or in capital and operation costs as well as in revenue levels via tariffs. In this context, it is desirable to establish the actual (ex post) unit costs at a disag-gregated level - i.e. 'level 3 costs'.

Economic analysis The economic analysis comprises the bulk of the ex post cost benefit analysis. It contains a presentation of the type of economic analysis applied, and of how cost and revenue elements included in the financial analysis have been con-verted to socioeconomic values and how other socioeconomic benefits (and costs) have been quantified via available data to enable calculation of the eco-nomic return. Furthermore, costs and benefits that cannot be expressed in monetary terms are treated in a qualitative manner. This also includes the analysis of unintended effects as well as accompanying measures, which are outside the project but intended to enhance the project success.

Risk assessment The risk assessment comprises both the calculation of the margin of error of the ex post costs benefit analysis and an assessment of risk mitigation measures applied during the construction and operation of the new infrastructure - here-under the consequences of mitigated and (both foreseen and unforeseen) non-mitigated risks.

3.6 Introduction to the ten selected projects The following section gives a brief introduction to the ten selected project case studies.

The ten projects group into three distinct types of environmental infrastructure projects: waste management (4 projects), water management (2 projects) and wastewater management projects (4 projects of which two also have elements of water supply).

This grouping is mapped in the figure overleaf.



57

.

Figure 3-1 Groupings of the 10 projects

For further details on the ten projects, the reader will find at two-page fact sheet for each of the projects in Appendix 1, while the full analysis of both the ex-ante and ex-post CBA for each of the 10 projects are available on the DG REGIO website6.

6 http://ec.europa.eu/regional_policy/sources/docgener/evaluation/expost2006/wpc_en.htm




58

.

4 Results of ex-post assessment 4.1 Results of ex ante analysis

4.1.1 Overview of ex ante context of the 10 projects The table below summarises across the 10 studies the main ex ante situation and features.

Table 4-1 Overview of ex ante context

CASES Lega

l com

plia

nce

Envi

ronm

enta

l

Hea

lth

Risk

miti

gatio

n

Oth

er

Self

cont

aine

d

Upg

radi

ng e

xist

ing

infr

astr

uctu

re

Part

of

'Mas

ter P

lan'

Ver

y re

asan

able

Reas

onab

le

Wea

k or

mis

lead

ing

Hig

h qu

ality

Med

ium

qua

lity

Poor

qua

lity

Hig

h qu

ality

Med

ium

qua

lity

Poor

qua

lity

01 Bulgaria/Waste Mngt X X X X X X X03 Czeck Rep/Pilsen: Combined water X X X X X X X06 Greece/Crete: Waste Mngt X X X X X09 Spain/Zaragoza: Water supply X X X X X X X13 Spain/Madrid: Waste/Energy recovery X X X X X17 Spain/Barcelona: Waste water X X X X X X X22 Ireland/Dublin: Waste water X X X X X X X27 Hungary/Szeged: Waste water X X X X X29 Poland/Szczecin: Waste water X X X X X X X X50 Portugal/Lipor: Waste management X X X X X

8 6 1 1 2 4 3 8 2 6 2 2 7 1 3 2 5

Main driverEx ante

assumptions

Quality of ex ante financial

analysis

Quality of ex ante

economic analysis

Project context

Although legal compliance has been the main driver for initialising the projects in all cases but Barcelona and Zaragoza cases7, the implemented projects were aimed at solving significant problems of upgrading the existing obsolete infra-structure and improving the quality of primary services delivered to the popula-tion.

7 other considerations such as City image in connection with the Olympic Games and flood risk mitigation were the main issues for Barcelona and water supply in the case of Zaragoza

Legal compliance as the main driver for project initialisation



59

.

As the assessment of the projects allowed for more than one driver, more than half of the projects also have the environment as the driver. Since the projects all have the objective of improving environmental infrastructure, this is not surprising.

MAIN DRIVER

Legal compliance 8

Environmental 6

Health 1

Risk mitigation 1

Other 2

No of projects

It appears that many projects are taken out of a context, and that they are part of a larger 'Master Plan' that contributes towards meeting national legislative re-quirements, e.g. a local or regional wastewater treatment plan or a waste man-agement plan.

Once more, it was possible to choose more than one category rendering a com-bination of being a part of a master plan and of upgrading of existing infrastruc-ture possible. Again, it is possible for a project to be in more then one category.

PROJECT CONTEXT

Self contained 4

Upgrading existing infrastructure 3

Part of 'Master Plan' 8

No of projects

The ex ante assumptions were seen as being relatively reasonable at the time and also the quality of the ex ante financial analysis were relatively acceptable although some flaws and miscalculations were observed in a few cases, e.g. in the Bulgarian and Zaragoza cases. The quality of the ex ante economic analysis varied considerably from case to case, and only the Plzen, Dublin and the Hun-garian cases included a well-founded economic analysis.

EX ANTE ASSUMPTIONS

Very reasanable 2

Reasonable 6

Weak or misleading 2

No of projects

Most of the financial analyses were of reasonable quality (sum of medium and high quality). One of the projects - the Bulgarian case - had submitted a finan-cial analysis of poor quality, where explanations and information about the value of the parameters were missing. In addition, the calculation of the total FNPV and FRR were missing in the analysis.

Project context

Quality of ex ante material



60

.

QUALITY OF FINANCIAL ANALYSIS

High quality 2

Medium quality 7

Poor quality 1

No of projects

When it came to the quality of the economic analyses, the result was somewhat poorer. For some of the projects, an economic analysis had never been made. Others yet had not provided the figures used or had not explained or justified figures used

QUALITY OF ECONOMIC ANALYSIS

High quality 3

Medium quality 2

Poor quality 5

No of projects

4.1.2 Ex ante Financial and economic indicators The key financial and economic ratios for the ex ante situation are summarised below.

Table 4-2 Key financial and economic ratios- ex ante



61

.

The ex ante analyses of the projects yielded very different results. Some pro-jects had a positive FNPV incl. CF, whereas others had a negative FNPV. If a project had a positive FNPV that exceeded the cohesion funding, it would have been critical, as the project would have been profitable in itself. This has not been the case, and based on this criterion, all projects seemed as good invest-ments. In addition, the FRR indicated the quality of the investments, and the conclusion drawn is similar in this case. In some cases, the original ex-ante in-dicators shown in the table below needed to be re-calculated in order to delete some methodological inconsistencies or mistakes encountered in the project dossier. This is the case for example of the Zaragoza project, where deprecia-tion costs were included as outflows, thus overestimating financial costs. In the same way, likely double counting of some benefits occurred in the ex-ante eco-nomic analysis, thus leading to highly positive results.

Figure 4-1 Ex ante CBA results for the 10 projects

-400

-300

-200

-100

0

100

200

300 Total inv costs (MEUR)

FNPV

ENPV

When looking at the results of the economic analysis, it is worth noting that there are cases where the ex-ante economic analysis was not existent or showed close to zero and even negative results (the Barcelona project). Only the Zaragoza, Madrid and Dublin projects turn out to be very good investments. However, one should remember that the quality of many of these analyses was poor, and consequently results are less robust and reliable. The Barcelona pro-ject featured one of the best economic analyses. The same conclusions can also be drawn based on the B/C ratios. The very positive ENPV in the Zaragosa and the Dublin cases can be found in the valuation of the benefits. In Zaragosa, the positive result is due to the high willingness to pay for drinking water, where



62

.

the reason for the positive result in Dublin had to do with including and esti-mating benefits that were not likely to materialise.

4.2 Ex-post assessments This section provides a picture of the ex-post assessments of the case studies.

4.2.1 General conclusions overview In the light of the results of the ex post analysis, it can be concluded the follow-ing:

• Only two projects (wastewater treatment in Zaragoza and two components of the Portuguese case) showed net positive impacts in terms of improved quality of drinking water supplied, improved efficiency in the management of the service (savings in purification) and environmental effects (health effects). Such net positive results are shown by a positive ENPV and sup-ported by the widespread perception of the project promoters, decision-makers and the citizens.

The remaining eight cases all provided more or less negative results due to dif-ferent reasons.

• In four cases (Czech Republic, Barcelona, Poland, Dublin) the projects im-plemented showed positive impacts in terms of better services delivered to the population and impacts on the natural environment, with also wider ef-fects in terms of recreational activities promoted, environmental awareness raised and institutional learning. Such positive results, were obtained at high costs, however, resulting in negative ENPV and B/C ratio lower than 1. There is a general perception that some relevant impacts were not cap-tured by the quantitative CBA, but whether those would suffice to alter the balance of the quantification obtained is unknown.

• In four cases (Hungary, Bulgaria, Madrid, Crete), the results of the CBA and the perception of the citizens and stakeholders support the argument that the projects under assessment produced a stream of benefits, which however did not outweigh their costs. Reasons for that are to be found in technical problems of overcapacity (Madrid), problems with demand fore-casts (Crete) or technical solutions to comply with environmental standards deemed too costly for the actual needs (Hungary and Bulgaria). Hence, the mere compliance with the environmental directives do not necessarily lead to evidenced positive impacts in net welfare terms.

Some of the projects in the sample were undertaken to comply with European regulatory requirements and in all cases to provide or enhance basic environ-mental infrastructure. In most cases, this was achieved at costs that exceeded the benefits that could be quantified, however.

It is conceivable that in some cases better evaluation methods might have re-sulted in the identification of benefits that would have contributed to attaining a positive balance, but this is unlikely to hold true in all cases.

Positive net impacts for some but not all projects...



63

.

Given that CBA was only undertaken at the very end of the project planning process, it is not surprising that some of the projects have negative ENPVs. They were not designed from the beginning to attain positive ENPVs.

In the case of projects that were undertaken to meet regulatory requirements, it might be impossible to achieve a positive ENPV. In such cases, the CBA result shows the net welfare loss caused by regulatory compliance. (In some cases, the regulation provides for exemptions when the cost of compliance is high, but this was not invoked in any of the cases analysed.)

Table 4-3 Ex-post general conclusions

Legal compliance as driver for

project initiation

Technical solution

Capacity issues

Budget reasonabili

ty

Number of citizens affected

Risk assessment

01 Bulgaria/Waste Mngt YES OK OK OK 228.000 The main challenge is affordability because of the low income level

03 Czeck Rep/Pilsen: Combined water

YES OK OK OK 170.000 Overall a sound project

06 Greece/Crete: Waste Mngt

YES OKOvercapacity (composting

facility)OK 80.000

Financial sustaianability an issue. Operations need to focus on improving recycling rate and market uptake of compost

09 Spain/Zaragoza: Water supply

not only OK OK OK 821.000 No major risk elements were identified

13 Spain/Madrid: Waste/Energy recovery

YES OK Over-capacity OK 3.000.000 No major risk elements were identified

17 Spain/Barcelona: Waste water

NO OK OK NO 130.000

It was concluded that the project is very likely never to generate an economic surplus. Most important variables were investment costs, value of beach days, avoided damage costs and the discount rate.

22 Ireland/Dublin: Waste water

YES Not optimalUnder-

capacityNO 1.145.000

The project is very unlikely to generate an economic surplus

27 Hungary/Szeged: Waste water

YES OK OK NO 170.000

The financial risk that actually materialized was an investment cost overrun. The economic feasibility of the sewage network expansion is subject to considerable uncertainty due to lack of supporting information. The WWTP expansion is not economically feasible with a high degree of certainty.

29 Poland/Szczecin: Waste water

YES OK OK OK 400.000 The project is both financially sustainable and likely to lead to an economic return in the future

50 Portugal/Lipor: Waste management

YES OK OK OK 1.400.000 The composting facility is the main reason for poor economic performance of the overall investment

General Characteristics

However, the types of risks facing the operational phase of the projects are of very different nature. The Plzen, Zaragoza and Madrid cases can be said to be rather sound projects with no major risk elements identified. In the Bulgarian case, the affordability of the citizens or indirectly through the municipalities to pay for the operation of the new landfills constitutes an actual concern, also since the population basis is decreasing. In the Crete case, there is also an issue on financial sustainability. However, improved operations and investment into accompanying measures, such as training of citizens to improve waste sorting

...with merely legal compliance not nec-essarily leading to positive impacts in economic welfare terms.



64

.

at the level of households and awareness campaigns towards the agricultural sector, could improve the resource efficiency and hence increase the revenue generating activities of the waste management plant. The Hungarian case in isolation is also a relatively sensible project, however, the problem here is that as long as the neighbouring countries do not observe the same high wastewater standards, the overall impact on the river water quality is limited.

4.2.2 Key financial and economic ratios - ex post The key financial and economic ratios for the ex post analysis are summarised below.

Table 4-4 Key financial and economic ratios - ex post

Investments have increased compared to the ex-ante results due to budget over-runs. These overruns had many different reasons, and often they could have been avoided if project preparation had been better. Note that some ex-post CBAs are only made for some of the components and that they are therefore not directly comparable with the ex-ante results.

FNPVs are small or even negative. The results are much more negative than the results of the ex-ante analyses. That means that among these projects, it was not possible to identify any good business cases, which directly justified the need for external funding.



65

.

Figure 4-2 Ex post CBA results for the 10 projects

The ENPVs of the projects are more positive in the ex-ante analyses than in the ex-post CBAs. This is mostly because the quality of the ex-ante economic analysis was poor rendering results unreliable, or because of an optimism bias in the ex-ante analysis. Given the fact that in some cases the ex-ante methodol-ogy had to be significantly revised due to mistakes or inconsistencies (as for the Zaragoza project), ex ante and ex post results are not comparable.

The fact that performance indicators of water supply, water management and solid waste project are usually not particularly high is, however, not surprising. Empirical analysis shows that the normal profitability for such projects is on average quite low as compared to other sectors8.

The results from the CBAs carried out in this project provide positive as well as negative NPVs as just mentioned. The result of the NPV should be interpreted as the sum of the positive (income/revenue) and negative (cost) effects of the project over time (cash flow). When a project has a positive NPV it means that it is a solid business case and that private companies should be willing to invest in the project as a profit is expected over time. When a negative NPV occurs, it is related to the fact that the costs of the project exceed the income over time. This is likely to happen in projects where a rather big initial investment is made and the expected income turns out to be much lower. When the loss of income exceeds the expected profit of the project, the NPV will become negative. If the result is very positive without Cohesion funding, one should consider if the funding is necessary for the project. On the other hand, a negative NPV does not mean that the project is not a good investment for DG REGIO, however, it

8 On the international comparison of rates of return see Del Bo, C. and Florio, M. (2010), Cost-benefit analysis and the rates of return of infrastructure projects: evidence from inter-national organizations, in Transition Studies Review, (2010) 17: 587-610,



66

.

would be advisable to consider the sustainability of the project, which could be done through a proper CBA.

Figure 4-3 Ex post B/C-ratio

Finally, the ex-post B/C ratios show that very few of the projects have a ratio above 1. The low ratios can partly be explained by the fact that a number of the benefits have only been qualitatively described in the CBAs due to lack of benefit estimates, e.g. Barcelona – improved marine water quality.

Another factor that can explain the low rations is the fact that a number of pro-jects have experienced overcapacity, which generates a relatively large invest-ment compared with the benefits that can be realised in the short term.

The fact that the projects are part of a larger set-up might result in the assign-ment of less benefits directly to the project, e.g. the Hungary project only cov-ers one section the River Tisza and the Bulgarian project is part of a master plan for national waste handling.

The B/C ratio expresses the division between the benefits and the costs related to the project. A B/C ratio above one indicates that the benefits exceed the costs related to the project and that the project provides a positive net benefit to that society from a socio economic view. When the B/C is lower than one, which is the case with some of the projects included in this analysis, it means that the project costs more than it provides through benefits to society. Often it can be very challenging to capture and monetise benefits, which implies that some are left out, not accurately valuated or only qualitatively described. When a project is a part of a master plan, it can be difficult to isolate the benefits of one inter-vention and include this in a CBA. This can lead to underestimation or even that a benefit is left out.



67

.

4.3 Comparison of ex post with ex ante results As the results of the projects differ, comparison is challenging. In section 4.2 above, some of the results from the ex ante and the ex post analyses are com-pared. The following main differences should be kept in mind:

• The different objectives for carrying out the ex ante and the ex post analy-ses

• Lack of expertise in valuation of benefits

• CBA is not a tool used in the project and therefore the quality requirement is limited.

4.4 Wider project outcomes The wider project outcomes are summarised in the table below.

An attempt to score the degree to which the project objectives have been meet has been made using a score 1-5, where 5 is the highest.

All projects are found to have fulfilled their objectives. For some projects, such as Plzen, a score 4 has been assigned as it is still too early to assess whether the retention tanks really can limit the number of overspills into the river. In the case of the Crete waste management project, a score of 4 has also been given due to the amount of waste still ending up in illegal fills although the project as such already has made considerable progress towards establishing a modern waste management operation in the Chania region of the island. For Bulgaria, the project has yet to be finalised.

Project objectives are more or less achieved



68

.

Figure 4-4 Wider project outcomes

Project outcomes

Fulfillment of objectivesScore 1-5

Financial sustainability Direct benefits Unrealised benefitsHindrances for realizing

benefits

Other project outcomes/unintended

effects

01 Bulgaria 4

Only the operational cost can be financed by the waste management tax No re-investement are included for e.g. sealing of the cells

Reduction of illegal waste disposal _ _ A first step toward a modern waste handling management system

03 Czeck Rep/Pilsen: Combined water

4

Yes due to the tariffs and since the affordability analysis of the total tariffs showed no risk of payment problems among household consumers.

Much improved quality of drinking water + reliability of supply of tapped waterReduced pollution of the river due to reduced risk of overspill of sewage systemConnection of approximately 1000 households to the central sewage system

Upgrading of the river environment (underway)

Perhaps slowed down now due the financial crisis

Enabled other projects to emerge, e.g. GreenwaysInstitutional learning, e.g. establishing of UKEP

06 Greece/Crete: Waste Mngt

4UncertainOP cost much higher than budgetRevenues lower than budget

Reduction of illegal waste disposal of 90000 tonnes of waste a yearRecycling of 30.000 tonnes of waste a year

Use of compost for soil enrichmentReduced use of pesticidesRefined sorting of waste100% waste management coverage

Lack of awareness among farmersLack of education of households to sort waste at sourceLack of reinforcement for illegal waste disposal

Waste management know-how to benefit rest of CreteIncreased environmental awareness among citizens

09 Spain/Zaragoza: Water supply

5The project is sustainable thanks to full cost recovery tariffs Significant improvement in water quality _ _

Reduction of potabilization treatment costs for municipalities. the project has also indirectly induced the municipality of Zaragoza to undertake actions that have radically reduced water losses and modified the behavior of the users, with beneficial effects on environment and safety of water supply.

13 Spain/Madrid: Waste/Energy recovery

5

The energy generation component generate a surplus. While the other components require operational funding form the municipality.

Closing off illegal landfill - stopping negative environmental impactsEnergy generation

Unutilized synergies (energy plant) with other facilities of the combined Madrid management facility

Contractual bindings of current contract with different operators

An important part of the combined Madrid Waste Management facility



69

.

Project outcomes

Fulfillment of objectivesScore 1-5

Financial sustainability Direct benefits Unrealised benefitsHindrances for realizing

benefits

Other project outcomes/unintended

effects

17 Spain/Barcelona: Waste water

5

Based on the financial indicators, the project is not sustainable. No income is generated. However; the economic assessment shows a sustainable investment.

Reduced risk of flooding (giving positive effects on quality of bathing water, traffic congestion and reduction of damage costs)Improved know how of water management. Qualitative benefits includes access to ground water for irrigation and street cleaning, improved ecosystem in marine water, public information activitiesand cleaner streets.

_ _

A lot of knowledge and experience has been gathered. The project has many visits from delegations from other cities and countries. There has been a lot of awareness rising among the population.

22 Ireland/Dublin: Waste water

3

The Project capacity is based on the information from the time of the application - and now it requires an upgrading to cater for unexpected increased demand - therefore it is not financially sustainable and requires the identification of further investment funds. Further, a revision of the user fee structure is needed.

The direct benefits include an increase in housing permissions and recreational benefits, in particular in connection with cleaner Dublin Bay beaches.

_

The design capacity of the new infrastructure is too low and it does not comply with the requirements of the newly designated sensitive waters. Hence, further investments are needed to fully reap the benefits.

_

27 Hungary/Szeged: Waste water

5

The project is financially sustainable despite its poor financial results, attributable to the high capital costs. But the fixed assets are not in the books of the operator. Tariffs are sufficent to cover operating and reinvestment costs and are inflation indexed.

The direct benefits are the extension of the sewage system to new users, who therefore avoid the cost of dealing with their own individual sewage solutions.

The objective of a cleaner Tisza River is unrealized because the dillution effects of the river are so strong (2400:1). The effects of the project might be more directly felt in the future, when incoming river water will be cleaner (icluding that of the Maros, which arrives from Romania)

_The project is excellently run. There has been some awareness rising among the population.

29 Poland/Szczecin: Waste water

5

The project appears to be financially sustainable as user fees have been allowed to increase in line with the increase in the quality of service provision.

Direct benefits to the inhabitants of Szczecin due to improved quality of waste water and water supply services, and from improved environmental conditions of the Oder River.

Since the new infrastructure has only been in operation for aroumnd a year, it is still early days for estimating e.g. possible impacts on business and housing developments.

_ Szczecin is no longer a HELCOM hotspot

50 Portugal/Lipor: Waste management

5

Financial sustainability is insured either by green or "guaranteed" administrative prices and close control of costs

Comprehensive and efficient management of urban solid waste generated in the second largest conurbation of the country with a population in excess of one million inhabitants

-

Usual implementation delays of a fairly large and multisectoral investmentAcceptance of compost quality by farmersAwareness to waste source separation + recycling of the public at large

Not detected



70

.

4.5 Typical components, benefit items per type of project and examples of calculation externalities

The following sections summarises the typical types of financial inflows and externalities as have been observed across the three different types of projects, e.g. waste management, waste water treatment and water management projects. Likewise, examples of how to calculate direct benefits and externalities are in-cluded.

4.5.1 Waste management projects Table 4-5 Waste management components and impacts

Waste management projects usually consist of one or more of the components appearing in the figure: restoration of illegal landfills, gasification facility, re-cycling or sorting facility, composting facility, construction of new landfill and biological or chemical treatment plant. Based on the cases studies, the table above summarises the main types of financial inflows for each of the compo-nents. The most common positive as well as negative impacts or externalities are likewise summarised for each of the components.

In the following, we have collected examples from the projects of calculations of externalities. Some of these examples originate from the original CBA, oth-ers have been drawn up as part of this ex post CBA exercise.

Reference: Madrid case; Component: sealing off landfill (done in connection with the ex post CBA)

Avoidance of landfill gas (LFG) emission

What is it worth that landfill



71

.

Landfill gas (LFG) contains about 55% methane, which is a powerful greenhouse gas. Over a 100-year time horizon, its greenhouse effect is 21 times that of CO2. In the absence of the project, LFG would seep out into the atmosphere. The project captures LFG and uses it to generate electricity, but for this purpose it does not matter if electricity is generated or not. It is enough for the methane to be burned. The project does indeed have the capacity to flare all of the gas collected, so doing so is a real, not just a theoretical, option. We es-timated that without the project, 470 million m3of landfill gas would have been released into the atmosphere over 30 years. We attributed a plus/minus 30% margin of error to this estimation.

The basis of the quantification of this benefit is the value of traded CO2 emis-sion rights, often called the price of carbon. This market price reflects the cost of GHG abatement. Given that at this cost GHG reductions can be achieved, it is not efficient to do so more expensively. This is why the price of carbon is a good measure of the benefits of GHG abatement. We forecast this value with a lower bound of EUR 5/MT, a modal value of EUR 15/MT and an upper bound of EUR 30/MT. The expected value of this distribution is EUR 16.7/MT. This value was adjusted by the fraction of methane in landfill gas (55%) and by the CO2 equivalent factor of methane of 21. The present value of this benefit is expected to be EUR 49.8 million.

Reference: Madrid case; Component: landfill restoration (done in connection with the ex post CBA).

Assessing the economic benefits of the land-scape restoration implies considering land use change as influencing the natural resources. One could argue that from a sustainability point of view and in order to be environmental responsi-ble, one would be obliged to bring a piece of land used for a given productive purpose back to the same conditions as prior to using the land for a productive purpose, e.g. landfill9, but this does not guarantee a satisfactory CBA result. To derive conclusions suitable for an ex post evaluation, one would have to com-pare the value of this reclaimed area with that of others that could be developed in and around Madrid. This task could not be attempted within the constraints of this study.

A second way of valuing landscape restoration is to regard the land surface as a potential carbon sink. Obviously, a surface covered with vegetation, trees, shrubs, etc would have a carbon sink value as opposed to an artificial surface. But this value is unlikely to be significant.

A third way of evaluation the value of landscape restoration could be the mar-ket price of agricultural or forest land. But again, the access restrictions in place, as well as the fact that other landfills continue to operate nearby, make 9 Ref: TEEB study, http://www.teebweb.org/ForPolicymakers/tabid/1019/Default.aspx

gas no longer is emitted di-rectly to the atmosphere?

Land-use opportu-nity benefits

How to value the benefits of landscape

restoration?



72

.

finding an adequate market value very difficult. On the other hand, there are examples that old landfill areas have been used for construction of airports (Oslo, Norway).

Reference: Crete case; Component: composting facility (Done in connection with the ex post CBA).

The lack of market uptake of compost for ag-ricultural purposes can be regarded as an un-realised benefit. The potential savings for the farmer has been calculated using market prices for both water for irrigation, fertilizer and compost. There are externalities con-nected with the use of fertilizer as it has negative effect on soil biodiversity as well as contribute to ground water contamination. There are positive external-ities with applying compost to the soil as the soil increases its ability to absorb moisture, hence reduce the need for water irrigation. This implies using market prices for water (for irrigation), fertilizer and compost. Net savings in the use of fertilizer and water for irrigation given that compost is used for the growing of olive trees.

Affordability Reference: Bulgaria case; Component: landfill (done in connection with the ex post CBA).

From an affordability perspective, the tax is justified in the project application, based on World Bank guidelines. However, it was clear from the site visits that the population finds the taxes paid for waste management to be on the high side. This was a com-mon acknowledgement among interviewees.

The salary levels in Bulgaria support this. Despite a large increase to an average monthly wage of EUR 32210 from EUR 150 at the time of application, it leaves a current yearly wage of approximately EUR 3,864 (2010 level).

In order to estimate the benefit, we have chosen to transfer a willingness-to-pay estimate based on an Israeli study11.

The calculation is based on the following assumptions:

• The transferred value equals EUR 3.77 per household per year. This value is derived from the Israeli value (USD 12.7) and is adjusted according to purchasing power parity and exchange rate. The estimate is from 2006, which is the same year as the first year of operation at the Russe site. This means that the value can be transferred without any adjustment in terms of time.

10 Exchange rate per day September 24th 2010, Danish National Bank 11 Aylon, Becker and Shani (2006) Economic aspects of the rehabilitation of the Hiriya landfill, Elsevier, Waste Management 26 (2006) 1313-1323

Environmental bene-fits of using compost

How to value the bene-fits of landscape resto-

ration?

What is the environ-mental value of using compost in agricul-

ture?



73

.

• Compared to the tax of about EUR 22, the willingness-to-pay estimate is quite lower.

• An average Bulgarian household produces about 1.5m3 waste at a willing-ness to pay value of EUR 3.77 per year. This is EUR 2.5/m3. Compared with the operating costs of the site, which are EUR 4.57m3, the willing-ness-to-pay is lower than the actual costs of operation.

• Waste are collected from 73,33312 households and taken to the Russe land-fill.

Based on the assumptions, the total willingness to pay is ENPV EUR 2.8 mil-lion over the 15-year period; that is the lifespan of the landfill.

Reference: Bulgaria case; Component: landfill (done in connection with the ex post CBA).

There is also a minor negative benefit from the new landfills. This benefit emerges because the new centralised landfill site is located up to 55 km further away compared with the old sites. The value of this additional transport can be valued as the cost of CO2 per km.

There are no transfer stations to handle waste. It was considered that transfer stations would not be cost-effective because the settlements from where the waste is transported are too small.

The calculations are based on the following assumptions:

• Total number of trips per year is 26,290. This is based on an average of five tonnes of waste per truck load divided by a total waste amount of 131,448 tonnes per year.

• The share of waste that will be transported the extra distance has been cal-culated based on the relative share of waste. In total 8% (5% and 3% re-spectively for two municipalities) has to transport the waste additional 15 km, and 3% have to transport the waste additional 55 km.

• The additional distance is based on the actual distance minus 5 km, which is assumed the average distance to the old waste dumps. The environ-mental cost of each additional km is EUR 0.03213

Based on these assumptions, the total negative environmental benefit is ENPV EUR 23,666 for the entire period of 15 years.

12 Base on the received amounts of waste from the different municipalities and the numbers of households in Russe (66.000). See table 1-3. 13 Danish Energy Agency 2010

Example of a nega-tive externality - CO2 emissions of waste trucks What are the environ-

mental negative im-pacts of collecting

waste?



74

.

4.5.2 Waste water projects In a similar manner, the table below summarises the typical, identified compo-nents of wastewater projects and the associated financial inflows as well as im-pacts.

Table 4-6 Waste water components and impacts

WASTE WATER Retention tanks Sewage network WWTP

Financial inflows None directlyPart of waste water fee/tariff

Part of waste water fee/tariff

Externalities

Positive

Reduced risk of floodingPositive recreational impactsImproved quality of river or sea water due to less risk of overspill from sewage network

Positive health impactsReduced need for septic tanks and hence costs savings at the level of the households if connected to the sewage network

Reduced pollution of rivers/seaReduction of pollution of soil, groundwater and surface watersPositive health impact Positive recreational impacts

NegativeIncreased risk of smell and noice close by

Typical Components and benefits

Wastewater projects usually consist of one or more of the components men-tioned in the figure: retention tanks, extension of sewage network allowing ad-ditional households to be connected to the central sewer system, construction of WWTP plant.

In the following, we have collected examples among the wastewater projects of calculations of externalities. Some of these examples originate from the original CBA; others were made during this ex post CBA exercise.

Reference: Barcelona case; Component: retention tank, but could also be any other of the waste water components. (done in connection with the ex post CBA).

The recreational value of the project includes the positive impacts on the life of people due to the improved bathing water quality. The improved quality is a result of fewer flooding occurrences causing system overloads, which in turn lead to less direct discharges of untreated sewage water into the sea. When sewage water is discharged into the sea, it causes both visually notable pollu-tion and exceeds limit values for water pollution.

Recreational value due to improved bathing water quality



75

.

In the ex ante study, the environmental benefit was estimated based on the differ-ence in the cost of treating water in the treatment plant and in the sea. The details of this estimate were not available to the consultants. As an alternative, we have used another approach to estimating the benefit.

The value of the improved water quality is estimated as the recreational value of more beach days where the sea is open to bathing. A number of assumptions have been made for this estimate:

• 3,500,000 people (including local people and tourists) visit the Barcelonan beaches every year during the season14.

• The season is from June to September, equalling 107 days.

• The recreational value of a day on the beach is estimated to be EUR 1615. This estimate is based on Lew & Larson (2008)16. The article reviews stud-ies estimating beach days to between USD 0.41 and USD 146.97; a range that represents the average of available academic studies. The figures have not been corrected for local differences, as it is a rough estimate, and a conservative estimate has been chosen to avoid overestimation.

• During the beach season, there is potentially three days with flooding inci-dents. This estimate is based on the yearly average of five floodings. Most heavy rainstorms occur during September and October. Therefore, half of the floodings can be expected to occur in the beach season.

• When sewage water is discharged into the sea due to flooding, it normally takes one day before the bathing water is back to its normal quality, i.e. one beach day is lost per flooding. This is most likely a conservative esti-mate, but one made to avoid overestimation of benefits.

Based on these assumptions, the total recreational value of more beach days is EUR 31.1 million in the period from 2000 to 2019. The net present value, dis-counted at 5.5%, is EUR 19.3 million.

14 Estimate provided by Spanish Authorities during site visit September 2010 15 Exchange rate EUR1 = USD 1.375 16 Law & Larson (2008) Valuing a Beach Day with a Repeated Nested Logit Model of Par-ticipation, Site Choice, and Stochastic Time Value, Marine Resource Economics, Volume 23, pp. 233-252

What is the economic value of a beach day?



76

.

4.5.3 Water projects Finally, for the water supply projects, the main components identified based on the case studies and the associated financial inflows and benefits items are pro-vided in the table below.

WATER SUPPLY Water tanks Acuaduct systemWater mains/Water network/Pumping

stationsWater treatment plant

Financial inflows

Externalities

Positive

Negative

Improved drinking water quality -> improved health conditionsReliability in supply

Typical Components and benefits

Part of drinking water tariff paid by the households and industry (incl. farming)

None identified

The value of the improved water quality enables the population with poor water quality of the health-oriented segment to drink tap water and save the difference between bottled water and tap water.

Reference: Zaragoza case; (done in connection with the ex post CBA).

The main benefit of the Zaragoza project is to provide drinking water of a better quality. The value of the willingness to pay for drinking water of a better qual-ity has been made on the following steps:

• The opportunity cost of drinking water is calculated on the basis of the al-ternative purchase of bulk water through tank lorries (backstop technol-ogy). Without the project, the municipalities would have continued to be supplied from the Ebro and the Canal imperial, and people would have consumed poor quality water. Alternatively, for having the same level of water quality, the bulk water should have been purchased on the market through tank lorries, at a given price.

• This price was estimated in 2000 by the project applicant at 0.15 Euro/m3. By comparing the value provided with data for similar services, it is possi-ble to infer that the estimate is credible and taken by adopting a conserva-tive approach.

• This value, converted in 2010 prices, is equal to 0.20 Euro/m3 and repre-sents the maximum price the municipalities were willing to pay for bulk water.

Example of willing-ness to pay for drink-ing water



77

.

• The benefit is calculated as the maximum price the municipalities were willing to pay for bulk water multiplied by the volumes supplied by acuaEbro S.A.

Over a time horizon of 27 years (25 years of full operation and 2 years of start up) the total benefit generated by the willingness to pay is equal to EUR 369,716,844 (discounted value EUR 180,180,397), accounting for 74% of the total projects quantified benefits.

Reference: Zaragoza case; (done in connection with the ex post CBA).

The positive externality on health provided by the improved water quality has been calculated following the guidelines of the Spanish Ministerial Regulation on Water Public Domains, according to which an economic value to the decon-tamination provided by the new system, measured by the difference between the contamination load carried by a unit volume of water without and with the new system, should be calculated.

A Unit of Contamination (UC) is defined as the contamination load deriving from 90 grams of materials in suspension and 61 grams of oxidizables carried by water supplied to one habitant per day. The formula is:

UC = K x V

where:

V= volume supplied (m3 per year)

K= coefficient that is positively correlated to the water quality since it depends on the degree of treatment necessary for water potabilization (from A3 to A1 level). Its values are given in the Annex IV, Reg. 894/1986.

The benefit in health is given by the formula:

(UC0 – UC1) x P

where:

UC0= contamination load before the intervention,

UC1= contamination load after the intervention,

P = price for Unit of Contamination.

Or:

(K0-K1) x V x P

where:

K0 = coefficient that corresponds to bad water quality (i.e. A3 level of treat-ment), supplied before the intervention. From Annex IV of Reg. 894/1986 its value is: K0 = 1.0 x 10-5

Example of health benefit calculation



78

.

K1= coefficient that corresponds to good water quality (i.e. A1 level of treat-ment), supplied after the intervention. From Annex IV of Reg. 894/1986 its value is: K1 = 0.1 x 10-5

V= volume supplied (m3 per year)

P = price for Unit of Contamination. This price has been fixed into the Regula-tion, amounting to 500,000 pesetas (equal to EUR 3,005.06). In 2010 prices, it is equal to EUR 7,800.71.

Over a time horizon of 27 years (25 years of full operation and 2 years of start up), the total benefit generated by the positive externality on health is equal to EUR 130,739,430 (discounted value EUR 63,715,470 ), accounting for 26% of the total projects quantified benefits.



79

.

5 CBA as a method This chapter is based on evidence from the case studies combined with evi-dence and input collected from the workshop held with Member States and pro-ject representatives to discuss the strength and weaknesses of the CBA method-ologies as applied by Member States, the effectiveness of CBA as a tool to sup-port project generation and project decision. The chapter will likewise report on the extent to which ex post CBA is practiced at project level as well as Member State level followed by a discussion of the appropriateness of the CBA as a methodology for evaluating project impacts at local, regional or national levels.

5.1 Strengths and weaknesses of the CBA methodologies applied by Member States

It is important to distinguish between project level, Master Plan level, national level and international level while looking at strength and weaknesses of how CBA methodologies are applied by Member States.

Based on our in-depth assessment of the 10 individual projects, it appears that CBA methodologies are not really used as part of project decision or project development. CBA is often outsourced to specialist consultancy companies. The drawback of outsourcing the conduction of the CBA is that some CBA thinking might be lost in the project and that the CBA becomes a mere 'ticking the box' for the Cohesion Fund application form. We have also experienced that the conduction of the CBA is not really done for the sake of the project, but rather because it is a requirement as part of the application for Cohesion Fund-ing.

It appears that many projects are often parts of larger master plans or strategic plans, where either a CBA or feasibility studies have been made so at least the CBA 'thinking', including option analysis, demand analysis, risk assessment has been taken into account. The master plans are not always a part of the individ-ual project or rather the individual projects are usually taken out of such larger contexts and the CBA thinking is lost.

Another issue is the timing, which implies that the master plan often comes a few years before the actual CF application, for which reason the reference year and some of the main assumptions might have changed.

Project level

Master Plan level



80

.

Figure 5-1 Timing of CBA

Some Member States (the Czech Republic) have developed CBA tools (the Ja-cobs model) to ease the CBA completion. This is, however, not meant to un-derpin the decision process, but rather to make the calculation.

Several Member states have developed national CBA guidelines.

In Spain, initiatives have been taken to establish indicators and parameters to ensure a harmonised approach.

When it comes to projects that have cross-border impacts, and such as is the case with the Hungarian Szeged wastewater project, cross-border effects are mostly omitted and are not taken into consideration while applying the CBA. This specific case highlights two important points. Firstly, that while evaluating impacts, it is important to look at the broader context, e.g. the entire catchment area. Secondly, that in ensuring the legal compliance, attention should be paid to the way in which environmental standards are achieved, the sustainability (with a time but also geographical scope) of such compliance and the way and extent to which compliance translates into socio-economic objectives.

One of the main reasons preventing the use of CBA to its full potential is the timing of the CBA. As has become evident in this study, projects are often taken out of a larger context and are often part of a larger 'Master Plan', hence the CBA is not integrated properly into the decision process.

The timing issue has been recognised in Hungary where efforts are now made to include CBA much earlier in the project development process.

The following quotation picked up from the Member State Workshop captures well how the CBA is used in practice:

"Decisions are mostly taken at a strategic level, so when the project will be im-plemented, then the CBA comes up with the result you want".

National level

International level

The timing of CBA was confirmed by Member states to be an issue

CBA not an inte-grated part of the decision making process…



81

.

Interestingly enough, it appears that the Member States do have access to CBA expertise as several specialised CBA consultancy companies have emerged to undertake the CBAs as required for the Cohesion Fund applications. It can be seen as a strength that the CBA expertise is there, although not always at the required quality, as proved by the ex ante re-appraisal undertaken within the present evaluation. The challenge is how to activate this capacity earlier in the process, to make the CBA thinking a more integrated part of the process of pro-ject definition and decision and to bring the quality of the available expertise at the required level.

5.2 Effectiveness of CBA as a tool supporting project generation and project decision of MS and EC

As explained in the previous sections, due to the present practice of applying the CBA, it is evident that the CBA is not used as basis for choosing among several alternatives. Nor is it used to its potential when fine-tuning the projects in terms of applying demand analysis as the basis for calculating the capacity of a given component or as the basis for identifying the main risk elements of a given project. In other words, the CBA is neither integrated in the project gen-eration nor in the decision process, but the CBA is rather the 'frosting on the cake' or a 'ticking the box' exercise done as part of the Cohesion Fund applica-tion.

Another interesting observation obtained from the case studies is that most pro-jects in fact are made up of several independent components each leading to different types of impacts. Despite this characteristic, the economic analysis, the ex ante economic CBA, was usually done at project level wherefore many impacts were not accounted for. Only by identifying the individual components the actual both financial flows and economic flows can be identified. The ex-ample of the Crete case is provided overleaf, Table 5-1.

Specialised compa-nies have emerged to conduct CBAs…

There is a need not only to look at the individual independ-ent components..



82

.

Table 5-1 Example of a project broken into components (Case: 06 Crete)

ComponentsA: Restoratation of the solid waste burial area

(illegal landfill)

B: Solid waste recycling facility

C: Compost facility of urban waste D: New Landfill E: Biological

treatment

Total: EUR 30 millions EUR 0 EUR 8.7 millions EUR 7 millions EUR 4.8 millions EUR 0.9 millions

Including EUR 9 millions for land aquisition, envrinmental protection work, building construction, trucks, etc

A precondition for funding

Two parellal conveyor belts: One for green bin waste with remains from first sorting of blue bin wasteCapacity 2*25 t/hr

Two compost facility lanes, each with a capacity of 20.000 tonnes of organic waste/year

Two landfill cells combnined capacity of 1 million tonnes, with expected lifetime of 40 years

Consist of several siloes for biological Sequence Batch Reactor. In a later stage extended to chemical process

Main outcomesOld illegal landfill is closed off

Creation of 40 permanent job positionsRessource efficiency due to recycling of paper, carboard, plactic and glass

Reduced need for both fertilizer and water for local farmers (mainly olive growers)

Aviodance of polution if waste alternatively were to to end up in illegal landfills, approx imately 65 thousands tonnes a year

The waste water envetually is clean enough to be used on site for irrigation and for spinkling internal roads to reduce dust

Revenue streamsRevenue generating

activities to be part of operational budget (financial analysis)

None Revenue of selling reycled material

Revenue of sales of compost

Revenue from various communes for waste handling

None

Socio-economic impact/externalities

(both positive and negative)

Avoidance of groundwater contaminationPolution avoidance in future (ground water, odor) due to termination of illegal landfilAvoidance of Landfill gas emmissionLandscape restoration/ecosystem benefits

Increased ressource efficiency (glass, aluminimum, metal, paper)

Net savings on fertilizer and irrigation costs if substituted with use of compost

Avoidance of polution if waste alternatively

was to end up in illegal landfills (e.g.

cost of cleaning illelegal landfills per

tonnes waste)Additional CO2

emmission of waste trucks

Proper optimization of the projects would have required separate analyses of these components, either independently, when feasible, or in different combina-tions of component elements, in the cases when the components interact. This type of design optimisation was generally not made in the ex-ante analyses ob-served, in which only global analyses were performed.

…but also at the big-ger picture…



83

.

In the example of the Madrid case, possible synergies with a separate project of the greater waste management facility remained unexplored as the two projects were not developed with that purpose in mind. Instead, two separate energy re-covery facilities were built; one to recover landfill gas from the old illegal land-fill and one to generate energy from the new landfills created.

In the example of Szeged, the Hungarian case, the wastewater plant was built to fulfil EU legal requirements, but due to the fact that the city’s effluent is dis-charged into a large river, the impact of the sewage treatment on the river’s wa-ter quality is small (barely at the threshold of measurement). The added fact that incoming waters are already polluted (some of it comes from an affluent coming from Romania) just exacerbates the lack of clear incremental benefits.

Figure 5-2 Project context and the bigger picture

A clear message from the Member State Workshop as expressed by several Member States was that although the CBA methodologies are interesting, they often appear more of an academic exercise, the reality often being very com-plex and difficult to depict with a set of Excel sheets. It has also been reported that the CBA do not always reflect the reality as things can change rapidly as new directives coming into place thus quickly outdating CBA studies. In re-sponse, several Member States called for a more simplified approach to the CBA.

Generally, lack of a common language and a shared understanding among pro-ject promoters, decision-makers at the Member State level and the EC about the need and use of the CBA is reported.

As to the application process, a need was expressed for better 'training' of those involved in reviewing the applications and in particular to use a more clear lan-guage (non-technical) and to make the process more transparent and to make communication between the beneficiary and the EC more smooth. Hence, more training at all levels would improve the common understanding of the use of CBA for project selection.

..to be able to iden-tify both potential synergies and poten-tial risks.

Member states shows scepticism about the usefulness of CBA…

..but more training at all levels might add to the CBA culture..



84

.

5.2.1 Recommendations for improving the effectiveness of CBA as a tool

Member States expressed an interest in having access to 'good practices of ex ante studies' as examples and learning for beneficiaries in undertaking ex ante CBA for own projects.

Such good practices could be supported by standard parameters for evaluating externalities. As there are 'standard' types of impacts across the same types of environmental infrastructure projects, e.g. methane emission, odours, etc., it might be beneficial to apply the same types of values against these parameters in order to compare across the same types of projects.

Some externalities are clearly very difficult to measure, and it could be useful to have access to reliable data instead of having to make surveys each time. This is in particular relevant when trying to assess citizens' 'willingness to pay' (WTP), e.g. access to updated EU data on WTP for the different types of envi-ronmental infrastructure projects.

While it is said at times that CBA is not necessary for compliance driven pro-ject, this argument should be strongly refused. In fact, environmental projects funded by the Cohesion Fund should not be seen as just mandatory expenditure in compliance with EU legislation, but as part of a development strategy (i.e. they must maximize the net social benefits).

First of all, as showed by the projects under assessment, legal compliance is often an initial driver but definitely not the end of the story, infrastructure de-velopment or upgrading often being an investment priority in order to promote regional development. Therefore a clear-cut distinction between purely compli-ance-driven projects and projects addressing actual environmental needs is not realistic. Moreover, such distinction between project appraisal adds to creating confusion about the methodology that should be consistent across member states, sectors, and funds.

Second, cost-effectiveness analysis provides no information about the benefits of projects, and unless done with CBA techniques might even get the costs wrong, by excluding externalities from consideration. If the social benefits of a project are not calculated according to a standard CBA, mere compliance with the legislation could be a suboptimal option. In some cases, in a given regional context, the environmental standards would needed to be exceeded, or rather to be complemented with other accompanying measures, in order to achieve wider social benefits. In other cases it would be appropriate to seek derogation from the standards. Examples of this case are also provided by the present study.

Third, as highlighted in the case studies, effects produced by environmental projects are manifold and they relate to direct impact as well as externalities. Such effects, relevant in a regional development context, cannot be captured by cost-effectiveness, which looks only at cost minimization for a given physical expected output. Only when exactly the same value of benefits is generated by different projects or the variants, cost-effectiveness can be recommended. In

Share good practice of ex ante CBA…

..including access to shared resources on parameters, e.g. common values on calculating external-ities

CBA is relevant also for compliance driven projects ....



85

.

that special case, a ratio between a physical measure and a monetary value is a reasonable shortcut. The environmental projects under the Cohesion Fund are very diverse in terms of pollutants, design, beneficiary population, etc. The lat-ter aspects are duly considered by CBA, and are not in CEA. Moreover, while the CBA offers a clear test (ENPV greater than zero, given a discount rate), the CEA has no standard test at all to offer: it only says that one project is less costly than another one, but still it can be the sub-optimal option.

The practice of CBA is a relatively new thing; however, Member States also acknowledged that the CBA guidelines have served the purpose of creating a culture of evaluation, but more training and improved access to common re-sources would be desirable.

5.3 Utility of ex post CBA from the point of view of project promoters, MS and EC

Among the ten case studies only one example of an ex post CBA was identi-fied, namely in the Plzen case. However, it was undertaken shortly after project completion and wider impacts beyond those already identified ex ante were not included.

This being said, it is worth mentioning that many of the projects are operated by a commercial operator that submits yearly reports on the projects. In addi-tion, data on environmental performance data, such as water quality or user sat-isfaction of services provided are published at web pages. A good example is the Plzen case, where the operator publishes on-line information about the per-centage of the citizens now using tap water.

While all projects following completion undergo a financial assessment in or-der to make financial corrections for the final report and the final payment, an economic ex post assessment to capture the actual impact to the society due to the intervention is not common practice. There are a number of reasons for that.

First, the timing of conducting the ex post CBA is important. Cohesion-funded projects are just starting to be completed and for an impact assessment to have any value, at least three to five years are required for impacts to materialise. Therefore, for most projects it is still too early to make a CBA.

Secondly, there are some methodological challenges in undertaking an ex post CBAs as the beneficiary quickly forgets the 'before project' situation and has got used to the benefit of the project, which makes it difficult to measure the impact of felt by the beneficiaries of the project, e.g. the citizens. Another chal-lenge is to be able to attribute to 'the after project situation' the factors that have led to the results; not all impacts can necessarily be traced back to the given project.

Thirdly, it is important to keep in mind the beneficiaries of an ex post CBA. The Member states were not in favour of introducing compulsory ex post CBAs as this requirement would just add to the pile of requirements already associ-

Economic ex post assessment not com-mon…



86

.

ated with project programming. However, from the point of view of DG REGIO, the rationale of conducting ex post impact assessments would be to add accountability into the objectives of the Cohesion Fund objectives and to learn about the channels of impact and in particular to take stake of lesson learnt to add value to future programming.

5.4 The appropriateness of ex post CBA to evaluate impact

The ten case studies performed in the framework of the present study showed how the ex-post CBA could be used as a tool to assess the impact of the pro-jects. As mentioned, the ex-post assessment was more an update of the ex ante analysis than a true ex post given the fact that the quantification of the identi-fied impacts was still mostly based on forecasts than on historical data (the lat-ter being only available for the first two/three years of the time horizon while the remaining 20 and even more were forecasts). Still, the CBA approach en-abled to identify some costs and benefits related to a number of actual or ex-pected impacts, as described in the table below.

Table 5-2 Impacts and effect identified in the ex-post analysis

Direct impact and external-ities included in the CBA (quantified)

Direct and indirect - not included

Wider impact and side ef-fects

Bulgaria (waste treat-ment)

Positive: improved waste man-agement (quantified with WTP of waste handling service) Negative: increased CO2 for transport to the landfill

Positive: decrease of pollu-tion of ground water with potential health impact

land agricultural use, afforda-bility risk, environmental aware-ness (e.g. initiating waste sort-ing)

Czech Republic (sew-age)

Positive: cost savings of septic tanks and of bottled water for drinking purposes, improved water quality, reduced pollu-tion (recreational use of water).

Positive: decrease in short-ages (reliability of supply), better water quality, re-duced ground pollution, landscape / image of the city improvement

Institutional learning (estab-lished a European Project Unit), Jacobs model, business agglom-eration effects on the river banks, environmental awareness (e.g. Green ways project)

Greece (waste man-agement)

Positive: improved waste man-agement (quantified with WTP), saving in fertiliser (use of compost), saved water for irri-gation Negative: increased CO2 for transport to the landfill

Positive: reduced ground pollution

environmental aware-ness/flagship project

Spain/Zaragoza (water supply)

Positive: improved water qual-ity (quantified with WTP), sav-ing on purification costs, health effect

- recreational and agglomeration effects, increased awareness about a more efficient use of water

Barcelona (waste wa-ter)

Positive: cost savings from re-duced flooding, reduction of risk of road congestion, im-proved bathing water quality

Positive: Access to ground-water for cleaning tanks, cleaning streets and irriga-tion

Improved ecosystem and biodi-versity of marine waters, public information and awareness (guided tours), perception of the city, recreational spaces created, increasing technological know-how

Hungary (waste water) Positive: improved waste re-moval service (quantified with WTP/avoided cost for septic

Positive: Improved quality of groundwater

recreational effects, sustenance of wildlife



87

.

Direct impact and external-ities included in the CBA (quantified)

Direct and indirect - not included

Wider impact and side ef-fects

tanks), improved surface water quality

Poland (waste water) Positive: recreational benefit Negative: increased pollution for energy use

Positive: Reduction in waste water pollution

Potential for tourism develop-ment

Portugal (solid waste) Positive: energy saving, avoided pollution, reduction of GHG emission, WTP for improved amenity of the site

Positive: energy saving from waste recycling

Environmental awareness

Madrid (solid waste) Positive: WTP for improved amenity of the site, reduction of GHG emission

Positive: visual amenity, use of land

Environmental awareness, edu-cational spill over

Dublin (waste water) Positive: WTP for improved water treatment, recreational benefit Negative: increased pollution for energy use

Positive: visual amenity from water quality im-provement

Public perception (blue flag), aquatic life, tourism develop-ment , environmental awareness (industries urged to have their own treatment system), health quality

Among the identified impacts, some were included in the CBA (quantified and monetised); the calculation of the economic indicators (ERR, ENPV and B/C ration) provided the quantification about the net socio-economic impact of such projects, with a positive impact evidenced by a positive ENPV or a B/C ratio higher than 1.

As it is reported in the Table 4-4, only one of the 10 projects reported positive CBA economic results, thus evidencing positive socio-economic net impacts. However, a number of additional impacts were also identified, but not included in the CBA calculations for reasons of data availability or methodological con-straints. Specifically, the main reasons for non-inclusion were:

• Lack of data for quantification (local data on environmental benefits not collected or available)

• Methodological constraints to monetise (e.g. lacking reliable benefit trans-fer values, lack of key parameters to value the main benefits and external-ities, contingent valuation often being the only, but too expensive, method to capture the willingness to pay).

The result is that the CBA carried out most likely underestimated the actual socio-economic and environmental benefits. However, it is worth noting that, while attention should be paid to double counting (some of the benefits not quantified may be already included, to some extent, in the quantified ones) in principle all the additional identified impacts (direct and indirect) can be as-sessed via a standard CBA and included in the calculations, if the proper condi-tions of data availability and suitable methodological tools (e.g. budget for a direct survey for contingent evaluation) are met.



88

.

Additional and broader impacts or side effects (institutional learning and in-creased environmental awareness being the most common) were also reported as outcomes of the assessed projects. In some cases, a strict causal link between the observed benefits and the project implementation may be difficult to detect, the project playing perhaps more a triggering role in the combination of differ-ent factors (for institutional learning for example). In some other cases (such as spill-over or agglomeration effects), care should be taken in assessing the net effect (displacement effect may have occurred elsewhere). Although some causal relationship can be assumed for such effects, the extent of it remains dif-ficult to quantify and the risk of double counting is high. Such effects are realis-tically not suitable for an ex-post CBA and should be better qualitatively de-scribed and reasoned.

To sum up, evidence from the present study shows that the CBA is an appropri-ate tool to assess direct and indirect socio-economic and environmental impact, the main challenge being in terms of resources and data constraints. More spe-cifically, the ex-post CBAs of projects are useful to:

• follow-up on past decisions in order demonstrate the effects brought about by the project and comparing it with alternative projects.

• learn from the experience, especially in terms of identified problems and reasons for deviations from ex-ante CBA. In this respect, the CBA is seen as a useful tool for project management, and it can suggest how to progress by showing what has gone wrong.

• be accountable for the use of public funds projects and ensure transparency throughout the process. Given the well-known reasons for ex-ante/ex-post results deviation, this would imply making transparent the source of over-estimation of benefits or underestimation of costs in the ex-ante selection phase (which may imply some political sensitiveness).

However, a number of issues should also be mentioned to further specify the assessment:

• The use of the ex-post CBA is relevant as much as the quality and use of the ex-ante CBA is improved. In fact, as long as the ex-ante CBA is carried out with a ‘tick the box’ kind of approach, more as an ex-post rationalisa-tion of a political decision already taken (therefore highly subject to ma-nipulation of the results), the political willingness to carry out the ex-post CBA may be poor and the use very limited.

• In relation to the above, it should be clear what the use of the results of the CBA could be. Here, two approaches can be identified: a soft approach suggesting that the ex-post CBA be used mainly as a learning exercise for the use of project management and for accountability purposes; alterna-tively, it can be used for some more binding result-based scheme. The cur-rent understanding about the use of ex-post CBA is more in the direction of the ‘soft’ approach, due also to the need, first, to improve the quality and strategic use of the ex ante CBA. As a further step, more in a long-term



89

.

perspective, however, a practice of systematic ex-ante/ex-post comparison at project level may ideally evolve in the direction of a performance-based scheme of funding. The main message, however, is that there should be a clear rationale for supporting the requirement of ex-post CBA and a com-mon understanding about how this can feed into the political process at all levels (EU, MS, project promoter).

• The timing of the ex-post exercise is crucial to understand the use of the ex-post results and the capacity to measure actual impacts. Three to five years after project completion has been deemed a proper timing for an ex-post exercise. However, it should be acknowledged that the impacts cap-tured by such an analysis are not the long term stabilised ones, but only the initial, more short-term ones. Still, the exercise is potentially extremely useful in order to plan corrective measures in case some unexpected events or structural changes in the context (e.g. the financial crises influenced some key macroeconomic parameters) have occurred in the meantime. A true ex-post CBA needs a longer timescale of at least 10-15 years (depend-ing also on the typologies of investments).

• Availability of data and methodological expertise are essential requirement in order feed into the calculations. For example, it has been stressed that some impacts may be difficult to capture with a CBA, such as for example wider impacts or impacts of less traditional typologies of infrastructure (such as ICT or R&D)

5.5 Practical treatment of intangible environmental costs and benefit and how they can be included as part of the decision making

As highlighted in the previous section, intangible environmental effects are the most difficult to quantify and monetise, and therefore most difficult to include into the CBA (ex-ante or ex post). At the same time, they are the most relevant aspect providing justification for an environmental infrastructure to be fi-nanced.

Lack of data and methodological expertise are the main causes for the shortages in including such elements in the economic analysis of CBA. While the better quality of a river or the reduced odour is observable and well-identifiable ef-fect, its estimation and monetisation is still rare. This is the main weakness re-ported for the ex ante but also ex post CBA, and the result is that the CBA failed to show the actual benefits of the implemented projects.

This may also explain the hesitation reported by some representatives of Mem-ber States about the effectiveness of including environmental intangible effects into the CBA, which may sometimes appear as more an academic exercise to decision makers less used and prepared on that. Lack of a ‘common language’ and understanding between CBA practitioners and policy-makers seems to be a gap which can be bridged for example by promoting trainings and capacity learning practices.



90

.

The uncertainty about the values and key parameters (WTP estimation, conver-sion factors etc.) to be used for the valuation of environmental intangible calls for harmonised procedures and the provision of key reference parameters. Envi-ronmental intangible effects need expensive technical instruments (e.g. contin-gent valuation) if practices about the consistent use of benefit transfer of a set of standardised rule are not immediately available. The promotion of empirical research to produce and make reference values and key parameters available to practitioners is an appropriate remedy to the current situation. In the (few) case studies where some guidance was given for the quantification of such benefits, the inclusion of those aspects in the CBA was quite straightforward. This is the case for example of the Zaragoza project, where national guidelines, already used in the ex ante analysis, provided the parameters for the health impact of improved quality of water (see section 4.5.3). To some extent also the use of the ‘Jacobs model’ in the Czech Republic, made the CBA exercise more man-ageable and the ex -ante/ex-post comparison more consistent.

Finally, the dissemination and public access of CBA reports, while providing a knowledge basis for benefit transfer approaches, may also foster transparency and accountability.

To sum up, although the inclusion of environmental intangibles into the CBA remains a key methodological challenge from a practical point of view (the theoretical foundations are there, but how to come up with the ‘right’ value is not straightforward). A major effort should be made by CBA practitioners, pro-ject promoters and decision-makers to improve the extent to which they are systematically quantified and a monetary value is attached to them. National authorities can play a role in facilitating such an exercise by developing guide-lines, indicating reference values and parameters and spreading good practices.

5.6 Relevance and potential utility of data obtained from CBA in macro-economic modelling

Public funding is often used for major infrastructure investments. The expecta-tions to the investments besides fulfilling the objectives is also to foster eco-nomic growth by enhancing factor productivity and promote convergence in income distribution and living standards. In principle, more efficient water sup-ply and sewage systems are expected to have both efficiency effects (private cost savings for households and firms) as well as effects on quality of life. Both types of effects may imply higher factor productivity. Macro-economic impacts on growth are therefore assumed as ultimate long-term effect of major envi-ronmental infrastructure. Macro-economic modelling can show the relationship between infrastructure and growth at an aggregate level. However, evidence is often mixed in terms of magnitude and types of effects when typologies of in-frastructure and spatial effects are taken into account.

CBA theoretical foundations lie in microeconomics, and figures used in both financial and economic analysis are at project level (investment cost, net pre-sent value) while macro-economic modelling needs aggregate values (aggregate



91

.

expenditure on infrastructures, GDP growth, GDP elasticity to infrastructure expenditures) for a given territorial unit (e.g. region, country, EU).

On top of this, while macro-economic modelling focuses mainly on effects on GDP, employment, productivity or added value, CBA adopts a more compre-hensive perspective of social welfare including also effects such as improve-ment in water quality, health effects, reduction in CO2 emissions and other en-vironmental externalities.

Although the standard CBA methodology in a general equilibrium framework implies that proper shadow prices are used also to capture the macro-effects, limitations in terms of data availability and the need for a comprehensive as-sessment of all the long-term contributions, may suggest to use a different method than CBA to assess such macro effects at project level.

While it is possible to apply CBA methods to evaluate the macro-economic ef-fects of policies that are identified by general equilibrium or macro-economic models, it is not possible to compute the size of such effects purely from bot-tom-up CBA analyses at project level.

Having said this, some explorations with using CBA results for macro-economic modelling may still be made. In case a database of key CBA ex-post indicators was available for a large sample of projects, the systematic analysis of such indicators could be used to test the results of macro-economic model-ling. In fact, the financial net present value reflects the net output generated at project level, and, in the same vein, data on employment created can be used to test data on employment generation from macro-economic models. Thus, if such micro indicators cannot directly feed into the macro-modelling, micro-indicators of a large sample of projects can usefully complement and test re-sults from macro-economic models. From a conceptual point of view, the same can be said for the economic analysis, where the social rate of return is to some extent an estimate of the elasticity of investment expenditures to growth.



92

.

6 Main findings and recommendations 6.1 What were the impacts of the projects investigated? Of the four waste management projects, the Bulgarian and Crete projects both constitute a first step towards creating a modern waste management handling infrastructure. In these two cases, the waste was not treated but simply disposed of into illegal dumping sites causing groundwater contamination and other in-appropriate implications such as non-pleasant odours, negative aesthetics, inap-propriate land use, bad image, etc. In the case of the Madrid and Portuguese waste projects, the situation is slightly different.

In the Madrid case, the project concerned sealing off an old illegal landfill that had accumulated around 22 million m3 of solid waste produced by Madrid's three million citizens over the last two decades covering an area of 110 hec-tares. At the same time, the project has transformed landfill gas into green elec-tricity producing a total of 1,925 million kWh worth of energy until 2017. The area of the old illegal landfill has been restored and replanted with trees, ponds have been established to the benefit of foraging birds, and by 2025 the area can be used for recreational purposes. The Portuguese waste project is yet another story as it concerns upgrading of an existing but outdated waste management system.

The wastewater management projects include the Dublin case and the Hungar-ian case. Wastewater components are also included in the Plzen and Polish cases. The Dublin case is an extraordinary story because it turned out that the project planners significantly underestimated the demand for wastewater treat-ment. With Ireland being a 'Celtic tiger', Dublin experienced a period of rapid economic growth between 1995 and 2007 - which however came to a dramatic halt in 2008. At the same time, the population of Dublin grew to unexpected heights. Hence, already from the start, the Dublin WWTS suffered from over-loading. Furthermore, due to the designation of the Liffey Estuary as sensitive water, there is a need for new infrastructure developments to comply with stricter discharge standards, and so an expansion of the wastewater treatment facilities is in process.

However, the Dublin Bay now enjoys a substantially cleaner water environ-ment. A proof that the water in the Dublin Bay has become usable for bathing and other water recreational use is that four beaches now comply with the water quality criteria of the Blue Flag Programme. Furthermore, it appears that a sub-

The waste projects have replaced illegal waste disposals with modern waste man-agement…

….and .turning past sins into future envi-ronmental activi-ties…

The impacts of waste water projects are very diverse.

…recreational bene-fits to the Dubliners due to a cleaner Dublin Bay



93

.

stantial number of housing permissions has been granted as new localities have been connected to the wastewater treatment system.

The Hungarian Szeged wastewater project is an excellent example of an inter-vention that, in line with nearly all the other nine projects, was initiated to comply with the Waste Water Directive and involved the construction of a WWTP and upgrading the sewer network completely according to plan. How-ever, the project is still not delivering the full benefits of the investment in terms of improved water quality of the River Tisza. The reason being that the outflows upstream (coming from Romania are not treated to the same standard as the wastewater generated from the City of Szeged.

Both the Plzen case and the Polish Szczecin case are combined water and waste projects. Both projects involve upgrading of existing, but outworn and inade-quate infrastructure. In the Plzen case, 1,400 households are now connected to the sewer system, and the city water supply is now reliable and of good quality. For a number of citizens tap water was simply not fit to drink before the pro-ject, and citizens were relying on bottled water for consumption of water. Thanks to the construction of two retention tanks, the number of overflows into the River Berounka has been reduced eliminating one of the main sources of pollution of the river. A cleaner river has, in turn, resulted crayfish to return to the river, people have started swimming in the river, beaches have been created and a number of projects are being implemented to restore river banks and cre-ate bicycle and walking paths along the river, e.g. Green Ways project.

The majority of the wastewater collection system in Szczecin was more than 60 years old, and some of it still suffered from damage inflicted during the Second World War. The wastewater treatment system relied on an obsolete technology of primary wastewater treatment causing polluted water to be directly dis-charged into the River Oder. This meant that Szczecin was on the hot spot list established under the Helsinki Convention - which is no longer the case with the new infrastructure. Furthermore, the users of the water systems experienced that the services were of a low quality. These problems are now solved.

The Barcelona and the Zaragoza cases are the only non-compliance driven pro-jects. The Barcelona project is about reducing the risk of flooding while the Zaragoza project involves building a 70 km long pipeline from the mountain down to Zaragoza to ensure fresh water to benefit the 821 thousand citizens in Zaragoza which now can rely on tapped water for their water demand to replace the purchase of bottled water. The project has also made the Municipality take other measures to reduce water consumption. In the Barcelona case, the main impacts are felt by the reduced risk of flooding and the fact that the beaches do not have stay closed for tourists for some days following a flooding.

In addition to the above-mentioned positive results, almost all the projects gen-erated positive side effects in terms of environmental awareness (e.g. change of attitudes in the wider public in terms of more efficient use of water – the Zaragoza case - or initiating waste sorting) and institutional learning (e.g. ca-pacity of the public administration to deal with EU projects – the EU unit in the

..unrealised benefits due to dilution ef-fects of the River Tisza

…and politically benefits since Szczecin is no longer on the hot spot list of Helcom

Environmental awareness and insti-tutional learning as common side effects



94

.

Czech case – or to manage complex and technologically innovative infrastruc-ture – the Barcelona case).

6.2 What have we learned that might improve the practice of ex ante CBA?

• That the CBA has to be introduced into practices earlier in the project defi-nition process in order to have an effect on the choice among various op-tions and in order to be used as basis for the decision process.

• That the CBA should be applied to the individual components but also take the larger context into account. As projects are often taken out of a context, e.g. being part of a larger master plan, the CBA needs to be related to the overall master plan.

There is a general lack of CBA thinking or culture among practitioners, access to best practice examples of how to assess externalities and even educational resources are recognised to be measures that might improve the CBA culture. In order to move away from the 'ticking the box' mentality of conducting CBAs, more training, more access to best practice CBAs, awareness of the use-fulness to undertake good quality and sound CBAs are required. Member States call for a more simple approach compared to the current CBA Guidelines, per-haps a more practical or common-sense approach is what it takes.

• That there is lack of data on how to evaluate environmental impacts. A critical issue identified in the report is the difficulty of doing “environ-mental valuation” in the context of these project appraisals. It is important to get the message right here. It is not that the methods are 'problematic' or 'uncertain' (although of course those critiques exist). Rather the problem seems to be that the data are lacking.

Firstly, benefits transfer protocols are desperately needed. This is essential to avoid subsequent work, which simply assumes that 'any number is better than no number'. Of particular importance here, given the paucity of valuation data in countries where many of these CP projects are sited, is the issue of when it is justifiable to transfer values across national boundaries and when it is not.

Secondly, more 'primary' valuation studies are required. In other words, there need to be some sort of concerted effort to establish a database of original val-ues – i.e. for important categories of environmental values affected by CP pro-jects. This might involve some a somewhat more 'centralised' EC effort to es-tablish these values.

6.3 How useful is CBA for ex post impact assessment?

CBA as an impact assessment method is not common practice, however, this study has used the CBA method for ex post assessment and proved its feasibil-ity for that purpose.

CBA earlier in the project selection process

Encourage CBA thinking among practitioners

Improve availability to environmental evaluation data



95

.

The ex post CBA is an appropriate quantitative tool for evaluating systemati-cally direct and indirect socio-economic and environmental impacts of infra-structure projects, provided that suitable resources are committed to it (data availability and methodological capacity and instruments);

There is however clear limitations to the use of CBA when it comes to captur-ing broader effects such as demonstration effects, institutional learning, distri-bution effects, etc. A complementary assessment to capture these broader ef-fects will provide a much better total coverage of the effects from the invest-ments. The CBA exercise is useful to structure the ex post thinking in order to isolate and identify such additional impacts.

The ex post CBA is relevant when it is related to the ex ante CBA. Its potential in terms as a learning and project management tool is enlarged when the ex ante CBA is of a good quality and actually used for project design and selec-tion. In this manner, a comparison of the expected versus actual performance is informative in terms of identifying sources of mismanagement or unforeseeable risks.

If too little time has elapsed since project realisation, then an update of the most relevant forecasts should be made rather than a brand new CBA, and it will still be an informative exercise. It will provide the assessment of the expected long-term impacts. The proper timing for such an analysis is three to five years after the project completion (although current practice at the Member State level and the present ex post exercise shows that an update at the start of operations is also a useful exercise).



96

.

Appendix I Fact sheets 10 projects

Project no. Country Ref. Title Sector

01 Bulgaria 2000BG16PPE002 Set of 6 Regional Waste Disposal Sites (Mon-tana, Ruse, Pernik, Sevlievo, Silistra, Sozopol)

solid waste

03 Czech Rep

2004CZ16CPE003 Plzen: Expansion of the water distribution and sewer infrastructure

Combined wa-ter and waste water

06 Greece 2000GR16CPE001 Gestion de déchets-Chania solid waste

09 Spain 2000ES16CPE035 Abastecimiento con agua a Zaragoza y corre-dor del Ebro

water

13 Pilot

Spain 2001ES16CPE012 Gestion de Residuos en Madrid-2001-grupo2, solid waste

17 Spain 2000ES16CPE060 SANEAMIENTO BARCELONA 2000-2004 water

22 Pilot

Ireland 2000IE16CPE001 Dublin regio Wastewater Treatment Scheme-Stage 5

water

27 Hungary 2000HU16PPE003 Szeged - establishing waste water treatment system

water

29 Poland 2000PL16PPE016 Sewage disposal for Szczecin water

50 Portugal 2002PT16CPE002 LIPOR - SERVIÇO INTERMUNICIPALIZADO DE GESTÃO DE RESÍDUOS DO GRANDE PORTO

solid waste



97

.

Investment: 57 MEUR 01 Bulgaria Waste Management

Project background and context Project objectives At the time of the initial ISPA application (end of the year 2000), Bulgaria’s focus was on achieving EU membership. Therefore, the primary rationale for the application was to seek support for reach-ing compliance with the EU environmental acquis and the general waste management standards. Historically, the waste management system in Bulgaria has hardly considered environmental impacts from waste disposal. Previously, many landfills were unregulated waste dumps, and based on this there was a general intention to make sites legal and to reduce negative environmental impacts.

The objective was to achieve solid waste management in full com-pliance with EU and Bulgarian Regulations and as a result avoid po-tential environmental damages from landfills. The investment included the development of five landfills forming an integrated system of facilities for disposal of municipal and haz-ardous waste, including the close-down of 100 old disposal sites (for Bulgaria as a whole the plan involved a reduction from around 700 old disposal sites to 50 sites).

Direct expected benefits include improved environmental and

health conditions. Furthermore, the project was expected to con-

tribute to integrating Bulgaria into the EU.

Overview of Project components

Main components

Component A: Landfill Montana

Component B1: Landfill Rousse

Component B2: Hazardous waste

Rousse

Component C1: Landfill Sevlievo

Component C2: Hazardous waste

Sevlievo

Component D: Landfill Silistra

Component E: Landfill Sozopol

Total investment 57 MEURO

Site not

visitedSite not

visitedSite not

visited

• Hazardous waste, size 0.55 ha - one cell that include 54 containers • The hazardous site is lower in elevation, providing a natural watershed

• Size of site 15 ha • Municipality, construction and industrial waste

• Size of site 11 ha • Municpal waste • 5 cells • Landfill and waste transfer station

Desription

• Size of site 19 ha • Disposal, municipality, construction and industrial waste• Receive waste from Montana and 22 villages

• Area of 27 ha - 3 cells for municipal, industrial and construction waste • Located at South Banks of Danube River • Recive waste from Russe and surrounding Municipalities

• 2 cells for hazardous waste located on an area of 2.6 ha

• Site size 11 ha (3 cells)• Waste from Sevlievo and 9 settlements

For the in-depth CBA analysis the Russe site was selected to represent the entire investment as all sites were rather similar.

Investment costs for Russe site were EUR 7.3 million.

Options analysis Technical analysis and assessment of capacity and utility The ex ante documentation did not include any option analysis. An option analysis could have contributed to avoiding some of the de-lays that did occur during the construction phase. In particular, the challenges with a mine in Sozopol could probably have been avoided or foreseen.

The landfills are part of the national waste management plan. For this reason it can be assumed that the sites have been screened at an earlier point, although no information was available from the stakeholders. The solution chosen was the minimum solution avail-able to fulfil the legislative requirements.

The pricing of the individual components seems reasonable.

The capacities seem reasonable, based on the situation at the time of estimation. Now, however, Bulgaria faces a changed socio-economic situation, e.g. people are moving toward the urban cen-tres, especially Sofia, which means that the rural areas have experi-enced a decline of the population.

For the Russe site, additional municipalities have started to use the site with the result that the expected utility of the site has been maintained despite a demographic collapse (ex ante 216,000 was forecasted, ex post 228,000 people are being served by the site).

With regard to hazardous waste, the Russe site has not received as much hazardous waste as expected. Generally, there is a tendency that this type of waste is delivered in bulks for which reason the capacity might still be used within the lifespan, and if it is not, the



98

.

major consequence is that the lifespan will be extended.

Financial analysis Economic analysis

The ex ante analysis was not sufficiently developed and did not dis-play any details of the assumptions. In the assessment we compared ex ante and ex post results for component B, Russe. The FRR was -1% in the ex-ante and not possible to calculate in the ex-post. With-out ISPA the results were comparable; however, with ISPA support included, the results were quite different. This was because the expected share of the ISPA support and the actual support differed.

Ex-ante: No economic analysis was undertaken. A cost-effectiveness analysis was intended, however, the actual usefulness of this was low.

Ex-post: the result is not positive. The reason for this was that the observed tangible benefits were few, and the result was a B/C ratio below 0.5.

Comparison of ex ante and ex post CBA results

FNPV M€

FRR %

ENPV M€

ERR %

B/C ratio

Ex ante 4.1 n/a n/a n/a n/a

Ex post -2.2 n/a -4,4 -4% 0.43

Project outcome – wider benefits The objectives of the project have been achieved, now the waste management for landfills is done as required in the directive. The project could have taken the waste management further by introducing much more recycling and reuse measures in the project. It could also have been beneficial if the population had been educated in sorting their waste at house-hold level.

What are the main lessons learned from the case study related to success of the project in terms of achieving the objectives of the project?

Being part of a wider waste management plan, the landfills constitute a first step toward a modern waste handling system. During the site visits recycling activities were emerging and this could possibly be the next step of Bulgarian waste handling.

How was CBA used to support the project decision process?

Only the financial analysis was made. The CBA was not undertaken and therefore not used in the decision support process. An estimation of the cost of handling waste per tonne in the different landfills was undertaken but the result did not bring about any change in the selection of landfills.

Could a more optimal use of CBA have changed or influences the decision process (and planning and execution of the project)?

The solution chosen is optimal, and it fulfils the objectives of reaching the environmental acquis at the time of application. However, it is difficult to assess whether a CBA would have arrived at the same solution or at a different solution.

Which other aspects might be more impor-tant than the CBA to support the decision process?

A good option analysis is important and could have been useful e.g. in relation to the So-zopol site, where pre-assessments could have detected that potential collapse of the mine.



99

.

Investment: 52 MEUR (75% funded) 03 Czech/Plzen Combined water and wastewater

Project background and context Project objectives

Plzen is the 4th largest City in Czech Republic with 170.000 inhabitants. It is located 90 km west of Prague at the confluence of 4 rivers (Rad-buza, Mze, Uslava) that form into River Berounka. The City's sewage system was established in 17th century and further developed in 19th century covering nearly the entire population of Plzen.

reduce pollution in surface water, water reservoirs, groundwater and soil

improve drinking water quality improve the basis for recreational activities

Overview of Project components and main direct benefits

Component A Component B Component CWater supply Sewage connection Retention tanks

Main investment Units

1. Water supply complex "Lobzy":Construction of drinking water reservoirs containing two chambers of each 10,000 m3, installation of chlorine dosage unit, pumping station and 8,290 m intake and distribution pipelines.

1. Construction of 15,520 m gravity sewerage pipelines providing sewerage connection for approx. 4,500 inhabitants2. Installation of 10 wastewater pumping stations for pumping of new connected areas to the existing sewerage system

1. Retention tank "Bolevec": Construction of a new storm water

outlet chamber and a 3,100 m3

retention tank with gravel and sand trap and flushing system

2. Water supply complex "Vinice":Construction of drinking water reservoirs containing two chambers of each 6,000 m3, installation of chlorine dosage unit, pumping station and 3,150 m intake and distribution pipelines.

3. Construction of 7,840 pressure mains connecting the new pumping stations to the existing sewerage system

2. Retention tank "Geva" Construction of a new storm water outlet chamber and

a 2,350 m3 retention tank with gravel and sand trap and flushing system

Direct benefits

Reliabil ity of water supply and improved quality of drinking water for 50-70% of the City of Plzeň's population of 170.000 inhabitants. El iminated problems of capacity, pressure and chlorinated water. Thereby increasing the comfort level of public water consumers, including hotel guests and other visitors tap water consumption.

Cost savings for the 1400 inhabitants for maintenance of septic tanksAvoidance of ground and soil pollution due to mis-maintenance of septic tanksReduced discharge of sewage water from outdated WWTP not fulfil l ing current requirements to cleaned sewage water

Reduction in the number of overflows from approximately 30 ex ante to in the order of 5-10 ex post (not known yet due to lack of operating data) leading to less pollution of surrounding rivers and groundwater

Water and Wastewater

Description



100

.

Options analysis Technical analysis incl. assessment of capacity and utility

The consults were told that an option analyses was conducted in order to identify the most feasible solutions - however, the feasibil-ity reports were not handed over for review. This means that it is not possible to justify if the option was cost-effective.

The technical solution is reasonable and the implementation of the project was efficient.

The demand for water and wastewater services was estimated ex ante with primary focus on the household demand.

All 1400 households that were supposed to be connected to the sewage system have actually been connected. Concerning the water retention tanks, it is still too early to say if they are of sufficient ca-pacity to reduce the number of overspills into the river as they just recently came into operation. The drinking water reservoirs also appear to be doing the job of ensuring reliable drinking water supply to the citizens of Plzen.

Comparison of ex ante and ex post CBA

FNPV M€ FRR % ENPV M€ ERR % B/C ratio

Ex ante -0.19 5.9% -0.3 2.6% 0.74

Ex post -0.06 6% -0.45 1.9% 0.66

The ex ante and ex post analyses were made based on the same conceptual idea. The consumer savings from being connected to the public systems were included and the general use and non-use value of the environmental improvement was included in the economic value flow.

Project outcome – wider benefits • The quality of the river waters has improved over the last 10 years,

• Crayfish is back in the river,

• People have started to swim in the rivers and a number of beaches have been cre-

ated.

• A number of projects are being implemented dealing with restoring the river banks

and creating bicycle and walking paths along the rivers, e.g. Green Ways project

• Institutional learning What are the main lessons learned from the case study related to success of the project in terms of achieving the objectives of the project?

In this specific case, it was found that the Municipality demonstrated a strong will to ensure that the project was promoted by establishing a unit that had direct responsibility to coor-dinate CF applications. In fact, the national Czech setup for analysing and implementing CF projects further seemed to improve the institutional environment to formulate, apply and implement the project. The case is in this respect subject to inspiration and demonstration.


The CBA was not used in the decisions making process. The financial calculations could have led to a review if the rate of support had fallen short of what was expected when the pro-ject was proposed (because of the inability of the city to finance the project without the expected level of EU support). The negative ENPV showed that the economic model results had no impact on the decision making.

Could a more optimal use of CBA have changed or influenced the decision process (and planning and execution of the project)?

Since the project was a compliance driven project the CBA did not influence the decision making process in the same way, as if it was a project formulated to improve the public infrastructure and the environmental, health and social impacts.


Decision making is largely influenced by the specific needs, but in this case the ability to get regional funds did also impact the decision. CBA was more something that was done to be able to apply for funds.



101

.

Investment: 28 MEUR (75% funded) 06 Greece/Crete Waste Management

Project context Project objectives

Crete has a population of 630.000. Of these 50,000 live in Chania. The island receives about 3 million tourists per year. It has a major olive production, with 40,000 hectares of olive trees in the Chania area.

The objectives of building the Solid Waste Recycling, Composting Plant and a new landfill site were to collect and treat waste pro-duced by the Municipalities of Chania, Akrotiri, Souda, Keramies, Eleftherios Venizelos, Therisos, Kydonia, Platania and Mousouri. In addition, excess capacity of the plant was likely to offer the possibil-ity of serving the entire Prefecture of Chania.




102

.

Options analysis Technical analysis and assessment of capacity and utility

Different options were considered, however the geo-graphic location chosen was politically motivated (next to the existing illegal landfill).

The technical option chosen including the recycling facility and the composting plant seems reasonable. So does the geographical location of the plant, approxi-mately 20 km from the city of Chania from which the majority of the waste origi-nates. The plant is located in a remote area. On the capacity of the plant, it ap-pears that the composting facility is oversized. Only one of two constructed com-posting plants is used and not even to full capacity. The capacity of the recycling & composting plant is not fully utilised, and there is a potential for obtaining a higher degree of recycling. Due to low utilisation of the recycling and compost facility, the disposal capacity is limited.

Financial sustainability Direct benefits

The financial key figures do not leave much room for negative influence on the cash-flow. It has to be un-derlined that operating income is about 5 per cent of total income so the project does cause a risk to the financial position of the operator and the municipality. More so, since the affordability analysis of the total tariffs showed no risk of payment problems among household consumers.

There are two main direct benefits: 1) resource efficiency due to the recycling of different types of plastic, different types of paper, glass, aluminium and iron, 2) production of compost as an output of the waste handling process. The produc-tion of compost potentially constitutes a major financial and environmental bene-fit that has not yet been realised. So far, the plant has experimented a lot with the quality and mix of producing compost. Some of the compost has been sold, but it appears that the production of compost remains very low.

• Comparison ex ante and ex post

FNPV M€

FRR %

ENPV M€

ERR %

B/C ratio

Ex ante 20 34% na na na

Ex post -3.7 neg -17 0.5% 0.22

Project outcome – wider benefits Due to the existence and operation of the plant, a number of follow-on effects have been ob-served as a direct outcome of the plant. Firstly, it should be mentioned that the Chania plant is the first of its sort on the island of Crete. The technical waste management know-how of the engineers of the plant, therefore are well recognised on the island of Crete and are feeding in to the combined waste management planning currently taking place on the island of Crete. 2) another outcome is the setting up of pilot projects with farms to demonstrate the added value of using the compost. 3) it is also worth mentioning that due to the construction of the plant and all the media attention in local TV, visiting school classes, education programmes in schools about sorting of waste, etc. are all contributing to an increased awareness of citizens concern-ing environmental issues.

What are the main lessons learned from the case study related to success of the project in terms of achieving the objec-tives of the project?

The objectives of the projects have been achieved, now the 90 thousands tons of waste are treated although there is still room for improvement in order to produce more recycled mate-rial and compost, and thereby minimising the amount of waste ending up in the landfill. The existence of non-realised benefits should not be underestimated, e.g. the importance of in-vesting a marginal amount to facilitate the education of users for waste sorting and education of farmers to understand the environmental advantages (and the potential costs savings in terms of water and fertilizer saving costs for the individual farmer) of compost.

How was CBA used to support the pro-ject decision process?

Only the financial analysis was made. The CBA was therefore not directly used in the decision support process.


The solution chosen is relatively optimal (despite some problems with the over and under ca-pacity). It is therefore difficult to assess whether an ex ante CBA would have arrived at the same solution or a different solution. Alternatively, only one composting unit could have been build initially with an option for another to emerge in case the market was to pick up.

Which other aspects might be more important than the CBA to support the decision process?

The main problems in taking full advantage of the benefits of the plant are related to behav-ioural aspects, e.g. (1) educating consumers in sorting the waste at the level of the households (2) educating local farmers to use compost to supplement or substitute the use of pesticides and to reduce the need for irrigation



103

.

Investment: 121 MEUR (85% funded) 09 Zaragoza, Spain Water supply


The project concerned the supply of bulk raw water to Zaragoza and to 22 surrounding urban centres located along the Ebro River, cover-ing a total population of 821,000 inhabitants (of which about 660,000 were located in Zaragoza).

The overall objective was to implement a water supply system ca-pable of guaranteeing constant distribution of good quality water in the target area.


The project design envisaged the construction of a new aqueduct system that connects the Sora tanks, in the Pyrenees mountains next to Rio Aragon – Canal de Bardenas, to the single water supply networks of the municipalities included in the target area, as well as the construction of first, intermediate and final deposits (Sora, Fuempudia and Casablanca) alongside the main pipe and a pumping station in la Loteta reservoir. The project consists of three compo-nents:

1) Component “Sora – Loteta” (green component in the Figure),

2) Component “Loteta – Zaragoza y corridor del Ebro” (blue compo-nent,

3) “Ramales de Jalon, Huerva-1 y Gallego” (red component.


The option analysis highlighted that the solution adopted is the

most cost-effective. Further use of chemicals would have resulted to

be ineffective and potentially harming for health. The use of tech-

niques of desalination of brackish water was a possible alternative,

but costly and still requiring a large use of chlorine.

Pipes were dimensioned according to higher excepted volumes. This

has produced an utilisation rate of the infrastructure not near 100%.

However, considering that the water speed is in any case ensured,

and the necessity to sustain peaks of demand, the larger pipes ca-

pacity can be accepted. In addition, the rate of utilisation of the

aqueduct will increase, when the construction of the component

“Ramales de Jalon, Huerva-1 y Gallego” ends.



104

.


The financial analysis demonstrated, both at ex ante and ex post stages; that the project is in need of co-financing. Financial sustain-ability is assured by the fact that the tariff is, year by year, full cost recovery. Also, investment and operating costs are in line with the type of project implemented. Delays in project implementation oc-curred.

The project is successful: when it entered into force in 2009 a sig-nificant decrease in overall average concentrations of water pollut-ants has been generated. For instance, in the cases of chloride and sodium, percentage reductions from 2007 to 2010 were 82.57% and 81.73%, respectively.

Comparison ex ante and ex post

FNPV M€

FRR %

ENPV M€

ERR %

B/C ratio

Ex ante – 71.6 n.a. 180.54 20.0 2.11

Ex post – 67.9 -0.81% 36.65 7.6 1.28

The two main causes of the deviation between ex ante and ex post economic results are a combination of an optimism bias and a fore-casting error:

1) Benefit double counting occurred at ex ante stage with er-roneous inclusion of tariffs and general welfare improve-ment.

2) Overestimation of the demand. In particular, since the methodology for monetizing the project’s benefits is such that it positively relates the latter with the volumes sup-plied. The demand aspect is crucial in this sense: lower the volumes supplied, lower the benefits.

Project outcome – wider benefits The project has indirectly induced the municipality of Zaragoza to undertake actions that have radically reduced water losses and modified the behaviour of the users, with benefi-cial effects on environment and safety of water supply. Although these are benefits not attributable to the project itself, as independent from it, a sort of positive leverage has been generated.


The project under analysis, and its success, is an example of how a lot of projects providing services to citizens actually reach their goals through the integration of new investments with already existing infrastructures. The partnership of different actors and institutions (each of them providing a different segment of the whole service) is also a necessary ele-ment. Thus, this project suggests how a perspective focused on the “service” rather than on the “infrastructure” should be always adopted in both project planning and appraisal. This implies a focus on the social benefits produced with the service provision, their nature and magnitude. In this way, integration between investments and actors is more clearly visible and distortions due to administrative requirements may be solved.


The CBA was used to calculate the amount of EU grant and to show the compliance of the project to the eligibility rules. However, the project presented in the ex ante analysis is only a hypothetical formulation drafted for administrative requirements. The project, as actually implemented, considers a more efficient and rational system of use of water of better qual-ity to be delivered at lower cost by exploiting the gravity force.


The administrative approach that led to the division of the project into “segments to be co-financed” is negatively affecting the role and importance of the CBA, which has been re-duced, in practice, to a mere administrative requirement to be fulfilled rather than a real tool to support the decision making. Fortunately, this has not generated negative conse-quences on the project. However, a more accurate CBA could have supported better the decision making process. Possible volume reductions should have been included into the risk analysis to assess the likelihood of a negative net benefit and then to take the decision.


No other aspects are envisaged as more important than CBA, which, if correctly applied to the whole project, resumes all the aspects necessary to capture the welfare change and the achievement of the objectives.



105

.

Investment: 74 MEUR (80% funded) 13 Spain/Madrid


The Valdemingómez landfill was in operation from 1978 until 2000. Over this period, it accumulated 21.7 million tons of solid waste over an area of 110 ha. The incorporation of European environmental regulations into the national legal code and the possibility of access-ing Community Funds led to the adoption of more effective tech-niques for urban waste treatment.

Up to date, EUR 414.2 million has been invested in the Valdem-ingómez Technological Park which comprise four treatment areas and a Visitors’ Center. Part of the investment (74mEUR) is 80% co-financed by European Union’s Cohesion Fund. The project, subject for the current ex-post evaluation, is the La Galiana Center and the objectives are to seal off of the 'old' landfill, the construction of an Energy Recovery Plan and landscape restoration on top of the old landfill.


Main components

Energy Recovery Plant Biogas Extraction Sealing off land fill site Landscape restoration

Development of Educational resources

Other Investments + admin costs

Seal the site

Manage leachates

Manage rainwater

Reducing foul smell

Use value (hindered by regulation)

Greenhouse gas reduction

Groundwater protection

Existence value

Health impact

Opportunity cost of scavenging

Use the LFG to generate electricity

Piping and infrastructure of wells to gather

the gas

Landscaping beyond what is

needed to seal the site

Create facilities for educational programmes

Desription

Value of electricity

Enables possibility to use

the LFG to generate energy

Use valueBenefits


The design of the project was part of the bidding process.

The main concerns related to the technical solution implemented are that (1) The installed capacity of the 8 landfill gas engines was to high, (2) Use of natural gas is up to 30% of the consumed energy and (3) Unclear why the energy recovery plant has to close in 2016 as utilisation of 1-2 gas engines is still possible and one gas engine can be operated with maximum load until 2018/19



106

.


The energy generation component generates a sur-plus. While the other components require opera-tional funding from the Commune.

Closing off illegal landfill - stopping negative environmental impacts

Energy generation

Unrealised benefits:

Unutilized synergies (energy plant) with other facilities of the combined Madrid management facility due to Contractual bindings of current contract with different operators

Comparison of ex ante and ex post

FNPVM€

FRR %

ENPV M€

ERR %

B/C ratio

Ex ante 10.1 6% 36.5 8.9% 1.3

Ex post -9.1 4 -47 n/a 1.08

Project outcome – wider benefits The project is an important brick in the combined Madrid Waste Management facil-ity. The main benefits of modern waste treatment facility however cannot be di-rectly attributed to the current part of the investment; hence WTP values can only be attributed to the benefits of not having the old landfill untreated.

Other benefits include the awareness rising among visitors to the Visitor Centre and the potential use of the restored landfill for recreational purposes (after 20 years).


It is important to identify and assess the individual components, but also to look at the broader context to be able to identify potential synergies.

The energy recovery plant clearly is over dimensioned and could operate longer than what is needed related to the landfill gas production, which is assumed to ter-minate by 2016.


The CBA was not really used, as the design of the project was part of the bidding process.


With the exception of the overcapacity problem of the energy recovery plant, the project is rather well executed, so difficult to say what would have been the added value of a more optimal CBA ex ante.

Which other aspects might be more important than the CBA to support the decision process?

Legal aspects concerning the concession contracts with the other parts of the Val-demingómez Technological Park to provide for changes to contracts in order to accommodate the utility of synergies, e.g. that the landfill gas to be produced from the new landfills could used in the energy recovery plant build under the Galiana project.



107

.

Investment: 69 MEUR (66% funded) 17 Barcelona, Spain Water supply


It is the particular location of Barcelona on the steep slope between the mountains, the high costs of flooding and event at the Olympics in 1992 that had influence on the decision to make a new strategy for the city to address the floodings and CSO in the city.

In this context a strategy (PECLAB) was developed by the municipal-ity and the current project is an integrated part of this strategy that in total involves construction of 25 retention tanks plus collectors and sewers.

The specific objectives for this project under evaluation were to:

• regulate flooding over a ten-year period, so that a major flood-

ing occurs only once in ten years in certain parts of the city

• avoid overflow (CSO) into the sea in the geographical area

covered by the retention tanks

• treat an additional 2 million m3 of sewage water by controlling

the water pleasure in the sewerage system Overview of Project components

Main components

Component A: Taulat

Component B: Doctors Dolsa

Component C: Joan Miro

Component D:

de Villalba

Total investment: EUR 69.4 million

Description

Total capacity: 51 000 m3 - Main sewer: 102m, 58m

Total capacity: 51 000 m3 - Main

sewers: 220m, 130m

Total capacity: 55 000 m3 - Main

sewer: 119m

Total capacity: 14 000 m3 - Plant, size: 5172 m2, tra-

pez shaped medium height 2.9m - Main

sewers: 657m, 379m, 264m, 192m,

550m

Reduction of flooding of building and cellars - especially important for shops

Reduction of emissions of sewerage water into the sea.

Increased number of beach days, because of fewer sewer discharges into the sea/increased quantity of treated water

Fewer or no traffic congestions because of flooding

Benefits

Improvement of quality of life has increased

Sanitary aspects – no rats and dirt from sewerage in the streets/hygiene Marine and ecosystem protection



108

.


Different options were considered at the time of the application; however, all other options were excluded because they were too expensive. A green option was discussed in the ex post assessment, but it was found to be unrealistic, given the specific context of Bar-celona.

The technical solution was optimal and was adjusted to the particu-lar needs of Barcelona – being a confined space that slopes down towards the sea.

The pricing of the individual components seems reasonable.

The scaling of the total capacity within the PECLAB strategy is easy to adjust as the retention tanks are built in phases. This means that if the four retention tanks under this project do not reach capacity limit, then future tanks will be downsized in order for the total ca-pacity to suit the needs.


The project is financially sustainable, as the city of Barcelona is will-ing to bear the operational costs in order to avoid negative conse-quences of flooding.

Because of the PECLAB strategy, the Barcelona city has achieved: Receiving waters quality improvement thanks to the retention tanks, industrial discharges control, and impact diagnosis of receiv-ing waters and design of corrective measures.

In addition to the system, access to ground water has been exca-vated at each retention tank. The ground water is used for cleaning the tanks, watering the public parks nearby and washing the streets.

Comparison ex ante and ex post

FNPV M€

FRR %

ENPV M€

ERR %

B/C ratio

Ex ante -43 4.87 16.7 14.43 1.47

Recalculated ex ante -52 Na -26 - 0.56

Ex post -58 -7.07 -1.1 5 .98

Project outcome – wider benefits A lot of knowledge and experience has been gathered among the technical specialists that have dealt with the project. Because the project is state-of-the-art, the project has many visits from delegations from other cities and countries. There has been a lot of awareness rising among the population, including visits from schools and other stakeholders to the tanks.


It is a good project with many direct benefits.

It has contributed positively to the success of the larger strategy for the Community (PECLAB).


The ex ante CBA developed in relation to the CF application was not used in the decision making process. Rather, the decision was based on input from an option analysis assessing various technical solutions and considering the cost-effectiveness of each of these. Fur-thermore, it appears that the detailed assessment of the project was included as part of the PECLAB strategy, including very detailed studies. The current ex ante is therefore based on a result of a comprehensive analysis.


The solution chosen is optimal (despite some problems with the over and under capacity for the individual tanks). As the entire PECLAB strategy was based on comprehensive as-sessments and option analysis, the CBA for the four tanks was not used per se in the deci-sion making process; rather the CBA way of thinking was the foundation of the design of the entire solution.


The political agenda was influenced from the world media during the Olympics.

A good option analysis and qualification of the related benefits was important for the pro-ject decision.



109

.

Investment: 500 MEUR (56% funded) 22 Ireland/Dublin Waste water


EU membership required the Irish authorities to implement the Urban Waste Water Directive (UWWD, 91/271/EEC) by 2001. This involved the provision of secondary treatment facilities.

The objective of the Dublin Region Waste Water Treatment Scheme (WWTS) was primarily to comply with the environmental legislation, and through this compliance improve the quality of life of the Dub-liners. In other words, the decision on the design of the Dublin WWTS was mainly steered by finding the technical solution that would comply with the UWWD in the cheapest way.

In addition to compliance with the UWWD, the aim of the project was to reduce the pollution in waters used for bathing and recrea-tion, to protect aquatic life and fish stocks according to the Water Quality Management Plan of Dublin Bay, and to improve tourism and industrial/commercial development in the area.


The project is divided into three main investment components:

1) the existing infrastructure;

2) the ultraviolet disinfection equipment - that is assumed to be accompanied with an upgrading of facilities such as toilets, life guards and machinery for removal of algal in order to meet the Blue Flag criteria for Dollymount Strand located at the spectacular North Bull Island, and

3) the future investment to upgrade the capacity of waste water treatment and the needed investments as a response to the designation of the Liffey Estuary as sensitive waters.

Options analysis Technical analysis and assessment of capacity and



110

.

utility

The project included extensive option analyses. A number of alternative options were analysed at the ex ante stage to deal with the fact that in the 1990's the already poor water quality deteriorated. Also, in the report "Ringsend STW Ex-pansion Preliminary Report (1993)" different site options for the expansion of the existing primary treatment plant were analysed. All sites were in the vicin-ity, i.e. more or less neighbour sites, to the site of the existing plant.

Subsequently, Dublin City Council decided to investigate the feasibility of using a compact treatment plant which would be restricted to the existing site area; and in the North Dublin Drainage Scheme Catchment Area Study Preliminary Report (1994), it was assumed that a new plant was constructed in the north drainage area and nine different locations of the plant were examined.

There was, however, a major local resistance for the location of a waste water treatment plant in the north area, and DC therefore decided to investigate the possibilities of a centralised solution, where all wastewater was treated at the Ringsend site.

The designed capacity was too low. With Ireland being a "Celtic tiger", Dublin experienced a period of rapid eco-nomic growth between 1995 and 2007. At the same time the population of Dublin grew to an unexpected high. Hence, the Dublin WWTS has already from the start of its operation suffered from overloading. Furthermore, due to designation of the Liffey Estuary as sensitive wa-ters there is a need for infrastructure developments to comply with stricter discharge standards. This expansion is already in process with geotechnical investigations related to a sea outfall tunnel expected to be carried out in 2010.


The Project is based on the information used - hereunder an upgrading to cater of unexpected high demand - not financially sustainable and require the identi-fication of further investment funds, and a revision of the user fee structure.

The direct benefits include an increase in housing per-missions and recreational benefits, in particular in con-nection with cleaner Dublin Bay beaches.

Comparison of ex ante and ex post

FNPV M€

FRR %

ENPV M€

ERR %

B/C ratio

Ex ante na na 118 6.7% 1.21

Ex post -675 -11.4% -325 -4.3 0.37

Project outcome – wider benefits Qualitative benefits discussed - but not included in the quantitative calculations - are the public perception of the public perception of the environmental improve-ments. This is very much related to water quality - as the ability to have the Blue Flag status is creating confidence among the population and generally they now regard the Bay as a great resource very closely located to a large city; improvements to aquatic life and fish stocks; to tourism and industry; and health impacts.


An actually CBA was not done ex ante only a cost effectiveness analysis.



111

.

Investment: 95 MEUR (50% funded) 27 Szeged, Hungary

Waste Water


Szeged is a city of nearly 170,000 inhabitants in southern Hungary. It has one of the oldest sewer systems in the country, dating back to the 19th Century. The system has, however, become obsolescent and inadequate now more than 100 years later. Only 72% of the city’s dwellings were connected to it, and 40% of the sewage col-lected was discharged raw into the Tisza River, with the rest receiv-ing only mechanical treatment.

The project being reviewed included (a) the expansion of the exist-ing waste-water treatment plant to include full biological treatment of up to 60,000 m3/day of sewage; (b) the increase of the length of the sewer network from 310 km to 718 km; and (c) the increase of the number of household connections by 15,470, thereby expanding coverage of the system to 100% of the population of Szeged and four additional towns (in total nearly 200,000 inhabitants).

The main purpose of the project was to comply with the Waste Wa-ter Directive. The directive requires mechanical treatment of sewage for cities the size of Szeged in all circumstances.


The project consisted of four components:

Component 1: Sewerage extension and development in Szeged city. In 16 districts of the city, 274 km pipeline was constructed and 11 126 households were connected. Streets without pavement have been paved to be used by heavy pipeline cleaning service cars.

Component 2: Sewerage construction in South-New Szeged region. Sewage network has been developed in four settlements (Deszk, Kübek-háza, Tiszasziget, Újszentiván) with a total pipeline length of 110 km, 3 391 households were connected.

Component 3: Construction of the WWTP. Mechanical treatment of the facility was completed by the construction of primary clarifiers fol-lowing the sand trap. Full biological treatment by activated sludge technology was built and installed. Primary and secondary sludge are handled with anaerobic digestion, and the biogas produced is utilized to generate electricity with gas engines. Hydraulic capacity of the newly built WWTP is 60 000 m3/d for dry weather flow and 120 000 m3/d for peak storm flow. Total biological capacity amounts to 230 000 LE (13 800 kg BOI5/d).

Component 4: Construction of two new sewer mains from the city centre and Újszeged to the WWTP. Collection of and combining the pre-sent discharges in Újszeged district on the left bank and Szeged city centre on the right bank of the river. From the city center a main was installed to transport the collected waste-water in the river bottom to the Újszeged bank and a gravitation main was established along the left bank of the river. The combined flow is directed back in the river bed to the WWTP. In addition, two existing pump stations were recon-structed and their capacity was increased, beside two newly pump stations.



112

.


The only options that were considered ex-ante were related to the technology for the WWTP. The tendering conducted allowed bid-ders to specify the technology. The choice between bidders implied a choice of technical solution as well. The document evaluating these options could not be obtained.

The volume of sewage treated has declined by about 10% since a

peak observed in 2008, but it is expected to remain steady. The

pollutant concentration has grown, however, the demand on the

treatment plant is more or less constant and appropriate for the

design capacity. The number of new sewer connections fell short of

the originally planned from 15,470 to 14,327.


The project is financially sustainable despite its poor financial re-sults, attributable to the high capital costs. But the fixed assets are not on the books of the operator. Tariffs are sufficient to cover op-erating and reinvestment costs and are inflation indexed.

The project’s principal tangible outcome is the extension of the

sewer service and the considerable cleaning of the effluent of the

sewage system.

About 15,000 new sewer connections were made. This solved a

problem for many households. In the Szeged area the ground water

levels are always high and very high during flooding periods, which

often caused individual septic tank systems to stop functioning

properly.

Comparison ex ante and ex post CBA results

FNPVM€

FRR %

ENPV M€

ERR %

/C ratio

Ex ante -25.8 0.9 29 9.9

Ex post -54.6 -1.5 -70.7 -5.2

Project outcome – wider benefits No unintended effect was detected. Because of the nature of the project, there are no important linkages that would cause effects beyond the direct consequences of the project.


The project was successful in achieving its objectives. The only negative aspect of its execu-tion was that of the investment cost overruns. Some of these were due to unforeseen technical requirements, others to administrative delays imposed in part by the process of coordination between the many layers of decision and review, spanning Szeged to Buda-pest to Brussels.


The CBA was done to justify the chosen solution. It played no role in the project’s decision process.


Only to a limited extent, due to the fact that the main objective of the project was to com-ply with the Waste Water directive. The directive requires mechanical treatment of sewage for cities of the size of Szeged in all circumstances. However, a properly conducted CBA could have justified an exemption (or at least a postponement for a decade or two) from the requirement to install secondary treatment. This could have yielded significant savings in capital and operating costs (EUR 20m in present value terms) and allowed wastewater tariffs to be lower for the population of Szeged


This project was largely driven by the need of legal compliance.



113

.

Investment: 281 MEUR (80% funded) 29 Poland/Szczecin Waste water

Project background and context and Project objectives

No adequate waste water collection system was available within a considerable part of the city and there was lack of reserve water supply from the water treatment plant and a missing connection of the northern districts of the city with the water distribution network of the remaining part of the left-bank (west of Oder River). The existing waste water treatment system was also not complying with environmental regulations.

With Szczecin being the country's seventh-largest city with a population of around 406,000 persons and the largest seaport in Poland on the Baltic Sea - the pollution of the Oder River had more widespread impacts than just for the city. Szczecin was on the “Hot Spot” list estab-lished under the Helsinki Convention (which is no longer the case with the new infrastructure).

In general, there was a need for a complete renovation of an outdated waste water treatment - that far from complied with environmental requirements, and of the water supply system in Szczecin. These project objectives have been achieved with the new financially sustainable infrastructure.




114

.


The Master Plan for water and waste water management developed for the City of Szczecin in 2000 discussed and evaluated five alterna-tive waste water discharge and treatment options. Two alternative solutions were proposed for further analysis and of these the pro-ject solution was elected via a feasibility study.

The technical solution - as well as the remaining part of the project - is well justified and in accordance with standard international prac-tice. Hence, no major technical shortcomings or failures have been identified. The infrastructure compares well to the demand, al-though the increased user fees have meant that the actual demand for water is slightly below what was forecasted at the time of plan-ning.

In general, there was a need for a complete renovation of an out-dated waste water treatment - that far from complied with envi-ronmental requirements, and of the water supply system in Szczecin. These project objectives have been achieved with the new financially sustainable infrastructure.


The project was financially sustainable. - Necessary investment in waste water treatment to comply with environmental regulations and to avoid being a HELCOM hotspot

- Improvements in quality of water supply – paid for by higher user fees

See more details for each component above.


FNPV M€

FRR %

ENPV M€

ERR

B/C ratio

Ex ante na 7.4 na na na

Ex post -52 1.9 -57 1.8 0.87

Project outcome – wider benefits It is envisaged that potential wider benefits from the new infrastructure development should be included in the assessment, but these are difficult to quantify. For example, the whole Baltic Sea perspective of being removed from the “Hot Spot” list established under the Helsinki Convention is likely to attract attention outside the borders of Szczecin.


A sincere option analysis meant that a good technical solution was chosen.

The beneficiary benefitted from a period of a low exchange rate for the Zloty, and so had some additional funds for unexpected expenses and a few extra infrastructure develop-ments.

The project has large policy attention.


The actual grant application stated that it was not applicable to carry out a CBA. Further-more, none of the stakeholders recalled any use of CBAs in the project decision making process.


Since there was a severe need to renovate the outdated waste water treatment - that far from complied with environmental requirements - and the water supply system; and since the technical option chosen is assessed to be well justified and in accordance with standard international practice, it is unlikely that a CBA would have changed the design significantly.


As just mentioned above, there was a severe need to renovate the outdated waste water treatment. Furthermore, being on the “Hot Spot” list established under the Helsinki Con-vention had high policy attention.



115

.

Investment: 80 MEUR (50% funded) 50 Portugal/Lipor


LIPOR is an association of city councils of the Porto metropolitan area, in the North of mainland Portugal. Since it was created in 1982, this asso-ciation has been implementing an integrated solid waste management system encompassing 8 municipalities of the Greater Porto sub-region.

The Greater Porto sub region has a population of 1.4 million (2007) and a surface of 817 km². It is a highly industrialised area that, together with the neighbouring sub regions, is the main source of the Portuguese ex-ports and home to one of the busiest Portuguese harbours, located in Leixões. Grande Porto serves as the commercial, educational, political and economical centre of northern Portugal.

The objective of the project was to complement and to extend some of the existing activities LIPOR and its city councils share-holders and to remedy some environmental liabilities, aiming at the consolidation of the integrated management system in the region.




116

.


The LIPOR Strategic Plan covering the period 2000-20014 did sug-gest that several technical options were considered during the preparation of the overall strategy.

A separate evaluation of all project components could have resulted in a more economically efficient project.

The rate of utilisation of the recycling facility is over 70%, for the composting facility it is around 50% and for the energy generation is still relatively low, which however is not uncommon during the first years of operation.


Financial sustainability is insured either by green or "guaranteed" administrative prices and close control of costs.

Comprehensive and efficient management of urban solid waste generated in the second largest conurbation of the country with a population in excess of one million inhabitants

For details on components please see above


Total CBA results are not available as analysis is carried out on component level for 3 out of 4 components.

FNPV

€ FRR %

ENPV M€

ERR %

B/C ratio

Ex ante 9.6 7.3 na na na

Ex post -42 <0 na na na



117

.

Project outcome – wider benefits None identified.


Some factors can affect the achievement of the objectives:

- Usual implementation delays of a fairly large and multisectoral investment

- Acceptance of compost quality by farmers

- Awareness to waste source separation + recycling of the public at large



118

.

Appendix II Literature

Agudelo, JI. (2001). THE ECONOMIC VALUATION OF WATER. Principles and methods. Value of Water Research Report Series No. 5. The Netherlands : IHE Delft

Alkire, S. and Sarwar, MB. (2009). Multidimensional Measures of Poverty & Well-being. Report Working Paper. Oxford Poverty & Human Development Initiative (OPHI) and Oxford Dept of International Development, Queen Eliza-beth House, University of Oxford, United Kingdom.

Atkinson, G. (2007). Environmental Valuation and Benefits Transfer: An Overview of Issues . In: Environmental and Resource Economics. Volume 37 (No. 1), Pages 43-61

Aylon, Becker and Shani (2006) Economic aspects of the rehabilitation of the Hiriya landfill, Elsevier, Waste Management 26 (2006) 1313-1323

Barbara, B., Harle, T., Glennie, E., Dillon, E. and Sjøvold E. (2007). Frame-work for operational cost benefit analysis in water supply. TECHNEAU.

Barca, F. (2009) Towards a territorial social agenda for the European Union. Report Working Paper. Ministry of Economy and Finance, Italy.

Barca, F. (2009). AN AGENDA FOR A REFORMED COHESION POLICY. A place-based approach to meeting European Union challenges and expecta-tions. Independent Report prepared at the request of Danuta Hübner, Commis-sioner for Regional Policy.

Barca, F. (2009). The Union and Cohesion Policy – Thoughts for Tomorrow. Transcript of the presentation at the Conference for the 5th Anniversary of Po-land’s Accession to the EU, Brussels, November 4, 2009.

Barkatullah, N. (2002). OLS and Instrumental Variable Price Elasticity Esti-mates for Water in Mixed-Effect Models Under a Multipart Tariff Structure. United Kingdom: London Economics.

Bonaccorsi, A. (2009). Towards better use of conditionality in policies for re-search and innovation under Structural Funds. The intelligent policy challenge. Report Working Paper. University of Pisa, Italy.

Coelli, T. Lloyd-Smith, J., Morrison, D. and Thomas, J. (1991). Hedonic pric-ing for a cost benefit analysis of public water supply scheme. In: The Austra-lian Jurnal of Agricultural Economics. Vol. 35 (No. 1).

Dumas, CF., Schuhmann, PW. and Whitehead, JC. (2004). Measuring the Eco-nomic Benefits of Water Quality Improvement with Benefit Transfer: An Intro-duction for Non-Economists. Department of Economics and Finance, Univer-



119

.

sity of North Carolina at Wilmington,Wilmington and Department of Econom-ics, Appalachian State University, Boone, USA

Dürrschmidt, W. and van Mark, M. (eds.)(2006). RENEWABLE ENERGY: EMPLOYMENT EFFECTS. Impact of the Expansion of Renewable Energy on the German Labour Market. Summary. Berlin: Federal Ministry for the Envi-ronment, Nature Conservation and Nuclear Safety (BMU)

Emerton, L. and Bos, E. (2004). Value. Counting ecosystems as water infra-structure. Gland, Switzerland and Cambridge, UK: International Union for Conservation of Nature and Natural Resources.

EPA (2000): Guidelines for Preparing Economic Analyses. U.S. Environmental Protection Agency

European Commission Diretorate-General for Regio -ISPA. (2001). ISPA Manual - Working Document. European Commission. Revision July 2001.

European Commission Diretorate-General for Regional Policy. (2008). Com-ment of DG REGIO evaluation unit on first deliverables of the ex post evalua-tion of cohesion policy 2000-2006 co-financed by the European Regional De-velopment Fund (Objectives 1 and 2): Work package 2 (Data feasibility study) and work package 1 (Task 1: Regional development trends). European Com-mission. Brussels, 02 July 2008

European Commission Diretorate-General for Regional Policy. (2009). Hearing on the first findings of ex post evaluation 2000-06 Conclusion. European Commission. Brussels, 23 June 2009

European Commission, DG Regional Policy. (2006). Methodological Working Document 4: Guidance on the methodology for carrying out cost-benefit analy-sis, The new programming period 2007 – 2013.

European Commission, DG Regional Policy. (2008). Guide to Cost Benefit Analysis of Investment Projects.

European Commission. (2003). Guide to Cohesion Fund 2000-2006. European Commission.

European Commission. (2008). AN AGENDA FOR A REFORMED COHESION POLICY. Hearing on Growth, Institutions and Policy -State Of The Art and Territorial Dimension 1-2 July 2008 Brussels (Berlaymont build-ing, room Jean Rey) Report Hearing Paper 1. European Commission.

European Commission. (2008). AN AGENDA FOR A REFORMED COHESION POLICY. Hearing on cohesion policy and regional innovation 15 September 2008 Brussels (DG REGIO building, CSM1) Report Hearing Paper 2. European Commission.



120

.

European Commission. (2008). AN AGENDA FOR A REFORMED COHESION POLICY. Hearing on cohesion policy and social inclusion 19 Sep-tember 2008 Brussels (DG REGIO building, CSM1) Report Hearing Paper 3. European Commission.

European Commission. (2008). Policymaker Seminars on cohesion policy: Brussels, 18th and 29th April, 5th and 16th June, 10th October 2008 Report Seminar Paper. Brussels: European Commission

European Commission. (2009). Ex post evaluation of cohesion programmes 2000-2006. Work package 10: "Efficiency: Unit cost of major projects" Sum-mery. European Commission.

European Commission. (2009). SIXTH EUROPEAN CONFERENCE ON EVALUATION OF COHESION POLICY WARSAW, 30 NOVEMBER-1 DECEMBER 2009 “NEW METHODS FOR COHESION POLICY EVALUATION: PROMOTING ACCOUNTABILITY AND LEARNING” PROVISIONAL CONFERENCE PROGRAMME 30 NOVEMBER 2009.

European Commission. (2010). Addressing the challenge of water scarcity and droughts in the European Union. P6_TA(2008)0473. European Parliament resolution of 9 October 2008 on addressing the challenge of water scarcity and droughts in the European Union (2008/2074(INI))(2010/C 9 E/06). 15.1.2010 Official Journal of the European Union.

European Commission. (n.d.). Cohesion fund guidelines introduction. Final. European Commission.

European Commission. (n.d.). Transition from ISPA to Cohesion Fund. Euro-pean Commission.

EUROPEAN COURT OF AUDITORS. (2008). IV(Notices) Notice from European Union Institutions and Bodies Court of Auditors. SPECIAL REPORT No 1/2008 concerning the procedures for the preliminary examina-tion and evaluation of major investment projects for the 1994-1999 and 2000-2006 programming periods together with the Commission’s replies (presented pursuant to Article 248(4), second subparagraph, EC). In: Official Journal of the European Union.

EUROPEAN COURT OF AUDITORS. (2008). THE INSTRUMENT FOR STRUCTURAL POLICIES FOR PRE-ACCESSION (ISPA), 2000-06 Special Report No 12 2008 (pursuant to Article 248(4), second subparagraph, of the EC Treaty). European Communities, Belgium.

Farole, T. , Rodríguez-Pose, A. and Storper, M. (2009) Cohesion Policy in the European Union:Growth, Geography, Institutions. Report Working Paper. London School of Economics, United Kingdom

Garcia, S. and Reynaud, A. (2004). Estimating the benefits of efficient water pricing in France. In: Resource and Energy Economics 26. Pages 1–25.



121

.

Gregersen, H.M., Arnold, J.E.M., Lundgren, A.L. and Contreras-Hermosilla, A. (1995). Valuing forests: context, issues and guidelines. Rome: Food and Agri-culture Organization of the United Nations.

HR Wallingford (2003): Handbook for the assessment of water demand. United Kingdom: Department for International Development in collaboration with the Department for Water Development Ministry of Rural Resources and Water Development, Zimbabwe, University of Zimbabwe, University of Zambia and University of Zululand.

JASPERS. (2008). Guidelines for cost benefit analysis of water and wastewater projects to be supported by the Cohesion Fund and the European Regional Development Fund in 2007-2013.

JASPERS. (2008). Guidelines for cost benefit analysis of water and waste wa-ter projects to be supportet by the Cohesion fund and the European Regional Development fund in 2007-2013. JASPERS (Joint Assistance to Support Pro-jects in European RegionS).

Law & Larson (2008) Valuing a Beach Day with a Repeated Nested Logit Model of Participation, Site Choice, and Stochastic Time Value, Marine Re-source Economics, Volume 23, pp. 233-252

Manzella, GP. and Mendez, C. (2009). The turning points of EU Cohesion pol-icy. Report Working Paper. European Investment Bank, Luxembourg and European Policies Research Centre, University of Strathclyde, United King-dom.

Monfort, P. (2009). Regional Convergence, Growth and Interpersonal Ine-qualities across EU. Report Working Paper.Directorate General Regional Pol-icy. European Commission.

Morton, MH. (2009). Applicability of Impact Evaluation to Cohesion Policy. Report Working Paper, Department of Social Policy & Social Work, University of Oxford, United Kingdom.

Nauges, C. and Thomas, A. (2003). Long run study of residential water con-sumption. In: Environmental and Resource Economics. Vol. 26 (Issue 1). Pages 25-43

OECD (2003). Social Issues in the Provision and Pricing of Water Services. OECD Publishing

Omtzigt, D. (2009). Survey on Social inclusion: Theory and Policy. Report Working Paper.Oxford Institute for Global Economic Development, Oxford University, United Kingdom.

RDC-Environment & Pira International (2003). Evaluation of costs and bene-fits for the achievement of reuse and recycling targets for the different packag-ing materials in the frame of the packaging and packaging waste directive



122

.

94/62/EC –Final consolidated report. Brussels, Belgiu: RDC-Environment & Pira International

RGLForensics, Frontier Economics and FABER MAUNSELLl /AECOM .(2009). Ex post evaluation of cohesion programmes 2000-2006. Work package 10: "Efficiency: Unit cost of major projects" First Interim Report. Task 1: Re-view of Existing Literature and Evaluations. Second Interim Report. Final ver-sion 3.0 1st of April 2009.

RGLForensics, Frontier Economics and FABER MAUNSELLl /AECOM .(2009). Ex post evaluation of cohesion programmes 2000-2006. Work package 10: "Efficiency: Unit cost of major projects" Final Report. European Commis-sion.

RGLForensics, Frontier Economics and FABER MAUNSELLl /AECOM .(n.d.). Ex post evaluation of cohesion programmes 2000-2006. Work package 10: "Efficiency: Unit cost of major projects" First Interrim Report. Task 1: Re-view of Existing Literature and Evaluations. Draft Version 2.1 European Commission.

Rufo, LS. (2004) Clean incineration of solid waste: a cost-benefit analysis for Manila. Singapore: Economy and Environment Program for Southeast Asia (EEPSEA).

Sawyer, S., Chung, L. and Renzetti, S. (2007) Cost-Benefit Analysis for Cleaner Source Water. Canada: Canadian Council of Ministers of the Envi-ronment

Schaafsma, M. and Brouwer, R. (2006). Overview of existing guidelines and manuals for the economic valuation of environmental and resource costs and benefits. AquaMoney

Seravalli, G. (2009). Competitive European regions through research and in-novation Different theoretical approaches to innovation policies. Report Work-ing Paper. Department of Economics,Faculty of Economics, Università degli Studi di Parma, Italy.

Silva, P. and Pagiola, S. (2003). A Review of the Valuation of Environmental Costs and Benefits in World Bank Projects. Washington, U.S.A.: The Interna-tional Bank for Reconstruction and Development/THE WORLD BANK

Turner, K., Georgiou, S., Clark, R., Brouwer, R. and Burke, J. (2004). Eco-nomic valuation of water resources in agriculture. From the sectoral to a func-tional perspective of natural resource management. Rome, Italy: Food and Ag-riculture Organization of the United Nations.

Wedgwood, A. and Sanson, K. (2003). Willingness-to-pay surveys - a stream-lined approach - Guidance notes for small town water services. Water Engeni-ering and Development Centre, Loughborough University, United Kingdom.

EUROPEAN COMMISSION Directorate-General Regional...

Documents

Transcript of EUROPEAN COMMISSION Directorate-General Regional...