Post on 18-Oct-2020
AEAT in Confidence AEAT/ED50287/Final Report
AEAT in Confidence
Mid-term evaluation on the implementation of the framework for co-operation in the field of accidental or deliberate marine pollution
Final Report to European Commission Directorate-General Environment Ref: B4-3070/2003/359591/MAR/D.3
March 2004
AEAT in Confidence
AEAT in Confidence AEA Technology i
Title Mid-term evaluation on the implementation of
the framework for co-operation in the field of accidental or deliberate marine pollution – Final Report
Customer European Commission Directorate-General
Environment Customer reference
Reference: B4-3070/2003/359591/MAR/D.3
Confidentiality, copyright and reproduction
This report has been prepared under contract by AEA Technology plc on behalf of the European Commission Directorate-General Environment (ref. B4-3070/2003/359591/MAR/D.3). This report may not be used for any other purposes, reproduced in whole or in part, nor passed to any organisation or person without the specific permission in writing of the Commission.
File reference AEAT/ED50287/Final Report Reference number AEAT/ED50287 /Final Report Report number AEAT/ENV/1656 Report status Final Contact Details AEA Technology plc
Telephone +44 870 190 6528 Facsimile +44 870 190 6615
AEA Technology is the trading name of AEA Technology plc AEA Technology is certificated to BS EN ISO9001:2000 and ISO14001
Name Signature Date
Author Madeleine Mcdonagh
Reviewed by Nigel Pratten
Approved by Heather Haydock
AEAT in Confidence
AEAT in Confidence AEA Technology ii
AEAT in Confidence AEAT/ED50287/Final Report
Mid-term evaluation on the implementation of the framework for co-operation in the field of accidental or deliberate marine pollution’ – Final Report
AEAT in Confidence AEA Technology iii
Executive Summary
AEA Technology Environment has been contracted by the European Commission Environment Directorate General to undertake a study entitled ‘Mid-term evaluation on the implementation of the framework for co-operation in the field of accidental or deliberate marine pollution’ (ref: B4-3070/2003/359591/MAR/D.3). This is the Final Report under that study to describe the work we have carried out, summarise the findings from our evaluation and make recommendations for the future. The Framework Programme has objectives related to supporting and supplementing Member States’ efforts to protect the marine environment, to contribute to improving their response capabilities, to facilitate efficient mutual assistance and to promote co-operation in order to provide for compensation for damage. The Commission is the only body that provides funding at an international level to promote co-operation between countries in the field of accidental or deliberate marine pollution. Since the Council Decision was adopted in 2000, the objectives of the Programme have been addressed by the development of a Community Information System and by actions within a three-year rolling plan. These actions have included Training and Information, Pilot Projects, Support and Information and Mobilisation of Experts. The mid-term evaluation has considered how the resources devoted to the programme have led to the various outputs, and how these in turn have led to the achievement of the objectives above. We have considered the linkages between the programme and other actions, policy instruments and previous related programmes. Linkages have included previous EC funding in this area, the activities of the International Maritime Organisation (IMO), relevant legislation /agreements/conventions and ongoing work within Member States. Accordingly, the key evaluation issues have been to establish: • Scope – to what extent the programme objectives are relevant to national and EU
needs; • Impacts – how successful the various actions have been and the value for money
they have provided; • Efficiency – how efficiently resources have been converted into outputs or impacts; • Effectiveness – how far the programme has contributed to achieving its objectives; • Sustainability – to what extent are the programme benefits effectively disseminated. Through examination of these issues, this study has also provided recommendations for continuing improvement, thus ensuring that the evaluation has been both retrospective and formative. In 2001, a separate Council Decision established a Community Mechanism to facilitate reinforced co-operation in civil protection assistance interventions, including marine pollution. This new instrument has covered the Mobilisation of Experts in particular and is therefore also considered within this evaluation. Our approach to obtaining the necessary information has involved evaluating data provided by the Commission and obtaining information from representatives from Member States and other stakeholders. A questionnaire has been used to assist in the collection of data and to ensure consistency. Visits have been made to 5 Member States,
AEAT in Confidence AEAT/ED50287/Final Report
Mid-term evaluation on the implementation of the framework for co-operation in the field of accidental or deliberate marine pollution’ – Final Report
AEAT in Confidence AEA Technology iv
other interviews have been carried out by telephone and additional information obtained via e:mail correspondence. Evaluating the impact of the Programme during a specific oil spill incident was felt to have the potential to provide an additional insight into the implementation and effectiveness of some key Programme Actions. The Baltic Carrier incident has been selected for this purpose because of the different ways in which the Programme and the Commission were involved and the accessibility of the various stakeholders. The Commission co-ordinated the mobilisation of both experts and observers in the early stages of the response operations and a project to assess the environmental effects of the spill was part-funded by the Framework Programme. This incident has therefore been used as a Case Study to assess the value of the interventions. The main findings of the study are summarised below. Scope and Rationale
• The Programme objectives are well-understood by stakeholders;
• The Programme complements activities at national, trans-national and regional levels, is perceived as adding value to these regional agreements and is recognised as unique in supporting co-operation at European level;
• Stakeholders believe that the Programme continues to be necessary;
• There are concerns from Member States about the evolving role of EMSA and the potential for duplication of effort; close co-ordination will be required.
Impacts
• While the impact of individual actions has not been measured, most stakeholders believe that the Programme is delivering benefit and good value for money;
• The level of co-operation between Member States is recognised as being very good;
• Stakeholders have different views over the most useful element of the Programme; the most commonly selected are training and workshops, exchange of information and enhancing co-operation between Member States;
• Views on the value of the Community Information System are highly polarised;
• Actions in Training and Information are valued, particularly by smaller countries that do not run their own national training programmes; however, some stakeholders have expressed some concerns over the variable quality of training courses;
• Stakeholders are positive about the value of Pilot Projects; however, there have only been 6 and it is too soon to draw clear conclusions on their impact and value for money;
• Stakeholders have a limited awareness of Support and Information Actions;
• Stakeholders are generally very positive about the Mobilisation of Experts through the Community Mechanism and the co-ordinating role of the Commission.
Process (Efficiency and Effectiveness)
• It is recognised that Commission staff have a high workload which is made worse by the Commission’s detailed procedures; however, against this context, Programme processes are felt to work well;
• Financing decisions are considered to be soundly based;
• Some Stakeholders would like the MCMP to have more influence in the evaluation of proposals.
Sustainability
AEAT in Confidence AEAT/ED50287/Final Report
Mid-term evaluation on the implementation of the framework for co-operation in the field of accidental or deliberate marine pollution’ – Final Report
AEAT in Confidence AEA Technology v
• Many of the Actions supported by the Programme are by their nature sustainable, e.g. training;
• Several Member States feel there is a lack of evaluation, feedback on benefits or critical appraisal of the outcomes of projects;
• Views are divided on the effectiveness of dissemination of project findings;
• Improvements in dissemination and learning will enhance the sustainability of the Programme.
Synthesis • The Programme is perceived as effective in meeting its objectives but has scope to
improve;
• Member States would generally like the focus on Training and Information actions to continue in future;
• There is a healthy list of topic areas that Member States would like to continue to be addressed within training and other actions;
• The current actions and coverage of the Programme are supported and there are no suggestions for major changes.
Baltic Carrier Case Study The Baltic Carrier Case Study has supported our general findings. All the Baltic Carrier actions have been favourably received by all parties and the mobilisation of experts and observers, in particular, was confirmed as very valuable. The need for improved dissemination of information to learn lessons from incidents has been reinforced by the Case Study. Key Recommendations. • The Programme should continue in its current format. • Consider the list of topics identified in this study when next reviewing priorities in the
three year rolling plan. • Ensure continuing involvement and dialogue with EMSA and the Regional Agreements
to ensure activities continue to complement. • Measure the impact of individual projects, follow-up after an appropriate time and
disseminate lessons to Member States. • Consider the future of the Community Information System and the possibility of
committing additional resources to support its maintenance and updating. • Review the need to hold a workshop to discuss and clarify the Community Mechanism
and the roles of experts and observers. • Review the procedures for letting training contracts and consider increasing
intervention to improve overall quality of courses. • Review all recommendations resulting from this mid-term evaluation and consider
priorities for action/implementation.
AEAT in Confidence AEAT/ED50287/Final Report
Mid-term evaluation on the implementation of the framework for co-operation in the field of accidental or deliberate marine pollution’ – Final Report
AEAT in Confidence AEA Technology vi
Contents
1 INTRODUCTION 1 1.1 OBJECTIVES OF THE STUDY 1 1.2 THE FRAMEWORK PROGRAMME 1 1.3 SCOPE OF THE EVALUATION 4 1.4 STRUCTURE OF THE REPORT 5
2 EVALUATION METHODOLOGY 6 2.1 DEVELOPMENT OF THE EVALUATION METHODOLOGY 6 2.2 ASSEMBLY AND ASSESSMENT OF INITIAL DATA 8 2.3 IN-DEPTH DATA COLLECTION 10
3 FINDINGS 12 3.1 OVERVIEW AND CONTEXT 12 3.2 IMPACTS 14 3.3 PROCESS (EFFICIENCY AND EFFECTIVENESS) 19 3.4 SUSTAINABILITY 21 3.5 SYNTHESIS 22 3.6 COMPARISON WITH PREVIOUS EVALUATION 24
4 BALTIC CARRIER CASE STUDY 25 4.1 BACKGROUND TO THE BALTIC CARRIER INCIDENT AND ACTIONS 25 4.2 MOBILISATION OF THE TASK FORCE 26 4.3 MOBILISATION OF OBSERVERS 29 4.4 PROJECT ON MONITORING AND ASSESSMENT OF ENVIRONMENTAL EFFECTS IN
GRONSUND 29
Appendices
APPENDIX 1 MAIN EVALUATION QUESTIONNAIRE APPENDIX 2 LIST OF STAKEHOLDERS PROVIDING INPUT APPENDIX 3 CASE STUDY QUESTIONNAIRES APPENDIX 4 TRAINING COURSE EVALUATION
AEAT in Confidence AEAT/ED50287/Final Report
Mid-term evaluation on the implementation of the framework for co-operation in the field of accidental or deliberate marine pollution’ – Final Report
AEAT in Confidence AEA Technology vii
AEAT in Confidence AEAT/ED50287/Final Report
Mid-term evaluation on the implementation of the framework for co-operation in the field of accidental or deliberate marine pollution’ – Final Report
AEAT in Confidence AEA Technology 1
1 Introduction
The European Parliament and the Council established a Community framework for co-operation in the field of accidental or deliberate marine pollution through its Decision no. 2850 of 20/12/2000. This framework set a legal basis for the role of the European Community in the field of response to marine pollution. The framework has been established for the period from 1 January 2000 to 31 December 2006 although, as the Decision was adopted in December 2000, actions started in 2001. In accordance with article 5 of the Decision, a mid-term evaluation should be carried out by 28/12/2003 and a report on this evaluation addressed to the European Parliament and to the Council. The European Commission Environment Directorate General has contracted AEA Technology Environment to undertake this study entitled ‘Mid-term evaluation on the implementation of the framework for co-operation in the field of accidental or deliberate marine pollution’ (ref: B4-3070/2003/359591/MAR/D.3). This is the Final Report of the study. It describes the work carried out, summarises the findings from the evaluation and makes recommendations for the future.
1.1 Objectives of the Study
The stated objectives of the evaluation are to:
• assess whether the Programme has fulfilled the objectives as identified at the outset;
• measure if the budget invested so far, as well as the organisation devoted to the implementation of the Programme, have led to results contributing to achieving its objectives.
The objectives of the Programme are:
• To support and supplement Member States’ efforts at national, regional and local levels for the protection of the marine environment, coastlines and human health against the risks of accidental or deliberate pollution at sea;
• To contribute to improving the capabilities of the Member States for response in case of incidents involving oil spills or other harmful substances at sea and also to contribute to the prevention of risks;
• To strengthen the conditions for and facilitate efficient mutual assistance and co-operation between Member States in this field;
• To promote co-operation between Member States in order to provide for compensation for damages according to the polluter pays principle.
1.2 The Framework Programme
The Council Decision no. 2850 of 20/12/2000 that established a legal framework for the role of the European Commission in marine pollution also set an agreed level of funding
AEAT in Confidence AEAT/ED50287/Final Report
Mid-term evaluation on the implementation of the framework for co-operation in the field of accidental or deliberate marine pollution’ – Final Report
AEAT in Confidence AEA Technology 2
over a 7 year period. Prior to that, the Civil Protection Unit had bid for funding on an annual basis, which had led to difficulties in forward planning. A committee (Advisory Committee on Pollution by Hydrocarbons and other harmful substances discharged at sea) consisting of representatives from maritime Member States had provided advice to the Unit and assisted in the identification of suitable projects for Commission financial support. Since 2000, the Programme has been managed by the Civil Protection and Environmental Accident Unit with support from the Management Committee on Marine Pollution (MCMP), consisting of expert representatives from Member States and accession countries. The establishing legislation (in Annex II of the Decision) defines the types of eligible actions and the level of Community financial contribution. The actions, which have to be implemented in close co-operation with the relevant national competent authorities, are selected according to their capacity to contribute to:
• providing information and building capacity to deal with pollution incidents;
• improving techniques for response and rehabilitation after incidents;
• providing better public information to help clarify risks;
• helping local competent authorities and other groups to co-operate and to reduce risks and improve response;
• providing operational support by mobilising experts to assist Member States with emergencies.
The arrangements since 2000 have included the introduction of more detailed and prescriptive procedures for managing the allocation and commitment of funds. Priority areas within the three-year rolling plan have been reviewed annually by the MCMP and approved by other Departments within the European Commission. Formal calls for proposals have been issued in the priority areas and a system has been implemented for evaluating proposals and distributing funding between the various different types of action as listed in Annex II of the Council Decision. These cover:
• Actions in training and information (courses, workshops and exercises) – EC funding provided up to a maximum of 75% of total budget;
• Actions for improving techniques and methods of response and rehabilitation (pilot projects) - EC funding provided up to a maximum of 75% of total budget;
• Support and information actions (environmental impact, conferences and events);
• Mobilisation of experts – EC funding provided to cover expenditure generated by mobilisation of experts following marine accidents.
The Commission also provides a facilitation and co-ordination function for observers to visit the affected country during incidents. The Commission is the only body that provides funding at an international level to promote co-operation between countries in the field of accidental or deliberate marine pollution. 1.2.1 Calls for Proposals Calls for proposals have been published in the Official Journal of the European Communities on an annual basis and have provided guidance on the aims of projects
AEAT in Confidence AEAT/ED50287/Final Report
Mid-term evaluation on the implementation of the framework for co-operation in the field of accidental or deliberate marine pollution’ – Final Report
AEAT in Confidence AEA Technology 3
sought within each priority area and the type of action required. Proposals have been assessed using a standard approach as follows: Evaluation Criteria Measure Eligibility – Exclusion Criteria 12 criteria; Yes/No assessment for each Eligibility - Selection Criteria 9 criteria; Yes/No assessment for each Within Priority Fields 6 criteria; score for each from 0 (not
sufficient) to 4 (very good) The assessments have been made following a clear set of rules. An evaluation team consisting of senior staff from different units/directorates rates the proposals and external experts are asked to assess specific technical aspects of proposals in cases where there is insufficient internal expertise. The Commission then approves the proposals and finalises the contracts. The overall process is as shown in the flow chart below. There have been Calls for Proposals in 2000, 2001 and 2002. The next Call is scheduled for 2004. 1.2.2 Community Information System In addition to the actions above, the Council Decision also required a Community Information System (CIS) to be established for the purpose of exchanging data between Member States (and others) on the preparedness for and response to accidental or deliberate marine pollution. Annex I of the Decision specified that the CIS should use a modern automatic data processing system and provide general background information at Community level on a home page on an Internet site and information related to the national intervention resources on national home pages. In addition, a printed version should be maintained in the form of a Community operational loose-leaf booklet with information on emergency management in each Member State. Although the Commission was required to set up the web site and a Community home page, responsibility was given to each Member State to open (within six months from the
Commission/Member States propose and discuss priority areas for action
3-year rolling plan submitted to MCMP and adopted by Commission
Call for Proposals is published in the Official Journal
Proposals are evaluated by the Commission and shortlisted
Commission approves proposals and finalises contracts
Project commences
Input from specialist experts, if required
AEAT in Confidence AEAT/ED50287/Final Report
Mid-term evaluation on the implementation of the framework for co-operation in the field of accidental or deliberate marine pollution’ – Final Report
AEAT in Confidence AEA Technology 4
date the Decision entered into force) and maintain their national home page. The national home pages are required to include information covering: (i) a description of national structures and of links between national authorities in the
field of accidental or deliberate marine pollution, including focal points to be addressed in questions of emergency response;
(ii) general information on existing teams and equipment for emergency response and clean-up, in particular: - strike teams (seaborne) consisting of spill response vessels, - strike teams (land-based) to combat shoreline pollution and to organise
temporary storage as well as to conduct actions for the rehabilitation of sensitive coastal areas,
- expert teams to carry out environmental monitoring of pollution and/or the impact of the combating techniques used, including chemical dispersion,
- other mechanical, chemical and biological means for combating pollution at sea and cleaning up coasts, including systems for lightening of oil tankers,
- aircraft for aerial surveillance, - location of stockpiles, - emergency towage capacity, - emergency number(s) for public use;
(iii) conditions for offering assistance. Member States are also required to update their national home pages as soon as changes have occurred. 1.2.3 Community Mechanism On 23 October 2001 a Council Decision established a Community Mechanism to facilitate reinforced co-operation in civil protection assistance interventions. This new instrument covers both civil protection and marine pollution and provides for the following:
• the identification of intervention teams (and other intervention support), assessment teams and/or co-ordination teams in the event of emergencies;
• the setting up and implementation of a training programme for intervention teams, assessment experts, and/or co-ordination teams;
• workshops, seminars and pilot projects on major aspects of interventions;
• the establishment and management of a Monitoring and Information Centre (operational on a continuous basis);
• the establishment and management of a common emergency communication and information system;
• other support action such as measures to facilitate transport of resources .
As the mechanism includes marine pollution, some of the funds available to support the above actions can potentially be used for relevant marine pollution activities – although it is apparent that most of the actions are already included within the Framework Programme.
1.3 Scope of the Evaluation
This mid-term evaluation has considered how the resources devoted to the Programme have led to the various outputs, and led to the achievement of the Programme objectives. We have considered the linkages between the Programme and other actions
AEAT in Confidence AEAT/ED50287/Final Report
Mid-term evaluation on the implementation of the framework for co-operation in the field of accidental or deliberate marine pollution’ – Final Report
AEAT in Confidence AEA Technology 5
and policy instruments. Linkages have included previous EC funding in this area, the activities of the International Maritime Organisation (IMO), relevant legislation/agreements/conventions and ongoing work within Member States. Accordingly, the key evaluation issues have been to establish:
• Scope – to what extent the Programme objectives are relevant to national and EU needs;
• Impacts – how successful the various actions have been and the value for money they have provided;
• Efficiency – how efficiently resources have been converted into outputs or impacts;
• Effectiveness – how far the Programme has contributed to achieving its objectives;
• Sustainability – to what extent are the Programme benefits disseminated effectively.
Through examination of these issues, this study has also provided recommendations for continuing improvement, thus ensuring that the evaluation has been both retrospective and formative. Our approach to obtaining the necessary information has involved evaluating data provided by the Commission and obtaining information from representatives from stakeholders including Member States. A questionnaire has been used to assist in the collection of data and to ensure consistency. Visits have been made to 5 Member States, other interviews have been carried out by telephone and additional information obtained via e:mail correspondence. Evaluating the impact of the Programme during a specific oil spill incident was felt to have the potential to provide an additional insight into the implementation and effectiveness of some key Programme Actions. The Baltic Carrier incident has been selected for this purpose because of the different ways in which the Programme and the Commission were involved and the accessibility of the various stakeholders. The Commission co-ordinated the mobilisation of both experts and observers in the early stages of the response operations and a project to assess the environmental effects of the spill was part-funded by the Framework Programme. This incident has therefore been used as a Case Study to assess the value of the interventions.
1.4 Structure of the Report
This report is structured as follows:
• Section 2 provides a description of the evaluation methodology;
• Section 3 provides a description of the key evaluation findings;
• Section 4 provides a Case Study of actions carried out following the oil spill from the Baltic Carrier.
Additional data including questionnaires, list of stakeholders providing input, and detailed evaluation of training courses are provided in Appendices.
AEAT in Confidence AEAT/ED50287/Final Report
Mid-term evaluation on the implementation of the framework for co-operation in the field of accidental or deliberate marine pollution’ – Final Report
AEAT in Confidence AEA Technology 6
2 Evaluation Methodology
Based on this basic understanding of the Programme, the approach to the evaluation study can be summarised as:
• clarification of the intervention logic;
• development of the evaluation methodology;
• assembly and assessment of initial data;
• in-depth data collection;
• evaluation of key quantitative and qualitative data.
More information is provided in the following sections.
2.1 Development of the Evaluation Methodology
The Terms of Reference for this study require a number of principal questions to be addressed in order to provide an evaluation that is both retrospective and formative. In addition, the study is required to propose and test a set of feasible indicators that can be used both for this evaluation and for subsequent monitoring of future actions. We have therefore identified five key themes for the evaluation:
• Overview and context;
• Impacts;
• Process (efficiency and effectiveness);
• Sustainability
• Synthesis.
Each of the principal questions together with supplementary questions have been grouped and categorised according to these themes within an evaluation framework. The basic evaluation framework is shown in Figure 2.1. Indicators Measures Theme 1: Overview and Context
• Programme objectives;
• Interactions with other Programmes;
• Barriers to co-operation;
• Gaps in the Programme.
Qualitative, based on views of stakeholders.
Theme 2: Impacts
• Programme successes;
• Added value;
• Programme shortcomings;
• Value for money;
Both qualitative and quantitative, including an assessment of Programme outputs and views of stakeholders. Case Study used to provide additional information.
AEAT in Confidence AEAT/ED50287/Final Report
Mid-term evaluation on the implementation of the framework for co-operation in the field of accidental or deliberate marine pollution’ – Final Report
AEAT in Confidence AEA Technology 7
• Actions;
• Programme value.
Themes 3: Process (Efficiency and Effectiveness)
• Funding decisions;
• Funding split;
• Management processes.
Qualitative, based on views of stakeholders.
Theme 4: Sustainability • Dissemination. Qualitative, based on views of stakeholders.
Theme 5: Synthesis
• Overall assessment of the Programme
• The future.
Primarily qualitative, based on views of stakeholders, but also including a semi-quantitative assessment of the effectiveness of the Programme.
Figure 2.1 Evaluation framework The detailed questions developed for each of the evaluation themes is provided in Appendix 1. While it has been possible to take a quantitative approach to evaluating the training element of the co-operation framework, the nature of the other elements of the framework and the relative sparsity of quantitative data means that the evaluation has been largely qualitative. We have, therefore, made significant use of stakeholders’ perceptions. The information was gathered using a structured questionnaire, by e-mail, a telephone interview or a face-to-face interview. As much as possible, the information was collected in a common format to facilitate comparison and synthesis. Several of the e-mail surveys were followed up by telephone interviews and follow up questions were used in both face-to-face and telephone interviews to test and supplement answers provided in response to the structured questionnaire. Information provided by stakeholders was analysed using a matrix mapping approach. A table was produced listing the key evaluation issues and the responses of individual stakeholders were then mapped on to the evaluation issues as columns of the table. It was therefore possible to compare views by scanning across the matrix. Responses to issues tended to fall in to 3 categories:
• The view was held by many stakeholders. The view may have been expressed in different terms by different people, and some stakeholders stated their views more strongly than others but there was a general consensus over the issue.
• The views of stakeholders were split on an issue.
• One or two stakeholders held a strong view over an issue while others were relatively neutral.
We have not carried out a detailed statistical analysis of replies because of the relatively small size of the survey population, instead we have either reflected the above categories in our comments in the body of the report or referred directly to the number of stakeholders who have supported a stated view.
AEAT in Confidence AEAT/ED50287/Final Report
Mid-term evaluation on the implementation of the framework for co-operation in the field of accidental or deliberate marine pollution’ – Final Report
AEAT in Confidence AEA Technology 8
2.2 Assembly and Assessment of Initial Data
Various data sources were used as the basis for an initial assessment of the Programme. The team within the Commission’s Civil Protection and Environmental Accident Unit co-operated in providing much of this valuable information. A list of these information sources is shown in Table 2.1. Table 2.1 List of information sources Item Brief description Applicability to the
evaluation ‘Texts Relating to Community Cooperation on Marine Pollution’, Directorate General Environment, Civil Protection Unit, January 2001
Relevant Council Decisions between 1978 and 2000.
Detailed information about the evolution of the Commission’s involvement in marine pollution.
‘Council Decision of 23 October 2001 establishing a Community mechanism to facilitate reinforced cooperation in civil protection assistance interventions’
Details of the Community Mechanism.
Provides information to assess how the Community Mechanism fits with the Framework Programme.
‘Status Report on the Actions Undertaken in the Context of Decision 2850/2000/EC’ Working Document 5/2/1 Annex, 5th Meeting of the MCMP, October 2003
Summary of annual priority fields, actions supported, contracting party, funding provided, project dates, summaries of actions.
Basic information about the Programme outputs.
MCMP meeting documents Agendas, working papers, minutes etc for all 5 of the meetings held between March 2001 and October 2003.
Provide information about the workings of the MCMP including information about proposals and projects and the comments of Member States.
EC’s Marine Pollution web site Framework programme objectives and overview, texts and legislation, three-year rolling plan, Community Information System, contingency organisation, useful publications and reports, and international co-operation.
Background information about the programme; links to other regional conventions and agreements; links to project reports.
‘Evaluation of the Community Framework for Co-operation in the Field of Accidental Marine Pollution’, Peter Hayward Associates, March 1999
An evaluation of Community action in the field of marine pollution from 1994 to 1999 and a forward look to the proposed programme for 2000 to 2004.
Enables a comparison to be made of the results, impacts and trends between this evaluation and the previous one.
Helsinki Convention; Barcelona Convention; Bonn Agreement; Lisbon Agreement
Details of other relevant international conventions and agreements.
To assist in the assessment of the value that the Framework Programme adds to the existing regional conventions and agreements.
Evaluation forms from training and information projects
Feedback from participants. To assess the perceived value of the courses and workshops to the participants.
Procedure for letting contracts under the Framework Programme
Information about the process for calls for proposals, proposal evaluation and funding
To help in assessing the efficiency of the process of managing the Framework.
European Maritime Safety Agency (EMSA) web site
Summary of the role of EMSA. To help in assessing the objectives of the Framework taking account of the Agency.
European Maritime Safety Agency Work Programme 2004
The origin, work plan and future development of EMSA.
Enables the roles of EMSA and the Commission Programme to be compared.
‘Communication from the Commission to the Council and the European Parliament: towards a strategy to protect and conserve the marine environment’, CEC COM(2002)539 final, 2/10/2002
Description of the foundation upon which a thematic strategy for the marine environment can be built.
To help in assessing the objectives of the Framework taking account of the thematic strategy.
AEAT in Confidence AEAT/ED50287/Final Report
Mid-term evaluation on the implementation of the framework for co-operation in the field of accidental or deliberate marine pollution’ – Final Report
AEAT in Confidence AEA Technology 9
Item Brief description Applicability to the evaluation
‘The “Baltic Carrier” Oil Spill: Monitoring and Assessment of Environmental Effects in Gronsund (DK)’, Storstrom County, 2002
Report of the environmental assessment project part-funded by the Programme.
Used for the case study.
‘Accident of the Oil Tanker “Baltic Carrier” off the Danish Coastline’, Final Report of the European Task Force in Denmark from 1st to 5th April, 2001
Report of the mission and recommendations of the Task Force.
Used for the case study.
An initial assessment was carried out to provide an appreciation of the main areas of activity. Most of the funds have been allocated to projects within the agreed priority areas according to their eligibility and using an evaluation process based on well-defined selection criteria. Some additional funds have been provided for projects within the action areas but outside of a specific call for proposals. These include the funds provided for mobilisation of experts and invitations from other international organisations to co-finance training courses or conferences. The expenditure committed since 2000 as reported on the EC web site (Rolling Plan 2000-2002, updated 17/9/03) amounted to Euro 2,277,598 and has been divided as follows: The figure below shows the number of proposals received, projects for which funds have been committed and projects completed.
No. of Projects
176
4
2
% Funding Allocated
52%
35%
10% 3%
Training & Information
Pilot Projects
Support & Information
Mobilisation of Experts
34 28
20
13
23
75
40
10203040506070
No.
2000 2001 2002 2003
Year
Proposals and Projects
Projects completed
Projects for which fundscommittedProposals Received
AEAT in Confidence AEAT/ED50287/Final Report
Mid-term evaluation on the implementation of the framework for co-operation in the field of accidental or deliberate marine pollution’ – Final Report
AEAT in Confidence AEA Technology 10
Details of projects supported in the Rolling Plan are summarised on the EU web site and information is provided on the project title, contractor and budget (including EC contribution). The dates of courses, workshops and conferences are given where known and summary reports (in pdf format) of completed actions can be viewed or downloaded via links directly from the web site.
2.3 In-depth data collection
Face-to-face interviews were used to obtain in-depth information and it was agreed with the Commission that it would be appropriate to interview five country representatives in this way.
Countries were selected for the face-to-face interviews according to the following criteria:
• to give a reasonable geographic spread;
• to include some with recent spill experience;
• to include one with a significant involvement in actions under the Programme.
It had been hoped to include Spain as one of the selected countries because of their experience following the accident involving the Prestige. However, this did not prove possible and an alternative was substituted. The maritime countries not selected for the face-to-face interviews were sent the electronic version of the questionnaire and invited to complete it themselves. They were also offered the option of a telephone interview, if that was preferable. Other selected stakeholders were contacted by telephone. A separate questionnaire was developed to gather information for the Baltic Carrier Case Study. A full list of individuals interviewed and their organisations is provided in Appendix 2. More information about the questionnaires is given below. 2.3.1 Main questionnaire The questionnaire was aimed at the main Programme stakeholders, including the National Authorities within maritime Member States, Commission stakeholders, and project beneficiaries. The interview questions were based on the information requirements within the five themes identified in Section 2.1. Visits were made to five Member States (Denmark, France, Italy, Portugal and UK) for face-to–face interviews with National Authorities. Questionnaires were returned by e:mail from a further six countries (Germany, Greece, Ireland, Netherlands, Norway, Sweden) and a telephone interview was conducted for Belgium. It was not possible to obtain feedback from the remaining two countries contacted.
Further telephone discussions were held with:
• a project beneficiary in France
• the European Maritime Safety Agency
• the International Tanker Owners Pollution Federation
• the International Maritime Organisation.
In addition, discussions were held with Commission officials within the Civil Protection and Environmental Accident Unit.
AEAT in Confidence AEAT/ED50287/Final Report
Mid-term evaluation on the implementation of the framework for co-operation in the field of accidental or deliberate marine pollution’ – Final Report
AEAT in Confidence AEA Technology 11
2.3.2 Case Study questionnaires Questionnaires were also developed for the various stakeholders interested in the Commission’s involvement with the Baltic Carrier incident ie Danish Authorities, Task Force Members, Environmental Effects project beneficiary, and Member States. The questionnaires are shown in Appendix 3. Replies were received from representatives of all these stakeholders.
AEAT in Confidence AEAT/ED50287/Final Report
Mid-term evaluation on the implementation of the framework for co-operation in the field of accidental or deliberate marine pollution’ – Final Report
AEAT in Confidence AEA Technology 12
3 Findings
The findings have been evaluated according to the five themes within the evaluation framework. Particular weight has been given to the comments from the Member State representatives and their feedback and views are discussed in some detail below. At appropriate places, input from other stakeholders and comments based on our own observations are included. The Baltic Carrier case study has been used to provide a specific focus for views and opinions. In each theme area, the key points and recommendations are summarised at the end of the section.
3.1 Overview and Context
3.1.1 Programme objectives The Programme objectives are generally well-understood in a broad sense although not necessarily expressed in the same way as the Commission. Many stakeholders recognise the role of the Programme in promoting co-operation between Member States, exchanging information and developing skills, particularly through provision of training. The co-ordinating role of the Commission during incidents and the mobilising of the Task Force were mentioned specifically by some Member States. 3.1.2 Interactions of the Programme with other Programmes The work within the Commission Programme is consistently perceived as complementing that at national and trans-national levels. Various specific trans-national organisations or activities were mentioned by Member States as having relevance to the Programme. Other more or less formal arrangements with neighbouring countries were also mentioned. Some overlap of objectives with regional agreements, in particular, is acknowledged but duplication of effort with the Programme is not seen as a current problem. Nevertheless, there is a need for ongoing dialogue to ensure continued complementarity with the regional mechanisms. European Maritime Safety Agency (EMSA) 6 Member States have expressed uncertainty about the role of EMSA and have raised concerns about the need to avoid duplication. The general perception is that the role of EMSA is continuing to evolve and that care should be taken to avoid EMSA repeating work already carried out elsewhere and relearning lessons already well-understood. Officials within both EMSA and the Commission are clear that their roles are different and complementary and there seems to be a strong commitment from both parties to maintaining their complementarity. EMSA is concerned mainly with at-sea operational matters and has a focus on prevention of accidents. The Programme is concerned with response methods and co-operation, communication and co-ordination between countries. Nevertheless, there appears to be the potential for some overlap in identifying and spreading best practice and in mobilising expertise during incidents. Good communication between EMSA, the Commission’s Unit and other stakeholders will continue to be essential to avoid future issues. It has been suggested by one stakeholder that the most effective arrangement may be to bring the MCMP and the Programme under the umbrella of EMSA. However, EMSA is not
AEAT in Confidence AEAT/ED50287/Final Report
Mid-term evaluation on the implementation of the framework for co-operation in the field of accidental or deliberate marine pollution’ – Final Report
AEAT in Confidence AEA Technology 13
constituted in a way that allows the range of activities possible within the Framework Programme. 3.1.3 Barriers to co-operation Co-operation between Member States is generally felt to be very good and this is confirmed by the experience during incidents. Interacting with representatives of other Member States at MCMP meetings is valued by some, although others feel that more could be made of the opportunity. It is recognised that meetings at other fora also help co-operation. A concern was raised about a perceived lack of clarity under the Community Mechanism as to who will pay for what in the event that experts or resources are provided by one Member State to another during an emergency. With the involvement of new countries, it is particularly important to avoid this issue becoming a barrier to co-operation. 3.1.4 Gaps in coverage No comments or concerns were expressed about the overall scope of the Programme. Some stakeholders mentioned specific areas that they feel should be addressed or given more attention. These included:
• Funding of port state control activities; • Surveillance of illicit/accidental spills; • Development of better equipment and technology for oil spill response; • Response to hazardous and noxious substances; • Review of procedures on board vessels; • Standardisation of procedures for cost recovery for equipment provided to other
Member States and consistency in obtaining compensation from P&I clubs.
Most of these areas can be considered along with other suggestions when reviewing priorities within the three-year rolling plan. The cost recovery issue that was also raised by a different Member State as a potential barrier to co-operation (see Section 3.1.3) could be a topic for discussion at a future MCMP meeting and could include a representative from ITOPF or a P&I club.
Overview and Context Key Points • The Programme objectives are well-understood by stakeholders;
• The Programme complements activities at national, trans-national and regional levels;
• There are concerns from Member States about the evolving role of EMSA and the potential for duplication of effort;
• The level of co-operation between Member States is recognised as being very good;
• There is a perceived lack of clarity about the mechanisms for payment when resources are provided by one Member State to another during an incident.
Recommendations for Action • Ensure continuing involvement and dialogue between the Unit and officials
in EMSA and the Regional Agreements to avoid duplication;
• Discuss the issue of mechanisms for payment for resources provided by one Member State to another at a future MCMP meeting;
AEAT in Confidence AEAT/ED50287/Final Report
Mid-term evaluation on the implementation of the framework for co-operation in the field of accidental or deliberate marine pollution’ – Final Report
AEAT in Confidence AEA Technology 14
• Consider the list of identified gaps in coverage of the Programme when next reviewing priorities in the three-year rolling plan.
3.2 Impacts
3.2.1 Programme successes Actions in Training and Information are highlighted by 5 Member States and are particularly valued by the smaller countries that do not have their own national training programmes, thus indicating additionality brought by the Programme funding. Various aspects of the Commission’s role during incidents are mentioned by a further 5 Member States. The Prestige is viewed as particularly successful for the part played by the Commission, and several areas of support at incidents are valued:
• Co-ordination of offers of assistance; • Co-ordination of observers; • Encouraging co-operation between Member States; • Role of the Monitoring and Information Centre; • Identifying lessons learnt.
Other areas of the Programme mentioned by Member States include facilitating of networks, pilot projects, workshops, exchange of experts between Member States and harmonising procedures in prosecution of MARPOL offenders. 3.2.2 Added value The Programme is widely perceived as adding value to the regional agreements and is recognised as unique in supporting co-operation at a European level. If it did not exist, the consensus seems to be that the regional and trans-national agreements would be the next best instruments for promoting co-operation. However, these agreements are not set up to provide funding for Actions in the same way as the Commission Programme and each one covers only a few countries. The Training and Information actions are viewed as providing particular added value. 3.2.3 Programme shortcomings In one case it is felt that different standards of preparedness exist in different regions and that requirements from the Programme therefore also vary. The implication is that it is not possible for individual actions within the Programme to be of equal value to all Member States. This is almost inevitable and, indeed, is likely to become more exaggerated as some less-experienced countries become more involved in the work of the Programme. However, the Programme has an important part to play in spreading good practice across all countries and raising standards within the least experienced. It therefore has to be accepted that the best-equipped and most experienced countries may not benefit directly to the same extent as some others. The number of regulations within the Commission and the strict procedures are seen as slowing down some activities. 3.2.4 Value for Money Most stakeholders feel that the Programme is providing good value for money. Some reservations have been expressed about the value for money from a few training courses where the quality has been poor.
AEAT in Confidence AEAT/ED50287/Final Report
Mid-term evaluation on the implementation of the framework for co-operation in the field of accidental or deliberate marine pollution’ – Final Report
AEAT in Confidence AEA Technology 15
One stakeholder would like the benefits of the Programme to be critically assessed before judging the value for money. This mid-term report provides some indication of the perceived benefits. However, it is also recommended that more effort is spent in evaluating and disseminating the outcomes and benefits of individual actions. 3.2.5 Community Information System Views on the CIS vary widely. Some stakeholders value it highly and consider it to be a very useful tool during emergencies. Others feel it has value in exchanging information that helps in emergency planning. One stakeholder considers it to be a tool for the general public rather than for national authorities and experts. Several stakeholders mention other tools that provide information for emergencies or explain that they would refer first to their regional agreements rather than consulting the CIS during emergencies. There is a view amongst some stakeholders that the CIS is not regularly reviewed, updated or improved. Those Member States that are less convinced about its value find it time-consuming and burdensome to keep updated. Two Member States proposed that it would be preferable to have a web site that contained links to other relevant and useful sites. The majority of Member States do hold positive views about the CIS and it is clearly used regularly and helpful to some. However, each Member State has responsibility for its own site and there may continue to be difficulties in persuading those Member States that do not value the CIS to spend the time in updating their sites. On balance we believe that the CIS is a tool that should be retained, particularly if improvements can be made to the updating process. It is therefore recommended that the Commission should consider ways of convincing all Member States of the value of maintaining their national pages. There is also scope for the Commission to make improvements to those pages for which it is responsible. The potential benefits include:
• Updated authoritative information on all sites; • Greater consistency of information; • Improved ease of use; • Identification and dissemination of best practice related to emergency response; • Extension of functionality eg inclusion of links to other useful web sites. 3.2.6 Actions in Training and Information Almost all Member States appreciate and value the training supported by the Programme and there is a consistent view that it is an extremely important way of informing and developing individuals and thus indirectly improving national approaches to prevention and intervention. Training and workshops are also seen as a very useful way of building links between individuals and Member States - the Catania workshop (organised in order to review the Prestige incident) was specifically mentioned as valuable in this regard. Workshops generally have a greater potential to influence national authorities than training courses. Appendix 4 provides some details from our analysis of course evaluation forms. The feedback from participants is generally very positive (although it is accepted that participants who have not had to pay to attend may be less critical). In particular, the key ratings for quality of trainers and overall ratings for the courses are high. This finding is at odds, to some extent, with the perceptions of some Member States that have commented on the variable quality of training courses. It seems possible that
AEAT in Confidence AEAT/ED50287/Final Report
Mid-term evaluation on the implementation of the framework for co-operation in the field of accidental or deliberate marine pollution’ – Final Report
AEAT in Confidence AEA Technology 16
positive markings on the course evaluation forms may not always be reflected by participants’ comments on their return to their home base. One suggestion is that the Commission should send a representative to every course and the representative should remain throughout its duration to make an independent assessment of its quality. This approach has the potential to drive up standards but clearly there is also a cost implication for the Commission. One stakeholder perceives that there is an expectation that the available budget for actions should be spread around Member States and that this sometimes compromises quality. They cite a specific example where it is asserted that a well-respected and well-known organisation would be the best choice to run a basic course on oil pollution but they have not been selected. The Member State representative of the country in which the organisation is located has commented in a different way. Their concern is that there are no funds available within the host country to co-fund training and that the organisation is therefore not able to respond to Calls for Proposals. It is assumed that it is not possible for the Commission to fund training courses at the 100% level. The alternative of making up the additional funding required by charging participants to attend is not attractive, particularly as the smaller countries might then feel inhibited from sending representatives. The recommended approach is therefore to focus on improving the quality of courses and for the Commission to be more proactive in reviewing course objectives, course content and standard of presenters. Feedback and impact assessment of training and information can also be improved and it is recommended that the longer-term impact of the training is assessed. It is proposed that the contractor is required to obtain feedback some 3-6 months after the course and to ask participants to evaluate changes resulting from the training. A short questionnaire should be devised for this purpose. The difficulty of obtaining such feedback from participants coming from many different countries is recognised but there are various approaches that could be adopted to facilitate this. All feedback should be reviewed and action taken as appropriate to improve the quality of future courses. 3.2.7 Pilot Projects Only six pilot projects have been supported by the Programme and only three have been completed with final reports available. Care must therefore be taken in drawing conclusions from this small sample size. “The Certification of Oil Spill Response Equipment” project was mentioned by 2 Member States as likely to be of particular value when purchasing new equipment. One beneficiary Member State commented that the pilot projects provide the greatest value for the Member State that proposes the project because they enable know-how and technical capacity to be built up and lessons can be learnt directly. The Programme funding can be essential in enabling such projects to take place. Another stakeholder felt that one of the projects, in particular, did not appear to be of general relevance. At this stage, the Pilot Projects do not generally appear to be leading to changes in strategy at a national level but this will be a matter to keep under review as projects progress. Part of the assessment of Pilot Project proposals should include their general relevance within Member States and for the findings to be widely replicated. Some limitations have been identified in the evaluation and dissemination of key information from Pilot Projects (and other actions). For example, it would be useful for Member States to have more opportunity to discuss the findings from projects and for the key benefits and lessons of general importance to Member States to be distilled from the work. In some cases, it would be valuable to follow-up the project in, say, 6 or 12
AEAT in Confidence AEAT/ED50287/Final Report
Mid-term evaluation on the implementation of the framework for co-operation in the field of accidental or deliberate marine pollution’ – Final Report
AEAT in Confidence AEA Technology 17
months time to establish what further developments have taken place and the implications for Member States. 3.2.8 Support and Information Actions These actions cover a wide range of activities within the current headings of:
• “Environmental Impact” (1 project carried out in the aftermath of the Baltic Carrier incident and discussed later as part of the Case Study);
• “Conferences and Events” (3 conferences for which the Programme provided less than 30% of the total funding);
• “Other Support Actions” (1 pilot project and this mid-term assessment project).
It is apparent that specific activities within this action area are not readily brought to mind by the Member States. The following may explain the reasons for this:
- the non-specific title of the action area - the disparate nature of these projects - the lower funding profile of the Commission in its support for conferences
Most Member States have not commented clearly on these actions but have generally responded positively in the expectation that the action will lead to enhanced co-operation. A Finnish municipality heard about the Baltic Carrier environmental impact project and was keen to receive a copy of the report, indicating its perceived value outside the host country. 3.2.9 Mobilisation of Expertise through the Community Mechanism Not all Member States have experience of this aspect but the comments from Member States are generally very positive about the principle of mobilising experts to assist during incidents and the role of the Commission in co-ordinating this activity. The continuous operation of the Monitoring and Information Centre and the wider reach of the Commission to countries that are not accessible by individuals through their regional agreements are particularly valued. The possibility of sending observers to incidents was seen as extremely valuable and the co-ordinating role of the Commission in this and the other areas is also found to be very useful and effective. However, there is considerable confusion surrounding this area within some Member States for the following reasons:
• Uncertainty about the continued existence of the Task Force; • Uncertainty about what is covered by the Community Mechanism and what remains
directly within the Framework Programme in this area; • Issues about who pays for what (see Section 3.1.3); • Lack of broad objectives or terms of reference for experts or observers at incidents.
Some stakeholders were particularly interested in discussing experiences during specific incidents and held some interesting views, as outlined below:
Prestige – The Commission’s wider access to resources was useful. However, the request for equipment did not give enough detail about requirements and all offers were taken up by the host country without consideration of their suitability. Perhaps an expert could have assisted in the Monitoring and Information Centre during the incident. The Commission requested nominations for a panel but it was unclear whether the request was for experts or observers or both.
Baltic Carrier – Again, there was some uncertainty about whether the Commission was seeking nominations for an expert group or an observer group or both. The observer group included some individuals with spill experience who were keen to share their idea
AEAT in Confidence AEAT/ED50287/Final Report
Mid-term evaluation on the implementation of the framework for co-operation in the field of accidental or deliberate marine pollution’ – Final Report
AEAT in Confidence AEA Technology 18
and thoughts with the response authorities. This was presumably beyond their remit but may not have been made clear to them. It is understood that the recent joint meeting between the Civil Protection and Marine Pollution committees (26 January 2004) has identified some specific practical procedures for mobilising experts and observers and this should help to resolve some of the issues identified here. Bearing in mind that the joint meeting has already addressed some issues, it is recommended that the Commission reviews whether there is a need to provide Member States with the opportunity to discuss the Community Mechanism further in order to ensure a common understanding of the operation of the Community Mechanism and related activities and to identify possible areas for improvement. A workshop could be a suitable forum for such a discussion, if still needed. 3.2.10 Programme Value The responses indicate that the stakeholders are very positive about the value of the Programme. A range of aspects are identified as “most useful” but training, exchange of information and enhancing co-operation are each mentioned by several stakeholders. Some stakeholders view all aspects as equally useful and are reluctant to pick out any specific aspects! A very small number of aspects are identified as “least useful”. Most useful aspects include:
• Training and workshops
• Developing co-operation
• Raising awareness of people and resources
• Monitoring and Information Centre
• Meetings of the MCMP Least useful aspects flagged:
• CIS
• Imbalance between Member States benefiting by hosting funded actions
Impacts Key Points • The Programme is perceived as adding value to the regional agreements and
is recognised as unique in supporting co-operation at European level;
• Many key areas of the Programme have been identified as “most useful” by different Member States; the most commonly selected are training and workshops, exchange of information and enhancing co-operation between Member States;
• Views on the value of the Community Information System are highly polarised but some Member States do use it regularly and find it helpful;
• Actions in Training and Information are highly valued, particularly by smaller countries that do not run their own national training programmes;
• Feedback from participants attending training courses is generally very positive but several stakeholders have commented on the perceived variable quality of the courses;
• Stakeholders are positive about the value of Pilot Projects but it is too soon to draw clear conclusions on their impact and value for money;
AEAT in Confidence AEAT/ED50287/Final Report
Mid-term evaluation on the implementation of the framework for co-operation in the field of accidental or deliberate marine pollution’ – Final Report
AEAT in Confidence AEA Technology 19
• Support and Information Actions are not well-recalled by stakeholders;
• Stakeholders are generally very positive about the Mobilisation of Experts and the co-ordinating role of the Commission. However, there is considerable confusion about some aspects, perhaps because this action is now supported by the Community Mechanism.
Recommendations for Action • Consider ways of bringing about improvements to the Community
Information System;
• Increase intervention to improve overall quality of courses;
• Ensure feedback from a higher proportion of training course participants and review; assess the longer-term impacts of training;
• When assessing Pilot Project proposals, ensure the general relevance of each project to Member States is fully considered;
• Review the outcome of projects and distil the key benefits and lessons of general importance to feed back to Member States; follow-up projects 6-12 months after completion to establish and report any further developments;
• Revise listings of actions within the various action areas (see also Section 3.5.2);
• Review the need to hold a workshop or other forum to discuss the Community Mechanism.
3.3 Process (efficiency and effectiveness)
3.3.1 Funding Decisions Member States correctly perceive their role within the MCMP as deciding the priorities whilst the Commission assesses the financial aspects according to its administrative arrangements. Two Member States commented specifically about the current proposal assessment process and would like the MCMP to have more influence in evaluating proposals. In general, stakeholders believe that the financing decisions are soundly based although it is felt that funding constraints mean that some worthy projects cannot be supported. One stakeholder has questioned the rationale for sending a Task Force to the Galapagos and the selection process for the Task Force. 3.3.2 Funding Split The budget is not separately allocated to each type of action but is distributed according to the agreed priority areas as discussed within the MCMP. This has resulted in the funds being spent as follows (see also Section 2.2):
Type of action Percentage of total funds spent Training & Information 52% Pilot Projects 35% Support & Information 10% Mobilisation of Experts 3%
AEAT in Confidence AEAT/ED50287/Final Report
Mid-term evaluation on the implementation of the framework for co-operation in the field of accidental or deliberate marine pollution’ – Final Report
AEAT in Confidence AEA Technology 20
Stakeholders are generally content with the split of funding and there are no issues with Training & Information receiving the majority – indeed, one stakeholder suggests this action could be increased to 65% at the expense of Pilot Projects. One stakeholder has suggested that more funding could be made available for Mobilisation of Experts (again at the expense of Pilot Projects) and another suggests that there should be a special ‘crisis’ budget for this Mobilisation of Experts. 3.3.3 Management Processes Several stakeholders comment about the lack of evaluation, feedback on benefits or critical appraisal of the outcomes of projects (see also Section 3.2.7). One suggestion is for the Commission to report in between meetings on the results and recommendations of training and projects to allow the MCMP meetings to focus more on strategic planning. Two stakeholders would specifically like to see improvements in the management of quality assurance and would like the MCMP to be more proactive in following-up on reports. Several comments are made about the strict procedures that have to be followed within the Commission, leading to a high workload for the staff within the Unit and slowing down progress. However, it is also noted that administrative aspects such as issuing of minutes and notification of meetings work well. The good contacts that exist between staff in the Unit and the MCMP are valued. One stakeholder feels that the role of the MCMP is not well defined and would like a clear definition of competences and rights. There are strongly held feelings about the specific issue of the length of notice given for training courses and workshop and the difficulties for Member States when the notice period is short. It is recommended that the Commission should agree with Member States the minimum acceptable time to be given for nominating participants for courses. A similar potential issue for course organisers is also identified whereby they are given a short timescale for organising a course and that can result in them having to bear higher costs. It has also been suggested that greater clarity could be given on the type and level of training being provided, its objectives and the required experience/interests of the participants. This information should always be provided. It is recommended that the detailed procedures for letting training contracts and inviting nominations from Member States are reviewed. Three Member States would like longer meetings of the MCMP and two of these suggest more two-day meetings. Longer meetings would give more time:
• To consider important issues; • For the MCMP to give its views; • For reports to be presented and discussed; • For building relationships between Member State representatives.
It is recommended that the Commission seeks views more formally about the support from Member States for more two-day meetings. The MCMP could be involved in identifying and agreeing specific topic areas for more detailed consideration and discussion on Day 2.
Process (efficiency and effectiveness) Key Points • Financing decisions are considered to be soundly based;
• Some stakeholders would like the MCMP to have more influence in the evaluation of proposals;
AEAT in Confidence AEAT/ED50287/Final Report
Mid-term evaluation on the implementation of the framework for co-operation in the field of accidental or deliberate marine pollution’ – Final Report
AEAT in Confidence AEA Technology 21
• Stakeholders are supportive of the majority of funding being provided for training actions.
Recommendations for Action • Review the detailed procedures for letting training contracts, particularly to
ensure adequate notice of training course dates can be given both to Member States and contractors;
• Obtain views from all Member States about their potential support for more two-day meetings of the MCMP.
3.4 Sustainability
3.4.1 Dissemination Views seem to be equally split between those who are happy with project dissemination, those who do not seem to be aware of dissemination activities and those who would like to see improvements. Specific suggestions for improvement include:
• Better use of the web site;
• Executive summaries written to a common format giving lessons learnt and benefits (see also Section 3.2.7);
• Action plan following a final report (next steps, MCMP recommendations for follow-up);
• Emphasis on high quality of reports that are well-written and distributed to Member States in hard copy;
• Wider dissemination of procedures, manuals, rules etc that result from projects.
There is a suggestion that the MCMP is sometimes too politically correct, hinting at an unwillingness on the part of the Committee members to criticise work carried out in another Member State. Dissemination of project outcomes and recommendations for ways in which stakeholders can make use of the findings is extremely important if full value is to be obtained from funding invested. This is an aspect of the Programme that should be kept under regular review. It is therefore recommended that the Commission considers:
• Providing detailed guidance to contractors on delivery of a project summary in a standard format to ensure conclusions, benefits and lessons learnt are clearly identified and set down. This is particularly important for pilot projects but can also be required in modified form for other actions;
• Ensuring there is adequate time at MCMP meetings to discuss completed projects (it is noted that action has already been taken by the Commission in this area and that time has been allocated for such a discussion at the meeting to be held on 1st March 2004);
• Identifying follow-up actions and a procedure to ensure actions are closed out.
Sustainability Key Points • Several stakeholders feel there is a lack of evaluation, feedback on benefits
AEAT in Confidence AEAT/ED50287/Final Report
Mid-term evaluation on the implementation of the framework for co-operation in the field of accidental or deliberate marine pollution’ – Final Report
AEAT in Confidence AEA Technology 22
or critical appraisal of the outcomes of projects;
• Views are divided on the effectiveness of dissemination of project findings.
Recommendations for Action • Allocate resources to improving the quality, dissemination, and follow-up of
project deliverables.
3.5 Synthesis
3.5.1 Overall Assessment of the Programme 6 stakeholders gave a percentage figure for the extent to which each felt the Programme has met its objectives. The figures varied from 70% to 95% and averaged 80%. These values indicate that the Programme is perceived as successful but with some scope for improvement. The most effective of the action areas are seen as:
• Training and information • Mobilisation of experts • MIC • Observer missions • CIS • Building relationships between Member States (and others) • Coordination activities • Exchange of experts (based on previous experience before the Framework
Programme)
Once again, training is judged to be particularly successful and was selected by several stakeholders. 3.5.2 The Future All stakeholders consider the work carried out within the Programme is still necessary and several are very definite about this. However, 4 stakeholders raise questions about the future role of EMSA and the need to avoid overlap and duplication. It is emphasised that operational activities are the responsibility of the Member States and that the Programme and EMSA should provide only co-ordination support during incidents. One stakeholder believes the Programme to be necessary but would welcome a critical analysis of the benefits to be certain. Another mentions the importance of the Programme as a focal point in Europe, particularly with the involvement of new countries. There is a long list of key areas identified by the stakeholders to address next, adding further weight to the perceived value of the Programme. Many of these topics have already had some coverage within the Programme but stakeholders would like more. The list covers:
• Actions in training and information – courses, workshops and exercises; • More basic training; • More training on anti-pollution matters for decision-makers; • Pilot projects; • Environmental impacts; • Exchange of experts; • Surveillance by aircraft; • Use of satellite observation;
AEAT in Confidence AEAT/ED50287/Final Report
Mid-term evaluation on the implementation of the framework for co-operation in the field of accidental or deliberate marine pollution’ – Final Report
AEAT in Confidence AEA Technology 23
• Certification on new anti-pollution equipment; • Detection of chemical spills; • Response to hazardous and noxious substances; • Pollution from offshore wind farms; • Workshops on recent incidents, response activities, difficulties encountered, lessons
learnt. • Improved system of collection of evidence of illicit discharges to assist in judicial
cases and in support of international conventions as well as European regulators;
A number of these areas are worthy of consideration at the next review of priority areas within the Programme. Some stakeholders suggest ways in which the Programme should be improved in future. One proposition is to hold a workshop on the topic of ‘The EU Programme: how to proceed and how to improve’. On a similar theme, another suggestion is to review the types of action within the Programme with involvement and comment from the MCMP. Other suggestions for improvement include:
• Better dissemination and promulgation of lessons learnt; • Enhanced web site; • More effective participation of Member States.
The matter of dissemination and promulgation of lessons learnt is a recurring theme and recommendations in this area are made in Section 3.4. Improvements to the web version of the CIS are discussed in Section 3.2.5. There is scope for improvement to the Commission’s marine pollution web site more generally and it is recommended that the site is reviewed with a view to updating its content, providing more links to other web sites and improving user-friendliness. Lack of time in MCMP meetings appears to be the main limitation to the full participation of Member States. More two-day meetings as suggested in Section 3.3.3 are an option to reduce this problem, provided the majority of Member States are prepared to commit the time.
Synthesis Key Points • The Programme is perceived as effective in meeting its objectives but with
scope to improve;
• Stakeholders are positive that the Programme continues to be necessary;
• There is uncertainty about the activities to be carried out within EMSA and concern to avoid overlap and duplication;
• Stakeholders would generally like the focus on Training and Information actions to continue in future;
• There is a healthy list of topic areas that stakeholders would like to be addressed in future within training and other actions;
• The current actions and coverage of the Programme are supported and there are no suggestions for major changes;
• Some possible improvements to the Programme have been identified. Recommendations for Action
AEAT in Confidence AEAT/ED50287/Final Report
Mid-term evaluation on the implementation of the framework for co-operation in the field of accidental or deliberate marine pollution’ – Final Report
AEAT in Confidence AEA Technology 24
• The Programme should continue in its current format;
• Potential improvements should be prioritised for action and implementation, taking account of resources available;
• Improvements to the Commission’s marine pollution web site should be considered.
3.6 Comparison with Previous Evaluation
The most recent previous evaluation was completed in 19991 and covered three periods of Community action:
• An ex post evaluation of the Programme from 1994-1997; • An intermediate evaluation of the ongoing Programme for 1998-1999; • An ex ante evaluation of the proposed Programme from 2000-2004.
The action areas within the Programme from 1994 onwards were similar to the current ones and support was provided for training and seminars, pilot projects, a conference, a programme to exchange experts between countries and a post-spill survey. Many of the findings were similar to those reported here:
• The Programme was relevant to the needs of Member States; • The Programme was generally considered to have met its objectives; • Most Member States considered training to be the most valuable action; • Opinions on the usefulness of the CIS varied widely (at this time, a web-based
version of the CIS was not available).
Various recommendations were made, some of which have been addressed by the Commission, notably:
• ‘Focus on the development of more specialised training courses’ – there have been courses supported by the Programme covering most of the specific areas mentioned;
• ‘Involvement of professionals from those maritime states seeking accession to the EU’ – Norway and Iceland have attended MCMP meetings and were invited to training courses and Estonia, Hungary, Lithuania and Slovenia attended the MCMP meeting in October 2003 for the first time;
• ‘Papers should be sent out in good time for meetings’ – no adverse comments were received and one Member State has specifically commented that issuing of minutes and notification of meetings works well.
One recommendation that was made previously and that appears again here is the need for more discussion of the results of pilot projects and ensuring their effective dissemination.
1 Peter Hayward Associates, ‘Evaluation of the Community Framework for Co-operation in the Field of Accidental Marine Pollution’, London UK, March 1999.
AEAT in Confidence AEAT/ED50287/Final Report
Mid-term evaluation on the implementation of the framework for co-operation in the field of accidental or deliberate marine pollution’ – Final Report
AEAT in Confidence AEA Technology 25
4 Baltic Carrier Case Study
The following activities funded or co-ordinated by the Commission were carried out under the Framework Programme during the Baltic Carrier incident:
• Mobilisation of experts • Mobilisation of observers • Project on Monitoring and Assessment of Environmental Effects in Gronsund
The incident therefore provides an opportunity to assess some very specific examples of key areas of activity within the Programme and to obtain feedback from the range of stakeholders involved in the various different actions. The purpose of this case study was therefore to evaluate in more detail some aspects of the Programme that were investigated in a more general way during the main assessment. The case study is not intended to review the incident itself in any detail but only to assess the impact of the Programme actions. To this end, questionnaires were devised to obtain the perceptions of stakeholders as follows:
• Danish authorities (Danish Emergency Management Agency – DEMA); • Task Force members; • Environmental Effects project leader (Storstrom County, Aquatic Environmental
Division); • Member States.
These questionnaires are reproduced in Appendix 3. Responses were obtained from representatives of each of these stakeholders, although only three Member States completed and returned the questionnaire. No questionnaire was prepared specifically for the observers as their role only emerged during the course of the assessment. Nevertheless, the views of one observer were obtained via a Member State questionnaire and a follow-up telephone discussion. The following sections provide a very brief background to the incident and the actions together with the key points made by the stakeholders.
4.1 Background to the Baltic Carrier Incident and Actions
On March 29th 2001 the tanker Baltic Carrier collided with the freighter Tern in the Baltic Sea between Denmark and Germany. Approximately 2,400 tonnes of heavy fuel oil were released into the sea and, although a significant amount of oil was recovered at sea, there was considerable pollution of the islands and coastline in the Gronsund area. The clean-up operations extended well into 2002 until they were officially ended in April 2002. The oil spill was the largest in Danish history. Soon after it became apparent that Denmark was responding to a major incident, the Commission offered help to the Danish Authorities (DEMA) and, with their agreement, issued a call for experts from amongst Member States to form a Task Force to visit and assist the Authorities. At around the same time, it was agreed with the Authorities that they would be prepared to host observers from Member States and the Commission therefore issued a call for nominations of observers.
AEAT in Confidence AEAT/ED50287/Final Report
Mid-term evaluation on the implementation of the framework for co-operation in the field of accidental or deliberate marine pollution’ – Final Report
AEAT in Confidence AEA Technology 26
The Task Force was made up of three experts who visited Denmark between 1st and 5th April 2001. They were hosted by DEMA and visited 10 different sites at which they offered suggestions and advice related to collection of oil in near-shore waters, shoreline clean-up and treatment of oily waste. Soon after their return, the Task Force issued a report of their activities. This is available on the Commission’s web site. The observers group was made up of representatives from a significant proportion of Member States, predominantly those from countries to the north of Europe. They were briefed by the Danish Authorities, provided with maps and assigned a co-ordinator to accompany them on visits. They were taken to see a forward control base, a polluted salt marsh, a waste handling facility and a vehicle clean-up operation. Storstrom County was the authority responsible for monitoring of the affected coastal waters and shoreline. After the accident, the Aquatic Environmental Division of Storstrom County worked with the National Environmental Research Institute (NERI) and Roskilde University (RUC) to devise a monitoring programme. The study was approved by the Commission for part-funding (Euro 97.3k, 49% of the total project cost) from Framework Programme funds in July 2001. The study investigated the ecological impact of the spill and its economic implications. It also evaluated the clean-up operation, the monitoring programme and the environmental effects of the spill. A final report of the study was published in 2002 and is available on the Commission’s web site.
4.2 Mobilisation of the Task Force
Table 4.1 summarises the views and perceptions of the various stakeholders. It must be recognised that the incident occurred nearly 3 years ago and some recollections of details related to the work of the Task Force will doubtless have faded. Nevertheless, there are some useful and interesting comments. The objectives of the Task Force as perceived by the Danish authorities and the Task Force itself are significantly different. This could be an artefact of the passage of time and difficulty in remembering but, if a real difference existed, this would be of concern. It does appear there is a need for clarification about the role of experts, as suggested by a Member State not involved with the Task Force at the Baltic Carrier incident. Related comments about the role of observers described in Section 4.3 reinforce the need for such a discussion. In addition, it would facilitate the mobilisation process and provide the potential for greater acceptance of expert views if countries could commit to the inclusion within contingency plans of some positive reference to hosting experts (and observers) during spill incidents and listening to advice given. Clearly there are some significant issues to debate and resolve but there could be clear benefits to all parties if there was such a commitment. Although the final report of the Task Force is available from the Commission’s web site, it appears that the Member States responding to the questionnaire are not aware of (or have forgotten) this fact. Clearly the experiences and lessons learnt from any incident will be felt and retained most clearly by the affected country and by the members of any Task Force sent to assist. Nevertheless, it should be of considerable value to other Member States to hear about those lessons that are generally applicable. Wherever possible, adequate time should be devoted in MCMP meetings or in other fora to provision of such feedback. The workshop arranged recently in Catania in the aftermath of the Prestige incident is a good example of one approach that appears to have been very much appreciated by the MCMP.
AEAT in C
onfiden
ceAEAT/E
D50287/F
inal
Rep
ort
M
id-t
erm
eva
luat
ion o
n t
he
imple
men
tation o
f th
e fr
amew
ork
for
co-o
per
atio
n
in t
he
fiel
d o
f ac
ciden
tal or
del
iber
ate
mar
ine
pollu
tion’ – F
inal Rep
ort
AEAT in C
onfiden
ce
AEA T
echnolo
gy
27
Tab
le 4
.1 F
eed
back
on
th
e M
ob
ilis
ati
on
of
Exp
ert
s at
the B
alt
ic C
arr
ier
Inci
den
t
Dan
ish
Au
tho
riti
es
Task
Fo
rce M
em
ber
Oth
er
Mem
ber
Sta
tes
Under
standin
g o
f th
e pro
cess
for
iden
tify
ing a
nd m
obili
sing e
xper
ts
One
does
under
stan
d t
he
pro
cess
, one
does
n’t a
nd o
ne
does
but
would
fin
d a
n
outlin
e use
ful. T
her
e co
uld
be
a sk
ills/
exper
ience
data
base
main
tain
ed
to iden
tify
and s
elec
t ca
ndid
ate
s to
re
pre
sent
the
Tas
k Fo
rce
duri
ng
inci
den
ts.
Obje
ctiv
es o
f th
e Tas
k Fo
rce
To s
uper
vise
and g
ather
info
rmat
ion f
or
futu
re inci
den
ts.
To d
eter
min
e w
ays
in w
hic
h t
he
EC
could
hel
p t
he
Dan
ish a
uth
orities
to
min
imis
e th
e dam
age
from
the
spill
, to
as
sist
in t
he
reco
very
and t
o s
upport
fu
ture
work
ass
oci
ated
with t
he
spill
.
Wer
e th
e obje
ctiv
es a
chie
ved?
DEM
A w
as s
atisf
ied w
ith t
he
Tas
k Fo
rce
report
. H
opef
ully
yes
. D
ifficu
lt t
o p
rovi
de
tech
nic
al ex
per
tise
duri
ng r
esponse
oper
atio
ns
when
the
auth
ori
ties
had
ve
ry little
tim
e to
hea
r outs
ide
advi
ce.
Adeq
uate
bri
efin
g a
nd a
dm
inis
trative
su
pport
and c
o-o
per
atio
n f
rom
the
Com
mis
sion a
nd t
he
Dan
ish a
uth
ori
ties
D
efin
itel
y ye
s to
all
aspec
ts.
Any
feed
back
pro
vided
about
the
work
of
the
Task
Forc
e?
Ver
y lim
ited
am
ount
pro
vided
.
Any
spec
ific
contr
ibution o
f th
e Tas
k Fo
rce
to t
he
clea
n-u
p o
per
ations
The
Task
Forc
e gav
e ad
vice
. Sev
eral
asp
ects
incl
udin
g d
iscu
ssio
n o
f m
echan
ical
rec
ove
ry t
echniq
ues
, build
ing a
tem
pora
ry s
tora
ge
site
and
eval
uation o
f th
e ex
tent
of
the
pollu
tion.
Not
gen
erally
aw
are.
One
Mem
ber
Sta
te
knew
that
the
Tas
k Fo
rce
bro
ught
exper
ience
of
resp
onse
to s
pill
s of
hea
vy f
uel
oils
.
Wer
e an
y re
com
men
dations
imple
men
ted?
All
advi
ce w
as w
elco
med
on h
igh
visc
osi
ty H
FO.
How
ever
, th
e only
sp
ecific
pro
posa
l im
ple
men
ted
conce
rned
a w
arnin
g f
rom
the
Tas
k Fo
rce
about
a la
rge
oil
slic
k th
at w
as
about
to d
rift
aw
ay w
ith t
he
tide.
As
a re
sult,
the
oil
was
conta
ined
.
The
Task
Forc
e dis
cuss
ed e
nvi
ronm
enta
l m
onitori
ng a
t so
me
length
and,
subse
quen
t to
the
main
oil
reco
very
oper
atio
ns,
a r
esea
rch p
rogra
mm
e w
as
initia
ted b
y th
e au
thori
ties
.
AEAT in C
onfiden
ceAEAT/E
D50287/F
inal
Rep
ort
M
id-t
erm
eva
luat
ion o
n t
he
imple
men
tation o
f th
e fr
amew
ork
for
co-o
per
atio
n
in t
he
fiel
d o
f ac
ciden
tal or
del
iber
ate
mar
ine
pollu
tion’ – F
inal Rep
ort
AEAT in C
onfiden
ce
AEA T
echnolo
gy
28
Will
the
exper
ience
and les
sons
lear
nt
by
the
Task
Forc
e re
sult in m
ore
ef
fect
ive
inte
rven
tion in o
wn/o
ther
co
untr
y?
Co-o
rdin
ation b
etw
een r
esponsi
ble
au
thori
ties
has
impro
ved in D
enm
ark
thro
ugh b
ette
r co
ntingen
cy p
lannin
g.
For
exam
ple
, th
ere
are
now
pre
-ap
poin
ted p
lace
s fo
r te
mpora
ry s
tora
ge
of
oily
was
te.
(Note
: it is
not
clea
r th
at
this
im
pro
vem
ent
has
res
ulted
direc
tly
from
the
Tas
k Fo
rce
invo
lvem
ent)
.
Yes
. Le
ssons
lear
nt
are
alw
ays
use
ful
for
impro
ving p
repar
ednes
s in
the
affe
cted
countr
y an
d m
ore
wid
ely
thro
ugh info
rmat
ion f
rom
the
Tas
k Fo
rce.
The
exper
ience
has
bee
n
par
ticu
larly
use
ful fo
r th
e co
untr
y su
pply
ing t
he
exper
ts.
None
can c
om
men
t sp
ecific
ally
bec
ause
not
awar
e of
less
ons
lear
nt
from
this
in
ciden
t. H
ow
ever
, one
Mem
ber
Sta
te is
very
support
ive
of
the
pri
nci
ple
of
additio
nal ex
per
tise
and a
dvi
ce f
rom
the
Tas
k Fo
rce,
pro
vided
the
nat
ional
au
thori
ties
inci
den
t co
mm
and s
truct
ure
is
gea
red t
o r
ecei
ve a
nd d
eal
const
ruct
ivel
y w
ith t
he
advi
ce –
could
be
cove
red w
ithin
contingen
cy p
lans.
Anoth
er M
ember
Sta
te is
stro
ngly
in
favo
ur
of
the
use
of
spec
ific
exp
ertise
fr
om
countr
ies
that
hav
e par
ticu
lar
spec
ialis
t kn
ow
ledge
or
capab
ilities
.
What
was
the
most
succ
essf
ul as
pec
t of
the
Task
Forc
e m
obili
sation?
The
sugges
tions
conce
rnin
g t
he
monitori
ng p
rogra
mm
e as
re
com
men
ded
in t
he
final re
port
.
Exc
han
ge
of
info
rmat
ion o
n t
he
way
s of
dea
ling w
ith a
spill
such
as
this
. O
nly
one
com
men
t – b
ringin
g
exper
ience
fro
m p
revi
ous
spill
s.
What
was
the
leas
t su
cces
sful as
pec
t?
Rec
om
men
dat
ions
about
clea
nin
g o
f pollu
ted b
each
mat
eria
l.
Wea
k in
volv
emen
t in
the
follo
w-u
p,
nota
bly
the
envi
ronm
enta
l m
onitori
ng
that
could
als
o h
ave
iden
tified
les
sons
for
the
futu
re f
or
oth
ers.
One
Mem
ber
Sta
te t
hin
ks t
hat
the
curr
ent
pro
cess
could
ignore
sig
nific
ant
exper
tise
within
the
EU
. Anoth
er
men
tions
the
lack
of
report
ing o
f th
e w
ork
of
the
Task
Forc
e.
Should
the
Com
mis
sion c
ontinue
to
pro
vide
fundin
g f
or
mobili
sation o
f ex
per
ts a
t fu
ture
inci
den
ts?
Yes
bec
ause
larg
e oil
spill
s m
ay
thre
ate
n s
ever
al c
ountr
ies.
D
efin
itel
y ye
s. T
he
Task
Forc
e or
obse
rver
s ca
n p
rovi
de
oil
spill
ex
per
ience
to t
he
affe
cted
countr
y an
d
can f
eedbac
k le
ssons
lear
nt
to o
ther
co
untr
ies
at
a Euro
pea
n lev
el.
Yes
. O
ne
sugges
tion is
that
it w
ould
be
ben
efic
ial if t
he
Com
mis
sion p
rovi
ded
fu
nds
for
Tas
k Fo
rce
exper
ts t
o join
IT
OPF
exp
erts
at
inci
den
ts o
uts
ide
Euro
pe.
Oth
er c
om
men
ts
No
Rea
dy
to r
evie
w e
xper
ience
s in
Euro
pe
or
else
wher
e w
her
e ex
change
of
info
rmation c
an h
elp in r
espondin
g t
o
spill
inci
den
ts o
r pla
nnin
g f
or
futu
re
spill
s.
The
role
of
the
exper
ts s
hould
be
dis
cuss
ed in d
epth
at
an M
CM
P m
eeting.
AEAT in Confidence AEAT/ED50287/Final Report
Mid-term evaluation on the implementation of the framework for co-operation in the field of accidental or deliberate marine pollution’ – Final Report
AEAT in Confidence AEA Technology 29
4.3 Mobilisation of Observers
The mobilisation of observers at the Baltic Carrier was perceived as useful and successful by those Member States that have commented on it. The Danish authorities appear to have been very efficient and effective in organising the tour for the observers and to have been particularly helpful in co-operating and providing a co-ordinator from DEMA to accompany the observers. However, some interesting comments have been received about aspects of the mobilisation and role of the observers in-country. The call for nominations from the Commission for an expert group was apparently confused with calls for an observer group. It seems that this resulted in some observers trying to give advice to the response authorities - but not to those in charge of the operations who might (or might not) have welcomed the advice. Comments from others suggest this experience has been repeated in other spill incidents. Once again, it seems it would be helpful to discuss and clarify the role of observers at incidents within a MCMP meeting. Brief Terms of Reference and/or guidance notes could be written and a procedure put in place to ensure observers receive such information before any involvement with the affected country. The apparent confusion about whether Member States were being asked to nominate experts or observers should be easily remedied by a more careful choice of wording by the Commission in any call.
4.4 Project on Monitoring and Assessment of Environmental Effects in Gronsund
Feedback was obtained from the beneficiary of the Programme funding (Storstrom County), the Danish authorities (DEMA) and three Member States. 4.4.1 Beneficiary Feedback The questions for Storstrom County were intended to address aspects of the funding process, project management and additionality of the funding. The Programme funding was felt to be important because of the limited money available from the local or regional environmental authorities. So, although it is judged that the project would have proceeded without Programme support, it would have been significantly smaller in scope. There were meetings between the Storstrom County representative in Brussels and staff in the Unit before the proposal was finalised. However, the Commission was not kept informed of progress during the project lifetime. It appears, therefore, that there was no mechanism for the Commission to be informed about possible delays or interim findings to enable it to intervene in case of problems or to identify possible beneficial changes in direction. It is good practice to require regular, brief reports (usually monthly) from contractors of progress against milestones for all projects with a duration greater than 1 or 2 months. Certainly the Commission should automatically require this reporting for all Pilot Projects and, where appropriate, for Training and Information projects (at the discretion of the Commission). In this case, the project was perceived by Storstrom County to have achieved its original objectives and to have been completed to schedule. Storstrom County was paid on time by the Commission, although some costs (layout, printing and translation) were found to be higher than expected.
AEAT in Confidence AEAT/ED50287/Final Report
Mid-term evaluation on the implementation of the framework for co-operation in the field of accidental or deliberate marine pollution’ – Final Report
AEAT in Confidence AEA Technology 30
The findings from the project have been widely disseminated through distribution of the report to Member States, the availability of the report on the Commission’s web site and through presentations in Denmark, France and Finland. 4.4.2 Feedback from Danish Authorities DEMA is aware of the project, has received a copy of the final report and therefore knows about the findings. The findings of particular interest from the monitoring programme relate to the remaining oil in the sediments and in mussels. DEMA is also interested in the evaluation of the clean-up operations. DEMA is supportive of the Commission continuing to fund such environmental impact projects and is keen to exchange experiences in order to improve their response to spills in future. 4.4.3 Feedback from Member States Two of the 3 individuals responding are aware of the project but have not seen the report. One Member State was particularly interested in the specific issues associated with clean-up of a marsh/reed-bed whilst another was interested in the method of monitoring. On the question of the need for continued funding for projects such as this, one Member State believes that most Member States are capable of funding these projects themselves but suggests that funds could be made available, if a special request is made. Another Member State is clear that funds should only be granted if the project is carried out with the involvement of the authorities in the country affected. There is also a question about whether funding for such projects could be recovered from the IOPC Fund. It is suggested that the Commission considers discussing the principles of this (and the possibility of recovering Task Force costs) with the Fund managers. 4.4.4 Case Study Findings versus Findings from the Main Evaluation All the Baltic Carrier actions have generally been favourably received by all parties and the mobilisation of experts and observers, in particular, is confirmed as very valuable. Some recommendations made earlier in this report, such as the need for improved dissemination of information, have been reinforced. If at all possible, more time should be allocated at MCMP meetings for presentation and discussion of findings, experiences and lessons learnt from incidents. The Case Study has highlighted some specific issues that the Commission should consider:
• Discussion and clarification within the MCMP of the role of experts in the Task Force and of observers;
• Consideration of obtaining some further formal commitment in principle from Member States to accept advice from the Task Force and to host observers;
• The requirement for contractors to report regularly on the progress of pilot projects and possibly of some other actions;
• The possibility of recovering funding from the IOPC Fund for the costs of the Task Force and of environmental assessment projects following spill incidents.
AEA Technology
Appendices
CONTENTS
Appendix 1 Main Evaluation Questionnaire Appendix 2 List of Stakeholders Providing Input Appendix 3 Case Study Questionnaires Appendix 4 Training Course Evaluation
Appendix 1 Main Evaluation Questionnaire
AEA Technology
Mid-term evaluation on the implementation of the framework for co-operation in the field of accidental or deliberate marine pollution Evaluation Questionnaire Theme 1 – Overview and Context 1.1 Your role What is your job and areas of responsibility? How do you link with the Programme? 1.2 Programme Objectives What is the Programme aiming to achieve?
1.3 Interactions of the Programme with other Programmes What other related activities are you aware of within your own country? What trans-national activities? How does this EC Programme link in with these other activities (eg regional agreements/conventions, European Maritime Safety Agency)? Is there any duplication? How can it be avoided? 1.4 Barriers to Co-operation Are there any barriers? What are they? 1.5 Gaps Are there any gaps in coverage that the EC Programme should fill? Theme 2 – Impacts 2.1 Programme Successes What have been the Programme’s greatest successes? Can you give specific examples and quantify the benefits, if possible? Why have you selected these? How was this success achieved? What role did the Programme play – did the EC have direct control or an indirect influence? 2.2 Added Value What would have happened if the EC Programme had not been in place? What has been the added value of the Programme? (refer back to answers given in 1.4 to check additionality) 2.3 Programme Shortcomings Are there any areas of the Programme that have not been successful? Why not? What could have been done to address the problems? 2.4 Value for Money Does the Programme give good value for money? Are the results obtained in proportion to the resources deployed? 2.5 Community Information System (CIS) Has the CIS helped the National Authorities when dealing with accidental or deliberate pollution incidents? How has it contributed?
AEA Technology
2.6 Actions in Training and Information Have these actions resulted in any modification of the national approach to prevention and intervention? To what extent? Was co-operation between Member States created or enhanced? Examples? What projects contributed directly to improving training and information? 2.7 Pilot projects Did any projects lead directly to investment in resources, procedures, equipment, or networking? To what extent were there improvements? 2.8 Support and Information Actions To what extent did they contribute to strengthening co-operation in risk prevention/response? 2.9 Mobilisation of Expertise To what extent did this lead to more effective intervention? 2.10 Programme Value What is the most useful aspect of the programme to you? What is least useful?
Theme 3 – Process (efficiency and effectiveness) 3.1 Funding Decisions Are the financing decisions soundly-based? Are the best proposals supported? Are the most necessary proposals supported? 3.2 Funding Split (currently 52% Training & Information; 35% Pilot Projects; 10% Support % Information; 3% Mobilisation of Experts) Is the allocation of funding between the various actions appropriate? If not, what changes should be made? 3.3 Management processes What have been the strengths and weaknesses of the overall management of the programme? In what ways did the Commission control or influence the outcome of projects? Are you aware of any data collected or indicators used to measure project performance compared with objectives? If so, are they appropriate? If not, what could be used? Should the Management Committee have a bigger role? Theme 4 - Sustainability 4.1 Dissemination Have project outputs been effectively disseminated? In what ways? Any suggestions for improvements? Theme 5 – Synthesis 5.1 Overall assessment of the Programme To what extent do you feel the Programme has met its objectives (give a percentage)? Which are the most effective of the action areas (CIS, Training and Information, Pilot Projects, Support and Information Actions, Mobilisation of Experts)? 5.2 The future Is the Programme still necessary? What are the key areas to address next? How should the Programme be improved in future?
AEA Technology
Appendix 2 List of Stakeholders Providing Input
NAME Status
Nature of Contact
Belgium
Thierry Jacques Belgian Representative, MCMP Email & telephone
Denmark
Peter Soeberg Poulsen Danish Representative, MCMP Email
Jorn Allan Pedersen Regional Commander, Danish Emergency Management Agency
Jakob Lysholdt Sorensen Associated Head of the Aquatic Environmental Division, Storstrom Region
France
Daniel Sylvestre French Representative, MCMP Face-to-face
Christophe Rousseau CEDRE Email & telephone
Stephane Le Floch Task Force Member at the Baltic Carrier
Germany
Berthold Tiefenbach German Representative, MCMP Email & telephone
Greece
Yannis Karageorgopoulos
(with input from the Merchant Marine Ministry)
Greek Representative, MCMP Email & telephone
Ireland
Eugene Clonan Irish Representative, MCMP Email & telephone
Italy
Francesco Valentini Italian Representative, MCMP Face-to-face
Netherlands
Sjon Huisman Netherlands Representative on MCMP
Email & telephone
Theo Kramer Netherlands Representative, MCMP Email & telephone
Norway
Jan Nerland Norwegian Representative, MCMP Email
Portugal
Raul Valente Portuguese Representative, MCMP Face-to-face
Sweden
Thomas Fago Swedish Representative, MCMP Email
NAME Status
Nature of Contact
United Kingdom
Toby Stone UK Representative, MCMP Face-to-face
Brian Elliott Observer at the Baltic Carrier Email & telephone
European Commission
Pia Bucella Head of Unit, Civil Protection and Environmental Accident Unit
Face-to-face
Gilles Vincent Deputy Head of Unit, Civil Protection and Environmental Accident Unit and Chairman of MCMP
Face-to-face, email & telephone
Ioanna Sgourdopoulou-Karra
DG ENV Unit D3 Face-to-face, email, telephone
Thomas de Lannoy DG ENV Unit D3 Face-to-face, email
Horst Miska Monitoring & Information Centre Face-to-face
European Maritime Safety Agency
James Wood EMSA Telephone
International Tanker Owners Pollution Federation
Tosh Moller ITOPF Email & telephone
International Maritime Organisation
Jean-Claude Sainlos IMO Email & telephone
Appendix 3 Case Study Questionnaires
Questions for Danish Authorities at the Baltic Carrier Incident
A. Mobilisation of Experts
Please give your name and position within your own organisation.
What was your own role during the Baltic Carrier incident?
What was your involvement with the European Commission’s Task Force (Gilles Vincent, Stephane Le Floch, and Bernard Le Guen) during the Baltic Carrier incident?
What do you think were the objectives of the Task Force?
Do you think they achieved their objectives? If not, why not?
What specific contribution did the Task Force make to the clean-up operations?
Did you implement any recommendations made by the Task Force?
Do you think the involvement of the Task Force at the Baltic Carrier incident will result in more effective intervention in Denmark in any future incident? If so, in what ways?
What do you think was the most successful aspect of the Task Force mobilisation?
What was the least successful aspect?
Do you think the Commission should continue to provide funding for the mobilisation of experts at future incidents?
Do you have any other comments about the involvement of the Commission’s Task Force at the Baltic Carrier incident?
B. Environmental Assessment survey following the accident of the Baltic Carrier
Are you aware that the Commission part-funded a project in 2001/02 to assess the environmental impact of the oil spill from the Baltic Carrier?
Were you informed about the findings from the project? If so, how?
Were any recommendations from the project of particular interest or relevance to you? If so, can you remember which ones?
Do you think the Commission should continue to provide funding for projects following spill incidents? If so, why?
Thank you for taking the time to complete this questionnaire! Please return it to Madeleine McDonagh at madeleine.mcdonagh@aeat.co.uk by Friday 23rd January.
Questions for Task Force Members at the Baltic Carrier Incident
Please give your name and position within your own organisation.
Why do you think you were selected to represent the Commission within their Task Force?
What were the objectives of the Task Force?
Did you achieve your objectives? If not, why not?
Did you receive adequate briefing and administrative support from the Commission?
Did you receive adequate administrative support and co-operation from the Danish response authorities?
Were you briefed adequately by the Danish authorities on your arrival?
Was there any resistance to your full involvement from the Danish authorities or others?
What specific contribution did the Task Force make to the clean-up operations?
Do you know if your recommendations were taken up by the Danish response authorities?
Do you think the involvement of the Task Force at the Baltic Carrier incident will result in more effective intervention in future:
(i) in Denmark? (ii) in your own organisation/country? (iii) in other countries?
If so, in what ways?
What was the most successful aspect of the Task Force mobilisation?
What was the least successful aspect?
Do you think the Commission should continue to provide funding for the mobilisation of experts at future incidents?
Do you have any other comments about your experience at the Baltic Carrier incident?
Thank you for taking the time to complete this questionnaire! Please return it to Madeleine McDonagh at madeleine.mcdonagh@aeat.co.uk by Friday 23rd January.
Questions for Member States about the Mobilisation of Experts at the Baltic Carrier Incident
Please give your name and country.
A. Mobilisation of Experts
Were you aware that oil spill experts were mobilised by the European Commission during the Baltic Carrier incident?
Do you understand the Commission’s process for identifying and mobilising experts?
Do you think that the Task Force sent to the Baltic Carrier incident was of an appropriate size and composition (there was one expert from the Commission and two from CEDRE)?
Was any feedback provided to you about the work of the Task Force?
Do you know if the Task Force made any specific contribution to the clean-up operations in Denmark? If so, what?
Do you think the experience and lessons learnt by the Task Force at the Baltic Carrier incident will result in more effective intervention in future in your own organisation/country? If so, in what ways?
From what you know, what do you think was the most successful aspect of the Task Force mobilisation at the Baltic Carrier incident?
What was the least successful aspect?
Do you think the Commission should continue to provide funding for the mobilisation of experts at future incidents?
Do you have any other comments about the involvement of the Commission’s Task Force at the Baltic Carrier incident?
B. Environmental Assessment survey following the accident of the Baltic Carrier
Are you aware that the Commission part-funded a project in 2001/02 to assess the environmental impact of the oil spill from the Baltic Carrier?
Were you informed about the findings from the project? If so, how?
Were any recommendations from the project of particular interest or relevance to you? If so,
can you remember which ones?
Do you think the Commission should continue to provide funding for projects following spill incidents? If so, why?
Thank you for taking the time to complete this questionnaire! Please return it to Madeleine McDonagh at madeleine.mcdonagh@aeat.co.uk by Friday 23rd January.
The Baltic Carrier Oil Spill Monitoring and Assessment of Environmental Effects in Gronsund AEA Technology is evaluating the European Commission’s marine pollution framework programme on behalf of DG ENV and has agreed with the Commission to use the Baltic Carrier incident as a case study. One way in which the Commission supported the activities in Denmark associated with the incident was to part-fund the project led by Storstrom County. We would like to understand your views about the support provided by the Commission for this project and would be very grateful if you could complete the short questionnaire below. Questions for Storstrom County, Aquatic Environmental Division
Please give your name and position within Storstrom County.
What was your role in the environmental assessment project?
Would the project have proceeded without the funding provided by the Commission?
Was the project completed within the original estimates of time and cost?
How was the Commission kept informed about progress?
Was there a meeting at any time between your organisation and a Commission representative to discuss the project?
Did the project achieve its original objectives?
Were you paid on time by the Commission?
Do you think the Commission should continue to provide funding for projects following spill incidents? If so, why?
Thank you for taking the time to complete this questionnaire! Please return it to Madeleine McDonagh at madeleine.mcdonagh@aeat.co.uk by Friday 23rd January.
Appendix 4 Training Course Evaluation
Introduction In total, there have been 18 courses/workshops run by 8 countries attended by an average of 23 participants /course representing an average of 10 countries/course. This Appendix provides an assessment of the evaluation forms collected from training course participants (not workshops) and provided by the Commission. Survey responses were received for all training courses EXCEPT the following:
• Analysis of recent major accidents
• Workshop – Defining the content of a course on Harmful Substances
110 individual responses were received from a minimum of 203 people who attended the 12 courses for which survey responses were received. For those courses where survey responses were received, the overall survey response rate is (at best) 54%.
This assessment involved designing an Excel spreadsheet to enable the aggregation and averaging of responses to be automated. The information on the evaluation forms was manually input to the spreadsheet together with individual’s comments where applicable. The findings are summarised below based on the questions in the forms. How well were the event’s objectives achieved?
• 75% of respondents answered this question
• 71% of those that answered this question thought that the event’s objectives were achieved “well”
• A further 20% thought the objectives were achieved “completely”.
• No respondents thought that the event’s objectives were “not very well” or “not at all” achieved
Were participants’ objectives achieved?
• 76% of respondents answered this question
• 63% of those that answered this question thought that their own objectives were achieved “well”
• A further 21% thought that their objectives were achieved “completely”
• 13% thought their objectives were “partly” achieved.
• 2% of those that answered this question thought that their own objectives had not been very well achieved, whilst no respondents thought that their objectives had “not at all” been achieved
Relevance of training to job
• 97% of respondents answered this question
• 53% of those that answered this question thought that the courses were “very relevant” to their job
• A further 46% thought that the courses were “relevant” to their job, whilst only 1% thought they were “of little relevance”
Content covered
• 98% of respondents answered this question
• 85% thought the content covered was “about right”
• 9% thought there was “too much” content
• 6% thought there was “too little” content
Level of content
• 98% of respondents answered this question
• 85% thought the level was “about right”
• 6% felt the courses were “too advanced”
• 8% felt the courses were “too elementary” Length of event
• 98% of respondents answered this question
• 74% thought the length was “about right”
• 19% thought the courses were “too long”
• 6% thought the courses were “too short”
Quality of trainers / key speakers
• 75% of respondents answered this question
• Those that answered this question gave the following grades:
* 0 = Poor; 3 = Very good
Quality of visual aids
• 97% of respondents answered this question
• Those that answered this question gave the following grades:
1%0
6%1
33%2
61%3
Proportion of respondents awarding grade
Grade for quality of visual aids
0%0
1%1
41%2
57%3
Proportion of respondents awarding grade
Grade for quality of trainers*
Quality of handouts and course documentation
• 97% of respondents answered this question
• Those that answered this question gave the following grades:
Overall rating for course events
• 98% of respondents answered this question
• Those that answered this question gave the following grades:
Should these types of events be repeated?
• 98% of respondents answered this question
• 97% of those that answered this question (105 out of 108 people) said “Yes”, these types of events should be repeated
• The three “No” responses referred to the following three courses:
- Human Environmental Life Protection
- Response to Oil Pollution on the Sea and on the Coast
- Airborne Pollution Surveillance
• In each case, only one participant thought the course should not be run again
0%0
5%1
35%2
61%3
Proportion of respondents awarding grade
Grade for quality of handouts
0%0
4%1
40%2
56%3
Proportion of respondents awarding grade
Overall grade for courses