Evaluation Arrangements for 2007-2013: A Decentralized Approach Laura Tagle Evaluation Unit –...
-
Upload
isabel-rivera -
Category
Documents
-
view
218 -
download
3
Transcript of Evaluation Arrangements for 2007-2013: A Decentralized Approach Laura Tagle Evaluation Unit –...
![Page 1: Evaluation Arrangements for 2007-2013: A Decentralized Approach Laura Tagle Evaluation Unit – Department for Development Policies – Ministry for Development.](https://reader033.fdocuments.us/reader033/viewer/2022061305/5514393e550346414e8b4580/html5/thumbnails/1.jpg)
Evaluation Arrangements for 2007-2013:
A Decentralized Approach
Laura TagleEvaluation Unit – Department for Development Policies – Ministry for
Development
Rossella TarantinoEvaluation Unit – Basilicata Region
Bruxelles 26-27 June 2008
![Page 2: Evaluation Arrangements for 2007-2013: A Decentralized Approach Laura Tagle Evaluation Unit – Department for Development Policies – Ministry for Development.](https://reader033.fdocuments.us/reader033/viewer/2022061305/5514393e550346414e8b4580/html5/thumbnails/2.jpg)
Presentation Outline
• Presenters’ background: collective evaluation capacity building
• Overall approach to evaluation: decentralization, responsibility and Evaluation plans
• Main issues:– Evaluation questions– Involvement of social and economic partners
• A regional perspective on a “central” endeavour
![Page 3: Evaluation Arrangements for 2007-2013: A Decentralized Approach Laura Tagle Evaluation Unit – Department for Development Policies – Ministry for Development.](https://reader033.fdocuments.us/reader033/viewer/2022061305/5514393e550346414e8b4580/html5/thumbnails/3.jpg)
National Evaluation System
• Work group: collective work for evaluation capacity development
• Individuals from:– Evaluation Unit of Ministry for Development
(responsible for work group coordination)– ISFOL (Agency responsible for evaluation of ESF)– INEA (Agency responsible for evaluation of Rural
Development)– Regional Evaluation Units
![Page 4: Evaluation Arrangements for 2007-2013: A Decentralized Approach Laura Tagle Evaluation Unit – Department for Development Policies – Ministry for Development.](https://reader033.fdocuments.us/reader033/viewer/2022061305/5514393e550346414e8b4580/html5/thumbnails/4.jpg)
Decentralization and responsibility
• Decentralization: – each Regional government responsible for evaluating its
interventions– Coordination ministry responsible for evaluating from
“central” point of view– Regional ownership: each Region asks the evaluations it
needs and times it for when it needs the results• No common evaluation questions• No prescribed timing
• Responsibility:– No sanction– Evaluation plans and evaluations are public– National Evaluation System guarantees guidance,
training, support, systematic observation and reputational mechanisms
![Page 5: Evaluation Arrangements for 2007-2013: A Decentralized Approach Laura Tagle Evaluation Unit – Department for Development Policies – Ministry for Development.](https://reader033.fdocuments.us/reader033/viewer/2022061305/5514393e550346414e8b4580/html5/thumbnails/5.jpg)
Evaluate the effects of regional policy
• Until the end of 2009, mainly ex post evaluations of past interventions
• Evaluate EU- and nationally funded interventions• Evaluate effects of joint interventions on areas/groups• Evaluation questions on specific issues, groups, areas,
NOT generic “program”
![Page 6: Evaluation Arrangements for 2007-2013: A Decentralized Approach Laura Tagle Evaluation Unit – Department for Development Policies – Ministry for Development.](https://reader033.fdocuments.us/reader033/viewer/2022061305/5514393e550346414e8b4580/html5/thumbnails/6.jpg)
Steering Groups
• Ensure involvement of socio-economic partners: Steering Group at Plan level
• Ensuring qualityand independence of each evaluation: Steering Group (+ other methods) at each evaluation level. No MA.
Risks:• creating a new profession: “Steering Group
member”• Duplicating the problems of Monitoring
Committees
![Page 7: Evaluation Arrangements for 2007-2013: A Decentralized Approach Laura Tagle Evaluation Unit – Department for Development Policies – Ministry for Development.](https://reader033.fdocuments.us/reader033/viewer/2022061305/5514393e550346414e8b4580/html5/thumbnails/7.jpg)
Evaluation plans
• One plan per Region/central ministry regarding all programs (past and present) (unitary approach)
• Need to build a new internal organization• 30 programs• Only two Regions are still developing the plan (out
of 21)
![Page 8: Evaluation Arrangements for 2007-2013: A Decentralized Approach Laura Tagle Evaluation Unit – Department for Development Policies – Ministry for Development.](https://reader033.fdocuments.us/reader033/viewer/2022061305/5514393e550346414e8b4580/html5/thumbnails/8.jpg)
Evaluations planned for 2008 e 2009
• Human Resources: 5• Education: 1• Urban policies: 1• Transport: 2• Implementation analysis:
13• Program evaluation: 18
• Research & innovation : 6• Enterprises: 3• Environment: 3• Cultural heritage: 1• Gender: 3• Area-based: 9• ICT: 2
![Page 9: Evaluation Arrangements for 2007-2013: A Decentralized Approach Laura Tagle Evaluation Unit – Department for Development Policies – Ministry for Development.](https://reader033.fdocuments.us/reader033/viewer/2022061305/5514393e550346414e8b4580/html5/thumbnails/9.jpg)
Satisfaction at innovation spread, but
• Out of 67 evaluations planned for 2008-2009, 31 (+ all the regional development evaluations) still have a generic “Structural Fund program” focus:– Inertia in accepting innovations from regulations and
from NSF– Need for general overview– Incomplete or contradictory support from EC
• SNV: – strengthening our common work on ECD with EC– Increase capacity of national coordination agencies
• Further work needs to be done to involve social & economic partners
• Is a Plan Steering Group the right way to go about it?
![Page 10: Evaluation Arrangements for 2007-2013: A Decentralized Approach Laura Tagle Evaluation Unit – Department for Development Policies – Ministry for Development.](https://reader033.fdocuments.us/reader033/viewer/2022061305/5514393e550346414e8b4580/html5/thumbnails/10.jpg)
The National Evaluation System and the evaluation plans
• “Guidelines for the organisation of the evaluation activities of the regional policies: the evaluation plans for 2007-13”;
• Working groups that went through the evaluation plans drafted by Regions and Ministers according to a peer group approach. These groups were created on volunteer basis and composed of both regional and national representatives.
This cross-regional and national scrutiny of the evaluation plans enabled a pooling of experiences and know-how. for instance, after going through the evaluation plan of the Liguria Region I changed/improved the evalution plan of my Region
![Page 11: Evaluation Arrangements for 2007-2013: A Decentralized Approach Laura Tagle Evaluation Unit – Department for Development Policies – Ministry for Development.](https://reader033.fdocuments.us/reader033/viewer/2022061305/5514393e550346414e8b4580/html5/thumbnails/11.jpg)
The evaluation plan: the unitary approach
• Each Region has a unitary strategy implemented through OP’s
• One Evaluation Plan for Each Region that assess:• ERDF OP• ESF OP• EARDF OP• OP funded by National Fund for deprived areas• Interventions financed by regional resources
![Page 12: Evaluation Arrangements for 2007-2013: A Decentralized Approach Laura Tagle Evaluation Unit – Department for Development Policies – Ministry for Development.](https://reader033.fdocuments.us/reader033/viewer/2022061305/5514393e550346414e8b4580/html5/thumbnails/12.jpg)
The evaluation plan: the unitary approach
to assess all the development policies implemented by the Region
–independently by the financial source – in order to have a global overview
of the effects that these policies produce.
Obviously in the respect of:
– Specific rules concerning each Fund;– The need of the European Commission and of the
single Managing Authorities to underline the specific community added value.
![Page 13: Evaluation Arrangements for 2007-2013: A Decentralized Approach Laura Tagle Evaluation Unit – Department for Development Policies – Ministry for Development.](https://reader033.fdocuments.us/reader033/viewer/2022061305/5514393e550346414e8b4580/html5/thumbnails/13.jpg)
• To coordinate the evaluations and avoid overlapping and duplication;
• To build a global overview of the individual evaluations and thus have a big picture of the effects of the regional development strategy;
• To assess the results produced jointly by different actions on the same area or target group
• To have a shared knowlegde to improve the quality of programs and of their implementation
The unitary approach- why?
![Page 14: Evaluation Arrangements for 2007-2013: A Decentralized Approach Laura Tagle Evaluation Unit – Department for Development Policies – Ministry for Development.](https://reader033.fdocuments.us/reader033/viewer/2022061305/5514393e550346414e8b4580/html5/thumbnails/14.jpg)
Specific evaluation activities focus on themes/areas/target groups and take into account results produced by different interventions
An ex-post evaluation in my region concerns the information society and the contribution to:
• The quality of the public services (i.e; health)
• The creation of community networks (i.e. Schools)
• The territorial and social inclusion
• The competitive capacity of SMEs
How to deal with organizational and financial issues?
• Pooling of funds (liguria)
• Steering Group with the participation of all Managing Authorities and departments involved in these policies (Basilicata)
• To address specific national resources (Emilia oRmagna)
The unitary approach-how
![Page 15: Evaluation Arrangements for 2007-2013: A Decentralized Approach Laura Tagle Evaluation Unit – Department for Development Policies – Ministry for Development.](https://reader033.fdocuments.us/reader033/viewer/2022061305/5514393e550346414e8b4580/html5/thumbnails/15.jpg)
• In some regional plans there is a divide between the ERDF and ESF evaluations
• In the implementation phase, the unitary approach is even more difficult
Therefore, for us (Regions) it is important:• EC common approach• EC Common guidance• EC (DG Regio and DG Employment + DG Agri) joint
meetings and analysis with National Evaluation Systems
The unitary approach: not easy
![Page 16: Evaluation Arrangements for 2007-2013: A Decentralized Approach Laura Tagle Evaluation Unit – Department for Development Policies – Ministry for Development.](https://reader033.fdocuments.us/reader033/viewer/2022061305/5514393e550346414e8b4580/html5/thumbnails/16.jpg)
EC support and guidance
• The joint analysis on the mid-term evaluation conducted on 2004 was very useful and gave important hints to improve the quality and awareness of evaluations: – focus on specific questions, stakeholders
involvement in the definition of questions and in the dissemination of evaluations results, emphasis on effects, etc
• Guidance
• Participation in our (SNV) ECD activities