8/6/2019 Evaluation-Goals and Principles_Benoit Nadler

http://slidepdf.com/reader/full/evaluation-goals-and-principlesbenoit-nadler 1/13

ENEN

Regional Policy 

EUROPEAN COMMISSION

Evaluation :

goals and principles

Evaluation :

goals and principles

Benoit Nadler DG REGIO

Bucharest ± 9 October 2008

8/6/2019 Evaluation-Goals and Principles_Benoit Nadler

http://slidepdf.com/reader/full/evaluation-goals-and-principlesbenoit-nadler 2/13

Regional Policy 

EUROPEAN COMMISSION

Evaluation 

2007 -13

Evaluation 

2007 -13

2

« ..improve the quality, effectiveness and consistency 

of  the assistance from the Funds and the str ategy and

implementation of  oper ational progr ammes« » - art 

47(1) regulation 1083/2006.

� Evaluation is a decision making tool :

� To be carried out together with the design of  projects /

progr ammes;

� To inter actively feed the decision making process

� Requiring initiative and strong ownership and

commitment from the competent authority (decision 

maker )

Main goalsMain goalsMain goalsMain goals

8/6/2019 Evaluation-Goals and Principles_Benoit Nadler

http://slidepdf.com/reader/full/evaluation-goals-and-principlesbenoit-nadler 3/13

Regional Policy 

EUROPEAN COMMISSION

Evaluation 

2007 -13

Evaluation 

2007 -13

3

Evaluation is not :

1. A bureaucr atic duty decoupled from the actual

decision making process (cf SEA)

2. An audit or a control « blaming » the responsible 

authorities for  possible weaknesses

Diffusion of  lessons learnt

accountability

Learning tool instead of irritatingLearning tool instead of irritatingbureaucratic requirement :bureaucratic requirement :

Learning tool instead of irritatingLearning tool instead of irritatingbureaucratic requirement :bureaucratic requirement :

8/6/2019 Evaluation-Goals and Principles_Benoit Nadler

http://slidepdf.com/reader/full/evaluation-goals-and-principlesbenoit-nadler 4/13

Regional Policy 

EUROPEAN COMMISSION

Evaluation 

2007 -13

Evaluation 

2007 -13

4

I m pa c ts

U tility

S t i ility

Soc i  

ty 

E c onom y 

E nv ir on-

men t 

¡ 

r og r ¢ 

mme

E va lu a ti onR l

ffi i y

ff ti

Ou tpu tsIn pu tsObje ctives

Ou tc m es /

esu lts

N eeds

p r  b lem s

is sues

³Realistic´ evaluation³Realistic´ evaluation -- the logicalthe logicalframeworkframework

³Realistic´ evaluation³Realistic´ evaluation -- the logicalthe logicalframeworkframework

8/6/2019 Evaluation-Goals and Principles_Benoit Nadler

http://slidepdf.com/reader/full/evaluation-goals-and-principlesbenoit-nadler 5/13

Regional Policy 

EUROPEAN COMMISSION

Evaluation 

2007 -13

Evaluation 

2007 -13

5

Level of 

assistanceObjective Indicators

 NSRF

 

Sustained economic

growth in a region

GDP per capita

OperationalProgramme

 

Improvecompetitiveness

of SMEs

Value added inmanufacturing sector 

Priority axis

 Reinforce research,

technological

developmentand innovation

Innovation rate

(e.g. patents)

Operations Increase

technological level

of firms

 Networking/

Collaborative

 projects

Connection Between Levels and EffectsConnection Between Levels and Effectson Interventionon Intervention

Connection Between Levels and EffectsConnection Between Levels and Effectson Interventionon Intervention

8/6/2019 Evaluation-Goals and Principles_Benoit Nadler

http://slidepdf.com/reader/full/evaluation-goals-and-principlesbenoit-nadler 6/13

Regional Policy 

EUROPEAN COMMISSION

Evaluation 

2007 -13

Evaluation 

2007 -13

6

� Evaluation shall be objective and

evidence-based :

 ± Need for  sound monitoring and reliable 

indicators;

 ± Appropriate targets

 ± Independent experts

� Proactive attitude needed to detect 

possible future difficulties : from mid

term evaluation to on-going evaluations

Evaluation and monitoringEvaluation and monitoringEvaluation and monitoringEvaluation and monitoring

8/6/2019 Evaluation-Goals and Principles_Benoit Nadler

http://slidepdf.com/reader/full/evaluation-goals-and-principlesbenoit-nadler 7/13

Regional Policy 

EUROPEAN COMMISSION

Evaluation 

2007 -13

Evaluation 

2007 -13

7

From regulation-based to needs-driven:

� Departure from goals initially set

� Significant progr amme revision

� Points of  interest (eg specific themes)

� Improving progr amme performance

From midFrom mid--term to ongoing evaluationterm to ongoing evaluationFrom midFrom mid--term to ongoing evaluationterm to ongoing evaluation

8/6/2019 Evaluation-Goals and Principles_Benoit Nadler

http://slidepdf.com/reader/full/evaluation-goals-and-principlesbenoit-nadler 8/13

Regional Policy 

EUROPEAN COMMISSION

Evaluation 

2007 -13

Evaluation 

2007 -13

8

� Strategic : focus on specific themes of  

str ategic importance for  oper ational

progr ammes. Contributes to the str ategic 

reporting.� Operational : review the quality and

relevance of  objectives, analysing date on 

financial and physical progress and providing 

recommendations� Evaluations carried out at the initiative of :

 ± The member State; 

 ± The Commission ± already sever al studies going 

on

Two t pes of evaluationTwo t pes of evaluationTwo t pes of evaluationTwo t pes of evaluation

8/6/2019 Evaluation-Goals and Principles_Benoit Nadler

http://slidepdf.com/reader/full/evaluation-goals-and-principlesbenoit-nadler 9/13

Regional Policy 

EUROPEAN COMMISSION

Evaluation 

2007 -13

Evaluation 

2007 -13

9

a) Where the monitoring s stems reveal a significantdeparture from the goal initiall set

� financial + physical monitoring data

� 10%-20% departure (specific features of OP, scale of  inter vention ±proportionality principle)

� ³ear ly warning mechanism´: in good time to allow progr amme ad justments

� Linked of  course to progr amme revision«

b) When programme revisions are proposed

Significant changes:

� Financial: between priority axes

� Content-related: revision of  objectives at progr amme or  priority level

Nature of  evaluation follows reason for  revision:

� Socio-economic changes, progr amme environment

� Changes in Community, national or  regional priorities

� Implementation difficulties

Operational evaluationOperational evaluationOperational evaluationOperational evaluation

8/6/2019 Evaluation-Goals and Principles_Benoit Nadler

http://slidepdf.com/reader/full/evaluation-goals-and-principlesbenoit-nadler 10/13

Regional Policy 

EUROPEAN COMMISSION

Evaluation 

2007 -13

Evaluation 

2007 -13

10

� Evolution of  socio-economic situation, change of  policy priorities, continued relevance of  str ategy

� Specific themes (eg innovation, SMEs)

� Contribution to str ategic reporting (Art. 27 and 28)

Progress towar ds meeting objectives: outputs, results

� Interf ace between monitoring and evaluation (quantification of  impact indicators)

� Delivery mechanisms

� Efficiency related issues

=> Dr aw conclusions to improve the quality and effectiveness

Be ond Legal Requirements: PolicBe ond Legal Requirements: Polic --Oriented EvaluationsOriented Evaluations

Be ond Legal Requirements: PolicBe ond Legal Requirements: Polic --Oriented EvaluationsOriented Evaluations

8/6/2019 Evaluation-Goals and Principles_Benoit Nadler

http://slidepdf.com/reader/full/evaluation-goals-and-principlesbenoit-nadler 11/13

Regional Policy 

EUROPEAN COMMISSION

Evaluation 

2007 -13

Evaluation 

2007 -13

11

� Dedicated staff within the managing authorities;

� Coor dination and consistency throughout the evaluations;

� Appropriate tr aining and commitment

� EC support : guidance (wor king documents) andavailability for  support (geogr aphical and evaluation units)

� The evaluation approach should not be dedicated to the mere cohesion policy but disseminate throughout the entire administr ation

Administrative capacitAdministrative capacitAdministrative capacitAdministrative capacit

8/6/2019 Evaluation-Goals and Principles_Benoit Nadler

http://slidepdf.com/reader/full/evaluation-goals-and-principlesbenoit-nadler 12/13

Regional Policy 

EUROPEAN COMMISSION

Evaluation 

2007 -13

Evaluation 

2007 -13

12

� Not mandatory, but suggested ± good

evaluations can have long lead times!

� Suggested content: proposals, resources, 

capacity-building

� Goal: good evaluations, not perfect plans

� « Living » document, updated over  time

Evaluation PlansEvaluation PlansEvaluation PlansEvaluation Plans

8/6/2019 Evaluation-Goals and Principles_Benoit Nadler

http://slidepdf.com/reader/full/evaluation-goals-and-principlesbenoit-nadler 13/13

Regional Policy 

EUROPEAN COMMISSION

Evaluation 

2007 -13

Evaluation 

2007 -13

13

Cost benefit anal sisCost benefit anal sisCost benefit anal sisCost benefit anal sisFor  ma jor  projects 

� Different purpose : project related

� Specific expertise required� Tailored sector al guidelines

� Attention to be paid to the under lying 

assumptions

� Economic expertise needed within the 

administr ation� Financial and economic analysis to apply for  

national projects