United We Stand. Evaluating, Cooperating, and other Unlikely Stories of Evaluation Capacity

Building in Italy

Martina Bolli, Silvia Ciampi, Francesco Giordano & Laura Tagle

Italy’s National Evaluation System

Content

1.Cohesion policy in Italy: long-term

background & contrasting

directions

2.The Italian evaluation policy

3.Efforts & (unlikely) achievements…

plus some drawbacks

COHESION POLICY IN ITALY: LONG TERM BACKGROUND & CONTRASTING DIRECTIONS

2007-2013 - Evaluation as one of the pillars of a (much needed) renewal of Italy’s regional policy

• Citizen-centered policies

• Strong focus on essential services—rather than on procedures & expenditure

• Emphasis on local knowledge

• Responsibility to regional governments

• Links with national policies

• Greater importance to administrative capacity

The regional (= cohesion) & rural development policy in Italy

• Intentionality of territorial goals

• Additionality vis-à-vis ordinary policy

Resources come from EU & National funds

Levels of programming: a complex framework

• National “umbrella” strategies for regional & rural development policies

• 7 Ministries develop their own programs• Each of the 21 Regions develops

– its multi-sector strategy– 5 Programs (each program is funded by one source)

“Naturally”, each of the 5 programs is usually managed by a separate Directorate

“Programs” end up being bureaucratic constructs rather than guides for action

Over time, silos have been created

Sources of money

• EU Regional Fund• National funds • EU Social Fund• EU Rural Development

Coordination organisms

• Ministry for economic development

• Ministry for agriculture• Ministry for labour

Other departments

….Beyond silos

2007-2013: intentional efforts to bridge gaps & introduce policy dialogue among and within

authorities in a new perspective

At the same time, EU regulations ask for separate

programmes by EU source of money (& within Italy the

momentum for integration dwindles)

The National Strategy for 2007-2013 tried to make the various sources of funding work together through common:

• Programming

• Implementation

• Monitoring

• Surveillance

• Evaluation

21 Regions (70%) 6 Ministries (30%)

Centre-North: 15% South: 85%

3bln€ to be given to Southern Regions on the basis of performance in waste, water, education and social services

Regional development strategyBetween 2007 & 2010: lower national resources (other than national cofinancing)•different distribution North-South•the instruments needed to implement nationally-relevant innovations (among which greater integration) dwindle

THE ITALIAN EVALUATION POLICY

Evaluation capacity building/1

From EU evaluation requirements towards a national policy

• Institutional building:– Creation of Evaluation Units within executive branch of

Regional authorities – Network of Evaluation Units

• Specialized: – Regional and Rural Development policies only

• Both demand & supply side– Improve regional authorities’ ability to Demand for & Use

evaluations

– Improve Evaluation Units ability to Manage and Conduct evaluations

Evaluation capacity building/2

• Organized guidance & support (National Evaluation System)

• Enabling approach:– No rigid prescriptions– Reputational mechanisms

• Legitimize & support innovators at regional level

Evaluation policy /1

• Nationally-funded interventions must be evaluated--not only those funded by EU

• Objects: – effects of a well-defined (however complex) intervention or group

of interventions on a problem, a social group, or an area—rather than a bureaucratic construct (e.g., a multisectoral “program”—tipically operating on a large area, affecting millionsof people)

– across bureaucratic borders: the evaluation object may have been financed by more than one source

– selectivity: choose to answer defined questions about “crucial & controversial issues”

Evaluation policy/2

• Each Region drafts an Evaluation Plan:– What is evaluated (and what is not)

– Who (internally or externally) performs each evaluation

– When

– Organization of evaluation function

– Resources (human and financial) devoted to evaluations

The National Evaluation System• UVAL – Ministry of Economic Development

(Coordinates)

• INEA – Ministry for Agricultural, Food and Forestry Policies

• ISFOL – Ministry of Labour

• Department for Equal Opportunities

• Evaluation Units of Regional Authorities

Its composition shows a clear intention to work together across silos

& involving Regions in a federal function

The National Evaluation System Each component differs from the others

• Central units:– Have differing rules, interpretations, goals, tasks &

status

• Regional evaluation units:– Conduct or manage evaluations…– … but also provide project selection, planning,

knowledge management to the Region– Have background and skills closer to regional fund

policies (or national funds) rather than rural development or active labour policies

EFFORTS & (UNLIKELY) ACHIEVEMENTS… PLUS SOME DRAWBACKS

Common guidance - include & adjust for differing rules, traditions, goals

Personalized support to Regions (all managing authorities together)

Ensure quality & evaluation of evaluations (....)

Observe processes & disseminate information (jointly)

Facilitate dissemination (conference-like meetings, workshops, create & maintain eval database www.dps.tesoro/valutazione.it)

A “federal” function performed across silos & by central & regional units

Some results

• All Regions developed an Evaluation Plan http://www.dps.tesoro.it/uval_linee_valutazione.asp#10

• Evaluations have been launched:– and even completed: 50! (in addition to the mandatory ex

post evaluations of rural development)

– and a lofty 60% available on the internet

• Evaluation units (established between 2000 & 2002) have started conducting internal evaluations

Still far from perfection but …

Why we are happy anyway…

• Activities started (and 50 concluded) even though – (or just because?) there are no enforceable

requirements (no sanctions) – there is still strong inertia in the system

• old evaluations fulfilled needs of managing authorities, national coordinators, and Commission rapporteurs

• restricted market with “marriages” between managing authorities & evaluators

– still little pressure from social partners & general public

• Innovations go together: evaluations seem to be used as an instrument by Regions willing to improve

• Innovations implemented where there has been more support from central state (not only on evaluation)

Thank you

Laura.tagle@tesoro.its.ciampi@isfol.it

Valutazione.uval@tesoro.it

Please send us feedback at these e-mail addresses