11

Evaluating administrative and institutional capacity building

International Evaluation & Methodology Conference

6-7 May 2010 Budapest

Anna Galazka European Commission,

DG Employment, Social Affairs & Equal Opportunities,Evaluation and Impact Assessment Unit

22

Outline

Rationale for the evaluation Rationale for the evaluation

Purpose and evaluation questionsPurpose and evaluation questions

Analytical framework and methodologyAnalytical framework and methodology

33

What is administrative capacity?What is administrative capacity?

““the process through which individuals, organizations the process through which individuals, organizations and societies obtain, strengthen and maintain the and societies obtain, strengthen and maintain the capabilities to set and achieve their own development capabilities to set and achieve their own development objectives over time” UNDPobjectives over time” UNDP

““the capacity to manage the complex processes and the capacity to manage the complex processes and interactions that constitute a working political and interactions that constitute a working political and economic system” World Bankeconomic system” World Bank

44

Rationale for the EvaluationRationale for the Evaluation

55

General aspectsGeneral aspects

Importance of administrative capacity for the Importance of administrative capacity for the economy and for the people;economy and for the people;

Administrative and institutional reforms in the last Administrative and institutional reforms in the last 20 years;20 years;

Role of the EU in the administrative capacity Role of the EU in the administrative capacity building.building.

66

ESF Regulation – the legal basisESF Regulation – the legal basis

Strengthening institutional capacity and the efficiency of:public administrations and public services at

national, regional and local level;and, where relevant, of the social partners and

non-governmental organisations,

with a view to reforms, better regulation and good governance especially in the economic, employment, education, social, environmental and judicial fields.

77

Purpose of the Evaluation & Evaluation Purpose of the Evaluation & Evaluation QuestionsQuestions

88

Purpose of the evaluationPurpose of the evaluation

To assess the relevance, effectiveness, efficiency To assess the relevance, effectiveness, efficiency and sustainability of the ESF 2007-13 interventions and sustainability of the ESF 2007-13 interventions in the administrative and institutional capacity in the administrative and institutional capacity building;building;

To provide recommendations on the priority for To provide recommendations on the priority for 2014-2020 programming period, particularly the role 2014-2020 programming period, particularly the role of the Commission.of the Commission.

99

Evaluation questionsEvaluation questions

What was the context in which administrative and institutional capacity building interventions have taken place?

What is the perceived impact of the EU pre-accession and ESF support to administrative and institutional capacity building in the Member States?

How could a future ESF priority on administrative and institutional capacity building look like? - What could be the role of the Commission?

1010

Analytical framework and methodologyAnalytical framework and methodology

1111

Methodology step by stepMethodology step by step

Scoping analysis – identifying issues and Scoping analysis – identifying issues and interventions relevant for administrative interventions relevant for administrative capacity building – literature reviewcapacity building – literature review

Context analysis - inventory of administrative Context analysis - inventory of administrative capacity building interventions in 10 MS - capacity building interventions in 10 MS - national reportsnational reports

Mapping of the relevant ESF interventionsMapping of the relevant ESF interventions

1212

Scoping – dimensions of administrative Scoping – dimensions of administrative capacity buildingcapacity building

Policy – organisational developmentPolicy – organisational development legislation;legislation; management;management; cooperation.cooperation.

Human Resources – individual capacity Human Resources – individual capacity developmentdevelopment people;people; competences/skills;competences/skills; operating budget.operating budget.

Systems and tools - development of instruments, Systems and tools - development of instruments, methods, guidelines, manuals, procedures, formsmethods, guidelines, manuals, procedures, forms

1313

Issues/Issues/

IntervenInterventionstions

CapacityCapacity

Po

licy Ma

nag

em

en

t Cycle

Po

licy Ma

nag

em

en

t Cycle

StructuresStructures Human ResourcesHuman Resources Systems Systems & tools& tools

LegislationLegislation ManagementManagement CooperatCooperationion

PeoplePeople SkillsSkills BudgetBudget

Preparatio

Preparatio

nnIm

plementatio

Implem

entationn

Monitoring/

Monitoring/

evaluationevaluation

1414

Context analysis – problem factors in MS Context analysis – problem factors in MS administrationsadministrations

an inherited political culturean inherited political culture

weak structures and cooperationweak structures and cooperation

skills and competence gaps amongst civil servantsskills and competence gaps amongst civil servants

1515

Mapping of ESF Mapping of ESF Administrative Administrative Capacity Building ActivitiesCapacity Building Activities

Types of activitiesTypes of activities Countries where Countries where this activity is this activity is implemented implemented

National (N) /National (N) /

Regional (R) /Regional (R) /

Local (L) Local (L)

Main indicator Main indicator used used

Implementation of Implementation of strategic plans strategic plans and/or and/or organisational organisational development plans development plans

RomaniaRomania

HungaryHungary

LithuaniaLithuania

GreeceGreece

N, R, LN, R, L

N, LN, L

N, R, LN, R, L

NN

Number of Number of administrative administrative bodies affected by bodies affected by organisational organisational development development measures measures

Activities to Activities to enhance enhance modernisation of modernisation of financial financial management management

PolandPoland

GreeceGreece

NN

NN

Number of Number of institutions receiving institutions receiving support for support for performance-based performance-based budgeting budgeting

1616

Example: Challenges to Human Resource Example: Challenges to Human Resource Development Strategies in National AdministrationsDevelopment Strategies in National Administrations

Can not be addressed by ESF interventions as Can not be addressed by ESF interventions as presented in the OPs:presented in the OPs:

Outdated strategy (RO);Outdated strategy (RO); Lack of responsibility (RO);Lack of responsibility (RO); Lack of awareness on importance HR (RO).Lack of awareness on importance HR (RO).

Can be addressed by ESF interventions as presented Can be addressed by ESF interventions as presented in the OPs:in the OPs:

Lack of sound analysis for HRD development (BG);Lack of sound analysis for HRD development (BG); Lack of trained staff in HR departments (BG);Lack of trained staff in HR departments (BG); Lack of integrated strategy (HU/SL/BG).Lack of integrated strategy (HU/SL/BG).

1717

Difficulties for the evaluationDifficulties for the evaluation

Lack of proper indicators Lack of proper indicators

Most of the capacity-building projects have Most of the capacity-building projects have just startedjust started

Limited transparency, dependency from Limited transparency, dependency from political decisionspolitical decisions

1818

Next steps

Case studies - selected projectsCase studies - selected projects

Validation of findings through national focus Validation of findings through national focus

groups groups

Interviews Interviews with key experts from international with key experts from international

organisations organisations

Developing recommendations Developing recommendations

Central workshop on the future role of ESF Central workshop on the future role of ESF

Final report planned for September 2010Final report planned for September 2010