Managing for Results Results Based Management in practice

Trine Eriksen and Alex Stimpson Financial Mechanism Office Vilnius, 11 December 2013

1

Johannes Hahn (Oct. 2013) “billions of euros of EU taxpayers' money has in the past been spent with no clear strategy or benchmarks” "The progress has to be measured and, crucially, communicated," said the commissioner.

RESULTS BASED MANAGEMENT

What is Results Based Management (RBM) RBM is a management strategy by which all actors, contributing directly or indirectly to achieving a set of results, ensure that their processes, products and services contribute to the achievement of desired results (outputs, outcomes and impact)

In the context of the Grants: A result is the output and the outcome of a development intervention A risk is an event that may occur and impede the objective

3

What is a result and a risk?

KEY CONCEPTS

Results achievement Beneficiary oriented Managing for results and reduced risks

4

In practice RBM is about • defining realistic expected results based on appropriate

analysis; • clearly identifying program beneficiaries and designing

programmes to meet their needs; • monitoring progress toward results and resources consumed

with the use of appropriate indicators; • identifying and managing risk; • increasing knowledge by learning lessons and integrating them

into decisions; and • reporting on the results achieved and resources involved.

RBM is supposed to help us:

– clarify what we mean by results and lets us deal with differences of understanding before a programme/project begins

– build capacity, because it clarifies for us what we need to concentrate on – produce more realistic schedules, forcing us to think through the preconditions

and sequence for actions – prepare programmes/projects for effective evaluations. – better identify unplanned results, as they occur, and assess if these are desirable,

or problematic, requiring support or coping strategies – making the links between inputs, funded activities and the results they should be

leading to, which reduces the potential for corruption

However…

• Agreeing on “clear” goals and objectives is often difficult • Using results-information to continually improve one’s own work and operations

requires time to reflect and acceptance from management • Ensuring that staff members can employ their creativity and resourcefulness to

engage on the “quest” for results requires a mind shift within the organisation In other words: red tape often remains and hampers the efforts of public agencies to increase their results-orientation

The challenge is to:

• Keep it simple • Getting stakeholder “buy-in” • Fostering organisational use of information • Making outcome accountability “operational”

and realistic • Measure • Ensure attribution • Linking financial and performance information

• The role of the PO is to keep the focus on the big picture during implementation – To create a simple and clear roadmap that guides

the project promoters

• What the donors (and we) want to be able to do, is to explain our results clearly to other people

Part I Quality at entry - Basis for good management

10

The Programming Cycle and Continuous Feedback

Planning

Programming

Budgeting

Implementation Monitoring

Evaluation

Reporting

PROGRAMMING CYCLE

11

Identify main programme elements of EEA/N Grants Objective (impact) Widespread improvements in societ or a sector

32 predefined objectives

Outcome(s) Intermediate effects on target groups and systems

160 predefined outcomes In DoRIS data base

Outputs Products and services delivered

Self-defined (at least two outputs per outcome)

In DoRIS data base

Activities Tasks transforming inputs to outputs

Self-defined Not in DoRIS

Inputs Financial, human and material resources

Budget + Budget in DoRIS

12

Programme logic

Inputs

Activities

Outputs

Expected outcomes

Objective

Overall objectives

if the inputs are available, then the activities will take place

which will contribute to the two overall objectives

if the activities take place, then the outputs will be produced

which will contribute to the fulfilment of the objective if the outputs are produced, then the expected outcome is likely to be achieved

Should be able to guarantee that this will happen

Likely to happen

Contribute to

Contribute to

13

ASSUMPTIONS

• The intervention logic is an aim-means chain and a cause-effect chain.

• It is assumed that “certain events” need to occur at one level in order to achieve the desired effect at the next level.

• It is not certain that these events happen; they represent uncertainties.

14

There are Assumptions between each level: Identify Assess Monitor Manage and

Mitigate

Impact/Objective

Outcome

Outputs (deliverables)

Activities

Inputs

15

Identify important assumptions

• Assumptions are worded as positive conditions (see objectives) • Assumptions are linked to the different levels of the

Programme Matrix. Each level must contain the necessary and sufficient conditions for the next level above.

• Start from bottom of matrix and work upwards • Are weighted according to importance and probability Example: Scholarship recipients return to assigned jobs

16

Programmes and projects

Programme proposal

Strategic approach

Projects Pre-defined projects

Pre-defined projects

17

EXAMPLE Green Industry Innovation

29

Increased competitiveness of green enterprises, including greening of existing industries, green innovation and green entrepreneurship

Realisation of the business opportunities of greening of the European economy

Environmental technologies in industry developed, improved or implemented by SMEs

Green products or material in industry developed or improved by SMEs

• Environmental technology development

• Implementation or replacement of the existing technology

• Processing sequence and/or synthesis of these innovations in order to run the processes at a higher efficiency and minimize waste, energy and emission generation

• New green product and material development or modification of the existing product and material to minimize environmental impact during or after its use.

Objective (pre-defined)

Expected outcome(s) (pre-defined)

Outputs

Types of projects

Call for proposals

Be clear about which outcomes and outputs you want the projects to contribute to. Which projects are necessary and sufficient? Project template in DoRIS: http://www.eeagrants.org/id/3192

19

Logical relationship

20

PART IV

21

Basis for results measurement

Basis for measuring results

Result based management

22

"The starting point for a results-oriented approach is ex ante setting of clear and measurable targets and outcome indicators. Indicators must be clearly interpretable, statistically validated, truly responsive and directly linked to policy intervention, and promptly collected and publicised." From the conclusions of the 5th Cohesion Report, adopted by the European Commission in November 2010

QUALITY AT ENTRY!

30

INDICATOR

A means to measure achievement, or to help assess a development

24

HOW SHOULD INDICATORS BE FORMULATED? • Once indicators have been identified, they should be

formulated to reflect measurable change. Indicators frequently utilize the following metrics to depict measurable amounts:

• Number; • Ratio; • Percentage; or • Rate; • Time Indicators shall whenever relevant, be broken down per gender

In DoRIS, always specify unit of measurement!

25

LESS IS MORE

EXAMPLES OF INDICATORS Example 1: Outcome: Cultural heritage restored, renovated and protected Indicator: Number of buildings of cultural heritage value restored or rehabilitated Example 2: Outcome: Awareness of cultural diversity raised and intercultural dialogue strengthened Indicator: Number of local cultural associations involved in the implementation of projects

26

INDICATORS

To describe how results will be measured, output indicators must be specific, observable and come from reliable data. • Does the indicator provide the most direct evidence

of the condition or result to be measured? • Is the indicator important, understood, and accepted

by important stakeholders?

27

Think about…

• Are data for the indicator collected in the same way over time?

• Will data be available for the indicator? • Will data be collected frequently enough? • Are data currently being collected? • If data are not currently collected, can cost effective

instruments for data collection be developed quickly?

28

Indicators related to Bilateral Relations

• All programmes should contribute to strengthened Bilateral Relations

• Use the Bilateral guideline. Four types of bilateral results expected: Extent of cooperation Shared results Wider effects Knowledge and mutual understanding 29

Indicators related to «Horizontal issues»

• A human-rights based approach and concerns for horizontal issues like fighting discriminiation, racism, hate speech etc. should guide defining the outputs and the indicators.

Example: % of members of X Parliament are minorities Example: Number of annual incidences of hate speech in Y school

30

Programme matrix - indicators

Sources of Verification

• When defining the indicators, consider the data sources. Where can you easily get data that are reliable and timely? • Specify your sources of information

• Assess where you can get easy access to data Example Ministry of Health’s statistics on cancer (health status of population) and various statistics on cancer detection and treatment services

Programme matrix – Sources of Verification

WHAT ARE BASELINES? • Baselines provide context for the setting of

targets and capture the situation before a development intervention begins, or at the beginning of a time period that will be monitored and assessed.

• Baselines describe the conditions prior to programming efforts. The present situation!

• Essential to make credible and meaningful assessments on progress towards outcomes. 34

Programme matrix – Baseline data

WHAT ARE TARGETS? • Targets set the level for the magnitude of change

expected by the end of a predetermined time period. They reflect what organizations commit themselves to achieve by the end of some duration of time.

• Targets provide tangible and meaningful points of

discussion with beneficiaries, stakeholders, and partners, and allow us to add further specificity to the outcomes from the results logic.

36

TARGETS

• When you have defined the indicators and know the baseline situation, you set the targets.

• What do you want the situation for the beneficiaries to be in 3 or 5 years?

• Examples: • 1. 60% of W 40+ called in for cancer test in Southern Region by year

2016. We want to increase the % of W 40+ called in from 30% in 2010 to 60% in 2016. • 2. Time lag between detection of cancer to operation is not more

than 3 months in 2016.

37

BASELINES AND TARGETS SUMMARY • A baseline is the value of the indicator before the

implementation of the project starts • Targets orients stakeholders to the tasks to be

accomplished • Targets help establish clear expectations • Targets serve as the guideposts for monitoring

whether progress is being made • Targets promote transparency

38

Part IV Quality at implementation • Monitoring • Reporting on results and risks • Evaluation

39

The Programming Cycle and Continuous Feedback

Planning

Programming

Budgeting

Implementation Monitoring

Evaluation

Reporting

PROGRAMMING CYCLE

40

PERFORMANCE MONITORING • Define processes and systems for performance

measurement; • Decide the intervals at which measurement will occur; • Assign roles and responsibilities; • Set standard requirements for project promoters • Build in adequate time and resources for analysis of

performance measurement data so that progress is reviewed and assessed.

• Use web site for communication with project promoters and other stakeholders

41

Performance monitoring

• Status regarding the output and outcome indicators is measured and reported towards the baseline situation (“where we were”), and also towards the future, desired situation (“the targets”).

Programme performance framework

Group work – Results based management • Questions for discussion: With the principles of results based management in mind, in your view: • Is there a clear intervention logic (link between

outputs/outcomes/objective)? • Are the Outputs sufficient to achieve the Outcome? • Are the indicators good measures of achievement? • Is it clear what the indicators refer to/mean?

44