CoCreate Consultancy SPORT FOR ALL CONFERENCE, LIMA, 2013€¦ · M&E ALIGNMENT M&E quality...

CoCreate Consultancy

SPORT FOR ALL CONFERENCE, LIMA, 2013

Andy Caldwell, Managing Partner Email: [email protected] Twitter: @CoCreateTweets Website: www.cocreateconsultancy.com

COCREATE – WHAT WE DO

Social Responsibility Partnership Strategies Working with businesses, professional bodies and NGOs,

we help to develop strategies that deliver social impact

and create shared value through partnerships.

CSR Impact Measurement We work with businesses and sports bodies, helping them

to develop simple measurement processes that evaluate

the social and commercial impacts of their CSR schemes.

Events and Conferences

From concept design to hosting and speaker input, we

design events that lead to collaborative action.

We are a growing network of consultants, dedicated to helping clients to

build partnerships and strategies that create shared value. We provide:

COCREATE – WHO WE WORK WITH

M&E IN SPORT – SESSION AGENDA

Today we will aim to cover:

0-5 minutes: Welcome, introduction, agenda

5-20 minutes: Exploration of the key challenges for M&E

in sport

20-30 minutes: Practical discussion and share back –

your M&E expertise and experiences

30-40 minutes: M&E Independent Quality Standards,

including Theory of Change

40 - 45 minutes: Practical discussion – M&E next steps

45 minutes: Close

M&E IN SPORT – THE HEADLINES

1. There is an increasing focus upon Impact Measurement in Sport

2. The size of your organisation makes a significant difference

3. Widespread lack of programme theory and M& E expertise and….

4. …..a lot of jargon about M&E which increases the complexity

5. There are distinctions & dependencies between:

Monitoring Evaluation Research

…..and these are not well understood

6. A lot of data being collected - but not optimally used

7. Growing need & desire to share learning & work together

8. Nobody wants to duplicate effort or start from ‘scratch’

WHAT IS MONITORING AND EVALUATION (M&E)?

Monitor: Verb Observe and check the progress or quality of (something) over a period of time; keep under systematic review.

Evaluate: Verb [ with obj. ] Form an idea of the amount, number, or value of.

WHAT IS MONITORING AND EVALUATION (M&E)?

OTHER COMMONLY USED WORDS;

ACTIVITY: What did we do? E.g. football training

INPUT: What resources did we use? E.g. number of coaches

OUTPUT: What’s the immediate result? E.g. number of football sessions

OUTCOME: What is the change that happened as a result of our activity? E.g. increased football skills

IMPACT: What is the overall change that we contributed to? E.g. healthier youth in our community

INDICATOR: This is how we measure our outputs and outcomes. E.g. number of participants or increase in self-reported football skills

BASELINE: linked to a specific indicator, this is the first measurement of that indicator, at the start of a project, or when you first start to collect data

TARGET: linked to a specific indicator, what are we aiming to achieve

WHY DO MONITORING AND EVALUATION (M&E)?

Compliance

Competition

Improvement

IMPACT

M&E IN SPORT – CHALLENGES WE HAVE FOUND

Sport can be difficult to evaluate….


Sport can have differing aims and objectives….

And external stakeholders with their own agendas….

Conclusions:

No impact measurement of SR

activities in place

Insufficient exchange of

knowledge taking place

11


Question: The

Association should lead

the implementation of an

Asian Social

Responsibility strategy

Average response 9.0

(out of 10)

Question: There is

strong data to support the

work of the Association to

date in creating positive

social impact

Average response 5.0

M&E IN SPORT – TAKING BEST PRACTICE FROM

INTERNATIONAL DEVELOPMENT COMMUNITY

WORKSHOP STRUCTURE

Discussion:

In small groups of three or four please discuss the following:

What are the key aspects of good Monitoring and Evaluation?

Please try to be specific about each aspect, and where possible give examples of what made/makes this aspect effective.

In 5 minutes I will collect some thoughts and ideas from you.

MAKING SENSE OF M&E - QUALITY STANDARDS

.

Quality Standards

These are the first

externally verified

quality standards in

M&E

They are designed to

simplify and explain

good practice in

the M&E process

Theory of

Change

M&E Alignment

Indicators

Data Collection Data

Management

Learning and

Improving

Reporting and

Accountability

M&E

Coordination

Theory of Change is a

way of thinking about

how your daily work

helps lead towards

your long-term vision

INFOCUS QUALITY STANDARDS – THEORY OF

CHANGE

Theory of

Change


CHANGE

A way of mapping

how activities

lead to impact

Takes account of the

broader context and

external factors

Fundamental for

strategic

planning, and

also helpful as a basis for M&E

M&E – THEORY OF CHANGE

Process for regular review of

why we do what we do

input outpu

t

outcom

e

Programme plans

“our vision

is…” “our

mission is…”

Strategy

Wider

context?

Assumptions?

A way of

thinking, not

just a document

Problem statement?

Our

data shows that

the children are

getting healthier…

but is that

because of us or

something else??

Teach

football

Parents

School

Friends Other NGOs

Government

programmes

Price of food

Children get

healthier


CHANGE

Theory of Change acknowledges external factors;

THEORY OF CHANGE – STREET LEAGUE (UK)

INFOCUS QUALITY STANDARDS

.

Quality Standards

These are the first

externally verified


M&E



good practice in

the M&E process

M&E Alignment

Indicators


Management

Theory of

Change

INFOCUS QUALITY STANDARDS – DATA

MANAGEMENT

Data

Management

INFOCUS QUALITY STANDARDS

.

Quality Standards

These are the first

externally verified


M&E



good practice in

the M&E process

M&E Alignment

Indicators


Management

Theory of

Change

Theory of

Change

Learning and

Improving

Reporting and

Accountability

M&E

Coordination

M&E - DEEPER EXPLORATION

Taking Action:

In pairs take five minutes to discuss the quality

of your organisations approach to M&E;

1. Is your approach drawing on all the Quality

Standards?

2. Where do you need to focus in order to

improve your M&E approach?

3. What will your next steps be with M&E?

M&E – FINAL THOUGHTS

1. Take the long view and build a strategy accordingly

2. Involve your key stakeholders from the outset

3. Ensure a joined-up approach to M&E with your partners, members and other stakeholders

4. Deliver to your organisational strengths and bring in partners to do what they do best

5. Be prepared to learn and adjust along the way

Online support

& Resources

Online

Resources

Q&A support, M&E, research and software

Annual review

M&E Training and

Support

1:1 Support : direct M&E

support from the inFocus team, both remotely and onsite

Essentials Training Course: 10 week M&E training programme for staff across 8 inFocus modules

Software

Entering data Managing data Reporting on data

Assess M&E Requirements

M&E Gap Assessment

Online survey and

recommendations Full Gap Analysis

including review of M&E documents, interview and project plan to address gaps

Implementation

3

1

2

On-going Support

THE IN FOCUS APPROACH

M&E – THEORY OF CHANGE

M&E quality standards for:

1. Defining the Theory of Change

1.1. The organisation has a robust theory of change which informs its planning process, its

implementation and its evaluation

1.2. The theory of change has been developed using a participatory approach – it is informed,

understood and owned by all staff and volunteers and representatives of the target group.

1.3. The theory of change is evidence based.

1.4. The theory of change acknowledges that there are many factors (external & internal) influencing

each change, not just the activities of your organisation.

1.5. The theory of change has been validated by stakeholders (including the local community and

funders)

1.6. The theory of change is regularly reviewed to capture new M&E data as well as data from external

sources and research.

M&E ALIGNMENT


2. Alignment of M&E

2.1. There is a strategic plan that exists based on the theory of change.

2.2. M&E is developed based on the strategic plan – it is fully integrated and aligned within the strategic plan.

2.3. There is clear alignment and harmonisation between what we plan, what we measure and what we report

on.

2.4. The organisation periodically uses findings and lessons learned from M&E to review and update its strategic

plan and theory of change

M&E INDICATORS


3. Measuring Progress with Indicators

3.1. Indicators have been identified to measure each key output and outcome identified by the organisation.

3.2. Indicators are measurable, meaningful and useful for the organisation.

3.3. There is a combination of both quantitative and qualitative indicators.

3.4. All indicators are developed based on a need and are actually used for reporting and/or learning purposes.

3.5. All indicators have an established baseline which is a starting point or a snapshot of where we are now.

3.6. All indicators have an established target which is a specific goal in mind or a snapshot of where we want to be

at a certain time

3.7. Existing and relevant indicators from other players in the same thematic field have been reviewed for their

applicability in the organisation’s M&E system.

M&E INDICATORS


4. Data Collection 4.1. It is clear how the information for each indicator will be collected.

4.2. Tools are manageable for the organisation to administer.

4.3. Data collectors (including staff, volunteers and beneficiaries) are equipped with the relevant skills to collect

data using the tools identified.

4.4. Tools have been tested for effectiveness and relevance

4.5. Relevant demographic data is collected (such as age and gender) so that the data can be interpreted more

meaningfully.

4.6. Where relevant, there is a clear and logical approach to sampling.

4.7. Where feasible and appropriate, the organisation looks to use a mix of methods/ tools for data collection.

4.8. Tools are developed in line with existing good practice and validated approaches.

M&E DATA MANAGEMENT


5. Data Management

5.1. Data is stored and managed effectively and efficiently where possible on a centralized database.

5.2. Data quality: Staff are equipped to ensure that data is managed with accuracy, integrity, reliability

and timeliness.

5.3. Data security: A process is in place to ensure that data is managed securely.

M&E LEARNING AND IMPROVING


6. Learning and Improving

6.1. The organisation regularly monitors progress against baselines and targets to inform continuous improvement

and learning for its programmes and services.

6.2. M&E data and findings are shared and discussed across the organisation to maximise learning.

6.3. Management ensures that plans are able to be translated into action.

6.4. The organisation periodically evaluates programmes according to: relevance, efficiency, effectiveness, impact

and sustainability.

6.5. Analysis of data is conducted as objectively as possible, making note of any potential bias or limitations to the

data.

6.6. Data is analysed and presented in a way that gives a clear picture of how well the organisation is achieving its

goals and where there are specific trends or issues.

6.7. Stakeholders and participants/beneficiaries are involved in the process of learning lessons and identifying

improvements.

M&E DATA MANAGEMENT


7. Accountability and Reporting

7.1. The organisation is able to provide timely, relevant and accurate progress reports or data to funders and

external stakeholders.

7.2. Results and lessons learned are shared with external partners and local communities.

7.3. The organisation uses M&E data effectively to communicate its story compellingly to potential funders and

other relevant stakeholders.

M&E LEARNING AND IMPROVING


8. Overall M&E Coordination

8.1. An overall plan for M&E is in place, which lays out what data collection and reporting will take place, by whom, at

what points in time, and for what purpose.

8.2. The work of collecting and recording data is appropriately spread across staff members and volunteers, so that no

individual is over-burdened.

8.3. One individual within the organisation is responsible for M&E.

8.4. An M&E capacity needs assessment has been conducted including training needs and resource allocation needs.

8.5. A system is in place so that the organisation is able to respond to changes in relation to M&E.

RECOMMENDATIONS AND FINAL THOUGHTS

Utilise your partnerships and networks

Regional/Global partner to manage

relationships

Local delivery NGO vetted by regional/global

partner

International governing bodies

– develop strategies

National Federations – align with and

deliver strategies

CoCreate Consultancy SPORT FOR ALL CONFERENCE, LIMA, 2013€¦ · M&E ALIGNMENT M&E quality...

Documents

Transcript of CoCreate Consultancy SPORT FOR ALL CONFERENCE, LIMA, 2013€¦ · M&E ALIGNMENT M&E quality...