SUSTAINABLE LIVELIHOODS - Meetupfiles.meetup.com/2038441/Eval Cafe SL Presentation... · Getting...

1 |

SUSTAINABLE LIVELIHOODS AN EVIDENCE-BASED APPROACH TO IMPROVING PRACTICE AND OUTCOMES

Evaluation Café Presentation

September 23, 2015

©Resources for Results and

2 |

Today’s Evaluation Café Session…

Agenda (35 minutes)

Part 1 – Why a Theory of Change?

Part 2 – Sustainable Livelihoods

Part 3 – Our Action Research Project

(Mini Case)

Part 4 – Results and Findings

Part 5 - Discussion

Purpose

To explore the merits of a theory of

change in promoting effective

program practice and improving

outcomes

Resources for Results &

3 |

Why a theory of change?

How many of you facilitate the development of Theories of Change in your evaluation work?


4 |

It can be:

A ‘road map’ for your project

A drawing that communicates the logical connections between your actions and desired outcomes

A statement of the chain of inter-connected outcomes and key milestones of progress

What is a Theory of change?


5 |

Evaluation and the Program Planning Cycle

Program design

Implementation planning

Design of evaluation and

monitoring

Ongoing results documentation

Program implementation

Reflection and

learning

Results assessment

Redesign


6 |

Envisioning and planning for outcomes...

Supports a developmental

approach to evaluation

Supports the planning of

complex, multi-faceted initiatives

Engages partners and supports a

collaborative, learning-based

approach

Enhances a results-orientation in

your planning

Promotes learning about

effective practices

Improves your results

Advances the policy agenda


7 |

The Sustainable Livelihoods Framework

A quick overview of our grounding Theory of Change


8 |

What is Sustainable Livelihoods?

The Sustainable Livelihoods approach offers a framework and language for understanding the assets of people who are socially excluded and living in poverty.


9 |

Think “livelihoods”

A “livelihood”

includes…

Livelihood thinking

goes beyond the idea

of “having a job”

Being a citizen who is fully included in society

Being able to work and earn income, and being a part of the economy

Empowerment and independence – the ability to make decisions and have a say

Long-term economic and social security


10 |

What does “sustainable” mean?

Sustainability is defined broadly as…

Resilience The ability to cope with and recover from life

shocks and stresses Thee ability to be more resourceful and

“entrepreneurial” in navigating the job market Future mode The ability to build assets starting with a small

resource base Social inclusion Fairness and the opportunity to be a part of

society and the economy


11 | Resources for Results &

12 |

Client Engagement Work


14 |

Sustainable Livelihoods - Features of the Approach

Client-centred – focuses on clients and

“where they are at”

Holistic – deals with all areas of clients’ lives,

not just one

Positive – looks at what assets clients have,

and proposes strategies to build on their

strengths

Results-oriented – starts with what clients

want and promotes their priorities

Problem solving – identifies and seeks to

deal with systems and policies that hold

clients back

15 |

Our Action Learning Project

A theory driven approach to program innovation

It is critical to build a foundation for participation before clients can move on to more substantial employment and income security gains. We now have a clear understanding of what must be in place for clients to participate in, and benefit from program interventions. LEF has now developed client-centred practices that intentionally build constructive, asset-building behaviours and resilience in LEF clients .

16 |

Introduction - LEF Sustainable Livelihoods Project

Strategy 1: Co-creating effective client

engagement practices

Embedding client-focused Sustainable Livelihoods

and motivational interviewing practice into LEF

client engagement processes through a

participatory, action learning approach

Strategy 2: Outcomes evaluation

Building an evidence-based strategy for learning

and continuous improvement of emerging CEW

practices

Project Purpose : • To enhance client engagement and retention in LEF’s programs and services • To advance LEF’s efforts to restructure and de-silo its organizational structure and program delivery. • To deepen the livelihood outcomes for clients.

Collaborative Learning to integrate Sustainable

Livelihood

Evidence-based approach


17 |

Early Stage Pathways to Livelihoods A preliminary theory of change developed through intensive workshops with CEWs

Welcome

Personal welcome and orientation to

organization's services

Personal, non-judgmental

registration and intake

Immediate internal and external referral

"Triage" and problem

identification

Problem solving to stabilize people (e.g.

access to income boosting

entitlements)

Put in place supports to participation (e.g.

child minding, transit)

Decision re: "fit" for problem solving and

coaching

Pathway 1: Develop the conditions for

participation

Pathway 2: Develop the capacity for

participation

Asset mapping and assessment

Promoting action planning practice

In house referral to further training and

support

Ongoing support for decisionmaking and

asset building

Hold clients accountable for

action and progress

Pathway 3: Coaching for

behaviour change/asset

building

"Anchor" relationship

continues parallel to program

participation

Promote self-direction and self-advocacy (e.g. goal

setting and decisionmaking)

Manage and sequence asset development

(including money)

Support transition to engagement at

organization


18 |

Client Engagement Worker: Creating a culture and process that engages LEF clients

Design changes to promote client

engagement ...

Streamlined client flow

Client-focused approach

A shift to holistic coaching

Program design that promotes

client accessibility (hours, location)

Evidence-based learning approach

The Concept

The Client Engagement Worker role has

been established at LEF to offer a single

entry point and personalized, ongoing

support for new clients who come to LEF.

Clients are engaged through a one-to-

one interaction that offers holistic,

client-centred coaching and supports,

breaking down silos which can create

barriers to service, and addressing the

immediate needs and priorities of clients.


19 |

Welcome

•Day 1:

•Orientation

•Problem Solving

•Triage/needs assessment

•Action planning

•Client Service Plan

Problem Solving

•Early Meetings:

•Building trust and a working partnership

•Continued problem solving as required

• Form filling as required

•Ongoing action planning

Asset Mapping

•Within 2-3 weeks:

•Early program look at assets

• Facilitated self-assessment

•Mapping strengths

• Setting priorities and planning for building assets

Ongoing Coaching

•Ongoing as required:

• Support for self-direction & independence

•Checking in and holding clients accountable

• goal setting and decision-making

• self-advocacy

Ongoing Mapping

•At 3 and 6 months:

• Facilitated client self-assessment

•Assessment of progress and outcomes

Building the conditions and capacity for success in settlement and employment

LEF Client Engagement Work What happens at each stage of client engagement?


20 |

The Next Generation of SL Tools

Based on the original CWF Sustainable Livelihood

tools, Mary Ferguson and Janet Murray worked

with LEF managers and staff to design, refine and

pilot the following tools :

Welcome Form – Client intake and consent form

What can we do for you? – An orientation to

LEF’s program offerings

My Plan 1 – A form to support client-led planning

and action

My Plan 2 – An asset mapping tool that supports

client self-assessment and longer-term planning

and asset building

LEF has partnered with Resources for Results, Eko Nomos and the Ontario Institute for Studies in Education (OISE) to develop and pilot a new level of Sustainable Livelihood tools and practices.


21 |

Motivational Interviewing integrated well into coaching and asset mapping practice

The Spirit of MI

MI Is a new client-centred practice that is

taking hold in the social services sector.

Grounded in empowering psychological

counselling, MI offers constructive and

empowering language and practices that

fit well with SL’s holistic, asset-based

approach.

COLLABORATION – Relationship as

collaboration or partnership

EVOCATION – Elicit vs. Impart

AUTONOMY – Ability to choose

COMPASSION - Empathy

GUIDING – vs. following or leading

(From: What is Motivational Interviewing? (PPT), Goldstein and Anthony,

OISE, 2015) Resources for Results &

22 |

LEF Client Flow – Sequencing Client Engagement priorities


23 |

Livelihood

Stage

Building the Conditions for

Participation

Building the Capacity for

Participation

Engaging in the

economy

Steps

1 Welcome 2 Problem

Solving

3 Asset

Mapping

4 Ongoing

Coaching

5 Follow-up –Asset

Mapping

Outcomes

Access to services

Action planning becomes a

client practice

Improved program enrollment

Increased client trust and

engagement

Ongoing employment and

settlement problems solved

Improved program retention

Realistic self-assessment

Strategy for building assets

Planning skills

Improved connections

Enhances skills to get and keep a

job

Enhanced decision making

Stronger self-organizing and

action

Ability to self-advocate

Increased independence

Improved stability

Enhanced hope and

wellbeing

Ability to think

about and plan for

the future

Employment

Onward education

Improved health

Coping Strategies Asset Building Strategies Future mode

Expected Outcomes of Client Engagement


26 |

Results and findings of our developmental evaluation

A Client outcomes survey validates our theory of change


27 |


28 |

Table 2 - Survey Likert scales – Aggregate Asset Area Scores and Sub-sets of Indicators/scores Mean N

Getting Connected 1.15 133

I am less alone than when I first came to the Learning Enrichment Foundation 1.00 130

I have better access to the supports and services I need 1.18 131

I have a strong connection to at least one staff person here 1.26 131

Solving Problems 1.04 133

I am more likely to seek help and advice for problems I face 1.08 132

I can find ways of solving the problems I face 1.03 131

I try out new ideas and ways of doing things 1.03 129

I am more likely to let people know what I think and what I need 1.06 132

Basic Needs .79 128

My housing is better .42 102

I have better access to information and resources 1.02 126

I have better access to computers and the internet .80 122

Building skills to find and keep a job .97 132

I have a better understanding of my strengths and abilities .94 130

I have a stronger set of skills to help me find and keep a job .91 131

I have better connections to help me find work 1.03 131

I have stronger English language skills 1.01 106

Action Planning .88 130

I am clear about what I want, and where I am going in my life 1.09 129

I often use action plans and lists to keep me on track .69 127

I am more able to get things done on my own .86 127

Money .32 113

I have a better understanding of my finances and how to deal with my money .63 106

My income has increased since I first came here .12 101

I am more financially stable now .15 103

Standing up for myself .80 128

I know how to deal with government to get what I need .38 125

I can stand up for myself 1.03 124

I have a better idea of what my rights are 1.08 118

Improving Health .81 130

I am less stressed than when I first came here .82 125

I am more healthy mentally since I came here - I am more positive and less 'down' .83 128

Building Identity 1.06 127

I have a clearer idea of who I am and what I can do 1.02 125

I can keep going no matter what happens 1.02 124

I am more hopeful about the future 1.18 125

I am more confident 1.11 127

I have more control over my life 1.00 121

Growing wellbeing .90 126

My life is more stable .71 123

I am better able to think about my future 1.02 125


29 |

Getting connected

1

1.18

1.26

I am less alone than when I first came to the Learning Enrichment Foundation

I have better access to the supports and services I need

I have a strong connection to at least one staff person here

Getting Connected (composite score: 1.15, N=133)

Note: These Likert scores are based on a scale from -2 to +2 where -2 means "Strongly disagree"; 0 means "Neutral" and +2 means "Strongly agree“.


30 |

Skills and Employability

0.94

0.91

1.03

1.01

I have a better understanding of my strengths and abilities

I have a stronger set of skills to help me find and keep a job

I have better connections to help me find work

I have stronger English language skills

Building skills to find and keep a job (composite score: 0.97, N=132)

1.09

0.69

0.86

I am clear about what I want, and where I am going in my life

I often use action plans and lists to keep me on track

I am more able to get things done on my own

Action Planning (composite score: 0.88, N=130)

1.08

1.03

1.03

1.06

I am more likely to seek help and advice for problems I face

I can find ways of solving the problems I face

I try out new ideas and ways of doing things

I am more likely to let people know what I think and what I need

Solving Problems (composite score: 1.04, N=133)

Note: These Likert scores are based on a scale from -2 to +2 where -2 means "Strongly disagree"; 0 means "Neutral" and +2 means "Strongly agree“.


31 |


32 |

Scale,

n = completed all items in the scale

Number of

items in

scale

Internal consistency

(Cronbach's Alpha)

Composite score -2

to +2 (all

participants)

Basic Needs, n=100 3 0.864 0.79±0.81

Growing wellbeing, n=121 3 0.830 0.90±0.68

Getting Connected, n=127 3 0.746 1.15±0.74

Solving Problems, n=128 4 0.861 1.04±0.67

Action Planning, n=124 3 0.809 0.88±0.69

Building skills to find and keep a job, n=102 4 0.842 0.97±0.78

Money, n=94 3 0.898 0.32±1.01

Building Identity, n=118 5 0.905 1.06±0.66

Standing up for myself, n=112 3 0.749 0.80±0.79

Improving Health, n=123 2 0.816 0.81±0.89

Table 1: Chronbach’s Alpha - Analysis of scales Note: composite score reported as mean ± SD. All scales have high degree of internal consistency (value of Cronbach's Alpha about 0.7 or above).


34 |

CEW Asset Mapping Pilot Asset Growth – pre/post assets (N=23)

ASSET AREA Pre-Training Post-Training Pre-Post Difference

Connections 6.42 7.63 1.21 (N=24)

Identity 7.23 8.26 0.74 (N=23)

Skills and Employability 6.79 7.46 0.67 (N=24)

Money 5.71 5.92 0.21 (N=24)

Foundation Building 6.58 7.46 0.33 (N=23)

Health 7.96 7.89 -0.41 (N=23)

Pre-Post Difference - change between Asset Maps


35 |

CEW Asset Mapping Pilot Livelihood Progress Case Study (N=8)

Sustainable Livelihood

Stage 5

Consolidation

Stage 4

Transition

Stage 3

Foundation Building

Stage 2

Engaging

Stage 1

Stabilizing


36 |

Promising Practices - CEW coaching and asset mapping support intentional asset building and livelihoods progress When combined with MI, Sustainable Livelihoods offers an

effective, client-centred practice that promotes outcomes.

We have confirmed that the following practices build a

foundation for client empowerment and independence:

1. Investment in long-term, face to face, “anchor” relationships of trust that focus on clients’ priorities – not funders’

2. Holistic, asset-based self-assessment using asset mapping resulting in livelihood development strategies.

3. Emphasis on the development of planning and decision-making skills through the ongoing re-enforcement of action planning, priority setting and problem solving skills.

4. Non-judgmental, motivational coaching that evokes clients’ own solutions to problems and desire for change.

5. Following up to hold clients accountable for their progress.


37 |

Finding 1 – Our theory of change has guided the development of evidence-based effective practices in client engagement

Finding 3 – There is strong potential for a Collective Impact approach and shared measures

Finding 2 – We now have a model of change. Our developmental evaluation has provided evidence validating our theory. Confirming that one-to-one client relationships build a foundation for participation and asset building


38 |

Thank You

Evaluation/ Report Credits: Janet M. Murray, Resources for Results Tel: 647-233-7711 Mary Ferguson, Eko Nomos Tel: 519-599-2410 Robyn Hoogendam, LEF Statistician: Anton Svendrovski Graphics: Nicholas Smith Graphic Production: Sara Gaudon

For SL resources go to: www.ekonomos.com

This report is the result of the intensive collaborative effort and persistence of a large team of people. We all worked hard on this project ! The authors wish to thank everyone who contributed their vision, resources and dedication to this project: Peter Frampton SL Design Team: Elisa Martinez-Reyes , LEF Robyn Hoogendam, LEF Dr. Immaculate Antony, OISE Dr. Abby Goldstein, OISE Janet M. Murray, Resources for Results Mary Ferguson, Eko Nomos Emmett Ferguson, Eko Nomos LEF Conductors: Roopa Dittakavi Cheryl Partridge Ray Stephens Fernando Rivas LEF CEW Staff

SUSTAINABLE LIVELIHOODS - Meetupfiles.meetup.com/2038441/Eval Cafe SL Presentation... · Getting...

Documents

Transcript of SUSTAINABLE LIVELIHOODS - Meetupfiles.meetup.com/2038441/Eval Cafe SL Presentation... · Getting...