Vibrant Communities Canada: Measuring Impact
Donna Jean Forster-Gill Tamarack-An Institute for Community Engagement
Why Vibrant Communities?
The Impetus
The Complex Nature of Poverty
“Poverty is a complex issue. There is no single cause and no one solution. Its successful reduction, and ideally its eradication, require a set of linked interventions undertaken by all orders of government working in collaboration with communities.”
Poverty Policy Sherri Torjman, Caledon Institute of Social Policy October 2008
What is Vibrant Communities?
A Pan-Canadian initiative exploring comprehensive, multi-sector approaches to poverty reduction
Launched in 2002 by three national partners
• Tamarack: An Institute for Community Engagement • The Caledon Institute of Social Policy • The J.W. McConnell Family Foundation
Vibrant Communities An experiment designed to test a specific
way to address the complex realities of poverty through local level action.
Theory of Change: Guided by 5 principles & assisted by extra supports provided by national sponsors –
local organizations and leaders could revitalize poverty reduction efforts in their
communities and generate significantly improved outcomes.
We believed that using a comprehensive, multi-sector approach communities can …
• Raise the local and national profile of poverty
• Build a constituency for change
• Encourage collaborative ways of working
• Begin to shift the systems underlying poverty
• Generate substantial changes for a large number of people living in poverty.
Who are the Vibrant Communities?
Part One – Exploring Principles
13 Vibrant Communities
+ Niagara Region
How does Vibrant Communities work?
What are the 5 components of Vibrant Communities?
Multi-Sector Collaboration
Business • Exper(se, credibility and
voice, connec(ons, funding and other resources, leadership
Nonprofit Organiza2ons • Exper(se, experience on
the ground, service delivery, ability to ramp up change efforts
Government • Exper(se, connec(ons to
elected officials, funding and other resources, policy change, leadership
Ci2zens with Lived Experience • Exper(se about the issues,
prac(cal and relevant solu(on, leadership, connec(ons to other ci(zens
Comprehensive Thinking & Action
address the interrelated root causes of poverty rather than its various symptoms
Create Community Knowledge
Community Asset Building
building on community strengths rather than focusing on deficits
Develop a Community Aspiration
Community Learning and Change
embracing a long-term process of learning and change rather than simply undertaking a series
of specific interventions
Create a Framework for Change
Poverty Reduction
a focus on reducing poverty as opposed to alleviating the hardships of living in poverty
Move towards Systemic Change
Impact of Vibrant Communities?
Collective Impact • Common agenda • Shared measurement
systems • Mutually reinforcing
activities • Continuous
communication • Backbone support
organization
– John Kania and Mark Kramer, Winter 2011
Measuring the impact of Vibrant Communities –
Shared Measurement
Original Approach • Logic Models, Outcomes and Outcome
Tracking. (Traditional Evaluation methods) • Used for first 2-3 years of project
SPEED BUMP: • Approach did not fit with what was
happening on the local level • Shifted to developmental evaluation
approach
Developmental Evaluation in Vibrant Communities Stream One –each local community to articulate the ‘theory of change’ guiding its work and to reflect annually on how those ideas were playing out in practice. Stream Two –communities to prepare brief stories describing their specific poverty reduction strategies: the challenge addressed, the strategy employed and the outcomes anticipated. These stories helped guide their outcome tracking efforts. Stream Three –communities complete semi-annual statistical reports focussed on two main targets: number of partners participating in their work and number of low-income households benefitting from their efforts. It also asked for an annual narrative report elaborating on the overall development of each community’s initiative, including its efforts to build community capacity and impact wider systems that contribute to poverty.
What was captured: • Stories and Lessons Learned
• Expected and unexpected outcomes
• Most significant change
• Statistical Reporting and Narrative reporting focused on 5 types of outcomes initially.
• Used sustainable livelihoods model with 5 outcomes areas, 13 indicators and many sub-indicators.
• After 5 years, this was revised to add outcomes around community capacity building and policy and systems change
Indicators of Community Change
• Changes in public policy • Changes in service and support systems • Changes in material resources • Changes in community-level assets
Policy and Systems Change
• Convening capacity • Multisectoral leadership • Collaboration • Community awareness
Community Capacity Building
• Personal assets • Physical assets • Social assets • Human assets • Financial assets
Individual and Household
Assets
Social Assets
Human Assets
Financial Assets
Inner resources
• Self-awareness • Self-esteem and self-confidence • Hope and motivation
Basic material goods & services
• Emergency supports • Food, Housing • Transportation • Dependent care
Relationships and Networks
• Civic participation • Support networks
Income, Savings
• Employment income • Non-employment
income • Savings and financial
assets • Reduced debt/costs
Skills, knowledge, education & health
• Health, Life skills • Financial literacy • Education • Employment Skills
Why Developmental Evaluation Worked for VC At the local level: required reflecting on the theory of change and upgrading it as required to better achieve desired outcomes, rapid response to a changing environment, and ability to capture the emerging insights and questions of participants. At the national level: it is about mining the on-the-ground experience of communities for patterns and themes that helped us understand the value of this approach to reducing poverty.
Challenges in measuring the impact of the
Vibrant Communities Initiative
Overall Challenges: • The sheer scope of the work: Lots of time and energy
required
• ‘attribution issues’: whose efforts are responsible for outcomes achieved or roles that different partners play.
• The guiding ideas are often left implicit making it hard to assess their validity.
• Evaluation processes need to be highly flexible due to emergent nature of the process.
• Outcomes are integral, but are often difficult to define and measure.
• The long-term focus must be reconciled with the need to track progress in the near- and mid-term.
• appropriate adjustments must be made along the way
Most significant challenge: The evaluation needed to accommodate substantial differences across the sites while continuing to meet collaborative objectives. National Level: • Managing the volume of data • Getting materials submitted on time • Lack of evaluation advisory group to guide the
work • More collaboration needed between evaluators
and local level • Takes lots of time and concentrated effort
Local Level Challenges: • Keeping up with evaluation demands
• Balance between hard numbers and story
• National Level questions did not always work well at the community level
• Staff turnover, inadequate evaluation training for the communities
• Often off the side of the desk rather than designated staff person
• Takes lots of time and concentrated effort
• Not single organization but collaborations
What have we learned?
How much more information do we need to know that a hungry child will not do well in school? Stop admiring the problem and get on with the work.
Mark Chamberlain CEO, Trivaris
Evaluating a comprehensive, multi-sector approach to poverty reduction is …
1. Hard work, messy and time consuming
2. Fluid, flexible and requires adapting to constantly changing conditions
3. Requires outside eyes to hold the evaluation pieces.
4. Orgs can benefit from training in evaluation
5. Everyone has to develop the evaluation framework
Common Success Factors • Influential and credible convener(s)
• Cross-sector, connected leadership table • Challenging community aspiration
• Clearly articulated purpose and approach
• High degree of resident mobilization
• Research which informs the work and captures shared impact
Reporting to the Community – Continuous
Communication
Vibrant Communities (2002 - 2010) Evaluation Report
Reflecting on Vibrant Communities: 2002-2006
Understanding the Potential & Practice of Comprehensive, Multi-sector Efforts to Reduce Poverty - The Preliminary Experiences of the Vibrant Communities Trail Builders
In From the Field - Exploring the First Poverty Reduction Strategies Undertaken by Trail Builders in the Vibrant Communities Initiative
To learn about the background of collecting the VC by the Numbers reports: http://tamarackcommunity.ca/downloads/vc/VC_By_the_Numbers_FAQs_032511.pdf
Where are we headed?
The Landscape has Changed…since 13 cities began to experiment…
• Municipally: 84 collaborative poverty reduction roundtables have
connected to Vibrant Communities • Provincially: 11 provinces and territories have or are developing
poverty reduction strategies • Federally: A new all-party Roundtable has been formed to focus on
poverty, the Government of Canada – HUMA committee, Senate Roundtable on Cities and Federation of Canadian Municipalities have identified poverty as a critical issue
Our Aspiration:
Imagine…100 cities reducing poverty
TOGETHER
Top Related