BBVT_Results Based Accountability_Presentation

Results-Based Accountability Vermont Nonprofit ConferenceMarch 20, 2012

Goals for Today(aka How I hope you will better off)

Basic understanding of Results-Based Accountability (RBA)•Why bother?•What is it?•How it can help you plan and manage and maximize•How it can help you communicate•Where to get more information & resources

Experience working with Performance Measures

Be ready to take next steps Benchmarks for a Better VT March 2012 2

Sources & ResourcesResults-Based AccountabilityTM (RBA)

Trying Hard Is Not Good Enough: How to Produce Measurable Improvements for Customers and CommunitiesMark Friedman (2005)

Results and Performance Accountability, Decision-Making, and BudgetingFiscal Policy Studies InstituteSanta Fe, New Mexicowww.resultsaccountability.comwww.raguide.org

Benchmarks for a Better VT March 2012 3

Wait a Second…


Results Accountability Performance Measures

Haven’t we heard about this before?

Everyone wants to:•Improve our programs & maximize impact•Leverage support for our programs

What gets in the way?

• “Too little time”• “Other priorities”• “Too complicated”• “Too many hard decisions”• “We don’t have expertise – or $$ to hire someone”• “They don’t get it”• “Doesn’t feel relevant”

• Overthinking…Trying too hardBenchmarks for a Better VT March 2012 5

Why Bother?Don’t I have enough to worry about already??


Why Bother?


What is Results Accountability?


Shifting our Thinking

Results Accountability is NOT:•a product•a year-end evaluation •software•(just) a reporting tool

Results Accountability IS a :•behavior discipline •framework for planning, doing, and evaluating •process for moving from Talk to Action•cycle for continuous improvement•tool for communicating


Shifting our Thinking

RBA is NOT:•fancy•new •a side track

RBA is :

•simple•the MAIN track

RBA is :•asking an answering common sense questions (that we already think about all the time)


Common Sense Questions• What do we want?• How will we recognize it?• How do things look?• What will it take to get the results we want?• Who are the partners with a role to play?

• What is our role in helping achieve the result?• How good of a job are we doing?


What’s the Big Deal? (1)Common Sense Questions•What do we want?•How will we recognize it?•How do things look now?

•What will it take to get the results we want?•Who are the partners with a role to play?•What is our role in helping achieve the result?•How good of a job are we doing?

Start with ENDS

Then talk about MEANS


What’s the Big Deal? (2) Common Sense Questions•What do we want?•How will we recognize it?•How do things look now?

•What will it take to get the results we want?•Who are the partners with a role to play?

•What is our role in helping achieve the result?•How good of a job are we doing?

COMMUNITY FOCUS

Can look at different levels……with appropriate accountability (more on that later)

PROGRAM FOCUS


What’s the Big Deal? (2 continued)Common Sense Questions•What do we want?•How will we recognize it?•How do things look now?



For each “thing” we do:

•Who do we serve?

•What do we do?

•How good of a job are we doing?

•What works to improve/strengthen?

•What do we propose to do?


What’s the Big Deal? (3) Common Sense Questions•What do we want?•How will we recognize it?•How do things look now?



Common Sense

Clear Language

No Jargon

Tells a Story


A few words about LANGUAGE

• Outcome, Result, Benchmark, Goal, Target

• It doesn’t matter what you call it, as long as people know what you mean.

You can call me anything you like, but just don’t call me late for dinner (or do I mean supper?).


Agreeing on Clear Language• What do we want?

• How will we recognize it?• How do things look now?

• What will it take?• Who are the partners?• What is our role?• How good of a job are we doing?

RESULTA condition of well-being for children, adults, families or communities as a whole

INDICATORHow we measure this condition


EXAMPLES RESULTS for All Peopleof the State, County, City or Neighborhood

A Prosperous Economy

A Clean Environment

Healthy and Safe Communities

Self-Sufficient Families

Children Ready for and Succeeding in School

Parents and Other Adults Healthy and Self-Sufficient

Elders Living with Dignity in Setting of Their Own ChoiceBenchmarks for a Better VT March 2012 18

Clear Language, continued• What do we want?• How will we recognize it?• How do things look now?• What will it take?• Who are the partners?

• What is our role? What will we do to help achieve the result?

• How good of a job are we doing?

STRATEGYAn action taken to improve the RESULT

PERFORMANCE MEASURESHow we know if strategies are workingBenchmarks for a Better VT March 2012 19

Benchmarks for a Better VT March 2012

Strategy ExamplesWhat we do to help achieve the desired result:

•Program/Service Delivery•Collaboration•System Change•Service Integration•Funding•Resource Development/Leveraging

20

Different Levels of Focus

RESULTS & INDICATORSare about the well being of populations

“All people are happy and healthy”


Different Levels of Focus

STRATEGIES & PERFORMANCE MEASURESare about the well being of program participants / service recipients

“Vaccinations for school children”


• Safe Community

• Crime Rate

• Average Police Department response time

• A community without graffiti

• % of Montpelier buildings without graffiti

• People have living wage jobs and income

• % of people in Windsor County with living wage jobs and income

• % of participants in job training who get living wage jobs

A quick test: RESULT, INDICATOR, or PERFORMANCE MEASURE?RESULT

INDICATOR

PERF. MEASURE

RESULT

INDICATOR

RESULT

INDICATOR

PERF. MEASURE

Different Levels of Focus = Different Levels of Accountability

1. Population Level- Focus = well being of population- Collective accountability: Many partners are responsible for achieving the desired result

2. Program (Strategy) Level- Focus = well being of program participants/service recipients- Program accountability: Program is responsible for its performance


Who should be accountable?

• Children in Brattleboro are ready for school• Children in grade 3 at St. Albans City School

learn to read• Job training participants gaining new skills• Employment rate in Bennington County• Employment in Burlington’s New North End

neighborhoods


What’s Appropriate Accountability?

WhoMany Partners

Program

WhatAll children are ready for school

Pre literacy skills in Head Start classroom


NO!!! Not appropriate!

Makes sense

Makes sense

Bringing Two Levels Together

See Handout “Results Accountability: The WHOLE PICTURE”


The Accountability Boundary

_____________________________________________** ACCOUNTABILITY BOUNDARY **

_____________________________________________

•Why is this so important?•OK to cross –but have to be AWARE


How does RBA fit with:

• Strategic planning• Outcomes evaluation• Logic Model• SSML• SWOT• PDCA


Break for LunchBurning questions?

Share them with the facilitators and fax to Amy

See you again at 12:45 sharp

After lunch we will focus in on:Program Accountability & Performance Measurement


REMINDER Different Levels of Focus = Different Levels of Accountability1. Population Level

- Focus = well being of population- Collective accountability: Many partners are responsible for achieving the desired result

2. Program (Strategy) Level- Focus = well being of program participants/service recipients- Program accountability: Program is responsible for its performance


REMINDER What’s the Big Deal?

Common Sense Questions•What do we want?•How will we recognize it?•How do things look now?



For each “thing” we do:

•Who do we serve?

•What do we do?

•How good of a job are we doing?

•What works to improve/strengthen?

•What do we propose to do?


Program AccountabilityPINK Handout

• Who do we serve?• What do we do?• How good of a job are we doing?

– How much do we do?– How well do we do it?– Is anyone better off? (Who? How so?)

• What works to improve/strengthen?• What do we propose to do?


Program Accountability

• Who do we serve?• What do we do?• How good of a job are we doing?

1. How much do we do?2. How well do we do it?3. Is anyone better off? (Who? How so?)

• What works to improve/strengthen?• What do we propose to do?


Performance Measures

Program Performance MeasuresCOMMON EXAMPLES

How much?•# People served•# Hours service offered•# Activities (by type)

How well?•Participant satisfaction•Use of best practice•Qualifications/training•External review

Anyone better off?•Change in skills, knowledge, behavior, circumstance, well being•Possible sources: program records, participant survey, external data


Program Performance MeasuresEXAMPLE

High SchoolHow much?•# students

How well?•Student teacher ratio

Anyone better off?•NECAP scores•Graduation rates•% students entering higher education or employment after graduation



Welfare to Work

How much?•# people trained

How well?•% child care and transportation needs met

Anyone better off?•% employed 6 months after program completion



Raising Children

How much?•# children, # hours spent

How well?•Parent training/education; use of best practice (child satisfaction??)

Anyone better off?•Child health•Child learning achievement•Child happiness & well being


Performance Measurement Challenges

How do we know if anyone is better off?

Required measuresLack of data Excess dataInappropriate dataInappropriate expectations


Choosing Performance Measures (1)

• Answer all 3 questions• Choose what YOU want to know

– What do we need to know to plan? – Evaluate our work? – Make it better?

• Don’t think (too much) about others • This is about the program


Choosing Performance Measures (2)

• Whole staff activity – Central part of planning, service delivery,

budgeting, communications, reporting

• Map out what you will measure & why• Look at what you already have• TRACK OVER TIME• TRY something. Did that work/help? Try

something else.


GROUP ACTIVITYImaginary Program: Cavity Free RichmondWho do we serve?Residents of Richmond, all ages, who do not have access to affordable dental care (e.g., underinsured, uninsured)

What do we do?•Free dental care in clinic (preventive care & treatment)•Presentations in schools about good dental care habits•Outreach to build awareness of services


GROUP ACTIVITYImaginary Program: Cavity Free RichmondYour Task: Identify possible Performance Measures

How could this program measure:1.How much it does?2.How well it does it?3.If anyone is better off?

15 minutes for group workReport out from Newport, Rutland, St. Albans


More Examples



Example: Early Care & Education

1. How much ?# children served; # receiving financial assistance

2. How well?STARS ratingStaff training & qualifications, best practices

3. Is anyone better off?Old measure: developmental statusNew measure: % needs addressed

48


Example: Youth Development

1. How much ?# and characteristics of youth served

2. How well?Licensing, external review, quality ratingStaff training & qualifications, best practices, standards

3. Is anyone better off?Youth feel safe at the program.Youth feel connected to caring adults/role models.Youth feel connected to their community.

49


Example: Residential Treatment

Is anyone better off?How to define this?

Complexity SimplicityWhat do we really need to know?

50

WRAPPING UP PROGRAM ACCOUNTABILITY


POPULATION ACCOUNTABILITYYELLOW Handout

A few key points:•Indicators = data•Don’t get lost in the data. Focus on RESULT.•How to pick good indicators? There’s a chapter for that. Let go of perfection.•When is Population Accountability useful?

– Clarifying accountability– Showing relevance– Bringing together partners.


GROUP ACTIVITYPopulation Accountability ExerciseImaginary Problem: •Community upset about fires in Winooski•Fire Department budget questioned•Plan for community meeting


GROUP ACTIVITYPopulation Accountability Exercise

What we want? RESULTAll people in Winooski are safe from fire

Your task today:Who are the partners with a role to play?10 minutesReport out from Montpelier & Bennington (if time)


Population Accountability: Examples


Population Accountability : Burlington Truancy Project

Started with a Problem: Burlington had highest drop out rate in VT. And it was going up.

What will it take to turn this around?Who needs to be at the table?


57

Burlington Truancy Project Shared accountability & partnership• Desired Result All Burlington students graduate from

high school • Indicator High school drop out rate• Baseline Not good! Highest dropout rate in VT

Dropout Rate for Burlington School District Students

0.0%1.0%2.0%3.0%4.0%5.0%6.0%7.0%8.0%9.0%

10.0%

1995

1996

1997

1998

1999

2000

2001

2002

2003

2004

2005

2006

2007

2008



Burlington Truancy project (cont.)

• What works? Address truancy to prevent drop out• Partners Non-profit service providers, school district,

police department, school administrators, juvenile court judges, attorneys, child

protection agencies, and United Way of Chittenden County

• Strategies A coordinated response to truancy:

- School District: new policies & procedures

- Juvenile Court & SA : monthly court time

- Non profits & state agencies: services that remove barriers


The Truancy Project: Key Indicator

Dropout Rate for Burlington School District Students

0.0%1.0%2.0%3.0%4.0%5.0%6.0%7.0%8.0%9.0%

10.0%

1995

1996

1997

1998

1999

2000

2001

2002

2003

2004

2005

2006

2007

2008

Example: Truancy Task Force & the EVEN BIGGER PictureWhat do we want?

Youth succeed in schoolHow will we recognize it?

School completion; CompetenciesWhat does it take?

1. Good educational opportunities2. Youth attend school3. Youth ready to learn



Example: Investing in Early Childhood (How UWCC fits in bigger picture)

ResultAll children enter school ready to learn

IndicatorKindergarten readiness

UWCC Strategies (the roles we play)1. Fund programs that provide parent support & education2. Provide access to high quality early education3. Volunteer programs supporting early literacy

RESULT

STRATEGY1

STRATEGY 2

OTHER INFLUENCES

61

RBA = Process

What about Product?


Reminder: Goals for Today aka How I hope you will better off

Brief Intro Results-Based Accountability (RBA)•Why bother?•What is it?•How it can help you plan•How can it help you manage…and maximize…and communication•Where to get more information & resources

65

Did we cover these?

If nothing else…

Key Features of RBA

3 Performance Measure questions2 Levels of focus (& accountability)1 Simple framework to help focus


Remember:• Anyone can do this, at any time. Period.

• This is a whole staff/whole organization process. All the time.

• Don’t worry about perfection. Sketch it (don’t etch it).

• Your program is not responsible for population level results.

• You have a role to play in population level results. Know what that is.

• Your program is responsible for helping its participants be better off. You have to know if and how.

• Keep RBA questions in your pocket, bring them with you. Say, “How much? How well? Anyone better off?” Say, “Who is responsible for that? For this?”

67

Feeling ready to give it a try?

• Buy the book – “There’s a chapter for that!”

• Check out the website

• Apply for the Performance Institute

• Start asking and answering the questions – Staff meeting– Board meeting– Coalition meeting

• JUST DO IT. Benchmarks for a Better VT March 2012 68

Any questions?

[email protected]://bbvt.marlboro.edu/


BBVT_Results Based Accountability_Presentation

Technology

Transcript of BBVT_Results Based Accountability_Presentation