Building Skills to Monitor & Evaluate Performance & Outcomes

Building Skills to Monitor and Evaluate

Performance and Outcomes

Joy A. Livingston, PhDDonna Reback, MSW, LICSW

Flint Springs Associates

www.flintspringsassociates.com

April 14, 2011

Workshop GoalsUnderstand steps needed to monitor &

evaluate performance & outcomes in order to market accomplishments and gain funding.

Learn to develop and use a logic model

Recognize the investment of time and resources required to monitor and evaluate performance and outcomes

4 Steps to Monitor and EvaluateDevelop a Logic Model

Gather Information

Analyze Information

Use Information

Developing a Logic ModelWhat is a logic model?

Why develop a logic modelProvides structure for conversations and

decisions about outcomes and performance measures

Guides development and implementation of monitoring and evaluation activities

Examines connection between what we do and results

Logic Model TemplateInputs/Resources

Activities Performance Measures

Outcome Indicators

Outcomes

What (resources, supports, commitments) do we need to carry out the initiative and achieve desired outcomes?

What do we need to do to achieve the desired outcomes?

How will we know we did what was needed?

How will we know we achieved the expected result?

What is the expected result of what we did?

Planned work Intended Results

Logic Model- Step 1:Identify and Define Outcomes

Agree on outcomesInterview individual staff and board

members

Identify areas of consensus and difference

Hold meeting/retreat to build uniform agreement on outcomes organization is willing to be held accountable for achieving

Why Track Outcomes?Make the case that program is effectiveFocus on activities toward

organization’s goals (don’t waste resources on activities not linked to outcomes)

Help set realistic and realizable goals for organization to be held accountable

Allow everyone in organization to be part of the conversation about outcomes

Inputs Resource

s


Outcome Indicators

Outcomes

Clients successfully plan, start and/or grow a business

Clients positively contribute to economy

Clients achieve financial stability and economic self-sufficiency

Logic Model – Step 2: Identify Outcome IndicatorsIdentify concrete, observable measures

Community-wide impactsIndividual client outcomes

Define measurable benchmarksBenchmarks are numerical points

which define success within a specified timeframe

InputsResources


Outcome Indicators

Outcomes

Clients complete written business plan

Clients starts new business or expand existing business – business continues

Client businesses are financially viable (profitable)


Client businesses contribute jobs

Client businesses generate tax revenues


Clients use budget Clients improve credit

score Clients have stable

employment Clients reduce use of

public assistance


Outcome Indicators BenchmarksClients complete written business plan

80% of clients complete business plan within 1 year of intake

Clients businesses are financially viable

50% increase in client businesses that are profitable (i.e., revenue less cost > 0) by 2012

Logic Model – Step 3: Determine Activities Leading to OutcomesExamine current activities and services

Consider if and how each would lead to desired outcomesBe willing to eliminate those activities

that aren’t linked to achieving outcomesConsider incorporating new activities

InputsResources


Outcome Indicators

Outcomes

Recruit participants

Provide 1-to-1 TA & counseling

Provide training courses

Client completes written business plan

Client starts new business or expands existing business – business continues

Client business is financially viable (profitable)





Provide 1-to-1 financial and social service counseling

Referrals within & outside agency




public assistance


Logic Model – Step 4: Identify Performance MeasuresConsider how you will measure the

implementation of agreed upon activities and servicesDetermine dosage (measures of amount,

extent of services and/or activities performed)Determine measures of quality (client

satisfaction, fidelity to evidence-based practice)

Identify benchmarks – numerical measures of successful implementation

InputsResources


Outcome Indicators

Outcomes




Number of clients recruited/retained

Number of clients receiving TA & types of TA provided

Number & types of courses offered; number participants; number meeting training objectives

Participant satisfaction with services










Number of clients receiving financial counseling

Number and type of referrals to other social services




public assistance


Performance Measures BenchmarksNumber of clients recruited/retained

During 2011, 100 new clients will be recruited; 75% will remain in program

Participants satisfied with services

85% of clients report satisfaction with program at 24 month follow-up

Logic Model – Step 5: Identify Inputs and Resources Needed

What needs to be in place to support implementation of activitiesFundingPoliciesSupport of stakeholdersSkilled, experienced staff

InputsResources


Outcome Indicators

Outcomes

Established MBDP and VWBC programs

Trained and experienced staff

Economic conditions increasing interest in small business




Number of clients recruited/retained

Number of clients receiving TA & types of TA provided

Number & types of courses offered; number participants; number meeting training objectives

Participant satisfaction with services








Established MBDP, VWBC and IDA programs

Trained and experienced staff

Federal and state funds to support programs



Number of clients receiving financial counseling

Number and type of referrals to other social services




public assistance


ExerciseIndividual organization workSmall group sharingReport out and Q&A

After Developing the Logic Model

Gather InformationIdentify what information needed to

measure performance and outcomes

Does the information already exist? If yes, where is it and what form is it in?If no, how will it be gathered?

Who will collect?

How will it be collected?

When will it be collected?

Analyze InformationRefer to logic model to structure

questions for analysis

Conduct systematic and objective analysis of quantitative and qualitative data

Review findings with relevant staff/board/stakeholder to determine:Did we do what we said we were going to

do (performance)?How well did we perform?Did we achieve desired outcomes?

Use Information

Internal - Continuous Quality ImprovementImprove, change activities and servicesInform strategic planning

External – Community Support and FundingMarket, tell our storyBuild support for our programGain funding

Building Skills to Monitor & Evaluate Performance & Outcomes

Business

Transcript of Building Skills to Monitor & Evaluate Performance & Outcomes