The Local Evaluation:A Brief Overview

Becca Sanders, M.S.W., Ph.D.Program Evaluator

beccaanddan@gorge.net

Training Objectives

The Local Evaluation

What is “it”?

Why does “it” matter?

Local Evaluation GoalsIn a Nutshell: Help figure out if good “stuff” is happening in our

System of Care.

If not, why not?If not, what to do about it?

Tis not a thing!There is no it!No such nutshell! Does not fit!

Local Evaluation Goals

In an Expanded Nutshell:

Is youth functioning increasing?• Family, Community, School

Are services?• Timely, Effective, Helping the “right” folks

Is system operating in accordance with a set of (SOC) principles?• Coordinated• Strength Based• User Friendly• Simplified• Responsive• Families as Equal Partners• Cultural and Linguistic Competence• Continuous Quality Improvement

Local Evaluation Efforts

The Local Evaluation Drives a 3-Car Train:

Train Car #1: The Local Data Reporting Car

Train Car #2: The Local Data Collection/ Analysis Car

Train Car #3: The Local Data Technical Assistance Car

Data Reporting: What does it look like now?

Stakeholder identifies reporting content

Evaluators run data/ develop a draft report

Draft report goes out to

15+ stakeholders

for review

Evaluators receive

feedback/ make

suggested changes

•Data disseminated electronically

•75+ stakeholders

• On a listserv

Currently On Board:•Monthly Reports •Posted on the KWRAPS Evaluation Website

Evaluator as Waitress: get the order right or we’ll send it back…

Data Reporting: What does it look like in the future?

Data disseminated a number of different

ways

Use data to Modify/ Improve/ $Maintain$ System of Care:

Quality Improvement to Achieve Better Outcomes

DATA-BASED DIALOGUE

More CateringMore Requests for DataDifferent Reporting Formats

•Annual Report•Individual Family Reports•Clinical Reports•Regular QI Process

More InterpretationIdentify Data Needs

Data Reporting Features

• Catered– Right off an open menu

• Stakeholder Driven– Topics generated in the field

• Accessible– Jargon free – Easy to interpret

• Relevant– To “the Big Picture” (sans nutshell)– To goals of the Local Evaluation

• In Accordance with System of Care Principles

Open Door Policy

Local Data Collection/ Analysis What does it Look Like?

Currently On Board:• Positive Behavior and Literacy Support• Child and Adolescent Functional Assessment ScaleBought the Ticket:• Wraparound Integrity Tool• Stigma MeasuresStill Planning the Trip:• Youth and Family as Equal Partners• Cultural and Linguistic Competence Assessment

Local Data Collection/ AnalysisFeatures

• Locally Sourced– Part of local efforts; multi-system sourcing

• Locally Housed– Transient piles– Various networks– Various agencies– Not yet collected

• Data piece will live on after grant ends– Central role in creating culture of continuous

quality improvement

Data-Related Technical Assistance

Partners at Various Stages: (On Board, Bought the Ticket, Planning the Trip)

1) What data do we have?2) How can evaluation help your efforts?

– Reporting– Measurement

3) How to analyze/ interpret data?

Who Drives the Train?Why is it Important?

The Evaluation Workgroup– Integrative Active Team subcommittee– Broad array of stakeholders (15+)– Meet quarterly– Largely electronic correspondence

• Describing • Improving • Sustaining

Ways to Get On BoardEnhance the Data Dialogue

• Sign up for the listserv– Notification of monthly report posting on web

• Read Reports, Ask Questions• Request a Topic/ Report• Request a Presentation• Request Technical Assistance

Open Door Policy!I’ll come to you!

http://www.wmich.edu/sociology/kzoowraps.html