“Next Generation” Technologies for the “Next Generation” Library User
Next Generation Impact Measurement | July 19, 2016
-
Upload
united-way-of-the-national-capital-area -
Category
Government & Nonprofit
-
view
170 -
download
0
Transcript of Next Generation Impact Measurement | July 19, 2016
![Page 1: Next Generation Impact Measurement | July 19, 2016](https://reader035.fdocuments.us/reader035/viewer/2022070514/587d62901a28ab32318b4c13/html5/thumbnails/1.jpg)
Next Generation Impact Measurement
Measuring What Matters to Nonprofits
![Page 2: Next Generation Impact Measurement | July 19, 2016](https://reader035.fdocuments.us/reader035/viewer/2022070514/587d62901a28ab32318b4c13/html5/thumbnails/2.jpg)
Why Do We NOT Know If We Are Making a Difference?
![Page 3: Next Generation Impact Measurement | July 19, 2016](https://reader035.fdocuments.us/reader035/viewer/2022070514/587d62901a28ab32318b4c13/html5/thumbnails/3.jpg)
What are Nonprofits Being Asked to Measure?
10% Have Some Output Data
Disconnected datasets<500 cases per dataset
No ”true” outcomes
1% have a Little Bit of Outcome Data
Disconnected datasets <250 cases per dataset
30%-50% aren’t using valid and reliable metrics
100% of 501c3s Have 990s
Shared Dataset600,000 cases
No program quality or outcome data
600,000
60,000 ….
$400M spent on filing 990s $12B $30B
![Page 4: Next Generation Impact Measurement | July 19, 2016](https://reader035.fdocuments.us/reader035/viewer/2022070514/587d62901a28ab32318b4c13/html5/thumbnails/4.jpg)
What’s Stopping Us?
![Page 5: Next Generation Impact Measurement | July 19, 2016](https://reader035.fdocuments.us/reader035/viewer/2022070514/587d62901a28ab32318b4c13/html5/thumbnails/5.jpg)
The MetricsChallenge #1
![Page 6: Next Generation Impact Measurement | July 19, 2016](https://reader035.fdocuments.us/reader035/viewer/2022070514/587d62901a28ab32318b4c13/html5/thumbnails/6.jpg)
Lots of Outputs Little Outcomes
The Balance of What We Measure
![Page 7: Next Generation Impact Measurement | July 19, 2016](https://reader035.fdocuments.us/reader035/viewer/2022070514/587d62901a28ab32318b4c13/html5/thumbnails/7.jpg)
The Unfair Economics of the Nonprofit Sector
Consumer Buyers “Bulk” Proxy Buyers
A Consumer Buys Personal Outcomes A Funder Buys Population Impact
![Page 8: Next Generation Impact Measurement | July 19, 2016](https://reader035.fdocuments.us/reader035/viewer/2022070514/587d62901a28ab32318b4c13/html5/thumbnails/8.jpg)
Postpartum Doula ServicesBought by Me, for Me
Postpartum Home Visitation ServicesBought by Others, for a Population
The Act of “Bulk” Proxy Buying Creates Unrealistic Accountability
![Page 9: Next Generation Impact Measurement | July 19, 2016](https://reader035.fdocuments.us/reader035/viewer/2022070514/587d62901a28ab32318b4c13/html5/thumbnails/9.jpg)
Postpartum Doula ServicesBought by Me, for Me
Postpartum Home Visitation ServicesBought by Others, for You
Outcome Metrics aren’t Realistic
Lower Stress Reduction in Child Abuse
![Page 10: Next Generation Impact Measurement | July 19, 2016](https://reader035.fdocuments.us/reader035/viewer/2022070514/587d62901a28ab32318b4c13/html5/thumbnails/10.jpg)
Postpartum Doula ServicesBought by Me, for Me
Postpartum Home Visitation ServicesBought by Others, for You
Outcome Metrics aren’t Realistic
Better Infant Care Skills and
Knowledge
Achievement of
Developmental Milestones
![Page 11: Next Generation Impact Measurement | July 19, 2016](https://reader035.fdocuments.us/reader035/viewer/2022070514/587d62901a28ab32318b4c13/html5/thumbnails/11.jpg)
Postpartum Doula ServicesBought by Me, for Me
Postpartum Home Visitation ServicesBought by Others, for You
Outcome Metrics aren’t Realistic
Improved Maternal Physical
Recovery
Reduction in Subsequent Pregnancies
![Page 12: Next Generation Impact Measurement | July 19, 2016](https://reader035.fdocuments.us/reader035/viewer/2022070514/587d62901a28ab32318b4c13/html5/thumbnails/12.jpg)
Postpartum Doula ServicesBought by Me, for Me
Postpartum Home Visitation ServicesBought by Others, for You
Outcome Metrics aren’t Realistic
Better Sibling Adjustment
Reduction in Risky
Behaviors
![Page 13: Next Generation Impact Measurement | July 19, 2016](https://reader035.fdocuments.us/reader035/viewer/2022070514/587d62901a28ab32318b4c13/html5/thumbnails/13.jpg)
Postpartum Doula ServicesBought by Me, for Me
Postpartum Home Visitation ServicesBought by Others, for You
Outcome Metrics aren’t Realistic
Better Communication with Providers
Kindergarten Readiness
![Page 14: Next Generation Impact Measurement | July 19, 2016](https://reader035.fdocuments.us/reader035/viewer/2022070514/587d62901a28ab32318b4c13/html5/thumbnails/14.jpg)
Show Us Don’t Show Us
As a Result, Our Current Impact Metrics…
How MuchWe Are Doing
Direct Results
What Works
![Page 15: Next Generation Impact Measurement | July 19, 2016](https://reader035.fdocuments.us/reader035/viewer/2022070514/587d62901a28ab32318b4c13/html5/thumbnails/15.jpg)
The MethodChallenge #2
![Page 16: Next Generation Impact Measurement | July 19, 2016](https://reader035.fdocuments.us/reader035/viewer/2022070514/587d62901a28ab32318b4c13/html5/thumbnails/16.jpg)
Does a Program Work for the Average Participant?
How Does a Program Work for Targeted Segments?
Means versus Me
![Page 17: Next Generation Impact Measurement | July 19, 2016](https://reader035.fdocuments.us/reader035/viewer/2022070514/587d62901a28ab32318b4c13/html5/thumbnails/17.jpg)
Did the Program Recipe Move the Average?
Were Always Above Average
Still Below Average
The Average Moved
![Page 18: Next Generation Impact Measurement | July 19, 2016](https://reader035.fdocuments.us/reader035/viewer/2022070514/587d62901a28ab32318b4c13/html5/thumbnails/18.jpg)
Which Program Ingredients Work for You?
You are like me! Can you tell me how you did it?
![Page 19: Next Generation Impact Measurement | July 19, 2016](https://reader035.fdocuments.us/reader035/viewer/2022070514/587d62901a28ab32318b4c13/html5/thumbnails/19.jpg)
Data AnalysisChallenge #3
![Page 20: Next Generation Impact Measurement | July 19, 2016](https://reader035.fdocuments.us/reader035/viewer/2022070514/587d62901a28ab32318b4c13/html5/thumbnails/20.jpg)
Analyses Describe What Just Happened
![Page 21: Next Generation Impact Measurement | July 19, 2016](https://reader035.fdocuments.us/reader035/viewer/2022070514/587d62901a28ab32318b4c13/html5/thumbnails/21.jpg)
The Unresolved Challenges of Impact MeasurementChallenge #3: Data Analysis
Analyses Describe What Just Happened
Amount, Frequencies,
Averages, Trends
Statistical Significance of
Outcomes
![Page 22: Next Generation Impact Measurement | July 19, 2016](https://reader035.fdocuments.us/reader035/viewer/2022070514/587d62901a28ab32318b4c13/html5/thumbnails/22.jpg)
Predict What Might Happen Provide Insight on What Might Work
Very Few Existing Analyses…
![Page 23: Next Generation Impact Measurement | July 19, 2016](https://reader035.fdocuments.us/reader035/viewer/2022070514/587d62901a28ab32318b4c13/html5/thumbnails/23.jpg)
Describes Never
As a Result, Our Current Impact Metrics…
Often, What We Just Did
Sometimes, Our Results
Diagnoses ProblemsPredicts What Might
HappenProvides Insights About
What To Do
![Page 24: Next Generation Impact Measurement | July 19, 2016](https://reader035.fdocuments.us/reader035/viewer/2022070514/587d62901a28ab32318b4c13/html5/thumbnails/24.jpg)
Measuring in IsolationChallenge #4
![Page 25: Next Generation Impact Measurement | July 19, 2016](https://reader035.fdocuments.us/reader035/viewer/2022070514/587d62901a28ab32318b4c13/html5/thumbnails/25.jpg)
We Don’t Share Stories, Insights or LessonsData Analytics Findings
![Page 26: Next Generation Impact Measurement | July 19, 2016](https://reader035.fdocuments.us/reader035/viewer/2022070514/587d62901a28ab32318b4c13/html5/thumbnails/26.jpg)
The Solution
![Page 27: Next Generation Impact Measurement | July 19, 2016](https://reader035.fdocuments.us/reader035/viewer/2022070514/587d62901a28ab32318b4c13/html5/thumbnails/27.jpg)
Step One: Come Together w/in Fields of Practice
![Page 28: Next Generation Impact Measurement | July 19, 2016](https://reader035.fdocuments.us/reader035/viewer/2022070514/587d62901a28ab32318b4c13/html5/thumbnails/28.jpg)
Come Together w/in Fields of PracticeThere’s no one-size-fits-all ideal business
performance and impact model for ALL for-profit service companies
Each ”industry” has it’s own standards, and when it comes to evaluating their services, each business within an industry competes on a set of common
market-driven (customer-driven need and demand) outcomes.
Academic Achievement Consulting/Capacity Building
Employment Career Advancement
It’s time to do the same within our
sectorYouth Development
Nonprofit Capacity Building
Senior Services
Workforce Development
Environmental Policy
Etc.
![Page 29: Next Generation Impact Measurement | July 19, 2016](https://reader035.fdocuments.us/reader035/viewer/2022070514/587d62901a28ab32318b4c13/html5/thumbnails/29.jpg)
Step Two: Agree on Shared Metrics
![Page 30: Next Generation Impact Measurement | July 19, 2016](https://reader035.fdocuments.us/reader035/viewer/2022070514/587d62901a28ab32318b4c13/html5/thumbnails/30.jpg)
Develop Shared Metrics
Target Population Field Practices Achievable Field Outcomes
![Page 31: Next Generation Impact Measurement | July 19, 2016](https://reader035.fdocuments.us/reader035/viewer/2022070514/587d62901a28ab32318b4c13/html5/thumbnails/31.jpg)
Step Three: Real-World Methods
![Page 32: Next Generation Impact Measurement | July 19, 2016](https://reader035.fdocuments.us/reader035/viewer/2022070514/587d62901a28ab32318b4c13/html5/thumbnails/32.jpg)
Natural ExperimentControlled Experiment
Conduct Natural Experiments
![Page 33: Next Generation Impact Measurement | July 19, 2016](https://reader035.fdocuments.us/reader035/viewer/2022070514/587d62901a28ab32318b4c13/html5/thumbnails/33.jpg)
Add Predictive Analytics
“Predictive analytics tells you what will [likely] happen”
Jeff Bertolucci, Information Week
![Page 34: Next Generation Impact Measurement | July 19, 2016](https://reader035.fdocuments.us/reader035/viewer/2022070514/587d62901a28ab32318b4c13/html5/thumbnails/34.jpg)
Add Prescriptive Analytics
“…prescriptive analytics tells you what to do about
it.”Jeff Bertolucci, Information Week
Find the Positive Deviants!
![Page 35: Next Generation Impact Measurement | July 19, 2016](https://reader035.fdocuments.us/reader035/viewer/2022070514/587d62901a28ab32318b4c13/html5/thumbnails/35.jpg)
Step Four: Make Meaning, Together
![Page 36: Next Generation Impact Measurement | July 19, 2016](https://reader035.fdocuments.us/reader035/viewer/2022070514/587d62901a28ab32318b4c13/html5/thumbnails/36.jpg)
Everyone Gathering Stories, Finding Positive Deviants and Sharing How They Did It
![Page 37: Next Generation Impact Measurement | July 19, 2016](https://reader035.fdocuments.us/reader035/viewer/2022070514/587d62901a28ab32318b4c13/html5/thumbnails/37.jpg)
Data-Driven Decision Making for Impact