The Promise of Programmatic, WTF Programmatic, December 2016
Using Data and Evidence to Strengthen Services for ... · • Creating Cross -Disciplinary...
Transcript of Using Data and Evidence to Strengthen Services for ... · • Creating Cross -Disciplinary...
![Page 1: Using Data and Evidence to Strengthen Services for ... · • Creating Cross -Disciplinary Workgroup • Identifying Priorities Across an Educational Continuum • Establishing Programmatic](https://reader036.fdocuments.us/reader036/viewer/2022063010/5fc2f0e3510b58528c0fc14b/html5/thumbnails/1.jpg)
Using Data and Evidence to Strengthen Services for Homeless Youth at Good Shepherd Services Beyond Housing 2014: A National Conversation on Child Homelessness and Poverty January 16, 2014
Panelists: Elizabeth Garcia, Program Director, Chelsea Foyer/Edwin Gould/Street Outreach Adrian Rodriguez, Program Manager, Chelsea Foyer Whitney Welshimer, Program Analyst, Program Evaluation and Planning Department Miranda Yates, Director, Program Evaluation and Planning Department
![Page 2: Using Data and Evidence to Strengthen Services for ... · • Creating Cross -Disciplinary Workgroup • Identifying Priorities Across an Educational Continuum • Establishing Programmatic](https://reader036.fdocuments.us/reader036/viewer/2022063010/5fc2f0e3510b58528c0fc14b/html5/thumbnails/2.jpg)
AGENDA
o Good Shepherd’s Approach to Program Evaluation and Planning
o Overview Chelsea Foyer Program
o Steps in Using Data for Strengthening Outcomes
o Q & A Panel
2
![Page 3: Using Data and Evidence to Strengthen Services for ... · • Creating Cross -Disciplinary Workgroup • Identifying Priorities Across an Educational Continuum • Establishing Programmatic](https://reader036.fdocuments.us/reader036/viewer/2022063010/5fc2f0e3510b58528c0fc14b/html5/thumbnails/3.jpg)
3
WHO WE ARE
Good Shepherd Services is a leading youth and family development and educational support agency in New York City. Focusing on under resourced communities in Brooklyn and the Bronx, we provide a broad array of individual, family and school-based services that give young people and families the opportunity to take ownership of their future, making a difference today and for the next generation.
![Page 4: Using Data and Evidence to Strengthen Services for ... · • Creating Cross -Disciplinary Workgroup • Identifying Priorities Across an Educational Continuum • Establishing Programmatic](https://reader036.fdocuments.us/reader036/viewer/2022063010/5fc2f0e3510b58528c0fc14b/html5/thumbnails/4.jpg)
WHAT WE DO
26,037 participants were served through
81 programs operated across
3 boroughs (Brooklyn, Bronx, Manhattan).
4
As a multi-service agency, we operate two networks of community-based youth development, education, youth justice, and family service programs in Brooklyn and the Bronx; group homes for adolescents; supportive housing for young adults; foster care and adoption services; and an in-service professional training program.
![Page 5: Using Data and Evidence to Strengthen Services for ... · • Creating Cross -Disciplinary Workgroup • Identifying Priorities Across an Educational Continuum • Establishing Programmatic](https://reader036.fdocuments.us/reader036/viewer/2022063010/5fc2f0e3510b58528c0fc14b/html5/thumbnails/5.jpg)
5
HOW WE WORK
We surround youth and their families with a variety of services that keep youth connected to family, school, and the community. o Youth and Family Development Approach o Commitment to Community o Partnership/Shared Resources
![Page 6: Using Data and Evidence to Strengthen Services for ... · • Creating Cross -Disciplinary Workgroup • Identifying Priorities Across an Educational Continuum • Establishing Programmatic](https://reader036.fdocuments.us/reader036/viewer/2022063010/5fc2f0e3510b58528c0fc14b/html5/thumbnails/6.jpg)
6
We are a Learning Organization committed to continuous quality improvement.
Across all of Good Shepherd Services’ programs, we assess positive impact by focusing on three core Development Outcomes: o Safety
o Belonging
o Skill-building
APPROACH TO EVALUATION
![Page 7: Using Data and Evidence to Strengthen Services for ... · • Creating Cross -Disciplinary Workgroup • Identifying Priorities Across an Educational Continuum • Establishing Programmatic](https://reader036.fdocuments.us/reader036/viewer/2022063010/5fc2f0e3510b58528c0fc14b/html5/thumbnails/7.jpg)
7 Source: Satterfield, Spring, Brownson, Mullen, Newhouse, Walker, & Whitlock (2009)
APPROACH TO EVALUATION Transdisciplinary Model of Evidence-Based Practice
![Page 8: Using Data and Evidence to Strengthen Services for ... · • Creating Cross -Disciplinary Workgroup • Identifying Priorities Across an Educational Continuum • Establishing Programmatic](https://reader036.fdocuments.us/reader036/viewer/2022063010/5fc2f0e3510b58528c0fc14b/html5/thumbnails/8.jpg)
CHELSEA FOYER Program Overview
o Opened in 2004. Based on Foyer model developed in the UK
o Provides 40 homeless, runaway, and foster care youth, ages 18-25, with supported transitional housing
o Funding from multiple public & private contracts
o Residents participate in a personalized program of services for up to 24 months
o Services infused with Good Shepherd Services’ signature strength-based youth and family development practices
8
![Page 9: Using Data and Evidence to Strengthen Services for ... · • Creating Cross -Disciplinary Workgroup • Identifying Priorities Across an Educational Continuum • Establishing Programmatic](https://reader036.fdocuments.us/reader036/viewer/2022063010/5fc2f0e3510b58528c0fc14b/html5/thumbnails/9.jpg)
o Case Management Services o Life-Skills Development o Workforce Development o Community Building o Housing and Aftercare Services
o Rigorous Application Process o Contract and Action Plan o Limited Structure/High Expectations o On-Site Support Services o Program Fee o Workforce Development Culture
CHELSEA FOYER
Key Program Components On-Site Support Services
![Page 10: Using Data and Evidence to Strengthen Services for ... · • Creating Cross -Disciplinary Workgroup • Identifying Priorities Across an Educational Continuum • Establishing Programmatic](https://reader036.fdocuments.us/reader036/viewer/2022063010/5fc2f0e3510b58528c0fc14b/html5/thumbnails/10.jpg)
A. DEFINE Program Planning
B. MEASURE Data Collection & Management
C. LEARN Analysis & Reporting
D. IMPROVE Using Findings
and Insights
10 Source: Adapted from Eckart-Queenan & Forti (2011)
PERFORMANCE MANAGEMENT LIFECYCLE
![Page 11: Using Data and Evidence to Strengthen Services for ... · • Creating Cross -Disciplinary Workgroup • Identifying Priorities Across an Educational Continuum • Establishing Programmatic](https://reader036.fdocuments.us/reader036/viewer/2022063010/5fc2f0e3510b58528c0fc14b/html5/thumbnails/11.jpg)
11
LOGIC MODEL
![Page 12: Using Data and Evidence to Strengthen Services for ... · • Creating Cross -Disciplinary Workgroup • Identifying Priorities Across an Educational Continuum • Establishing Programmatic](https://reader036.fdocuments.us/reader036/viewer/2022063010/5fc2f0e3510b58528c0fc14b/html5/thumbnails/12.jpg)
12
PROGRAM OUTCOMES
![Page 13: Using Data and Evidence to Strengthen Services for ... · • Creating Cross -Disciplinary Workgroup • Identifying Priorities Across an Educational Continuum • Establishing Programmatic](https://reader036.fdocuments.us/reader036/viewer/2022063010/5fc2f0e3510b58528c0fc14b/html5/thumbnails/13.jpg)
13
OUTCOMES PROGRESS REPORT
![Page 14: Using Data and Evidence to Strengthen Services for ... · • Creating Cross -Disciplinary Workgroup • Identifying Priorities Across an Educational Continuum • Establishing Programmatic](https://reader036.fdocuments.us/reader036/viewer/2022063010/5fc2f0e3510b58528c0fc14b/html5/thumbnails/14.jpg)
14
OUTCOMES PROGRESS REPORT
![Page 15: Using Data and Evidence to Strengthen Services for ... · • Creating Cross -Disciplinary Workgroup • Identifying Priorities Across an Educational Continuum • Establishing Programmatic](https://reader036.fdocuments.us/reader036/viewer/2022063010/5fc2f0e3510b58528c0fc14b/html5/thumbnails/15.jpg)
15
USING DATA TO STRENGTHEN OUTCOMES: STEPS
1. Identify Priority Outcome to Strengthen
2. Deepen Understanding of this Outcome
3. Develop Strategy to Strengthen Results
4. Implement New Strategy
5. Re-Evaluate
![Page 16: Using Data and Evidence to Strengthen Services for ... · • Creating Cross -Disciplinary Workgroup • Identifying Priorities Across an Educational Continuum • Establishing Programmatic](https://reader036.fdocuments.us/reader036/viewer/2022063010/5fc2f0e3510b58528c0fc14b/html5/thumbnails/16.jpg)
1. Identify Priority Outcome to Strengthen
16
USING DATA TO STRENGTHEN OUTCOMES
41%
33% 31%
0%
20%
40%
60%
80%
100%
FY11 (n=68) FY12 (n=74) FY13 (n=69) %
Enr
olle
d Active Year
Enrolled in Educational/Vocational Program FY11-13
FY 2013 Priority Outcome: Employment FY 2014 Priority Outcome: Education
43% 43% 56%
0%
20%
40%
60%
80%
100%
FY10 (n=35) FY11 (n=30) FY12 (n=36) % E
mpl
oyed
at D
isch
arge
Discharge Year
Employment at Discharge FY10-12
![Page 17: Using Data and Evidence to Strengthen Services for ... · • Creating Cross -Disciplinary Workgroup • Identifying Priorities Across an Educational Continuum • Establishing Programmatic](https://reader036.fdocuments.us/reader036/viewer/2022063010/5fc2f0e3510b58528c0fc14b/html5/thumbnails/17.jpg)
2. Deepen Understanding of this Outcome
17
USING DATA TO STRENGTHEN OUTCOMES
• Performance trends across time • Performance variation by demographic characteristics • Performance variation by prior experience
![Page 18: Using Data and Evidence to Strengthen Services for ... · • Creating Cross -Disciplinary Workgroup • Identifying Priorities Across an Educational Continuum • Establishing Programmatic](https://reader036.fdocuments.us/reader036/viewer/2022063010/5fc2f0e3510b58528c0fc14b/html5/thumbnails/18.jpg)
3. Develop Strategy to Strengthen Outcome Performance
18
USING DATA TO STRENGTHEN OUTCOMES
FY 2013 Priority Outcome: Employment
Research evidence:
Practitioner expertise:
-WORC Model - Literature on workforce development
-Participant Satisfaction Survey -Internal Data
- Internal (Case Managers, Vocational Specialist) - External (Mentoring USA, Reciprocity Foundation)
Participant needs and values:
Decision Making
![Page 19: Using Data and Evidence to Strengthen Services for ... · • Creating Cross -Disciplinary Workgroup • Identifying Priorities Across an Educational Continuum • Establishing Programmatic](https://reader036.fdocuments.us/reader036/viewer/2022063010/5fc2f0e3510b58528c0fc14b/html5/thumbnails/19.jpg)
3. Develop Strategy to Strengthen Outcome Performance
19
USING DATA TO STRENGTHEN OUTCOMES
FY 2014 Priority Outcome: Education
- Case Managers, Director of Post-Secondary Initiatives - Cross-Agency Workgroup
Practitioner Expertise:
Research evidence:
- National Clearinghouse on Families and Youth - Community College Research Center
- Participant Satisfaction Survey -Internal Data
Participant needs and values:
Decision Making
![Page 20: Using Data and Evidence to Strengthen Services for ... · • Creating Cross -Disciplinary Workgroup • Identifying Priorities Across an Educational Continuum • Establishing Programmatic](https://reader036.fdocuments.us/reader036/viewer/2022063010/5fc2f0e3510b58528c0fc14b/html5/thumbnails/20.jpg)
3. Develop Strategy to Strengthen Outcome Performance:
20
USING DATA TO STRENGTHEN OUTCOMES
FY 2013 Priority Outcome: Employment
Youth Employed at Discharge Experience More:
• Workshops
• Internships/Fellowships
• Employment during first 30 days at Foyer
FY 2014 Priority Outcome: Education
Youth Enrolled in School Are More Likely To:
• Be employed at discharge
• Make a program payment
• Save more money by discharge
Using Internal Data to Understand Participant Success
![Page 21: Using Data and Evidence to Strengthen Services for ... · • Creating Cross -Disciplinary Workgroup • Identifying Priorities Across an Educational Continuum • Establishing Programmatic](https://reader036.fdocuments.us/reader036/viewer/2022063010/5fc2f0e3510b58528c0fc14b/html5/thumbnails/21.jpg)
4. Implement New Strategy
21
USING DATA TO STRENGTHEN OUTCOMES
FY 2013 Priority Outcome: Employment “Stepping Stones to Success”
FY 2014 Priority Outcome: Education
“Future Focus Initiative” In Progress…
• Assessment Process • Comprehensive WORC Assessments • Meetings with Vocational Specialist
• Skill-Building Workshops • Career Club (“Express to Success”) • Financial Literacy • Project Voice (new) • Mentoring USA (new)
• Career Exploration • Reciprocity Foundation Programming
• Supported Experiential Learning
• Expanded Internship Opportunities
• Training on Education-Specific Pathways and Options
• Creating Cross-Disciplinary Workgroup
• Identifying Priorities Across an Educational Continuum
• Establishing Programmatic Curriculum and Processes
• Defining Multi-Phase Process with Milestones
![Page 22: Using Data and Evidence to Strengthen Services for ... · • Creating Cross -Disciplinary Workgroup • Identifying Priorities Across an Educational Continuum • Establishing Programmatic](https://reader036.fdocuments.us/reader036/viewer/2022063010/5fc2f0e3510b58528c0fc14b/html5/thumbnails/22.jpg)
5. Re-Evaluate
22
USING DATA TO STRENGTHEN OUTCOMES
FY 2013 Priority Outcome: Employment FY 2014 Priority Outcome: Education
TBD 43% 43% 56%
66%
74%
0%
20%
40%
60%
80%
100%
FY10 (n=35)
FY11 (n=30)
FY12 (n=36)
FY13 (n=29)
FY14 (n=19) %
Em
ploy
ed a
t Dis
char
ge
Discharge Year
Employment at Discharge FY 10-14
![Page 23: Using Data and Evidence to Strengthen Services for ... · • Creating Cross -Disciplinary Workgroup • Identifying Priorities Across an Educational Continuum • Establishing Programmatic](https://reader036.fdocuments.us/reader036/viewer/2022063010/5fc2f0e3510b58528c0fc14b/html5/thumbnails/23.jpg)
23
10 VALUES GUIDING THE PROGRAM EVALUATION & PLANNING PROCESS
Mission-Driven
Aspirational
Partnered
Learning-oriented
Strategic
Evidence-informed
Iterative
Authentically empowering
Tailored
Flexible
![Page 24: Using Data and Evidence to Strengthen Services for ... · • Creating Cross -Disciplinary Workgroup • Identifying Priorities Across an Educational Continuum • Establishing Programmatic](https://reader036.fdocuments.us/reader036/viewer/2022063010/5fc2f0e3510b58528c0fc14b/html5/thumbnails/24.jpg)
PANEL CONVERSATION
24
![Page 25: Using Data and Evidence to Strengthen Services for ... · • Creating Cross -Disciplinary Workgroup • Identifying Priorities Across an Educational Continuum • Establishing Programmatic](https://reader036.fdocuments.us/reader036/viewer/2022063010/5fc2f0e3510b58528c0fc14b/html5/thumbnails/25.jpg)
•Dworsky, A. (2010). Supporting Homeless Youth During the Transition to Adulthood: Housing Based Independent Living Programs Retrieved from http://www.highbeam.com/doc/1G1-224934236.html
•Eckhart-Queenan, J. & Forti, M. (2011). Measurement as Learning: What Nonprofit CEOs, Board Members, and Philanthropists Need to Know to Keep Improving. Boston: MA: The Bridgespan Group. Retrieved from http://www.bridgespan.org/measurement-as-learning.aspx
•Good Shepherd Services (2012). Good Practice Produces Good Outcomes: Good Shepherd Services’ Commitment to Evidence-Based Practice. Retrieved from http://www.goodshepherds.org/about/results.html •Kirk, A. Visualising Data blog http://www.visualisingdata.com •Morton, M.M. (2012). Applying Evidence-Based Practice to Runaway and Homeless Youth Services. Administration for Children & Families. Retrieved from http://www.rhyttac.ou.edu/webinars/topical/968-applying-evidence-based-practice-to-runaway-and-homeless-youth-services
•Patton, M. Q. (2013). Utilization-Focused Evaluation Checklist. Retrieved from http://www.wmich.edu/evalctr/wp-content/uploads/2013/03/UFE_checklist_2013.pdf •Satterfield, J.M., Spring, B., Brownson, R.C., Mullen, E.J., Newhouse, R.P., Walker, B.B., & Whitlock, E.P. (2009). Toward a transdisciplinary model of evidence-based practice. The Milbank Quarterly, 87 (2), 368-390. Retrieved from http://www.ncbi.nlm.nih.gov/pmc/articles/PMC2698591/?tool=pubmed
•The Urban Institute (2003). Keys to Outcome Management. Washington, DC: Urban Institute. Retrieved from http://www.urban.org/UploadedPDF/310776_KeySteps.pdf
•W. K. Kellogg Foundation (2004). Logic Model Development Guide: Using Logic Models to Bring Together Planning, Evaluation, and Action. Battlecreek, MI: Author. Retrieved from http://www.wisconsin.edu/edi/grants/Kellogg_Logic_Model.pdf
RESOURCES
25
![Page 26: Using Data and Evidence to Strengthen Services for ... · • Creating Cross -Disciplinary Workgroup • Identifying Priorities Across an Educational Continuum • Establishing Programmatic](https://reader036.fdocuments.us/reader036/viewer/2022063010/5fc2f0e3510b58528c0fc14b/html5/thumbnails/26.jpg)
THANK YOU!
For more information about Good Shepherd Services, please visit www.goodshepherds.org Contact: Elizabeth Garcia, LMSW Program Director, Chelsea Foyer/Edwin Gould /Street Outreach [email protected] Miranda Yates, Ph.D. Director of Program Evaluation and Planning [email protected]
26