Post on 29-Jan-2016
description
Improving Practice to Wraparound Improving Practice to Wraparound Through Use of Fidelity MeasuresThrough Use of Fidelity Measures
Enrica Bertoldo, Quality Support ManagerEnrica Bertoldo, Quality Support ManagerEleanor Castillo, Director of Outcomes & Quality Assurance Eleanor Castillo, Director of Outcomes & Quality Assurance
Mary Ann Wong, Research SpecialistMary Ann Wong, Research SpecialistVeronica Padilla, Manager of Outcomes & EvaluationsVeronica Padilla, Manager of Outcomes & Evaluations
26th26th Annual California Mental Health Advocates for Children & Youth Annual California Mental Health Advocates for Children & Youth Conference, Asilomar, CaliforniaConference, Asilomar, California
May 4, 2006May 4, 2006
EMQ Children & Family EMQ Children & Family ServicesServices
Our MissionOur MissionTo work with children and their families To work with children and their families to transform their lives, build emotional, to transform their lives, build emotional, social and familial well-being, and to social and familial well-being, and to transform the systems that serve them.transform the systems that serve them.
Our PhilosophyOur Philosophy Family Voice Family Voice Team Based Team Based Collaborative/IntegrativeCollaborative/Integrative Community-Based Community-Based Culturally CompetentCulturally Competent Individualized Individualized Strength-BasedStrength-Based Natural Supports Natural Supports PersistencePersistence Outcomes-Based Outcomes-Based
PurposePurpose
Given the complexity of Wraparound, existing Given the complexity of Wraparound, existing fidelity measures are utilized to improve fidelity measures are utilized to improve practice.practice.
This workshop will share successes and This workshop will share successes and challenges in the implementation of fidelity challenges in the implementation of fidelity measures (WFI-3.0 & WOF-2.0) that affect measures (WFI-3.0 & WOF-2.0) that affect practice and supervision to Wraparound.practice and supervision to Wraparound.
A pilot in utilizing a Wraparound supervisory A pilot in utilizing a Wraparound supervisory adherence measure (W-SAM) will also be adherence measure (W-SAM) will also be discussed.discussed.
What is Wraparound?What is Wraparound?
Team-based, individualized service Team-based, individualized service planning processplanning processPrinciples of community-based, strength-Principles of community-based, strength-based, family-centered, and culturally based, family-centered, and culturally competent competent Wraparound is described as a promising Wraparound is described as a promising practice (Burns, Hoagwood, & Maultsby, practice (Burns, Hoagwood, & Maultsby, 1998)1998)
11 Core Elements of Wraparound11 Core Elements of Wraparound
1.1. Voice and ChoiceVoice and Choice2.2. Youth/Family TeamYouth/Family Team3.3. Community-Based Community-Based
Services/SupportsServices/Supports4.4. Cultural Cultural
CompetenceCompetence5.5. Individualized Individualized
Services/SupportsServices/Supports6.6. Strengths-Based Strengths-Based
Services/SupportsServices/Supports
7.7. Natural SupportsNatural Supports8.8. Continuation of Continuation of
Services/SupportsServices/Supports
9.9. CollaborationCollaboration10.10. Flexible Resources/ Flexible Resources/
Funding Funding11.11. Outcome-Based Outcome-Based
Services/SupportsServices/Supports
Essential Elements of Wraparound Essential Elements of Wraparound
Element 1: Voice and ChoiceElement 1: Voice and ChoiceFamilies must be full and active partners at every level of the wraparound Families must be full and active partners at every level of the wraparound process. If the team cannot reach consensus, the final decision should be process. If the team cannot reach consensus, the final decision should be up to the caregiver.up to the caregiver.
Element 2: Youth and Family TeamElement 2: Youth and Family TeamWraparound is a team-driven process involving caregivers, youth, natural Wraparound is a team-driven process involving caregivers, youth, natural supports, and community services working together to develop, supports, and community services working together to develop, implement, and evaluate the individualized plan.implement, and evaluate the individualized plan.
Element 3: Community-based Services and SupportsElement 3: Community-based Services and SupportsServices and supports that the youth and family receive should be based Services and supports that the youth and family receive should be based in their community. The family should not have to leave their community if in their community. The family should not have to leave their community if more restrictive services are necessary.more restrictive services are necessary.
Element 4: Cultural CompetenceElement 4: Cultural CompetenceThe team should not only be respectful of the family’s beliefs and The team should not only be respectful of the family’s beliefs and traditions, but also actively seek to under-stand the family’s unique traditions, but also actively seek to under-stand the family’s unique perspectives and convey them to others.perspectives and convey them to others.
Element 5: Individualized ServicesElement 5: Individualized Services This means that services and supports are tailored to the unique This means that services and supports are tailored to the unique
situation, strengths, and needs of teach individual, and may involve situation, strengths, and needs of teach individual, and may involve existing categorical services and informal sup-ports; modifying existing existing categorical services and informal sup-ports; modifying existing services and supports; and or creating new services and supports. services and supports; and or creating new services and supports. Further, the team should create a specific plan to meet the family’s goals Further, the team should create a specific plan to meet the family’s goals and crisis/ safety plan to manage potential emergencies.and crisis/ safety plan to manage potential emergencies.
Element 6: Strengths-Based ServicesElement 6: Strengths-Based ServicesThe focus of the team should be on what is working and going well for the The focus of the team should be on what is working and going well for the family. While goals may be drawn up based on the family’s needs, the family. While goals may be drawn up based on the family’s needs, the plan should capitalize on the family’s positive abilities and characteristics.plan should capitalize on the family’s positive abilities and characteristics.
Element 7: Natural SupportsElement 7: Natural SupportsServices and supports should reflect a balance of formal and informal Services and supports should reflect a balance of formal and informal community and family supports rather than a reliance on formal community and family supports rather than a reliance on formal professional services.professional services.
Element 8: Continuation of CareElement 8: Continuation of CareServices and supports must be provided unconditionally. In a crisis, Services and supports must be provided unconditionally. In a crisis, services and supports should be added rather than placing the youth with services and supports should be added rather than placing the youth with a new provider.a new provider.
Element 9: CollaborationElement 9: CollaborationThe team should coordinate services and supports so they seem The team should coordinate services and supports so they seem seamless to the family rather than disjointed.seamless to the family rather than disjointed.
Element 10: Flexible Funding and ResourcesElement 10: Flexible Funding and ResourcesSuccessful wraparound teams are creative in their approach to service Successful wraparound teams are creative in their approach to service delivery and have access to flexible funds and resources to implement delivery and have access to flexible funds and resources to implement their ideas.their ideas.
Element 11: Outcome-Based ServicesElement 11: Outcome-Based ServicesSpecific, measurable out-comes should be monitored to assess the youth Specific, measurable out-comes should be monitored to assess the youth and family’s progress toward goals.and family’s progress toward goals.
The description of each element is taken directly from The description of each element is taken directly from Wraparound Fidelity Index 3.0Wraparound Fidelity Index 3.0 (2002), (2002), produced by the Wraparound Evaluation and Research Team at the University of Vermont. produced by the Wraparound Evaluation and Research Team at the University of Vermont.
What is Fidelity to Wraparound?What is Fidelity to Wraparound?
Treatment Fidelity = The degree to which a Treatment Fidelity = The degree to which a program is implemented as intended (Rast & program is implemented as intended (Rast & Bruns, 2003; Moncher & Prinz, 1991)Bruns, 2003; Moncher & Prinz, 1991)
Adherence to the 11 Core Wraparound Adherence to the 11 Core Wraparound Elements (e.g., WFI-3.0; Suter et al., 2002)Elements (e.g., WFI-3.0; Suter et al., 2002)
Adherence to the Child and Family Team Adherence to the Child and Family Team Process (e.g., WOF-2.0; Epstein et al., 2002)Process (e.g., WOF-2.0; Epstein et al., 2002)
Why Measure Fidelity?Why Measure Fidelity?
Measuring fidelity is essential to families, Measuring fidelity is essential to families, providers, policy makers and researchersproviders, policy makers and researchers
1.1. Without measuring fidelity, how do you ensure the Without measuring fidelity, how do you ensure the Wraparound process is occurring?Wraparound process is occurring?
2.2. Without outcomes, the Wraparound process is just Without outcomes, the Wraparound process is just one more fadone more fad
3.3. High fidelity has been associated with positive High fidelity has been associated with positive outcomes in some studiesoutcomes in some studies
4.4. It improves quality assuranceIt improves quality assurance5.5. It helps agencies secure more funding by proving It helps agencies secure more funding by proving
outcomesoutcomes6.6. It can even help create legislation on how families It can even help create legislation on how families
and kids receive servicesand kids receive services
More Why’s…..More Why’s…..
All nationally recognized Wraparound programs All nationally recognized Wraparound programs have extensive fidelity measurements in place. have extensive fidelity measurements in place.
Why?Why?Without fidelity measurement standards firmly in Without fidelity measurement standards firmly in
place, Wraparound programs and team place, Wraparound programs and team members have difficulty knowing what they are members have difficulty knowing what they are doing well and what they need to improve in doing well and what they need to improve in order to achieve the ideal as presented in the order to achieve the ideal as presented in the Wraparound vision.Wraparound vision.
Fidelity Implementation MeasuresFidelity Implementation Measures
Wraparound Fidelity Index-3.0 (WFI-3.0)Wraparound Fidelity Index-3.0 (WFI-3.0)Measures fidelity to Wraparound principles through Measures fidelity to Wraparound principles through structured interviewsstructured interviews
Administered to Youth, Caregiver, & Resource Administered to Youth, Caregiver, & Resource FacilitatorFacilitator
Higher Score = Greater FidelityHigher Score = Greater Fidelity
Wraparound Observation Form-2.0 (WOF-2.0)Wraparound Observation Form-2.0 (WOF-2.0)Measures fidelity to Wraparound through Measures fidelity to Wraparound through observation of the Child and Family Team meetingsobservation of the Child and Family Team meetings
Higher Score = Greater FidelityHigher Score = Greater Fidelity
Fidelity to WraparoundFidelity to Wraparound
EMQ Sacramento has collected the EMQ Sacramento has collected the Wraparound Fidelity Index (WFI) since Wraparound Fidelity Index (WFI) since 2001 2001
Data from WFI-3.0: Data from WFI-3.0: January 2003 – September 2005January 2003 – September 2005
A Comparison of EMQ Data & A Comparison of EMQ Data & National DataNational Data
WFI Total Scores by Respondent: WFI Total Scores by Respondent: EMQ & National DataEMQ & National Data
77
74
74
81
73
67
67
79
0 20 40 60 80 100
Total
Youth
Caregiver
Facilitator
EMQ
National
*National WFI-3 Dataset came from the Wraparound Evaluation and Research Team presentation for the 18th Annual Research Conference on Systems of Care and Children's Mental Health in Tampa on 3/7/05 by Eric J. Bruns, Ph.D. for "Is it Wraparound Yet?" - Bootstrapping wraparound fidelity standards using the WFI .
WFI Elements: EMQ & National DataWFI Elements: EMQ & National Data
81
83
88
73
66
87
80
86
83
62
56
83
0 20 40 60 80 100
Strengths
Individualized
Cultural Competence
Community
Youth/Family Team
Voice/Choice
EMQ
National
*National WFI-3 Dataset came from the Wraparound Evaluation and Research Team presentation for the Technical Assistance Partnership Webinar on 4/5/04 by Eric J. Bruns, Ph.D. for "Ensuring High-Quality Wraparound"
WFI Elements: EMQ & National DataWFI Elements: EMQ & National Data
77
68
68
85
63
85
66
76
76
47
0 20 40 60 80 100
Outcome-Based
Flex Funds
Collaboration
Continuation
NaturalSupports
EMQ
National
*National WFI-3 Dataset came from the Wraparound Evaluation and Research Team presentation for the Technical Assistance Partnership Webinar on 4/5/04 by Eric J. Bruns, Ph.D. for "Ensuring High-Quality Wraparound"
The Relationship Between The Relationship Between Fidelity & Outcomes Fidelity & Outcomes
The Relationship Between The Relationship Between Fidelity & Outcomes Fidelity & Outcomes
Previous research has found an association Previous research has found an association between greater Wraparound fidelity and between greater Wraparound fidelity and better child and family outcomes (Bruns, better child and family outcomes (Bruns, 2004), but the relationship has not been clearly 2004), but the relationship has not been clearly understoodunderstood
EMQ (2005) completed a study to further EMQ (2005) completed a study to further understand the relationship between fidelity to understand the relationship between fidelity to the 11 core elements of Wraparound and the 11 core elements of Wraparound and treatment outcomes treatment outcomes
DemographicsDemographics
Average Age at Admission: 14 yearsAverage Age at Admission: 14 years
63% Male; 37% Female63% Male; 37% Female
62% Caucasian; 26% African-American; 62% Caucasian; 26% African-American; 8% Latino; 2% Asian/Pacific Islander; 8% Latino; 2% Asian/Pacific Islander; 1% Native-American; 1% Other1% Native-American; 1% Other
Average Length of Stay: 15 monthsAverage Length of Stay: 15 months
Fidelity to WraparoundFidelity to Wraparound
Implementation Measure: Implementation Measure:
Wraparound Fidelity Index – 3.0 (WFI-3.0)Wraparound Fidelity Index – 3.0 (WFI-3.0)
Collection since 2003Collection since 2003
Sample in study included the following:Sample in study included the following:
146 WFI’s collected from Youth146 WFI’s collected from Youth
124 WFI’s collected from Caregivers124 WFI’s collected from Caregivers
183 WFI’s collected from Resource 183 WFI’s collected from Resource FacilitatorsFacilitators
Outcome Measures at DischargeOutcome Measures at Discharge
Child and Adolescent Functional Assessment Scale Child and Adolescent Functional Assessment Scale (CAFAS; Hodges, 2000)(CAFAS; Hodges, 2000)
Measures youth’s level of functioning; completed by Resource FacilitatorMeasures youth’s level of functioning; completed by Resource FacilitatorLower Score = Greater FunctioningLower Score = Greater FunctioningExit Total CAFAS ScoreExit Total CAFAS Score
Child Behavior Checklist (CBCL; Achenbach, 2001)Child Behavior Checklist (CBCL; Achenbach, 2001)Measures youth’s level of behavior problems; administered to CaregiverMeasures youth’s level of behavior problems; administered to CaregiverLower Score = Lower Behavior ProblemsLower Score = Lower Behavior ProblemsExit Total, Externalizing, & Internalizing CBCL ScoresExit Total, Externalizing, & Internalizing CBCL Scores
Living Arrangement at Discharge: Community or Living Arrangement at Discharge: Community or FacilityFacility
CBCLTotal
CBCL Externalizing
CBCL Internalizing
CAFAS Total
Living in Community
1. Voice and Choice .12 .07 .12 .15* .47
2. Youth and Family Team -.04 -.04 .01 .07 .10
3. Community-Based Services
-.19* -.25** -.21* -.40** .21**
4. Cultural Competence .06 .01 .02 .10 .15
5. Individualized Services .09 .06 .01 .11 .05
6. Strengths-Based -.10 -.09 -.14 .08 .05
7. Natural Supports -.19* -.12 -.13 -.07 .13
8. Continuation of Services -.06 -.02 -.12 .12 -.02
9. Collaboration -.04 -.03 -.17* .13 -.11
10. Flexible Resources/Funds -.06 .04 -.14 .05 .20**
11. Outcome-Based Services -.16 -.04 -.18* .14* .21**
12. RF Total Fidelity -.09 -.20 -.08 -.15 .16
13. Caregiver Total Fidelity -.04 -.02 -.12 .04 -.15
14. Youth Total Fidelity -.03 -.01 -.05 .02 -.05
15. WFI Total Fidelity -.10 -.09 -.15 .05 .04
Correlations Among WFI Scores and Outcomes at Discharge
Note: *p<.05; **p<.01
Significant ResultsSignificant Results
Greater community-based services is related Greater community-based services is related to lower impairment on the CBCL and CAFAS to lower impairment on the CBCL and CAFAS at dischargeat dischargeGreater community-based services is related Greater community-based services is related to community living arrangement at dischargeto community living arrangement at discharge
Community-Based ServicesCommunity-Based Services
CBCL TotalCBCL Total -.19*-.19*
CBCL ExternalizingCBCL Externalizing -.25**-.25**
CBCL InternalizingCBCL Internalizing -.21*-.21*
CAFAS TotalCAFAS Total -.40**-.40**
Living in CommunityLiving in Community .21**.21**
Note: *p<.05; **p<.01
ResultsResults
Greater fidelity to the provision of Community Greater fidelity to the provision of Community Services/Supports in Wraparound is related to Services/Supports in Wraparound is related to positive outcomespositive outcomes
No other elements had as significant correlations No other elements had as significant correlations to outcomes as Community Services/Supportsto outcomes as Community Services/Supports
No significant relationship found between Total No significant relationship found between Total Fidelity Scores and outcomes at dischargeFidelity Scores and outcomes at discharge
Correlations were consistently low even when Correlations were consistently low even when significantsignificant
ImplicationsImplications
Implications for teams to support a Implications for teams to support a youth’s school attendance and youth’s school attendance and involvement in work, training, and other involvement in work, training, and other community activities (e.g., church, community activities (e.g., church, sports, art, etc.) as well as support sports, art, etc.) as well as support community-based living situations for community-based living situations for youth (as an alternative to residential or youth (as an alternative to residential or institutional care) in order to improve institutional care) in order to improve outcomes.outcomes.
Challenges of the WFIChallenges of the WFI
Direct feedback to teams was not provided Direct feedback to teams was not provided
The utility of the WFI in its relationship to The utility of the WFI in its relationship to outcomes was limited based on EMQ’s outcomes was limited based on EMQ’s analysis of the dataanalysis of the data
Time intensive and limited resources Time intensive and limited resources available to conduct 3 structured available to conduct 3 structured interviews per youthinterviews per youth
EMQ Practice Standards Initiative EMQ Practice Standards Initiative
Initiative to Improve PracticeInitiative to Improve Practice
Quality Improvement Project launched in Quality Improvement Project launched in January 2004 to implement practice standards January 2004 to implement practice standards and feedback tools across positions and feedback tools across positions
Family FacilitatorFamily Facilitator Family SpecialistFamily Specialist Family PartnerFamily Partner
Goal was to standardize job positions, provide Goal was to standardize job positions, provide role clarity, and improve consistency in job role clarity, and improve consistency in job performance and supervisionperformance and supervision
Example of EMQ Feedback Tool:Example of EMQ Feedback Tool:Family FacilitatorFamily Facilitator
Scoring:Scoring:Opportunity for Improvement (1)Opportunity for Improvement (1)Acceptable (2)Acceptable (2)Exemplary (3)Exemplary (3)Not Observed (*)Not Observed (*)
3 Dimensions:3 Dimensions:A.A. Families receive services that are consistent with Families receive services that are consistent with wraparound valueswraparound valuesB.B. Families receive services that are collaborative, Families receive services that are collaborative, integrated and adhere to best practicesintegrated and adhere to best practicesC.C. Children and Families served are safe and stable at Children and Families served are safe and stable at home and in the communityhome and in the community
Facilitator Field Feedback ToolFacilitator Field Feedback ToolA.A. Families receive services that are consistent with Families receive services that are consistent with
wraparound valueswraparound values1.1. Arrived on time and was prepared for the CFTArrived on time and was prepared for the CFT2.2. Started the CFT with “what’s working” to maintain strength based Started the CFT with “what’s working” to maintain strength based
focus and future orientationfocus and future orientation3.3. Facilitated the CFT to validate and incorporate ideas, requests, and Facilitated the CFT to validate and incorporate ideas, requests, and
concerns of all membersconcerns of all members4.4. Evidence that needs and strategies were reviewed within the CFT Evidence that needs and strategies were reviewed within the CFT
to assure the family voice and preference guide all activitiesto assure the family voice and preference guide all activities5.5. Evidence that plans were reviewed for progress on current Evidence that plans were reviewed for progress on current
measurable, observable goals and updated as needed to address measurable, observable goals and updated as needed to address new and changing needsnew and changing needs
6.6. Evidence that the next CFT and staff visits were scheduled Evidence that the next CFT and staff visits were scheduled according to family preferenceaccording to family preference
Facilitator Field Feedback ToolFacilitator Field Feedback ToolB.B. Families receive services that are collaborative, integrated and Families receive services that are collaborative, integrated and
adhere to best practicesadhere to best practices1.1. For children open six months or longer, evidence that the CFT was comprised of For children open six months or longer, evidence that the CFT was comprised of
at least 50% non-paid informal membersat least 50% non-paid informal members2.2. Facilitated the CFT to allow pertinent agenda items to be covered according to Facilitated the CFT to allow pertinent agenda items to be covered according to
family preference and time availabilityfamily preference and time availability3.3. Demonstrated engagement and rapport with all team members including system Demonstrated engagement and rapport with all team members including system
partnerspartners4.4. Therapy, medications and adjunctive services were evaluated relative to plan Therapy, medications and adjunctive services were evaluated relative to plan
goals as needed goals as needed 5.5. Specific tasks were assignedSpecific tasks were assigned
C.C. Children and Families served are safe and stable at home and in Children and Families served are safe and stable at home and in the communitythe community
1.1. Evidence that team has a plan B to back up plan AEvidence that team has a plan B to back up plan A2.2. If the CFT was for a child in out of home placement, this was addressed as a If the CFT was for a child in out of home placement, this was addressed as a
priority with specific plan, tasks, and target datepriority with specific plan, tasks, and target date3.3. Evidence that strategies and interventions developedEvidence that strategies and interventions developed emphasize the use of emphasize the use of
typical resources in the child’s communitytypical resources in the child’s community
EMQ Feedback ToolsEMQ Feedback Tools
Data analysis showed that the tools were not Data analysis showed that the tools were not measuring what was intended; therefore, tools measuring what was intended; therefore, tools were discardedwere discarded
Led to an examination of the national Led to an examination of the national standards for Wraparoundstandards for Wraparound
EMQ Sacramento decided to collect data on EMQ Sacramento decided to collect data on the Child and Family Team process using the the Child and Family Team process using the Wraparound Observation Form – Version 2.0 Wraparound Observation Form – Version 2.0 (WOF-2.0)(WOF-2.0)
Wraparound Observation Form – Wraparound Observation Form – Second Version (WOF-2)Second Version (WOF-2)
Measure to assess the implementation of the Wraparound Measure to assess the implementation of the Wraparound approach by direct observation of youth and family team approach by direct observation of youth and family team meetingsmeetings
WOF has demonstrated good inter-rater reliability WOF has demonstrated good inter-rater reliability (kappa=.886; Nordess & Epstein, 2003)(kappa=.886; Nordess & Epstein, 2003)
48 items that measures 8 characteristics: 48 items that measures 8 characteristics:
Community-BasedCommunity-Based Unconditional Care Unconditional Care IndividualizedIndividualized Measurable Outcomes Measurable Outcomes Family-DrivenFamily-Driven Management of Team Meeting Management of Team Meeting Interagency Collaboration Interagency Collaboration Care Coordinator Care Coordinator
Implementation of the WOFImplementation of the WOF
WOF data collection began July 2005WOF data collection began July 2005
Feedback provided in individual Feedback provided in individual supervision if completed by direct supervision if completed by direct supervisorsupervisor
Verbal feedback provided to facilitators Verbal feedback provided to facilitators after the child and family team meeting if after the child and family team meeting if completed by neutral observer. Copy of completed by neutral observer. Copy of tools also provided to supervisorstools also provided to supervisors
Wraparound Observation Form – Wraparound Observation Form – Second Version (WOF-2)Second Version (WOF-2)
Community-based resources (5 items)Community-based resources (5 items)
1. Information about resources / interventions in the area is offered to 1. Information about resources / interventions in the area is offered to the team. the team.
2. Plan of care includes at least one public and/or private community 2. Plan of care includes at least one public and/or private community service/resource.service/resource.
3. Plan of care includes at least one informal resource.3. Plan of care includes at least one informal resource.4. When residential placement is discussed, team chooses 4. When residential placement is discussed, team chooses
community placements for child(ren) rather than out-of-community community placements for child(ren) rather than out-of-community placements, whenever possible.placements, whenever possible.
5. Individuals (non-professionals) important to the family are present 5. Individuals (non-professionals) important to the family are present at the meeting.at the meeting.
Individualized services for the family (9 items)Individualized services for the family (9 items)
6. If an initial plan of care meeting, the parent is asked what 6. If an initial plan of care meeting, the parent is asked what treatments or interventions he/she felt worked/didn’t work prior to treatments or interventions he/she felt worked/didn’t work prior to WRAP.WRAP.
7. Facilitator advocates for services and resources for the family 7. Facilitator advocates for services and resources for the family (e.g., identifies and argues for necessary services). (e.g., identifies and argues for necessary services).
8. All services needed by family are included in plan (e.g., no 8. All services needed by family are included in plan (e.g., no needed services were not offered).needed services were not offered).
9. Barriers to services or resources / interventions are identified and 9. Barriers to services or resources / interventions are identified and solutions discussed.solutions discussed.
10. The steps needed to implement the plan of care are clearly 10. The steps needed to implement the plan of care are clearly specified by the team.specified by the team.
11. Strengths of family members are identified and discusses at the 11. Strengths of family members are identified and discusses at the meeting.meeting.
12. Plan of care that includes life domain(s) goals, objectives, and 12. Plan of care that includes life domain(s) goals, objectives, and resources/interventions is discussed (or written).resources/interventions is discussed (or written).
13. Plan of care goals, objectives, or interventions are based on 13. Plan of care goals, objectives, or interventions are based on family/child strengths.family/child strengths.
14. Safety plan/Crisis plan developed/reviewed.14. Safety plan/Crisis plan developed/reviewed.
Family-driven services (10 items)Family-driven services (10 items)
15. Convenient arrangements for family’s presence at meeting are 15. Convenient arrangements for family’s presence at meeting are made (e.g., location, time, transportation, day care made (e.g., location, time, transportation, day care arrangements).arrangements).
16. The parent/child is seated or invited to sit where he/she can be 16. The parent/child is seated or invited to sit where he/she can be included in the discussion.included in the discussion.
17. Family members are treated in a courteous fashion at all times.17. Family members are treated in a courteous fashion at all times.18. The family’s perspective is presented to professionals from other 18. The family’s perspective is presented to professionals from other
agencies. agencies. 19. The family is asked what goals they would like to work on.19. The family is asked what goals they would like to work on.20. The parent is asked about the types of services or 20. The parent is asked about the types of services or
resources/interventions he/she would prefer for his/her family.resources/interventions he/she would prefer for his/her family.21. Family members are involved in designing the plan of care.21. Family members are involved in designing the plan of care.22. In the plan of care, the family and team members are assigned 22. In the plan of care, the family and team members are assigned
(or asked) tasks and responsibilities that promote the family’s (or asked) tasks and responsibilities that promote the family’s independence (e.g., accessing resources on own, budgeting, independence (e.g., accessing resources on own, budgeting, maintaining housing).maintaining housing).
23. The team plans to keep the family intact or to reunite the family.23. The team plans to keep the family intact or to reunite the family.24. Family Members voice agreement/disagreement with plan of 24. Family Members voice agreement/disagreement with plan of
care.care.
Interagency collaboration (7 items)Interagency collaboration (7 items)
25. Staff from other agencies who care about or provide 25. Staff from other agencies who care about or provide resources/interventions to the family are present at the meeting.resources/interventions to the family are present at the meeting.
26. Staff from other facilities or agencies (if present) have an 26. Staff from other facilities or agencies (if present) have an opportunity to provide input opportunity to provide input
27. Informal supports (if present) have an opportunity to provide 27. Informal supports (if present) have an opportunity to provide input.input.
28. Problems that can develop in an interagency team (e.g., turf 28. Problems that can develop in an interagency team (e.g., turf problems, challenges to authority) are not evident or are resolved.problems, challenges to authority) are not evident or are resolved.
29. Staff from other agencies describe support resources 29. Staff from other agencies describe support resources interventions available in the community.interventions available in the community.
30. Statement(s) made by a staff member or an informal support 30. Statement(s) made by a staff member or an informal support indicate that contact/ communication with another team member indicate that contact/ communication with another team member occurred between meetings.occurred between meetings.
31. Availability of alternative funding sources is discussed before 31. Availability of alternative funding sources is discussed before flexible funds are committed.flexible funds are committed.
Unconditional Care (3 items)Unconditional Care (3 items)
32. Termination of Wrap services is discussed because of the 32. Termination of Wrap services is discussed because of the multiplicity or severity of the child’s/family’s behaviors /problems.multiplicity or severity of the child’s/family’s behaviors /problems.
33. Termination of other services (not Wrap) is discussed because of 33. Termination of other services (not Wrap) is discussed because of the multiplicity or severity of the child’s/family’s behaviors the multiplicity or severity of the child’s/family’s behaviors /problems./problems.
34. For severe behavior challenges (e.g., gangs, drugs), discussion 34. For severe behavior challenges (e.g., gangs, drugs), discussion focuses on safety plans/crisis plans (e.g., service and staff to be focuses on safety plans/crisis plans (e.g., service and staff to be provided) rather than termination.provided) rather than termination.
Measurable Outcomes (3 items)Measurable Outcomes (3 items)
35. The plan of care goals are discussed in objective, measurable 35. The plan of care goals are discussed in objective, measurable terms.terms.
36. The criteria for ending Wrap involvement are discussed.36. The criteria for ending Wrap involvement are discussed.37. Objective or verifiable information on child and parent functioning 37. Objective or verifiable information on child and parent functioning
is used as outcome data.is used as outcome data.
Management of team meeting (5 items)Management of team meeting (5 items)
38. Key participants are invited to the meeting (e.g., family members, 38. Key participants are invited to the meeting (e.g., family members, CPS worker, teacher, therapist, others identified by family).CPS worker, teacher, therapist, others identified by family).
39. Current information about the family (e.g., social history, 39. Current information about the family (e.g., social history, behavioral and emotional status) is gathered prior to the meeting behavioral and emotional status) is gathered prior to the meeting and shared at meeting (or beforehand).and shared at meeting (or beforehand).
40. All meeting participants introduce themselves (if applicable) or are 40. All meeting participants introduce themselves (if applicable) or are introduced.introduced.
41. The family is informed that they may be observed during the 41. The family is informed that they may be observed during the meeting.meeting.
42. Plan of care is agreed on by all present at the meeting.42. Plan of care is agreed on by all present at the meeting.
Facilitator (6 items)Facilitator (6 items)
43. Facilitator makes the agenda of meeting clear to participants.43. Facilitator makes the agenda of meeting clear to participants.44. Facilitator reviews goals, objectives, interventions and/or 44. Facilitator reviews goals, objectives, interventions and/or
progress of plan of care.progress of plan of care.45. Facilitator directs (or reflects) team to discuss family/child 45. Facilitator directs (or reflects) team to discuss family/child
strengths.strengths.46. Facilitator directs (or reflects) team to revise/update plan of care.46. Facilitator directs (or reflects) team to revise/update plan of care.47. Facilitator summarizes content of the meeting at the conclusion of 47. Facilitator summarizes content of the meeting at the conclusion of
the meeting.the meeting.48. Facilitator sets next meeting date/time.48. Facilitator sets next meeting date/time.
EMQ WOF-2 Results:EMQ WOF-2 Results:Total Element Scores (N=30)Total Element Scores (N=30)
7795
63
88929077
57
235
37128
102343
0
20
40
60
80
100
Comm
unity
Indiv
idua
lized
Family
-Driv
en
Collabo
ratio
n
Uncond
itiona
l Car
e
Out
com
es
Man
agem
ent
Facilit
ator
Undesired
Desired
Results of WOFResults of WOF
EMQ Sacramento shows higher scores on EMQ Sacramento shows higher scores on elements of Family-driven, Collaboration, elements of Family-driven, Collaboration, Unconditional Care & Management of CFT Unconditional Care & Management of CFT MeetingsMeetingsEMQ Sacramento continues to be challenged EMQ Sacramento continues to be challenged by lower scores on Community-based by lower scores on Community-based Resources, inclusive of natural supportsResources, inclusive of natural supportsIn contrast to the results of the WFI for In contrast to the results of the WFI for Outcomes (high scores), the WOF showed low Outcomes (high scores), the WOF showed low scores on this element scores on this element Slight increase in Total WFI scores since the Slight increase in Total WFI scores since the implementation of the WOFimplementation of the WOF
Back to the WFIBack to the WFI
Could the use of the WOF impact Could the use of the WOF impact WFI scores?WFI scores?
A look at WFI scores across timeA look at WFI scores across time
WFI Total Scores Across Time WFI Total Scores Across Time
75 7570
76
0
20
40
60
80
100
1/04-6/04 7/04-12/04 1/05-6/05 7/05-12/05
WOF Implementation Begins 7/05
WFI Respondent Scores WFI Respondent Scores Across TimeAcross Time
81 79 82 84
7264
74 7771 68
72 71
0
10
20
30
40
50
60
70
80
90
Facilitator Caregiver Youth
1/04-6/04
7/04-12/04
1/05-6/05
7/05-12/05
High Scores on Voice and Choice, Cultural Competence, Individualized Services, Strengths, Outcome-Based Services Low Scores on Natural Supports, Community-Based Services, and CFT Element
Challenges of the WOFChallenges of the WOF
Limited resources in staff to become neutral Limited resources in staff to become neutral observersobserversManagers wanted to use the WOF as a Managers wanted to use the WOF as a supervision toolsupervision toolWillingness and comfort level of staff to Willingness and comfort level of staff to participate in direct observationparticipate in direct observationStaff perception of fear of families being Staff perception of fear of families being judged by observersjudged by observersSponsorship by management Sponsorship by management Competing priorities in data collectionCompeting priorities in data collection
Next StepsNext Steps
Supervisors will continue to utilize the Supervisors will continue to utilize the WOF to provide feedbackWOF to provide feedbackOperational realities limit the use of Operational realities limit the use of neutral observersneutral observersEMQ will no longer collect the WFI due EMQ will no longer collect the WFI due to limitations in resources, the utility of to limitations in resources, the utility of the tool for feedback, and its relationship the tool for feedback, and its relationship to outcomesto outcomesContinue to develop practice standardsContinue to develop practice standards
ReferencesReferences
Achenbach, T. M., & Rescorla, L.A. (2001). Achenbach, T. M., & Rescorla, L.A. (2001). Manual for the ASEBA Manual for the ASEBA school-age forms & profiles. school-age forms & profiles. Burlington, VT: University of Vermont, Burlington, VT: University of Vermont, Research Center for Children, Youth, & Families. Research Center for Children, Youth, & Families.
Epstein, M.H., Nordess, P.D., & Hertzog, M. (2002). Epstein, M.H., Nordess, P.D., & Hertzog, M. (2002). Wraparound Wraparound Observation Form-Second VersionObservation Form-Second Version..
Epstein, M.H., Nordess, P.D., et al. (2003). Epstein, M.H., Nordess, P.D., et al. (2003). Assessing the Wraparound Assessing the Wraparound Process During Family Planning Meetings.Process During Family Planning Meetings.
Hodges, K. (2000). Child and Adolescent Functional Assessment Scale Hodges, K. (2000). Child and Adolescent Functional Assessment Scale (CAFAS). (CAFAS). CAFAS Self-Training Manual and Blank Scoring Forms CAFAS Self-Training Manual and Blank Scoring Forms (2003)(2003). Ann Arbor, MI . Ann Arbor, MI
Nordness, P.D., & Epstein, M.H.(2003). Nordness, P.D., & Epstein, M.H.(2003). Reliability of the Wraparound Reliability of the Wraparound Observation Form Second Version.Observation Form Second Version.
Suter, J., Force, M., Bruns, E., Leverentz-Brady, K., & Burchard, J. Suter, J., Force, M., Bruns, E., Leverentz-Brady, K., & Burchard, J. (2002). (2002). Manual for training, administration, and scoring of the WFI 3.0.Manual for training, administration, and scoring of the WFI 3.0. Burlington: University of Vermont. Burlington: University of Vermont.
Contact InformationContact Information
Enrica BertoldoEnrica Bertoldo ebertoldo@emq.orgebertoldo@emq.org Quality Support Manager, SacramentoQuality Support Manager, Sacramento
Mary Ann WongMary Ann Wongmwong@emq.orgmwong@emq.org
Research Specialist, SacramentoResearch Specialist, Sacramento
Wraparound Supervisor Wraparound Supervisor Adherence Measure (W-SAM): Adherence Measure (W-SAM):
A Pilot A Pilot
Eleanor Castillo, Ph.D.Eleanor Castillo, Ph.D.
Veronica PadillaVeronica Padilla
RationaleRationale
Supervision identified as a critical Supervision identified as a critical aspect of Wraparoundaspect of Wraparound
1.1. No tool to date to measure this aspect of No tool to date to measure this aspect of WraparoundWraparound
Concept adapted from Multisystemic Concept adapted from Multisystemic Treatment’s (MST) Supervisor Treatment’s (MST) Supervisor Adherence Measure (Adherence Measure (SAMSAM))
RationaleRationale
WFI-3.0 measures adherence to WFI-3.0 measures adherence to Wraparound principles from the Wraparound principles from the Youth, Caregiver, and Resource Youth, Caregiver, and Resource Facilitator perspectiveFacilitator perspective
WOF-2.0 measures the Child and WOF-2.0 measures the Child and Family Team aspect of WraparoundFamily Team aspect of Wraparound
DescriptionDescription
Facilitator rates the Clinical Program Facilitator rates the Clinical Program Manager (CPM)Manager (CPM)
40 Items40 Items
Likert Scale (1 = Never; 5 = Almost Likert Scale (1 = Never; 5 = Almost Always)Always)
Administered quarterlyAdministered quarterly
Highly reliable (alpha = .84-.98)Highly reliable (alpha = .84-.98)
Process ImplementationProcess Implementation
Sponsorship from the DirectorsSponsorship from the Directors
Program quality improvement versus Program quality improvement versus “mandate”“mandate”
Immediate feedbackImmediate feedback
Quality ManagementQuality Management
TrainingTraining
EvaluationEvaluation1.1. Overall Program ResultsOverall Program Results
2.2. Individual CPMIndividual CPM
Performance AppraisalPerformance Appraisal1.1. 360360
2.2. AnnualAnnual
Quality ManagementQuality Management
3-Tiered: Agency-wide, Program, and 3-Tiered: Agency-wide, Program, and IndividualIndividual
1.1. Training and CoachingTraining and Coaching2.2. EvaluationEvaluation
Performance AppraisalPerformance Appraisal1.1. Quarterly 360 feedbackQuarterly 360 feedback2.2. AnnualAnnual
Correlation between supervisor focus on Correlation between supervisor focus on the Wraparound process and youth the Wraparound process and youth outcomesoutcomes
Quality ManagementQuality Management
Trend analysesTrend analyses1.1. Fidelity consistency over timeFidelity consistency over time
2.2. What factors affect fidelityWhat factors affect fidelityLearning curve with new manager or Learning curve with new manager or facilitatorsfacilitators
Next StepsNext Steps
Continue pilotContinue pilot
Conduct further analyses on the Conduct further analyses on the psychometric properties of the psychometric properties of the scalescale
1.1. Factor analysesFactor analyses
2.2. Items reductionItems reduction
Contact Information:Contact Information:
Eleanor Castillo, Ph.D.Eleanor Castillo, Ph.D.
Director of Outcomes & Quality Director of Outcomes & Quality AssuranceAssurance
Email: ecastillo@emq.orgEmail: ecastillo@emq.org