Update on Family Outcomes Work Don Bailey and Melissa Raspa ECO Advisory Board Meeting March 7,...

Update on Family Outcomes Work

Don Bailey and Melissa Raspa ECO Advisory Board Meeting

March 7, 2012

Goals for Today

• Provide update on family outcomes APR data• Introduce ECO family outcomes and

experiences measurement system self-assessment tool

• Discuss future of family outcomes

Early Childhood Outcomes Center

Part CMeasurement Approaches

• Tools– NCSEAM survey (n = 25)– ECO Family Outcomes Survey (n = 24)

• FOS-R (n = 3)

– State-developed survey (n = 6)• Methodology

– Census (n = 44)– Sampling (n = 11)

3Early Childhood Outcomes Center

Part CPopulation Surveyed

Timing All families Families > 6 months

Total

Point in time 25 12 37

At exit 3 6 9

At IFSP meeting 3 2 5

IFSP or exit 2 1 3

Total 33 21 54


Part CResponse Rates

• Overall response rate = 36%• Tool

– ECO: 41%– NCSEAM: 31%– State: 31%

• Methodology:– Census: 37%– Sample: 34%

Part CResponse Rates

• Distribution– In person: 46%– Multiple methods: 37%– Mail: 27%

• Return– In-person: 75%– Multiple methods: 36%– Mail: 27%


Part C Representativeness

Criteria States (N) States (%)

Race/ethnicity 41 75

Geography (district, county) 17 31

Gender 14 25

Child’s age (at referral, at survey) 13 24

Disability/eligibility category 6 11

Length of time in services 6 11

Income 3 5


Part CAPR Trend Data

4A Know rights

4BCommunicate

needs

4CDevelop and learn

0102030405060708090

100

80 8187

81 8388

84 8590

86 8690

FFY 06-07FFY 07-08FFY 08-09FFY 09-10


Part CTrend Data – Indicator 4A

FFY 06-07 FFY 07-08 FFY 08-09 FFY 09-100

5

10

15

20

25

30

> 50%50-59%60-69%70-79%80-89%90-100%


Part CTrend Data – Indicator 4B

FFY 06-07 FFY 07-08 FFY 08-09 FFY 09-100

5

10

15

20

25

30

35

> 50%50-59%60-69%70-79%80-89%90-100%


Part CTrend Data – Indicator 4C

FFY 06-07 FFY 07-08 FFY 08-09 FFY 09-100

5

10

15

20

25

30

50-59%60-69%70-79%80-89%90-100%


Part CAPR by Survey Tool

4AKnow rights

4BCommunicate

needs

4C Develop and learn

0102030405060708090

10088 91 91

82 808989 89 91

ECONCSEAMState


Part BMeasurement Approaches

• Tools– NCSEAM (n=43)– Adapted NCSEAM or ECO (n=8)– State (n=8)

• Methodology– Census (n=21)– Sampling (n=35)


Part BResponse Rates

• Overall response rate = 24%• Distribution

– Multiple methods: 30%– Mail: 17%

• Return– In-person: 75%– Multiple methods: 36%– Mail: 27


Part BAPR Trend Data


Next Steps

• APR analyses for FFY 2010-2011• Examine data quality

– APR by response rates• Develop template for states to share family

outcomes data• Provide TA to states on as-needed basis


FAMILY OUTCOMES AND EXPERIENCES MEASUREMENT SYSTEM

SELF ASSESSMENT TOOL

Early Childhood Outcomes Center 17

Purpose

• Provides a common language and organizing structure for discussing family outcomes and experiences

• Assist states in developing a measurement system that captures various aspects of family outcomes and experiences, such as– benefits that families receive from program participation– family satisfaction with services– perceived helpfulness of the services– family involvement with the service delivery system


Framework and Self-Assessment

• FRAMEWORK– Set of components and quality indicators– Provides the structure for the self-assessment

• SELF-ASSESSMENT – Scale that provides criteria for levels of

implementation within each quality indicator– Rating assigned based on level of

implementation within each indicator 19


Partner States

• Connecticut• Illinois• Minnesota• Texas


Framework

• Components– 7 key areas of a measurement system

• Quality indicators– 16 specific items that further describe each of

the components• Elements

– Various examples of what constitute quality at the indicator level


22

Purpose

Data Collection and Transmission

Analysis

Reporting

Using Data

Evaluation

Cross-System Coordination

Components Quality Indicators

Purpose

1. State has articulated purpose(s) of COMS.

Data Collection and Transmission

2. Data collection procedures are carried out efficiently and effectively.

3. Providers, supervisors, and others involved in data collection have the required knowledge, skills, and commitment.

4. State's method for entering, transmitting, and storing data is effective and efficient.

Analysis

5. State identifies accountability and program improvement questions related to child outcomes.

6. Local programs identify accountability and program improvement questions related to child outcomes.

7. State agency analyzes data in a timely manner.

8. Local programs analyze data in a timely manner.

9. State agency ensures completeness and accuracy of data.

Reporting

10. State agency interprets, reports, and communicates information related to child outcomes.

11. Local programs interpret, report, and communicate information related to child outcomes.

Using Data

12. State agency makes regular use of information on child outcomes to improve programs.

13. Local programs makes regular use of information on child outcomes to improve programs.

Evaluation14. State evaluates its COMS regularly.

Cross-system Coordination

15. Part C and 619 coordinate child outcomes measurement.

16. Child outcomes measurement is integrated across early childhood (EC) programs statewide.

17. Child outcomes measurement is aligned with state’s early learning guidelines/standards.

18. State has a longitudinal data system to link child outcomes data from EC program participation to K–12 data.

Elements

a. State has…b. State has…c. State agency..d. Representative..e. State agency…f. State ……g. State provides…h. State has..

Self-Assessment

• Provides guidance to states on what constitutes a high quality family experiences and outcomes measurement system.

• Assists states in setting priorities for improving their measurement system

• Provides information to assist states in advocating for resources for systems development

23

How Does it Work?

• Each quality indicator has multiple elements• Evidence for the extent of implementation for each

element is provided• Each element is rated as

• NY = Not Yet or Don’t know• IP = In Process• IF = Fully Implemented• NA = Not applicable (only where applicable)

• The quality indicator is given a rating based on the ratings of the elements


Self-assessment Scoring

25

Implementation of ElementsQuality

Indicator Score

All elements are fully implemented 7

Nearly all elements are fully implemented and the rest are in process 6

Most of the elements are fully implemented and the rest are in process. 5

At least one element is fully implemented and the rest are in process 4

All of the elements are in process 3

Some of the elements are in process 2

None of the elements are yet in process 1

Challenges with Family Framework and Self-assessment

• Going beyond the APR: Family outcomes and experiences

• Balancing expectations: Reality vs. ideal • Scoring system

– Difficulty in reaching 7– Not applicable items


Next Steps

• Finalize framework and self-assessment tool and post on ECO web site

• National conference call to introduce to broader audience

• Provide TA to states on as-needed basis


WHAT IS THE FUTURE OF FAMILY OUTCOMES?

Context

• Family-centered (FC) practices are endorsed by many professional organizations as central to high-quality practice– Institute of Medicine– American Academy of Pediatrics– American Hospital Association– American Speech-Language Hearing Association– Division for Early Childhood

We know….

• Meta-analyses of research provides strong evidence that FC practices lead to many benefits for families and children

Dunst, Trivette, & Hamby (2007)

• Meta-analysis of 47 studies of family-centered practices in a variety of settings

• Higher use of FC practices was associated with:– Satisfaction with services*– Self-efficacy beliefs*– Parenting behaviors– Personal and/or family well-being– Social support*– Child behavior and functioning

*Strongest associations

Pitrowski, Talavera, & Mayer (2009)

• Meta-analysis of 13 articles evaluating Health Steps, a pediatric care model based on– strength-based primary prevention– the assumption that child well-being and family-centered care are

necessarily linked– supporting the role of parents– positive, caring, and enduring relations with parents.

• Parents in HS were– More satisfied with services– Exhibited more desirable parenting practices– More satisfied in their role as parents– Interacted more sensitively and appropriately with child

Broom & Enriquez (2009)

• Reviewed 9 randomized trials of family-centered interventions in treating children with Type 1 diabetes

• Family-centered practices were associated with– Improved health outcomes for children– Reduced diabetes-related conflict in families– Improved family relations

Two studies using the National Survey of Children with Special Health Care Needs

• When families perceived a strong partnership between the child’s family and the child’s service provider– children’s health outcomes were improved– children received more services– families were more satisfied with services

Also

• Our NEILS analyses and our ECO papers show that family-centered practices are related to family outcomes

Epley, Summers & Turnbull (2011)

But research also shows that FC practices are not widely implemented

• Medical Expenditure Panel Survey and National Survey of CSHCN children found that only about 2/3 of families reported getting family-centered care

• A hospital survey found that only about half used FC rounds in pediatric in-patient settings

• Minority families generally report fewer FC services and less satisfaction

Dunst (2012) argues that EI has not evolved in tandem with what we know

So, if…..

• Professional organizations widely endorse FC practices, and…

• Research shows that FC practices make a difference…

• What should we do about the future of FC practices and family outcomes in early intervention and preschool programs?

Use of Family-Centered Practices

• We don’t know at a state or national level the extent to which programs are actually using family-centered principles and practices

• We don’t know what factors predict or are associated with variability in implementation of F-C practices

• NEEDED: A comprehensive description of the status of F-C practices in EI and preschool programs, and a clear understanding of why some programs are more family-centered than others

Family-centered principles, when integrated across all program activities, ought to lead to

family outcomes.

Referral and intake Determining eligibility Child assessment Family assessment Team meetings and

decision-making Service provision and

coordination Parenting practices Community support

Understand child’s abilities and special needs

Know rights and advocate effectively

Help child develop and learn

Have support systems Access the community

Family-Centered Principles

Program Activities Family Outcomes Shared philosophy Families as partners Focus on strengths Family choice of goals

and services Collaboration and

coordination of service Effective communication Flexibility Community-based

What is it about F-C practices that leads to better outcomes?

• With the exception of the NEILS study and other smaller studies, we don’t know the full range of outcomes that F-C practices can affect

• The pediatric literature has far more studies of this than EI

• NEEDED: Sophisticated large-scale studies modeling how and which F-C practices influence a variety of outcomes

The desirability & applicability of F-C practices & family outcomes in preschool

• Nothing is known about F-C practices in preschool programs for children w/disabilities

• The ECO family outcomes have never been accepted or seriously discussed for preschool

• We have some evidence that family outcomes are less likely to be achieved with older children

• NEEDED: Stakeholder opinions and consensus regarding the desirability of FC practices & appropriate family outcomes in preschool

F-C practices and child outcomes

• Is helping families a good outcome in and of itself, OR must we show that this results in benefits for children?

• We have danced around this topic for 20 years, with no resolution

• NEEDED: Either we need consensus on the argument that family benefit is a worthwhile outcome OR we need to show that F-C practices and family outcomes are important to maximize child benefit

Potential Year 5 Family Activities

• Help states use framework to improve systems for family outcomes assessment

• Targeted TA to states with low family helpfulness ratings

• Collect, compile and analyze data from states that have FOS data that can be linked with child outcomes data

• Explore with one or two states or programs use of FOS in preschool programs

Update on Family Outcomes Work Don Bailey and Melissa Raspa ECO Advisory Board Meeting March 7,...

Documents

Transcript of Update on Family Outcomes Work Don Bailey and Melissa Raspa ECO Advisory Board Meeting March 7,...