1

Early Childhood and Accountability

OSEP’s Project Director’s Meeting

August 2006

2

Jennifer Tschantz, OSEP Lou Danielson, OSEP Kathleen Hebbeler, SRI International Corry Robinson, University of CO Mary McLean, University of WI-Milwaukee Beth Rous, University of Kentucky Pat Trohanis, University of NC, NECTAC

3

Objectives

Provide background to OSEP’s early childhood outcomes work

Hear diverse perspectives on critical issues related to early childhood assessment and accountability

Facilitate a dialogue on these critical issues

4

Why the focus on early childhood outcomes?

Improve results for young children with

disabilities and their families

Meeting a need in the field

Development of outcomes for general early

childhood programs

Address GPRA, PART, and IDEA 2004

5

Approach

Short-term: obtain data from the States regarding child and family outcome GPRA indicators used to assess program performance at the Federal Level

Long-term: support the development of State early childhood outcome data collection used for program improvement by States, local programs and service providers

On-going stakeholder involvement

6

OSEP’s Key Investments

Early Childhood Outcomes (ECO) Center – Funded Fall 2003– Provides leadership and technical guidance

General Supervision Enhancement Grants (GSEGs)– 2004 = 18 awards focused on ec outcomes– 2006 = 9 award focused on ec outcomes

National Early Childhood Technical Assistance Center (NECTAC)

– Provision of TA and 2 national conferences

7

State Performance Plan (SPP) and Annual Performance Report(APR)

Each state required to submit SPP in December 2005 and an APR beginning Feb. 2007

Part C SPP has 14 indicators Part B SPP (includes preschool) has 20

indicators Early childhood outcomes are part of the

SPP/APR

8

Critical Events

Spring 2005 – Public input on what should be collected with regard to child and family outcomes

Summer 2005 – OSEP released the reporting requirements

December 2005 – States submitted their plans for outcome data collection in their State Performance Plan

Spring 2006 – States begin collecting data February 2007 – Status and entry data due February 2008 – first progress data due

9

OSEP Reporting Requirements: Part C and Preschool Child Outcomes

Percent of children who demonstrate improved:

Positive social emotional skills (including positive social relationships)

Acquisition and use of knowledge and skills (including early language/communication [and early literacy])

Use of appropriate behaviors to meet their needs

10

Reporting Categories

a. % of children who reach or maintain functioning at a level comparable to same-age peers

b. % of children who improve functioning but do not achieve functioning comparable to same age peers

c. % of children who do not improve functioning

3 outcomes x 3 percentages = 9 numbers

11

Proposed Change to Reporting Categories

CURRENT

a. % of children who reach or maintain functioning at a level comparable to same-aged peers

b. % of children who improve functioning (not in a)

c. % of children who did not improve functioning

PROPOSED CHANGE

a. % of children who maintain functioning at a level comparable to same-aged peers

b. % of children who improved functioning to reach a level comparable to same-aged peers

c. % of children who improve functioning but did not reach a level comparable to same-aged peers

d. % of children who did not improve functioning

12

Additional information about child indicators

Child must be in program at least 6 months Outcomes are broad and functional All outcome areas apply to all children regardless of

area receiving services These progress indicators compare entry to exit data

for each child, requiring a minimum of two data points Not mandating assessment tools, States have flexibility

to choose own tool or tools.

13

Analysis of State Performance Plans

ECO summarized State Plans for collecting child outcome data

Based on SPPs submitted in December 2005 Many States doing more than what they sent in

and many state plans have evolved since December

14

SPP Analysis:Part C Outcomes Data Sources

Data Source # %

Formal assessment instruments

45 80%

Parent report 25 45%

Observation 14 25%

Clinical opinion 10 18%

IFSP goals & objectives 6 11%

Record review 4 7%

Not reported 6 11%

15

SPP Analysis:Preschool Outcomes Data Sources

Data Source # %

Formal assessment instruments

45 80%

Observation 12 21%

Parent report 11 19%

Teacher/provider report 8 14%

IEP goals & objectives 1 2%

Clinical opinion 1 2%

Not reported 10 17%

16

Commonly Reported Assessment Instruments: Part C

Of 28 states who listed specific assessment instruments:– HELP- 15 states– BDI/BDI-2- 13 states– AEPS- 11 states– Creative Curriculum- 6 states– ELAP- 6 states

Not yet determined- 23 states

17

Commonly Reported Assessment Instruments: Preschool

Of 31 states who listed specific assessment instruments:

– BDI/BDI-2- 9 states– Creative Curriculum- 8 states– Brigance- 7 states– High Scope COR- 6 states– AEPS- 5 states– State developed assessments- 7 states

Not yet determined or not reported- 27 states

18

State approaches to assessments

One assessment selected by state List of assessments developed by state;

programs pick Programs can use whatever they have

been using

19

ECO Child Outcomes Summary Form (COSF)

States need to be able to aggregate data across tools –need a “common metric” to which data from different assessments can be converted

COSF– Provides “scores” directly on each of the 3 outcomes– Allows different assessment data to be transformed to a

common metric (1 to 7 scale)– Allows for increments of significant progress to be tracked

over time– Allows for any degree of progress to be tracked over time

21

Use of the form

ECO envisioned the form as the final step in a team process where the child’s functioning was discussed and consensus was reached

Alternatives: One professional completing, team members completing the form individually

“Behind the Scenes” Alternative: Converting online assessment data directly to the 7-point scale

22

Current Activities

States currently focused on training Development of guidance materials on the

COSF Reliability studies on the COSF Encouraging states to start thinking about how

they will use these data

23

Summary

All States are required to submit data on 3 outcomes for all children participating in Part C and Part B Preschool programs.

States implementing a variety of approaches to produce these data

Information from assessments are critical– Single assessment statewide– Small set of approved assessments– Any assessment in use

24

Summary

States are making critical decisions related to assessment right now.

25

Questions for the Panel

26

Assessment Tools / Use of Data

Is it professionally acceptable (and will the data be valid) to use instruments designed for screening and eligibility, progress monitoring, or other purposes for accountability?

27

Assessment Tools / Use of Data

Can and should the same assessment data be used for accountability, program improvement and individual progress monitoring purposes?

Can data providers are collecting for other purposes (i.e., progress monitoring) be credible for accountability?

28

Training

What is your sense of the current level of expertise of the field with regard to early childhood assessment?

What has to be done to increase the overall level of knowledge of providers?

29

Families

What is the role of the family in early childhood assessment for accountability purposes?

How do we include families in a meaningful way?

30

Assessment Practices

How can we ensure that accountability has a positive impact on assessment practices in early childhood and promotes authentic assessment?