Using Perkins Data

To Improve CTE

Sharon Enright, Ph.D.Ohio Department of Education

Office of Career-Technical Education1

NCLA ConferenceOctober 1, 2010

Greetings from OhioThe Buckeye State!

2

Four areas of CTE Accountability:

• Secondary (ODE)• Postsecondary (OBR)• Adult (ODE and OBR)• Tech Prep (ODE and OBR)

3

Ohio’s CTE Accountability System

Negotiated State Targets with OVAE• Obtained input from Locals

prior to negotiating State targets.

• FY08 – Secondary only (1S1, 1S2, 4S1)

• FY09 – Secondary, Postsecondary, Adult

• FY10 and FY11 – Secondary, Postsecondary, Adult

4

Renegotiated State Targets with OVAE• FY09 – Renegotiated some

Postsecondary and Adult targets• FY10 and FY11 – Renegotiated

Placement targets (5S1, 4P1, 4A1)–State and county unemployment

rates continue to go up

5

Statewide Longitudinal Data System (SLDS)• Perkins data is submitted by Ohio locals in SLDS

• Secondary – Education Information Management System (EMIS), PK-12

• Postsecondary – Higher Education Information Data System (HEI), 13-20+

• Ohio is working on an integration plan for PK-20+ SLDS

6

Local and State Perkins Data • LOCALS – Submit course, student-

level and other data to State via SLDS.

• STATE – Aggregates LOCAL data:–Calculates LOCAL performance results

–• Reports some local Secondary results to

USDOE via EDFacts.• Publishes local performance reports.

–Calculates STATE performance results –• Reports state results to USDOE/OVAE via

CAR & EDFacts.• Publishes state performance report.

.7

FY2009 State Performance

No State Improvement Plans Needed

• Secondary – Exceeded State targets for all indicators

• Postsecondary – Exceeded State targets for all indicators

• Adult –Did not meet State target for Placement (4A1), but exceeded 90%

8

FY2009 Statewide Performance Reports• Goal – Create user-friendly State

reports• Publish on the Web

–Ohio Department of Education (ODE)

–Ohio Board of Regents (OBR)• Review reports with State and

local CTE staff 9

Ohio is on a journey to improve CTE student

performance…

And we’re not on this journey alone.

10

Ohio’s strategies to improve Statewide

performance results should

also help improve Local performance

results.11

12

CTE Performance, Data and Accountability Web site

13

Secondary CTE Performance Reports

Review of Secondary State Data

14

State Staff Review – Meeting #1

Reviewed FY09 Enrollment Data• State Performance Report• Also pulled additional data to review

15

16

Career-Technical Education (CTE) and Tech Prep Student Counts (unduplicated)In each of the following categories, unduplicated means a student is counted only one time in a category, but a student may be counted in more than one category. A student may be counted only as a CTE Participant; or as a CTE Participant and a CTE Concentrator; or as a CTE Participant, a CTE Concentrator and a CTE Concentrator Who Left Secondary Education.

2007-2008

2008-2009

Total CTE Participants*A secondary student who has earned credit in one (1) or more courses in any career and technical education (CTE) workforce development program area.

137,961 127,085

Total CTE ConcentratorsA secondary student who has completed a minimum of 50% of the high school credits allowed for a single career and technical education (CTE) workforce development program (e.g., health sciences or marketing), and has enrolled for additional credit at the secondary level.

32,272 32,747

Total CTE Concentrators Who Left Secondary EducationCTE Concentrators who have graduated or withdrawn from high school.

26,230 27,098

Total Secondary Tech Prep ParticipantsA secondary student who has earned credit in one (1) or more courses in any career and technical education (CTE) Tech Prep workforce development program area.

Not Calculated 22,989

Total Secondary Tech Prep ConcentratorsA secondary student who has completed a minimum of 50% of the high school credits allowed for a single career and technical education (CTE) Tech Prep program (e.g., health sciences or marketing), and has enrolled for additional credit at the secondary level.

8,908 9,870

Total Secondary Tech Prep Concentrators Who Left Secondary EducationSecondary Tech Prep Concentrators who have graduated or withdrawn from high school.

7,710 8,530

* CTE Participant is calculated differently in 2008-2009 than in previous years.

FY09 CTE Enrollment Data – Pg. 2

17

2008-2009 Statewide Student Subgroup Counts (unduplicated)Statewide student counts are disaggregated by student subgroups. These data support statewide accountability and continuous improvement initiatives.

Student Counts by Student Subgroups

CTE Participants

CTE Concentrator

s

CTE Concentrator

s Who Left Secondary Education

Secondary Tech Prep

Participants

Secondary Tech Prep Students (Concen.)

Secondary Tech Prep Students (Concen.) Who Left

Secondary Education

All Students 127,085 32,747 27,098 22,989 9,870 8,530

Male 72,571 17,810 14,512 12,971 5,458 4,669

Female 54,514 14,937 12,586 10,018 4,412 3,861

American Indian or Alaskan Native 189 56 43 29 19 18

Asian or Pacific Islander 996 258 201 221 105 88

Black (not Hispanic) 19,574 4,137 3,048 2,617 1,092 880

Hispanic 2,991 649 458 439 189 154

White 100,741 27,158 22,966 19,303 8,314 7,269

Unknown – Multi-Racial 2,594 489 382 380 151 121

Individuals with Disabilities 22,610 7,085 5,418 2,958 1,246 1,044

Economically Disadvantaged 43,822 10,216 8,010 6,547 2,609 2,129

Limited English Proficient 1,224 180 98 109 50 34

Single Parents 1,212 615 523 266 129 109

Migrant Status 26 7 3 2 1 1

CTE Nontraditional Enrollees 23,950 6,728 5,541 5,008 2,071 1,805

FY09 CTE Enrollment Data, Disaggregated – Pg. 3

State Staff Review – Meeting #1Reviewed FY09 Enrollment Data• CTE Participant Count – Lower in FY09

than FY08 (New calculation method in FY09)

• CTE Concentrators – Suspect under-reporting

• CTE Student Sub-groups:– Students with Disabilities – CTE has higher

enrollment rate than general school population, grades 9-12

– Single Parents – Need to improve identification and reporting

18

State Staff Review – Meeting #2

• Did Ohio meet all state targets?• Did Ohio meet 90% of state

targets?• How did student sub-groups

perform?• How did the LOCALS perform?

19

Reviewed FY09 Performance Data

20

FY09 Performance Data– Pg. 2Did We Meet All Targets?Did We Meet 90% of All Targets?

Secondary CTE Workforce Development Perkins Core Indicators of PerformanceDefinitions for the indicators of performance are in the Ohio Perkins Five-Year State Plan for Career-Technical Education. The Ohio Department of Education, Office of Career-Technical Education, reached agreement with the U. S. Department of Education on state performance targets for each Perkins core indicator of performance.

Perkins Core Indicators of Performance2007-2008

State Performance

Rate

2008-2009State

Performance Rate

2008-2009State

Performance Target

Met 90% of 2008-2009

State Performance

Target

Academic Attainment—Reading/Language Arts (1S1) 93.89% 93.31% 87.00% YesAcademic Attainment—Mathematics2 (1S2) 92.09% 91.68% 83.00% YesTechnical Skill Attainment, (2S1) n/a 63.63% 61.00% YesSecondary School Completion3,4 (3S1) n/a 95.40% 93.00% Yes

Regular or Honors Diploma n/a 95.23%General Educational Development (GED) Diploma5 n/a 0.17%

Student Graduation Rates4 (4S1) 93.38% 94.63% 73.60% YesSecondary Placement3,4 (5S1) n/a 88.49% 87.00% Yes

Postsecondary Education or Advanced Training n/a 53.00%Employment6 n/a 58.22%Military6 n/a 3.93%Apprenticeship6 n/a 3.25%Placement Status Known7 n/a 90.07%

Nontraditional Participation2,3 (6S1) n/a 26.67% 20.00% YesNontraditional Completion2,3 (6S2) n/a 22.81% 17.00% Yes

[1] If 90% of the state performance target is not met for any Perkins core indicator of performance, Ohio must submit an improvement plan to the U.S. Department of Education.[2] 2008-2009 results are based on CTE Concentrators who left secondary education in 2008-2009.[3] Performance results were not calculated for these indicators and there were no performance targets in 2007-2008.[4] 2008-2009 performance results are based on CTE Concentrators who left secondary education in 2007-2008 (including summer graduates).[5] Secondary School Completion disaggregated indicator data – a student is included in only one category of disaggregated data.[6] Secondary Placement disaggregated indicator data – a student may be included in more than one category of disaggregated data.7 Percent of CTE Concentrators who left secondary education in 2007-2008 whose placement status was known. Not an indicator of performance.

21

FY09 CTE Performance Data, Disaggregated – Pg. 32008-2009 Statewide CTE Performance Rates by Student SubgroupStatewide performance rates are disaggregated by student subgroups on each Perkins core indicator of performance. These data support statewide accountability and continuous improvement initiatives.

(1S1) (1S2) (2S1) (3S1) (4S1) (5S1) (6S1) (6S2)

All Students 93.31% 91.68% 63.63% 95.40% 94.63% 88.49% 26.67% 22.81%

Male 91.66% 90.89% 60.83% 95.26% 94.43% 89.22% 5.05% 5.41%

Female 95.22% 92.60% 67.96% 95.57% 94.86% 87.63% 55.89% 43.11%

American Indian or Alaskan Native 93.02% 79.07% 58.33% 97.37% 94.59% 82.86% 30.40% 32.50%

Asian or Pacific Islander 96.02% 95.52% 72.50% 95.41% 93.26% 88.70% 29.42% 27.78%

Black (not Hispanic) 90.14% 85.84% 51.53% 92.59% 93.00% 76.72% 28.34% 24.42%

Hispanic 93.44% 93.44% 46.40% 93.10% 92.42% 81.96% 28.99% 25.97%

White 93.69% 92.39% 65.43% 95.86% 94.91% 90.34% 26.35% 22.49%

Unknown – Multi-Racial 94.49% 92.91% 62.75% 95.47% 94.41% 82.27% 27.45% 23.85%

Individuals with Disabilities 68.18% 61.81% 47.63% 95.00% 93.88% 79.90% 20.80% 18.95%

Economically Disadvantaged 88.42% 85.52% 56.54% 93.37% 92.67% 81.76% 27.13% 23.35%

Limited English Proficient 93.75% 92.71% 32.50% 84.30% 84.26% 82.83% 26.76% 18.29%

Single Parents* 87.95% 86.23% 27.07% 23.96%

Migrant Status 100.00% 100.00% 50.00% 66.67% 66.67% 100.00% 18.75% 33.33%

CTE Nontraditional Enrollees 94.49% 92.05% 58.80% 95.96% 95.64% 87.56%

Secondary Tech Prep Concentrators 97.51% 96.34% 64.00% 97.01% 96.18% 92.54% 22.80% 22.77%

* Single Parents data collected for the first time in FY2009. Not collected for indicators using one-year lag data (2S1, 3S1, 4S1 and 5S1).

22

CTE Indicators of Performance# Locals Did Not

Meet 90%

Academic Attainment—Reading (1S1) 1Academic Attainment—Mathematics (1S2) 1Technical Skill Attainment (2S1) 23Secondary School Completion (3S1) 2Student Graduation Rates (4S1) 0Placement (5S1) 5Nontraditional Participation (6S1) 7Nontraditional Completion (6S2) 8

FY09 Local Performance – 35 Did Not Meet 90% of One (1) or More Targets

What can State Staff do to help improve 2S1?

23

• Improve guidance and technical assistance provided.

• Improve published resources – Example is CTE Assessment Matrix:–State CTE Assessments– Industry Assessments

• Improve local data verification reports.

Local Administrators Conference

24

• Did Ohio meet all state targets?• Did Ohio meet 90% of state

targets?• How did student sub-groups

perform?• How did the LOCALS perform?

Reviewed FY09 Performance Data, same questions

Local Administrators Discussion

25

• Explain more about CTE Participant.

• Explain more about CTE Concentrator.

• How is 3S1 different than 4S1?• Explain more about 5S1.• How is 2S1 calculated? Which

students have to take technical assessments? When do we report what data? MUCH discussion.

Strategies for Improving CTE Performance Results• Understand CTE accountability

(CTE students, CTE indicators, how indicators are calculated);

• Improve data quality – more complete, accurate and reliable data;

• Help students perform at higher levels;

• And more…

26

Understand CTE Accountability• Improve Guidance Documents

(published on ODE Web)• Conduct Webinar sessions• Increased engagement of local

CTE leaders

27

28

Data Quality – EMIS data system (PK-12 SLDS)

• All data for Secondary CTE reported in EMIS

• Student-level data (student identifier)

• For FY09, CTE pulled EMIS data from:– Multiple Years -- FY09, FY08, FY07– Four EMIS reporting periods each

year

Data Quality -- Strategies for Improving EMIS Data• Improve CTE EMIS data verification

reports• Conduct Webinar sessions and

trainings• Develop new CTE Data Tools to

help Locals –Monitor data while reporting periods

are open–Track performance results 29

30

We are conducting Webinars for CTE on different EMIS reporting periods, with expanded information on EMIS reporting and how to use and interpret the CTE EMIS reports.

Develop More Capacity to Work with Locals• Build more capacity among ODE state

staff to understand CTE data and the CTE EMIS data reports.

• ODE state staff can work one-on-one with local CTE leaders on:– Monitoring local performance data– Tracking local preliminary performance

data– Developing Performance Improvement

Plans31

Improving Student Performance

• Focus on Specific CTE Indicators:–Webinar on improving

Nontraditional Participation and Nontraditional Completion

• Focus on Student Sub-groups:–CTE Initiatives on Students With

Disabilities 32

Local CTE Performance Improvement Initiatives• Benchmarking own performance

results with other locals.• High engagement of staff and

students in CTE performance.• Using CTE Program performance

results as a gauge of program quality and viability.

33

What different strategies are

you employing to improve CTE

performance?34

Perkins IV – “National” CTE Data(?)• Much variability in definitions of CTE

students and indicators of performance.

• Much variability in CTE state targets and performance results.

• Difficult to tell a “National Story” about CTE.

35

36

FY2008 CTE AccountabilityAll States and DC, PR Guam, VI (54 total)

IndicatorState Target

Range

Actual Performance

Range

# Met 90% of Target

High Low High Low Yes No

1S1 97.73 22.30 97.55 25.29 48 4

1S2 96.97 11.00 97.50 11.26 47 5

4S1 95.00 50.00 100.00 42.38 51 1

37

Perkins V – Looking ahead…

38

Might we strive for more consistency

instudent and

indicator definitions?

Thank you for participating in this

discussion on improving CTE performance results!

And may your performance results continually improve.

Sharon Enright, Ph.D.Ohio Department of Education

Office of Career-Technical Educationsharon.enright@ode.state.oh.us

614-644-681439