Data use and its role inData use and its role in ...

Data use and its role inData use and its role in increasing graduation rates

Dr. Deborah JonasExecutive Director for Research and Strategic

PlanningVirginia Department of EducationVirginia Department of Education

February 2010

Factors associated with increased likelihood of droppingincreased likelihood of dropping

out of school• Elementary school

– Poor behavior, including aggressive behavior.– School performance as measured by behavior, academics, and p y , ,

attendance.– Children who repeat grades in K-4 are five times more likely to

drop out of school.• Middle and high school• Middle and high school

– Poor in-school behavior.– Failing grades in mathematics.

Failing grades in English– Failing grades in English.– Poor attendance.– Being retained in grade.

February 2010

Adapted from: Bost, L. & Klare, M. (2007). National Dropout Prevention Center for Students with Disabilities. http://www.ndpc-sd.org/documents/Teleseminars/Bost-071016/Policies_Procedures_teleseminar_10-16-2007.pdf.

Additional risk factors for high school st dentsschool students

• Entering ninth grade two or more grade levels behind their peers.

• Being retained in ninth grade.• Poor transition into high school—a large percentage of

high school dropouts fall “off-track” between ninth andhigh school dropouts fall off track between ninth and tenth grade.

February 2010

Adapted from: Bost, L. & Klare, M. (2007). National Dropout Prevention Center for Students with Disabilities. http://www.ndpc-sd.org/documents/Teleseminars/Bost-071016/Policies_Procedures_teleseminar_10-16-2007.pdf

Do the indicators apply for st dents ith disabilities?students with disabilities?

• In Chicago public schools the same indicators wereIn Chicago public schools the same indicators were predictive of graduates for students with disabilities and students without disabilities:*

Grade point average– Grade point average– Course failures– and on-track1 status

• Students with disabilities had lower graduation rates than students without disabilities.

• Students with disabilities who were on-track forStudents with disabilities who were on track for graduating in their freshman year were two to six times more likely to graduate that students who were off-track.

February 2010

*Gwynne, J., Lesnick, J., Hart, H.M., & Allensworth, E.M. (2009). What matters for staying on-track and graduating in Chicago Public Schools: A focus on students with disabilities. Chicago, IL: Consortium on Chicago School Research.1Failed no more than one core course and accumulated at least 5 full-year course credits

How can data inform our work?How can data inform our work?• Advances in technology and data availability make it possible to better

predict who is at risk of dropping out.• Research has identified factors most predictive of students dropping out of

school (e.g., Allensworth & Easton, 2005; Neild & Balfanz, 2006).

• Using these data can provide schools and school divisions with diagnostic g p gtools that:

– Inform leaders about the specific challenges and the magnitude of those challenges at each school.

– Match interventions to students based on challenges.I f th d l t f t t i l t i d ti t– Inform the development of strategic plans to improve graduation rates.

– Support school staff’s ability to communicate with students and parents about changes needed to support students’ graduation goals.

• VDOE is using a multi-pronged approach to using data and analysis to t h l di i isupport school divisions.

• As we collect more specific data, we can improve the data tools we offer to our schools.

February 2010

Allensworth, E.M. & Easton, J. Q. (2005). The on-track indicator as a predictor of high school graduation. Chicago: Chicago, IL: Consortium on Chicago School Research.

Neild, R.C. & Balfanz, R. (2006). Unfulfilled promise: The dimensions and characteristics of Philadelphia’s dropout crisis, 2000-2005. Philadelphia Youth Network, The Johns Hopkins University, and the University of Pennsylvania.

Multi-pronged approach to data seuse

• Comprehensive reporting of cohort graduates, dropouts, students who complete school, and those who stay in schoolschool.

• Conducting research using existing data available to the statestate.

• Build data tools that divisions and schools can use as part of their improvement processpart of their improvement process.

February 2010

Comprehensive reportingComprehensive reporting

The Cohort Report and information available regarding students

th h hi h h lprogress through high school

Results from Virginia’s Cohort Report 2009

SubgroupAdjusted

cohort %

Graduated%

GED%

Certificate% Still

Enrolled%

Dropout

% long-term

leave of absence

% unconfirmedSubgroup cohort Graduated GED Certificate Enrolled Dropout absence unconfirmed

All Students 98043 83.22% 3.64% 0.33% 2.68% 7.93% 0.45% 1.76%Female 48093 86.40% 2.81% 0.35% 2.13% 6.50% 0.41% 1.40%Male 49950 80.15% 4.44% 0.32% 3.21% 9.31% 0.49% 2.10%Black 26433 75.69% 3.36% 0.58% 4.82% 11.26% 0.90% 3.40%Hispanic 6812 72.28% 3.07% 0.69% 2.89% 19.14% 0.28% 1.64%White 58003 87.01% 4.03% 0.19% 1.78% 5.55% 0.29% 1.16%Asian 4953 93.30% 1.31% 0.18% 1.47% 3.31% 0.10% 0.32%American Indian 301 78.41% 5.98% 0% 3.65% 9.30% 1.33% 1.33%Native Hawaiian 89 83.15% 4.49% 0% 3.37% 6.74% 0% 2.25%Other 6795 91.11% 2.02% 0.24% 1.84% 4.05% 0.19% 0.56%Students with Disabilities 11901 82.49% 3.81% 0.66% n/a 12.46% 0.01% 0.55%Students with Disabilities anytime 13962 79.45% 4.23% 0.60% 0.01% 15.19% 0.01% 0.51%Economically Disadvantaged 24009 73.19% 5.50% 0.66% 4.61% 11.95% 0.81% 3.28%Economically Disadvantaged anytime 35370 70.87% 5.53% 0.62% 4.61% 14.59% 0.86% 2.91%Limited English Proficient 3920 68.78% 1.63% 1.58% n/a 26.12% 0.08% 0.82%Limited English Proficient anytime 5951 74.86% 1.97% 1.13% 0.66% 20.79% 0.05% 0.55%Migrant 45 66 67% 6 67% 6 67% 0% 20% 0% 0%

Data for the state, school divisions, and schools can be downloaded at:

Migrant 45 66.67% 6.67% 6.67% 0% 20% 0% 0%Migrant anytime 76 60.53% 3.95% 6.58% 0% 28.95% 0% 0%Homeless 587 66.61% 4.77% 1.19% 5.28% 17.89% 1.19% 3.07%Homeless anytime 1395 62.01% 5.38% 0.79% 6.59% 21.22% 1.29% 2.72%

February 2010

Data for the state, school divisions, and schools can be downloaded at: http://www.doe.virginia.gov/statistics_reports/graduation_completion/cohort_reports/index.shtml

More information available regarding st dent stat sregarding student status

• Detailed cohort reports available to the public show theDetailed cohort reports available to the public show the numbers (rather than percents), including number of students earning each diploma type.

• Authorized users can view details of all students in SSWS On-Time Graduation Rate application.– Status of students in future cohorts (e g class of 2010 2011Status of students in future cohorts (e.g., class of 2010, 2011,

2012, and 2013).– Details of cell sizes less than 10.– Updated status of students in previously reported cohorts (can be

d t i t f t d t till ll d i h l)used to review outcomes of students still enrolled in school).

• Review an up-to-date list of students in each adjusted h t

February 2010

cohort.

Take a closer lookTake a closer lookVirginia Diplomas Non-diploma completion

credentials

Subgroup CohortAdvanced

studies Standard Modified standard Special

Grad Rate GED

Certificate of Program Completion

Cohort Completion

RateDropout

RateAll St d t 98 102 43 343 34 340 1 827 2 076 83 2 3 665 327 87 2 8 1

credentials

Students 98,102 43,343 34,340 1,827 2,076 83.2 3,665 327 87.2 8.1Female 48,106 24,309 15,862 682 700 86.4 1,368 169 89.6 6.6Male 49,996 19,034 18,478 1,145 1,376 80.1 2,297 158 85 9.4Black 26,476 6,905 11,269 755 1,080 75.6 941 154 79.7 11.6Hispanic 6,812 2,151 2,548 139 84 72.3 210 47 76 19.3White 58,019 30,017 18,719 873 856 87 2,374 110 91.3 5.6Asian 4,953 3,384 1,174 31 31 93.3 66 < 94.8 3.3Students with Disabilities 11,956 1,169 4,746 1,827 2,076 82.1 461 78 86.6 12.8Disabilities 11,956 1,169 4,746 1,827 2,076 82.1 461 78 86.6 12.8Students with Disabilities anytime 14,026 1,620 5,571 1,827 2,076 79.1 598 84 84 15.5

February 2010

ExerciseExerciseGraduation

RateCompletion

RateDropout

Rate

All students 83.2 87.2 8.1

Students with Disabilities 82.1 86.6 12.8

Students with Disabilities Anytime 79.1 84 15.5

• What else do the cohort report data tell you?you?

• What else would you want to know?• Where can you find other information?

February 2010

Data to considerData to consider

• Division

• School

• Student

February 2010

Use Existing DataUse Existing Data

Outcomes of Virginia’s students on factors associated

with dropout

Data on the following slides show indicators associated with dropout, not necessarily causes of dropping out.

Retentions in student cohortsRetentions in student cohorts

NonRetention/promotion Dropouts

Non-dropouts

Retained in high schoolg2009 62% 12%2008 59% 12%

Retained in ninth grade2009 36% 9%2008 38% 6%

February 2010

Students drop out in all grade le elslevels

Percent of Dropouts by Grade: Graduation Cohort of 2009p y

23% 17% 23% 23%33%

19%

37%23%90%

100%

27%

21%

27% 31%26%

26%

28%

21%

33% 37%

50%60%70%80%

33%

28%

29%

29% 23%21%

28%

20%

35%

28%

20%30%40%50%

22%33%

22% 23% 20%27%

14% 21%

0%10%20%

Region 1 Region 2 Region 3 Region 4 Region 5 Region 6 Region 7 Region 8

February 2010

Grade 9 Grade 10 Grade 11 Grade 12

Age at high school exitAge at high school exit

Percent of students in cohorts who were two orPercent of students in cohorts who were two or more years overage for grade

Diploma Dropouts

pearners

2009 70% 9%2008 68% 8%

February 2010

AttendanceAttendance

NonDropouts

Non-dropouts

Attendance < 80% of their last year of schooly2009 65% 7%2008 65% 7%

Attendance < 80% one year before leaving high school

2009 35% 4%2009 35% 4%2008 29% 4%

February 2010

Attendance decreases over four ears for st dents ho drop o t*years for students who drop out*

STATEWIDEAverage Days Absent for all students enrolled in

2007-2008

Non-dropout Dropout 2004-2005 8 192005-2006 8 222006 2007 9 282006-2007 9 282007-2008 10 30

*L it di l ti f l d t d t 2008

February 2010

*Longitudinal perspective of annual dropout data, 2008Source: Holian, L. (2009). Status of High School Dropouts in the State and Two Southwest Regions. Presented at the VSBA meeting, March 11. REL Appalachia.

Students who drop out in Virginia ha e more risk factorshave more risk factors

Risk factors analyzed for students enrolled in 2008• Changing schools during the school year• Changing schools during the school year• Repeating the current grade• Being absent more than 20 percent of the time• Two or more years overage for grade• Two or more years overage for grade

Number of Risk FactorsGrade level 0 1 or more 2 or more 3 or more 4

Non-dropout 73 8% 26 2% 10 4% 4 0% 0 9%Ninth grade

Non-dropout 73.8% 26.2% 10.4% 4.0% 0.9%Dropout 4.6% 95.4% 77.6% 53.0% 18.4%

Tenth gradeNon-dropout 81.5% 18.5% 4.3% 0.9% 0.1%Dropout 8.7% 91.3% 57.7% 21.0% 3.4%p

Eleventh gradeNon-dropout 83.7% 16.3% 3.5% 0.6% 0.0%Dropout 13.9% 86.1% 45.6% 14.2% 1.3%

Twelfth gradeNon-dropout 6.1% 93.9% 14.0% 1.8% 0.1%

February 2010

Twelfth gradeDropout 6.2% 93.8% 63.9% 63.9% 1.2%

*Longitudinal perspective of annual dropout data, 2008Source: VDOE data analysis conducted with technical assistance from Laura Holian, Ph.D., REL Appalachia. 2009.

More recent findings…work in progressprogress

• VDOE is interested in understanding more about relationships• VDOE is interested in understanding more about relationships between assessment data and graduation/dropout.– In part because we don’t collect course outcomes data statewide;– It is also critical that we understand how high stakes assessments relate

to critical student outcome indictors, including graduation and dropout.• Preliminary results*:

– Freshmen who failed the statewide reading assessment 2 years in a row are 38% more likely to drop out after controlling for other risk factors.y p g

– Freshmen who failed the statewide mathematics assessment 2 years in a row are 31% more likely to drop out after controlling for other risk factors.High absenteeism is the most predictive factor that we have available for– High absenteeism is the most predictive factor that we have available for analysis.

• VDOE plans to continue this work adding more data as it becomes available.

February 2010

*Virginia’s analyses of the associations between SOL data and dropout are supported by technical assistance provided by the Regional Educational Laboratory at Appalachia.

ExerciseExercise

• What do you know about the students who dropout of your division/school?– Attendance – Behavior– Course Failure

• Where might you find the data you need to understandWhere might you find the data you need to understand the indicators of drop in your school/division?

• If the data are not readily available, does your division/school have a plan to provide it?division/school have a plan to provide it?

• Once you have the data, then what?

February 2010

Postsecondary outcomesPostsecondary outcomes

February 2010

Postsecondary enrollmentPostsecondary enrollmentYear Graduated/Completed High School

Postsecondary enrollment 2006 2007 2008

Enrolled within one year Percent enrolled

4-year institution 38% 38% 37%


Less than 2 year institution < 1% < 1% < 1%

Total 60% 63% 62%Total 60% 63% 62%

Enrolled anytime since high school completion



Less than 2 year institution < 1% < 1% < 1%

Total 69% 68% 62%

February 2010

Total 69% 68% 62%Preliminary analyses of data from the National Student Clearinghouse. NOTE: Approximately 45 percent of students who enroll in Virginia’s two-year colleges require developmental coursework.

Postsecondary enrollment rates for students with disabilities infor students with disabilities in

VirginiaPostsecondary enrollment of high school graduates and completers , 2006 and 2007, Virginia's public high schools,

students with disabilities

45 0%

30.1%

38.4%

25 0%30.0%35.0%40.0%45.0%

11.6%

0 5%5.0%10.0%15.0%20.0%25.0%

0.5%0.0%5.0%

Enrollment Rates

Four‐Year Two‐Year Less than Two‐Year Any

February 2010

Lichtenberger, E., Dietrich, C., Kamulladeen, R., & O’Reilly, P.A. (2010). Postsecondary enrollment: Summary of Phase I. Blacksburg, VA: Virginia Tech.

VDOE Data ToolsVDOE Data Tools

February 2010

Watch List Report for K-12Watch List Report for K 12• Provides authorized users with information about

students who may be at risk of not being successful in y gschool.

• Information can be used to inform school-wide decisions that impact student instruction and support services andthat impact student instruction and support services and help ensure that interventions are tailored to students' individual needs.

• Provides school and student level “flags” for:– Attendance– SOL performance– Students two or more years overage for grade– Students who were retained (forthcoming)

• Currently available through Virginia’s Education

February 2010

Currently available through Virginia s Education Information Management System (EIMS) with enhancements planned

Watch List Report*Attendance # Failed English- # Failed Math

# with 2 or More Yrs # with 1 or

Watch List Report

Grade < 80% Reading SOL 2 Yrs SOL 2 Yrs Overage More FlagsKG 1% 0% 0% 0% 1%

1 1% 0% 0% 0% 1%

2 0% 0% 0% 1% 1%2 0% 0% 0% 1% 1%

3 1% 1% 0% 2% 3%

4 0% 3% 3% 1% 6%

5 0% 2% 3% 2% 6%5 0% 2% 3% 2% 6%

6 1% 3% 3% 2% 8%

7 2% 5% 13% 4% 19%

8 2% 7% 9% 4% 17%

9 3% 0% 2% 6% 10%

10 2% 0% 3% 7% 11%

11 2% 1% 3% 3% 8%

February 2010

12 6% 0% 2% 5% 13%

*sample data from one school division

Early warning indicator toolkitEarly warning indicator toolkit

• Uses data to identify students in ninth grade at risk of not succeedingUses data to identify students in ninth grade at risk of not succeeding in high school.

• Modeled on two tools:

– Boston City Schools’ Composite Learning Index: http://www.bpe.org/school_dev/cli

– Early warning system tool from the National High School Center: http://www betterhighschools org/topics/DropoutWarningSigns asphttp://www.betterhighschools.org/topics/DropoutWarningSigns.asp

• Will include a user’s guide and examples of how to use the data.

• Provides information on rising 8th graders and monitoring/tracking t l f 9th d t d ttool for 9th grade students.

• Includes early warning indicators and intervention tracking capability.

February 2010

A preview of Virginia’s Early Warning System and GuidanceWarning System and Guidance

Document

Cycle of the Early Warning System Tool Data Analysis ProcessTool Data Analysis Process

February 2010

From: A Practitioners’ Guide to Analyzing Virginia’s Early Warning Systems Data—DRAFT document. Developed by the National High School Center in collaboration with the Appalachia Regional Comprehensive Center.

VDOE supports a tiered approach to inter entionapproach to intervention

February 2010

From: A Practitioners’ Guide to Analyzing Virginia’s Early Warning Systems Data—DRAFT document. Developed by the National High School Center in collaboration with the Appalachia Regional Comprehensive Center .

A functioning tiered system…A functioning tiered system…

• Has buy-in from multiple levels• Is clearly defined and understood by• Is clearly defined and understood by

stakeholders throughout the system.Has clear protocols that permit student to• Has clear protocols that permit student to move through the tiers in both directionsseamlessly based on needsseamlessly based on needs.

February 2010


Early warning indicator toolkitEarly warning indicator toolkitSemester One Student Report This report provides detailed student level information on the early warning indicators for Semester One.

PRE-

HS A

lert

EWS

Aler

t

Last Name First Name Student ID Grade

Flag for First 20 Day Count Attendance

Flag for Q1 Attendance

Flag for S1 Attendance

Flag for Course Fs Flag for GPA At

tenda

nce

Tier 1. Attendance

Intervention2. Attendance Intervention Be

havio

r Tier

1. Behavior Intervention

Student 2 Example 2 1 9 # No No No No No # 1 Attendance plan home visit 1 0

((Imported From DATA ENTRY Student Information Form) (Monitor for On- or Off-Track to On-time Graduation)Student Information Semester One Indicators of Risk

p p

! Student 3 Example 3 2 9 # Yes No No Yes Yes O 2 attend 2 attend 2 2 advisory

Student 5 Example 5 3 9 # No No No Yes No O 1 0 0 1 0

Student 6 Example 6 4 9 # Yes No No Yes No O 1 0 0 1 0

! Student 7 Example 7 5 9 # Yes No No Yes No O 2 Attendance plan home visit 1 0

! Student 8 Example 8 6 9 # No No No No No # 3 home visit 0 1 0

! Student 9 Example 9 7 9 # No No No No No # 2 Attendance plan 0 1 0

! Student 10 Example 10 8 9 # No No No No No # 3 home visit 0 1 0! Student 10 Example 10 8 9 # No No No No No # 3 home visit 0 1 0

February 2010


Early warning indicator toolkitEarly warning indicator toolkitBEGINNING OF SCHOOL YEAR: Students by Tiered

InterventionIntervention

100%

60%

80%

20%

40% Tier 3

Tier 2

Tier 1 From: A Practitioners’ Guide to Analyzing Vi i i ’ E l W i

0%

Tier 3 0.0% 5.4% 0.0%

Ti 2 14 3% 3 6% 25 0%

Attendance Behavior Academic

Virginia’s Early Warning Systems Data—DRAFT document. Developed by the National High School Center in collaboration with the Appalachia Regional

February 2010

Tier 2 14.3% 3.6% 25.0%

Tier 1 85.7% 91.1% 75.0%

Appalachia Regional Comprehensive Center.

Early warning indicator toolkitEarly warning indicator toolkitFULL YEAR ‐ All Students

60

Not Flagged (On‐Track) All Students Flagged ( Off‐Track)

40

50

20

30

From: A Practitioners’ Guide to Analyzing Vi i i ’ E l W i

0

10

All St d t Fl d ( Off T k) 2 31 9

Flag for Attendance Flag for Course Fs Flag for GPA


February 2010

All Students Flagged ( Off‐Track) 2 31 9

Not Flagged (On‐Track) 52 23 47


Early warning indicator toolkitEarly warning indicator toolkitEND OF YEAR ‐ Student Status (based on Full Year flag status)

Not Flagged (On Track) All Students Flagged ( Off Track)

45

50

Not Flagged (On‐Track) All Students Flagged ( Off‐Track)

25

30

35

40

10

15

20

25

From: A Practitioners’ Guide to Analyzing Vi i i ’ E l W i

0

5

All Students Flagged ( Off‐Track) 36 5 4

Enrolled Retained Dropped Out


February 2010

All Students Flagged ( Off Track)

Not Flagged (On‐Track) 10 0 1


Use data to inform practiceUse data to inform practice• Data on risk factors for dropout are a means to understanding the types of

challenges that divisions schools and individual students face in achievingchallenges that divisions, schools, and individual students face in achieving graduation goals.

• The next steps are to use the data to identify interventions aligned with areas of need.

• Research on risk factors and evidence-based programs shows important lessons learned about interventions programs (Hammond, Linton, Smink, & Drew, 2007):

Use multiple strategies to help assure program impact– Use multiple strategies to help assure program impact.– The strategies chosen to support at-risk students should be evidence-based,

aligned with the risk factors they need to target, and grounded in best practices.– When practitioners adopt existing programs, research suggests the programs

h ld b f ll i l t d d i l t d th d i dshould be fully implemented and implemented as they were designed.– The likelihood of dropping out increases as the number of risk factors increases,

and prevention strategies should take this into account and target as many risk factors as possible.

February 2010

• Looking at data only provides a starting point! Hammond, C., Linton, D., Smink, J. & Drew, S. (2007). Dropout risk factors and exemplary programs: A technical report. Clemson,

SC: National Dropout Prevention Center, Communities In Schools, Inc.

Questions?Questions?

Deborah Jonas,Ph.D.Executive Director for Research and Strategic Planningg g

Virginia Department of [email protected]

804-225-2067

February 2010

Data use and its role inData use and its role in ...

Documents

Transcript of Data use and its role inData use and its role in ...