Lessons Learned from Working with Absenteeism Data: A Local Evaluation

informing positive community change

Presented by: Dr. L. Shon Bunkley, CRPOctober 16, 2008

CRP’s local evaluation of a pilot truancy reduction project

Program background 6 months in operation (Dec ‘07 – May ‘08) Collaboration between school district, police,

county prosecutor, job and family services, juvenile court

Targets elementary and middle school students Program components

3 unexcused absences = Warning letter from Prosecutor

5 unexcused absences = Mediation session 10 unexcused absences = Truancy board hearing 15 unexcused absences = Referral to Juvenile Court

CRP’s local evaluation of a pilot truancy reduction project (cont’d)

Major measures of success All pilot schools meet or exceed the

attendance standards set by CCS Reduce by 50% the number of referrals to

juvenile court from pilot schools Reduce by 75% the rate of unexcused

absences for students who participate in a mediation session

Several secondary measures, like 30 consecutive days of no absences

CRP’s local evaluation of a pilot truancy reduction project (cont’d)

Evaluation methodology Literature review Interviews

Staff Stakeholders Parents and youth

Analysis of secondary data Student absences School attendance rates School referrals to juvenile court

CRP’s evaluation findings

Between December and February, 199 students were mediated

Unexcused absences among mediated students at the pilot schools declined anywhere from 36% to 76%

Nearly 100 (50%) mediated students had 30 days of no unexcused absences following mediation

Too early to gauge school level improvements in attendance and referrals to juvenile court

Lesson 1: Define measures and outcomes

Definition of truancy Excused vs. unexcused absences

Definition of referral Eligible vs. charges filed

Lesson 2: Caution against the assessment of impact of a pilot project Assessing impact in 1st year is made difficult by:

Project being operational for a limited amount of time Not enough time to iron out the “kinks” of the project Not enough time to expect to see marked change Limited amount of data for analyses

Influence of several other district initiatives aimed at reducing truancy, such as truancy sweeps

High student mobility within the district Seasonal/time-specific attendance patterns Moderating factors (e.g., substance abusing parent)

Lesson 3: Be aware of competing interests

Client need for data to support effectiveness of the project In need of additional funding Desire for replication

Evaluator’s responsibility to keep the evaluation objective and balanced

Lesson 4: Clarify data elements and collection procedures

What data elements are needed Student vs. school-level data Program stats vs. school/district stats Ancillary indicators/facilitators of success, like

GPA Who will be responsible for collecting what

data School district vs. program staff Timelines will be dependent upon when the

district releases data Reconciliation of attendance data

Lesson 5: Get clarity and agreement on how data will be analyzed Rate of absences vs. number of

absences Average rate vs. median rate of

absences Inclusion/exclusion of students mediated

at the end of the school year

Up Front…

1. Define measure and outcomes2. Caution against the evaluation of the

impact of a pilot project3. Be aware of competing interests4. Clarify data elements and collection

procedures5. Get clarity and agreement on how

data will be analyzed

For more information

Dr. L. Shon Bunkley, Senior Research Associate

Community Research Partners(614) 224-5917, ext. 104sbunkley@communityresearchpartners.org