SUMMER LEARNING PROJECTSUMMER LEARNING PROJECT

Overview of Outcome Measures Overview of Outcome Measures

SUMMER 2011SUMMER 2011University of California, IrvineUniversity of California, Irvine

Department of EducationDepartment of Education

California Afterschool Outcome Measures ProjectCalifornia Afterschool Outcome Measures ProjectSUMMER LEARNING PROECT SUMMER LEARNING PROECT 20112011

California Afterschool Outcome Measures ProjectCalifornia Afterschool Outcome Measures ProjectSUMMER LEARNING PROECT SUMMER LEARNING PROECT 20112011

Overview of key elementsOverview of key elements: : Participation open to all 8 Summer Learning Participation open to all 8 Summer Learning

Communities and their respective sitesCommunities and their respective sites

Technical assistance provided Technical assistance provided • Email and telephone help during survey administrations & help Email and telephone help during survey administrations & help

interpreting scores at the end of summer programinterpreting scores at the end of summer program

Web-based surveys of student performance Web-based surveys of student performance collected from students and program staff at collected from students and program staff at the beginning and end of summer session the beginning and end of summer session 20112011

Confidential summary report of survey Confidential summary report of survey resultsresults

• Programs receive scores of positive behavior change and skill Programs receive scores of positive behavior change and skill development for their site and across all sites development for their site and across all sites

Option to Administer Option to Administer Student and Staff SurveysStudent and Staff Surveys

StudentStudent—Elementary & Middle School——Elementary & Middle School—(5/6 subscales)(5/6 subscales)• Work HabitsWork Habits• MisconductMisconduct• Reading Efficacy Reading Efficacy • Math EfficacyMath Efficacy• Social EfficacySocial Efficacy• Program Environment [at end of summer session only]Program Environment [at end of summer session only]

Program StaffProgram Staff——Reports on Students Reports on Students (5 subscales)(5 subscales)• Work HabitsWork Habits• Aggressive Behavior with PeersAggressive Behavior with Peers• Prosocial Behavior with PeersProsocial Behavior with Peers• Task PersistenceTask Persistence• Social SkillsSocial Skills

STUDENT SURVEY STUDENT SURVEY ADMINISTRATIONADMINISTRATION

Time to Complete Survey Time to Complete Survey

The majority of students (80%) take The majority of students (80%) take less than 10 minutes less than 10 minutes to to complete the survey online.complete the survey online.

Some students may take up 10-15 Some students may take up 10-15 minutes with a few students requiring minutes with a few students requiring extra support. extra support. See instructions for Slow Readers and English See instructions for Slow Readers and English

Language Learners available on website.Language Learners available on website.

Timeline Survey Timeline Survey Administration Administration PRE-SURVEYS: PRE-SURVEYS:

WEEK 1 & 2 OF SUMMER SESSION WEEK 1 & 2 OF SUMMER SESSION

POST-SURVEYS: POST-SURVEYS:

LAST TWO WEEKS OF LAST TWO WEEKS OF final final SUMMER SUMMER SESSION SESSION

Surveys can be administered any time during the 2-week survey window

[at the beginning and end of your summer program] and can be done in multiple sessions.

Surveys can be administered any time during the 2-week survey window

[at the beginning and end of your summer program] and can be done in multiple sessions.

California Afterschool Outcome Measures ProjectCalifornia Afterschool Outcome Measures ProjectSUMMER LEARNING PROJECT SUMMER LEARNING PROJECT 20112011

Instructions available on Project Instructions available on Project

website: website: afterschooloutcomes.orgafterschooloutcomes.org

Click on TABClick on TABClick on TABClick on TAB

Select Appropriate Instructions

Download & Read Prior to Survey

Administration

Summer Outcome Measures Overview Ppt.Initial Instructions for Site Coordinator to Administer Online SurveysSite Coordinator Instructions to Train Staff to Administer

Student SurveysStudent Survey InstructionsSite Coordinator Instructions to Administer Staff SurveysStaff Survey InstructionsSpecial Instructions for ELL and Special Needs Students

Example Walk-through of Example Walk-through of STUDENT SURVEY ADMINISTRATIONSTUDENT SURVEY ADMINISTRATION

STUDENT INDIVIDUAL SURVEY ACCESS CODESSTUDENT INDIVIDUAL SURVEY ACCESS CODES

Give students their individual assigned student codeGive students their individual assigned student code—highlighted in —highlighted in yellow yellow on the on the Request for Survey Request for Survey Codes TemplateCodes Template——this code will be used for pre- and this code will be used for pre- and post- surveys and will allow for analysis of individual post- surveys and will allow for analysis of individual student behavior changes.student behavior changes.

Make sure you Make sure you request codes request codes for all students for all students enrolled in the summer learning program enrolled in the summer learning program prior to prior to your site’s pre-survey administration dateyour site’s pre-survey administration date. The excel . The excel form can be found on our website. form can be found on our website.

Please provide the student Please provide the student State ID numbers* State ID numbers* as as well. This will allow us to track them.well. This will allow us to track them.

* The CSID (student State ID #) and other student information is necessary for cross-referencing and verifying incoming survey data, and for linking survey data longitudinally.

GO TO PROJECT WEBSITE: GO TO PROJECT WEBSITE: afterschooloutcomes.orgafterschooloutcomes.org

1. Click on “Summer Learning Project” tab at top.

GO TO PROJECT WEBSITE: GO TO PROJECT WEBSITE: afterschooloutcomes.orgafterschooloutcomes.org

1. Click on “Summer Learning Project” tab at top.

2. SELECT appropriate survey: PRE-Participation Survey if administering at start of program POST-Paricipation Survey if administering at end of program period3. Click on appropriate level: Elementary or Middle School

Example First PageExample First PageElementary Student Elementary Student

PRE-Participation SurveyPRE-Participation Survey

Example First PageExample First PageElementary Student Elementary Student

PRE-Participation SurveyPRE-Participation Survey

GREEN BAR indicates Pre-Participation survey

4. Student a. Selects language [English or Spanish] b. Enters assigned survey code:

4. Student a. Selects language [English or Spanish] b. Enters assigned survey code:

ClCliick downck downFor SpanishFor Spanish

To review survey & test link on computers prior to administering use

TEST ACCESS CODE: 52911

To review survey & test link on computers prior to administering use

TEST ACCESS CODE: 52911

5. When student has finished survey this page should appear.

5. When student has finished survey this page should appear.

Program Staff Survey:Program Staff Survey:Reports on Students in Afterschool ProgramReports on Students in Afterschool Program

Staff Staff need both their need both their individual access codeindividual access code and and the the survey code for each student survey code for each student for whom they for whom they will be completing a report.will be completing a report.

Each Each student report student report takes takes 3-5 minutes 3-5 minutes to to complete.complete. Staff may complete student reports in multiple sessions. Staff may complete student reports in multiple sessions.

For exampleFor example: A staff completing reports for 20 students can do : A staff completing reports for 20 students can do four reports a day (a 20-minute session) on five different days.four reports a day (a 20-minute session) on five different days.

Ideally, staff should complete reports at the end of Ideally, staff should complete reports at the end of the summer session on the same students for whom the summer session on the same students for whom they completed reports at the beginning of the they completed reports at the beginning of the summer. However, if there are changes in staff or summer. However, if there are changes in staff or group assignments then staff can report on different group assignments then staff can report on different students.students.

Questions about Outcome MeasuresQuestions about Outcome Measures

For answers to frequently asked questions For answers to frequently asked questions

go to Project Website click on “go to Project Website click on “FAQFAQ” tab:” tab:

ViView other ew other informationinformation

about the Project & about the Project & download download

instructions instructions [click on “Summer Learning Project”

tab]

for administering the for administering the outcome measures outcome measures

Questions before or during survey Questions before or during survey administration… Contact us by email at:administration… Contact us by email at:

afterschool@uci.edu afterschool@uci.edu

Questions before or during survey Questions before or during survey administration… Contact us by email at:administration… Contact us by email at:

afterschool@uci.edu afterschool@uci.edu