Pennsylvania Department of Education PROGRESS & & ACHIEVEMENT Pennsylvania Value-Added Assessment...

Pennsylvania Department of Education

PROGRESSPROGRESS&&

ACHIEVEMENTACHIEVEMENT

Pennsylvania Value-Added Assessment System


Session Overview

History of PVAAS

Pennsylvania’s Intent for PVAAS

PVAAS Methodology and Reports

Timelines for PVAAS Implementation

Data Needed for PVAAS Analyses

Costs

District Preparation for PVAAS

Supports


A Focus on A Focus on StandardsStandards


A Standards-based System

Goal of high achievement, college readiness

Standards aligned

Assessments aligned

Safety nets aligned

Curriculum aligned

Instruction aligned

Instructional materials aligned


Data Driven Decision Making Tools Provided to Districts

Leading for Learning!Getting Ready for the PSSAGetting Results!Data ToolsData ReportsPA GROWContext ToolsQuality ReviewsTutoring EvaluationsPVAAS

Student Planning Process

Focus: Classroom of Students

Who: Teacher

Periodic Grade Level Planning Process

Focus: Groups of StudentsWho: Teacher Teams

How: Regular 1-2 hour meetings

Annual Building-wide Planning ProcessFocus: All Students

Who: School-wide TeamHow: Data Retreat, School Planning Process

District-Level Support(Budgetary Support, Professional Development, Resources and Time)

Student Learning Data

School Structures for Data-Informed Decision Making

School LevelPSSA & PVAAS

Standardized AssessmentsDistrict End-of-Year Tests

Final Benchmark Test

Classroom Level Initial: PSSA/PVAAS/final tests – student levelCyclical: •Benchmark Data – Student LevelContinuous•Individual Classroom Assessments•Progress Monitoring

Grade/Course LevelInitial: PSSA/PVAAS/final tests – class/subgroup levelsCyclical:

•Benchmark Data - grade level•District quarterly assessments•Common Classroom Data•Classroom Summaries

Demographic/Perceptual/Process Data

School Level •School Demographics•Discipline Data•Attendance Data•Mobility Rate

Grade/Course Level•Class Demographics •Class Engagement Data•Satisfaction Data•Attendance Data

Classroom LevelQualitative Data•Student Historical Information•Student Medical Information•Student Learning Information

© Lancaster-Lebanon IU13


History of PVAAS

2002: Recommended by Pennsylvania

League of Urban Schools (PLUS) to

State Board of Education

2002: State Board of Education passed

resolution for “value-added approach

across the Commonwealth”

2002 (SY04-05): 3 Pilot Phases

100 Districts, including the School District of

Philadelphia and Pittsburgh City School

District


History of PVAAS

Fall ‘04 PDE Statewide Workgroup

Led by PA Secretary of Education, Dr. Zahorchak

50+ Members

Districts

Governor’s Office

Higher Education

Statewide Associations including, but not limited to:

PSEA, PFT, PASA, PSBA, PAIU, PAESSP, PASCD

Legislators

Business and Industry

SY05-06 Statewide Implementation - 501 Districts

1st Reporting in Fall 06


PA’s Intent for PVAAS

Another Tool for Continuous Improvement

Measure of Growth to Add to

Achievement/Status Measures

Not to be Used in Isolation

No Teacher Level Analyses


PA’s Intent for PVAAS

Integration of PVAAS into existing PDE initiatives:School Improvement Planning

Distinguished Educator Training

Governor’s Institutes

EAP

4Sight

Progress Monitoring

Content Initiatives: Literacy and Mathematics

Response to Intervention

PIL

PIMS (PA Information Management System)


What is PVAAS?What is PVAAS?


Student Learning Measures

Status MeasuresPSSA

% proficient or not proficient

Other local assessment data

Improvement MeasuresSafe Harbor

Other local assessment data

NEW MEASURE: Progress/Growth MeasurePVAAS


Value Added Analysis

Provides a measure of what “effect”

schooling has in a single year

Formally described: Mixed model multivariate

longitudinal analyses

– The statistical methodology has been reviewed by

nationally renowned statisticians since the 1980’s

– The reporting has been used by educators for more

than 10 years


PVAAS IS NOT:

Another test

Being used for teacher

level accountability in

Pennsylvania


PVAAS IS:

A statistical analysis of existing assessment

data

A precise measurement of student

progress/growth over time

Progress data to add to achievement data as

schools are analyzing their work with

students and making improvement plans


How is PVAAS analysis done?

Year 1 Year 2 Year 3

PSSA PSSA PSSA (7th)

5th 6th Predicted Actual

1. We collect all of the individual student data that is available for a minimum of two years.

2. All of these data are used to develop individual student profiles.

Reading 1280 1300

Math 1330 1305

1290 ??

1320 ??

Student ATesting Scores

Pool of Testing Histories

Computer

Prediction Model 1

Prediction Model 2

Prediction Model 6

Prediction Model 3

Prediction Model 4

Prediction Model 5

The PVAAS Methodology – Part 1

The PVAAS Methodology – Part 2

Student ATesting Scores

Prediction Model 1

Prediction Model 2

Prediction Model 6

Prediction Model 3

Prediction Model 4

Prediction Model 5

Student D Predicted Score

Student E Predicted Score

Student A Predicted Score

Student B Predicted Score

Student F Predicted Score

Student C Predicted Score

Mean of Student Scores for a particular school

Mean of Predicted Scores for a particular school

School Effect Mean of Student Scores – Mean of Predicted Scores

PVAAS Ratings of School EffectRemember:

School Effect Mean of Student Scores – Mean of Predicted Scores

So, if the students perform as expected, School Effect 0.

The ratings are in terms of the expected value of 0.

-2 SE’s 0 +2 SE’s

Not Detectably

DifferentAboveBelow

Standard Error (SE) is a measure of variation in the prediction


Achievement and Progress


Types of PVAAS Reporting

District Level

By School

School Level

By Grade

Reading and Math

Student Reports

Longitudinal Reports

“Projections” on Future PSSAs

Diagnostic Reports

By Quintiles

By Performance Levels

Diagnostic Subgroup Reports

Custom Reporting/Querying

District Summary Report

School Report

Above Expected Progress

Below Average

Achievement


Student Projections

Search for students

by subgroup or achievement

Identified students meeting search requirements


How Can PVAAS How Can PVAAS Help Me?Help Me?


Key Learning from PVAAS

Value-added information will tell you:

How much progress students have made in each subject area and grade level

How much progress students at different prior achievement levels have made.

How students’ progress in one curricular area or program compares to their progress in others


PSSA and PVAAS – The Power of 2!

Without data all we have is an opinion!

PSSA tells us where a student is.

PVAAS tells us where that student is headed.

Location and direction are critical partners in the

quest for having all students achieve proficiency

The data from both achievement and progress

measures gives schools valuable information to

make data informed decisions.


PA Proposal to USDOE

PA submitted proposal to USDOE to use

“projections” as means to recalculate

proficiency/AYP; did not include value-

added reporting as part of proposal

PA is waiting to hear from USDOE


Timelines for Implementation

Timeline Released in 2002

SY01-02 and SY02-03: Pilot Phase I

SY03-04: Pilot Phase II

SY04-05: Pilot Phase III

SY05-06: Statewide Implementation

All 501 Districts Receive PVAAS Reporting in Fall ‘06

Pilot districts get comprehensive PVAAS reporting

401 Districts Receive Limited Reporting

Grades 4 & 6

SY06-07: Comprehensive Reporting for All Districts


Data Needed for PVAAS

PSSA reading and math will be the only data

needed for statewide implementation

Only common assessment for all 501 districts

No data preparation is needed

No purchase of any data is needed

All PSSA data is sent direct from DRC to SAS,

Inc.


Costs Covered by PDE

Costs for PSSA Data Submission

Costs for PVAAS Analyses

Costs for PVAAS Web-Based Reporting

Costs for Communication Materials

Costs for Professional Development Materials

Costs for Professional Development SessionsAvailable through SY06-07

Regional sessions

Videoconferences

Webinars


How Can Districts Prepare?

Plan to Use PVAAS Data as part of Continuous

Improvement Planning

District Level

Building Level

Grade Level

Content level

Subgroup Level

Student Level


How Can Districts Prepare?

Professional Development on PVAAS

District Administrative Team

Building Level Administrators

Teacher Leaders

Classroom Teachers

Other Key Constituents

Communications to Board and Community


Supports

Communications KitsPosted on PDE Website

Professional Development Materials

Professional Development SupportsStaff Trained:

PVAAS Statewide Core Team/PATTAN

PDE Staff

IU Staff

Distinguished Educators

PVAAS Statewide Work Group Members


For Information, Resources or Training

PVAAS Core Team:

[email protected]

717-560-4611

Pennsylvania Department of Education PROGRESS & & ACHIEVEMENT Pennsylvania Value-Added Assessment...

Documents

Transcript of Pennsylvania Department of Education PROGRESS & & ACHIEVEMENT Pennsylvania Value-Added Assessment...