PVAAS Statewide Core Team Fall, 2011

72
Pennsylvania Value-Added Assessment System (PVAAS): Effectively Communicate PVAAS with your Board and Community PVAAS Statewide Core Team Fall, 2011

description

Pennsylvania Value-Added Assessment System (PVAAS): Effectively Communicate PVAAS with your Board and Community. PVAAS Statewide Core Team Fall, 2011. Session Description. - PowerPoint PPT Presentation

Transcript of PVAAS Statewide Core Team Fall, 2011

Page 1: PVAAS Statewide Core Team Fall, 2011

Pennsylvania Value-Added Assessment System (PVAAS):

Effectively Communicate PVAAS with your Board and Community

PVAAS Statewide Core TeamFall, 2011

Page 2: PVAAS Statewide Core Team Fall, 2011

Session Description This 60 minute session will provide strategies and

approaches for discussing PVAAS with your local school board and community, as well as communicating PVAAS results. Approaches that have been used by actual districts in Pennsylvania will be discussed. Materials and resources to use as tools for communication about PVAAS to your local school board and community will be reviewed. Opportunities for questions and discussion will be provided.

There are trainer notes included for this presentation.

Page 3: PVAAS Statewide Core Team Fall, 2011

Agenda Documents & Suggested Handouts to

use with School Board Members Considerations for Communicating

PVAAS to School Boards and Your Community

PVAAS Public Reporting Site Examining the School Board

PowerPoint template Questions

Page 4: PVAAS Statewide Core Team Fall, 2011

Suggested Approach for InitialSession with School Board

Keep it simple…keep it conceptual Discuss what PVAAS is and is not

• Start with a focus on Scatter Plots: Achievement vs. Growth Summarize important reports!

• Suggest starting with Reading and Math only• District Value-Added Report• Value-Added Summary Report (building level

results)• District (Single Grade) Projection Summary Report• Performance Diagnostic Summary Report (if needed)

Go For The BIG IDEAS!

Page 5: PVAAS Statewide Core Team Fall, 2011

Suggested Approach for Discussion with Community Members

Keep it simple…keep it conceptual Discuss what PVAAS is and is not Start with a focus on Scatter Plots: Achievement vs.

Growth (Across Grades Only!) Summarize important information!

• District & School Value-Added Across Grades Only

• School Search Average Growth Index Across Grades Only

Go For The BIG IDEAS!

Page 6: PVAAS Statewide Core Team Fall, 2011

Supports and Resources to Prepare & Present at

PVAAS to School Board Members and/or Community

Page 7: PVAAS Statewide Core Team Fall, 2011

Show: DVD for School Boards

PVAAS for School Board Members

by School Board Members

Tom Gentzel, PSBA 2 Superintendents

School Board Members 12 minutes

Email [email protected] for a copy of the DVD

Page 8: PVAAS Statewide Core Team Fall, 2011

Suggested Resources & Documents for Board Members

Copy of PPT Presentation

Guides to PVAAS Reporting 2011• Guide to PVAAS Reporting for LEAS 2011• Guide to PVAAS Public Reporting 2011

Page 9: PVAAS Statewide Core Team Fall, 2011

Suggested Resources & Documents for Board Members

Copies of Key District Reports• District Value-Added Summary Reports• District Value-Added Reports• District Projection Summary Reports

District Scatter Plots• Web-based Scatter Plots (demonstrate on

public reporting site) Grade level or Across Grades

Page 10: PVAAS Statewide Core Team Fall, 2011

Example

Page 11: PVAAS Statewide Core Team Fall, 2011

PDE PVAAS Webpage

Help Menus, PPT Presentations with Trainer Notes, Resource Materials, Video Clips, & District Case Studies

Page 12: PVAAS Statewide Core Team Fall, 2011

SAS PVAAS Website

New & Archived Webinars, Podcasts, Guide to PVAAS Reporting for LEAs, Guide to PVAAS Public Reporting,

& Intermediate Unit Supports

https://pvaas.sas.com

Page 13: PVAAS Statewide Core Team Fall, 2011

Key Concepts for EVERY

PVAAS Session

Page 14: PVAAS Statewide Core Team Fall, 2011

Key Concepts: ANY PVAAS Session

Achievement Progress Measures students’

performance at a single point in time

Compares student performance to a standard

Critical to students’ post-secondary opportunities

Highly correlated with demographic factors, such as socioeconomic status

Measures students’ progress across time/years

Compares student performance to his/her own prior performance

Critical to ensuring students’ future academic success

Not correlated with demographic factors

Page 15: PVAAS Statewide Core Team Fall, 2011

Key Concept:ANY PVAAS Session

PVAAS reporting reflects the effectiveness of your district’s or school’s Standards-Aligned System

PVAAS reporting reflects the district’s or school’s system regarding curriculum, assessment, and instruction

Page 16: PVAAS Statewide Core Team Fall, 2011

Key Concepts: ANY PVAAS Session

There is NO relationship between demographic variables such as socioeconomic status and growth.

• There are high achieving schools making high growth;• There are high achieving schools making low growth;• There are low achieving schools making high growth; • There are low achieving schools making low growth.

Growth reporting reflects what WE do with students in terms of academic growth in schools/districts.

-- VAAS can remove the effects of factors not under the control of the school. (McCaffrey, Lockwood, Koretz & Hamilton, 2003; Ross, Wang, Sanders, Wright & Stringfield, 1999a; Wright, Horn & Sanders, 1997).

Page 17: PVAAS Statewide Core Team Fall, 2011

Key Concept:ANY PVAAS Session

PVAAS District and School Value-Added Reports do NOT provide a measure of teacher effectiveness!

Often people ask about the meaning of these reports, especially in situations where there is only one classroom teacher in a grade level. This is NOT a report on the effectiveness of the classroom teacher, even in that situation!

Teacher effectiveness reporting would be done using a different statistical analyses than the one used for reporting on district or school effectiveness. Additional data would be collected as well.

Page 18: PVAAS Statewide Core Team Fall, 2011

DECISION POINTS

Who? - What? -When? - How?

Page 19: PVAAS Statewide Core Team Fall, 2011

Decision Point: Audience

• What are the backgrounds of school board members?

• Do the board members have experience with growth data/PVAAS?

• Do the board members have any background knowledge/foundation in Statistics?

Page 20: PVAAS Statewide Core Team Fall, 2011

Decision Point: Intended Goal for Session (s)

PVAAS Overview?

Review of Actual District/School Data?

Part of Planning Process?• Comprehensive Planning• School Improvement Planning

Page 21: PVAAS Statewide Core Team Fall, 2011

Decision Point: Focus of Data

District? School? Grade? Subgroups? Which subjects?

• Math, Reading, Science, Writing Remember, less is more!

Page 22: PVAAS Statewide Core Team Fall, 2011

Decision Point: Reports NeededDistrict-wide Focus

• Scatter Plots• District Value-Added

Think ahead if there is only one building per grade

• Value-Added Summary/Individual schools• District (Single Grade) Projection Summary

Reports• (Optional) Performance Diagnostic Summary

If your audience is interested in growth of students at different achievement levels

Page 23: PVAAS Statewide Core Team Fall, 2011

Examples of Reports

Page 24: PVAAS Statewide Core Team Fall, 2011

Talking About Your Results

Think about your message before sharing your results! What do the results

mean? Do you have a plan of

action regarding the areas needing improvement (Yellow and Red)?

What is your plan of action for attending to the Green, Light Blue, and Dark Blue areas?

Page 25: PVAAS Statewide Core Team Fall, 2011

Talking About Your Results

Think about your message before sharing your results!

What do the results mean? Do you have a plan of action regarding the areas

needing improvement (Yellow and Red)? What is your plan of action for attending to the

Green, Light Blue, and Dark Blue areas?

Page 26: PVAAS Statewide Core Team Fall, 2011

Decision Point: Session Facilitators(s)

Principal Team of Principals District Administrator(s) District Administrator(s) + Principal(s) Support from Intermediate Unit PVAAS

Contacts PVAAS Statewide Team for PDE

• Regional sessions for district administrators and School Board members

Page 27: PVAAS Statewide Core Team Fall, 2011

OTHER CONSIDERATIONS

Page 28: PVAAS Statewide Core Team Fall, 2011

PVAAS Projections are an Approved Way for Schools

to Make AYP(Growth Model)

Page 29: PVAAS Statewide Core Team Fall, 2011

PVAAS Public Reporting Site Value-Added (Growth) Reports at District and

School Levels• Math, Reading, Science, and Writing• Grades 4 through 8, and 9-11

School Search Capability• Allows users to find and view the achievement

and progress of groups of students in public schools across the Commonwealth of Pennsylvania.

• Can search for similar schools based on grade levels tested and various demographics.

Web-based scatter plots

Page 30: PVAAS Statewide Core Team Fall, 2011

Preparing for Parent Requests

Parents may ask about data or students.

What is available at the student level? Projections!• Percent likelihood of a student reaching Basic,

Proficient and/or Advanced levels on future PSSA

Some PA districts have shared PVAAS student projections with parents.

Plan in advance for how to respond such requests

Page 31: PVAAS Statewide Core Team Fall, 2011

Questions?

Comments?

Email [email protected] for a PowerPoint version of this

session.

Page 32: PVAAS Statewide Core Team Fall, 2011

Presenting PVAAS to an Individual School Board Power Point Template

with Trainer Notes

Page 33: PVAAS Statewide Core Team Fall, 2011

Pennsylvania Value-Added Assessment System (PVAAS):A Compass for Improvement

Presented at [District] School Board Meeting[Date]

Page 34: PVAAS Statewide Core Team Fall, 2011

DVD for School Boards

PVAAS for School Board Members

by School Board Members

Tom Gentzel, PSBA 2 Superintendents

School Board Members 12 minutes

Page 35: PVAAS Statewide Core Team Fall, 2011

TWO Types of PVAAS Information

Today

Looking Back/Evaluation…

Value-added Growth Reports For Groups of Students

Looking Forward/Planning…

PVAAS Projection Reports For Individual Students and Groups of Students

Page 36: PVAAS Statewide Core Team Fall, 2011

PVAAS IS: Analysis of existing assessment data

• Longitudinal PSSA scores in available content areas

Math, Reading, Science, Writing• Provides reliable estimates

A measure of a school’s effectiveness

Critical Relationship: Growth Data vs. Achievement

Data

Page 37: PVAAS Statewide Core Team Fall, 2011

PVAAS:

Is not another test

Critical concept• Is not a comparison of

single test scores

Page 38: PVAAS Statewide Core Team Fall, 2011

Review of Literature

Is there a relationship between… Demographics vs. Achievement?

YES

Demographics vs. Growth?NO

Demographics are inherently controlled for in PVAAS due to the use of all longitudinal data on students!

Page 39: PVAAS Statewide Core Team Fall, 2011

High Achievement

Average Achievement

Low Achievement

Current Data Collection Approaches

Achievement is based on each student’s ending point.

Page 40: PVAAS Statewide Core Team Fall, 2011

PVAAS Adds Another Variable To Achievement…GROWTH!

Below Growth

Standard

At or AboveGrowth

Standard

Growth is based on each student’s starting point!!

Page 41: PVAAS Statewide Core Team Fall, 2011

Viewing Achievement and Progress Together

PVAAS Web-based Scatter Plots

Page 42: PVAAS Statewide Core Team Fall, 2011

Scatter Plot Quadrants and Their Meanings

Quad 2 - SlippingHigher

AchievementLower Growth

Quad 1 - Excelling

Higher Achievement

Higher Growth

Quad 4 - Improving

Lower Achievement Higher Growth

Quad 3 - Underperforming

Lower AchievementLower Growth

Page 43: PVAAS Statewide Core Team Fall, 2011

Example: Regional Scatter Plot

Page 44: PVAAS Statewide Core Team Fall, 2011

Example: Regional Scatter Plot: Same Schools, Different Subject

Page 45: PVAAS Statewide Core Team Fall, 2011

Web-based Scatter Plotson PVAAS Public Site

Page 46: PVAAS Statewide Core Team Fall, 2011

Example

Page 47: PVAAS Statewide Core Team Fall, 2011

District Scatter Plots

Math and Reading

Page 48: PVAAS Statewide Core Team Fall, 2011

District Scatter Plot – Grade 4Math and Reading

Page 49: PVAAS Statewide Core Team Fall, 2011

District Scatter Plot – Grade 5Math and Reading

Page 50: PVAAS Statewide Core Team Fall, 2011

District Scatter Plot – Grade 6Math and Reading

Page 51: PVAAS Statewide Core Team Fall, 2011

District Scatter Plot – Grade 7Math and Reading

Page 52: PVAAS Statewide Core Team Fall, 2011

District Scatter Plot – Grade 8Math and Reading

Page 53: PVAAS Statewide Core Team Fall, 2011

District Scatter Plot – Grade 11Math and Reading

Page 54: PVAAS Statewide Core Team Fall, 2011

Overview of PVAAS Reporting

Page 55: PVAAS Statewide Core Team Fall, 2011

PVAAS Reporting District Level Reporting School Level Reporting Grade Level Reporting

• Reading (Grades 4-8, 11)• Math (Grades 4-8, 11)• Science (Grades 4, 8, & 11)• Writing (Grades 5, 8, & 11)

Subgroup Reporting (ex. IEP, ED, ELL) Student Level Reporting

• Projections to Future PSSA Performance• Reading, Math, Science, Writing• Grades vary based on subject area

Page 56: PVAAS Statewide Core Team Fall, 2011

Have all of our groups of students met the standard for PA Academic Growth?

Grades 4-8?Grades 9-11?

Value-Added Growth Reports

Page 57: PVAAS Statewide Core Team Fall, 2011

PVAAS Value-AddedGrowth Descriptors

DB

Significant evidence that the district/school exceeded the standard for PA Academic Growth

LBModerate evidence that the district/school exceeded the standard for PA Academic Growth

GEvidence that the district/school met the standard for PA Academic Growth

YModerate evidence that the district/school did not meet the standard for PA Academic Growth

RSignificant evidence that the district/school did not meet the standard for PA Academic Growth

Page 58: PVAAS Statewide Core Team Fall, 2011

District/School Value-added ReportGrades 4 through 8: Math & Reading

Question for Educators:Are all grade levels meeting the standard for PA Academic Growth?

NOT on Public Site

Page 59: PVAAS Statewide Core Team Fall, 2011

District/School Value-added ReportGrades 9-11 Math & Reading; Science; & Writing

Question for Educators:Are all grade levels meeting the standard for PA Academic Growth?

NOT on Public Site

Page 60: PVAAS Statewide Core Team Fall, 2011

District Value-added Summary Report

Question for Educators: Which schools at which grade levels are exceeding the standard for PA Academic Growth?

NOT on Public Site

Page 61: PVAAS Statewide Core Team Fall, 2011

How well are we supporting our high achieving students or low achieving

students?

Performance Diagnostic Summary Reports

Page 62: PVAAS Statewide Core Team Fall, 2011

Performance Diagnostic Growth Descriptors,

Grade 4-8, Math/Reading

Exceeded the standardfor PA Academic Growth

Met the standardfor PA Academic Growth

Did not meet the standard for PA Academic Growth

Page 63: PVAAS Statewide Core Team Fall, 2011

Performance Diagnostic Summary Report

Questions for Educators: How effective are schools across the district in making academic progress with groups of students at different predicted performance levels? Are there difference in growth patterns across schools?

NOT on Public Site

Page 64: PVAAS Statewide Core Team Fall, 2011

What is the likelihood that students will be reach a Proficient level or

higher on a future PSSA? What is their likelihood of reaching an

Advanced level?

PVAAS Projection Summary ReportsPVAAS Student Projection Reports

Page 65: PVAAS Statewide Core Team Fall, 2011

Student Projection Methodology

Is this the same methodology as the one used to estimate growth?• No, the projection methodology is a separate modeling

process that focuses on individual students.

What data are used in this methodology?• ALL available longitudinal data in both Reading and Math

are used in projection calculations for all four subjects.

What students have projections? In what subjects?• Projections are available for all students who have a

minimum of 2 years of historical data. The next slide details the grade levels and subjects in which projections are reported.

Page 66: PVAAS Statewide Core Team Fall, 2011

PVAAS Fall 2011 Reporting: Projections to Basic, Proficient, Advanced

Math and Reading: 3 4 or 5 4 5 or 6 5 6 or 7 6 7 or 8 7 8 8 11

Writing: 3 5 4 5 5 8 6 8 7 8 8 11

Science: 3 4 4 NA 5 8 6 8 7 8 8 11

Grade PSSA

Last Taken

Grade PSSA

Last Taken

Grade PSSA

Last Taken

Page 67: PVAAS Statewide Core Team Fall, 2011

NOT on Public Site

District (Single Grade)Projection Summary Report

Page 68: PVAAS Statewide Core Team Fall, 2011

PVAAS Student Projection Report

NOT on Public Site

Page 69: PVAAS Statewide Core Team Fall, 2011

PVAAS Help Menus

Page 70: PVAAS Statewide Core Team Fall, 2011

Questions:

PVAAS Materials or Statewide Implementation

[email protected]

717-606-1911

PVAAS Report Web Site

https://pvaas.sas.com

Page 71: PVAAS Statewide Core Team Fall, 2011

www.pde.state.pa.us

333 Market Street Harrisburg, PA 17126

Page 72: PVAAS Statewide Core Team Fall, 2011

Questions from this webinar session?

[email protected]

717-606-1911