Post on 06-Feb-2016
description
Pennsylvania Value-Added Assessment System (PVAAS):
PVAAS Public Site Tour
PVAAS Statewide Team for PDEFall 2011
Session Description This 60 minute session will provide an overview of
the PVAAS public reporting site.
All data reported on the public site will be described as to the intent and meaning of the results. This includes the district and school level data, the School Search feature and the web-based scatter plots.
Materials and resources to use as tools for communication about PVAAS will be reviewed.
Opportunities for questions and discussion will be provided.
Agenda
Achievement and Progress Data What’s New? Available Data & Features Website Demonstration Suggested Resources for
Communicating PVAAS Questions
Achievement vs. Progress
Achievement
The final result of an academic experience
Highly correlated with demographic factors, such as socioeconomic status
Affected by factors outside the school
Progress
Dependent upon what happens as a result of schooling
Not correlated with demographic factors
Is the concept underlying value-added analysis and reporting
Achievement vs. ProgressAchievement
Measures students’ performance at a single point in time
Compares student performance to a standard
Critical to students’ post-secondary opportunities
Progress
Measures students’ progress across time/years
Compares student performance to his/her own prior performance
Critical to ensuring students’ future and continued academic success
Stair-Step Expectations
Grade Three Grade Four Grade Five Grade 3 Test Grade 4 Test Grade 5 Test
In a perfect world students would start school at about the same academic level. Proficiency levels could simply be achieved by students gaining age
appropriate knowledge and skills each year.
Differentiated Reality
Grade Three Grade Four Grade Five
Students begin school at different places, and they often progress at different rates. Yet all students
are expected to achieve the same level of academic proficiency each year.
Grade 3 Test Grade 4 Test Grade 5 Test
The Need for Progress Measures
Grade Three Grade Four Grade Five
To measure school effectiveness in this “differentiated” world, we need to pay attention not
only to proficiency levels but also to how much progress students make in a given year.
Grade 3 Test Grade 4 Test Grade 5 Test
What is Value-Added?
Imagine a child’s physical growth curve
Parent or doctor measures the child’s height at various ages
A graph can be constructed to illustrate the height of the child as shown here
What is Value-Added? Let’s apply the same
process to education
This chart measures the growth, or progress, for a group of students at various grade levels
This is value-added following the growth, or progress, of students over time to estimate growth during a year of schooling.
With PVAAS, Pennsylvania educators get an indicator as to whether they are
making appropriate academic progress for their students!
Value-Added is… A statistical analysis used to measure a district’s
or school’s impact on the academic progress rates of groups of students from year to year.
Conceptually, a growth measure is approximately the difference between current achievement (current results) and prior achievement (prior results) with achievement being measured by an appropriate assessment, such as the PSSA.
PVAAS is NOT a simple comparison of two scores!
Value-Added & PVAAS
Pennsylvania’s model for value-added is called PVAAS - the Pennsylvania Value-Added Assessment System.
PVAAS is based on the EVAAS Methodology – the Education Value-Added Assessment System.
The EVAAS methodology has been nationally reviewed and published.
Benefits of Value-Added Offers an objective, more accurate way to measure
the influence districts and schools have on students’ academic progress:
Administrators and Teachers can:• Monitor the progress of all groups of students from low-
achieving to high-achieving ensuring growth opportunities for all students
• Make more informed, data-driven decisions about where to focus resources to help students make greater progress and perform at higher levels.
• Align professional development efforts in the areas of greatest need.
• Identify best practices and implement instructional strategies and programs that best meet the needs of all students.
Key Concepts in Understanding PVAAS Reporting
PVAAS reporting reflects the effectiveness of your district’s or school’s Standards-Aligned System
PVAAS reporting reflects the district or school’s system regarding curriculum, assessment, and instruction
Overview of PVAAS Public Reports &
Features
Achievement + Growth
Achievement results (PSSA) and growth results (PVAAS) must be used together to get a complete
picture of student learning. To view the achievement results
for Pennsylvania's public districts/schools, go to:
http://paayp.emetric.net/
Public Home Screen
PVAAS Public Reports
Use of Reports Tab to Select & View Reports
PVAAS Public Reports
Use of Tests Tab to View Reports at Different Grade Levels for the District and School Value-Added Summaries
PVAAS Public Reports Value-Added Summary Reports
• District/LEA and School Level data only• Math and Reading
Grades 4-8 and 9-11• Science and Writing, Grades 9-11
School Search Capability• Allows users to find and view the progress of
local schools, charter schools, and full-time CTCs across Pennsylvania.
• Can search for similar schools based on grade levels tested, various demographics, Intermediate Unit (IU) region and/or county.
PVAAS Public Reports (continued)
Web-based scatter plots • Math, Reading
Grades 4-8 & Grades 9-11• Science and Writing, Grades 9-11
Reports on Public Site
The data on the PVAAS public site come directly from the reports on the district/school password-protected site.
This session will include a cross-walk between these two sites to show the source of the data on the public reporting.
Purpose of District & SchoolPVAAS Data
• Provides users with information to assist them in evaluating the overall effectiveness of a district/LEA or school on the academic progress of groups of students.
• This report is NOT a report on teacher effectiveness!
What is New?
Since Feb 2011 Public Reporting Release:• New: Web-based Scatterplots• New: Color Scheme/Legends• New: Addition of Science and Writing for
Grades 9-11 • New: Rollover Text Feature• New: Redesigned School Search Feature
Web-based Scatter Plots
Accessing Scatter Plots
The scatter plot report can be accessed directly within the orange “Reports” tab
Customizing Scatter Plots
Users can switch
subjects
Users can also switch
the the X and Y Axis
Roll-Over Feature!
Website Demonstration
https://pvaas.sas.com
District and School Value-Added Reports
PVAAS Value-added Growth Descriptors
Grades 4-8 Math & ReadingGrades 9-11 Math, Reading, Science & Writing
Example of District Value-Added Summary Report
Grades 4-8, Math & Reading
Example of School Value-Added Summary Report
Grades 4-8, Math & Reading
Where can I find this information on other reports?
Public Site: School Value-Added Summary Report
Restricted Site: School Value-Added Report
Example: Growth Measure over Grades Relative to the Growth Standard
Growth Measure over Grades Relative to the Growth Standard (on the public report) is the SAME as the Growth Measure over Grades Relative to the Growth
Standard on the School Value-Added Report (password-protected site)!
3.24th Grade Growth
0.05th Grade Growth
2 1.6 Growth Measure over Grades Relative to Growth Standard
What is the Growth Measure over Grades Relative to the State?
• Represents the average growth across the grade levels served between 4 and 8 compared to the average progress of all students in Pennsylvania at the same grade levels.
• It is the average academic growth of the district’s or school’s students, compared to the academic growth of students statewide.
• Answers the question, “How much did the district/LEA/school impact the academic progress of its students compared to the progress of other students in Pennsylvania in those same grade levels?”
Example: Growth Measure over Grades Relative to the State
Growth Measure over Grades Relative to the State (on the public report) is the SAME as the Growth Measure over Grades Relative to the State on the School
Value-Added Report (password-protected site)!
1.6Growth Measure over Grades Relative to Growth Standard
for the school
0.9 Growth Measure over Grades Relative to the State
?Average Growth over
Grades Relative to Growth Standard for the State
What is Growth Measure over Grades Relative to the Growth Standard for the State?
2.9Grade 4 State
3-Yr-Avg
-1.4Grade 5 State
3-Yr-Avg
2 .75 Average Growth over Grades Relative to Growth Standard for the State
Example: Growth Measure over Grades Relative to the State
Growth Measure over Grades Relative to the State (on the public report) is the SAME as the Growth Measure over Grades Relative to the State on the School
Value-Added Report (password-protected site)!
1.6Growth Measure over Grades Relative to Growth Standard
for the school
0.9 Growth Measure over Grades Relative to the State
.75Average Growth over
Grades Relative to Growth Standard for the State
Example of District Value-Added Summary Report
Grades 9-11, Math, Reading, Science & Writing
Example of School Value-Added Summary Report
Grades 9-11, Math, Reading, Science & Writing
Where can I find this information on other reports?
Public Site: School Value-Added Summary Report
Restricted Site: School Value-Added Report
Is it appropriate to compare the amount of progress made by a district/school to
another district/school?
• Without taking the Standard Error into account, it is NOT possible to directly compare growth measures across districts/schools.
• The color-coding of the growth measures does in fact take into account the Standard Error.
o Note the link to the color code legends on each report.
• The Average Growth Index found on the School Searchreport feature takes the Standard Error into account and allows a more direct comparison across schools.
SCHOOL SEARCH FEATURE
Purpose of School Search
• Users can find and view the progress of public schools across Pennsylvania and search for similar schools based on grade levels tested, various demographics, Intermediate Unit (IU) region, and/or county.
Which schools are included when I use School Search?
Schools with at least one tested grade in common as the “reference school” you selected.
Example: Your reference school is a grade 6-8 school• Other schools included in the search may
include grade 6-7 schools, grade 7-8 schools, K-6 schools, etc. – unless you specific a grade range.
Accessing School Search
PVAAS School Search is accessed the via links at the top of the page or using the
orange Reports tab
Predictive Text to Assist in Locating Schools
Example: the user enters the word “Pink” and school names containing the word “Pink” appear as options to select.
School Search Results
School Search Results
Finding Similar Schools
Comparison Schools
Website Demonstration
https://pvaas.sas.com
How are schools compared? PVAAS: Average Growth Index What is an Index?
• A numerical scale used to compare variables with one another or with some reference number
Analogy: Consumer Price Index• A measure of the average change over a period
of time• Statistical Indicator• Reflects patterns
The PVAAS Average Growth Index allows viewers to compare growth across schools.
What is the Average Growth Index?• A measure of student progress across the tested grade
levels in a school.
• This index is a value based on the average growth across grade levels and its relationship to the standard error so that comparison among schools is meaningful. If the standard error is not accounted for, users might get a skewed picture of the relative effectiveness of different schools.
• For grades 4 through 8, the Average Growth Index is calculated by dividing the Growth Measure over Grades Relative to the Growth Standard by the corresponding Standard Error.
• For grades 9 through 11, the Average Growth Index is calculated by dividing the Growth Measure by the corresponding Standard Error.
Example: Average Growth Index, Grades 4-8
Average Growth Index= Growth Measure over Grades Relative to the Growth Standard divided by the Standard Error
What does the Average Growth Index mean?
• Average Growth Index (AGI) = 0: The average achieving student in this school met the standard for PA Academic Growth.
• Average Growth Index (AGI) > 0: The average achieving student in this school exceeded the standard for PA Academic Growth. A larger AGI provides more evidence that the average achieving student in this school exceeded the standard for PA Academic Growth.
• Average Growth Index (AGI) < 0: The average achieving student in this school did not meet the standard for PA Academic Growth. A smaller AGI provides more evidence that the average achieving student in this school did not meet the standard for PA Academic Growth.
Why can I not find a district/school on the PVAAS public site?
Reporting was suppressed for the following schools:• Districts/schools with less than 10 students
PA public reporting requirements• Districts/schools serving one grade level
Act 104 legislative intent was district and building level data
Example: Grade 11 only school Example: Grade 11-12 schools Example: Grade 6 only school
Suggested Resources for Communicating
PVAAS
PVAAS Help Menus onPublic Reporting
PDE PVAAS Webpage
• Help Menus
• PowerPoint Presentation
swith Trainer
Notes
• Resource Materials
• Video Clips
• District Case Studies
Suggested Resources
Guide for PVAAS Public Reporting 2011• Differences between Achievement and Progress• Concept of Value-Added• Benefits of Value-Added• Description of Reporting • Frequently Asked Questions• Glossary of Key Terms & Acronyms
Available on the PVAAS login page!
24/7 Suggested Resources
Archived webinars and PowerPoint presentations detailing the public reporting site
One-page handout titled “What is PVAAS?”
PVAAS Podcasts, FREE on iTunesU• Introduction to PVAAS• Value-Added Reporting• School Search
Why PVAAS?
• Provides users with information to assist them in evaluating the overall effectiveness of a public district or public school on the academic progress of groups of students.
Questions?
Comments?
Questions:
PVAAS Materials or Statewide Implementation
pdepvaas@iu13.org
717-606-1911
PVAAS Report Web Site
https://pvaas.sas.com
www.pde.state.pa.us
333 Market Street Harrisburg, PA 17126