PVAAS F2F Overview:Evaluating Growth,

Projecting Performance

Fall 2013

F2F Overview 2013

Goal of SessionBy the end of the session, participants will be able to answer the

following questions:

• What is PVAAS?

• How does PVAAS measure growth?

• What does PVAAS tell us about growth in our district and school(s)?

• What does PVAAS tell us about how students are projected to perform on a future state assessment?

2

F2F Overview 2013

WHAT IS PVAAS?

3

F2F Overview 2013 4

What is PVAAS?

• Longitudinal analysis of existing state assessment data

• Reliable estimates of academic growth & projections of future performance

• Effectiveness of the educational system

F2F Overview 2013 5

What is PVAAS?

• Growth of a group of students over time• NOT a simple comparison of test scores

Grade 3

Grade 4

Grade 5 Grade 6

F2F Overview 2013

PVAAS Addresses Complex Issues!

Research-based Statistical Modeling

•Minimizes the effects of measurement error associated with any one test scoreMeasurement

Error

•Accounts for mobility of studentsMobility

•Compensates for missing dataMissing Data

6

F2F Overview 2013

Demographics and Growth

As long as you can sufficiently account for the prior achievement of the student,…

7

…there is NO significant relationship between demographics and student progress, or growth!

F2F Overview 2013

Prior AchievementHow can we adequately account for prior achievement?

• The SAS EVAAS team has found that the only way to adequately account for prior achievement is to use ALL available test data for each student to dampen the effects of measurement error.

• PVAAS does NOT need to adjust for the demographic status of students directly since it is accounting for students’ prior achievement.– In other words, demographics are inherently controlled for in

PVAAS by the use of all longitudinal data on all students.

8

ActivityWork with a partner to decide if each statement is true or false.

9F2F Overview 2013

F2F Overview 2013 10

PVAAS Analyses

PSSA Writing

Keystone Exams:Algebra I, Biology,

Literature

PSSA Science

PSSA Reading

PSSA Math

LOOKING AHEADPlanning for

Students’ Needs:Student

Projections to Future Tests

LOOKING BACKEvaluating School

Effectiveness:Value Added &

Diagnostic Reports

What is PVAAS?

F2F Overview 2013

PVAAS Value-Added ReportingHow is growth measured in PVAAS?

11

PVAASValue-Added Reporting

Testing Subjects – Grades Methodologies

Assessments in consecutive

years

Mathematics, 4-8Reading, 4-8 Growth Standard

Assessments not in consecutive

years

Writing – 5 & 8 Science – 4 & 8

Keystones – Algebra I, Biology, & Literature

Predictive

12F2F Overview 2013

F2F Overview 2013

Measuring Academic Growth

Students don’t all start the year at the same place academically

13

F2F Overview 2013

Measuring Academic Growth

• Standard for PA Academic Growth– Regardless of their entering achievement level, students

should not lose ground academically

• Is it not a reasonable goal to maintain the achievement level of a group of students?– Not necessarily increase their achievement level, but

maintain their achievement level.

14

F2F Overview 2013 15

PVAAS Value-AddedGrowth Indicators

F2F Overview 2013

Measuring Growth in PVAAS• Remember, PVAAS does NOT compare the

achievement of one group of students in a particular subject and grade with the previous year’s students in the same subject and grade.

• PVAAS does NOT use the percentages of students at various academic performance levels to measure growth.

• PVAAS uses students’ scale scores within those performance levels to better estimate growth.

16

F2F Overview 2013 17

Growth Measures Not Reportedfor Prior Years

• PA’s state assessment system is transitioning to include Keystone end-of-course exams and statewide assessments aligned to the PA Common Core over the next several years.

• Growth Measures in prior years will NOT be included in fall 2013 reporting (SY12-13).

• Anticipated that Growth Measures from up to 3 previous years with 3-year averages will be yielded and reported beginning with fall 2015 reporting (SY14-15).

F2F Overview 2013

PVAAS Value-AddedGrowth Measures

• For Science, Writing and Keystone content areas, Growth is measured in scale score points.

• The Growth Measure is NOT reported in years and cannot be referenced in terms of number of year’s growth.

• The Growth Measure is NOT reported in percentages and cannot be referenced in terms of an increase or decrease in percentage points.

18

F2F Overview 2013

Purpose of Value-Added & Performance Diagnostic Reports

Assessing curricular needs across grades, subjects, and Keystone content areas

Evaluating instructional needs of students at various achievement levels and in different subgroups

Ensuring instructional coherence/alignment• Curriculum Instruction Assessment• Supports, including scheduling and staffing needs

Monitoring implementation and effectiveness of grade-level or course-specific programs

19

F2F Overview 2013

What Are Projections?

Given a specific set of circumstances…

…what’s the most likely outcome?

20

F2F Overview 2013

Setting Expectations

How do teachers set expectations for their students?

• Informationo What do they know about this

student?

• Experienceo How have students like this one

performed in the past?

21

F2F Overview 2013 22

PVAAS ProjectionsWhat are they based on?

Expectations based on what we know:

About this student:

»Prior test scores

»Across subjects

F2F Overview 2013 23

PVAAS ProjectionsWhat are they based on?

About other students:

» Prior test scores across subjects

» Scores on the test to which we are projecting

F2F Overview 2013

Given this student’s testing history, across subjects…

…what is the student likely to score on an upcoming test, assuming the average schooling experience?

What Are PVAAS Projections?

24

F2F Overview 2013

Student Projection Methodology

What data are used in calculating projections?

25

Projection TO…. Data Used to Calculate ProjectionPSSA Math PSSA Math and ReadingPSSA Reading PSSA Math and ReadingPSSA Science PSSA Math, Reading, and Science (in grades available)PSSA Writing PSSA Math, Reading, and Writing (in grades available)Keystone content areas (Algebra I, Literature, Biology)

PSSA Math, Reading, and Science (in grades available)

F2F Overview 2013

Student Projection Methodology

Which students have projections? In what subjects?

– Projections are available for all students who have a minimum of 3 data points. The exception to this are projections to grade 4 where only 2 data points from grade 3 are needed.

– The next two slides detail the grade levels, subjects, and Keystone content areas in which projections are reported.

26

PVAAS Projection Reporting: Projections to Keystones (Basic, Proficient, Advanced)

27

• Anticipating projections to future Keystone exams will be available to students who last tested in grade 5

• Remember, projections to a Keystone exam are NOT grade specific

• The projection probability indicates the probability of a student reaching the indicated performance level on a future Keystone exam – not necessarily at a specific grade level.

F2F Overview 2013

F2F Overview 2013

PVAAS Student Projection Report

28

F2F Overview 2013

PVAAS Student Projection Report

29

F2F Overview 2013

PVAAS Student Projection Report

30

F2F Overview 2013 31

LOOKING AHEAD

LOOKING BACK

F2F Overview 2013

PA’s Intent for PVAAS

• Measure of Growth to Add to Existing Achievement Measures

• Indicator of Likelihood of Future Student Performance to Add to Current Achievement Data

• Another Tool for Continuous Improvement

• Not to Be Used in Isolation

32

F2F Overview 2013 33http://www.education.state.pa.us/pvaas

Additional PVAAS Supports

www.pde.state.pa.us

PENNSYLVANIAValue Added Assessment System

______________________________________________________________________

PVAAS

333 Market Street Harrisburg, PA 17126