Questions to:Kristen Lewald, Ed.D.

kristen_lewald@iu13.org

PVAAS Statewide Core Team

PVAAS Update to PAIUCurriculum Coordinators

December 2010

Final Numbers

2,400+ Participants!350+ Districts and

Charters!Thank you for all of your efforts in making PVAAS

consultations a success again this year!

So…where are we now?

PVAAS Webinars – Archived & Available!

PVAAS webinars that took place in August and September 2010 are still available for viewing!1. Go to http://elluminate.iu13.org click on

Recordings 2. Click on the month the webinar was originally

held using the calendar

Ideas for Follow-up to Fall Sessions with Districts

Faculty RolloutUSING the projections

Ex. Are the right students getting interventions? Ex. Scheduling current 8th graders

Use grade 7 to 8 projections; more reliable at predicting future performance than most recent PSSA

High Growth schools forums By grades, subjects Based on Multiple years quintiles document

Need support for follow-up with local districts?

SY2009-2010 Scatterplots

As of November 4, 2010

scatterplots became

available for every IU via a

secure document library on IU 13’s SharePoint site.

Trouble with scatterplot files?

Contact pdepvaas@iu13.or

g

Single-Year High Growth & Achievement Quintiles

As of November 15,

2010

Single-Year Quintile files

became available for

every IU via a secure

document library on IU

13’s SharePoint site.

Trouble with quintile files?

Contact pdepvaas@iu13.

org

Single Year of Quintile 5 Growth and/or Quintile 5 Achievement

    Gain/Growth

Achievement

Include:5 in both growth & achievement

School  A

5 5

Include:5 in growth but not achievement

School   B

5 1, 2, 3 or 4

Include:5 in achievement but not growth

School  C

  5

NOT included:Does not have 5 in growth nor in achievement

School  D 1, 2, 3 or 4 1, 2, 3 or 4

Multiple Years (2 or 3 yrs) of Quintile 5 for Growth

Available NOW!!!

Consecutive Year High Growth

Quintile files will be posted for every IU via a secure document

library on IU 13’s

SharePoint site.

Consecutive years of Quintile 5 Growth Results

    SY09-10

SY08-09

SY07-08

Include/last 2 consecutive years

School  A

5 5  

Include/last 3 consecutive years

School   B

5 5 5

NOT included, 2/3 non-consecutive years

School  C

  5 5

NOT included, 2/3 non-consecutive years

School  D

5   5

Currently Enrolled Submission: Window 1 & 2

• 100+ Districts participated in the first and/or second submission window

• What this does for them?

• Future Enrollment Deadline• January 7, 2011

• Target 2-3 districts in your region to encourage their use of this FREE, OPTIONAL service

• Those with the highest mobility will benefit the most• All middle schools and all high schools benefit

Currently Enrolled via Projection Summaries

How can districts use this report?

Currently Enrolled via Student Search

New feature on Student Search to include Currently Enrolled:

Projection Summaries: Projections to Advanced

NEW!!!!!

Graduation Rate now in PVAAS!

Graduation rates are now included in the School Search feature within PVAAS

WHY? Growth results on grade 11 reports are on

those students that did not drop out by the end of grade 11 testing

Schools should also consider how many students DID drop out along the way from the end of grade 8 to the end of grade 11 when considering this report

Graduation Rate now in PVAAS!

Graduation rates are now included in the School Search feature within PVAAS

Getting Results Worksheet PDFs

Now available in the PVAAS “Reports” menu!

• MISSING DATA• MEASUREMENT ERROR

Issues with SIMPLE, TRANSPARENT comparisons of PSSA scores at the

District, School, Student levels

Issues with SIMPLE, TRANSPARENT comparisons of PSSA scores at the District, School, Student

levels

• The issue of missing data is a significant issue for districts/school, meaning students who do not have a PSSA score for 2 consecutive years. They missed a test.

• A simple comparison of current and prior scores could only occur for students who have those 2 test scores. Other students would not be included from current/prior year analyses, meaning many students would not be included which would impact the results.

• Think of a district and the impact of this on their growth measures. Simple methods exclude these students who are mobile/transient as they have missing test scores. They have a history of what is called fractured records. It really impacts subgroups too as they have a history of fractured records like the ELL subgroup for example.

• PVAAS is able to address this issue of missing data as it is a longitudinal analysis that uses all available prior data on students.

Issues with SIMPLE, TRANSPARENT comparisons of PSSA scores

• There is too much measurement error on any one given test score or a pair of test scores.

• Using a simple difference of 2 scores, or 2 sets of scores, creates bias in the measures.

• Measurement error is a type of "noise". It is a noise around a value that is trying to measure some other value.

• For instance, when we have a test score for a student, it is trying to measure the true knowledge of that student in a particular subject/grade. However, there is measurement error or noise in that test score.

• This is a normal thing in all assessments; some have more "noise" than others. For example, teacher made tests…VERY noisy!

• This issue of error/noise can be seen on a student’s PSSA report that goes home to mom and dad. See the example below and on next slide- this is a student’s actual recent PSSA report from SY09-10.

• His math scaled score on the PSSA was a 1720-- HOWEVER, read the statement below the number line…..

• Almost a 200 point range!!! SO what is his true understanding of mathematics...???

Issues with SIMPLE, TRANSPARENT comparisons of PSSA scores

Individual Student Report

• This is a normal issue for assessment, but a reality that simple, transparent models like the comparison of 2 sets of scores does not address. • Think why we all took the SAT more than one time!!! To have measurement error hopefully work to our advantage and get a higher score the 2nd time we took it or the 3rd time……. • The PVAAS methodology uses all available test scores on each student to mitigate the effects of measurement error on the analyses; basically to mitigate these kinds of issues.

Issues with SIMPLE, TRANSPARENT comparisons of PSSA scores

Handout for YOU to give to YOUR Districts

Summary handout for you to give to your districts from these updates Superintendents Asst Supt/Directors of C & I Directors of Special Ed Principals Teacher Leaders

Will send email with document enclosedWill also post on Sharepoint

pdepvaas@iu13.org

717-606-1911

Questions? Unmet Needs?