PVAAS School Consultation Fall 2011

PVAAS School ConsultationFall 2011

PVAAS Statewide Core [email protected]

mailto:[email protected]

PVAAS 2011 Consultation Guide

2

Purpose of Consultation• Goal: To model a process for the inspection,

analysis and interpretation of your 2011 data

• Focus: PAAYP, eMetric, and PVAAS Data for Reading and Math

• This process is designed to give a school the “30,000 foot view” of the status of achievement and growth– Whole school data– Grade level data– Subgroup data

Consultation Guide Pg. 1

10/9/2011


3

Two key questions to keep in mind as you examine your data!

Based on the system’s prior year achievement goals/targets:

– Is the system (school/district) serving all students adequately?

– Is the system (school/district) serving all students equitably?

Purpose of Consultation

10/9/2011


4

Agenda• The Power of Two: Web-Based Scatter Plots……15

min• PVAAS Reports………………………………………35

min– Value-Added Reports, Performance Diagnostic Reports,

School Projection Summary

• PA AYP Report………………………………………..5 min

• PSSA Data Interaction Reports, eMetric ………...20 min– Performance Level Summary, Reporting Categories

Summary, Matched Comparison Report• Summary and Next Steps………………………….15

min

10/9/2011

The Power of Two!

Considering Achievement and Growth TOGETHER!

C.G. Pg. 2


Quad 2 - SlippingHigher

Achievement,Lower Growth

Scatter Plot Quadrants and Their Meanings

10/9/2011 6

Quad 1 - Excelling

Higher Achievement,Higher Growth

Quad 4 - Improving

Lower Achievement, Higher Growth

Quad 3 - Underperforming

Lower Achievement,Lower Growth

C.G. Pg. 2


NEW! Web-Based Scatter Plots

7

C.G. Pg. 2

https://pvaas.sas.com

10/9/2011


8

Questions For Later Analysis1. Based upon the position of the schools in

district, is the system serving all students adequately in the chosen subject area?

2. How are the schools in the district performing in comparison to all the schools in the state?

C.G. Pg. 2

10/9/2011


9

Consultation Guide• Decide on focus for your team to make the most

of the time:– What subject you/others will focus on – Reading or Math?– What grade level(s) within the subject?– What subgroup within the grade and subject?

• Purpose is to model a process that can be used to determine the status of achievement & growth in your school.

• The next step is to begin to investigate WHY you are getting these results.

C.G. Pg. 2

10/9/2011


10

PVAAS ReportsC.G. Pg. 7

https://pvaas.sas.com

10/9/2011


11

TWO Types of PVAAS Information

Today

Looking Back/Evaluation…Value-added Growth Reports For

Groups of Students

Looking Forward/Planning…

PVAAS Projection Reports For Individual Students

and Groups of Students

C.G. Pg. 7

10/9/2011


12

School Value-Added ReportsThese reports provide information on how effective a school was in supporting groups of students to at least meet the standard for PA Academic Growth.

C.G. Pg. 7

10/9/2011


NEW! Legends and Color Codes

• Alignment of colors and interpretative legends across all Value-Added Reports (all subjects and grade levels) – New 5 color-coding scheme

• Same for Reporting 4-8 and 9-11!– All legends are consistent in color and language– Use of the data becomes easier as there is no

confusion about what the colors indicate• New 5 Color Codes

– Allow for more differentiation among schools exceeding the standard for PA Academic Growth

– Allow for more differentiation in Science and Writing reporting, as well as grades 9-11 Reading and Math reporting

1310/9/2011


1410/9/2011


NEW! PVAAS Value-addedGrowth Descriptors

1510/9/2011


16

Meeting the standard for PA Academic Growth

Growth Measure= 0• Grades 4-8 - A group meets the standard

for PA Academic Growth when they maintain their relative achievement level from one year to the next in relation to the statewide distribution (2006 baseline).

• Grades 9-11 – A group meets the standard for PA Academic Growth when their actual achievement meets their predicted achievement- based on the average schooling experience.

10/9/2011


17

Questions1. Are the students in your school making the

growth needed to meet the school’s/districts achievement goals/targets?

2. Describe the pattern of growth within each grade over the last three years.

3. What patterns do you see when examining the entering achievement level of students at each grade level over the past years (looking at achievement vertically for the grade)?

4. What do the Growth Measures tell you about the system’s success at meeting the needs of students at their achievement level?

C.G. Pg. 7

10/9/2011


18

School Value-Added Reports

Navigate & investigate your Value-Added Reports and answer the questions in your guidebook.

C.G. Pg. 7

10/9/2011


19

Performance Diagnostic Report

The Performance Diagnostic Report allows you to view growth patterns disaggregated by predicted PSSA performance levels (Below Basic, Basic, Proficient, and Advanced).

C.G. Pg. 8

10/9/2011


20

Interpretation of the Performance Diagnostic

Report

Zero (0) Line

Growth

Met standard for PA Academic Growth

Green

Less Than standard for PA Academic Growth

Pink

Exceeded standard for PA Academic Growth

Blue

What the whiskers tell us…

C.G. Pg. 8

10/9/2011


A

B

C

21

Patterns of Growth

C.G. Pg. 8

10/9/2011


22

Questions1. How does the growth pattern for the current

year (blue bars) compare to the growth pattern for previous years (gold bars) in same grade and subject?

2. What patterns of growth do you see across grades in each subject area?

3. What do these patterns suggest about the system’s success at meeting the needs of students at varying achievement levels?

4. What might these patterns suggest about the system’s curriculum, instruction, materials & resources, or interventions?

C.G. Pg. 8

10/9/2011


23

Performance Diagnostic Report

Navigate & investigate your Performance Diagnostic Reports for each grade level and complete the questions in your guidebook.

C.G. Pg. 8

10/9/2011


24

Questions for a Subgroup1. What patterns of growth do you see for

the selected subgroup across grades in each subject area?

2. Does the pattern of growth for the selected subgroup mirror the pattern of growth for the grade as a whole? Make note of any concerns.

3. What do these patterns suggest about the system’s success at meeting the needs of students at varying achievement levels within each subgroup?

C.G. Pg. 9

10/9/2011


25

Performance Diagnostic Report - Subgroups

Navigate & investigate your Performance Diagnostic Reports by subgroup for each grade level. Complete the table and answer the questions in your guidebook.

C.G. Pg. 9

10/9/2011


26

TWO Types of PVAAS Information

Today

Looking Back/Evaluation…Value-added Growth Reports For

Groups of Students

Looking Forward/Planning…

PVAAS Projection Reports For Individual Students and Groups of Students

10/9/2011


Projections to Proficient or Advanced

27

Grade PSSA Last Taken

3 4 or 54 5 or 65 6 or 76 7 or 87 88 11

Project to these Grades

10/9/2011


28

How do I know that the projections

are accurate?• The projection is precise, and is created using a model that

has been reviewed and approved by four different peer review panels and the GAO (US Government Accountability Office).

• The most recent 2008 growth model proposal to USDOE includes information regarding the statistical model and projection reliability study.(

http://www.ed.gov/admins/lead/account/growthmodel/pa/index.html)

• Recent studies have confirmed that the PVAAS projections (even as far as 3 years into the future) are more reliable at looking at the future performance of a student than the most recent PSSA score.

10/9/2011

http://www.ed.gov/admins/lead/account/growthmodel/pa/index.html

http://www.ed.gov/admins/lead/account/growthmodel/pa/index.html


29

School (Single Grade) Projection Summary Report

This report provides educators with the probability of students scoring at or above a Proficient level on a future PSSA.

C.G. Pg. 10

10/9/2011


30

Questions1. What percentage of students fall in 70%-100%

probability range of reaching proficiency? What does this suggest about the degree to which the core program is meeting the needs of the majority of your students?

2. What percentage of students fall in the 40%-70% probability range of reaching proficiency? What does this suggest about the needs for supports or interventions for students that are at risk of not reaching proficiency?

3. What percentage of student fall in the 0- 40% probability range of reaching proficiency? What does this suggest about the need for intensive supports and interventions for students in this range?

C.G. Pg. 10

10/9/2011


31

School (Single Grade) Projection Summary Report

Navigate & investigate your Single Grade Projection Summary Reports for each tested grade and answer the questions in your guidebook.

C.G. Pg. 10

10/9/2011


32

Questions for a Subgroup1. In what probability range do most of the

students in this subgroup fall in each tested grade? Greater than 70%, Between 40% to 70%, or Less than 40%?

2. What do the data suggest about the degree to which your core program is meeting the needs of the selected subgroup of students?

3. What interventions, supports, and/or services have been provided for these students?

C.G. Pg. 11

10/9/2011


33

School (Single Grade) Projection Summary Report - Subgroups

Navigate & investigate your Single Grade Projection Summary Reports by Subgroup for each tested grade and answer the questions in your guidebook.

C.G. Pg. 11

10/9/2011


34

PA AYP

http://paayp.emetric.net/

C.G. Pg. 3

10/9/2011


35

PA AYP Report• Public Site• Has a school met the AYP targets?

– Academic Performance in Reading & Math– Test Participation in Reading & Math– Attendance or Graduation

• Which student groups count for AYP purposes?– A total of 40 or more students in the tested

grades in the school

C.G. Pg. 3

10/9/2011


36

Questions1. Did your school make AYP for performance

for the whole school and all measurable subgroups? (72% Reading, 67% Math)

2. Did your school make AYP for participation for the whole school and all measurable subgroups?

3. Did the school make AYP for Attendance or Graduation?

C.G. Pg. 3

10/9/2011


37

PAAYP Data Table

Navigate & investigate your PAAYP Data Table and answer the questions in your guidebook.

2011 AYP TargetsReading – 72%Math – 67%

C.G. Pg. 3

10/9/2011


38

PSSA Data Interaction – eMetric

https://solutions1.emetric.net/pssa/

C.G. Pg. 4

10/9/2011


39

eMetric• Password-protected site• Grade level data

– Multiple years• Student groups within grade levels

– Ethnicity– Gender– ED, IEP, LEP– Title I

C.G. Pg. 4

10/9/2011


40

eMetricC.G. Pg. 4

10/9/2011


41

Graphical Summary: Performance Levels

• Provides summary reports in the form of graphs, including bar graphs, pie charts, histograms, or line graphs.

• Allows for selection of subject areas, demographic variables, multiple grades, and multiple years.

C.G. Pg. 4

10/9/2011


42

Questions1. Did every grade meet the AYP performance

targets for reading and math? (72% Reading, 67% Math)

2. Did every relevant subgroup within each grade meet the AYP performance targets for reading and math? (72% Reading, 67% Math)

3. Does the pattern of performance for the selected subgroup mirror the pattern of performance for the grade as a whole?

C.G. Pg. 4

10/9/2011


43

Navigate & investigate your Performance Levels Summary for each grade level and subgroup and answer the questions in the guidebook.

Graphical Summary: Performance Levels

C.G. Pg. 4

10/9/2011


44

Graphical Summary: Reporting Categories

• Provides a bar graphs which display the percentage of score points earned for each reporting category.

• Allows for selection of different subject areas and demographic variables at each grade level.

C.G. Pg. 5

10/9/2011


45

Questions1. At each grade level, did all students

perform satisfactorily (at least 60% correct) in all reporting categories in the targeted subject area?

2. Does the pattern of performance across reporting categories for the selected subgroups mirror the pattern of performance across reporting categories for the grade as a whole?

C.G. Pg. 5

10/9/2011


46

C.G. Pg. 5

Navigate & investigate your Reporting Category Summary for each grade level and subgroup and answer the questions in the guidebook.

Graphical Summary: Reporting Categories

10/9/2011


47

Matched Comparison

• Compares PSSA performance levels for two years to determine if students have moved to higher PSSA performance levels in the subject area this past year.

C.G. Pg. 6

10/9/2011


48

Questions For Later Analysis

1. In each grade level, have more students moved to higher PSSA performance levels, remained in the same performance levels, or moved to lower performance levels?

2. How do these results impact your school’s achievement goals?

C.G. Pg. 6

10/9/2011


49

Matched Comparison

Navigate & investigate your Matched Comparison reports for each grade levels and answer the questions in the guidebook.

C.G. Pg. 6

10/9/2011


50

Summary of Essential Findings

• Based on your system’s achievement goals/target, how would you answer the two main questions?– Is the system (school/district) serving all

students adequately?– Is the system (school/district) serving all

students equitably?• List three findings from today’s session

under the subject area you investigated to share with the group.

C.G. Pg. 12

10/9/2011


51

Next Steps• Share the findings from this session with

staff Or, even better….

• Replicate the process with staff– Whole school – half focusing on math; half

focusing on reading– Grade level teams – focusing on math and

reading for their grade only– Departmental – focusing on subject area

C.G. Pg. 13-14

10/9/2011


52

What about other data?• What other data sources are

available for staff? – Benchmark Assessments (4-Sight, MAPS,

Acuity)– CDT – Classroom Diagnostic Tool– DIBELS, AIMSweb, G-MADE, GRADE, Study

Island, STAR– District/School Common Assessments

• Who has access to these data?• How often is it shared?

C.G. Pg. 13

10/9/2011


53

Why are we getting these results?

C.G. Pg. 13-14

10/9/2011


54

How can we support you in the future?

• Discuss with your team what types of support you might find useful from your IU PVAAS contact(s), as well as any other resources or materials needed from the PDE PVAAS Statewide Team.

• Detach and return page 19 of the Consultation Guide to your session facilitator.

C.G. Pg. 19

10/9/2011


55

Questions:

PVAAS Materials or Statewide Implementation

[email protected]

717-606-1911

PVAAS Report Web Sitehttps://pvaas.sas.com

10/9/2011

www.pde.state.pa.us

56PVAAS 2011 Consultation Guide10/9/2011

PVAAS School Consultation Fall 2011

Documents

Transcript of PVAAS School Consultation Fall 2011