Graeme Cosslett

NZCER

2010

Unpacking PAT Tests from the NZCER Marking

Website

Building a greater understanding of the online reporting

Reporting at the:

• Individual Student level

• Class level

• Schoolwide level

Understanding:

Scale scores, Stanines, Box plots, Measurement error, Interpreting reports, Generating the reports

How are they doing?

Class of 2010

Test scores

Alex 22 Juliet 31

Allen 25 Kelly 18

Amy 40 Michael 36

Andrew 7 Nichole 30

Bean 6 Patrick 35

Becca 19 Paul 10

Bill 11 Peter 31

Catherine 7 Rose 18

Cathy 23 Sara 24

Christina 9 Simon 1

Daniel 18 Spencer 26

Dianna 41 Steve 14

Gary 24 Tessa 15

Janine 14 Tom 28

Jessie 14 Tony 16

Joy 6 Tracy 25

Grouped test scores

0-4 5-9 10-14 15-19 20-24 25-29 30-34 35-39 40-44

Dianna 41 Daniel 18

Amy 40 Kelly 18

Michael 36 Rose 18

Patrick 35 Tony 16

Juliet 31 Tessa 15

Peter 31 Janine 14

Nichole 30 Jessie 14

Tom 28 Steve 14

Spencer 26 Bill 11

Allen 25 Paul 10

Tracy 25 Christina 9

Gary 24 Andrew 7

Sara 24 Catherine 7

Cathy 23 Bean 6

Alex 22 Joy 6

Becca 19 Simon 1

Average 20.1

St.Dev 10.5

Median 18.5

Ordered test scores

How well are these students doing?

It depends!

Questions left unanswered

• What kind of knowledge and skill do students have at different achievement levels?

• How much better is a student doing who gets 35 compared to 25?

• How do scores on this test compare with scores on a harder or easier test?

• What score should I expect the students to get in a year’s time?

We can start with a measurement scale.

How do we answer these kinds of questions?

Measurement scales

• Allow you to convert a test score to a location on an equal interval scale.

• You can then see how one score compares with another, no matter which PAT test was used.

• You compare a student’s location on the scale with national norms for different year levels.

Measurement scales

• You can locate the difficulties of each test question on the same scale.

• We can then compare a student’s location on the scale with the location of different questions.

Measuring progress

More comprehension skills

Less comprehension skills

By Year 6, 50% of students have

reached this level

Questions involve longer passages containing abstract

ideas and more complex vocabulary

Create your own reading comprehension and mathematics scales that cater for achievement from Year 4 to 10. Describe the characteristics of tasks at different points on each scale.

Tasks that can be done using counting strategies.

How difficult?

• Put the tasks in order from least difficult to most difficult.

• Locate each task somewhere on your scale

• Which tasks are the most difficult? Why?• How would a typical Year 7 do on each of

the tasks?• On each of your scales locate where you

think the average Year 7 student would be

20 patm

34 patm

66 patm

75 patm

Year 7 Student

Year 10 Student

Comparing students and items

Students will typically answer correctly:

• 50% of the test items that are at the same location on the scale as they are

• more than 50% of the test items that are below their own location on the scale

• less than 50% of the test items that are above their own location on the scale.

Increasing skill as a reader

As students improve in comprehension they demonstrate increasing skill in:

– using abstract information– using separated information– using multiple pieces of information– using implied information– rejecting competing information– using vocabulary– using grammatical structures.

Vocabulary demands

Word List Dog & Pukeko (%) Square Eyes (%)

1 83.33 69.2

2 11.1 10.0

3 2.8 2.5

4 1.4 6.7

5 - 2.5

6 - -

7 - 1.7

8 - 0.8

other 1.4 6.66

Percentage of words in word list one to 10

The items in PAT Reading Comprehension according to text

Unpacking grammatical structures

Year 6

Year 7

Average progression in reading vocabulary for New Zealand students

Year 5

Year 4

Year 8

Year 9

Year 10

Year 6

Year 7

Average progression in mathematics for New Zealand students

Year 5

Year 4

Year 8

Year 9

Year 10

PAT:Mathematics Scale

Year 3

The PAT:Mathematics Scale

Stanines

Stanines

Stanines

Where on the scale?

Two students score 50% on PAT Mathematics tests.

Student one has sat Test 5 and student two has sat Test 6.

Test 6 is harder than Test 5.

Who will be located higher on the scale?

What Tom got right

1 202 223 234 245 267 288 299 32

10 3313 3414 3516 3617 3919 40

Tomsixsix

2855.9 ± 3.6

sevensix

Toms’ Maths Report

• Which national year group is Tom being compared with?

• How good is stanine 7?• How many Geometry and Measurement

questions did Tom get wrong?• Which was the easiest Geometry and

measurement question?• Was Tom expected to get question 30

correct?

SamYear 6March, 20082044.1 3.55

Year 6

Question types

• Retrieval

Comprehension without needing to infer

• Local inference

Comprehension of implied information from parts of the text

• Global inference

Comprehension of implied information from across the text as a whole

Local inference

• What does the poet compare herself with in the last verse? (Inferring the meaning of a simile from 1 line.)

• Why did Glen shrug? (Inferring a character’s intention from implied information across 6 sentences.)

• What was Adam imagining when he was playing with the water cannon? (Inferring a character’s thoughts from information in 4 paragraphs.)

Global inference

• What is the main purpose of this text?

• Which sentence would the author most likely agree with?

• What impression of the brain does this text create?

• What will probably happen next in the story?

PAT:Vocabulary items

90 patv

70 patv

50 patv

28patv

16 patv

Year 7 Student

Year 10 Student