Understanding e-asTTle – a longitudinal analysis

Mercy Mhuru

September 2020

Released under the Official Information Act 1982

education.govt.nz

Longitudinal analysis of progress using E-asTTle

2

» E-asTTle has end of year assessment results for students in Year levels 4-10 in the period 2011-2019.

» Average end-of-year scores across year levels show that students’ levels of achievement increase at a decreasing rate.

» However, the composition of students from which the averages are calculated is different. The students from which average scores per year level are drawn from are different with each year level.

» Some students have only one assessment while others get assessments in consecutive years during their schooling years.

» To show the progress that students make in E-asTTle assessments, we identify students with consecutive assessments across different year levels.

Released under the Official Information Act 1982

education.govt.nz

Students that have long histories of assessments are representative of the

broader population.

3

Released under the Official Information Act 1982

education.govt.nz

Students with frequent assessments

4

>= CL 2

CL3

CL4

CL 5

CL 6These are average

scores for students

observed from Year 4-10

1200

1300

1400

1500

1600

1700

1800

Year10Year9Year8Year7Year6Year5Year4

E-as

TTLe

Mat

hem

atic

s Sc

ore

>=CL 2

CL 3

CL 4

CL 5

CL 6

1200

1300

1400

1500

1600

1700

1800

Year10Year9Year8Year7Year6Year5Year4

E-as

TTle

Rea

ding

Sco

re

The longest trajectory of consecutive

assessments is 7 years

The 7-year trajectory shows the progress of students that have been assessed in

consecutive years from Year 4-10.

Released under the Official Information Act 1982

education.govt.nz

Yearly progress increases at declining rates across year levels.

5

<= CL 2

CL 3

CL 4

CL 5 Progress increases at a decreasing rate

1200

1300

1400

1500

1600

1700

Year10Year9Year8Year7Year6Year5Year4

E-as

TTle

Rea

ding

Sco

res

>= CL 2

CL 3

CL 4

CL 5

1200

1300

1400

1500

1600

1700

Year 10Year 9Year 8Year 7Year 6Year 5Year 4

E-as

TTle

Mat

hem

atic

s Sc

ores

For all the 3-yearly trajectories across years 4-10, the highest

yearly progress is 44 points from year 4 to year 5 in Mathematics

The highest yearly progress is 53 points from year 4 to year 5 in Reading. Progress declines in

subsequent year levels

Released under the Official Information Act 1982

education.govt.nz

Rate of progress is lower in Y8 to Y9

6

>= CL 2

CL 4

CL 51562

1542

1511

1474

CL 3

1200

1300

1400

1500

1600

1700

1800

Year10Year9Year8Year7Year6Year5Year4

E-as

TTLe

Mat

hem

atic

s Sc

ore

Progress slows down from Year 8 to Year 9

>=CL 2

CL 3

CL 4

CL 5

CL 6

15621535

1513

1476

1200

1300

1400

1500

1600

1700

1800

Year10Year9Year8Year7Year6Year5Year4

E-as

TTle

Rea

ding

Sco

re

Then picks up as students move from Year 9 to 10

Yearly progress is the lowest from Year 8 to Year 9 in Mathematics and Reading, whichever trajectory is picked.

Released under the Official Information Act 1982

education.govt.nz

Yearly progress is lower than the expected 50 points

7

• The highest yearly progress is 44 points in Mathematics and 53 points in Reading from Year 4 to Year 5.

• As students progress to higher year levels, yearly progress declines in both Mathematics and Reading.

• The lowest yearly progress is noted when students move from Year 8 to Year 9.

• Students with 3 year trajectories of linked scores in Mathematics and Reading in year levels 7-9 and 8-10 show that yearly progress from Year 8 to Year 9 is between 22-25 points.

• It is not clear why we observe lowest yearly progress as students with linked scores move from Year 8 to Year 9.

Released under the Official Information Act 1982

education.govt.nz

Slow-down effect for students with no school moves

8

<= CL 2

CL 3

CL 4

CL 5

1200

1300

1400

1500

1600

1700

1800

Year10Year9Year8Year7Year6Year5Year4

E-as

TTLe

Mat

hem

atic

s Sc

ore

>=CL 2

CL 3

CL 4

CL 5

CL 6

1200

1300

1400

1500

1600

1700

1800

Year10Year9Year8Year7Year6Year5Year4

E-as

TTle

Rea

ding

Sco

re

The dip in year 8-9 progress disappears

when looking at students with no school moves

We only observe the declining rate of

progress persisting until year 10

Released under the Official Information Act 1982

education.govt.nz

Slowdown effect is more pronounced in low decile schools

9

<= CL 2

CL 3

CL 4

CL 5

CL 6

Deciles 9 & 10

Students in deciles 9/10 progress 26 points from year 8-9 and year 9-10

Deciles 1 & 2

Compared to 17 points progress from year 8-9 and 28 points progress

from year 9-10 for students in deciles1 and

2 .

1200

1300

1400

1500

1600

1700

1800

Year10Year9Year8Year7

E-as

TTle

Rea

ding

Sco

re

Released under the Official Information Act 1982

education.govt.nz

Do top performers also slow-down?

10

<= CL 2

CL 3

CL 4

CL 5

CL 6

Progress = 32 pointsYear 9-10

Progress = 29 pointsYear 8-9

Progress = 36 points Year 7-8

1200

1300

1400

1500

1600

1700

1800

1900

Year10Year9Year8Year7

E-as

TTLe

Mat

hem

atic

s Sc

ore

Students who are consistently above average

in consecutive assessments also

slowdown in progress from Year 8 to Year ,

although the decline is very minimal compared to all

students

Average yearly progress is less than 50 points for Maths for

above average students

Released under the Official Information Act 1982

education.govt.nz

Progress is different for students within the same groups

11

<= CL 2

CL 3

CL 4

CL 5

Top 10% achieversCL 6

Above average students

16351606

Bottom 10% students

13991387

Below average students

14821465

1711

1674

1200

1300

1400

1500

1600

1700

1800

1900

Year10Year9Year8Year7

E-as

TTLe

Mat

hem

atic

s Sc

ore

Y8-Y9 progress for students below average is 12 points lower than the progress made by

above average students

Students in the bottom 10% almost flat-line

across the 4 years from Y7-Y10. In Y8-Y9, their

progress is 3 times lower (12 points) than the

progress for top 10% students (36 points)

Released under the Official Information Act 1982

education.govt.nz

Prior scores predict future scores

12

1200

1400

1600

1800

2000

1000 1200 1400 1600 1800 2000

Pred

icte

d M

aths

E-a

sTTl

e Sc

ore

Prior Maths E-asTTle Score

On average, learning progress increases from one year to the

next

X

1200

1400

1600

1800

2000

1000 1200 1400 1600 1800 2000

Pred

icte

d R

eadi

ng E

-asT

Tle

Scor

ePrior Reading E-asTTle Score

However, some students’ score remain

relatively stable or they regress from one year level to the next

Most students with linked scores have positive progress from one year level to the next. Some students regress in scores from one year to the next. This could be because of

structure/topics covered in e-asTTle assessment taken.

Released under the Official Information Act 1982

education.govt.nz

Most prior and current scores are positively related

13

Threshold score in Year 7(score 1472)

Threshold score in Year (score 1571)

1100

1200

1300

1400

1500

1600

1700

1800

1900

1100 1200 1300 1400 1500 1600 1700 1800 1900

E-as

TTle

Mat

hem

atic

s Sc

ore

(Yea

r 10)

E-asTTle Mathematics Score (Year 7)

28,319 students (27%)consistently score below threshold in year 7 and year 10

18,246 students (17%)score above threshold in year 7 and then regress to score below threshold

6,642 students (6%)score below threshold in year 7, and then progress to score above threshold in year 10

51,979 students (49%)consistently score above threshold in year 7 and year 10

Released under the Official Information Act 1982

education.govt.nz

Longitudinal analysis identifies students who aren’t progressing

14

• About 17% of students with linked scores show a decline in learning. In Year 7, they score above threshold score and then score below threshold in Year 10.

• This group shows a concerning trend although a number of factors are at play e.g. the structure/topics covered in e-asTTle assessment taken and school moves from intermediate to secondary school.

• About 6% of students with linked scores show improvement in learning, from scoring below threshold in Year 7 to achieving above threshold scores in Year 10.

• This group represents a group of students where progress in learning is positive. Identifying unique characteristics within this group may unlock some factors that contribute positively to the learning process.

• About 27% of students with linked scores consistently score below threshold in Years 7 and 10. This group potentially identifies students who require additional support to progress as expected across curriculum levels.

Released under the Official Information Act 1982