Post on 09-Aug-2021
Understanding e-asTTle – a longitudinal analysis
Mercy Mhuru
September 2020
Released under the Official Information Act 1982
education.govt.nz
Longitudinal analysis of progress using E-asTTle
2
» E-asTTle has end of year assessment results for students in Year levels 4-10 in the period 2011-2019.
» Average end-of-year scores across year levels show that students’ levels of achievement increase at a decreasing rate.
» However, the composition of students from which the averages are calculated is different. The students from which average scores per year level are drawn from are different with each year level.
» Some students have only one assessment while others get assessments in consecutive years during their schooling years.
» To show the progress that students make in E-asTTle assessments, we identify students with consecutive assessments across different year levels.
Released under the Official Information Act 1982
education.govt.nz
Students that have long histories of assessments are representative of the
broader population.
3
Released under the Official Information Act 1982
education.govt.nz
Students with frequent assessments
4
>= CL 2
CL3
CL4
CL 5
CL 6These are average
scores for students
observed from Year 4-10
1200
1300
1400
1500
1600
1700
1800
Year10Year9Year8Year7Year6Year5Year4
E-as
TTLe
Mat
hem
atic
s Sc
ore
>=CL 2
CL 3
CL 4
CL 5
CL 6
1200
1300
1400
1500
1600
1700
1800
Year10Year9Year8Year7Year6Year5Year4
E-as
TTle
Rea
ding
Sco
re
The longest trajectory of consecutive
assessments is 7 years
The 7-year trajectory shows the progress of students that have been assessed in
consecutive years from Year 4-10.
Released under the Official Information Act 1982
education.govt.nz
Yearly progress increases at declining rates across year levels.
5
<= CL 2
CL 3
CL 4
CL 5 Progress increases at a decreasing rate
1200
1300
1400
1500
1600
1700
Year10Year9Year8Year7Year6Year5Year4
E-as
TTle
Rea
ding
Sco
res
>= CL 2
CL 3
CL 4
CL 5
1200
1300
1400
1500
1600
1700
Year 10Year 9Year 8Year 7Year 6Year 5Year 4
E-as
TTle
Mat
hem
atic
s Sc
ores
For all the 3-yearly trajectories across years 4-10, the highest
yearly progress is 44 points from year 4 to year 5 in Mathematics
The highest yearly progress is 53 points from year 4 to year 5 in Reading. Progress declines in
subsequent year levels
Released under the Official Information Act 1982
education.govt.nz
Rate of progress is lower in Y8 to Y9
6
>= CL 2
CL 4
CL 51562
1542
1511
1474
CL 3
1200
1300
1400
1500
1600
1700
1800
Year10Year9Year8Year7Year6Year5Year4
E-as
TTLe
Mat
hem
atic
s Sc
ore
Progress slows down from Year 8 to Year 9
>=CL 2
CL 3
CL 4
CL 5
CL 6
15621535
1513
1476
1200
1300
1400
1500
1600
1700
1800
Year10Year9Year8Year7Year6Year5Year4
E-as
TTle
Rea
ding
Sco
re
Then picks up as students move from Year 9 to 10
Yearly progress is the lowest from Year 8 to Year 9 in Mathematics and Reading, whichever trajectory is picked.
Released under the Official Information Act 1982
education.govt.nz
Yearly progress is lower than the expected 50 points
7
• The highest yearly progress is 44 points in Mathematics and 53 points in Reading from Year 4 to Year 5.
• As students progress to higher year levels, yearly progress declines in both Mathematics and Reading.
• The lowest yearly progress is noted when students move from Year 8 to Year 9.
• Students with 3 year trajectories of linked scores in Mathematics and Reading in year levels 7-9 and 8-10 show that yearly progress from Year 8 to Year 9 is between 22-25 points.
• It is not clear why we observe lowest yearly progress as students with linked scores move from Year 8 to Year 9.
Released under the Official Information Act 1982
education.govt.nz
Slow-down effect for students with no school moves
8
<= CL 2
CL 3
CL 4
CL 5
1200
1300
1400
1500
1600
1700
1800
Year10Year9Year8Year7Year6Year5Year4
E-as
TTLe
Mat
hem
atic
s Sc
ore
>=CL 2
CL 3
CL 4
CL 5
CL 6
1200
1300
1400
1500
1600
1700
1800
Year10Year9Year8Year7Year6Year5Year4
E-as
TTle
Rea
ding
Sco
re
The dip in year 8-9 progress disappears
when looking at students with no school moves
We only observe the declining rate of
progress persisting until year 10
Released under the Official Information Act 1982
education.govt.nz
Slowdown effect is more pronounced in low decile schools
9
<= CL 2
CL 3
CL 4
CL 5
CL 6
Deciles 9 & 10
Students in deciles 9/10 progress 26 points from year 8-9 and year 9-10
Deciles 1 & 2
Compared to 17 points progress from year 8-9 and 28 points progress
from year 9-10 for students in deciles1 and
2 .
1200
1300
1400
1500
1600
1700
1800
Year10Year9Year8Year7
E-as
TTle
Rea
ding
Sco
re
Released under the Official Information Act 1982
education.govt.nz
Do top performers also slow-down?
10
<= CL 2
CL 3
CL 4
CL 5
CL 6
Progress = 32 pointsYear 9-10
Progress = 29 pointsYear 8-9
Progress = 36 points Year 7-8
1200
1300
1400
1500
1600
1700
1800
1900
Year10Year9Year8Year7
E-as
TTLe
Mat
hem
atic
s Sc
ore
Students who are consistently above average
in consecutive assessments also
slowdown in progress from Year 8 to Year ,
although the decline is very minimal compared to all
students
Average yearly progress is less than 50 points for Maths for
above average students
Released under the Official Information Act 1982
education.govt.nz
Progress is different for students within the same groups
11
<= CL 2
CL 3
CL 4
CL 5
Top 10% achieversCL 6
Above average students
16351606
Bottom 10% students
13991387
Below average students
14821465
1711
1674
1200
1300
1400
1500
1600
1700
1800
1900
Year10Year9Year8Year7
E-as
TTLe
Mat
hem
atic
s Sc
ore
Y8-Y9 progress for students below average is 12 points lower than the progress made by
above average students
Students in the bottom 10% almost flat-line
across the 4 years from Y7-Y10. In Y8-Y9, their
progress is 3 times lower (12 points) than the
progress for top 10% students (36 points)
Released under the Official Information Act 1982
education.govt.nz
Prior scores predict future scores
12
1200
1400
1600
1800
2000
1000 1200 1400 1600 1800 2000
Pred
icte
d M
aths
E-a
sTTl
e Sc
ore
Prior Maths E-asTTle Score
On average, learning progress increases from one year to the
next
X
1200
1400
1600
1800
2000
1000 1200 1400 1600 1800 2000
Pred
icte
d R
eadi
ng E
-asT
Tle
Scor
ePrior Reading E-asTTle Score
However, some students’ score remain
relatively stable or they regress from one year level to the next
Most students with linked scores have positive progress from one year level to the next. Some students regress in scores from one year to the next. This could be because of
structure/topics covered in e-asTTle assessment taken.
Released under the Official Information Act 1982
education.govt.nz
Most prior and current scores are positively related
13
Threshold score in Year 7(score 1472)
Threshold score in Year (score 1571)
1100
1200
1300
1400
1500
1600
1700
1800
1900
1100 1200 1300 1400 1500 1600 1700 1800 1900
E-as
TTle
Mat
hem
atic
s Sc
ore
(Yea
r 10)
E-asTTle Mathematics Score (Year 7)
28,319 students (27%)consistently score below threshold in year 7 and year 10
18,246 students (17%)score above threshold in year 7 and then regress to score below threshold
6,642 students (6%)score below threshold in year 7, and then progress to score above threshold in year 10
51,979 students (49%)consistently score above threshold in year 7 and year 10
Released under the Official Information Act 1982
education.govt.nz
Longitudinal analysis identifies students who aren’t progressing
14
• About 17% of students with linked scores show a decline in learning. In Year 7, they score above threshold score and then score below threshold in Year 10.
• This group shows a concerning trend although a number of factors are at play e.g. the structure/topics covered in e-asTTle assessment taken and school moves from intermediate to secondary school.
• About 6% of students with linked scores show improvement in learning, from scoring below threshold in Year 7 to achieving above threshold scores in Year 10.
• This group represents a group of students where progress in learning is positive. Identifying unique characteristics within this group may unlock some factors that contribute positively to the learning process.
• About 27% of students with linked scores consistently score below threshold in Years 7 and 10. This group potentially identifies students who require additional support to progress as expected across curriculum levels.
Released under the Official Information Act 1982