Feedback from CEM Assessments: Individual Pupil Records & Predictions
description
Transcript of Feedback from CEM Assessments: Individual Pupil Records & Predictions
Feedback from CEM Assessments:
Individual Pupil Records & Predictions
Belfast, March 6th 2013Neil Defty
Business & Development ManagerCEM
• To inform professional judgement
• To start a conversation
Baseline Assessment Data
USING BASELINE DATA FOR TEACHING AND LEARNING
IF FEEDBACK DATA IS TO BE TRUSTED THEN PROBABLY:
• The students understood the purpose of the assessments
• And each student did their best at the time…….
• As a year group, how able are they?
• What strengths and weaknesses does the group have?
• How did each student perform?
Bands, percentiles, standardised scores…
0
50 60 70 80 90 100 110 120 130 140 150
Standardised scores
DC B
A
105 20 301 40 90 958070 996050Percentiles:
25 75
Alis: year 12 students, two baseline profiles for the same school
Nationally, 25% in each band
Band Profile Graph: all MidYIS cohort
Checking this graph each year will give you an immediate overview of your intake.
A school with a ‘completely average’ intake would have 25% of pupils within each band.
Band D Band A
3526
21 17
0
10
20
30
40
50
60
70
80
90
100
D C B A
Perc
ent
Maths: Percent in each BandYear 7 MidYIS Test 2011/2012
17
37
2025
0
10
20
30
40
50
60
70
80
90
100
D C B A
Perc
ent
Vocabulary: Percent in each BandYear 7 MidYIS Test 2011/2012
Band A Band ABand D Band D
1121
33 35
0
10
20
30
40
50
60
70
80
90
100
D C B A
Perc
ent
Non-Verbal: Percent in each BandYear 7 MidYIS Test 2011/2012
2924 26
21
0
10
20
30
40
50
60
70
80
90
100
D C B A
Perc
ent
Skills: Percent in each BandYear 7 MidYIS Test 2011/2012
2924 26
21
0
10
20
30
40
50
60
70
80
90
100
D C B A
Perc
ent
Skills: Percent in each BandYear 7 MidYIS Test 2011/2012
38
2619 17
0
10
20
30
40
50
60
70
80
90
100
D C B A
Perc
ent
Proof Reading: Percent in each BandYear 7 MidYIS Test 2011/2012
24 23 2330
0
10
20
30
40
50
60
70
80
90
100
D C B A
Perc
ent
Perceptual Speed & Accuracy: Percent in each BandYear 7 MidYIS Test 2011/2012
Vocabulary Maths Non Verbal Skills MidYIS Score
St. S
core
Ban
d
St. S
core
Ban
d
St. S
core
Ban
d
St. S
core
Ban
d
St. S
core
Ban
d
Average 100.6 95.9 105.3 99.2 98.1
St. error 1.1 1.1 1.1 1.1 1.1
On another day, with 95% certainty, the score of 100.6 (+/- 1.1 x 2) • would not be higher than 100.6 + 2.2 = 102.8• would not be lower than 100.6 – 2.2 = 98.4
Standardised Scores Sort Data By...
Mean Score: 100
Standard Deviation: 15
Scores Standardised On A Nationally Representative Sample of Schools
Vocabulary Maths Non Verbal Skills MidYIS ScoreSt
. Sco
re
Ban
d
St. S
core
Ban
d
St. S
core
Ban
d
St. S
core
Ban
d
St. S
core
Ban
d
100.6 95.9 105.3 99.2 98.1
1.1 1.1 1.1 1.1 1.1
134 A 119 A 116 A 87 D 130 A95 C 97 C 88 D 85 D 95 C99 C 104 B 112 A 110 B 101 B93 C 94 C 120 A 91 C 93 C
116 A 99 C 118 A 84 D 109 B93 C 95 C 120 A 113 A 93 C92 C 83 D 112 A 94 C 87 D
102 B 102 B 112 A 108 B 102 B
Proof Reading PSA
St. S
core
Ban
d
St. S
core
Ban
d
93.7 102.2
1.1 1.1
101 B 83 D76 D 93 C97 C 114 A80 D 99 C
102 B 78 D119 A 107 B80 D 102 B
106 B 107 B
50 60 70 80 90 100 110 120 130 140 150 Standardised Test Score
Stanine 1
2 3 4 5 6 7 8 9
4% 4 % 7% 7% 12% 12% 17% 17% 20%
(See www.cem.org/midyisiprbooklet for report: “Using MidYIS Individual Pupil Records to Inform Teaching and Learning”)• To start a conversation
Individual Pupil Records (IPRs)
Individual Pupil Record 75Date Of Birth: 24/12/97 Student Number:Sex: F Form:
Band Stanine Percentile Standard Score
Vocabulary B 5 53 101
Maths B 5 52 101
Non-Verbal C 5 46 98
Skills C 5 46 99
MidYIS Score B 5 52 101
50
60
70
80
90
100
110
120
130
140
150
Vocabulary Maths Non-Verbal Skills MidYIS Score
Natio
nally
Sta
ndar
dise
d Sc
ores
Nationally Standardised Scores with 95% Confidence Band
Individual Pupil Record LAUREN BAKERDate Of Birth: 28/11/99 Student Number: J870396003050Sex: F Form: 7C
Band Stanine Percentile Standard Score
Vocabulary C 4 27 91
Maths B 6 68 107
Non-Verbal B 6 75 110
Skills D 2 10 81
MidYIS Score C 5 45 98
50
60
70
80
90
100
110
120
130
140
150
Vocabulary Maths Non-Verbal Skills MidYIS Score
Natio
nally
Sta
ndar
dise
d Sc
ores
Nationally Standardised Scores with 95% Confidence Band
INSIGHT Pupil IPR
Band Stanine Percentile Standard Score
KS3 Equivalent
Speed Reading B 6 69 107 6c
Text Comprehension B 5 60 104 5a
Passage Comprehension C 5 45 98 5b
Overall Reading B 5 59 103 5a
Number & Algebra D 3 19 87 4a
Handling Data B 6 61 104 6a
Space, Shape & Measures D 4 23 89 5c
Overall Mathematics C 4 31 93 5b
Biology A 7 82 114 6a
Chemistry A 9 96 127 7a
Physics A 8 89 118 7c
Overall Science A 8 93 122 7b
Vocabulary B 6 65 106
Non Verbal B 6 70 108
Skills A 8 92 121
Overall Ability C 4 36 94
N2a
c3bac
4bac
5bac
6bac
7bac
8ba
Spee
d R
eadi
ng
Text
Com
preh
ensi
on
Pass
age
Com
preh
ensi
on
Ove
rall
Rea
ding
Num
ber &
Alg
ebra
Han
dlin
g D
ata
Spac
e, S
hape
& M
easu
res
Ove
rall
Mat
hem
atic
s
Biol
ogy
Che
mis
try
Phys
ics
Ove
rall
Scie
nce
KS3
Leve
lKS3 Equivalent Levels with 95% Confidence Band
IPRs in KS3 Levels
NATIONAL AVERAGE
The code:
j: reduces size
k: infills
m: inverts
“Predictions”
BASELINE SCORE
G
RA
DE
G
RA
DE
* ** *** *******************
* ** *** ********************************
* ** *** ********************************* *** ** *
* ** *** ************************************ *** ** *
* ** *** ******************************** **** ** *
* ** ***************************** *** ** *
* ** *** ******************* ** * *
* ** *********** ** *
How is a ‘prediction’ generated?
C
A*
A
B
50 100 150
3 key points are:
• The higher the baseline score the higher the final grade
• Any one grade is achievable from a range of baseline scores
• From any baseline score, a range of grades are possible
BASELINE SCORE
G
RA
DE
G
RA
DE
* ** *** *******************
* ** *** ********************************
* ** *** ********************************* *** ** *
* ** *** ************************************ *** ** *
* ** *** ******************************** **** ** *
* ** ***************************** *** ** *
* ** *** ******************* ** * *
* ** *********** ** *
Subject National trend line (regression line)
How is a ‘prediction’ generated?50% on or above the trend line
50% on or below the trend line
‘PREDICTION’ (expected grade)C
A*
A
B
2008 GCSE Maths (representative sample)
0
1
2
3
4
5
6
7
8
9
0 20 40 60 80 100
baseline score
Res
ult
‘Predictions’…...are based on
Average performance by similar pupils in past examinations
The problem with the word ‘prediction’ is…?
An alternative is ‘expected’ grade
Predictions
4 points = D
6.6 points = A/B Trend line
FACTORS THAT WILL INFLUENCE RELIABILITY OF PREDICTIONS:
• Knowledge of student • Parental support/home life• Peer influences/social life• Student attitude, interest, language• Expectations of staff• Department/institution ethos • Resources • Quality of teaching and learning: pace of lessons• Understanding how children learn………• And the reliability of the predictions......
29
Correlation = 1
Res
ult
Correlation = 0
Correlation = 0.7
subjectMIDYIS( Yr 7)
to GCSEART 0.49BIOL 0.56ENG LANG 0.7FRENCH 0.64HISTORY 0.63MATHS 0.74MUSIC 0.57
CORRELATIONS
SUBJECTGCSE to A2
ART and DESIGN 0.54BIOLOGY 0.69ECONOMICS 0.65ENG LIT 0.71FRENCH 0.6HISTORY 0.69MATHS 0.6MUSIC 0.65PHYS EDUC 0.65PSYCHOLOGY 0.62
50 60 70 80 90 100 110 120 130 140 1501
2
3
4
5
6
7
8
MidYIS Year 7 to GCSE 2011 Value Added Analysis Art & Design
Biology
English
French
History
Mathematics
Science
MidYIS Test Score
GC
SE P
oint
s Sc
ore
The graph below shows the middle 2/3 of some subject trend lines
0 0 15
19
39
27
9
10
5
10
15
20
25
30
35
40
45
U G F E D C B A A*
Perc
ent
Grade
Individual Chances Graph for Student 4- GCSE EnglishMidYIS Score 101 MidYIS Band B
Teacher's Adjustment : 0 grades / levels / points
Prediction/expected grade: 5.1 grade C
Most likely grade
Not a label for life...just another piece of information
• The Chances graphs show that, from almost any baseline score, students come up with almost any grade - - -there are just different probabilities for each grade depending on the baseline score.
• In working with students these graphs are more useful than a single predicted or target grade
• Chances graphs show what can be achieved: – By students of similar ability– By students with lower baseline scores
Predictions & Chances Graphs to GCSE subjects based on Year 9 nationally standardised scores
Please refer to the Information Sheet for details of the method used to make these predictions
0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0
Student Surname Forename Sex Form CEM Id Cust Mid
YIS
Scor
e
Mid
YIS
Band
Addi
tiona
l App
lied
Scie
nce
Addi
tiona
l Sci
ence
Art &
Des
ign
Biol
ogy
Busi
ness
Stu
dies
Che
mis
try
Des
ign
& Te
chno
logy
Dra
ma
Engl
ish
Engl
ish
Lite
ratu
re
Fren
ch
Geo
grap
hy
1 F 9140005J 105 B 4.4 5.3 5.8 5.9 5.1 5.9 5.3 5.2 5.4 5.5 5.0 5.12 F 9140005K 111 A 4.7 5.8 6.1 6.3 5.6 6.3 5.7 5.6 5.8 5.9 5.4 5.63 F 91400042 108 B 4.6 5.6 6.0 6.1 5.4 6.1 5.5 5.4 5.6 5.7 5.2 5.44 F 91400043 101 B 4.2 5.0 5.6 5.7 4.8 5.7 5.0 5.0 5.1 5.3 4.7 4.7
Teac
her's
Ad
just
men
t (G
rade
s)
Predictions based on 2011 Regression Analysis
MidYIS Year 9 Test Academic Year 2011/2012
GCSE Point PredictionsPoints To Grades Adjust
CEM Centre Scale
QCA Scale
Display Legend
Upper Quartile of Schools
Prior Value added
Generate File for MIS Import
to place school at 75th percentile of value added results
Predictions & Chances Graphs to GCSE subjects based on Year 9 nationally standardised scores
Please refer to the Information Sheet for details of the method used to make these predictions
0.2 0.2 0.2 0.2 0.2 0.2 0.2 0.2 0.2 0.2 0.2 0.2
Student Surname Forename Sex Form CEM Id Cust Mid
YIS
Scor
e
Mid
YIS
Band
Addi
tiona
l App
lied
Scie
nce
Addi
tiona
l Sci
ence
Art &
Des
ign
Biol
ogy
Busi
ness
Stu
dies
Che
mis
try
Des
ign
& Te
chno
logy
Dra
ma
Engl
ish
Engl
ish
Lite
ratu
re
Fren
ch
Geo
grap
hy
1 F 9140005J 105 B 4.6 5.5 6.0 6.1 5.3 6.1 5.5 5.4 5.6 5.7 5.2 5.32 F 9140005K 111 A 4.9 6.0 6.3 6.5 5.8 6.5 5.9 5.8 6.0 6.1 5.6 5.83 F 91400042 108 B 4.8 5.8 6.2 6.3 5.6 6.3 5.7 5.6 5.8 5.9 5.4 5.64 F 91400043 101 B 4.4 5.2 5.8 5.9 5.0 5.9 5.2 5.2 5.3 5.5 4.9 4.9
Teac
her's
Ad
just
men
t (G
rade
s)
Predictions based on 2011 Regression Analysis
MidYIS Year 9 Test Academic Year 2011/2012
GCSE Point PredictionsPoints To Grades Adjust
CEM Centre Scale
QCA Scale
Display Legend
Reset Adjustments
Prior Value Prior Value addedadded
Generate File for MIS Import
Predictions & Chances Graphs to GCSE subjects based on Year 9 nationally standardised scores
Please refer to the Information Sheet for details of the method used to make these predictions
0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0
Student Surname Forename Sex Form CEM Id Cust Mid
YIS
Scor
e
Mid
YIS
Band
Addi
tiona
l App
lied
Scie
nce
Addi
tiona
l Sci
ence
Art &
Des
ign
Biol
ogy
Busi
ness
Stu
dies
Che
mis
try
Des
ign
& Te
chno
logy
Dra
ma
Engl
ish
Engl
ish
Lite
ratu
re
Fren
ch
Geo
grap
hy
1 F 9140005J 105 B 4.4 5.3 5.8 5.9 5.1 5.9 5.3 5.2 5.4 5.5 5.0 5.12 F 9140005K 111 A 4.7 5.8 6.1 6.3 5.6 6.3 5.7 5.6 5.8 5.9 5.4 5.63 F 91400042 108 B 4.6 5.6 6.0 6.1 5.4 6.1 5.5 5.4 5.6 5.7 5.2 5.44 F 91400043 101 B 4.2 5.0 5.6 5.7 4.8 5.7 5.0 5.0 5.1 5.3 4.7 4.7
Teac
her's
Ad
just
men
t (G
rade
s)
Predictions based on 2011 Regression Analysis
MidYIS Year 9 Test Academic Year 2011/2012
GCSE Point PredictionsPoints To Grades Adjust
CEM Centre Scale
QCA Scale
Display Legend
Upper Quartile of Schools
Prior Value added
Generate File for MIS Import
Or insert own valuesand click Adjust
Predictions & Chances Graphs to GCSE subjects
based on Year 9 nationally standardised scores
Please refer to the Information Sheet for details of the method used to make these predictions
0.0 0.3 1.4 0.7 0.0 0.7 0.0 0.9 1.3 0.8 0.7 1.4
Student Surname Forename Sex Form CEM Id Cust Mid
YIS
Scor
e
Mid
YIS
Band
Addi
tiona
l App
lied
Scie
nce
Addi
tiona
l Sci
ence
Art &
Des
ign
Biol
ogy
Busi
ness
Stu
dies
Che
mis
try
Des
ign
& Te
chno
logy
Dra
ma
Engl
ish
Engl
ish
Lite
ratu
re
Fren
ch
Geo
grap
hy
1 F 9140005J 105 B 4.4 5.6 7.2 6.6 5.1 6.6 5.3 6.1 6.7 6.3 5.7 6.52 F 9140005K 111 A 4.7 6.1 7.5 7.0 5.6 7.0 5.7 6.5 7.1 6.7 6.1 7.03 F 91400042 108 B 4.6 5.9 7.4 6.8 5.4 6.8 5.5 6.3 6.9 6.5 5.9 6.84 F 91400043 101 B 4.2 5.3 7.0 6.4 4.8 6.4 5.0 5.9 6.4 6.1 5.4 6.1
Teac
her's
Ad
just
men
t (G
rade
s)
Predictions based on 2011 Regression Analysis
MidYIS Year 9 Test Academic Year 2011/2012
GCSE Point PredictionsPoints To Grades Adjust
CEM Centre Scale
QCA Scale
Display Legend
Upper Quartile Upper Quartile of Schoolsof Schools
Reset Adjustments
Generate File for MIS Import
Predictions & Chances Graphs to GCSE subjects based on Year 9 nationally standardised scores
Please refer to the Information Sheet for details of the method used to make these predictions
0.0 0.3 1.4 0.7 0.0 0.7 0.0 0.9 1.3 0.8 0.7 1.4
Student Surname Forename Sex Form CEM Id Cust Mid
YIS
Scor
e
Mid
YIS
Band
Addi
tiona
l App
lied
Scie
nce
Addi
tiona
l Sci
ence
Art &
Des
ign
Biol
ogy
Busi
ness
Stu
dies
Che
mis
try
Des
ign
& Te
chno
logy
Dra
ma
Engl
ish
Engl
ish
Lite
ratu
re
Fren
ch
Geo
grap
hy
1 F 9140005J 105 B C/D B/C A A/B C A/B B/C B A/B A/B B/C A/B2 F 9140005K 111 A C/D B A*/A A B/C A B/C A/B A A/B B A3 F 91400042 108 B C/D B A*/A A B/C A B/C A/B A A/B B A4 F 91400043 101 B D B/C A A/B C A/B C B A/B B B/C B
Teac
her's
Ad
just
men
t (G
rade
s)
Predictions based on 2011 Regression Analysis
MidYIS Year 9 Test Academic Year 2011/2012
GCSE Grade PredictionsGrades To Points Adjust
CEM Centre Scale
QCA Scale
Display Legend
Upper Quartile Upper Quartile of Schoolsof Schools
Reset Adjustments
Generate File for MIS Import
Prediction/expected grade: 6.4 grade A/B
Most likely grade
Predictions & Chances Graphs to GCSE subjects based on Year 9 Independent Sector standardised scores
Please refer to the Information Sheet for details of the method used to make these predictions
Student Surname Forename Sex Form CEM Id Cust Mid
YIS
Scor
e
Mid
YIS
Ban
d
Addi
tiona
l Sci
ence
Art &
Des
ign
Biol
ogy
Busi
ness
Stu
dies
Che
mis
try
Cla
ssic
al C
ivilis
atio
n
Des
ign
& Te
chno
logy
Dra
ma
Engl
ish
Engl
ish
Lite
ratu
re
Fren
ch
Geo
grap
hy
Ger
man
1 F 9140005J 87 D 5.8 6.6 6.4 5.7 6.4 5.8 6.1 6.1 6.1 6.2 5.7 6.1 5.82 F 9140005K 94 C 6.1 6.8 6.7 6.0 6.7 6.1 6.4 6.4 6.5 6.4 6.1 6.5 6.23 F 91400042 90 C 6.0 6.7 6.6 5.9 6.6 6.0 6.3 6.3 6.3 6.3 5.9 6.3 6.04 F 91400043 81 D 5.5 6.5 6.1 5.4 6.1 5.6 5.8 5.9 5.9 5.9 5.4 5.8 5.6
Predictions based on 2011 Independent Sector Regression Analysis
MidYIS Year 9 Test Academic Year 2011/2012
GCSE Point PredictionsPoints To Grades
CEM Centre Scale
QCA Scale
Display Legend
Generate File for MIS Import