® BiomarkersofWellness. ® What Gets Measured- Gets Managed Gets Managed Peter Drucker.
“ I often say that what gets measured, gets done. Margaret Spellings [email protected] ...
-
Upload
anthony-warner -
Category
Documents
-
view
222 -
download
0
Transcript of “ I often say that what gets measured, gets done. Margaret Spellings [email protected] ...
“ I often say that what gets measured, gets done.
Margaret Spellings
Knowing every student, Knowing their potential
Why should we use data in our work
Without data, you are just another person with an opinion
Andreas Schleicher. OECD, Head of Indicators and Analysis Division
Winning is a game of inches.Humphrey Walters
Performance Comparisons
S-RateA 91%
B 73%
C 82%
D 55%
E 5%
F 99.20%
Performance Comparisons
S-Rate Hospital TypeA 91% Orthopaedic
B 73% Accident and Emergency
C 82% General Surgery
D 55% Coronary Unit
E 5% Hospice
F 99.20%
Performance Comparisons
S-Rate Hospital TypeA 91% Orthopaedic
B 73% Accident and Emergency
C 82% General Surgery
D 55% Coronary Unit
E 5% Hospice
F 99.20% Maternity
First Major Principle of Fair Evaluation
What goes in affects what comes out
Performance Comparisons
S-Rate Hospital TypeA 91% Orthopaedic
B 73% Accident and Emergency
C 82% General Surgery
D 55% Coronary Unit
E 5% Hospice
F 99.20% Maternity
Performance Comparisons
S-Rate Hospital Type Good Avg PoorA 91% Orthopaedic 97% 95% 93%
B 73% Accident and Emergency
C 82% General Surgery
D 55% Coronary Unit 48% 43% 38%
E 5% Hospice
F 99.20% Maternity
Second Major Principle of Fair Evaluation
Essential to compare like with like
Lies, damn lies and statistics – Mark Twain
He uses statistics as a drunken man uses lampposts - for
support rather than illumination.
(Andrew Lang)
Statistics are no substitute for judgment.(Henry Clay)
Data Availability and Data Literacy
Ineffective
Embedded
Ignored Dangerous
Low High
Data Availability
Dat
a L
iter
acy
Hig
hL
ow
Data LiteracyON YOUR TABLES, DISCUSS: DO YOU HAVE
1. Sufficient data:– to enable the key questions and factors to be explored
2. Sufficient access:– to systems which enable key elements of data to be linked
3. Sufficient experience and understanding:– to find the smallest amount of data needed - and how best to present it
4. Sufficient embedding:– such that individuals have an appropriate view about the reliability of
data5. Sufficient confidence:
– to be able to justify why we are NOT doing something as well as the things we have decided to do
6. Sufficient humility:– to enable our own assumptions to be challenged
Terminology
C
D
A An estimateA target
B A guess Daft
Calculating what you would expect a group of pupils to achieve, based upon the progress of similar pupils last year, is ?
Terminology
CAn estimate
Calculating what you would expect a group of pupils to achieve, based upon the progress of similar pupils last year, is ?
Past knowledge = estimate
Using Estimates with StudentsYour target grade
is …I thought I could do better
How do they expect me to achieve that?
I’ll show them!
I can get that easily
If you make average progress, you might get a…
Let’s look at the range of grades achieved by similar students last year ….
…. what will you aim to achieve?Interesting .. Maybe I could do that …If one in five did that last year…?
Using Estimates with Students
What factors impact upon pupil achievement
AAttitudeAttendanceAptitude
MMotherMath capabilityMovement between schools
B Behaviour N Neighbourhood
CCrimeChallengeComputer access
OOpportunity
DDataDadDependents
PPrior attainmentPolicy
EEngagementEnglishEnvironment
QQuestioning abilityQuality of provision
FFoodFamily R
Reading
GGender
SSchoolSpecial needsSocial welfare
HHome life
TTeacher qualityTest ability
IImportance of EducationIntervention U
UnderstandingUniform
J Job aspirations V Variation
KKnowledge
WWillingness to learnWriting
L Lifestyle XYZ
What factors impact upon pupil achievement
16.678 + 0.0054*(KS1 APS squared) + 0.672 *KS1 APS + 0.033*(KS1 reading points - KS1 APS) + 0.271*(KS1 maths points - KS1 APS) + 0.2750 (if in care) - 0.681*IDACI score - 1.528 (if School Action) - 2.437 (if Action Plus or Statemented) - 0.509 (if joined at start of or during Y6) - 0.306 (if joined at start of or during Y5) - 0.227 (if joined at start of or during years 3 or 4) - 0.272 (if female) - 0.626*(age within year where 1 Sept= 1.00, 31 Aug = 0.00)+ for EAL pupils only (2.173 + 0.0036*(KS1 APS squared) - 0.1762 *KS1 APS )+ ethnicity coefficient+ for FSM pupils only ( - 0.327 + FSM/ethnicity interaction)
UK KS2 to KS4 CVA
Simple Value added
Time
Ac
hie
ve
me
nt
Time
Ac
hie
ve
me
nt
Better than average = Positive Value Added
Lower than average = Negative Value Added
In the UK, we take 589,000 pupils and look at the average of what happened
KS2 APS KS4 APS
Different models = different estimates
Time
Att
ain
me
nt
Different characteristics are used in complex mathematical models to create estimates based on a number of characteristics... Different estimates are created.
Differences• If two assessments are different
– One might be wrong
– They might BOTH be wrong
– They might be assessing different things
TriangulationAnalysis A Analysis B
Teachers Professional Judgement
Basis for action Investigate Further
Check Accuracy Challenge Assumptions
UK GCSE outcomes at age 16
What would/could this look like for Nashville?
What would the input variable be?
What would the output variable
be?
35.935.835.735.635.535.435.335.235.135.034.934.834.734.634.534.434.334.234.134.033.933.833.733.633.533.433.333.233.133.032.932.832.732.632.532.432.332.232.132.031.931.831.731.631.531.431.331.231.131.030.930.830.730.630.530.430.330.230.130.029.929.829.729.629.529.429.329.229.129.028.928.828.728.628.528.428.328.228.128.027.927.827.727.627.527.427.327.227.127.026.926.826.726.626.526.426.326.226.126.025.925.825.725.625.525.425.325.225.125.024.924.824.724.624.524.424.324.224.124.023.923.823.723.623.523.423.323.223.123.022.922.822.722.622.522.422.322.222.122.021.921.821.721.621.521.421.321.221.121.020.920.820.720.620.520.420.320.220.120.019.919.819.719.619.519.419.319.219.119.018.918.818.718.618.518.418.318.218.118.017.917.817.717.617.517.417.317.217.117.016.916.816.716.616.516.416.316.216.116.015.915.815.715.615.515.415.315.215.115.0
6500
60005500
50004500
4000
35003000
25002000
1500
1000500
0
Pupils' Key Stage 2 Points
No'
s of
Pup
ils
National distribution of Key Stage 2 Pointsfor 600,000 pupils annually
12 18 24 30 36
Low Below Avge Above High
What would/could this look like for Nashville?
Low Below Avg Above High
8% 26% 57% 83% 95%
The Wensleydale SchoolA Specialist Science College
Richmond Road, Leyburn North Yorkshire DL8 5HY
www.wensleydale.n-yorks.sch.uk
What might you do to exceed average?
The only judgements that can be made…
Mainstream secondary schools ranked
UCI
LCI
Statistically Average Statistically AboveStatistically Below
UCI
LCI
UCI
LCI
Always check if the confidence intervals cross the magical 1000 median?