Post on 22-Feb-2016
description
“ I often say that what gets measured, gets done.
Margaret Spellings
david.moran@tribalgroup.comwww.portal4isp.com ”
Knowing every student, Knowing their potential
Why should we use data in our work
Without data, you are just another person with an opinion
Andreas Schleicher. OECD, Head of Indicators and Analysis Division
Winning is a game of inches.Humphrey Walters
Performance Comparisons
S-RateA 91%
B 73%
C 82%
D 55%
E 5%
F 99.20%
Performance Comparisons
S-Rate Hospital TypeA 91% Orthopaedic
B 73% Accident and Emergency
C 82% General Surgery
D 55% Coronary Unit
E 5% Hospice
F 99.20%
Performance Comparisons
S-Rate Hospital TypeA 91% Orthopaedic
B 73% Accident and Emergency
C 82% General Surgery
D 55% Coronary Unit
E 5% Hospice
F 99.20% Maternity
First Major Principle of Fair Evaluation
What goes in affects what comes out
Performance Comparisons
S-Rate Hospital TypeA 91% Orthopaedic
B 73% Accident and Emergency
C 82% General Surgery
D 55% Coronary Unit
E 5% Hospice
F 99.20% Maternity
Performance Comparisons
S-Rate Hospital Type Good Avg PoorA 91% Orthopaedic 97% 95% 93%
B 73% Accident and Emergency
C 82% General Surgery
D 55% Coronary Unit 48% 43% 38%
E 5% Hospice
F 99.20% Maternity
Second Major Principle of Fair Evaluation
Essential to compare like with like
Lies, damn lies and statistics – Mark Twain
He uses statistics as a drunken man uses lampposts - for
support rather than illumination.
(Andrew Lang)
Statistics are no substitute for judgment.(Henry Clay)
Data Availability and Data Literacy
Ineffective
Embedded
Ignored Dangerous
Low HighData Availability
Dat
a Li
tera
cy Hig
hLo
w
Data LiteracyON YOUR TABLES, DISCUSS: DO YOU HAVE
1. Sufficient data:– to enable the key questions and factors to be explored
2. Sufficient access:– to systems which enable key elements of data to be linked
3. Sufficient experience and understanding:– to find the smallest amount of data needed - and how best to present it
4. Sufficient embedding:– such that individuals have an appropriate view about the reliability of data
5. Sufficient confidence:– to be able to justify why we are NOT doing something as well as the things
we have decided to do6. Sufficient humility:
– to enable our own assumptions to be challenged
Terminology
C
D
A An estimateA target
B A guess Daft
Calculating what you would expect a group of pupils to achieve, based upon the progress of similar pupils last year, is ?
Terminology
CAn estimate
Calculating what you would expect a group of pupils to achieve, based upon the progress of similar pupils last year, is ?
Past knowledge = estimate
Using Estimates with StudentsYour target grade
is …I thought I could do better
How do they expect me to achieve that?
I’ll show them!
I can get that easily
If you make average progress, you might get a… Let’s look at the range of grades achieved by similar students last year …. …. what will you aim to achieve?Interesting ..
Maybe I could do that …If one in five did that last year…?
Using Estimates with Students
What factors impact upon pupil achievement
AAttitudeAttendanceAptitude
MMotherMath capabilityMovement between schools
B Behaviour N Neighbourhood
CCrimeChallengeComputer access
OOpportunity
DDataDadDependents
PPrior attainmentPolicy
EEngagementEnglishEnvironment
QQuestioning abilityQuality of provision
F FoodFamily R Reading
GGender
SSchoolSpecial needsSocial welfare
H Home life T Teacher qualityTest ability
I Importance of EducationIntervention U Understanding
UniformJ Job aspirations V Variation
K Knowledge W Willingness to learnWriting
L Lifestyle XYZ
What factors impact upon pupil achievement
16.678 + 0.0054*(KS1 APS squared) + 0.672 *KS1 APS + 0.033*(KS1 reading points - KS1 APS) + 0.271*(KS1 maths points - KS1 APS) + 0.2750 (if in care) - 0.681*IDACI score - 1.528 (if School Action) - 2.437 (if Action Plus or Statemented) - 0.509 (if joined at start of or during Y6) - 0.306 (if joined at start of or during Y5) - 0.227 (if joined at start of or during years 3 or 4) - 0.272 (if female) - 0.626*(age within year where 1 Sept= 1.00, 31 Aug = 0.00)+ for EAL pupils only (2.173 + 0.0036*(KS1 APS squared) - 0.1762 *KS1 APS )+ ethnicity coefficient+ for FSM pupils only ( - 0.327 + FSM/ethnicity interaction)
UK KS2 to KS4 CVA
Simple Value added
Time
Ach
ieve
men
t
Time
Ach
ieve
men
t
Better than average = Positive Value Added
Lower than average = Negative Value Added
In the UK, we take 589,000 pupils and look at the average of what happened
KS2 APS KS4 APS
Different models = different estimates
Time
Atta
inm
ent
Different characteristics are used in complex mathematical models to create estimates based on a number of characteristics... Different estimates are created.
Differences• If two assessments are different
– One might be wrong
– They might BOTH be wrong
– They might be assessing different things
TriangulationAnalysis A Analysis B
Teachers Professional JudgementBasis for action Investigate Further
Check Accuracy Challenge Assumptions
UK GCSE outcomes at age 16
What would/could this look like for Nashville?
What would the input variable be?
What would the output variable
be?
35.935.835.735 .635.535 .435.335.235.135 .034.934 .834 .734.634.534 .434.334.234 .134.033.933 .833 .733.633.533 .433.333.233 .133.032.932 .832 .732.632 .532.432.332 .232.132.031 .931.831.731 .631 .531.431 .331.231.131 .030.930.830 .730 .630.530.430 .330.230.130 .029.929 .829.729.629 .529.429.329 .229.129 .028 .928.828.728 .628.528.428 .328.228.128 .027 .927.827.727 .627.527.427 .327.227.127 .026 .926.826 .726.626.526.426 .326.226 .126.025.925.825 .725.625 .525.425.325.225 .125.024 .924 .824.724.624 .524.424.324 .224.124.023 .923 .823.723.623 .523.423 .323.223.123 .022.922 .822.722 .622.522.422 .322.222.122 .021.921 .821.721 .621.521 .421.321.221 .121.020.920 .820.720.620 .520.420 .320 .220.120.019 .919.819.719 .619.519.419 .319.219 .119 .018.918.818 .718.618.518 .418.318.218 .118 .017.917 .817.717.617.517 .417.317 .217.117 .016.916 .816.716.616 .516.416.316 .216.116.015.915 .815.715 .615.515.415 .315.215.115 .0
650060005500500045004000350030002500200015001000500
0
Pupils' Key Stage 2 Points
No's
of P
upils
National distribution of Key Stage 2 Pointsfor 600,000 pupils annually
12 18 24 30 36
Low Below Avge Above High
What would/could this look like for Nashville?
Low Below Avg Above High
8% 26% 57% 83% 95%
The Wensleydale SchoolA Specialist Science College
Richmond Road, Leyburn North Yorkshire DL8 5HY
www.wensleydale.n-yorks.sch.uk
What might you do to exceed average?
The only judgements that can be made…
Mainstream secondary schools ranked
UCI
LCI
Statistically Average Statistically AboveStatistically Below
UCI
LCI
UCI
LCI
Always check if the confidence intervals cross the magical 1000 median?