The EEF approach
54
How do you choose a washing machine? Using evidence to inform practice Dr Jonathan Sharples www.educationendowmentfoundation.org.uk
description
How do you choose a washing machine? Using evidence to inform practice Dr Jonathan Sharples www.educationendowmentfoundation.org.uk. The EEF approach. “We must give educators and politicians the information they need to make wise decisions for children”. Estelle Morris. - PowerPoint PPT Presentation
Transcript of The EEF approach
How do you choose a washing machine? Using evidence to inform practice
Dr Jonathan Sharples
www.educationendowmentfoundation.org.uk
The EEF approach
Stand on the shoulders of previous progress i.e. healthcare, engineering• • Research is seen as something done to, not with, or for, or by the
profession – culture change
“We must give educators and politicians the information they need to make wise decisions for children”
Estelle Morris
What is evidence-informed practice?
How do we generate useful evidence for practice?
What can be done to make research more accessible to busy professionals?
How might we build capacity in professional settings to use research evidence at scale?
Make abstract concepts a bit more tangible...
www.alliance4usefulevidence.org
Current moves towards more evidence-informed policy/practice…
Cabinet Office ‘What works’ centres
Royal College of TeachersResearchED
Need for good evidence is increasingly agreed
Weak link is the coupling of evidence to practice, at scale –‘Knowledge mobilisation’
Challenges – Fragmentation in the system, weak links to training, OfSTED?
What are the challenges in accessing and using reliable research evidence?
Sifting reliable research conclusions from the rest
Not skilled to judge the validity of claims
Can be a conflict with existing school practices
Knowing where to look for useful information
Too much information is available
Senior Leadership Teams
Time, time, time!
Researchers
Policy makers
Practitioners
What would an evidence-using and creating culture look like?
What do we need to get there?
2013/14 - Events series on knowledge mobilisation in practice
www.cebenetwork.org
The EEF by numbers
83 evaluationsfunded to
date
3,000 schools
participating in projects
34 topics in
the Toolkit
16 independent evaluation
teams
600,000
pupils involved in EEF projects
14 members of EEF team
£220mestimated spend over lifetime of
the EEF
6,000 heads
presented to since launch
10 reports
published
Applying evidence in practice
External evidence summarised in the Toolkit can be used to inform choices.
Step 2: Identifying possible solutions
Evaluate the impact of your decisions and identify potential improvements for the future.
Step 4: Did it work?
Mobilise the knowledge and use the findings to inform the work of the school to grow or stop
the intervention.
Step 5: Securing and spreading change
Applying the ingredients of effective implementation.
Step 3: Giving the idea the best chance of success
Identify school priorities using internal data and professional judgement.
Step 1: Decide what you want to achieve
10
Applying evidence in practice
Generate a question using data, professional judgement and values.
How can we engage and support struggling readers in Year 7?
Step 1: Decide what you want to achieve
Applying evidence in practice
Ensure that you start from the best position by seeking internal and external
knowledge.
What has been shown to be effective in raising outcomes for struggling readers?
Step 2: Identify possible solutions
A Pupil Premium Scenario – Struggling Readers
What do you decide to spend the money on?How do you make the most of that investment?
Number on Roll %FSM Pupil Premium Allocation
750 10% £67,500
Parents Class size reduction
Head of EnglishOne to one tuition
SENCOEmploy more
Teaching Assistants
Teaching and Learning Toolkit
• The Toolkit is an accessible, teacher-friendly summary of educational research. ‘Which?’ for education
• Practice focused: tries to give schools the information they need to make informed decisions and narrow the gap.
• Based on meta-analyses conducted by Durham University.
Teaching and Learning Toolkit
• The Toolkit is an accessible, teacher-friendly summary of educational research. ‘Which?’ for education
• Practice focused: tries to give schools the information they need to make informed decisions and narrow the gap.
• Based on meta-analyses conducted by Durham University.
The Toolkit is a starting point for making decisions
Overview of value for money
Cost per pupil
Effe
ct S
ize
(m
on
ths
ga
in)
£00
10
£1000
Feedback
Meta-cognitive
Peer tutoring Pre-school
1-1 tutoringHomeworkICTOutdoor
learning
Parental involvement
Sports
Summer schools
After school
Individualised learning
Learning styles
ArtsPerformance
pay
Teaching assistants
Smaller classes
Ability grouping
Phonics
Independent learning
Using the Toolkit
Use the evidence as a starting point for discussion.
Dig deeper into what the evidence actually says
Understand the ‘active ingredients’ of implementation
Example: Teaching Assistants
Better magazine and Best Evidence in Brief
Three/year. Free for first year – www.betterevidence.org
Free fortnightly research digest – [email protected]
Applying evidence in practice
?How will you organise the tuition during classtime?
How much training do TA’s need?
Implementation matters: have you thought about what the approach means for teaching and
learning?
What are the ‘active ingredients’ for making best use of teaching assistants for struggling readers?
Step 3: Give the idea the best chance of success
Implementation matters: how is as important as what the evidence says
Shortage of practical vehicles (interventions, CPD training) to help get evidence working in practice, at scale and with rigour (eg AfL)
In the US, healthcare workers failure to wash hands effectively is major cause of death – $billions
Researcher created a checklist for surgical teams. Trial showed 66% reduction in infection rates, ~1500 lives in 18 months. Packaged the principles of handwashing into a practical intervention.
EEF Projects
We are working to fund, develop and evaluate projects that:
• Build on existing evidence.
• Will generate significant new understanding of ‘what works’.
• Can be replicated cost effectively if proven to work.
Examples: One-to-one support with teaching assistants, lesson observation, using mobile devices for feedback,
Switch on Reading
• RCT in 19 schools with 300 pupils• Attainment measured using
standardised literacy measures • Independent evaluation by Durham
University • Observations and interviews to inform
how and why the approach might be working
http://educationendowmentfoundation.org.uk/projects/category/primary
• One to one literacy intervention with children in Year 7 who are struggling with literacy (not achieving level 4 at KS2)
• Based on Reading Recovery. Delivered by teaching assistants, 20mins/day over 10 weeks.
• Previous research shows a positive effect (inc. Reading Recovery).
Switch On Reading
GroupNumber of
pupilsEffect size*
Estimated months’ progress
All pupils 308 +0.24 +3
Lower attainers
156 +0.39 +5
FSM-eligible 98 +0.36 +4
SEN reported 225 +0.31 +4
Switch On Reading - Conclusions
Key Conclusions
1. Switch-on Reading appears to be effective for weak and disadvantaged readers at the stage of transition to secondary school.
2. It can be delivered by teaching assistants after two-days of training, and full training and support is required for all relevant staff.
3. Challenges to successful implementation may include timetabling and the availability of age-appropriate texts.
4. There is a tendency for some staff to stray away from the explicit schedule and this is likely to reduce the programme’s impact. Regular monitoring will increase fidelity but may also increase cost.
Catch Up Numeracy
• One to one intervention with children in Years 2 to 6 who are struggling with numeracy
• Two 15 minute sessions with TAs per week for 30 weeks
GroupNumber of
pupilsEffect size
Estimated months’ progress
Catch Up Numeracy vs. control
108 +0.21 +3
Equivalent time one to one support vs.
control102 +0.27 +4
Catch Up Numeracy -Conclusions
Key Conclusions
1. Within this trial, one-to-one support by TAs led to a significant gain in numeracy skills.
2. Catch Up Numeracy makes similar significant gains, but there is little evidence that it provided any additional gains in numeracy outcomes over and above those from one to one teaching itself.
3. Schools can find it challenging to run two 15 minutes sessions per week, due to timetabling and other issues.
4. Structured interventions, such as Catch Up Numeracy, should be planned into the timetable from the start of the new school year to ensure they are given priority and status.
What have we learned?
• Teaching assistants, given the right support and training, can make a significant contribution to pupil attainment
• Schools should take care to understand how specific programmes are having an impact in their school.
• Implementation matters: brief, well-structured 1-2-1 sessions over a sustained period, with appropriate support and training
Not all positive findings though….
IEE Evidence for Impact database… coming soon
Applying evidence in practice
Did the approach work, what made it work, and how can it be improved next time?
Can we demonstrate that our readers are making progress?
Is it worth the effort?
Step 4: Put energy into evaluation
The Toolkit is a starting point for making decisions
Overview of value for money
Cost per pupil
Effe
ct S
ize
(m
on
ths
ga
in)
£00
10
£1000
Feedback
Meta-cognitive
Peer tutoring Pre-school
1-1 tutoringHomeworkICTOutdoor
learning
Parental involvement
Sports
Summer schools
After school
Individualised learning
Learning styles
ArtsPerformance
pay
Teaching assistants
Smaller classes
Ability grouping
Promising
May be worth it
Requires careful consideration
Phonics
Independent learning
Using the Toolkit
Use the evidence as a starting point for discussion.
Dig deeper into what the evidence actually says
Understand the ‘active ingredients’ of implementation
Example: Teaching Assistants
Better magazine and Best Evidence in Brief
Three/year. Free for first year – www.betterevidence.org
Free fortnightly research digest – [email protected]
Applying evidence in practice
?How will you organise the tuition during classtime?
How much training do TA’s need?
Implementation matters: have you thought about what the approach means for teaching and
learning?
What are the ‘active ingredients’ for making best use of teaching assistants for struggling readers?
Step 3: Give the idea the best chance of success
Implementation matters: how is as important as what the evidence says
Shortage of practical vehicles (interventions, CPD training) to help get evidence working in practice, at scale and with rigour (eg AfL)
In the US, healthcare workers failure to wash hands effectively is major cause of death – $billions
Researcher created a checklist for surgical teams. Trial showed 66% reduction in infection rates, ~1500 lives in 18 months. Packaged the principles of handwashing into a practical intervention.
Switch on Reading
• RCT in 19 schools with 300 pupils• Attainment measured using
standardised literacy measures • Independent evaluation by Durham
University • Observations and interviews to inform
how and why the approach might be working
http://educationendowmentfoundation.org.uk/projects/category/primary
• One to one literacy intervention with children in Year 7 who are struggling with literacy (not achieving level 4 at KS2)
• Based on Reading Recovery. Delivered by teaching assistants, 20mins/day over 10 weeks.
• Previous research shows a positive effect (inc. Reading Recovery).
Switch On Reading
GroupNumber of
pupilsEffect size*
Estimated months’ progress
All pupils 308 +0.24 +3
Lower attainers
156 +0.39 +5
FSM-eligible 98 +0.36 +4
SEN reported 225 +0.31 +4
Switch On Reading - Conclusions
Key Conclusions
1. Switch-on Reading appears to be effective for weak and disadvantaged readers at the stage of transition to secondary school.
2. It can be delivered by teaching assistants after two-days of training, and full training and support is required for all relevant staff.
3. Challenges to successful implementation may include timetabling and the availability of age-appropriate texts.
4. There is a tendency for some staff to stray away from the explicit schedule and this is likely to reduce the programme’s impact. Regular monitoring will increase fidelity but may also increase cost.
Catch Up Numeracy
• One to one intervention with children in Years 2 to 6 who are struggling with numeracy
• Two 15 minute sessions with TAs per week for 30 weeks
GroupNumber of
pupilsEffect size
Estimated months’ progress
Catch Up Numeracy vs. control
108 +0.21 +3
Equivalent time one to one support vs.
control102 +0.27 +4
Catch Up Numeracy -Conclusions
Key Conclusions
1. Within this trial, one-to-one support by TAs led to a significant gain in numeracy skills.
2. Catch Up Numeracy makes similar significant gains, but there is little evidence that it provided any additional gains in numeracy outcomes over and above those from one to one teaching itself.
3. Schools can find it challenging to run two 15 minutes sessions per week, due to timetabling and other issues.
4. Structured interventions, such as Catch Up Numeracy, should be planned into the timetable from the start of the new school year to ensure they are given priority and status.
What have we learned?
• Teaching assistants, given the right support and training, can make a significant contribution to pupil attainment
• Schools should take care to understand how specific programmes are having an impact in their school.
• Implementation matters: brief, well-structured 1-2-1 sessions over a sustained period, with appropriate support and training
IEE Evidence for Impact database… coming soon
Applying evidence in practice
Did the approach work, what made it work, and how can it be improved next time?
Can we demonstrate that our readers are making progress?
Is it worth the effort?
Step 4: Put energy into evaluation
Use DIY guide to evaluate innovations
A traditional school approach to intervention impact
Year 9 Class29 students
Data indicates low rates of attainment
Year 9 Class29 students
Data indicates 32% of them increased their
levels
29 students receive an
intervention to improve their attainment
Conclusion? Oral feedback was responsible for the rise in attainment
Control vs. Treatment
Stages of the guide
Applying evidence in practice
Moving from what we know to what we do.
Have we captured and embedded effective small group tuition in our school?
Could it make an impact in other areas?
Step 5: Making innovation stick
