How do we prepare our students for a world we cannot possibly imagine? Dylan Wiliam .
Transcript of How do we prepare our students for a world we cannot possibly imagine? Dylan Wiliam .
![Page 1: How do we prepare our students for a world we cannot possibly imagine? Dylan Wiliam .](https://reader034.fdocuments.us/reader034/viewer/2022052317/56649e545503460f94b4b1a4/html5/thumbnails/1.jpg)
How do we prepare our students for a world we cannot possibly imagine?
Dylan Wiliam
www.dylanwiliam.net
![Page 2: How do we prepare our students for a world we cannot possibly imagine? Dylan Wiliam .](https://reader034.fdocuments.us/reader034/viewer/2022052317/56649e545503460f94b4b1a4/html5/thumbnails/2.jpg)
Overview2
Why education matters The changing world of work The role of teacher quality in improving education Why most of the strategies being pursued in the
U.S. won’t work What we should be doing
![Page 3: How do we prepare our students for a world we cannot possibly imagine? Dylan Wiliam .](https://reader034.fdocuments.us/reader034/viewer/2022052317/56649e545503460f94b4b1a4/html5/thumbnails/3.jpg)
What is the purpose of education?3
Four main philosophies of education Personal empowerment Preparation for citizenship Cultural transmission Preparation for work
All are important Any education system is a (sometimes uneasy)
compromise between these four forces
![Page 4: How do we prepare our students for a world we cannot possibly imagine? Dylan Wiliam .](https://reader034.fdocuments.us/reader034/viewer/2022052317/56649e545503460f94b4b1a4/html5/thumbnails/4.jpg)
Recession (2008-2010) and recovery (2010-2012)
Those with a high school diploma or less lost 5.6 million jobs in the recession, and lost a further 230,000 jobs in the recovery
Those with an Associate’s degree lost 1.75 million jobs in the recession, but gained 1.6 million jobs in the recovery
Those with at least a Bachelor’s degree gained 187,000 jobs in the recession, and gained a further 2 million jobs in the recovery
Carnevale, Jayasundera, and Cheah (2012)
4
![Page 5: How do we prepare our students for a world we cannot possibly imagine? Dylan Wiliam .](https://reader034.fdocuments.us/reader034/viewer/2022052317/56649e545503460f94b4b1a4/html5/thumbnails/5.jpg)
The world of work is changing…
Education level Change in salary 1978 to 2005Postgraduate qualification +28%BA/BSc +19%Some college 0%High school diploma 0%High school dropout -16%
Economic Policy Institute (2010)
![Page 6: How do we prepare our students for a world we cannot possibly imagine? Dylan Wiliam .](https://reader034.fdocuments.us/reader034/viewer/2022052317/56649e545503460f94b4b1a4/html5/thumbnails/6.jpg)
…in ways that are hard to predict
Skill category Percentage change 1969-1999
Complex communication +14%Expert thinking/problem solving +8%Routine manual –3%Non-routine manual –5%Routine cognitive –8%
Autor, Levy & Murnane (2003)
![Page 7: How do we prepare our students for a world we cannot possibly imagine? Dylan Wiliam .](https://reader034.fdocuments.us/reader034/viewer/2022052317/56649e545503460f94b4b1a4/html5/thumbnails/7.jpg)
Computers in medical diagnosis7
Pilot study of the use of neural nets to predict biopsy results Sample: 1,787 men with a serum prostate-specific antigen (PSA)
concentration > 4.0 ng/ml Data:
Age Maximum PSA concentration over all visits Average PSA concentration over all visits Maximum digital rectal examination over all visits Maximum transrectal ultrasonography results over all visits Change of PSA value between visits
Positive predictive value (% of those predicted with positive biopsies) Specialist urologists: 34% Artificial neural nets: 77%
Snow, Smith and Catalona (1994)
![Page 8: How do we prepare our students for a world we cannot possibly imagine? Dylan Wiliam .](https://reader034.fdocuments.us/reader034/viewer/2022052317/56649e545503460f94b4b1a4/html5/thumbnails/8.jpg)
Off-shoring and automation
Off-shoreable Not off-shoreable
SkilledRadiographerSecurity analystTax accountant
Surgeon (?)BricklayerHairdresser
UnskilledFood packagerData entry clerkCall centre operator
Grocery store clerkReceptionistRetail salesperson
![Page 9: How do we prepare our students for a world we cannot possibly imagine? Dylan Wiliam .](https://reader034.fdocuments.us/reader034/viewer/2022052317/56649e545503460f94b4b1a4/html5/thumbnails/9.jpg)
The coming war for jobs (Clifton, 2011)
Right now 7 billion people on earth 5 billion adults 3 billion people who want to work 90% of these want to work full time
As a consequence 2.7 billion full-time formal jobs are wanted with only 1.2 billion full-time formal jobs available
A shortfall of 1.5 billion jobs So, for every US worker, there are 10 people who
would like their job…
![Page 10: How do we prepare our students for a world we cannot possibly imagine? Dylan Wiliam .](https://reader034.fdocuments.us/reader034/viewer/2022052317/56649e545503460f94b4b1a4/html5/thumbnails/10.jpg)
Surprising jobs are being automated…
![Page 11: How do we prepare our students for a world we cannot possibly imagine? Dylan Wiliam .](https://reader034.fdocuments.us/reader034/viewer/2022052317/56649e545503460f94b4b1a4/html5/thumbnails/11.jpg)
Meet Maddie Parlier…
Davidson (2012)
![Page 12: How do we prepare our students for a world we cannot possibly imagine? Dylan Wiliam .](https://reader034.fdocuments.us/reader034/viewer/2022052317/56649e545503460f94b4b1a4/html5/thumbnails/12.jpg)
So the model that says learn while you’re at school, while you’re young, the skills that you will apply during your lifetime is no longer tenable. The skills that you can learn when you’re at school will not be applicable. They will be obsolete by the time you get into the workplace and need them, except for one skill. The one really competitive skill is the skill of being able to learn. It is the skill of being able not to give the right answer to questions about what you were taught in school, but to make the right response to situations that are outside the scope of what you were taught in school. We need to produce people who know how to act when they’re faced with situations for which they were not specifically prepared. (Papert, 1998)
There is only one 21st century skill12
![Page 13: How do we prepare our students for a world we cannot possibly imagine? Dylan Wiliam .](https://reader034.fdocuments.us/reader034/viewer/2022052317/56649e545503460f94b4b1a4/html5/thumbnails/13.jpg)
Successful education?
The test of successful education is not the amount of knowledge that a pupil takes away from school, but his appetite to know and his capacity to learn. If the school sends out children with the desire for knowledge and some idea how to acquire it, it will have done its work. Too many leave school with the appetite killed and the mind loaded with undigested lumps of information. The good schoolmaster is known by the number of valuable subjects which he declines to teach.
The Future of Education (Livingstone, 1941 p. 28)
13
![Page 14: How do we prepare our students for a world we cannot possibly imagine? Dylan Wiliam .](https://reader034.fdocuments.us/reader034/viewer/2022052317/56649e545503460f94b4b1a4/html5/thumbnails/14.jpg)
Where’s the solution?
Structure: Smaller/larger high schools K–8 schools/“All-through” schools
Alignment: Curriculum reform Textbook replacement
Governance: Charter schools Vouchers
Technology: Computers Interactive whiteboards
Workforce reforms
![Page 15: How do we prepare our students for a world we cannot possibly imagine? Dylan Wiliam .](https://reader034.fdocuments.us/reader034/viewer/2022052317/56649e545503460f94b4b1a4/html5/thumbnails/15.jpg)
Pause for reflection
What’s the most interesting, surprising, or challenging thing you have heard so far?
See if you can get consensus with your neighbors
![Page 16: How do we prepare our students for a world we cannot possibly imagine? Dylan Wiliam .](https://reader034.fdocuments.us/reader034/viewer/2022052317/56649e545503460f94b4b1a4/html5/thumbnails/16.jpg)
16
School effectiveness
Three generations of school effectiveness research: Raw results approaches:
Different schools get different results. Conclusion: Schools make a difference.
Demographic-based approaches: Demographic factors account for most of the variation. Conclusion: Schools don’t make a difference.
Value-added approaches: School-level differences in value-added are relatively small. Classroom-level differences in value-added are large. Conclusion: An effective school is a school full of effective
classrooms.
![Page 17: How do we prepare our students for a world we cannot possibly imagine? Dylan Wiliam .](https://reader034.fdocuments.us/reader034/viewer/2022052317/56649e545503460f94b4b1a4/html5/thumbnails/17.jpg)
We need to focus on classrooms, not schools
In the USA, variability at the classroom level is at least four times that at school level. As long as you go to school, it doesn’t matter very
much which school you go to. But it matters very much which classrooms you are in.
It’s not class size. It’s not the between-class grouping strategy. It’s not the within-class grouping strategy.
![Page 18: How do we prepare our students for a world we cannot possibly imagine? Dylan Wiliam .](https://reader034.fdocuments.us/reader034/viewer/2022052317/56649e545503460f94b4b1a4/html5/thumbnails/18.jpg)
And most of all, on teachers18
Take a group of 50 teachers: Students taught by the most effective teacher in that group
of 50 teachers learn in six months what those taught by the average teacher learn in a year.
Students taught by the least effective teacher in that group of 50 teachers will take two years to achieve the same learning
(Hanushek & Rivkin, 2006) And furthermore:
In the classrooms of the most effective teachers, students from disadvantaged backgrounds learn at the same rate as those from advantaged backgrounds
(Hamre & Pianta, 2005).
![Page 19: How do we prepare our students for a world we cannot possibly imagine? Dylan Wiliam .](https://reader034.fdocuments.us/reader034/viewer/2022052317/56649e545503460f94b4b1a4/html5/thumbnails/19.jpg)
How do we improve teacher quality?
A classic labor force issue with two (non-exclusive) solutions: Replace existing teachers with better ones. Help existing teachers become even more effective.
![Page 20: How do we prepare our students for a world we cannot possibly imagine? Dylan Wiliam .](https://reader034.fdocuments.us/reader034/viewer/2022052317/56649e545503460f94b4b1a4/html5/thumbnails/20.jpg)
Here’s what we know about teaching20
We don’t know who will be good teachers We can’t always guarantee good teaching:
from teachers’ qualifications by classroom observation by looking at “value-added” test scores
![Page 21: How do we prepare our students for a world we cannot possibly imagine? Dylan Wiliam .](https://reader034.fdocuments.us/reader034/viewer/2022052317/56649e545503460f94b4b1a4/html5/thumbnails/21.jpg)
Teacher qualifications
21
![Page 22: How do we prepare our students for a world we cannot possibly imagine? Dylan Wiliam .](https://reader034.fdocuments.us/reader034/viewer/2022052317/56649e545503460f94b4b1a4/html5/thumbnails/22.jpg)
Teacher qualifications and student progress22
Mathematics Reading
Primary Middle High Primary Middle High
General theory of education coursesTeaching practice coursesPedagogical content coursesAdvanced university coursesAptitude test scores
Harris and Sass (2007)
Mathematics Reading
Primary Middle High Primary Middle High
General theory of education courses —Teaching practice courses — +Pedagogical content courses + +Advanced university courses — +Aptitude test scores —
![Page 23: How do we prepare our students for a world we cannot possibly imagine? Dylan Wiliam .](https://reader034.fdocuments.us/reader034/viewer/2022052317/56649e545503460f94b4b1a4/html5/thumbnails/23.jpg)
Teacher observations
23
![Page 24: How do we prepare our students for a world we cannot possibly imagine? Dylan Wiliam .](https://reader034.fdocuments.us/reader034/viewer/2022052317/56649e545503460f94b4b1a4/html5/thumbnails/24.jpg)
Framework for teaching (Danielson 1996)
Four domains of professional practice1. Planning and preparation2. Classroom environment3. Instruction4. Professional responsibilities
Links with student achievement (Sartain, et al. 2011) Domains 1 and 4: no impact on student achievement Domains 2 and 3: some impact on student achievement
24
![Page 25: How do we prepare our students for a world we cannot possibly imagine? Dylan Wiliam .](https://reader034.fdocuments.us/reader034/viewer/2022052317/56649e545503460f94b4b1a4/html5/thumbnails/25.jpg)
A framework for teaching (Danielson, 1996)
Domain 2: The classroom environment 2a: Creating an environment of respect and rapport 2b: Establishing a culture for learning 2c: Managing classroom procedures 2d: Managing student behavior 2e: Organizing physical space
Domain 3: Instruction 3a: Communicating with students 3b: Using questioning and discussion techniques 3c: Engaging students in learning 3d: Using assessment in instruction 3e: Demonstrating flexibility and responsiveness
![Page 26: How do we prepare our students for a world we cannot possibly imagine? Dylan Wiliam .](https://reader034.fdocuments.us/reader034/viewer/2022052317/56649e545503460f94b4b1a4/html5/thumbnails/26.jpg)
Reading
Unsatisfactory Basic Proficient Distinguished-0.6
-0.4
-0.2
0
0.2
0.4
0.6
2a2b2c2d2e3a3b3c3d3e
Teac
her v
alue
-add
ed
![Page 27: How do we prepare our students for a world we cannot possibly imagine? Dylan Wiliam .](https://reader034.fdocuments.us/reader034/viewer/2022052317/56649e545503460f94b4b1a4/html5/thumbnails/27.jpg)
Mathematics
Unsatisfactory Basic Proficient Distinguished-0.8
-0.6
-0.4
-0.2
0
0.2
0.4
0.6
2a2b2c2d2e3a3b3c3d3e
Teac
her v
alue
-add
ed
![Page 28: How do we prepare our students for a world we cannot possibly imagine? Dylan Wiliam .](https://reader034.fdocuments.us/reader034/viewer/2022052317/56649e545503460f94b4b1a4/html5/thumbnails/28.jpg)
Observations and teacher quality28
Unsatisfactory Basic Proficient Distinguished-15
-10
-5
0
5
10
15
20
Reading Mathematics
Perc
enta
ge c
hang
e in
rate
of l
earn
-in
g
Sartain, Stoelinga, Brown, Luppescu, Matsko, Miller, Durwood, Jiang, and Glazer (2011)
So, the highest rated teachers are 30% more productive than the lowest rated
But the best teachers are 400% more productive than the least effective
![Page 29: How do we prepare our students for a world we cannot possibly imagine? Dylan Wiliam .](https://reader034.fdocuments.us/reader034/viewer/2022052317/56649e545503460f94b4b1a4/html5/thumbnails/29.jpg)
Imprecision of lesson observations29
Achieving a reliability of 0.9 in judging teacher quality through lesson observation is likely to require observing a teacher teaching 6 different classes, and for each lesson to be judged by 5 independent observers.
Hill, Charalambous and Kraft (2012)
![Page 30: How do we prepare our students for a world we cannot possibly imagine? Dylan Wiliam .](https://reader034.fdocuments.us/reader034/viewer/2022052317/56649e545503460f94b4b1a4/html5/thumbnails/30.jpg)
Teacher value-added
30
![Page 31: How do we prepare our students for a world we cannot possibly imagine? Dylan Wiliam .](https://reader034.fdocuments.us/reader034/viewer/2022052317/56649e545503460f94b4b1a4/html5/thumbnails/31.jpg)
Teacher value-added
Value-added quintile: Student fixed-effects modelValu
e-adde
d quintile: Tradition
al mod
el
1 2 3 4 5
1 38 22 24 16 0
2 26 28 15 20 11
3 20 20 20 24 16
4 13 24 26 13 24
5 9 5 12 28 47
Goldhaber, Goldschmidt, and Tseng (2013)
![Page 32: How do we prepare our students for a world we cannot possibly imagine? Dylan Wiliam .](https://reader034.fdocuments.us/reader034/viewer/2022052317/56649e545503460f94b4b1a4/html5/thumbnails/32.jpg)
Change in mathematics teacher over time
0 1 2 3 4-0.3
-0.2
-0.1
5.55111512312578E-17
0.1
0.2
0.3
0.4
0.5
Quintile of teacher value-added
5th 4th 3rd 2nd 1st
Years of teaching experience
Valu
e-ad
ded
(stu
dent
SD
s)
Atteberry, Loeb and Wyckoff (2013)
![Page 33: How do we prepare our students for a world we cannot possibly imagine? Dylan Wiliam .](https://reader034.fdocuments.us/reader034/viewer/2022052317/56649e545503460f94b4b1a4/html5/thumbnails/33.jpg)
Correlation of initial and later performance33
Base+1 Base+2 Base+3 Base+4 Base+50
0.1
0.2
0.3
0.4
0.5
All teachers New teachers
Year of prediction
Corr
elati
on o
f val
ue-a
dded
with
bas
e ye
ar
Mathematics
![Page 34: How do we prepare our students for a world we cannot possibly imagine? Dylan Wiliam .](https://reader034.fdocuments.us/reader034/viewer/2022052317/56649e545503460f94b4b1a4/html5/thumbnails/34.jpg)
Correlation of initial and later performance34
Base+1 Base+2 Base+3 Base+4 Base+50
0.1
0.2
0.3
0.4
0.5
All teachers New teachers
Year of prediction
Corr
elati
on o
f val
ue-a
dded
with
bas
e ye
ar
Reading
![Page 35: How do we prepare our students for a world we cannot possibly imagine? Dylan Wiliam .](https://reader034.fdocuments.us/reader034/viewer/2022052317/56649e545503460f94b4b1a4/html5/thumbnails/35.jpg)
Issues with value-added models for teachers35
Different (reasonable) models of value-added give very different estimates of teacher quality (Goldhaber, Goldschmidt and Tseng, 2013)
Teacher value-added in their first year accounts for less than 5% of the variation in teacher quality in their fifth year of teaching (Atteberry, Loeb and Wyckoff, 2013)
Teachers benefit students for at least three years after they stop teaching them (Rothstein, 2010)
![Page 36: How do we prepare our students for a world we cannot possibly imagine? Dylan Wiliam .](https://reader034.fdocuments.us/reader034/viewer/2022052317/56649e545503460f94b4b1a4/html5/thumbnails/36.jpg)
36
To recap The highest rated teachers generate learning 30% faster
than the lowest rated teachers But the most effective teachers generate learning 400%
faster than the least effective teachers So the best observation systems we have capture less than
10% of teacher quality (in reality, less than this because most teachers are in the middle two categories)
There is therefore a real danger that teachers will “game the system” by aping features of observation protocols while becoming less effective.
![Page 37: How do we prepare our students for a world we cannot possibly imagine? Dylan Wiliam .](https://reader034.fdocuments.us/reader034/viewer/2022052317/56649e545503460f94b4b1a4/html5/thumbnails/37.jpg)
So what’s to be done?
![Page 38: How do we prepare our students for a world we cannot possibly imagine? Dylan Wiliam .](https://reader034.fdocuments.us/reader034/viewer/2022052317/56649e545503460f94b4b1a4/html5/thumbnails/38.jpg)
Expertise38
Grandmaster chess players don’t have higher IQs than average chess players
Top surgeons don’t have higher IQs, medical school grades, or higher manual dexterity than average surgeons
In general, measures of general ability account for 4% of the variability in performance
![Page 39: How do we prepare our students for a world we cannot possibly imagine? Dylan Wiliam .](https://reader034.fdocuments.us/reader034/viewer/2022052317/56649e545503460f94b4b1a4/html5/thumbnails/39.jpg)
The role of deliberate practice39
Music professors at the Hochschule der Kuenst (Academy of Music) Berlin identified 10 violin students who had the potential for careers as international soloists (“best” students)
The professors also identified a sample of 10 good, but not outstanding students (“good” students)
Researchers recruited another 10 students training to be music teachers who specialised in the violin (“Music Ed” students)
An additional 10 middle-aged professional violinists from two local orchestras were recruited to the study
Groups were matched in sex (7f, 3m) and for the first three groups, age
![Page 40: How do we prepare our students for a world we cannot possibly imagine? Dylan Wiliam .](https://reader034.fdocuments.us/reader034/viewer/2022052317/56649e545503460f94b4b1a4/html5/thumbnails/40.jpg)
How much do violinists practice?
4 6 8 10 12 14 16 18 200
5
10
15
20
25
30
35
Music Ed Good Best Professionals
Age
Hou
rs o
f pra
ctie
per w
eek
Ericsson, Krampe, and Tesch-Römer (1993)
![Page 41: How do we prepare our students for a world we cannot possibly imagine? Dylan Wiliam .](https://reader034.fdocuments.us/reader034/viewer/2022052317/56649e545503460f94b4b1a4/html5/thumbnails/41.jpg)
Violinists’ hours of practice (cumulative)41
4 6 8 10 12 14 16 18 200
2000
4000
6000
8000
10000
Music Ed Good Best Professionals
Age
Cum
ulati
ve h
ours
of p
racti
ce
![Page 42: How do we prepare our students for a world we cannot possibly imagine? Dylan Wiliam .](https://reader034.fdocuments.us/reader034/viewer/2022052317/56649e545503460f94b4b1a4/html5/thumbnails/42.jpg)
These differences are substantial…
Hours of practice by age 18
Music Education students 3420
Good violin students 5301Best violin students 7410Professional musicians 7336
By the age of 18, the best violinists have accumulated 40% more practice than good violinists
Since the amount of deliberate practice being undertaken by the best students once they are adults is close to the maximum possible, it is, essentially, impossible for the good students to catch up to the best.
42
![Page 43: How do we prepare our students for a world we cannot possibly imagine? Dylan Wiliam .](https://reader034.fdocuments.us/reader034/viewer/2022052317/56649e545503460f94b4b1a4/html5/thumbnails/43.jpg)
Talent is over-rated…43
![Page 44: How do we prepare our students for a world we cannot possibly imagine? Dylan Wiliam .](https://reader034.fdocuments.us/reader034/viewer/2022052317/56649e545503460f94b4b1a4/html5/thumbnails/44.jpg)
General conclusions about expertise44
Elite performance is the result of at least a decade of maximal efforts to improve performance through an optimal distribution of deliberate practice
What distinguishes experts from others is the commitment to deliberate practice
Deliberate practice is an effortful activity that can be sustained only for a
limited time each day neither motivating nor enjoyable—it is instrumental in
achieving further improvement in performance
![Page 45: How do we prepare our students for a world we cannot possibly imagine? Dylan Wiliam .](https://reader034.fdocuments.us/reader034/viewer/2022052317/56649e545503460f94b4b1a4/html5/thumbnails/45.jpg)
Expertise
According to Berliner (1994), experts: Excel mainly in their own domain Often develop automaticity for the repetitive operations that are
needed to accomplish their goals Are more sensitive to the task demands and social situation when
solving problems Are more opportunistic and flexible in their teaching than novices Represent problems in qualitatively different ways than novices Have faster and more accurate pattern recognition capabilities Perceive meaningful patterns in the domain in which they are
experienced Begin to solve problems slower but bring richer and more personal
sources of information to bear
45
![Page 46: How do we prepare our students for a world we cannot possibly imagine? Dylan Wiliam .](https://reader034.fdocuments.us/reader034/viewer/2022052317/56649e545503460f94b4b1a4/html5/thumbnails/46.jpg)
Effects of experience in teaching
0 1 2 3 to 5-5
-4
-3
-2
-1
0
1
Years of teaching experience
Extr
a m
onth
s pe
r yea
r o f
lear
ning
46
0 1 2 3 to 5-5
-4
-3
-2
-1
0
1
Years of teaching experience
Mathematics Reading
Rivkin, Hanushek and Kain (2005)
![Page 47: How do we prepare our students for a world we cannot possibly imagine? Dylan Wiliam .](https://reader034.fdocuments.us/reader034/viewer/2022052317/56649e545503460f94b4b1a4/html5/thumbnails/47.jpg)
Implications for education systems
Pursuing a strategy of getting the “best and brightest” into teaching is unlikely to succeed
Currently all teachers slow, and most actually stop, improving after two or three years in the classroom
Expertise research therefore suggests that they are only beginning to scratch the surface of what they are capable of
What we need is to persuade those with a real passion for working with young people to become teachers, and to continue to improve as long as they stay in the job.
![Page 48: How do we prepare our students for a world we cannot possibly imagine? Dylan Wiliam .](https://reader034.fdocuments.us/reader034/viewer/2022052317/56649e545503460f94b4b1a4/html5/thumbnails/48.jpg)
So what should teachers improve?
![Page 49: How do we prepare our students for a world we cannot possibly imagine? Dylan Wiliam .](https://reader034.fdocuments.us/reader034/viewer/2022052317/56649e545503460f94b4b1a4/html5/thumbnails/49.jpg)
Two key levers for system improvement49
A broad and balanced curriculum Effective learning environments
![Page 50: How do we prepare our students for a world we cannot possibly imagine? Dylan Wiliam .](https://reader034.fdocuments.us/reader034/viewer/2022052317/56649e545503460f94b4b1a4/html5/thumbnails/50.jpg)
21st Century skills?
Cognitive competencies Cognitive processes and strategies Knowledge Creativity
Intra-personal competencies Intellectual openness Work ethic/conscientiousness Positive core self-evaluation
Inter-personal competencies Team-work Leadership
Pellegrino and Hilton (2012)
![Page 51: How do we prepare our students for a world we cannot possibly imagine? Dylan Wiliam .](https://reader034.fdocuments.us/reader034/viewer/2022052317/56649e545503460f94b4b1a4/html5/thumbnails/51.jpg)
21st century skills: What are they, really?51
Many “21st century skills” appear to be universal, generalizable capabilities, such as Creativity Critical thinking Problem-solving Collaboration Communication
There is little evidence that they are They are best thought of as audit tools for the
examination of the curriculum in each school subject
![Page 52: How do we prepare our students for a world we cannot possibly imagine? Dylan Wiliam .](https://reader034.fdocuments.us/reader034/viewer/2022052317/56649e545503460f94b4b1a4/html5/thumbnails/52.jpg)
Principles for curriculum design52
Balanced Rigorous Coherent Vertically integrated Appropriate Focused/parsimonious Relevant
![Page 53: How do we prepare our students for a world we cannot possibly imagine? Dylan Wiliam .](https://reader034.fdocuments.us/reader034/viewer/2022052317/56649e545503460f94b4b1a4/html5/thumbnails/53.jpg)
Effective learning environments53
Effective learning environments: Create student engagement (pedagogies of engagement) Are well-regulated (pedagogies of contingency) Develop disciplinary habits of mind (pedagogies of formation)
![Page 54: How do we prepare our students for a world we cannot possibly imagine? Dylan Wiliam .](https://reader034.fdocuments.us/reader034/viewer/2022052317/56649e545503460f94b4b1a4/html5/thumbnails/54.jpg)
Formative assessment…
Fuchs & Fuchs (1986) Natriello (1987) Crooks (1988) Bangert-Drowns et al. (1991) Kluger & DeNisi (1996) Black & Wiliam (1998) Nyquist (2003) Dempster (1991, 1992)
Elshout-Mohr (1994) Brookhart (2004) Allal & Lopez (2005) Köller (2005) Brookhart (2007) Wiliam (2007) Hattie & Timperley (2007) Shute (2008)
54
![Page 55: How do we prepare our students for a world we cannot possibly imagine? Dylan Wiliam .](https://reader034.fdocuments.us/reader034/viewer/2022052317/56649e545503460f94b4b1a4/html5/thumbnails/55.jpg)
… an evolving concept (Brookhart, 2007)
Conceptualization
Information about the learning process…
… that teachers can use for instructional decisions…
…and students can use to improve performance…
…which motivates students
Source(s)
Scriven (1967)
Bloom, Hastings and Madaus (1971)
Sadler (1983; 1989)
Natriello (1987); Crooks (1988); Black and Wiliam (1998)
![Page 56: How do we prepare our students for a world we cannot possibly imagine? Dylan Wiliam .](https://reader034.fdocuments.us/reader034/viewer/2022052317/56649e545503460f94b4b1a4/html5/thumbnails/56.jpg)
56
“An assessment functions formatively to the extent that evidence about student achievement is elicited, interpreted, and used by teachers, learners, or their peers, to make decisions about the next steps in instruction that are likely to be better, or better founded, than the decisions they would have taken in the absence of the evidence that was elicited.”
![Page 57: How do we prepare our students for a world we cannot possibly imagine? Dylan Wiliam .](https://reader034.fdocuments.us/reader034/viewer/2022052317/56649e545503460f94b4b1a4/html5/thumbnails/57.jpg)
Mapping out the terrain
Annual
Interim
Weekly
Daily
Instructional Guidance
(“formative”)
Describing Individuals
(“summative”)
Institutional Accountability (“evaluative”)
FunctionFunction
Hourly
Tim
esca
le
High-stakes accountability
Academic promotion
Hinge-pointquestions
End-of-unit tests
Benchmark
Exit pass
Common formative assessments
End-of-course exams
Before the end-of-unit tests
Growth
57
![Page 58: How do we prepare our students for a world we cannot possibly imagine? Dylan Wiliam .](https://reader034.fdocuments.us/reader034/viewer/2022052317/56649e545503460f94b4b1a4/html5/thumbnails/58.jpg)
Main approaches to formative assessment
Professional Learning Communities “…an inclusive group of people, motivated by a shared learning vision, who support and work with each other, finding ways, inside and outside their immediate community, to enquire on their practice and together learn new and better approaches that will enhance all pupils’ learning.” (Stoll et al., 2006)
Two main approaches Professional learning communities (PLCs)
focused on student achievement Teacher learning communities (TLCs)
focused on improvements in teacher expertise
![Page 59: How do we prepare our students for a world we cannot possibly imagine? Dylan Wiliam .](https://reader034.fdocuments.us/reader034/viewer/2022052317/56649e545503460f94b4b1a4/html5/thumbnails/59.jpg)
Complementary processes
Quality control Common assessments Improvement through better
team work and systems Focus on individual outcomes for
students Regular meetings focused on
data 16 points on PISA (in two to three
years)
Quality assurance Highly structured meetings Improvement through
increased teacher capacity Focus on teachers’ individual
accountability for change Regular meetings focused on
teacher change 30 points on PISA (in two to
three years)
59
Data-driven PLCs Classroom FA TLCs
![Page 60: How do we prepare our students for a world we cannot possibly imagine? Dylan Wiliam .](https://reader034.fdocuments.us/reader034/viewer/2022052317/56649e545503460f94b4b1a4/html5/thumbnails/60.jpg)
Unpacking classroom formative assessment
Where the learner is going Where the learner is How to get there
Teacher
Peer
Learner
Clarifying, sharing and
understanding learning
intentions
Engineering effective discussions, tasks, and
activities that elicit evidence of learning
Providing feedback that
moves learners forward
Activating students as learningresources for one another
Activating students as ownersof their own learning
60
![Page 61: How do we prepare our students for a world we cannot possibly imagine? Dylan Wiliam .](https://reader034.fdocuments.us/reader034/viewer/2022052317/56649e545503460f94b4b1a4/html5/thumbnails/61.jpg)
Examples of techniques
Learning intentions “sharing exemplars”
Eliciting evidence “mini white-boards”
Providing feedback “match the comments to the essays”
Students as owners of their learning “colored cups”
Students as learning resources “pre-flight checklist”
![Page 62: How do we prepare our students for a world we cannot possibly imagine? Dylan Wiliam .](https://reader034.fdocuments.us/reader034/viewer/2022052317/56649e545503460f94b4b1a4/html5/thumbnails/62.jpg)
And one big idea
Where the learner is going Where the learner is How to get there
Teacher
Peer
Learner
62
Using evidence of achievement to adapt
what happens in classrooms to meet
learner needs
![Page 63: How do we prepare our students for a world we cannot possibly imagine? Dylan Wiliam .](https://reader034.fdocuments.us/reader034/viewer/2022052317/56649e545503460f94b4b1a4/html5/thumbnails/63.jpg)
Formative assessment and other priorities
Formative assessment is an integral part of many current policy priorities: Framework for teaching (Danielson) Common formative assessments (DuFour) Differentiated instruction (Tomlinson) Response to (instruction and) intervention
63
![Page 64: How do we prepare our students for a world we cannot possibly imagine? Dylan Wiliam .](https://reader034.fdocuments.us/reader034/viewer/2022052317/56649e545503460f94b4b1a4/html5/thumbnails/64.jpg)
Framework for teaching (Danielson 1996)
Four domains of professional practice1. Planning and preparation2. Classroom environment3. Instruction4. Professional responsibilities
Links with student achievement (Sartain, et al. 2011) Domains 1 and 4: no impact on student achievement Domains 2 and 3: some impact on student achievement
64
![Page 65: How do we prepare our students for a world we cannot possibly imagine? Dylan Wiliam .](https://reader034.fdocuments.us/reader034/viewer/2022052317/56649e545503460f94b4b1a4/html5/thumbnails/65.jpg)
The framework in detail
Domain 2: The classroom environment 2a: Creating an environment of respect and rapport 2b: Establishing a culture for learning 2c: Managing classroom procedures 2d: Managing student behavior 2e: Organizing physical space
Domain 3: Instruction 3a: Communicating with students 3b: Using questioning and discussion techniques 3c: Engaging students in learning 3d: Using assessment in instruction 3e: Demonstrating flexibility and responsiveness
65
![Page 66: How do we prepare our students for a world we cannot possibly imagine? Dylan Wiliam .](https://reader034.fdocuments.us/reader034/viewer/2022052317/56649e545503460f94b4b1a4/html5/thumbnails/66.jpg)
Formative assessment and domain 3
Communicating with students
Using questioning and discussion techniques
Engaging students in learning
Using assessment in instruction
Demonstrating flexibil-ity and responsiveness
Sharing learning intentions with students
Eliciting evidence
Feedback
Students as learning resources
Students as owners of their learning
Framework for teaching Classroom formative assessment
66
![Page 67: How do we prepare our students for a world we cannot possibly imagine? Dylan Wiliam .](https://reader034.fdocuments.us/reader034/viewer/2022052317/56649e545503460f94b4b1a4/html5/thumbnails/67.jpg)
Differentiated instruction: not a new idea
Differentiation in action (Stradling & Saunders, 1993) Differences in
educational goals curriculum structure course content learning tasks teaching approach pace of learning assessment review
67
![Page 68: How do we prepare our students for a world we cannot possibly imagine? Dylan Wiliam .](https://reader034.fdocuments.us/reader034/viewer/2022052317/56649e545503460f94b4b1a4/html5/thumbnails/68.jpg)
Most definitions of DI are vague
“While the concept of ‘differentiated instruction’ can be defined in many ways, as good a definition as any is ensuring that what a student learns, how he/she learns it, and how the student demonstrates what he/she has learned is a match for that student's readiness level, interests, and preferred mode of learning.” (Tomlinson, 2004 p. 188)
“To differentiate instruction is to recognize students' varying background knowledge, readiness, language, preferences in learning and interests; and to react responsively. Differentiated instruction is a process to teaching and learning for students of differing abilities in the same class.” (Hall, Strangman, & Meyer, 2011)
68
![Page 69: How do we prepare our students for a world we cannot possibly imagine? Dylan Wiliam .](https://reader034.fdocuments.us/reader034/viewer/2022052317/56649e545503460f94b4b1a4/html5/thumbnails/69.jpg)
Differentiated instruction and formative assessment
69
Aspects of differentiated instruction (Hall, Strangman & Meyer, 2008) FA?
Content
Several elements and materials are used
Align tasks and objectives to learning goals 3Instruction is concept-focused and principle-driven
Process
Flexible grouping is consistently used
Classroom management benefits students and teachers
Products
Initial and on-going assessment of student readiness and growth 3Students are active and responsible explorers 3Vary expectations and requirements for student responses 3
Misccelaneous
Clarify key concepts and generalizations
Use assessment as a teaching tool 3Emphasize critical and creative thinking as a goal in lesson design
Engaging all learners is essential 3Balance between teacher-assigned and student-selected tasks
![Page 70: How do we prepare our students for a world we cannot possibly imagine? Dylan Wiliam .](https://reader034.fdocuments.us/reader034/viewer/2022052317/56649e545503460f94b4b1a4/html5/thumbnails/70.jpg)
Response to (instruction and) intervention
“Response to intervention integrates assessment and intervention within a multi-level prevention system to maximize student achievement and reduce behavior problems. With RTI, schools identify students at risk for poor learning outcomes, monitor student progress, provide evidence-based interventions and adjust the intensity and nature of those interventions depending on a student’s responsiveness, and identify students with learning disabilities.” (National Center on Response to Intervention, 2010)
Two “creation myths” for RT(I)I A protocol for preventing academic failure (progress monitoring,
early—research-based—intervention) An alternative to IQ testing in the identification of
learning disabilities
70
![Page 71: How do we prepare our students for a world we cannot possibly imagine? Dylan Wiliam .](https://reader034.fdocuments.us/reader034/viewer/2022052317/56649e545503460f94b4b1a4/html5/thumbnails/71.jpg)
Response to (instruction and) intervention
Key points Tier 1 must be high-quality, evidence-based
instruction Student progress must be monitored Failure to progress triggers additional support
Formative assessment Makes tier 1 instruction as effective as it can be Allows assessment of progress (for tier 2 assessment)
71
![Page 72: How do we prepare our students for a world we cannot possibly imagine? Dylan Wiliam .](https://reader034.fdocuments.us/reader034/viewer/2022052317/56649e545503460f94b4b1a4/html5/thumbnails/72.jpg)
So much for the easy bit
![Page 73: How do we prepare our students for a world we cannot possibly imagine? Dylan Wiliam .](https://reader034.fdocuments.us/reader034/viewer/2022052317/56649e545503460f94b4b1a4/html5/thumbnails/73.jpg)
A model for teacher learning
Content, then process Content (what we want teachers to change):
Evidence Ideas (strategies and techniques)
Process (how to go about change): Choice Flexibility Small steps Accountability Support
73
Science
Design
![Page 74: How do we prepare our students for a world we cannot possibly imagine? Dylan Wiliam .](https://reader034.fdocuments.us/reader034/viewer/2022052317/56649e545503460f94b4b1a4/html5/thumbnails/74.jpg)
Supportive accountability
What is needed from teachers: A commitment to:
The continual improvement of practice Focus on those things that make a difference to students
What is needed from leaders: A commitment to engineer effective learning
environments for teachers by: Creating expectations for continually improving practice Keeping the focus on the things that make a difference to
students Providing the time, space, dispensation, and support for
innovation Supporting risk-taking
74
![Page 75: How do we prepare our students for a world we cannot possibly imagine? Dylan Wiliam .](https://reader034.fdocuments.us/reader034/viewer/2022052317/56649e545503460f94b4b1a4/html5/thumbnails/75.jpg)
Teacher learning communities
![Page 76: How do we prepare our students for a world we cannot possibly imagine? Dylan Wiliam .](https://reader034.fdocuments.us/reader034/viewer/2022052317/56649e545503460f94b4b1a4/html5/thumbnails/76.jpg)
We need to create time and space for teachers to reflect on their practice in a structured way, and to learn from mistakes.
Bransford, Brown & Cocking (1999)
“Always make new mistakes.”Esther Dyson
“Ever tried. Ever failed. No matter. Try again. Fail again. Fail better.”
Beckett (1984)
76
![Page 77: How do we prepare our students for a world we cannot possibly imagine? Dylan Wiliam .](https://reader034.fdocuments.us/reader034/viewer/2022052317/56649e545503460f94b4b1a4/html5/thumbnails/77.jpg)
Teacher learning communities
Plan that the TLC will run for two years Identify 10 to 12 interested colleagues:
Conscripts vs. volunteers Composition:
Similar assignments (e.g., early years, math/science) Mixed subject/mixed phase Hybrid
Secure institutional support for: Monthly workshops (75–120 minutes each, inside or outside school
time) Time between workshops (two hours per month in school time) for
collaborative planning and peer observation Any necessary waivers from school policies
77
![Page 78: How do we prepare our students for a world we cannot possibly imagine? Dylan Wiliam .](https://reader034.fdocuments.us/reader034/viewer/2022052317/56649e545503460f94b4b1a4/html5/thumbnails/78.jpg)
A “signature pedagogy” for teacher learning
Every monthly TLC workshop should follow the same structure and sequence of activities:
Activity 1: Introduction (5 minutes) Activity 2: Starter activity (5 minutes) Activity 3: Feedback (25–50 minutes) Activity 4: New learning about formative assessment
(20–40 minutes) Activity 5: Personal action planning (15 minutes) Activity 6: Review of learning (5 minutes)
78
![Page 79: How do we prepare our students for a world we cannot possibly imagine? Dylan Wiliam .](https://reader034.fdocuments.us/reader034/viewer/2022052317/56649e545503460f94b4b1a4/html5/thumbnails/79.jpg)
Every TLC needs a leader
The job of the TLC leader(s): To ensure that all necessary resources (including
refreshments!) are available at workshops To ensure that the agenda is followed To maintain a collegial and supportive environment
But most important of all: It is not to be the formative assessment “expert”
79
![Page 80: How do we prepare our students for a world we cannot possibly imagine? Dylan Wiliam .](https://reader034.fdocuments.us/reader034/viewer/2022052317/56649e545503460f94b4b1a4/html5/thumbnails/80.jpg)
Peer observation
Run to the agenda of the observed, not the observer: Observed teacher specifies focus of observation:
e.g., teacher wants to increase wait time Observed teacher specifies what counts as evidence:
Provides observer with a stopwatch to log wait times Observed teacher owns any notes made during the
observation
80
![Page 81: How do we prepare our students for a world we cannot possibly imagine? Dylan Wiliam .](https://reader034.fdocuments.us/reader034/viewer/2022052317/56649e545503460f94b4b1a4/html5/thumbnails/81.jpg)
Summary
Raising achievement is important Raising achievement requires improving teacher quality Improving teacher quality requires teacher professional
development To be effective, teacher professional development must
address: What teachers do in the classroom How teachers change what they do in the classroom
Classroom formative assessment + teacher learning communities: A point of (uniquely?) high leverage
81
![Page 82: How do we prepare our students for a world we cannot possibly imagine? Dylan Wiliam .](https://reader034.fdocuments.us/reader034/viewer/2022052317/56649e545503460f94b4b1a4/html5/thumbnails/82.jpg)
Thank you