1 “Best Practice” & The Habits We MUST Change Grant Wiggins December 2006.
Grant Wiggins on Grading - Authentic Education · 4 Grant Wiggins on Grading 10 Criteria for any...
Transcript of Grant Wiggins on Grading - Authentic Education · 4 Grant Wiggins on Grading 10 Criteria for any...
1
Grant Wiggins on Grading
1
GradesGrades
2
Provocations...• What is a grade’s purpose? What follows for what it
should represent, regardless of tradition?• Should we - would we - give grades if we didn’t have to?• Is it acceptable for teachers at the same grade and
teaching the same course to view the same or similarstudent work differently? What follows for gradingpolicy and practice?
• Why calculate the ‘mean’ when it penalizes progress andover-rewards inconsistency?
• Why isn’t every public school giving grades against statestandards if those standards are obligatory?
• Why do parents like grades? Why is this interest sound,even if teachers don’t like giving them?
3
Looking closely at habit• Always tricky: habit runs so deep, we
rationalize it without realizing it• We ask that you work hard to keep an
open mind and resist the ‘Yes, but...”reaction that is inevitable
2
Grant Wiggins on Grading
4
Essential Qs of Grading• Why grade? Why not?• Audience: For whom are we grading?• Purpose: What is the primary purpose of a
grade? (What if our purposes conflict?)• What should we grade? What shouldn’t we
grade?• How should a grade be determined?• How consistent should grades and grading
policy be (across time and across teachers)?• How is quality control in grading best
achieved?
5
An indication of unthinkingand harmful habit
• I received the following email: “I was meeting with our high school Advanced
Placement Teachers, who were expressingconcerns about our open enrollment processand the high failure rate. One math teachersaid that while a particular student was nowmaking grades in the 80's, she had made a 12on an initial test, “so there is no way she's goingto make a passing grade for the first nineweeks.”
6
Other countries• In Belgium, France, Morocco, Portugal, Peru,Venezuela, Iran and Tunisia a 20-pointgrading scale is used, in which 20 is thehighest grade and 0 is the lowest.
• The "passing" grade is usually 10• Grades of 10-11 is "adequate".• Grades of 12 or 13 are "passable"(better than
adequate)• Grades of 14 to 15 are "good" (better than "passable")• Grades of 16 to 17 are regarded as excellent and
outstanding, respectively. From this point on, you havetruly mastered the course.
• Grades of 18 to 19 are nearing perfection.• Grades of 20 are just perfect.
3
Grant Wiggins on Grading
7
Denmark• The current Danish scale is called the 13-scale and consists of
10 grades ranging from 00 to 13, with 00 being the worst.• 00 completely unacceptable performance.• 03 very hesitant, very insufficient and unsatisfactory
performance.• 5 hesitant and not satisfactory performance.• 6 just acceptable performance.• 7 mediocre performance, slightly below average.• 8 average performance.• 9 good performance, a little above average.• 10 excellent but not particularly independent performance.• 11independent and excellent performance.• 13 exceptionally independent and excellent performance.
8
Where is it written thatwe must, as teachers,…• Give only one (aggregate )grade to each
student in our class, even if the transcriptonly permits one grade?
• Calculate grades using the mean score (asopposed to, say, the median or the trend)?
• Grade each student in a diverse classroomagainst the same standard?
• Keep different weighting and gradingvariables constant all year?
• Assess and grade all students at the samepoint in time (thereby making it impossible todo a thorough and timely job of grading)?
9
Judging grading policiesand practices
• No progress without sound criteria: We need criteria by which we can
evaluate grading policies and practices ina pedagogically wise and defensiblemanner
Only agreed-upon criteria can free usfrom the tyranny of bad habits that wetry to rationalize
4
Grant Wiggins on Grading
10
Criteria for any gradingsystem to meet:
• Honest feedback about one’s standing• Fair to each student and other students• Transparent and without mystery• Credible to clients & constituencies• Valid assessment against key long-term
learning goals• Useful (actionable) and user-friendly
information about performance and howto improve
• Pedagogically wise - it sends the rightmessage and gets the incentives right forlearners
11
Exercise: So, whatfollows?
• In small groups, we’ll analyze somepractical implications for individualand school-wide grading, given eachcriterion: Fair Honest Transparent Credible Useful & User-Friendly Pedagogically wise
12
Symbols and theirmeaning
• A big problem may not be “grades” but asingle grade, hiding vast differences inperformance, habits, and attitudes! Why do we only put single grades on report
cards and transcripts when disaggregated datawould greatly help the reader understand whatthe learner has accomplished and who thelearner is as a learner?
5
Grant Wiggins on Grading
14
Break out the IndependentVariables in Grading
• Validity, transparency, and usefulnessrequire knowing at least these 3 elements,broken out:
What is my level of achievement? What has been my progress (against
standards)? How are my work habits and attitudes?
15
Independent Variables inGrading
• Most Grading Systems conceal more thanthey reveal!
Achievement (against standards) Sub-achievement (discrete competencies
making up overall achievement) Progress over time (against standards) Habits and attitudes (effort, open to
learning, etc.)
6
Grant Wiggins on Grading
16
Problems with the(Single) Grade
• Different Work, Same Grade Average Achievement, Great Progress
vs. High Achievement, No progress Different sub-scores on multiple
rubrics, SAME AVERAGE Wild swings vs. consistency - SAME
AVERAGE
17
sub-grades vital in asubject
• “Yes, I know I got a ‘B’ in LanguageArts, but break it down for me” by – Grades against each state standard Control over varied genres of reading Control over varied genres of writing Each criterion used in rubrics related to
my writing The quality of my participation
18
Some implications,explored in this conference
• Thus, we are not against grades! We are againstdumb grading systems!
Conventional summary grades,calculated by computing thearithmetical ‘mean’, areindefensible (despite thelongstanding habit of doing so)
Failure to give grades against statestandards is irresponsible
Providing only a single grade isunhelpful
7
Grant Wiggins on Grading
19
Beyond the Mean: ithides vital information
• The “average” (the mean) hides or under-rewards:
Trend of work over time - progress Consistency of work quality Key feedback on independent variables
making up the grade Degree of true mastery against an
objective standard
20
Beyond the mean: OtherMethods of “Averaging”
• Beyond the mean: Reliable pattern (median, mode) Consistency (range, Standard Deviation.) Throw out the highest and lowest scores
(Olympics) Factor in degree of difficulty of the tasks
(as in music, diving, gymnastics)
21
Beyond the ‘average’:Other Methods for
Determining Grades• think of diving, karate, chess:
Total score over time (e.g. Olympics) Final mastery level (chess, karate,
ACTFL) Measure of progress against standards
over time (pre/post, the value-added)
8
Grant Wiggins on Grading
22
Varying the weightingand honoring difference
• Why penalize early effort or prior level? Don’t grade (or average in) all formative work INITIALLY grade Effort and Progress more than
level of achievement in the first month(s),gradually making level of achievement moresignificant
Have LEVELS in your class, via pre-testing; make itlike skiing - Novice/Intermediate/Expert - wherethat designation is given along with the grade - Awork for a novice, but C work for Expert
Emphasize some rubrics more than others initially,then give grades against all rubrics by the end
23
Provide sub-grades for betterfeedback
• A standards-based report card: Grade control over core genres in
Language Arts, state standards in mathand science
Distinguish between mastery of contentvs. mastery of processes
24
Feedback: on which keytasks?
• Key to reform: to view assessment in termsof longer-term learning goals, not justgrades on quizzes of recent content;
• Feedback against – Key competencies/exit standards Key transfer tasks Key habits of mind Key long-term inquiries (essential questions)
9
Grant Wiggins on Grading
25
Longitudinal Progressvia Rubrics
• Rubrics to track progress over timeagainst standards On a novice-expert continuum in addition
to rubrics for judging increasingsophistication of ideas and processes• cf. Rubrics in ACTFL, American Literacy
Profiles, Meisels’ Work Sampling System,chess/karate/bridge etc.
26
Longitudinal Progressvia Rubrics
• Too Many Rubrics are Task-Specific A reporting and grading system should
align, for tracking progress over time Rubrics should be more generalized and
linked to exit-level standards• Too many grading systems and rubrics
conflate descriptive LEVEL withjudgment about WORK QUALITY(regardless of level)
27
ACTFL Example• Novice-Low: Oral production consists of isolated words
and perhaps a few high-frequency phrases. Essentiallyno functional communicative ability.
• Novice-Mid: Oral production continues to consist ofisolated words and learned phrases within verypredictable areas of need, although quantity isincreased. Vocabulary is sufficient only for handlingsimple, elementary needs and expressing basiccourtesies. Utterances rarely consist of more than two orthree words and show frequent long pauses andrepetition of interlocutor's words…. Some Novice-Midspeakers will be understood only with great difficulty.
10
Grant Wiggins on Grading
28
ACTFL Example• Novice-High Able to satisfy partially the
requirements of basic communicative exchangesby relying heavily on learned utterances butoccasionally expanding these through simple re-combinations of their elements…. Shows signs ofspontaneity although this falls short of realautonomy of expression…. Vocabulary centers onareas such as basic objects, places, and mostcommon kinship terms. Pronunciation may still bestrongly influenced by first language. Errors arefrequent and, in spite of repetition, some Novice-High speakers will have difficulty beingunderstood even by sympathetic interlocutors.
29
ACTFL• Advanced Able to satisfy the requirements of
everyday situations and routine school and workrequirements…. Can narrate and describe withsome details, linking sentences together smoothly.Can communicate facts and talk casually abouttopics of current public and personal interest, usinggeneral vocabulary. Shortcomings can often besmoothed over by communicative strategies, such aspause fillers…. Circumlocution which arises fromvocabulary or syntactic limitations very often is quitesuccessful, though some groping for words may stillbe evident. The Advanced-level speaker can beunderstood without difficulty by nativeinterlocutors.
30
UK natl curriculum:science
• Level 4 Pupils recognize that scientific ideas are based on evidence. Intheir own investigative work, they decide on an appropriate approachfor example, using a fair test to answer a question. Where appropriate,they describe, or show in the way they perform their task, how to varyone factor while keeping others the same. Where appropriate, theymake predictions. They select information from sources provided forthem. They select suitable equipment and make a series of observationsand measurements that are adequate for the task. They record theirobservations, comparisons and measurements using tables and barcharts. They begin to plot points to form simple graphs, and use thesegraphs to point out and interpret patterns in their data. They begin torelate their conclusions to these patterns and to scientific knowledge andunderstanding, and to communicate them with appropriate scientificlanguage. They suggest improvements in their work, giving reasons.
11
Grant Wiggins on Grading
31
From UK natl. curriculum• Level 8 Pupils give examples of scientific explanations or models that
have had to be changed in the light of additional scientific evidence. Theyevaluate and synthesize data from a range of sources. They recognize thatinvestigating different kinds of scientific questions requires differentstrategies, and use scientific knowledge and understanding to select anappropriate strategy in their own work. They decide which observationsare relevant in qualitative work and include suitable detail in theirrecords. They decide the level of precision needed in comparisons ormeasurements, and collect data enabling them to test relationshipsbetween variables. They identify and begin to explain anomalousobservations and measurements and allow for these when they drawgraphs. They use scientific knowledge and understanding to drawconclusions from their evidence. They consider graphs and tables ofresults critically. They communicate findings and arguments usingappropriate scientific language and conventions, showing awareness of arange of views.
32
Reading LevelsAssessment
• Lexile Scores• Degrees of Reading Power• Meisel’s Work Sampling System• American Literacy Profiles• UK Reading rubrics
33
12
Grant Wiggins on Grading
34
NO justification for fittinggrades to a bell curve
• True both mathematically and pedagogically Mathematics: the point of a curve is to show the
normal distribution of a vast number ofelements, likely to be ‘naturally’ distributed -e.g. the cholesterol numbers for an entirepopulation
Education: We aim for performanceimprovement across the board; a normal curve isa sign of failure in a small class
• Note: so-called “curving” of a set of test grades is different, andmay reflect a wise adjustment based on the hunch that the rawgrades are not reliable
35
Bloom et al:Grading on a Curve
• Grading on a curve - unjustified: “There is nothing sacred about the
normal curve. It is the distribution mostappropriate to chance and randomactivity. Education is a purposefulactivity, and we seek to have thestudents learn what we have to teach…”
36
Bloom et al:Grading on a Curve
“If we are effective in our instruction, thedistribution of achievement should be verydifferent from the normal curve. In fact,we may insist that our efforts have beenunsuccessful to the extent that [grades]approximate the normal distribution.”• Evaluation to Improve Learning, p. 53
13
Grant Wiggins on Grading
37
Why fitting to a curve iswrong - and wrong-headed• More generally:
Fitting grades to a normal curve is toexaggerate student differences and reportagainst norms not standards.
We should seek the opposite: let the gradesfall where they may, against worthystandards.
Standards and expectations are lowered ifwe fail to give disinterested grades againstclear criteria and models
38
Example of “Mike” -a high-mobility student
Year Location Grade
1989 Salem C+ 1990 Vero Beach C -1991 Bakersfield C1992 Albuquerque A1992 San Antonio A +1992# Los Angeles C-1993 Los Angeles B+
# 1st year of mainstreaming
39
“Mike” (continued)
Year Location Score
1989 Salem 2681990 Vero Beach 2501991 Bakersfield 2771992 Albuquerque 3411992 San Antonio 3711992# Los Angeles 2321993 Los Angeles 318
14
Grant Wiggins on Grading
40
41
42
15
Grant Wiggins on Grading
43
Grading Criteria,Grading Criteria,explainedexplained
44
Key tension: Honest andFair
It is difficult but essential to be both; mostreports err on one side or the other
Honest: a dispassionate account of studentstrengths and weaknesses against standards andnormed expectationsFair: mindful of extenuating circumstances,personal strengths and challenges, local norms,and reasonable expectations of that individual
45
Fair• Honors idiosyncracies and appropriate
extenuating circumstances Don’t compare apples and oranges:
• Don’t confuse “behind” with “differentdevelopmental path”
• Don’t emphasize narrow and arbitrary test types• Be explicit about special-student status
16
Grant Wiggins on Grading
46
Fair• We work consciously and carefully to
eliminate bias and capriciousness Bloom et al.: “Unfortunately, extraneous
factors, such as the student’s speech,sociability, personality can unconsciouslyinfluence a teacher in grading”
47
Honest• Faithfully describes absolute levels ofperformance Honest reports do not flinch from stating the
level of performance against standards, andmaking predictions about likely exit status
Dishonest to report only the strengths - cf.some narrative and portfolio systems
48
Credible• Means that it is trustworthy and
believable information for ALLconstituencies - implying: disinterested scoring and grading on a
regular basis external validation in terms of tasks,
criteria, standards
17
Grant Wiggins on Grading
49
To be credible a grade(judgment) must be... Based on multiple & varied assessment, and
sensitive to MI and learning styles Comparable grades/marks Given by someone who consistently monitors
based on instruction Measuring achievement toward standards Inter-rater reliability Measured over time More than one person has input
50
To be credible a grade(judgment) must ...
Be specific about what the child knows and is able to do Not confuse what a child does with what a child can do
(“potential” separated from “achievement”) Knowing the criteria and standards upon which the grade
was given Discrepant data not swept under the rug! Fair opportunities to learn Observable, measurable data is central to a judgment
51
Credibility: QualityControl in Grading
• Grades are too often unreliable orinvalid (and inconsistent acrossteachers) Agreement on “anchors” and grading
criteria needed Grading policies and oversight needed Need to link grades to external standards
18
Grant Wiggins on Grading
52
Useful• Refers to the reader’s ability to use
and profit from the report Links back to clarity about purpose(s) and
audience(s) Not likely to be useful if fails to assess
and evaluate strengths and weaknesses
53
User-Friendly• Refers to intelligibility and ease of
access of information Many reforms of grading and reporting
have not been user-friendly, though highlyvalued by teachers
Few parents understand why letter gradesare inadequate and are suspicious ofreforms
54
User-friendly (feasible)• Must be manageable for teachers
We need to rethink our use of timeto ensure rich, timely, and accurateinformation• Why do we unthinkingly report for ALL students
on the same day? Why not stagger report cardsby sections of the alphabet to lighten the loadand ensure better reports?
19
Grant Wiggins on Grading
55
Toward CredibleGrading & Scoring
• Setting standards locally requires: on-going faculty group scoring of student
work system-wide rubrics and 'anchors' for
shared standards on-going study of performance results
leading to recommendations validating local grading standards
56
Credible
Valid, supported analysis, using astandard approachA trustworthy source, an expertjudgmentBelievable, plausible, based on acareful consideration
57
Achievement, defined
Performance against academic/intellectual standards
Measure of “pure” performance level,without extenuating factors or normsExemplars, criteria and specifications areexplicit and deliberately referenced
20
Grant Wiggins on Grading
58
Progress, defined
Performance measured over timeagainst explicit standards
Requires a longitudinal assessmentsystem, on a novice-expert continuum
We need to ensure there is both pre-testingand post testingLongitudinal rubrics
59
Purposes: Assess vs. Evaluate• Assess: comment on and analyze work
against valid criteria - so as to givevalue-neutral feedback & guidance
• Evaluate: summarize implications ofresults, by assigning a value to theoverall performance, against somestandard(s) or expectation(s)
6060
“Objective” &“Subjective”
• Human judgments can and must be as“objective” as possible! Objective: disinterested, valid, and reliable
judging against standards Subjective (in the bad sense): Biased or
capricious judgment Subjective (in the good sense): A human
judgment based on a variety of information, notmechanically computed
• Beware of defining machine-grading as “objective” andhuman scoring as “subjective”
21
Grant Wiggins on Grading
61
“Objective” &“Subjective”
• Thus, to say that local grades are “sosubjective” is misleading The issue is sound professional judgment We need more disinterested judging locally to
make scores and grades defensible - objective• E.g. IB, Advanced Placement, group scoring of writing or
music against standards and rubrics, etc.
62
A Mantra:• Give fewer grades and more feedback
We tend to give too much evaluation, toosoon
We tend to “grade” without helping thestudent really grasp the feedback thegrade is based on
Giving better feedback enables thestudent to take greater control of givingthemselves (or seeking) advice, sooner
63
Excellent feedback• The most common answers to our
exercise on good feedback: Timely user-friendly - in approach and amount Descriptive & specific re: performance Consistent Expert Accurate Honest, yet constructive Derived from concrete standards On-going
22
Grant Wiggins on Grading
64
for furtherinformation...Contact me:
[email protected] grading and reporting bulletinboards at authenticeducation. orgwww.bigideas.org Our monthly resourceon a theme related to teaching andassessing
The archive contains an issue devoted tograding (winter 2005)