Developing Evaluation Rubrics or Scoring Guides

for UMass-Dartmouth January 27, 2012

What is a Rubric?A Scoring Tool

◦Lays out specific expectations (criteria) for an assignment or task

◦Describes different levels of performance on those criteria

Why Use a Scoring Guide?Increase reliability when gradingClarify expectations for studentsReduce time spent gradingProvide detailed feedback in a

timely mannerCommunicate with others about

criteria and standards

When to use a Rubric?For any high-stakes assessment

situation

When several different people are doing the assessment

When the criteria or expectations need to be clear!

Sharing OneFor a 200-level course in writing

arguments

For one of five assignments, assigned after mid-term

Steps in Creating a RubricFirst, Reflection

◦Why did you create this task or expectation? How does it relate to the readings or content of the course? What skills do students need to complete this project successfully? What scaffolding has the course provided thus far?

Next Step: ListingAfter completing the task

description, you will want to make a list of all of the qualities or characteristics you are looking for as students complete this task.

Third, Grouping and LabelingDecide on priorities, as your

rubric cannot cover everything. Also choose what you will call the levels of performance and what adjectives you think best describe the features.

Finally, Describing each Level of Performance

Repeating key words or phrases across all levels is appropriate

Example: Support for Claim

Writer offers an abundance of vivid examples, concrete details, and/or relevant passages of text to support the claim.

Writer offers adequate support (examples, details, passages) for the claim.

Writer offers little support for the claim or has depended on repetition as support.

Use of evidence shows original thought or unusual connections.

Use of evidence is solid but may be predictable or not particularly striking.

Use of evidence is entirely predictable, thin, or not relevant.

Let’s Get StartedFor the assignment you brought

with you, write the task description--concisely. Look at your assignment sheet, think about the readings or context, and repeat some of the language students will be given re: the product, performance, or task.

NextDecide what to call your three

levels of performance or achievement

Examples:◦ Exemplary, Proficient, Emerging◦ Excellent, Good, Developing◦ Strong, Satisfactory, Weak◦ Amazing, Effective, Needs Work◦ Above Average, Average, Below Average◦ Good, Fair, Poor

Identify 5-6 criteria or dimensionsExamples:

◦Clear Thesis, Adequate Support, Logical Organization, . . .

◦Eye Contact, Announces Topic, Elocution, . . .

◦Follows Procedure, Conducts Follow-up, . . .

◦Greets Clients by Name, Maintains Confidentiality of Files, . . .

Describe each LevelMake sure the adjectives indicate

qualitative differences

Use qualifiers

Other than adjectives or qualifiers, repetition is appropriate

HintsMost instructors find it easiest to

describe first the top level and then the bottom level, saving the middle for last.

If you describe FIVE levels of performance, you will work much harder, and students will see them as A, B, C, D, and F.

Good AdviceIdentify only features that can be

measured “objectively.”

◦Not, for example, ◦“seems to enjoy the task”◦or◦“has a poor attitude”

More Good AdviceDo not cram the “Dimensions”

column (far left) full of descriptors or information.◦Stick to 1-3 word phrases

Do not name a category “overall”!

IssuesWhat about products that “don’t

fit” anywhere on the scoring guide?