Dr. Mike Fulton and Dr. Jennifer Schneider

Pattonville School District

The Role of Rubrics in Assessment by Design: Do You Think Like An Assessor or Like An Activity Designer?

Two Approaches to Thinking about Assessment

When thinking like an assessor, we ask-

What would be sufficient and revealing evidence of understanding?

Given the goals, what performance tasks must anchor the unit and focus the instructional work?

What are the different types of evidence required by desired results?

Against what criteria will we appropriately consider work and assess levels of quality

Did the assessments reveal and distinguish those who really understood from those who only seem to? Am I clear on the reasons behind learner mistakes?

When thinking like an activity designer (only), we ask--

What would be fun and interesting activities on this topic?

What projects might students wish to do on this topic?

What tests should I give, based on the content I taught?

How will I give students a grade (and justify it to their parents)?

How well did the activities work? How did the students do on the test?

Continuum of Assessments

Performance Assessments Academic Prompts Quiz and Test Items Informal Checks for Understanding

Performance tasks Complex challenges that mirror the issues and problems faced by

adults. Ranging in length from short-term task to long-term, multistaged projects, they yield one or more tangible products and performances. They differ from academic prompts in the following ways: Involve a real of simulated setting and the kind of constraints and

opportunities an adult would find in a similar situation Are based on a specific purpose that relates to the audience Typically require the student to address an identified audience (real

or simulated) Allow students greater opportunity to personalize the task Are not secure: task, evaluative criteria, and performance standards

are known in advance and guide student work

Performance tasks examples

CA W3A 8th grade: Compose a variety of texts, using narrative, descriptive, expository, and/or persuasive features in various formats, including workplace communication (e.g., business letter with a correctly addressed envelope, email communications)

SC VII1B Physics Scientific inquiry relies upon gathering evidence from qualitative and quantitative observations

H2A 2nd grade Identify healthy food choices (see MyPyramid.gov) in each of the five food groups (sorting play food or pictures into food groups. Compare and contrast the effects of nutrient-dense and empty-calorie foods on the body (e.g., soda/skim milk, chips/apple; cookies/carrots)

Academic Prompts Open-ended questions or problems that require the students to think

critically, not just recall knowledge, and to prepare specific academic response, product, or performance. Such questions or problems Require constructed response to specific prompts under school and

exam conditions Are often “ill structured,” requiring the development of a strategy Involve analysis, synthesis, and evaluation Typically require an explanation or defense of the answer given

and methods used Require judgment-based scoring based on criteria and performance

standards May or may not be secure Involve questions typically only asked of students in school

Academic Prompts- Examples CA 4th grade released itemRead this sentence from the article.“Eliza Scidmore was there, watching her dream take root in the ground.”What does “watching her dream take root in the ground” mean? Explainyour answer, using information from the article.

MA 8th grade released itemJoe has a sales job that pays him $3,000 per month and he also

earns 10% of his monthly sales as a commission.On the line below, write an expression that can be used to find

Joe’s total earnings for a month. Let d represent his sales for the month, in dollars. expression_____________

One month Joe’s total sales were $34,000. What was the amount of Joe’s earnings for that month?_______________

Quiz and Test Items Familiar assessment formats consisting of

simple, content-focus items that Assess for factual information, concepts, and

discrete skills Are convergent, typically having a single, best

answer May be easily scored using an answer key or

machine Are typically secure (i.e., items are not known in

advance)

Informal Checks for Understanding

Ongoing assessments used as part of the instructional process. These assessments provide feedback to the teacher and students. They are not typically scored or graded. Teacher questioning Observations Examining student work Think-alouds

From Criteria to Rubric Rubric is a criterion based scoring guide consisting of

a fixed measurement scale and descriptions of the characteristics for each score point. Rubrics describe degrees of quality, proficiency, or understanding along a continuum. By what criteria should performance be judged and

discriminated? Where should we look and what should we look for to judge

performance success? How should the different levels of quality, proficiency, or

understanding be described and distinguished from one another?

Two Types of Rubrics

Analytic-divides a product or performance into distinct traits or dimensions and judges each separately

Holistic-provides an overall impression of student’s work

Rubric on Writing Good RubricsSuperioridentifies a complex task that draws on higher order thinking skills has goals that are clearly stated and attainable clearly describes attributes across a defined range of abilities judges characteristics or behaviors that are observable and measurable is age appropriate but encourages growth has wording/vocabulary/concepts appropriate to the ones who will be using it is stated in positive terms: WHAT TO DO

Goodidentifies a less complicated task that uses mid-level thinking skills has goals that are a little ambiguous, but attainable partially describes attributes across a range of abilities mostly judges characteristic that are observable describes characteristics that are age appropriate has wording that is somewhat appropriate for target audience is mostly stated in positive terms

Pooridentifies a simple task that draws on low-level thinking skills expresses goals that are unclear, unattainable, or unrealistic does not describe a range of abilities or attributes offers judgments that are merely opinions describes characteristics that are not age appropriate has wording that is inappropriate for the targeted audience is stated only in terms in negative terms: what NOT to do

Based on Nancy Osborne's Rubric on Writing Good Rubrics

Examples of Top level descriptors Development of Big or Important Ideas: The paper is clear and focused.

I hold the reader’s attention. Relevant anecdotes and details enrich the central theme

Organization: The organization enhances and showcases the central idea or theme. The order, structure, or presentation of information is compelling and moves the reader through the text.

Word Choice: The words convey the intended message in a precise, interesting, and natural way. The words are powerful and engaging.

Voice: The writer speaks directly to the reader in a way that is individual, compelling, and engaging. The writer crafts the writing with an awareness of and respect for the audience and the purpose for writing.

Word Choice: The words convey the intended message in a precise, interesting, and natural way. The words are powerful and engaging.

Designing and Refining Rubrics Based on Student Work

Gather samples of student performance that illustrate the desired understanding or proficiency

Sort student work into different “stacks” and write down the reasons

Cluster the reasons into traits or important dimensions of performance

Write a definition of each trait Select samples of student performance that illustrate

each score point on each trait Continuously refine

1. Responds fully to the assignment 

EXCELLENT VERY GOOD   ADEQUATE  WEAK  POOR

2. Presents a clear statement of topic, thesis or dominant impression

EXCELLENT VERY GOOD   ADEQUATE  WEAK  POOR

3. Exercises good critical thinking 

EXCELLENT  VERY GOOD   ADEQUATE  WEAK  POOR

4. Expresses its purpose clearly and persuasively throughout 

EXCELLENT  VERY GOOD   ADEQUATE  WEAK  POOR

5. Invokes and uses relevant facts correctly

EXCELLENT VERY GOOD  ADEQUATE  WEAK  POOR

Rubrics for Comparison

http://www.sci.edu/classes/ellertsen/rubrics/gradingrubric.html

Rubrics for ComparisonBeginning Developing Accomplished Exemplary

Use of Internet

Needs frequent assistance to access

internet sites and locate meaningful

information.

Accesses internet sites with minimal assistance, gathers

some relevant information.

Accesses internet sites easily, only

gathers some relevant

information.

Easily accesses internet sites; consistently locates and

records meaningful information.

Research Does not answer any questions suggested in

the guidelines. Only found 1-2

animals

Answers some questions.

3-4 animals used as examples.

Examples too general.

Answers some questions and includes a few

other interesting facts.

5 animals given as examples.  but

details sometimes too general.

Answers most questions and gives detailed information.

6 or more animals used for examples.

http://www.tlt.ab.ca/projects/Div3/Grade9/diversity/report-rubric2.html

Rubrics for Comparison

LEVEL 4 – All steps in the task – market survey, list of available and scarce resources, final product, advertisement, and Market Day Success Report – are created in an independent manner. All work meets the criteria for a quality product.

LEVEL 3 – All steps in the task – market survey, list of available and scarce resources, final product , advertisement, and Market Day Success Report – are completed with some help. All work meets the criteria for an adequate product.

LEVEL 2 – Most steps in the task – market survey, list of available and scarce resources, final product, advertisement, and Market Day Success Report – are completed but much help is needed. Some work shows lack of understanding.

LEVEL 1 - Few steps in the task – market survey, list of available and scarce resources, final product, advertisement, and Market Day Success Report – are completed. Those that are complete show lack of understanding.