Rubrics for Complex Papers/Projects

Academic Assessment Workshop

May 14-15, 2009

Bea Babbitt, Ph.D.

Complex Project

Can be used to gather information on multiple learning outcomes for the course or program.

Examples include end-of-semester paper or project, semester-long project, research study, clinical experience.

Assessment Definition

Assessment is an ongoing process aimed at understanding and improving student learning. It involves making our expectations explicit and public; setting appropriate criteria and high standards for learning quality; systematically gathering, analyzing, and interpreting evidence to determine how well performance matches those expectations and standards; and using the resulting information to document, explain, and improve performance.

~Tom Angelo, AAHE Assessment Forum

Can you answer the following questions?

What do you want students to learn? How well do you want them to learn it? What evidence of learning will you require? Do students understand your expectations? How well do individual students or groups of students

perform? What’s the relationship between instruction and various

components and levels of performance? When performance doesn’t match expectations, what

actions can be taken to improve performance? Do the actions taken result in improved performance?

10 sure signs you need a rubric

You are getting carpal tunnel syndrome from writing the same comments on almost every student paper.

You have graded all your papers and worry that the last ones were graded slightly differently from the first ones.

10 signs

You want students to complete a complex assignment that integrates all the work of the term and are not sure how to communicate all the varied expectations easily and clearly.

You give a carefully planned assignment that you never used before, and to your surprise, it takes the whole class period to explain it to students.

10 signsStudents ask many questions about the long

narrative description of the assignment in the syllabus.

Students can’t explain the assignments or expectations to the Writing Center or other tutorial services.

All or most of your class were unaware of academic expectations so basic you neglected to mention them (e.g., spell check, citations)

10 signs

Students provide many excuses why they couldn’t possibly meet your expectations; you’d like to say, “here are the standards, you’re an adult, you weigh all your responsibilities and choose how well you would like to do”.

You’ve designed a common assignment across course sections, but aren’t sure faculty are using a common grading scale.

What’s a rubric?

Scoring toolLays out specific expectations for an

assignmentDivides an assignment into its component

partsProvides a detailed description of what

constitutes acceptable or unacceptable levels of performance for each of those parts

At what level will you assess?

Level 1. Assessing individual student learning within courses

Level 2. Assessing individual student learning across courses

Level 3. Assessing coursesLevel 4. Assessing programs

UNLV Program Assessment: Includes aspects of both 3 & 4

Level 5. Assessing the institution

Why rubrics for complex papers/projects?

Defines major components of paper/project Communicates expectations Conveys feedback to students Saves grading time Insures cross-section/cross-instructor scoring

consistency (with training, of course) Facilitates aggregating scores across

students/sections to determine group performance on each skill component, or program or general education objective.

Parts of a Rubric

Task description (the assignment)Scale (e.g. Exceeds expectations, meets

expectations, below expectations)Dimensions (a breakdown of the

skills/knowledge demonstrated through the assignment)

Descriptions (describes the dimension in detail at each level of the scale) set out on a grid (matrix).

Basic Rubric Grid Format

Scale Level 1

Scale Level 2

Scale Level 3

Scale Level 4

Dimension 1

Dimension 2

Dimension 3

Dimension 4

Dimension 5

Assignment TitleTask Description:

Rubric with Scale: High to Low

Exemplary Proficient Basic Novice

Dimension 1

Dimension 2

Dimension 3

Dimension 4

Dimension 5

Short EssayTask Description (complete):

Add Dimensions

Exemplary Proficient Basic Novice

Structure

Analysis

Evidence

Add dimension

Add dimension

Short EssayTask Description (complete):

Out-to-lunch Glimmer of understanding

So-so Name this level

Structure One paragraph Three or more paragraphs, arbitrarily divided, word music rambling in a thick haze.

Multiple paragraphs adhering to some formulaic notion of an essay -- intro., body, conclusion.

Originality. Structure driven by discourse.

Analysis The lights are completely out. It's a power failure

There is mild evidence of thought. The obvious and banal are represented as great insight

Depth of understanding beyond the obvious but is weighted down with questionable logic, gross assumptions, and a lack of perspective.

Original thought, challenging assumptions, and shedding new light on old subjects.

Evidence What evidence? It's all unsupported claims.

Perfunctory, sporadic support is juxtaposed with unsubstantiated assumptions and bromides. The occasional citation is incomplete

Most claims are supported with evidence, properly cited; reliable sources are given weight equal to internet crackpot claims.

The analysis is driven by the evidence which is extensive and from a variety of reliable sources, all of them properly and thoroughly cited.

Etc.

Short EssayTask Description:

Partial Rubric (Humorous)

Write Descriptions

Examples, examples, examples

Starter Resources

Internet Resources for Higher Education Outcomes Assessment http://www2.acs.ncsu.edu/UPA/assmt/resource.htm

Miller, R., & Leskes, A. (2005). Levels of assessment: From the student to the institution. Washington, D.C.: Association of American Colleges and Universities.

UNLV Assessment Home Page http://provost.unlv.edu/Assessment

Winona State University Sample Rubrics http://www.winona.edu/air/rubrics.htm