On Common Ground: 6-TRAIT Scoring for the Kansas Writing Assessment

Presentation Objectives1. Examine how the 6+1 TRAIT model was developed

for teachers by teachers.

2. Discuss the ways the 6-TRAIT model has been applied both nationally and here in Kansas.

3. Explore how the 6-TRAIT model is used within the context of the Kansas Writing Assessment.

4. Practice scoring for the Kansas Writing Assessment.

5. Discuss issues with scorer objectivity and bias.

Activity One

The Writing Sneeze:

Write continuously for three

minutes using the following

sentence starter as a

springboard:

My fears in scoring student essays for the Kansas Writing Assessment are…

Why Worry About Writing? Approximately 70% of students in grades 4-12 are low-

achieving writers. (Persky et al., 2003)

Thirty-five percent of high school graduates in college and 38% of high school graduates in the workforce feel their writing does not meet expectations for quality. (Achieve, Inc., 2005)

About half of private employers and more than 60% of state government employers say writing skills impact promotion decisions. (National Commission on Writing, 2005)

Writing remediation costs American businesses as much as $3.1 billion annually. (National Commission on Writing, 2004)

Among the Recommendations for Improving Student Writing

1. School districts should insist that writing be taught in all subjects and at all grade levels.

2. Every teacher should be required to successfully complete a course in writing theory and practice as a condition for teacher licensing.

3. Schools should aim to double the amount of time most students spend writing.

(National Commission on Writing, 2003)

Effective Practices to Improve Student Writing in Grades 4 to 12

1. Writing Strategies2. Summarization3. Collaborative Writing4. Specific Product Goals5. Word Processing6. Sentence-Combining7. Prewriting8. Inquiry Activities9. Process Writing Approach10. Study of Models11. Writing for Content

Learning

The Writing Next report highlights 11 instructional practices that research has identified as having shown strength and consistency in improving student writing quality.

Which of these practices can be facilitated and enhanced with the 6-TRAIT model?

(Graham and Perin, 2007)

Effective Practices to Improve Student Writing in Grades 4 to 12

1. Writing Strategies2. Summarization3. Collaborative Writing4. Specific Product Goals5. Word Processing6. Sentence-Combining7. Prewriting8. Inquiry Activities9. Process Writing Approach10. Study of Models11. Writing for Content

Learning

Using the 6-TRAIT

model of assessment

and instruction can

facilitate and

complement eight of

these eleven

instructional practices.

(Graham and Perin, 2007)

The 6+1 TRAIT ModelWhat it is…• a common

vocabulary for teachers and students

• a vision of the characteristics of strong writing

• an analytical scoring method

• a system for driving and managing student learning

What it is NOT…• a packaged

curriculum• a standards

document• a ‘canned’ program• a ‘convenience’ to

ease the grading load• a way to make

teaching ‘easier’

Development of the 6+1 TRAIT Modelfor teachers, by teachers

In 1984, drawing from the pioneering work of Paul

Diederich (1974) and Donald Murray (1982) and with

the help of researchers from the NWREL, a group of

17 teachers from the Beaverton, Oregon, school

district set out to create a scoring guide (a rubric) that

would describe what ‘good’ writing looks like…

Teachers spent weeks pouring over thousands of

examples of student writing, sorting them into high,

middle, and beginning levels, and documenting their

reasons for the rankings…

Next, they began sifting through their documented

reasons—combining and condensing—until they

arrived at six essential traits:

Development of the 6+1 TRAIT Modelfor teachers, by teachers

1. Ideas & Content – the heart of the message

2. Organization – the internal structure

3. Voice – the personal tone and flavor

4. Word Choice – the vocabulary a writer chooses

5. Sentence Fluency – the rhythm and flow

6. Conventions – the mechanical correctness

+1. Presentation – form, layout, eye-appeal

The 6+1 Traits of Writing

focused and clear controlling idea or

theme provides detail and

support that enriches writer is selective shows insight writes from knowledge

and experience

Ideas & Content

thread of central meaning

crafts a clear beginning, middle, and end

effective sequencing good pacing smooth transitions information is given in

the right doses at the right times

Organization

person behind the words

heart and soul appropriate for

audience conviction text is full of life personal, individual,

expressive

Voice

Word Choice rich, colorful, precise

language

strong and natural

vocabulary

energetic verbs

precise nouns and

modifiers

skill in using everyday

words well

Sentence Fluency rhythm and flow

easy to read a out loud

poetic, musical

variety of sentence

beginnings

variety of sentence structures

cadence, power, movement

mechanical correctness consistent control of

spelling, punctuation, grammar, indenting, capitalization, etc.

minimal typographical errors to distract a reader

appearance is irrelevant

Conventions

Presentation uniform spacing

legible handwriting or

appropriate use of fonts

and sizes

appealing use of white

space

effective use of bullets,

lists, graphs, maps, tables,

illustrations, sidebars,

sub-headings as necessary

Activity TwoWhat Teachers Value in Writing:

With a partner…

Read the selection of writing and generate a list of three to five characteristics that make this a ‘good’ piece of writing…

Now, for each of the characteristics you and your partner wrote down, identify under which of the traits that characteristic would fall.

1. Ideas & Content – the heart of the message

2. Organization – the internal structure

3. Voice – the personal tone and flavor

4. Word Choice – the vocabulary a writer chooses

5. Sentence Fluency – the rhythm and flow

6. Conventions – the mechanical correctness

+1. Presentation – form, layout, eye-appeal

The 6+1 Traits of Writing

Remember those 17 teachers in Oregon that spent weeks pouring over thousands of examples of student writing and documenting their reactions and comments?

Keep in mind they were also sorting that writing into three categories: high, middle, and beginning levels.

The result of their work was a draft of a rubric that would become the foundation for the 6+1 TRAIT model…

It looked a little something like this…

Development of the 6+1 TRAIT Modelfor teachers, by teachers

The 6+1 TRAIT Continuum

5 = Strong• shows skill and control; many

strengths present; writing is publishable

4 = Effective• strengths outweigh weaknesses; one

draft away from being publishable

3 = Developing• balance of strengths and weaknesses

2 = Emerging• weaknesses outweigh strengths

1 = Beginning• writer not yet showing any control

4

5

3

2

1

The 6+1 TRAIT Continuumpoints worth noting

• A ‘5’ represents a strong paper, not necessarily a perfect one.

• There may be a considerable difference in the “distance” between a ‘1’ and a ‘2’ and the distance between a ‘4’ and a ‘5.’

• The goal is for each student to improve with each subsequent draft.

4

5

3

2

1

Since 1984, both the rubric and the 6+1 TRAIT model have been revised nearly 20 times and been modified by a host of schools, districts, states, and even foreign countries.

In fact, Presentation (the seventh trait) wasn’t added to the model until 1997—based on the recommendations of classroom teachers—to help separate issues of correctness (conventions) from issues of appearance.

Development of the 6+1 TRAIT Modelfor teachers, by teachers

Flexibility and Adaptability

All this to prove a fairly simple point…

The 6-TRAIT Writing Model is just that—a model.

It is flexible and can be adapted to suit specific

assignments, specific purposes, and—most

importantly—the needs of particular students.

Flexibility and Adaptability Even a simple

Google search will reveal…• 4-point rubrics• 5-point rubrics• 6-point rubrics• Rubrics for teachers• Rubrics for students• Rubrics for assessment• Rubrics for instruction

The traits, however, remain constant:

1. Ideas and Content

2. Organization

3. Voice

4. Word Choice

5. Sentence Fluency

6. Conventions

7. Presentation

What We Value in Student Writing“We believe that any scoring model is by design a

value system. In that spirit, we hope to share this

model in a way that enables users to clarify and

define their own values about writing, and to learn a

method of sharing those values with students and

with other teachers, so that all of us who work with

the assessment of writing can understand writing

better and teach it more effectively.”

– from NWREL’s first publication of the 6-TRAIT model (1986)

Flexibility and Adaptability

Adapt the 6-Trait Model to suit your needs, the needs of your classroom, and the needs of your students.

Unlike this teacher…

The 6-TRAIT Model in Kansas

First used in 1989 in the KCK district

1992 — the first pilot of the Kansas Writing

Assessment which is built upon the 6-TRAIT model

(about one third of Kansas school districts

participate)

1994 to present — the Kansas Writing Assessment

continues biennially, requiring 6-TRAIT scoring at

both the local and state level

The 6-TRAIT Model in Kansas In 2003, 6-TRAIT rubrics were adapted to create the

Kansas Modified 6-TRAIT Rubric.

In 2007 work began on updating the 6-TRAIT rubrics

for the narrative, expository, and persuasive writing

on the Kansas Writing Assessment.

Changes to format and layout of the rubrics.

The six traits and the descriptor language,

however, remains largely the same.

Recent Changes to Rubricsfor the Kansas Writing Assessment

Changes to format:

descriptors organized under four criteria

within each trait

landscape orientation

“checkbox” bullets

Changes to content: descriptors added for the ‘2’ and ‘4’ levels point level labels of the rubric were changed to

better reflect the stages of a writing process

Rubrics are available for download on the KSDE Writing Homepage: <www.ksde.org/Default.aspx?tabid=1726>

New Point Level LabelsWhat was…

Strong

Maturing

Developing

Emerging

Beginning

Is now…

Publishing

Polishing

Drafting

Shaping

Inventing

4

5

3

2

1

The “Old” Rubric

The “New” Rubric

Two readers are always preferable to only one.

Scorers should be encouraged to discuss their ratings with other scorers.

Periodically, scorers should physically mark a copy of the rubric as they score to personally verify their ratings.

Suggested Scoring Procedures

Suggested Scoring Procedures are available on the KSDE Writing Homepage: <www.ksde.org/Default.aspx?tabid=1726>

Scoring an Individual Essay

For each trait… Determine which set of descriptors best

describe the essay for each of four criteria within a given trait.

Then, scorers should use the four criteria levels they identified to inform the rating of the overall trait.

Suggested Procedure for Scoring an Essay

Guidelines for Discussing Scores The goal is not to persuade or convince individuals

to change their scores. Keep the discussion firmly grounded in the language

of the scoring rubric. Discussion facilitates professional learning and

allows readers to better understand, clarify, and apply the criteria and descriptors contained within the scoring rubric.

The ultimate goal of is to ensure that student writing is being scored consistently and accurately.

Updated Training Manual is available on the KSDE Writing Homepage: <www.ksde.org/Default.aspx?tabid=1726>

Includes scored samples essays written by Kansas students that can be used for training/calibration purposes.

Activity Three

Practice Scoring

Using pre-scored sample essays from the

KSDE Training Manual for 6-TRAIT scoring,

let’s practice scoring student writing at the

5th grade, 8th grade, and high school levels.

How Did We Do?

Using the scoring

chart from the

training manual, let’s

check to see how we

did!

Issues with Scorer Objectivity & Bias Aspects/Characteristics of student writing NOT

considered when scoring for the Kansas Writing Assessment:

handwriting lightness or darkness of writing neatness formatting (skipping lines, unusual margins, font size/style) presence or absence of a title length of the piece of writing absence or use of technology

A number of factors can unknowingly influence a scorer’s perception of student writing in either a positive or negative direction.

An illustrative (not exhaustive) list of potential pitfalls to be mindful of…

Other Examples of Potential Scorer Bias

Personal Standards of Quality

Readers often have personal standards for what makes quality writing. Rationalizing scores with personal thoughts such as, Three misspelled words means a piece can only score a 3 or below in Conventions, or Only an essay that is completely error free can receive a top score, is a form of bias. When scoring for the Kansas Writing Assessment, the rubric must be followed.

Other Examples of Potential Scorer Bias

Other Examples of Potential Scorer Bias

Referring to the Prompt

Many writers write excellent papers but do not refer

directly in any way to the language of the prompt.

This is fine. Do not demand an obvious reference

(e.g., “My most memorable experience was the time

when…”) or demand that students include any of the

example ideas suggested in the prompt.

Other Examples of Potential Scorer Bias

Reactions to Content

Personal reactions to the specific content of the essay

may influence scoring. If the writer’s values are not

the same as the scorer’s or the writer’s choice of

overall theme or specific details are unappealing, this

may unduly affect scoring. If you cannot objectively

read such a paper, pass it on for scoring by someone

else.

Profanity

Occasionally a student will include profanity in his or

her writing. If you cannot objectively read a paper

with profanity, pass it on for scoring by someone else.

Other Examples of Potential Scorer Bias

Persona or Tone

Personal reactions to the persona or tone of the

writing can influence scoring. Scorers should take

into account any thoughts such as What a cute kid! or

How conceited! that may affect the accuracy of their

scoring.

Other Examples of Potential Scorer Bias

Style or Usage Prejudice

Some idiosyncratic preferences in style or usage (e.g.,

the use of a lot, get, or that is; the halo effect of a

well-turned phrase; the use of a particular cliché) can

create bias.

Other Examples of Potential Scorer Bias

Prior Experience with Student

Deeply ingrained personal classroom assessment

experiences with specific students can corrupt

accurate scoring. Thoughts such as, Pablo is the best

writer in the class, so his essay must receive straight

5s, or Samantha never does well with writing; I’m

sure her essay is horrible, can influence scoring.

Other Examples of Potential Scorer Bias

1. Refer often to the scoring rubric. Do not rely just on

your memory or your intuition.

2. Physically mark copies of the rubric while scoring

to ensure your judgments are being made based

upon the characteristics the rubric provides.

3. Remember to score each trait individually without

allowing the score from one trait to influence your

scoring of another trait.

Other Friendly Reminders for Scoring

4. Think of a 3 as the point on the scoring continuum

where strengths and weaknesses balance. Any score

above a 3 indicates dominant strengths; any score

below a 3 indicates dominant weaknesses.

5. Do not dwell on a particular essay’s weaknesses.

Focus your attention on identifying the set(s) of

descriptors that best describe the characteristics of

the essay.

Other Friendly Reminders for Scoring

6. Remember that you are assessing the writing—not

the writer—and only a single performance at that.

7. Keep in mind that the prompt is only meant to

motivate the writer and provide a springboard for

the student to begin generating ideas. Readers

should score the quality of the writing, not the

student’s adherence to the prompt.

Other Friendly Reminders for Scoring

8. If the essay is off-topic, score it according to the rubric to the best of your ability.

9. If you think a paper might be a crisis paper (e.g., suicide, child abuse, drug abuse), flag it and report it to your scoring session leader or testing coordinator but continue scoring.

10. If the papers are starting to blend together in your mind and all are starting to read the same, take a break.

Other Friendly Reminders for Scoring

11. When you need to share an essay or talk at length about

scoring, please do so with respect toward other scorers.

Even quiet talking can make it hard for those around you to

concentrate.

12. Your pace should be brisk but comfortable. Take time to

read each paper thoroughly; do not skim. Our priority is

scoring accuracy, not scoring speed. If you become stuck

with a particular essay, ask for help.

Other Friendly Reminders for Scoring

The 6+1 TRAIT Model in KansasKansas educators believe the 6+1 TRAIT model has:

reinforced the teaching of a writing process; helped many teachers who were unsure or

inexperienced with teaching writing to do so with confidence in their classrooms;

provided a common vocabulary to talk about writing with students and colleagues;

helped students understand what improvement a piece of writing needs;

The 6+1 TRAIT Model in KansasKansas educators believe the 6+1 TRAIT model has:

motivated students to revise and improve their writing; helped teachers to diagnose student needs; helped teachers to adjust and individualize instruction;

and provided a method for sharing information about

individual student writing ability with administrators, parents, and other non-specialists.

Other Available Resources

Glossary of Writing Terminology provides clarification on commonly

confused and/or misunderstood terms

in reference to their use on the Kansas

Writing Assessment

might be appropriate for both students

and educators

available on the KSDE Writing

Homepage: <www.ksde.org/Default.aspx?

tabid=1726>

Other Available ResourcesRubrics for Incorporating Research and Citing Sources

Grades 3-7 Grades 8-12

• Although not a part of the Kansas Writing Assessment, these

rubrics are formatted like those for the assessment and help

educators communicate our expectations for academic writing

and avoiding plagiarism.

• Available on the KSDE Writing Homepage:

<www.ksde.org/Default.aspx?tabid=1726>

Other Available Resources

• illustrative lists of the various types and genres of writing that might fit into each mode (narrative, technical, expository, persuasive)

• helps communicate that most writing often blends several modes or moves back and forth among modes

• available on the KSDE Writing Homepage: <www.ksde.org/Default.aspx?tabid=1726>

Illustration of Writing Modes Chart

Activity FourTeacher Reflection:

1. One piece of information presented today that I already knew was…

2. One piece of information presented today of which I was not aware was…

3. One idea from this presentation that I intend to use in my classroom is…

4. The most controversial idea I heard today was…

5. The idea I’d most like to hear more about is…

Communication from KSDE about Writing

• KSDE Writing Homepage—Standards, Assessment, and Resources <www.ksde.org/Default.aspx?tabid=1726>

• KSDE Writing ListServ

(email Matt to be added to this list)• Contact Matt Copeland directly

• Phone (785) 296-5060

• Email mcopeland@ksde.org

BibliographyAchieve, Inc. (2005). Rising to the Challenge: Are High School Graduates Prepared for College

and Work? Washington, DC: Author. Retrieved January 16, 2008, from http://www.achieve.org/node/548

Graham, S., & Perin, D. (2007). Writing Next: Effective Strategies to Improve Writing of Adolescents in Middle and High Schools—A Report to Carnegie Corporation of New York. Washington, DC: Alliance for Excellent Education. Retrieved January 16, 2008, from http://www.all4ed.org/publication_material/reports

National Commission on Writing (2003). The Neglected ‘R’: The Need for a Writing Revolution. Retrieved January 16, 2008, from http://www.writingcommission.org/report.html

National Commission on Writing. (2004). Writing: A Ticket to Work… or a Ticket Out: A Survey of Business Leaders. Retrieved January 16, 2008, from http://www.writingcommission.org/report.html

National Commission on Writing. (2005). Writing: A Powerful Message from State Government. Retrieved January 16, 2008, from http://www.writingcommission.org/report.html

Northwest Regional Education Laboratory. (1986). “The Six-Trait Analytic Model.” Portland, OR: Author.

Persky, H.R., Daane, M.C., & Jin, Y. (2003). The Nation’s Report Card: Writing 2002. (NCES 2003—529). U.S. Department of Education. Institute of Education Sciences. National Center for Education Statistics. Washington, DC: Government Printing Office.