Teachers Manual a to A

7/27/2019 Teachers Manual a to A

1/20

A Guide to Classroom

Assessment

By Kim Hartley


2/20

A Teachers Manual

A Guide to Assessment in Society and Environment

2

Contents

Introduction to Assessment 3Validity

Reliability

Bias

Essays 6

Portfolios 7

Performance Assessment 9

Selected Response 11Multiple Choice

True/Flase

Completion

Matching

Self Assessment 15

Peer Assessment 15

Creating Quality Rubrics 16

Reference List 20


3/20

A Teachers Manual


3

Purpose Use Task Agent ofJudgement

Basis ofJudgement

Score

1

The interpretations of students assessment results are valid only to the degree to which you canpoint to evidence that supports them.

2

The uses you make of your assessment results are valid only to the degree to which you can pointto evidence that supports thier correctness and appropriateness.

3

The interpretations and uses of your assessment results are valid only when the values implied bythem are appropriate.

4

The interpretations and uses you make of your assessment results are valid only when theconsequences of these interprettions and uses are consistent with appropriate values.

Introduction

Assessment is the process of collecting and interpreting evidence for some purpose. Assessment is a way inwhich teachers, educators and students can map progress in learning and development and can occur in

many different forms. Whenever possible students should be involved in the assessment process as this

promotes student engagement and chievement. Assessments can serve different purposes as can be used as

a form oflearning, as learning, orforlearning. Summative assessment is the assessment of learning. This

type of assessment is used for the purpose of reporting the achievement of individual students at a

particular time. Formative assessment is the assessment for Learning and is intended to contribute directly

to the learning process through feedback. This type of assessment is used to inform decisions about learning

experiences and to report on what has been achieved. Peer and Self assessments are used as learning

because they are used as part of the learning process.

System of Assessment; Variables. (Harlen 2007, p.15)

Assessment Validity

Validity is the soundness of your interpretations and uses of students assessment results. Validity isconcerned with the soundness, trustworthyness, or legitimacy of the claims or inferences that are made on

the basis of obtained scores (Mcmillan 2007, p.64). Validity is always determined by professional judgement,

in classrooms this judgement is made by the teacher. An analysis is done by accumulating evidence that

would suggest that an inference or use is appropriate and whether the consequences and interpretations are

reasonable and fair. Validity comes from three forms of evidence; Content-related evidence, Criterion-

related evidence and Construct-related evidence (Mcmillan 2007, p. 65).

The Four Principles of Validation (Messick, 1994)


4/20

A Teachers Manual


4

BIASTestEnvironment

Incorrect TestCriteria

Test Usage

Test Takers

Test Content

ObsrvedScore

TrueScore

Error

You can improve assessment validity by asking other teachers or professionals to judge the assessment for

clarity and by using different methods for assessing the same thing. Appropriate vocabulary, sentence

structure and item delivery is necessary to ensure that all students can access the information within the

item and adequate time should be given to students.Validity can also be improved by comparing studentsscores on similar but different tasks (Mcmillan 2007, p.69)

Assessment Reliability

Reliability is concerned with consistancy, stability and dependability of the scores (Mcmillan 2007, p.71)

Assessment reliability is the degree to which students results stay the same when; the same task is

completed on two seperate occasions, another teacher marks work and gives the same answer, same result

from a different but equivelant task (Brookhart 2007, p. 69). Statistical methods can indicate the degree of

reliability and the approximate size of the measurement errors in the assessment. Concept of error in

assessment is critical to our understanding of reliability. When we assess something, we get an observed

score or result. This observed score is a product of what the true or real ability or skills plus some degree of

error. Reliability is directly related to error. For each assessment there is some degree of error, so low,

medium or high reliability. Error is measured by (SEM) Standard Error of Measure. SEM is determined by a

formula that takes into account the reliability and standard deviation of the test. (Mcmillan 2007, p.71)

Assessment reliability can be affected by internal factors such as the student s health, motivation, if they

are tired, in a bad mood or are distracted. These can all create negative error which would underestimate

the students true score. External factors such as the test directions and the room temperature and

atmosphere also affect reliability. To ensure that assessment reliability is high it is recomended to use

shorter assessments more often.

Assessment Bias

Bias can be found in the assessment tasks, its contents, process or problem. Bias is present if the assessment

distorts performance because of the students ethnicity, gender, race or religion. There are two forms of

assessment bias; unfair penalisation and offensiveness. Offensiveness occurs if the content of the

assessment offends, upsets, distresses, angers or creates negive effects for particular students or subgroups


5/20

A Teachers Manual


5

of students. This negative affect makes it less likely that the students will perform as well as they otherwise

might, lowering the validity of the inferences. (mcmillan 78) Unfair penalisation is bias that disadvantages a

student because of content that makes it more difficult for students from some groups to perform as

compared to students from other groups (Mcmillan 2007, p. 79). Assessment bias can be minimised byhaving others review your work and by becoming aware of bias elements in assessments.


6/20

A Teachers Manual


6

Essays

Essays are a good assessment to use for evaluating a students in depth knowledge of a topic but may besubjected to sampling. To ensure that this type of assessment is valid, the teacher must structure the

question carefully to ensure that the response given by the students will demonstrte the students ability to

meet the specificed marking criteria. This type of assessment is very end heavy and requires alot of time to

mark the essays effectively. A rubic cube is used by teachers to assist in the reliballity of scores for essays.

Bias can exist in essay questions if the question is not written clearly and concisely.

There are two forms of essay questions, restricted response and extended response. Restricted response

required students to produce brief answers or short essays. Restricted response items restrict or limit both

the content of students answers and the form of their written response. This is done by the way the

restricted response task is phrased. (Brookhart 2007, p.190). Extended response requires students to write

essays in which they are free to express their own ideas and to use their own organisation of their answers.Usually no single answer is consideredcorrect. A student is free to chose the way to respons. The degree ofcorrectness or merit of a students response can be judged only by a skilled teacher who is informed on the

subject (Brookhart 2007, p.190) This type of assessment should be used to assess the students ability to

explain cause-effect relationships, ability to present arguments and formulate valid conclusions. This

assessment technique requires that students use high order thinking skills as they have to have a deep

understanding of the topic and complex information (Mcmillan 2007, p.312).

Advantages assesses deep understanding, complex thinking, and reasoning skills, provides students with

flexability in their response, evaluates students ability to communicate their reasoning, reduces guessability

and encourges students to study content widely and in depth.

Disadvantages soring and reading is highly time consuming, especially if giving meaninful feedback, there

can be issues with unreliable scoring. This type of assessment does not provide very good example of

content knowledge as relatively few questions are asked and therefore validity is also decreased (Designing

Test Questions, 2003).

Examples of Essay Questions

Arguably the novel has responded more quickly and fully to new ideas than any other literary genre. It is a

social and moral document, as well as an art form. Do you agree? Discuss with detailed reference to two

texts.

This is a good essay question because it offers an opinion and asks the student to respond in their chosen way whichprovides structure for what needs to be said in the response but freedom in the way the response is given and

perspective the student takes. This question requires the student to use high order thinking skills to answer it and is

directly related to learning targets.

The complex movement from pleasure to understanding in the reading of fiction. What does this mean?

This is another good example of an essay question. The student is required to explain the given statement and in doing

so they will be employing high order thinking skills and a deep understanding of literature and its impact on readers.

The topic is clearly defined and it is clear what is required of the student.

Explain the importance of religion.

This essay question is too broad and is open to interpretation. It is unclear what the learning objectives would be with

this question unless it is to assess a general superficial understanding of religion. It is unclear what sort of response is

required from the student.


7/20

A Teachers Manual


7

Purpose ofPortfolio

what learningtargets andcurriculum goalswill the portfolioserve

will the portfoliobe summative,formative or both

Organisation

what types ofentries should beincluded

what number ofentries will berequired

what is thetimeframe for eachentry

Use in Practice

when will thestudents work ontheir portfolios

how will theportfolio mark beweighed

how will theportfolio fit intoclassroom routine

Evaluation ofPortfolio

is it necessary tohave an overallportfolio score

How often will theportfolio need tobe scored

will evaluations ofevery entry counttowards theportfolios grade

Portfolio

A portfoliio is a limited collection of a students work used either to present the students best work or todemonstrate the students educational growth over time. The works put into the portfolio are limited to

those that best serve the portfolios purpose (Brookhart 2007, p.249) Portfolios may be used to evaluate

students abilities and improvement. Types of portfolios include best work, showcase and growth. Best works

portfolios are summative assessments where as growth and learning portfolios are formative assessments.

Portfolios can include; Videos, tests, group work, reports, drafts, photos and comments. Concerns are

reliability due to sampling and rater consistancy and validity as this is a potential source of test bias.

Brookheart, p291-292

Advantages allow reflection and growth, create collabortive climate amongst students, provides

oppertunity to assume responsibility for own learning, contributes to self awareness and self esteem,

provides meaningful picture of student growth, allows for the integration of instruction and assessment,

provides concrete and tangible evidence for communication.

Disadvantages increased workload for the teacher, decreases instructional time, interfering with teaching

and learning, controversy over reliability and validity of data collected as well as standardised portfolio

content, requires alot of teacher knowledge about data gathering and interpreting data.


8/20

A Teachers Manual


8

Example Rubric Whole Portfolio (Grades 3-12)

Strong Developing Not there yet Not

Attempted

Change over time The student selects materialthat clearly demonstrates

growth in one or more

specific areas.

The samples show evidence

of some growth, but the

growth is limited.

The samples in the

portfolio do not show

evidence of noticeable

student growth or

change over time. Either

noticeable growth has

not occurred, or the

student has not selected

the samples of work that

would illustrate that

growth clearly.

Diversity The portfolio clearlydemonstrates that the

student has tried a variety of

tasks/projects/assignments/

challenges. There is great

variety in the kinds of work

presented or the

outcomes/skills

demonstrated.

The portfolio reflects somediversity. Tasks are not all

parallel and do not all

demonstrate identical

outcomes.

The portfolio reflectsminimal diversity. All

tasks represented are

more or less alike, and

demonstrate the same

outcomes/skills.

Evidence of

thinking

The work in the portfolio

provides evidence that the

student has identified,

analyzed, planned strategies,

and worked through the

solution to a problem or

question.

The work in the portfolio

shows some evidence of

thinking, reasoning, analyzing

or problem solving, but the

student may not have worked

all the way through a

solution, or may have missed

opportunities to pull together

interesting conclusions or plot

alternate strategies. Still, the

students work overall shows

signs of planning and

purposeful effort.

The student has not

included in the portfolio

any work that clearly

demonstrates purposeful

planning of strategies,

problem solving, and

analysis of a situation,

reasoning out a

conclusion, or

considering alternative

solutions.

Self-reflection Several (or more) examplesof self-reflection show

thoughtful consideration of

personal strengths and needsbased on in-depth

understanding of criteria.

Self-reflections included

within the portfolio provide at

least a superficial analysis of

strengths and needs, whichmay or may not be tied to

specific criteria for judging

performance or growth.

Either no self-reflection is

included within the

portfolio, or the self-

reflection is rudimentary.

Structure and

Organisation

The student has formatted

and arranged the portfolio in

a way that invites the reader

inside. Items within the

portfolio are clearly labeled

and dated; the sequence is

purposeful.

The portfolio is arranged and

formatted in a way that

enables the reader to make

sense of it with a little work.

Most items within the

portfolio are labeled, dated or

both.

Arrangement and

formatting of the

portfolio make it difficult

for the reviewer to

determine when and

under what

circumstances it was

assembled. Few items (if

any) are clearly labeled or

dated.

Arter, J., and Chappuis, J., Creating and Recognising Quality Rubrics, Educational Testing Services, New Jersey, 2006


9/20

A Teachers Manual


9

Performance Assessment

Assessment based on authentic tasks such as activities, exercises or problems that require students to showwhat they can do. They have more than one acceptable solution. They call for students to create a response

to a problem and then explain or defend it. Uses higher order thinking skills such as cause and effect analysis,

experimentation, problem solving. To ensure high reliability ensure that the tasks and responses tap into a

kind of processing required. Performance assessment targets deep understanding of a topic as well as a

students resoning, presentation and communication skills. It is authentic and engaging with no singular

answer correct. But has low inter-rater reliability, can be prone to sampling and is time consuming.

Rubric for Performance Assessment is very difficult to complete. One suggestion is to identify the overall

performance or task to be assessed and perform it yourself, list the important aspects of the performance or

task, limit the number of performancne criteria, express performance criteria in terms of observable pupil

behaviours or characteristics, dont use amibigius words, arrange criteria in order they are likely to beobserved.

Two examples of performance assessment are oral presentations and role play. Bias in oral presentations can

be present for several reasons. One of the most obvious reasons would be the test takers ability to remain

calm and confident during the presentation and to maintain control over their nerves. Reliabiity for role

plays are low as the teachers impression or interpretation of the presentation will be reflected in the

marking. Performance assessment is difficult to construct and grade and time consuming to give and take

but provides a way to measure skills and abilities which would otherwise be impossible to assess (Designing

Test Questions, 2003).

Example Rubric

Oral Presentation (Grades 4-12)

Strong Middle Weak Not Attempted

Content

Ideas

Information

accuracy

Audience

understanding

Ideas are focusedand supported withrelevant details andexamples. Contentis relevant for thetask.

Information isaccurate.

The speaker haschosen the mostsignificant informationand stays with thetopic.

Support is attempted,but doesnt go farenough.

Ideas are reasonablyclear, but there aresome problems withaccuracy. The

speaker generallystays on the topic, butdoesnt develop aclear theme.

The listener is left withquestions. Thereseems to be someholes in theinformation.

There is littlecontrolling idea, thespeaker is still insearch of a topic.

Information is limited,unclear, or incorrect.

The presentation maybe repetitious orsound like a collectionof disconnectedthoughts.

Not Attempted

Organisation

Sequence of events

The speaker helpsthe listenerunderstand thesequence of ideas

The sequence andrelationships arefairly easy to follow,but sometimes you

Ideas that go togetherare not put together.

Not Attempted


10/20

A Teachers Manual


10

Easy to follow ideas

Opening and

closing

throughorganizational aids.

Listeners can put the

ideas in an outline.

The opening drawsthe listener in; theclosing leaves asense of closure andresolution.

have to makeassumptions toconnect the ideas.

An outline of the

ideas requiresinferences.

The presentation hasa recognizableopening and closing,but there is littlesense of anticipationor closure.

Listeners would have

trouble putting theideas into an outline.

There is no openingor closing. Detailsseem to fit wheretheyre placed.Sequencing is

confusing.

Delivery

Volume

Visual Aids

Pronounciation

Disfluencies

Pacing

Volume is loudenough to be heardand understood.

Visual aids are usedeffectively to supportand enhance

meaning.

Pronunciation andenunciation are clearenough to beunderstood and areused to emphasizeimportant points.

The speaker exhibitsvery few disfluencies,

such as ah, um,and youknow.

Pacing is right for theaudience. Thespeaker knows whento slow down andwhen to speed up.

The speaker can beheard and volumedoesnt distract thelistener.

Visual aids, whileunderstandable, dontadd much to the

presentation.

Pronunciation and/orenunciationare generally clearenough to beunderstood.

While the speakerexhibits disfluencies,they dont detractfrom the presentationenough to interferewith meaning.

Pacing is fairly good,but at times thespeaker goes too fastor too slow for thelisteners to keep up.

The speaker cant beheard and/orchanges in volumedistract the listener.

Visual aids areconfusing, do notrelate to the point

being made, ordistract the listener.

Pronunciation and/orenunciationdetract from beingable to understandthe speaker.

Disfluencies, such asum, ah, and you

know, detract fromunderstanding whatis being said.

Pacing is awkward.

Not Attempted

Language Use

Language and

Volcabulary

Communication

Words and phrasesare accurate, to thepoint, create picturesin the listenershead, and/or result inemphasizing theintended points.

The speakerconsciously useslanguagetechniques such asvivid language,emotional language,

humor imagery,metaphor, and simile.

The speaker usesbland language that,while not detractingfrom the message,does little to enhanceit.

Words and grammarare accurate andcommunicate, butdont capture thelisteners attention.

Grammar andvocabulary detractfrom being able tounderstand thespeakers message.

Word and phraseseither sound like athesaurus on theloose or are sonondescript, such asthing and stuff that

the listener loosesattention.

Not Attempted



11/20

A Teachers Manual


11

Selected Response

These type of assessments are commonly known as closed assessments, meaning that there is a correctanswer for each question which is not open to interpretation. The advantages of this type of assessment is

that it can test large groups in a relatively short period of time and that the test has a high reliability. It is also

very quick to mark and in most cases can be marked by a computerised system.

Multiple Choice

Multiple Choice Tests are more complecated than they may seem. The first part of a Multiple Choice test is

the item stem, or question, then there are the response alternatives which can be used to complete the stem

or answer the question it proposes. Of these response alternatives there will be one correct answer and two

or more incorrect answers, known as distractors. Multiple choice questions are able to test high order

thinking skills only if they are constructed well. in order to answer such a question the studnets would

require more time to select their answer.

Advantages versitile, reliable, efficient and accurate, objective, wide sampling of content. Can test on a

large scale with reliable and unbias marking, even computerised marking.

Disadvantages time consuming and difficult to construct, may favour simple recall, depends heavily on

students reading ability. Once a multiple choice question has been used it should not be used in its entirety

again, this adds to the time consuming task of writing questions.

Example

1. The fundamental differences between men and women are what?a. Their ability to do two things at once. unrealistic options improve guessability

b. Physical and psychological differences. make sure correct answers position is randomc. The ability to drive properly unrealistic options improve guessabilityd. All of the above this option only serves to be a filler

2. The following foods are not considered to be healthy to eat on a daily basis.a. Breads and cereals

b. Butter, ice-cream, jellybeans, biscuits, chocolate, lollies, custard, chicken and cheese

sausages and cakes. make all choics of equal length the reduce guessabilityc. Bananas and apples avoid easily guessable items and encourage Best Choice responses

3. William Shakespear is a famous playwrite and many of his plays are still studies in schools today.What play did William Shakespeare write? Eliminate irrelevant information in the stem

a. Follow Thy Fair Sunb. Oh Mistress Minec. Redemptiond. Easter Wingse. Queen and Huntressf. Sonnet of Black Beauty limit responses to 3-4

4. In what situation should you phone 000 and what information do you need to provide theopperator? Only use one problem per stem

a. Only in an emergency and any information they ask you to provideb. Only in an emergency and your name, your location and details of the emergency


12/20

A Teachers Manual


12

c. Only if you need an ambulance and any information they ask you to provided. All of the above avoid using all of the above

True/False

The stem, or question, makes a statement and the test taker is required to indicated if the statement is true

or false. True/False items have an increased guessability as there are only two options available to the test

taker. They are quick to complete and are frequently used to test low level skills. This types of assessment

works best when you wish to evaluate knowledge level content, evaluate students understanding of popular

misconceptions and when testing concepts which have two logical responses (Designing Test Questions,

2003).

Advantages easy to write and develop, quick to score, easily administed to large numbers of students, less

discriminating, can be used as a written or oral assessment, limits bias due to poor writing or reading skills,

can sample large amount of knowledge in a short amount of time, objective nature limits bias in scoring.

Disadvantages - may overestimate learning due to guessability,often leads to test trivial facts, encourages

rote memorization, a large number of items are needed for reliability, it is difficult to discriminate between

students who know the content material and those who dont.

Example

1. Atoms of an element always have same atomic mass?a. True

b. Flase2. According to modern research, all atoms of an element are identical.

a. Trueb. False avoid absolute language

3. According to the Education Department there is a gap between the achivements made byAbroiginal students and non-Aboriginal students and to remedy this the government is

implementing programs to help close this gap and these programs are having positive results.Although not all Aboriginal students are benefiting from these initiatives it is generally thought

that within twenty years all studnets whether Aboriginal or non-Aboriginal will be achieving atsimilar levels.

a. True do not copy questions straight from the text bookb. False avoid long complicated sentences

4. Most of the illegal immirgrants who come to Australia are poor and unskilled labourers who willrequire welfare benefits in order to live in Australia.

a. Trueb. False use popular misconceptions to test studnets knowledge

5. It is not raining outside and there are puddles on the ground.a. True

b. False only use one central idea for each question

6. Fruits are sweet to eat.


13/20

A Teachers Manual


13

a. Trueb. False ask questions that are obviously true or false

Completion Items

A partial statement is presented to the student and the student must select the correct response to

complete the statement.

Advantages provides a wide sampling of content, efficiently measures lower levels of cognative ability,

minimize guessing, provides objective measure to student achievement,

Disadvantages are difficult to construct, difficult to measure more than recall of basic information, mayinclude irrelevant clues to answer, time consuming to score compared to T/F and MCQ, more than one

answer may be correct.

Example

1. The local council provides _____________________ for local residence to ______________on a __________________ basis. do not omit too many key words or the meaning will be lost

2. The house was painted blue and red with green ______________lining the driveway. Thesetrees __________________shade for the ________________who lived in the

_______________. only omit meaningful words and avoid clues

Matching Items

Matching items effectively and efficiently measures the extent to which students know related facts,

assosiations and relationships. Assosiations include terms with definitions, persons with descriptions, dates

with events and symbols with names. By using this type of assessment the teacher can obtain a good sample

of a large amount of knowledge and if the question and answer is constructed properly, can assess at the

knowledge and comprehension levels. This types of assessment is easily and objectively scored. Constructing

good matching items is more difficult then creating completion or short-answer items, but not as difficult as

preparing multiple choice items. Poor matching items are constructed where there is insufficient material to

include in the item and irrelevant information is added (McMillan 2007, p.172).

Advantages spacesaving and objective way of assessing a number of important learning targets, such as

students ability to identify relationships and assosiations between two things.

Disadvantages students can use rote memorisation to learn the elements in two lists limited to the

assessmetn of memorised factual information, does not assess higher order thinking, time consuming for

students.

Example

Match the author to the title of their novel.

1. F. Scott Fitzgerald a. The Catcher in the Rye


14/20

A Teachers Manual


14

2. Vladimir Nabokov b. The Grapes of Wrath3. Mary Durack c. Lolita4. John Steinbeck d. The Great Gatsby

5. D.J. Salinger e. Lady Chatterleys Lover6. D.H. Lawrence f. Kings in Grass Castles

Use more responses than premesis to avoid perfect matching

Put reponse into a logical order, e.g. alphabetical order or chronological

Place premesis and responses on the same page to avoid confusion and distractions

Match these significant events to the date when they occured.

1. Outbreak of WW1 a. 14972. Titanic sank b. 18843. Federation of Australia c.19014. America was discovered d. 19025. End of World War 2 e. 19146. America declared war on Iraq f. 19127. Twin Towers terrorism attack g. 19298. Berlin wall was torn down h.19459. First man landed on the moon i. 196110. Electricity was invented j. 196811.Women were given the vote in Australalia k. 1969

12. Osama Bin Laden was killed l. 197813. Martin Luther King Jr. was assasinated m. 199614. First colour TV broadcast in Australia n. 200115. First test tube baby born o. 200316. Dolly the sheep is cloned p. 2011

Limit items to 15 or less, ideal number would be between 4-8

Usse homogenous material for each exercise, this example test knowledge that spans several content areas.

To test more than simple recall or basic knoledge

Match the beginning of each sentence to its other half.

1. All the names of the days of the week... a. year and four seasons2. There are twelve months in a b. and in ay3. English is the lanuage people speak in c. long trunk whihc it uses like a hand4. An elephant has a d. piglet and these piglets are pink5. A baby pig is called a e. Australia, England and America, as

well as many other countries, but is the.

national language for these countries.

Other languages are spoken in these

countries also.

Responses short and logical

Avoid grammatical clues to correct answers. (do not use incomplete sentences as premesis)


15/20

A Teachers Manual


15

Self Assessment

Students self assess naturally, teacher assessment is not sufficiently transparent, to deepen studentslearning experience, let the students into the assessment culture, help students towards autonomous

learners, students gain more feedback. Students assess their own performance and can reduce the

assessment workload of academic staff. The process of assessment is itself an inherently valuable learning

experience.

Advantages encourages studnets to become reflective and self critical learners, enables students to take

greater responsibility for, and become actively involved in the assessment for learning process, enhanse the

quality and depth of the learning that takes place, facilitates self pacing enabling studnets to develop target

setting and time-management skills., provides students with instant feedback, students are aware of what is

expected of them, students can learn from their mistakes, can complement other modes of assessment.

Disadvantages students lack experience, students may look up questions before trying to answer them,

hard to create good and valid Self Assessment Questions, totally dependant on the studnets willingness to

partiicipate, is not favoured for summative assessments.

Peer Assessment

Peer assessment involves students assessing the performance of other students.

Advantages involves students in the assessment process, provides students with greater sense of

ownership, students more aware of objectives and learning outcomes, enhanses the quality of the learning

process, creates a benchmark which students can use to reflect the standard of their own work, facilitates a

sense of partnership in learning within the classroom, suits various styles such as formative, summative,

informal or formal.

Disadvantages student opinion may be over subjective, lack of experience, requires careful planning,

implementation and monitoring, students need to be trained to assess properly, some students may not wnt

the responsibility of marking other students work, if not monitored sufficeintly then fairness and

effectiveness will me impacted, may cause undue stress and anxiety for students.


16/20

A Teachers Manual


16

Creating Quality Rubrics

There are two types of rubrics; holistic rubrics and analytical rubrics. Holistic rubrics requires the teacher toscore the produce or process as a whole without using individual criteria. Analytical rubrics require the

teacher to appoint a mark to each seperate criteria and to then add up the individual marks to obtain a

whole score.

Example Rubric A Rubric for designing quality rubrics

1 Weak 3 Medium 5 Strong

Covers the right

content

You cant tell what learning

target(s) the rubric isintended to assess, or you can

guess at the learning targets,

but they dont seem important.

The rubric doesnt seem to

align with the content

standards/learning targets it is

intended to assess.

You can think of many

important dimensions of a

quality performance or product

that are not in the rubric, or

content focuses on irrelevant

features.

Much of the content represents the

best thinking in the field, but thereare a few places that are

questionable.

Some features dont align well with

the content standards/learning

targets it is intended to assess.

Much of the content is relevant,

but you can easily think of some

important things that have been

left out or that have been given

short shrift, or it contains an

irrelevant criterion or descriptor

that might lead to an incorrect

conclusion about the quality of

student performance.

The content of the rubric

represents the best thinking inthe field about what it means to

perform well on the skill or product

under consideration.

The content of the rubric aligns

directly with the content

standards/ learning targets it is

intended to assess.

The content has the ring of

truthyour experience

as a teacher confirms that the

content is truly what you

do look for when you evaluate the

quality of a student performance or

product. In fact, the rubric is

insightful; it helps you organize

your own thinking about what it

means to perform well.

Criteria are well

Organised

The rubric is holistic when an

analytic one is better suited

to the intended use or learning

targets to be assessed;

or the rubric is an endless list

of everything; there is noorganization.

The rubric seems mixed up

descriptors that go together

dont seem to be placed

together.

The rubric is out of balance

features of more importance

are emphasized the same as

features of less importance.

Descriptors of quality work are

represented redundantly in

more than one criterion to the

extent that the criteria are

really not covering different

The number of criteria needs to be

adjusted a little: either a single

criterion should be made into two

criteria, or two criteria should be

combined.

Some details that are used to

describe a criterion are in the

wrong criterion, but most are

placed correctly.

The emphasis on some criteria or

descriptors is either too small or

too great; others are all right.

Although there are instances when

the same feature is included in

more than one criterion, the

criteria structure holds up pretty

well.

The rubric is divided into easily

understandable criteria as needed.

The number of criteria reflects the

complexity of the learning target. If

a holistic rubric is used, its because

a single criterion adequatelydescribes performance.

The details that are used to

describe a criterion go together;

you can see how they are facets of

the same criterion.

The relative emphasis on various

features of performance is right

things that are more important are

stressed more; things that are less

important are stressed less.

The criteria are independent. Each

important feature that contributes

to quality work appears in only one

place in the rubric.


17/20

A Teachers Manual


17

things.

Number of LevelsFits Targets and

Uses

The number of levels is not

appropriate for the learning

target being assessed or

intended use. There are so

many levels it is impossible to

reliably distinguish between

them, or too few to make

important distinctions.

Teachers might find it useful to

create more levels to make finer

distinctions in student progress, or

to merge levels to suit the rubrics

intended use. The number of levels

could be adjusted easily.

The number of levels of quality

used in the rating scale makes

sense. There are enough levels to

be able to show student progress,

but not so many levels that it is

impossible to distinguish among

them.

Levels DefinedWell

No levels are defined.

Wording of the levels, if

present, is vague or confusing.

It is unlikely that independent

raters could consistently rate

work the same, even with

practice.

Rating is almost totally based

on counting the number

or frequency of something,

even though quality is more

important than quantity.

Wording tends to be evaluative

rather than descriptive of the

work.

Only the top level is defined. The

other levels are not defined.

There is some attempt to define

terms and include descriptors, but

some key ideas are fuzzy in

meaning.

You have a question whether

independent raters, even with

practice, could assign the same

rating most of the time.

There is some descriptive detail in

the form of words, adjectives, and

descriptive phrases, but counting

the frequency of something or

vague quantitative words are

also present. Wording is

descriptive, not evaluative.

Each score point (level) is defined

with indicators and/or descriptors.

There is enough descriptive detail

in the form of concrete indicators,

adjectives, and descriptive phrases

that allow you to match a studentperformance to the right score.

Two independent users, with

training and practice, assign the

same rating most of the time.

If counting the number or

frequency of something is

included as an indicator, changes in

such counts really are indicators of

changes in quality.

Levels Parallel Levels are not parallel incontent and there is no

explanation of why, or the

explanation doesnt make

sense.

The levels are mostly parallel in

content, but there are

some places where there is an

indicator at one level that

is not present at the other levels.

The levels of the rubric are parallel

in contentif an indicator of

quality is discussed in one level, it is

discussed in all levels. If the levels

are not parallel, there is a good

explanation why.



18/20


19/20

A Teachers Manual


19

Example Rubric Essay Scoring Criteria, Secondary Social Studies (0= no response, 5= highest

understanding)

0

No response

1

No facts

2

Not there yet

3

Understanding

4

High

Understanding

5

Excellent

UnderstandingPrior

knowledge,

facts and

events

No facts/events

mentioned that

are not found in

the text of the

debates

No facts One to two

pieces of

information that

are not found in

the text of the

debates

Three to four

pieces of

information that

are not found in

the text of the

debates

Five to six pieces of

information that

are not found in the

text of the debates

Seven or more

pieces of

information that

are not found in

the text of the

debates.

Number of

principles orconcepts

No response No principles

/concepts

One principle

/concept

Two principles

/concept

Three principles

/concept

Four or more

principles/concept

Porportion of

text detail

No information

from text

No information

from text

Material from

the text accounts

for about 14%

of the essay

Material from the

text accounts for

about 12% of the

essay

Material from the

text accounts for

about 34% of the

essay

The essay uses or

is based on

material from the

text only

Argumentation No response Argument is

hard to

decipher

Very weak

argument

Argument present Supported

argument

Strong and

convincing

argument

Misconceptions No response One or more

serious

misconceptions

central to the

essay

At least one

serious

misconception

Several minor

errors and/or a

moderate

misconception

Very minor

misconception

No

misconceptions


In this rubric there are many errors.

1. The student could have four principles of poor quality and get a better mark than someonewho has three really good principles. This is the same for the information presented by the

student.

2. There are too many levels present.3. There is an uneven amount of space between each level, such as argument is hard to decifer

and very weak argument.4. The same criteria is used for a mark of 0 and a mark of 1 for the criteria Portion of text detail,Prior knowledge, facts and events and Number of principles or concepts.


20/20

A Teachers Manual


20

References

Arter, J., and Chappuis, J., Creating and Recognising Quality Rubrics, Educational Testing Services, New

Jersey, 2006

Broadfoot, P.,An Introduction to Assessment, Continuum International Publishing Group, 2007.

Brookhart, s., and Nikito, A., Educational Assessment of Students, Pearson Education, New Jersey, 2007.

Designing Test Questions, Grayson Walker Teaching Resource Centre, 2003; Accessed 21/05/2011,

http://www.utc.edu/Administration/WalkerTeachingResourceCenter/FacultyDevelopment/Assessment/asse

ssment.html

Harlen, W.,Assessment of Learning, Sage Publications, London, 2007.

McMillan, J., Classroom Assessment; Principles and Practice for Effective Standards-based Instruction,

Pearson Education, 2007.
http://www.utc.edu/Administration/WalkerTeachingResourceCenter/FacultyDevelopment/Assessment/assessment.htmlhttp://www.utc.edu/Administration/WalkerTeachingResourceCenter/FacultyDevelopment/Assessment/assessment.htmlhttp://www.utc.edu/Administration/WalkerTeachingResourceCenter/FacultyDevelopment/Assessment/assessment.htmlhttp://www.utc.edu/Administration/WalkerTeachingResourceCenter/FacultyDevelopment/Assessment/assessment.htmlhttp://www.utc.edu/Administration/WalkerTeachingResourceCenter/FacultyDevelopment/Assessment/assessment.html

Teachers Manual a to A

Documents

Transcript of Teachers Manual a to A