1

Beyond Self and Peer AssessmentDavid Boud

2

Outline

Part 1. What do we know about self and peer assessment?

Part 2. How can we think differently about assessment: developing informed judgement?

Part 3. How can we involve students as active agents in making judgements?

3

What is self assessment?

the involvement of students in identifying standards and/or criteria to apply to their work and making judgements about the extent to which they have met these criteria and standards.

4

What does this imply?

Students may not be the only judges Self assessment may be facilitated or left to chance There may be different understandings of ‘involvement’ and

‘making judgments of learning” There may be different practices involved Self assessment doesn’t necessarily involve student

involvement in summative assessment Self assessment is not done in isolation from others or from

standards

5

What do we know about self assessment?

Self assessment is a necessary skill for learning within the course after the course

Learners can be realistic in making self assessments inexperienced learners and those new to an area tend tp overrate

themselves advanced learners are more realistic and can tend to underrate

themselves Context strongly influences ratings

incentives for students to overrate tend to work Practice in making judgements improves self assessment

One-off uses of self assessment don’t have much impact

6

What else do we know?

Self assessment is not a method or technique, it simply represents who is the main agent in making judgements

Self assessment is best if not used in isolation from students considering other input eg. of peers, of teachers, etc

Self assessment practices are extraordinarily varied and must be designed to fit the circumstances of the subject of the stage of development of the learner

7

Five qualitatively different ways of experiencing student self-assessment

Category A: as ensuring students’ behavioural compliance Category B: as allowance for students’ contingent

judgments of their knowledge Category C: as providing feedback on students’ judgments

of requisite standards in the program of study Category D: as developing students’ judgments of their

proficiency in the program of study Category E: as sustaining students’ ability to self-assess

beyond the program of study(Tan 2006)

8

Self assessment for formative and summative purposes

Self assessment for formative purposes can be fostered by learning tasks

In limited circumstances student marks can be used for summative purposes

9

Self/peer assessment may be used for grading when:

there is a high trust, high integrity learning environment students are rewarded for high integrity marking marks are moderated by staff so that deviations need to be

justified blind peer marking is used as a check random staff marking is used as a check students have had ample opportunity to practice and develop

their skills criteria have been sufficiently unambiguously defined to

minimise misinterpretation of grade boundaries effort is explicitly excluded as a criterion

10

Conditions in which self marking may be justified

when students are new to the knowledge domain and cannot yet recognise good work

when it is a preliminary stage to self assessment proper when it is used in association with distancing devices to

help students look afresh at their work when rating scales used do not have connotations of what

is socially desirable when the sub-components of the task, not global marking

are emphasised when all scales and points on scales are explicit

11

Key features: self assessment

Active involvement in process, not following a recipe

Students involved in determining criteria, not just self-marking

Link to learning outcomes May involve peers at some stage Emphasis on informing judgment

12

Is self assessment flawed?

Recent medical education literature points to the limitations of self assessment. How should we regard what they show?

Based upon meta analyses that typically demonstrate moderate correlations between self-judgement and those of teachers (~0.3-0.4)

Many of the empirical studies on which they are based are limited and were not designed to calibrate self assessment.

Use multiple sources of feedback to calibrate Self assessment alone has, of course, considerable limitations for

summative assessment purposes. However, there is no choice but to persist with improving self

assessment if learning is about developing judgement.

13

Peer assessment research 1

‘Peer assessment seems adequately reliable and valid in a wide variety of applications, although virtually all of the current literature considers reliability of marks or grades rather than more detailed, formative assessment. Levels of acceptability to students are varied and do not seem to be a function of actual reliability. Students find peer assessment through tests, marks, or grades demanding but anxiety reducing. Learning gains in terms of test performance, skill performance, or subjective measures are frequently reported.’ (p. 268)

Topping, K. (1998) Peer assessment between students in colleges and universities, Review of Educational Research, 68, 3, 249-276.

14

Peer assessment research 2

‘Peer assessment and feedback of a more detailed, open-ended nature have been associated with improved confidence and better presentation and appraisal skills. The relatively high number and quality of studies of peer assessment of writing suggest outcomes at least as good as teacher assessment, and sometimes better. peer assessment of group and project work has been positive in terms of student perceptions. Similarly, peer assessment of professional skills shows adequate reliability but limited outcome data, often in participant perceptions. However, these again show outcomes at least equivalent to teacher assessment.’ (p. 268)

Topping, K. (1998) Peer assessment between students in colleges and universities, Review of Educational Research, 68, 3, 249-276.

15

Using peer assessment

Use formatively in conjunction with self assessment

Peer feedback without summative elements can be used very widely

Peer assessment with a summative flavour must be used carefully otherwise it can inhibit the very learning it seeks to promote

16

Key features: peer assessment

As before for self assessment Focus on peer assessment when communicating

ideas to others is important Use guidelines for giving and receiving feedback Focus on qualitative peer feedback, downplay or

eliminate ratings and grading

17

Giving and receiving feedback

Offering feedback

Be realistic Be specific Be sensitive to the person’s goals Be timely Be descriptive Be consciously non-judgmental Don’t compare Be diligent Be direct Be positive Be aware

Receiving feedback

Be explicit Be attentive Be aware Be silent

18

3. Returning to assessment generally

A. What to consider about assessment in higher education

B. Contrasting models of educational assessment

C. Thinking about developing students’ judgement

D. Implications of viewing assessment as about informing judgement

19

A. What to consider in assessment in higher education

Assessment is about judgement. Currently: judging learning outcomes against standards

Assessment must contribute to learning for learning now for future learning

Assessment is about both informing students’ judgements as well as making judgements on them Summative assessment alone is to risky

Students must necessarily be involved in assessment Assessment is a key influence in their formation and they are active

subjects.

20

B. Contrasting models of educational assessment

Scientific measurement model Practice derived from theory

Knowledge is a ‘given’ for practical purposes

Knowledge is ‘impersonal’ and context free

Discipline-driven Deals with structured problems

Judgemental model Practice and theory (loosely)

symbiotic Knowledge is understood as

provisional Knowledge is a human construct

and reflects context Problem-driven Deals with unstructured

problems(Hager and Butler 1996)

21

C. Thinking about developing judgement

Students must develop the capacity to make judgments about their own learning Otherwise they cannot be effective learners now or in the future

We can never provide them with as much or as detailed feedback as students need. Some kinds of feedback inhibit judgment through fostering

dependency and compliance. Capacity for self assessment is central to informing

judgment But simply adding self assessment activities is not sufficient.

Communities of judgment beyond ourselves need to be engaged with (peers, practitioners, professional bodies).

22

D. Assessment as informing student judgement implies

Always look to what the consequences of assessment are for learning

Focus on fostering reflexivity and self-regulation through every aspect of a course, not just assessment tasks

Recognise the variety of contexts in which learning occurs and is utilised—judgement is not independent of context

Stage opportunities for developing informed judgement throughout programs

Assessment must be integrated with learning and integrated within the program and over time

23

The problem of judgement

Judgement is more elusive than it appears Making judgements are context specific and context

dependent There are intrinsic biases in making judgements Judgement is always a subjective act, especially when the

acts of people are judged Getting to self-assessment is essential to judgement, but it

is always flawed

24

Students as active agents in developing judgment

Student agency Communities of judgment Self assessment Role of peers

25

Why involve students?

How can we justify not involving them? But what does it mean to involve them?

Developing judgment is about more than acquiring knowledge and skills, it involves practice in discernment. More opportunities for this are needed.

Students learn a lot through contributing to the learning of others engaging with criteria, formulating ideas, taking account of

the other Students are a massively under-utilised resource at a

time of resource constraints

26

Figure 1. Elements of informed judgement.

ELEMENTS OF

INFORMED

JUDGEMENT

How achievable? What tasks may be involved?

Examples of sources

1. Identifying self as active learner

Encourage learners to identify themselves as such Construct tasks and learning activities to render learning visible Implement participatory and active approaches; build learner agency and construct active learners Encourage engagement with standards and criteria/ problem analysis

Boud and Solomon, 2003, Collins et al., 1991, Shulman, 2000, Torrance, 1995 Woodward, 1998; Winter, 2003 Dierick and Dochy, 2001

2. Finding ways of moving from what’s known to what it is desirable to know

Recognise that judgement does not come fully formed and that staging is needed to develop expertise Scaffold knowledge and assessment expertise Encourage discernment of variation Encourage engagement with peers and others Locate knowledge in local practice (situated learning and communities of practice).

Hogan and Pressley 1997, Dochy, 2001 Bowden and Marton, 1998 Falchikov, 2001 Lave and Wenger ,1991; Wenger, 1998 Goldfinch et al., 1999

3. Testing and judging

Apply research and analytic skills; resort to evidence Practice self assessment Widen and utilise types and sources of evidence of learning Encourage seeking of feedback from many sources including research

Boud, 1995 Falchikov, 1996; Black and Wiliam, 1998; Sadler, 1998: Hounsell, 2003

4. Developing these skills over time

Consider risk and confidence of judgement Integrate learning over time and courses Require portrayal of outcomes for different purposes

Kahnemann, Slovic and Tversky, 1982 Ewers and Searby, 1997

5. Embodying reflexivity and commitment

Foster awareness of personal and professional practice Foster reflexivity

Edwards et al 2003

27

Some practices for involving students

Don’t mistake practices for purposes!

Not the practice but the purpose that counts. Self assessment Peer assessment Hybrids

Ultimately all assessment must be about informing the judgment of learners as it is only they who can learn.

28

Workshop task

1. Identify an assessment activity you wish to modify to enhance its contribution to learning in the longer term and developing student judgement. Make notes about what you will change.

2. In groups of three. Take turns in sharing your planned assessment activity and getting feedback.

3. Identify an issue that has arisen to bring back to the group.

29

Issues arising from the workshop task:

30

Practicalities

A. Common features for any assessment innovation

B. Choosing appropriate tasks and processes

C. Giving and receiving feedback

31

A. Common features in any assessment innovation

Assume students will be resistant, if they are not then they’re probably not behaving rationally

Never underestimate the importance of providing a compelling rationale and reiterating it in different ways

Act confidently. Listen to their concerns, but don’t change what you are doing unless they suggest a better way of doing it.

Be much more explicit than you imagine must be necessary, give full guidelines/deadlines, etc. in writing

Reassure students that cheating/ collusion will be detected if grading is involved

Be prepared to discuss tangible benefits to them

32

B. Choosing appropriate tasks and processes

ask learners to make judgments on matters on which it is reasonable for them to do so

use cues for success which are embedded in the content as much as possible

choose situations in which there are external sources of judgment which can be drawn upon or, multiple sources of evidence are available

avoid situations in which criteria for success are matters of opinion or taste avoid incentives for mis-assessment choose specific and concrete rather than the global and abstract task limit the number of criteria which must be considered simultaneously at first develop detailed guidelines on how the process is to be undertaken

33

Assessment is not enough

We can’t consider assessment separately from teaching and learning processes. All are about informing judgment.

Alignment between and integration of learning activities is needed

Choosing assessment practices chooses what students will learn

34

The new agenda

Not just constructive alignment, but alignment of assessment now with long term learning goals

Breakdown the binary between assessment and pedagogy

Revisit assessment from the perspective of lifelong learning

Revisit pedagogy from the perspective of lifelong assessment

35

References

Boud, D. (1995). Enhancing Learning through Self Assessment. London: Kogan Page.

Boud, D. (2000). Sustainable assessment: rethinking assessment for the learning society. Studies in Continuing Education, 22, 2, 151-167.

Boud, D. and Falchikov, N. (2006). Aligning assessment with long term learning, Assessment and Evaluation in Higher Education, 31, 4, 399-413.

Boud, D. and Falchikov, N. (Eds.) (2007) Rethinking Assessment in Higher Education: Learning for the Longer Term. London: Routledge.

Falchikov, N. (2005). Improving Assessment through Student Involvement. London: Routledge.

Gibbs, G. (2006). How assessment frames student learning. In Clegg, K. and Bryan, C. (Eds.) Innovative Assessment in Higher Education. London: Routledge.