VIEWING SUMMATIVE ASSESSMENT

THROUGH DIFFERENT LENSESPeter Reed

eAssessment Scotland Keynote Presentation

@Reedyreedles#eas14

CC BY flickr photo by ** RCB **

What’s your role?

CC BY-NC flickr photo by mattcornock

What do you see when you think about assessment?

CC BY-ND flickr photo by Adam Hinett

LEARNING TECHNOLOGIST

PROJECT COORDINATOR ACADEMIC

SELF REFLECTION

STUDENTS PEERS

BROKFIELD’S LENSES

LITERATURE

SELF REFLECTION

STUDENTS PEERS

AM I CRITICALLY REFLETIVE?

PROGRAMME SYSTEM(S)

LITERATURE

STUDENTSTHE

INSTITUTIONRECRUITMENT& RETENTION

NSS as driver for change

What can we do to improve the student experience?

Is there a danger in using the NSS as a driver for change?

A BIGGER PICTURE…

FROM THIS…

CCBY-NC-ND flickr photo by Stanford EdTech

WHERE STRATEGIC CHANGE IS NEEDEDTO THIS?

CC BY-NC-SA flickr photo by MizzD

SYSTEMS & ‘THE SYSTEM’

STUDENT AS PRODUCER !=

TRADITIONAL COURSEWORK

RECEIPTING

We're operating in a world in which the rules and processes we have were not invented for

‘Practice remains stubbornly resistant to change’ (JISC, 2012)

FABRIC & CULTURE

FABRIC & CULTURE

WRITTEN EXAMS ONLINE EXAMS

WORK-BASED FORMS ELECTRONIC SIGNATUARES

PATCH UP VS REBUILD

Bigger picture & Perspectives

Rebuilding ‘The System’

CC BY-NC flickr photo by That Canadian Grrl

FOCUSING THE PICTURE

SELF REFLECTION

STUDENTS PEERSLITERATURE

LITERATURE & SUPPORT

SELF REFLECTION

STUDENTS PEERSLITERATURE

TECHNOLOGY & THE LIFECYCLE

EFFICIENCY

ENHANCEMENT

TRANSFORMATION

SPECIFYING

SETTING

SUPPORTING

SUBMITTING

MARKING & FEEDBACK

RECORDING GRADES

REFLECTING

Very modular?

CONSEQUENTIAL VALIDITY

"Students experience the interaction effects of one form of

assessment on another. In any given month they may have to

complete ten assessment tasks, in another month only one. The ways

in which they approach each of these will be influenced by the others.

A task which is intrinsically interesting and which may be approached

meaningfully at any other time may be given short shrift when it is

located among a thicket of examinations.

Very little attention has been given to the compounding effects of

assessment even when we know that it is the total array of demands

in a given period which influences how each one is tackled.”

(Boud p3, in Knight 1995)

ASSESSMENT TIMELINES

What does assessment look like through the student lens? As a whole?

What do you see?

Represents low stakes assessments

Represents medium stakes assessment

Represents high stakes assessments

Week 3 6 9 12

Week 3 6 9 12

Week 3 6 9 12

Week 3 6 9 12

ASSESSMENT TIMELINES

Bunching effect

Represents low stakes assessments

Represents medium stakes assessment

Represents high stakes assessments

Week 3 6 9 12

Week 3 6 9 12

Week 3 6 9 12

Week 3 6 9 12

ASSESSMENT TIMELINES

Little/no feed forward within a module

Represents low stakes assessments

Represents medium stakes assessment

Represents high stakes assessments

Week 3 6 9 12

Week 3 6 9 12

Week 3 6 9 12

Week 3 6 9 12

ASSESSMENT TIMELINES

…or between modules

Represents low stakes assessments

Represents medium stakes assessment

Represents high stakes assessments

Week 3 6 9 12

Week 3 6 9 12

Week 3 6 9 12

Week 3 6 9 12

ASSESSMENT TIMELINES

Strategic planning of summative assessments (ESCAPE Project, Hertfordshire)JISC Design Studio on Transforming A&F

Week 3 6 9 12

Week 3 6 9 12

Week 3 6 9 12

Continuous or low stakes assessments throughout

Split one high stakes summative into two medium stakes assessments & low stakes

Traditional summative assessments at the end of module

Represents low stakes assessments

Represents medium stakes assessment

Represents high stakes assessments

Week 5 10 25 403020 5035 4515

Y1

Y2

Y3

Y4

Y5

Represents formative assessments

Represents re-sit opportunity

Represents high stakes assessments

SEMESTER 1 SEMESTER 2 SEMESTER 3

ASSESSMENT TIMELINES

Draft MBCHB Curriculum

ASSESSMENT TIMELINES

Week 10 13 16 22 26 29 39 433024

YEAR

1

1.011.02

1.031.04

1.051.06

1.071.08

WRITTEN EXAM2 X 1.5 HR PAPERS

RESEARCH & SCHOLARSHIP

Week 4 8 13 26 38 433121 4934 45

2.012.02

2.032.04

2.052.06

2.072.08

OSCERESEARCH & SCHOLARSHIP

WRITTEN EXAM2 X 1.5 HR PAPERS

RESIT

YEAR

2Represents online formative assessments

Represents re-sit opportunity

Represents summative assessments

RESIT

What things pop out at you?

ASSESSMENT TIMELINES

SELF REFLECTION

STUDENTS PEERSLITERATURE

STUDENT WORKLOADGOOD PRACTICE LOST

GOOD PRACTICE LOST

PROGRAMME SYSTEM(S)WEIGHTINGS & FORMATS

IMPACT ON PROCESSES

FEED-FORWARD

LANGUAGE

SYSTEM PERFORMANCE

TESTA Project @ Dundee

USING THE LENSES

SELF REFLECTION

STUDENTS PEERSLITERATURE

ONE MORE THING

‘Learning analytics’

‘refers to the interpretation of a wide range of data

produced by and gathered on behalf of students in

order to assess academic progress, predict future

performance, and spot potential issues’ (L. Johnson,

Smith, Willis, Levine, & Haywood, 2011).

ONE MORE THING

“Assessment analytics offers the potential for

student attainment to be measured across time, in

comparison with individual students’ starting points

(ipsative development), with their peers and/or

against benchmarks or standards” (Ellis, 2013, p.

663).

ONE MORE THING

ONE MORE THING

Ahead of Mark’s keynote, consider:

• How/what might we analyse?

• Reflect through the student lens - ethical issues

your data being analysed?

• Staff lens – what about your data?

• How might we use data to inform future

practice?