Electronic Management of Assessment A tour of the UK landscape 09/09/2015 CC BY-NC-ND 2.0 Marius...

Electronic Management of AssessmentA tour of the UK landscape

09/09/2015

CC BY-NC-ND 2.0 Marius Kluzniak

Pre-departure

09/09/2015

Electronic management of assessment: a tour of the UK landscape

» Jisc Assessment & Feedback programme 2011-2014

» University baseline reviews

» Baseline review of landscape published 2012

CC BY-NC-ND 2.0 Nicole Hanusek

Travel advisory (1)

09/09/2015


» Traditional culture stubbornly resistant to change

» Devolved responsibility & varied practice

» Limited opportunity for students to engage with process of making academic judgements

» Issues with degree structure

CC BY-NC-ND 2.0 Richard Lee

Travel advisory (2)

09/09/2015


» Investment in technical infrastructure not yet delivering efficiencies

» Preference for established technologies but innovation still difficult to scale up

» Current practice not promoting employability

CC BY-NC-ND 2. 0 Soilse

Travel essentials

09/09/2015


» Lifecycle Model provides our ‘map’ & ‘phrase book’

» Academic model against which processes and systems can be mapped

» Common framework for different stakeholders to discuss issues

» Means of clarifying requirements for suppliers

CC BY-NC 2.0 Marcus Ramberg

The Assessment and Feedback Lifecycle

09/09/2015


Based on an original idea developed by Manchester Metropolitan University

Starting point

09/09/2015


» Educational principles egREAP project University of Strathclyde;University of Hertfordshire Assessment for learning

» Appreciative Inquiry approaches

» University of Ulster Viewpoints project

CC BY-SA 2.0 Michael Coglan

Principles at Queen’s University Belfast

09/09/2015


Supporting longitudinal development

09/09/2015


» Assessment for learning rather than of learning

» More formative opportunities

» Feed forward

» Ipsative approaches

» Assesment scheduling

» Access to information

CC BY-NC-SA 2.0 CrazyFast

Assessment timelines

09/09/2015


University of Dundee course redesign using University of Hertfordshire assessment timelines tool

The role of feedback

09/09/2015


» Feedback is a ‘black box’

» Programme teams don’t discuss feedback

» Useful analytical tools available

CC BY-NC-SA 2.0 xdxd vs xdxd

What kind of feedback do you give?

09/09/2015


» P1 - Praise. Motivating but if used indiscriminately can appear insincere.

» P2 - Recognising Progress (ipsative feedback). Can be motivating and informs students about their learning. Lack of progress serves as an early warning.

» C - Critique. How work falls short of expectations or criteria; can be discouraging if not accompanied by information on how to improve.

» A - Advice. Help students take future action to improve.

» Q - Clarification requests. Asking learners to think more deeply about their work and generate actions themselves.

» O - Unclassified statements. Neutral comments, for example that describe the piece of work but do not make any judgement.

(adapted from Institute of Education feedback profile)

Electronic Management of Assessment

09/09/2015


» Jisc EMA project

» Heads of e-Learning Forum surveys

» Questionnaire (65 HEIs)

» Think tank & workshops

» Interviews

» Discussions on EMA blog

» Landscape review reporthttp://repository.jisc.ac.uk/5599/1/EMA_REPORT.pdf

CC BY 2.0 acme

http://repository.jisc.ac.uk/5599/1/EMA_REPORT.pdf

http://repository.jisc.ac.uk/5599/1/EMA_REPORT.pdf

Localised initiatives

09/09/2015


e-Submission e-Marking e-Feedback e-Exams0

20

40

60

80

100

120

How extensively has your organisation applied EMA?

N/ASmall scale pilotsLarge scale pilotsMandated locallyMandated organisation wideDon't know

Pain points

09/09/2015


Systems integration

09/09/2015


» Limited range of core technologies

» Integration loose & requiring manual intervention

» More universities say their assessment systems are ‘completely separate’ than ‘highly integrated’

CC BY-NC-SA 2.0 Cesar Harada

Business processes

09/09/2015


» Lots of local variation

» Only 12% of universities have ‘highly standardised’ processes

» Different interpretations of policy

» Variation hidden till EMA tried


Top 10 EMA challenges

09/09/2015


Top 20 EMA challenges

09/09/2015

Working towards solutions

09/09/2015


» Validation of priorities

» Workshop to generate solution ideas

» Validation and refinement of solutions

» Talking to others: EUNIS, SURF, UNINETT

» Toolkit in development

Project timeline

09/09/2015


EMA workflows project – mapping requirements

Ongoing capture of resources – ‘EMA toolkit’

Landscape review

– challeng

es prioritise

dFeedback hub scoping

Single online

resource

Summer 2014

Spring/Summer 2015 October 2015

Pilots

Autumn/Winter 2015


Souvenir shop

09/09/2015


» A&F Baseline review: http://www.webarchive.org.uk/wayback/archive/20140614114153/http://www.jisc.ac.uk/media/documents/programmes/elearning/Assessment/JISCAFBaselineReportMay2012.pdf

» A&F Final report:http://repository.jisc.ac.uk/5450/4/Jisc_AF_Final_Synthesis_Report_Oct_2013_v2.pdf

» EMA Landscape review: http://repository.jisc.ac.uk/5599/

» Jisc EMA blog: http://ema.jiscinvolve.org/wp/

CC BY-NC-ND-SA 2.0 ericalaspada

http://www.webarchive.org.uk/wayback/archive/20140614114153/http:/www.jisc.ac.uk/media/documents/programmes/elearning/Assessment/JISCAFBaselineReportMay2012.pdf




http://repository.jisc.ac.uk/5450/4/Jisc_AF_Final_Synthesis_Report_Oct_2013_v2.pdf

http://repository.jisc.ac.uk/5450/4/Jisc_AF_Final_Synthesis_Report_Oct_2013_v2.pdf

http://repository.jisc.ac.uk/5599/

http://ema.jiscinvolve.org/wp/

http://ema.jiscinvolve.org/wp/

Prefer it gift wrapped?

09/09/2015


» Look out for our Assessment Toolkitdue to launch in October 2015

jisc.ac.uk

Except where otherwise noted, this work is licensed under CC-BY-NC-ND


Stay in touch…

Contact

Lisa GraySenior Co-Design Manager, Student [email protected]

09/09/2015

Gill [email protected]

http://www.jisc.ac.uk/

Electronic Management of Assessment A tour of the UK landscape 09/09/2015 CC BY-NC-ND 2.0 Marius...

Documents

Transcript of Electronic Management of Assessment A tour of the UK landscape 09/09/2015 CC BY-NC-ND 2.0 Marius...