RIDE 2010 presentation - Formative e-assessment: case stories, design patterns, and future scenarios
-
Upload
centre-for-distance-education -
Category
Education
-
view
997 -
download
0
description
Transcript of RIDE 2010 presentation - Formative e-assessment: case stories, design patterns, and future scenarios
![Page 1: RIDE 2010 presentation - Formative e-assessment: case stories, design patterns, and future scenarios](https://reader036.fdocuments.us/reader036/viewer/2022062617/54d1ad8f4a7959964d8b47a0/html5/thumbnails/1.jpg)
Formative e-assessment: case stories, design patterns, and
future scenarios
Harvey MellarLondon Knowledge Lab
Institute of Education, University of London
http://feasst.wlecentre.ac.uk/
http://www.jisc.ac.uk/whatwedo/projects/feasst.aspx
![Page 2: RIDE 2010 presentation - Formative e-assessment: case stories, design patterns, and future scenarios](https://reader036.fdocuments.us/reader036/viewer/2022062617/54d1ad8f4a7959964d8b47a0/html5/thumbnails/2.jpg)
Overview
Short term, scoping study commissioned by JISC, and supported by the Centre
for Excellence in Work-based Learning for Education Professionals
• Methodology
• Desk research
• Literature review• Comparing frameworks
• Five Practical Enquiry Days
• Combination of collaborative reflection, report back from teams, and guest plenaries
• Launch day, three Planet workshops, developers' day
(Adopted and adapted the Planet Project's Participatory Methodology for
Practical Design Patterns - http://patternlanguagenetwork.wordpress.com)
• Wiki for collaborative authoring of patterns http://purl.org/planet/
Groups.FormativeEAssessment/
![Page 3: RIDE 2010 presentation - Formative e-assessment: case stories, design patterns, and future scenarios](https://reader036.fdocuments.us/reader036/viewer/2022062617/54d1ad8f4a7959964d8b47a0/html5/thumbnails/3.jpg)
WHAT IS FORMATIVE ASSESSMENT?
![Page 4: RIDE 2010 presentation - Formative e-assessment: case stories, design patterns, and future scenarios](https://reader036.fdocuments.us/reader036/viewer/2022062617/54d1ad8f4a7959964d8b47a0/html5/thumbnails/4.jpg)
A definition
“An assessment functions formatively when evidence about student achievement elicited by the assessment is interpreted and used to make decisions about the next steps in instruction that are likely to be better, or better founded, than the decisions that would have been made in the absence of that evidence”
(Dylan Wiliam)
![Page 5: RIDE 2010 presentation - Formative e-assessment: case stories, design patterns, and future scenarios](https://reader036.fdocuments.us/reader036/viewer/2022062617/54d1ad8f4a7959964d8b47a0/html5/thumbnails/5.jpg)
Five strategies
Feed up Feed back Feed forward
Black, P., & Wiliam, D. (2009). Developing the theory of formative assessment. Educational Assessment, Evaluation and Accountability, 21(1), 5-31.
![Page 6: RIDE 2010 presentation - Formative e-assessment: case stories, design patterns, and future scenarios](https://reader036.fdocuments.us/reader036/viewer/2022062617/54d1ad8f4a7959964d8b47a0/html5/thumbnails/6.jpg)
Moments of contingency
Teachers design their instruction to yield evidence about student achievement, by carefully crafting hinge-point questions, for
example. These create ‘moments of contingency’, in which the direction of the instruction will depend on student responses. Teachers provide feedback that engages students, make time in class for students to work on improvement, and activate students as instructional resources for one another.
(Leahy, Lyon, Thompson and Wiliam, 2005)
![Page 7: RIDE 2010 presentation - Formative e-assessment: case stories, design patterns, and future scenarios](https://reader036.fdocuments.us/reader036/viewer/2022062617/54d1ad8f4a7959964d8b47a0/html5/thumbnails/7.jpg)
PARTICIPATORY METHODOLOGY FOR PRACTICAL DESIGN PATTERNS
![Page 8: RIDE 2010 presentation - Formative e-assessment: case stories, design patterns, and future scenarios](https://reader036.fdocuments.us/reader036/viewer/2022062617/54d1ad8f4a7959964d8b47a0/html5/thumbnails/8.jpg)
8
Case Stories Workshop
Engender collaborative reflection among practitioners by a structured process of sharing stories of successful practice
![Page 9: RIDE 2010 presentation - Formative e-assessment: case stories, design patterns, and future scenarios](https://reader036.fdocuments.us/reader036/viewer/2022062617/54d1ad8f4a7959964d8b47a0/html5/thumbnails/9.jpg)
9
Pattern Mining Workshop
Shift from anecdotes to transferable design knowledge by identifying commonalities across case stories, and capturing them in a semi-structured form
![Page 10: RIDE 2010 presentation - Formative e-assessment: case stories, design patterns, and future scenarios](https://reader036.fdocuments.us/reader036/viewer/2022062617/54d1ad8f4a7959964d8b47a0/html5/thumbnails/10.jpg)
The core template
• Context– Where, when, who (all the things you can’t change)
• Problem (pick one!)– We want to do A under condition B but are
constrained by C
• Solution
(in any order that
works for you)C o n t e x t
Problem Solution
When, Where, Who
What are we trying to achieve / solve?
Cookbook: ingredients, procedure, expected
outcomes
![Page 11: RIDE 2010 presentation - Formative e-assessment: case stories, design patterns, and future scenarios](https://reader036.fdocuments.us/reader036/viewer/2022062617/54d1ad8f4a7959964d8b47a0/html5/thumbnails/11.jpg)
11
Future Scenarios Workshop
Validate design patterns by applying them to novel real problems in real contexts
![Page 12: RIDE 2010 presentation - Formative e-assessment: case stories, design patterns, and future scenarios](https://reader036.fdocuments.us/reader036/viewer/2022062617/54d1ad8f4a7959964d8b47a0/html5/thumbnails/12.jpg)
![Page 13: RIDE 2010 presentation - Formative e-assessment: case stories, design patterns, and future scenarios](https://reader036.fdocuments.us/reader036/viewer/2022062617/54d1ad8f4a7959964d8b47a0/html5/thumbnails/13.jpg)
CASES
![Page 14: RIDE 2010 presentation - Formative e-assessment: case stories, design patterns, and future scenarios](https://reader036.fdocuments.us/reader036/viewer/2022062617/54d1ad8f4a7959964d8b47a0/html5/thumbnails/14.jpg)
Creature of the week (Judy Robertson)
Situation
Large class (138), first and second year computer science students. Assignment: create a virtual pet in Second Life
Task Engage and motivate the students show examples of good work which others could
learn from show students their work is valued. build a sense of community.
http://purl.org/planet/Cases/creatureoftheweek
![Page 15: RIDE 2010 presentation - Formative e-assessment: case stories, design patterns, and future scenarios](https://reader036.fdocuments.us/reader036/viewer/2022062617/54d1ad8f4a7959964d8b47a0/html5/thumbnails/15.jpg)
![Page 16: RIDE 2010 presentation - Formative e-assessment: case stories, design patterns, and future scenarios](https://reader036.fdocuments.us/reader036/viewer/2022062617/54d1ad8f4a7959964d8b47a0/html5/thumbnails/16.jpg)
CoMo (Niall Winters, Yishay Mor)
Situation Royal Veterinary College Hospital rotations as part of the training
Task Allow students to capture critical incidents in text
and image Support sharing of clinical experiences and co-
reflection
http://purl.org/planet/Cases/CoMo
![Page 17: RIDE 2010 presentation - Formative e-assessment: case stories, design patterns, and future scenarios](https://reader036.fdocuments.us/reader036/viewer/2022062617/54d1ad8f4a7959964d8b47a0/html5/thumbnails/17.jpg)
![Page 18: RIDE 2010 presentation - Formative e-assessment: case stories, design patterns, and future scenarios](https://reader036.fdocuments.us/reader036/viewer/2022062617/54d1ad8f4a7959964d8b47a0/html5/thumbnails/18.jpg)
Open mentor (Denise Whitelock)
http://purl.org/planet/Cases/OpenMentor
![Page 19: RIDE 2010 presentation - Formative e-assessment: case stories, design patterns, and future scenarios](https://reader036.fdocuments.us/reader036/viewer/2022062617/54d1ad8f4a7959964d8b47a0/html5/thumbnails/19.jpg)
PATTERNS
![Page 20: RIDE 2010 presentation - Formative e-assessment: case stories, design patterns, and future scenarios](https://reader036.fdocuments.us/reader036/viewer/2022062617/54d1ad8f4a7959964d8b47a0/html5/thumbnails/20.jpg)
Try Once, Refine Once
(Aliy Fowler)
http://purl.org/planet/Patterns/TryOnceRefineOnce
![Page 21: RIDE 2010 presentation - Formative e-assessment: case stories, design patterns, and future scenarios](https://reader036.fdocuments.us/reader036/viewer/2022062617/54d1ad8f4a7959964d8b47a0/html5/thumbnails/21.jpg)
ProblemLack of immediate feedback for students leads to fossilisation of errors and misconceptions
providing immediate feedback in an iterative fashion can also hinder effective learning since students are able to "grope their way" step-by-step to a correct solution without necessarily having to think about each answer as a whole.
![Page 22: RIDE 2010 presentation - Formative e-assessment: case stories, design patterns, and future scenarios](https://reader036.fdocuments.us/reader036/viewer/2022062617/54d1ad8f4a7959964d8b47a0/html5/thumbnails/22.jpg)
Context
Class size Large (30-300)
Content Skills\facts
Mode of instruction Blended\on-line. Computer tested
![Page 23: RIDE 2010 presentation - Formative e-assessment: case stories, design patterns, and future scenarios](https://reader036.fdocuments.us/reader036/viewer/2022062617/54d1ad8f4a7959964d8b47a0/html5/thumbnails/23.jpg)
Solution• Students are posed questions of a type which elicit answers that
can contain multiple errors
• If a student's answer is entirely correct a mark of 100% is awarded
• If their answer contains errors, a mark is given which contributes to a percentage of the total mark for the question, along with detailed - yet generic - feedback on the location and type of the errors
• Students are then permitted a second attempt in which to refine their answer
• The mark for the 2nd attempt contributes to remaining percentage of the total mark for the question
• Feedback on any remaining errors is also given, along with the correct answer(s)
• No further attempts are permitted
![Page 24: RIDE 2010 presentation - Formative e-assessment: case stories, design patterns, and future scenarios](https://reader036.fdocuments.us/reader036/viewer/2022062617/54d1ad8f4a7959964d8b47a0/html5/thumbnails/24.jpg)
Feedback on
Feedback
(Linda McGuigan)
http://purl.org/planet/Patterns/FeedbackonFeedback
![Page 25: RIDE 2010 presentation - Formative e-assessment: case stories, design patterns, and future scenarios](https://reader036.fdocuments.us/reader036/viewer/2022062617/54d1ad8f4a7959964d8b47a0/html5/thumbnails/25.jpg)
Good feedback should
Alert learners to their weaknesses. Diagnose the causes and dynamics of these. Include operational suggestions to improve the
learning experience. Address socio-emotive factors.
Tutors know this, but are pressed for time, or are not aware of their feedback strategies
Large teaching organisations are not equipped to provide tutors with personal feedback on their teaching
Problem
![Page 26: RIDE 2010 presentation - Formative e-assessment: case stories, design patterns, and future scenarios](https://reader036.fdocuments.us/reader036/viewer/2022062617/54d1ad8f4a7959964d8b47a0/html5/thumbnails/26.jpg)
Context
Large scale, technology supported, graded courses many tutors instructing many students
Feedback is mediated by technology that allows it to be captured and processed in real time
Topic of study is subject to both grading and formative feedback
![Page 27: RIDE 2010 presentation - Formative e-assessment: case stories, design patterns, and future scenarios](https://reader036.fdocuments.us/reader036/viewer/2022062617/54d1ad8f4a7959964d8b47a0/html5/thumbnails/27.jpg)
Solution
Embed a mechanism in the learning and teaching system that regularly captures tutor feedback, analyses it, and presents them with graphical representation of the types of feedback they have given. Ideally, this should also include constructive advice as to how to shift from less to more effective forms.
In computer supported environments (e.g. VLEs), this mechanism could be integrated into the system, providing tutors with immediate analysis of their feedback, as well as long-term aggregates.
![Page 28: RIDE 2010 presentation - Formative e-assessment: case stories, design patterns, and future scenarios](https://reader036.fdocuments.us/reader036/viewer/2022062617/54d1ad8f4a7959964d8b47a0/html5/thumbnails/28.jpg)
SCENARIO
![Page 29: RIDE 2010 presentation - Formative e-assessment: case stories, design patterns, and future scenarios](https://reader036.fdocuments.us/reader036/viewer/2022062617/54d1ad8f4a7959964d8b47a0/html5/thumbnails/29.jpg)
High achievers
When using Try Once Refine Once, there is a risk that high-achievers do not receive feedback
So
• Use Showcase Learning to celebrate students’ work and provoke feedback from peers and tutors
• Use Feedback on Feedback to alert tutors to the problem
![Page 30: RIDE 2010 presentation - Formative e-assessment: case stories, design patterns, and future scenarios](https://reader036.fdocuments.us/reader036/viewer/2022062617/54d1ad8f4a7959964d8b47a0/html5/thumbnails/30.jpg)
AUGMENTED DOMAIN MAP
![Page 31: RIDE 2010 presentation - Formative e-assessment: case stories, design patterns, and future scenarios](https://reader036.fdocuments.us/reader036/viewer/2022062617/54d1ad8f4a7959964d8b47a0/html5/thumbnails/31.jpg)
Reminder of the five strategies
Feed up Feed back Feed forward
Black, P., & Wiliam, D. (2009). Developing the theory of formative assessment. Educational Assessment, Evaluation and Accountability, 21(1), 5-31.
![Page 32: RIDE 2010 presentation - Formative e-assessment: case stories, design patterns, and future scenarios](https://reader036.fdocuments.us/reader036/viewer/2022062617/54d1ad8f4a7959964d8b47a0/html5/thumbnails/32.jpg)
![Page 33: RIDE 2010 presentation - Formative e-assessment: case stories, design patterns, and future scenarios](https://reader036.fdocuments.us/reader036/viewer/2022062617/54d1ad8f4a7959964d8b47a0/html5/thumbnails/33.jpg)
REPORTS
![Page 34: RIDE 2010 presentation - Formative e-assessment: case stories, design patterns, and future scenarios](https://reader036.fdocuments.us/reader036/viewer/2022062617/54d1ad8f4a7959964d8b47a0/html5/thumbnails/34.jpg)
http://www.jisc.ac.uk/media/documents/projects/scopingfinalreport.pdf
![Page 35: RIDE 2010 presentation - Formative e-assessment: case stories, design patterns, and future scenarios](https://reader036.fdocuments.us/reader036/viewer/2022062617/54d1ad8f4a7959964d8b47a0/html5/thumbnails/35.jpg)
Practical design patterns for teaching and learning with
technology
A book for Sense Publisher's 'Technology Enhanced Learning' series
Editors: Yishay Mor (London Knowledge Lab), Steven Warburton (King's College London) and Niall Winters (London Knowledge Lab)
http://www.practicalpatternsbook.org/Home