Technology- enhanced examinations at the University of Bradford John Dermo, Centre for Educational...
-
Upload
georgiana-jefferson -
Category
Documents
-
view
220 -
download
1
Transcript of Technology- enhanced examinations at the University of Bradford John Dermo, Centre for Educational...
![Page 1: Technology- enhanced examinations at the University of Bradford John Dermo, Centre for Educational Development, University of Bradford University of Edinburgh.](https://reader038.fdocuments.us/reader038/viewer/2022103022/56649ce65503460f949b3644/html5/thumbnails/1.jpg)
Technology-enhanced examinations at the University of Bradford
John Dermo, Centre for Educational Development,
University of Bradford
University of Edinburgh 23rd April 2012
![Page 2: Technology- enhanced examinations at the University of Bradford John Dermo, Centre for Educational Development, University of Bradford University of Edinburgh.](https://reader038.fdocuments.us/reader038/viewer/2022103022/56649ce65503460f949b3644/html5/thumbnails/2.jpg)
2
Technology enhanced exams at UoB
• A strategic approach to technology enhanced assessment
• Drivers: including the student perspective• Developing a facility for e-assessment• Innovative e-assessment at UoB
![Page 3: Technology- enhanced examinations at the University of Bradford John Dermo, Centre for Educational Development, University of Bradford University of Edinburgh.](https://reader038.fdocuments.us/reader038/viewer/2022103022/56649ce65503460f949b3644/html5/thumbnails/3.jpg)
3
e-Learning Pathfinder Programme“Embedding Support Processes for e-Assessment”
Pathfinder Project 2007-8
http://www.bradford.ac.uk/educational-development/educational-research/pathfinder-caa/
![Page 4: Technology- enhanced examinations at the University of Bradford John Dermo, Centre for Educational Development, University of Bradford University of Edinburgh.](https://reader038.fdocuments.us/reader038/viewer/2022103022/56649ce65503460f949b3644/html5/thumbnails/4.jpg)
4
JISC Institutional Exemplar Project“Integrating thin client systems and smart card technology for flexible, accessible and secure E-assessment.”
ITS4SEA Project 2007-9
http://www.jisc.ac.uk/whatwedo/programmes/institutionalinnovation/its4sea.aspx
Institutional Innovation Programme
![Page 5: Technology- enhanced examinations at the University of Bradford John Dermo, Centre for Educational Development, University of Bradford University of Edinburgh.](https://reader038.fdocuments.us/reader038/viewer/2022103022/56649ce65503460f949b3644/html5/thumbnails/5.jpg)
5
Drivers: reasons for focus on online assessment
• University e-Strategy (2004-9)• LTA strategy and Academic Framework
(2009-14)• National Student Survey• Increased use of e-learning• Student expectations
![Page 6: Technology- enhanced examinations at the University of Bradford John Dermo, Centre for Educational Development, University of Bradford University of Edinburgh.](https://reader038.fdocuments.us/reader038/viewer/2022103022/56649ce65503460f949b3644/html5/thumbnails/6.jpg)
6
• In 20 years HE has seen radical and unprecedented change (Education Act, 1992; Dearing Report,
1997; Roberts Report, 2003; Leitch Report, 2006; Browne, 2010; CSR, 2010)
• Learners have different expectations and assumptions about their HE experiences
• The student body has become dramatically more heterogeneous
• Impact of student fees
![Page 7: Technology- enhanced examinations at the University of Bradford John Dermo, Centre for Educational Development, University of Bradford University of Edinburgh.](https://reader038.fdocuments.us/reader038/viewer/2022103022/56649ce65503460f949b3644/html5/thumbnails/7.jpg)
7
NATIONAL UNION OF STUDENTS’ PRINCIPLES OF EFFECTIVE ASSESSMENT (2009)
1. Should be for learning, not simply of learning
2. Should be reliable, valid, fair and consistent
3. Should consist of effective and constructive feedback
4. Should be innovative and have the capacity to inspire and motivate
5. Should measure understanding and application, rather than technique and memory
6. Should be conducted throughout the course, rather than being positioned as a final event
7. Should develop key skills such as peer and reflective assessment
8. Should be central to staff development and teaching strategies, and frequently reviewed
9. Should be of a manageable amount for both tutors and students
10. Should encourage dialogue between students and their tutors and students and their peers
![Page 8: Technology- enhanced examinations at the University of Bradford John Dermo, Centre for Educational Development, University of Bradford University of Edinburgh.](https://reader038.fdocuments.us/reader038/viewer/2022103022/56649ce65503460f949b3644/html5/thumbnails/8.jpg)
8
“Please, Sir, may I have some more exams?”
Becka Colley, Dean of Students, University of Bradford. Keynote speech at eAssessment Scotland, 2011http://www.slideshare.net/rcurrant/please-sir-may-i-have-some-more-exams
With apologies to Charles Dickens and Ronald Searle
![Page 9: Technology- enhanced examinations at the University of Bradford John Dermo, Centre for Educational Development, University of Bradford University of Edinburgh.](https://reader038.fdocuments.us/reader038/viewer/2022103022/56649ce65503460f949b3644/html5/thumbnails/9.jpg)
9
There was a strong preference to retain two exam sessions. This was felt to be more manageable in terms of revision. Students were not in favour of one exam session at the end of the academic year. A staged approach with summative assessment was viewed as the most appropriate. Also, January exams give an opportunity for students to ‘trial’ their exam revision and writing skills.
[UBU feedback on review of the academic year, June 2011]
Student Union Feedback
![Page 10: Technology- enhanced examinations at the University of Bradford John Dermo, Centre for Educational Development, University of Bradford University of Edinburgh.](https://reader038.fdocuments.us/reader038/viewer/2022103022/56649ce65503460f949b3644/html5/thumbnails/10.jpg)
10
• Dedicated support• Move towards integrated
processes and systems• Workflow model and clear roles
and communication channels• Approved policies• Training materials and sessions,
briefings• Develop new flexible space for
e-assessment
Developments at UoB
![Page 11: Technology- enhanced examinations at the University of Bradford John Dermo, Centre for Educational Development, University of Bradford University of Edinburgh.](https://reader038.fdocuments.us/reader038/viewer/2022103022/56649ce65503460f949b3644/html5/thumbnails/11.jpg)
11
100 seat e-assessment cluster
![Page 12: Technology- enhanced examinations at the University of Bradford John Dermo, Centre for Educational Development, University of Bradford University of Edinburgh.](https://reader038.fdocuments.us/reader038/viewer/2022103022/56649ce65503460f949b3644/html5/thumbnails/12.jpg)
12
![Page 13: Technology- enhanced examinations at the University of Bradford John Dermo, Centre for Educational Development, University of Bradford University of Edinburgh.](https://reader038.fdocuments.us/reader038/viewer/2022103022/56649ce65503460f949b3644/html5/thumbnails/13.jpg)
13
Smartcards: link each exam to a different card
VMware and Sun Ray servers
![Page 14: Technology- enhanced examinations at the University of Bradford John Dermo, Centre for Educational Development, University of Bradford University of Edinburgh.](https://reader038.fdocuments.us/reader038/viewer/2022103022/56649ce65503460f949b3644/html5/thumbnails/14.jpg)
14
Benefits of e-assessment suite for online assessments
• Can re-image software available very quickly between exams
• Easy to control software available, flexible using Appsense • Can link assessment to card to allow multiple simultaneous
exams in single location• Easier to invigilate – more secure• Room easier to find• More pleasant environment• Easier to manage extra time students
![Page 15: Technology- enhanced examinations at the University of Bradford John Dermo, Centre for Educational Development, University of Bradford University of Edinburgh.](https://reader038.fdocuments.us/reader038/viewer/2022103022/56649ce65503460f949b3644/html5/thumbnails/15.jpg)
15
First used January 2009, since when the use for summative e-assessment has gradually increased:
Usage: 2500 different students per year, from more than 30 modules covering range of subject areas across disciplines
“I’ve already told the DVC that we will need another cluster like this within 2 years”
Associate Dean (Teaching and Learning) Life Sciences
“This is just what I always wanted but we couldn’t do on the PCs!”
Lecturer in School of Management
![Page 16: Technology- enhanced examinations at the University of Bradford John Dermo, Centre for Educational Development, University of Bradford University of Edinburgh.](https://reader038.fdocuments.us/reader038/viewer/2022103022/56649ce65503460f949b3644/html5/thumbnails/16.jpg)
16
• Growing interest in e-assessment
• Further demand on facilities, especially for summative assessment
• More efficient use of space (smartcards, flexible scheduling)
• “Overflow” room to increase capacity to 130
• Single rooms available when necessary
![Page 17: Technology- enhanced examinations at the University of Bradford John Dermo, Centre for Educational Development, University of Bradford University of Edinburgh.](https://reader038.fdocuments.us/reader038/viewer/2022103022/56649ce65503460f949b3644/html5/thumbnails/17.jpg)
17
Some Student Feedback
Strongly Disagree
Disagree No Opinion Agree Strongly Agree
0
20
40
60
80
100
120
The room is appropriately set out for a summative assessment.
Strongly Disagree
Disagree No Opinion Agree Strongly Agree
0
20
40
60
80
100
The questions were easy to read and answer on the computer screen.
Strongly Disagree
Disagree No Opinion Agree Strongly Agree
0
20
40
60
80
100
I preferred doing the assessment on the computer rather than on paper.
Strongly Disagree
Disagree No Opinion Agree Strongly Agree
0
20
40
60
80
100
Using computerized tests would be appropriate for other summative tests or exams.
![Page 18: Technology- enhanced examinations at the University of Bradford John Dermo, Centre for Educational Development, University of Bradford University of Edinburgh.](https://reader038.fdocuments.us/reader038/viewer/2022103022/56649ce65503460f949b3644/html5/thumbnails/18.jpg)
18
Varied use of e-assessment suite
• exam periods: block booked by exams office for summative e-assessment
• term time – managed by room bookings for• one off e-assessments• regular teaching sessions
• otherwise: open 24/7 as self access cluster for students
![Page 19: Technology- enhanced examinations at the University of Bradford John Dermo, Centre for Educational Development, University of Bradford University of Edinburgh.](https://reader038.fdocuments.us/reader038/viewer/2022103022/56649ce65503460f949b3644/html5/thumbnails/19.jpg)
19
24/7 Cluster Use
• Room in regular use• “vibrant environment”• Pleasant learning space• Encourages group work• Demand during exam period• But… limitations of thin-clients?
![Page 20: Technology- enhanced examinations at the University of Bradford John Dermo, Centre for Educational Development, University of Bradford University of Edinburgh.](https://reader038.fdocuments.us/reader038/viewer/2022103022/56649ce65503460f949b3644/html5/thumbnails/20.jpg)
20
Challenges
How to manage the resource 365 days a year?Who owns the facility?How to get the most out of the investment?How to manage larger groups?Booking the room?Server capacity to allow 100+ logins? Expanding the use of the room to include essay exams (cf interest in e-submission and e-marking)
![Page 21: Technology- enhanced examinations at the University of Bradford John Dermo, Centre for Educational Development, University of Bradford University of Edinburgh.](https://reader038.fdocuments.us/reader038/viewer/2022103022/56649ce65503460f949b3644/html5/thumbnails/21.jpg)
21
Integrating Institutional Processes
![Page 22: Technology- enhanced examinations at the University of Bradford John Dermo, Centre for Educational Development, University of Bradford University of Edinburgh.](https://reader038.fdocuments.us/reader038/viewer/2022103022/56649ce65503460f949b3644/html5/thumbnails/22.jpg)
22
But don’t forget the human processes which go with the technical systems
![Page 23: Technology- enhanced examinations at the University of Bradford John Dermo, Centre for Educational Development, University of Bradford University of Edinburgh.](https://reader038.fdocuments.us/reader038/viewer/2022103022/56649ce65503460f949b3644/html5/thumbnails/23.jpg)
23
Innovation requires commitment and acceptance of responsibility across the institution
instructors course teams invigilators students department admin exams office disabilities office senior managementIT services support teamsroom bookings line managersestates
![Page 24: Technology- enhanced examinations at the University of Bradford John Dermo, Centre for Educational Development, University of Bradford University of Edinburgh.](https://reader038.fdocuments.us/reader038/viewer/2022103022/56649ce65503460f949b3644/html5/thumbnails/24.jpg)
24
Lessons Learned at Bradford?
Challenges are not just technology-basedCommunication is keyYour systems are only as good as the data in themAutomated systems require humans to play their part
![Page 25: Technology- enhanced examinations at the University of Bradford John Dermo, Centre for Educational Development, University of Bradford University of Edinburgh.](https://reader038.fdocuments.us/reader038/viewer/2022103022/56649ce65503460f949b3644/html5/thumbnails/25.jpg)
25
We all know what an exam looks like, don’t we??
From this… …to this.
![Page 26: Technology- enhanced examinations at the University of Bradford John Dermo, Centre for Educational Development, University of Bradford University of Edinburgh.](https://reader038.fdocuments.us/reader038/viewer/2022103022/56649ce65503460f949b3644/html5/thumbnails/26.jpg)
26
Examples of authenticity
Department of Biomedical Sciences: • Combination of web-based genetic
research tool (BLAST) with MCQs to deliver authentic, real-life tasks which require application of knowledge and understanding in a real-life professional context
• Can be assessed automatically = great for large numbers of students
![Page 27: Technology- enhanced examinations at the University of Bradford John Dermo, Centre for Educational Development, University of Bradford University of Edinburgh.](https://reader038.fdocuments.us/reader038/viewer/2022103022/56649ce65503460f949b3644/html5/thumbnails/27.jpg)
27
School of Medical Sciences: • Use of formative “e-assessment for
learning” (in F42 and via VLE) to deliver automatic generic feedback to students.
• Work has been carried out on delivering question level feedback and topic based feedback – students would appear to best appreciate a combination of these.
Examples of authenticity
![Page 28: Technology- enhanced examinations at the University of Bradford John Dermo, Centre for Educational Development, University of Bradford University of Edinburgh.](https://reader038.fdocuments.us/reader038/viewer/2022103022/56649ce65503460f949b3644/html5/thumbnails/28.jpg)
28
School of Optometry: • Use F42 for “Active problem-based learning” • Formative group based case studies• Information is automatically and gradually
released during process• Students make joint decisions, then receive
email feedback. • Group focus initially then individual
summative process
Examples of authenticity
![Page 29: Technology- enhanced examinations at the University of Bradford John Dermo, Centre for Educational Development, University of Bradford University of Edinburgh.](https://reader038.fdocuments.us/reader038/viewer/2022103022/56649ce65503460f949b3644/html5/thumbnails/29.jpg)
29
• “Hybrid” assessments
• Some questions automatically marked, others done by a human
• More efficient use of human resources to allow time to be spent on students and feedback processes.
Other Developments
![Page 30: Technology- enhanced examinations at the University of Bradford John Dermo, Centre for Educational Development, University of Bradford University of Edinburgh.](https://reader038.fdocuments.us/reader038/viewer/2022103022/56649ce65503460f949b3644/html5/thumbnails/30.jpg)
30
• School of Health Studies, School of Management and School of Lifelong Education and Development:
• Using e-peer assessment• Students grading performance and/or give
feedback collaborative group tasks. • This assessment can take place either all
together in the e-assessment suite or remotely via the VLE.
Other Developments
![Page 31: Technology- enhanced examinations at the University of Bradford John Dermo, Centre for Educational Development, University of Bradford University of Edinburgh.](https://reader038.fdocuments.us/reader038/viewer/2022103022/56649ce65503460f949b3644/html5/thumbnails/31.jpg)
31
“Activity-based Student-centred” (AbSc) teaching and learning: an Engineering perspective at Bradford
• 8 e-Assessments in 12 weeks in F42 (including 6 bi-weekly assessments, one e-assessment on lab work and one mid-term e-assessment)
• Supported by online quizzes – self assessment with feedback done in own time in VLE
• Final summative e-assessment in F42
Other Developments
![Page 32: Technology- enhanced examinations at the University of Bradford John Dermo, Centre for Educational Development, University of Bradford University of Edinburgh.](https://reader038.fdocuments.us/reader038/viewer/2022103022/56649ce65503460f949b3644/html5/thumbnails/32.jpg)
32
F42