Eassessment Bob Rotheram
-
Upload
kirsteen-macdonald -
Category
Education
-
view
886 -
download
0
Transcript of Eassessment Bob Rotheram
Assessment and feedback: technology to the rescue?
Bob RotheramNational Teaching Fellow
Outline ‘Objective testing’
‘Social Policy Question Resource’ (SPQR) Digital audio for assessment feedback
‘Sounds Good’ With each:
What to use technology for How to do it Limitations and obstacles
Objective testing “Objective tests require a user to choose
or provide a response to a question whose answer is pre-determined … the marking of the responses is completely non-subjective because no judgement has to be made on the correctness or otherwise of an answer at the time of marking. … An objective test is only as objective as the test designer makes it.” (Bull & McKenna 2004)
Some question types
multiple-choice (MCQ) multiple response text match response/fill in the blank true/false, yes/no Likert scale numeric essay
Objective testing: objections?
Well?
MCQ: easy?
MCQ: feedback
Feedback: recommendations statement on whether chosen
option correct or not (if incorrect) statement giving
correct answer and why correct clear up possible misunderstanding
about wrong choice source of info on correct answer language clear and not discouraging
Bloom’s Taxonomy (1956)
knowledge comprehension application analysis synthesis evaluation
Knowledge questions What word means the same as…? What is the most important difference
between…? Which one of the following sequences
shows the correct order of…? What are the major classifications of…? Which method is the most useful for…? What evidence best supports the theory
of…?
Comprehension questions
Which one of the following is closest in meaning to the term…?
The statement ‘…’ means that … [Various facts are presented.]
Which of the reasons listed below best explains this?
Objective testing: some uses Summative
assessment Diagnostic tests Formative tests Informal quizzes Revision aid Subject intro
Individual questions v. flexible, ‘granular’, ‘learning objects’: insert into lectures gather further info
via web links preparation for
classes …
Objective testing:limitations, obstacles
Other assessment also needed Up-front effort required Question-checking Maintenance, shelf-life Collaboration Data portability
Writing questions
Bull & McKenna, (2004), Blueprint for Computer-Assisted Assessment
PASS-IT (2004), Good Practice Guide in Question and Test Design
In your discipline?
Use? Potential/advantages? Limitations/disadvantages?
Sounds Good ‘Quicker, better assessment using
audio feedback’ Coursework:
formative, summative, individual, group Leeds Met (2008) Leeds Met, Newman, Northampton,
York St John (2008-9) [Subject centres: Engineering, GEES]
Sounds Good: some numbers
£35k + £15k (JISC) 38 lecturers 1200+ students Cohort sizes: 3 to 151 Levels: 1 to doctoral 37 presentations!
Marian Tuck’s coursework
Book review length, tips assessment criteria
Read it Audio feedback
listen (as Marian Tuck) how did it feel? other comments?
Tools, techniques
Digital audio feedback MP3 recorders ‘Audacity’ software ‘WIMBA’ voice tools
Audio files via: Email VLE
Early comments
(Student) “Very helpful. It felt like the tutor was able to expand more… . Often when you read feedback, things can get misunderstood or meant in a different way. I felt this way was very clear.”
(Staff) “I think with practice this will get quicker as I get more used to things.”
More staff opinions Staff like audio feedback:
Quality, quantity… “I was able to give … more detailed
and pertinent feedback … [It] became almost an online tutorial.”
“An ideal medium to assist in the development of skills and confidence of students.”
Student opinions
Students like audio feedback! personal, detail, careful consideration
“Very helpful. It felt like the tutor was able to expand more… . Often when you read feedback, things can get misunderstood or meant in a different way. I felt this way was very clear.”
On the other hand…
Some students prefer written feedback Should staff oblige?
Effort to produce, student ‘skimming’…
Some want audio + written Whether/how to do it?
Audio feedback:Practice guidelines Handheld recorder more convenient?
record direct to MP3; USB port Keep files short (<5 mins?) ‘Good enough’ sound quality (32kbps
mono?) Get approval for audio use
See www.soundsgood.org.uk
Time saving? Don’t expect to save time
immediately Time to send audio files
Problem if many students Best chance of saving staff time if:
give lots of feedback write slowly but record speech quickly comfortable with technology easy to send audio feedback
Is audio feedback worth it?
Experienced practitioner Better, but may not be quicker Give an extended trial Worth it for some assessment, esp.
if detail, personal touch valuable to groups
Other uses of audio!
Technology to the rescue?
Not a magic bullet Use appropriately
In favourable circumstances Make informed choices on methods
Technology can bite! “To err is human, but to really screw
things up requires a computer.” (Anon)
Sources (1)
Bull J & McKenna C (2004), Blueprint for Computer-Assisted Assessment, RoutledgeFalmer, London
PASS-IT (2004), Good Practice Guide in Question and Test Design, http://staffnet.kingston.ac.uk/~ku36708/RRR/docs/goodpracticeguide.pdf
Sources (2)
Rotheram B, (2009) Practice tips on using digital audio for assessment feedback, http://sites.google.com/site/soundsgooduk/downloads/Audio_feedback_tips_3.pdf