e-FeedbackWho’s using it and why?

Simon StarrCanterbury Christ Church University

LTEULearning and Teaching Enhancement Unit

Session Outline

1. Context• what’s the deal with feedback? • e-feedback research

2. Approaches to e-Feedback at Christ Church

Faculty of Business and Management

Department of Geographical and Life Science

Department of Computing

3. Developing University Strategy

• Turnitin GradeMark

4. Questions & Discussion

Context – National Students’ Voice

“Feedback: Should be for learning, not just of learning … Should relate to clear criteria … Should be legible and clear”

“tutors and students have differing impressions about what is expected from assessment and how a student can achieve a high mark”

“Objectives for assessment and grade criteria need to be clearly communicated to, and fully understood by, student. Subsequent feedback should be provided primarily in relation to this.”

NUS Feedback Amnesty Briefing Paperhttp://www.nus.org.uk/en/Campaigns/Higher-Education/Assessment-feedback-/

Context – Local Students’ Voice

“[Arts & Humanities] students were in favour of electronic feedback, yet would like to see it customised for each assignment”

“[Business & Management] students found that they were often unable to read tutor’s [sic] handwriting”

“[Education] students understood that electronic feedback would be easier to read and more structured”

“Students highlighted that feedback would be especially useful if staff used a marking grid to show the students [sic] competence in various areas, 100% of the [Health & Social Care] Reps present agreed that typed and electronic feedback would be preferable”

“[Social & Applied Sciences] students agreed that typed comments were preferable”

Minutes of Student UnionFaculty Council Meeting 4th March 2009

Context – University Policy

“Students have an entitlement to timely and constructive feedback”

“The language of feedback should be accessible, constructive and relate directly to the criteria against which the assessment is conducted”

“The use of statement banks and electronic templates can speed up writing feedback and contribute to the quality of feedback”

Principles for Effective FeedbackAccepted by Academic Board 2009/10

Context – Research at Liverpool John Moores

198 L4 Pharmaceutical Science and Science students (2007)

Feedback template inc. assessment criteria and statement

banks

48 e-feedback (e), 150 traditional (t) feedback

Survey (n=169, e=40, t=129)

Avg. 0.9 Likert point higher for e-feedback compared to

traditional• Amount of feedback I received: e=4.0, t=2.6• Information on where I went wrong: e=3.8, t=2.8• Identified aspects I did well: e=3.9, t=2.7

Markers support further use• 2-3 mins avg. time saving/script = 2 hrs for 40 scripts• Offset against programming criteria/statement banks …

Denton, P., Madden, J., Roberts, M & Rowe, P. (2008) ‘Students’ response to traditional and computer-based formative feedback: A

comparative case study’, British Journal of Educational Technology, 39 (3) pp.486-500

Context – Research at University of Wales

200 L4/5/6 undergrad. criminology essays (2004/5)

Feedback template inc. assessment criteria and statement

banks

Survey (n=95)• 79% more aware of assessment criteria• 69% more “able to identify ways to improve in the future”• 67% more “motivated to improve in the future”

Average 4% improvement in performance per student L1-2

Marking time reduced, student queries reduced

Adopted by department as standard

Case, S. (2007) ‘Reconfiguring and realigning the assessment feedback processes for an undergraduate criminology degree’, Assessment &

Evaluation in Higher Education, 32 (3) pp.285-299

@Christ Church 1: Business & Management

Module: Introduction to Financial Accounting (L4)

Assignment: Financial Report; ~400 students in 113 self-selecting groups

Marking Arrangements: 5 markers feeding back grade, summary- and criteria-related feedback• assessment grid “plugged in”

Technology: Feedback 15 software (LJMU)

e-Feedback Driver: time –vs- amount and quality of

feedback

@Christ Church 1: Business & Management

Module Lead: Nicky Leatherbarrow

@Christ Church 1: Business & Management

Criteria-related feedback …

@Christ Church 1: Business & Management

… enabled using Feedback 15 software

@Christ Church 1: Business & Management

Example Feedback 15 Report

@ Christ Church 2: Geographical & Life Sciences

Module: Organic/Environmental Chemistry ~25 students (L5)

Module: Pollution Science ~15 students (L6)

Assignments: lab reports; research tasks; numerical problems

Marking Arrangements: single marker - criteria-related feedback, summary key points and stock phrases e.g.

• "the axis in the graph should be labelled“• "references should be given in the Harvard format“

Technology: FormFeed (Heriot-Watt) – similar to Feedback 15

e-Feedback Driver: try out new technology - early adopter

@ Christ Church 2: Geographical & Life Sciences

Module Leader: Emilia Bertolo

“I preferred this method to conventional marking: the time is spent more creatively … with almost no time wasted in generating generic feedback again and again.”

“nice to be able to use office to write the feedback, since it catches spelling errors (as a non-native English speaker, that was great)”

“modifying the marking grid to make sure I was not leaving any irrelevant feedback took me longer than what I had expected”

No formal evaluation with students but asked for comments – no negative feedback …

“Informally, a couple of students told me that they liked it, and another mentioned how nice it was that the feedback was typed …”

“One of the external examiners really liked, he thought the feedback was very meaningful … I consider this the greatest success of the experience”

@Christ Church 3: Department of Computing

Modules/Assignments: various

Marking Arrangements: since 2002 feeding back criteria-related marks, stock phrases and overall comment, auto sum mark, e-mail reports to students

Technology: In-house Excel-Word-Outlook e-mail

e-Feedback Driver: R&D interest

@Christ Church 3: Department of Computing

@Christ Church 3: Department of Computing

Module Leader: Dave Bennett

Used by almost all academic staff• most use for simple addition• some made enhancements e.g. automatic comment

based on grade and standard marking grid

No formal evaluation with students but …

“No major issues encountered so far…”

“[students] like getting it by email”

Paper on criteria for developing e-Feedback systems in progress …

Summary

Experience@Christ Church echoes research

Students like:

• clarity – type-written• structure & consistency – criteria driven• “feedforward”• potential for electronic return of feedback?

Tutors like:

• stock phrases to save time• criteria-related feedback to guide students’ learning• potential for electronic return of feedback?

Several technologies available – key is “plugging in” assessment grids; stock phrases; electronic return?

Where Next? Developing University Strategy …

Strategic assessment enhancement 2010/11:

• 3 week turnaround

• institutionally supported e-Submission and e-Feedback systems

Piloting Turnitin GradeMark April/May

• quality of feedback: criteria-related feedback and detailed stock phrases

• ease: web-based, access anywhere; intuitively programme in and share assessment grids

• lesser focus on speed …

• … e-submission; e-return of feedback?

GradeMark Demo

Marking exercises …?

Questions?

Q Are there any opportunities for e-feedback for your

students?

Q What might some of the drawbacks be?

Q Wait a minute, can’t this be done on paper anyway?