Developing assessment patterns that work through TESTA
-
Upload
tansy-jessop -
Category
Education
-
view
32 -
download
0
Transcript of Developing assessment patterns that work through TESTA
Developing assessment patterns that work through TESTA
Professor Tansy JessopTESTA Workshop
University College Dublin10 February 2017
The plan for this session
1. Brief overview of TESTA2. Two case study puzzles3. Assessment patterns which help
students to learn
Mixed methods approach
Programme Team
Meeting
Assessment Experience
Questionnaire(AEQ)
TESTAProgramme
Audit
Student Focus Groups
Growth of TESTAWidespread use
TESTA….
“…is a way of thinking about assessment and feedback”
Graham Gibbs
It enables you to see the whole elephant
Defining the terms
• Summative assessment carries a grade which counts toward the degree classification.
• Formative assessment does not count towards the degree (either pass/fail or a grade), elicits comments and is required to be done by all students.
TESTA Case Study X: what’s going on?
• Plenty of varieties of assessment, no exams• Reasonable amount of formative assessment (14 x)• 33 summative assessments• Masses of written feedback on assignments (15,000 words)• Learning outcomes and criteria clearly specified….looks like a ‘model’ assessment environment
But students:• Don’t put in a lot of effort & distribute their effort across few topics• Don’t think there is a lot of feedback or that it very useful, and
don’t make use of it• Don’t think it is at all clear what the goals and standards are• …are unhappy
Case Study Y: what’s going on?
• 35 summative assessments• No specified formative assessment• Learning outcomes and criteria wordy and woolly• Marking by connoisseurship, tacit, professional judgements….looks like a problematic assessment environment
But students:• Put in a lot of effort and distribute their effort across topics• Have a very clear idea of goals and standards • Are able to evaluate their work and have a good idea of how to
‘close the gap’
Assessment patterns which work
1. Part of principled whole curriculum design 2. Plenty of formative assessment3. Students distribute effort and spend time on task4. Challenging and high expectations5. Feedback connects across modules6. Students interact with and internalise standards
Assessment patterns (n=73 programmes)Characteristic Low Medium High
Volume of summative assessment
Below 33 40-48 More than 48
Volume of formative only Below 1 5-19 More than 19
% of tasks by examinations Below 11% 22-31% More than 31%
Variety of assessment methods
Below 8 11-15 More than 15
Written feedback in words Less than 3,800 6,000-7,600 More than 7,600
Deep and Surface Learning (Marton and Saljo (1976)
Deep Learning• Meaning• Concepts• Active learning• Generating knowledge• Relationship new and
previous knowledge• Real-world learning
Surface Learning• External purpose• Topics• Passive process• Reproducing knowledge• Isolated and
disconnected knowledge• Artificial learning
• High summative on most UK, NZ and Indian degrees (range 12 -227 in UK)
• Pedagogies of control
• Low formative: ratio of 1:8 summative to formative
• Weakly practised and understood
1. Curriculum design question
Does IKEA 101 work for complex learning?
..has led to over-emphasising knowing (Barnett and Coate 2005)
• Knowing is about content• Acting is about becoming
a historian, actor, psychologist, or philosopher
• Being is about understanding yourself, orienting yourself and relating your knowledge and action to the world
Knowing
Being
Acting
The best approach from the student’s perspective is to focus on concepts. I’m sorry to break it to you, but your students are not going to remember 90 per cent – possibly 99 per cent – of what you teach them unless it’s conceptual…. when broad, over-arching connections are made, education occurs. Most details are only a necessary means to that end.
http://www.timeshighereducation.co.uk/features/a-students-lecture-to-rofessors/2013238.fullarticle#.U3orx_f9xWc.twitter
A student’s lecture to her professor
Assessment is mainly sort of the topical knowledge and the topics never relate. We'll never do something again that we’ve already studied, like we learn something and then just move on
(TESTA focus group data).
…and fragmenting content…
2. Plenty of formative but…
“Innovations that include strengthening the practice of formative assessment produce significant and often substantial learning gains”
(Black and Wiliam, 1998, p.40).
Take five
• Which quote strikes you most and why?
• What do students value about formative assessment?
• What are the challenges of doing formative?
It was just a practice and didn’t really matter what we did and we could learn from mistakes so that was quite useful.
He’s such a better essay writer because he’s constantly writing. And we don’t, especially in the first year when we really don’t have anything to do. The amount of times formative assignments could have taken place…
It’s good to know you’re being graded because you take it more seriously.
I would probably work for tasks, but for a lot of people, if it’s not going to count towards your degree, why bother?
Students talking about formative
3 & 4. Effort and challenging tasks
• Chickering and Gamson (1987)
• Time-on-task (Gibbs 2004)
• Arum and Roksa (2011)
The problem is that…
• Summative drives student effort
• ‘Pedagogies of control’
• Too much marking for staff, too little effort by students
• Assess less, learn more
• Measure less, learn more
Typical student effort graph
Almost 50% of students in USA leave with no significant gains in academic writing, complex reasoning and critical thinking skills…
(Arum and Roksa, 2011)
Significant learning gains for students who 1) Read > 40 pages a week of academic writing
2) Write > 20 pages per semester for each unit
From this educational paradigm…
Transmission Model
Social Constructivist Model
ReferencesArum, R and Roksa, J. (2011) Academically adrift: Limited Learning on College Campuses. University of Chicago Press.Boud, D. and Molloy, E. (2013) ‘Rethinking models of feedback for learning: The challenge of design’, Assessment & Evaluation in Higher Education, 38(6), pp. 698–712. doi: 10.1080/02602938.2012.691462.Gibbs, G. & Simpson, C. (2004) Conditions r which assessment supports students' learning. Learning and Teaching in Higher Education. 1(1): 3-31.Harland, T., McLean, A., Wass, R., Miller, E. and Sim, K. N. (2014) ‘An assessment arms race and its fallout: High-stakes grading and the case for slow scholarship’, Assessment & Evaluation in Higher Jessop, T. and Tomas, C. 2016 The implications of programme assessment on student learning. Assessment and Evaluation in Higher Education. Published online 2 August 2016. Jessop, T. and Maleckar, B. (2014). The Influence of disciplinary assessment patterns on student learning: a comparative study. Studies in Higher Education. Published Online 27 August 2014 http://www.tandfonline.com/doi/abs/10.1080/03075079.2014.943170Jessop, T. , El Hakim, Y. and Gibbs, G. (2014) The whole is greater than the sum of its parts: a large-scale study of students’ learning in response to different assessment patterns. Assessment and Evaluation in Higher Education. 39(1) 73-88.Nicol, D. (2010) From monologue to dialogue: improving written feedback processes in mass higher education, Assessment & Evaluation in Higher Education, 35: 5, 501 – 517.O'Donovan, B , Price, M. and Rust, C. (2008) 'Developing student understanding of assessment standards: a nested hierarchy of approaches', Teaching in Higher Education, 13: 2, 205 — 217Sadler, D. R. (1989) ‘Formative assessment and the design of instructional systems’, Instructional Science, 18(2), pp. 119–144. doi: 10.1007/bf00117714.