Welcome. First Year Assessment ReDesign – A Programme Approach Galvin, Á, O’Neill, G, Noonan,...

Welcome

First Year Assessment ReDesign – A Programme Approach

Galvin, Á, O’Neill, G, Noonan, E, Jennings, D.

University College Dublin

Introduction

• Evaluation of a programme approach to the review and re-design of first year assessment to achieve effective learning and students engagement, while still being efficient to staff.

• Central strand of a wider strategic initiative – Focus on First Year –… “to foster early and lasting student engagement”. (UCD Education Strategy, 2009-2014)

Literature Review Curriculum Design & Programme Mapping• …importance of addressing assessment from a strategic

perspective (Knight, 2000)• …efficiency of the programme approach & importance of

alignment of assessment with programme philosophy, outcomes & GAs, i.e. curriculum mapping (Fink, 2003; Ornstein & Hunkins, 2009; Wiggins & McTighe, 2005)

Curriculum change processes• …clear sense of purpose, strong leadership & local ownership –

all important factors in successful curriculum change initiatives (Blackmore & Kandiko, 2012; Dempster et al, 2012)

• …“discipline-based department teams rather than individuals can be strategic targets for effective change” (Healy et al, 2013: p.42)

The UCD 1st Year Module and Programme Design Principles

Proj

ect P

erso

nnel

:D

ean,

Pro

gram

me

Lead

, Sta

ge

Coor

dina

tors

Proj

ect

Pers

onne

l: Pr

og. L

eade

rs,

Stag

e &

M

odul

e Co

ordi

nato

rs

Workshop A:Articulation of

Programme & Stage Outcomes

Workshop B: Mapping Programme

Outcomes to assessment activities and identifying opportunities and issues

Workshop C: 1st Assessment Redesign against UCD Assessment

Principles

Equivalence document

Specific Assessment

ReDesign workshops

Project Methodology

7

Develop a coherent approach to use of assessment – mapping assessment to programme outcomes at each Stage

1= to indicate students are INTRODUCED to the programme outcome 1 A= ASSESSED (Summative)

2= to indicate the outcome is REINFORCED and the students afforded opportunity to practice 2 Blank= NOT ASSESSED3= indicates that students have had sufficient practice and can now demonstrate MASTERY 3

1 2 3 A 1 2 3 A 1 2 3 A 1 2 3 A 1 2 3 A 1 2 3 A 1 2 3 A 1 2 3 A 1 2 3 A 1 2 3 A

Programme Outcome 1 1 A 1 A 1 1 1 1 1 1 1 1 10.00 0.00 0.00 10 2

Programme Outcome 2 2 0.00 1.00 0.00 2 0

Programme Outcome 3 3 0.00 0.00 1.00 3 0

Programme Outcome 4 0.00 0.00 0.00 0 0









Sem

este

r One

Frequencyof occurance

Sem

este

r Two

Total Introduce

d

Total Reinforce

d

Total Master

y

Wieghted Total

How Often

Assessed?

Module name Module nameModule nameModule nameModule name Module nameModule nameModule nameModule nameModule name

Research Methodology (I)

• Face-to-face interviews with individual Programme Leaders (n=7) – a representative from each of five participating programmes was interviewed

• Focus: – Reflections on the project process– Extent to which project objectives were achieved– Analysis of changes to assessment

Research Methodology (II)

• Online survey circulated to First Year Module Coordinators (n=35)

• 22 MCs completed survey (62% response rate), representing 31 (76%) of modules in the project

• Focus: Extent & nature of planned changes to assessment, aligned with first year design principles

Research Results: Programme Leaders’ PerspectiveMotivation for getting involved• Existing interest/concern/need

• Programme Dean was the main driver

Strengths of the project process• Flexible & adaptable approach – customised to local needs

• The workshops – well structured/outcomes orientated/efficient • The role of T&L facilitators – ‘outside’ perspective & new ideas

Measuring the success of the project• Examples of changes to 1st Year assessment cited, e.g.

Assessment FOR learning; some revision to weighting, timing & quantity BUT query around the extent of changes implemented –

radical revision of Stage 1 had not occurred (yet)• Articulation of programme & stage outcomes – highly valued

• New stage 1 ‘concept’ module introduced

I think the fact that they were adaptable was very important, a real

strength

The mapping exercise was very useful…it highlighted the kinds of demands on

students & ‘bunching’ of assessments

So if there is someone watching from outside it’s always interesting, because first of all they tidy up

what you’re doing and make sense of it, and second of all they point out to you things that are

invisible to you because you are ‘native’.

We knew already there were a couple of modules that we weren’t completely happy with the mode of

assessment, and we have restructured those as a result of

going through assessment redesign.

Change takes time – at least two-three years.

The more we zoned in on the first year modules and what we would do differently it felt like the law of diminishing returns. The

‘big picture’ programme stuff was great, but when it came to ‘just change that bit in that

module’, it was like ‘so what’.

Research Results: Programme Leaders’ Perspective

Research Results: Module Coordinators’ Perspective Assessment Design Principles W* N ** %

Consider the demands of other parallel modules in the stage when planning my assessments

34 14 64%

Reduce assessment work-load for staff, i.e. attention to word-count, reducing number of submissions

33 11 50%

Design a more efficient & effective sequencing of the learning & assessment activities

30 10 45%

Reduce assessment workload for students, i.e. reduce number & size of assessment tasks

30 8 36%

Allow more assessment FOR learning opportunities, i.e. in/out of class activities where the primary focus is to allow students get feedback on their progress

30 12 56%

Introduce more authentic assessment 28 9 41%

Develop multiple opportunities for collaborative learning 25 7 32%

Create more time for introducing learners to key challenging discipline or subject concepts

24 7 32%

W* = Weighting to include the amount (a little, a lot); N** = Number of MCs who applied this principle to their module(s)

Research Results: Module Coordinators’ Perspective

• Strong focus on ‘efficiency’ of assessment– 14 MCs (64%) planned to ‘considering the demands

of other parallel modules when planning their assessments’.

• Assessment FOR learning opportunities planned by 12 MCs (56%)– In-class quizzes, in-class discussion groups, in-class

group work (despite large class sizes)– Online formative MCQs & problem-solving

activities

Discussion & Conclusions• Some areas of consistency, as well different

emphasis, emerged from the two data sets• Positive feedback on curriculum mapping tool• Key success factors of project: – Macro approach to curriculum change involving

wider community of colleagues in collaborative process

– Flexible & adaptable approach– Workshop format was both effective & efficient

Key Recommendations• Programme review process is transferrable to

other strategic projects • Striking the balance between the long-term

impacts of a more strategic approach versus the achievement of immediate module-level changes - local priorities will be key driver

• Further research on the curriculum mapping tool needs to be carried-out & disseminated

…..“learner centred, evidence informed design, developed in the peer-supported environment, is fundamental to successfully embedding new modes of delivery to pedagogic innovations.”

(Dempster et al, 2012, p.135)

Welcome. First Year Assessment ReDesign – A Programme Approach Galvin, Á, O’Neill, G, Noonan,...

Documents

Transcript of Welcome. First Year Assessment ReDesign – A Programme Approach Galvin, Á, O’Neill, G, Noonan,...