Designing effective assessment

Designing and implementing effective assessment

National University of Singapore September 28, 2015

David CarlessUniversity of Hong Kong

The University of Hong Kong

Overview1. Two assessment designs2. Your views of good assessment design 3. Principles of assessment design 4. Challenges & Implications


Competing assessment functions

• Judging student achievement • Satisfying accountability needs

• Stimulating productive student learning


Research process The University of Hong Kong


HISTORY CASE STUDY

Making History course• Foundation level, year 1, 110 students


Making History LOsCritical engagement with representations of

past; interpret connections between past & present;


History Assessment Fieldwork report (30%): Museum visit

Individual project (40%): draft 10%, final 30%

Participation (30%):tutorial participation 15%short weekly written responses 15% (cf. Carless & Zhou, 2015)


Museum report1000 words or podcast

Issues: key messages; use of space; coverage and omissions


Sample Project (1) Relationship between film & history


Sample Project (2) Walking tour of sites of Hong Kong identity


Project stages• Topic brief• Draft (10%)• Final version (30%): 3000 words


Use of FacebookStudents could upload drafts of work in progress & receive peer feedback



LAW CASE STUDY

Tort Law• Core 1st year course: 180+ students


Tort Law LOs

• Explain common torts & their functions• Think critically about legal issues• Analyze tort issues


Tort Law Assessment Reflective Media Diary (20%)

1st sem test (20%) [or test 10% + photo essay 10%]

Final Exam (60%) [or 40% + 20% research essay]


Reflective Media Diary• identify tort-related events in local media;

track developments; provide legal analysis;

• portfolio-like: collecting, selecting, editing and analyzing material over time.


RMD featuresSteady student engagement

Promotes reading habits

Incomplete, authentic problems

Workload friendly for teachers


Photo essayStudents photo tort law situations

Write a short legal analysis


Photo essay features

Student-identified legal issuesCreative …. Iterative

Not easy to grade reliablyMinor option so might be ignored


Use of exemplarsSharing of samples to illustrate expectations


Same-day exam feedback Discussion immediately after exam


Exam debriefing Supplemented by online discussion

Novel angles may be added to marking scheme


Two-stage examsIndividual 80%

Group 20%


Carl Wieman

SHARING

What do you see as the key features of effective assessment task design?


Learning-oriented assessment

A major priority in all assessment should be to promote effective student learning processes (Carless, 2014)



Productive assessment task design

Student self-evaluative capacities

Student engagement with feedback

Learning-oriented assessment framework

TASK DESIGN PRINCIPLES


(1) Integrated with instruction and ILOs


Constructive alignment

(2) Spreading student effort/sustained engagement


(3) Mirroring real-life uses of the discipline


(4) Integrated and coherent


(5) Incorporates feedback dialogues


(6) Supports student in understanding quality


(7) Flexibility & choice


(8) Encourage deep approaches to learning


Programme-based approaches

Cumulative … integrated … coherent


Issues & Challenges The University of Hong Kong

Student responseStudents interpret assessment tasks based on their own preferences, experiences and motivations


Double dutyCompeting priorities in assessment

Reliability, Fairness,Workload, QA etc


Lack of autonomyMany teachers lack individual autonomy: pulled in different directions by competing priorities (James, 2014)


Limited incentivesGood assessment design: difficult but may not be rewarded (Norton et al., 2013)


Teacher X factorTeacher determination to overcome barriers & strive for learning-oriented assessment


Trust in teachers Distrust as barrier for innovative assessment (Carless, 2009)


Assessment literacyNeed for development in assessment (for learning) literacy of university teachers (cf. Price et al., 2012)


THANK YOU


Designing effective assessment

Education

Transcript of Designing effective assessment