Outcomes-based Teaching and LearningOutcomes-based Teaching and LearningPre-workshop 3Pre-workshop 3

Designing Designing Assessment Tasks and Grading CriteriaAssessment Tasks and Grading Criteria

Intended Outcomes of Pre-workshop ExercisesIntended Outcomes of Pre-workshop Exercisesfor members of each groupfor members of each group

1. 1. Review Review existing assessment tasks (ATs) in relation to existing assessment tasks (ATs) in relation to alignment with subject ILOs. alignment with subject ILOs.

2. 2. IdentifyIdentify areas requiring changes. areas requiring changes.3. 3. ReviseRevise or or re-design re-design ATs to align to subject ILOs.ATs to align to subject ILOs.4.4. DiscussDiscuss and and shareshare your ideas with your group your ideas with your group

members. members.

Before the WorkshopBefore the Workshop

a. design assessment tasks to address the revised a. design assessment tasks to address the revised subject IlOs. subject IlOs. Consider if the ATs require the students to engege in Consider if the ATs require the students to engege in

the ILO verbs? Do the weightings reflect the relative the ILO verbs? Do the weightings reflect the relative importance of the ILOs?importance of the ILOs?

b. design grading criteria for either the ILOs or theb. design grading criteria for either the ILOs or the assessment tasks.assessment tasks.

Discuss with your group members when doing the above Discuss with your group members when doing the above exercises. exercises.

Complete the exercises to be ready for discussion and sharing Complete the exercises to be ready for discussion and sharing in the workshops on 2 December 2006. in the workshops on 2 December 2006.

Refer to the Pre-workshop reference provided and the Refer to the Pre-workshop reference provided and the revised subject ILOs in Pre-workshop Exercise 1, conduct revised subject ILOs in Pre-workshop Exercise 1, conduct Pre-workshop Exercises 4, 5 & 6 to:Pre-workshop Exercises 4, 5 & 6 to:

Outcomes-based Teaching and Learning (OBTL) Outcomes-based Teaching and Learning (OBTL) using Constructive Alignmentusing Constructive Alignment

ILO:ILO: What the What the studentstudent has to learnhas to learn

Teaching:Teaching:Engaging the Engaging the student in the student in the verb in the ILOverb in the ILO

Assessment:Assessment:How wellHow wellthe student the student has met the has met the ILOILO

Designing Assessment Tasks (ATs)Designing Assessment Tasks (ATs)

Steps:Steps:

1.1. Selecting a practicable task that embodies the target Selecting a practicable task that embodies the target ILO verb.ILO verb.

2.2. Making a judgment on how well the ILO has been Making a judgment on how well the ILO has been met by the students' performance on the ATs – met by the students' performance on the ATs – developing grading criteria.developing grading criteria.

Assessment Tasks (ATs)Assessment Tasks (ATs)

.. Provide students the opportunity to demonstrate whether or Provide students the opportunity to demonstrate whether or not they have achieved the ILOs and what level their not they have achieved the ILOs and what level their

performance is in those ILOs. performance is in those ILOs.

.. Provide the evidence allowing teachers to make a judgment Provide the evidence allowing teachers to make a judgment about the level of a student’s performance against the ILOs about the level of a student’s performance against the ILOs and to awardand to award a final grade. a final grade.

.. Should be appropriately designed or selected to address the Should be appropriately designed or selected to address the ILOs that we want to assess.ILOs that we want to assess.

.. Different assessment tasks address different ILOs.Different assessment tasks address different ILOs.There would therefore be different kinds of assessment task.There would therefore be different kinds of assessment task.

A range of different assessment tasks may be required to A range of different assessment tasks may be required to address the range of ILOs of a subject.address the range of ILOs of a subject.

Questions to be asked in selecting assessment tasks:Questions to be asked in selecting assessment tasks:

1. Are the assessment tasks able to address all ILOs?1. Are the assessment tasks able to address all ILOs?2. Are the assessment tasks aligned to their appropriate 2. Are the assessment tasks aligned to their appropriate ILOs? Are the students required to engage in the verbs ILOs? Are the students required to engage in the verbs identified in the ILOs? identified in the ILOs?3. Are the assessment tasks practicable with respect to 3. Are the assessment tasks practicable with respect to available time and resources? available time and resources?4. Do the assessment tasks reflect the relative importance of 4. Do the assessment tasks reflect the relative importance of the subject ILOs? the subject ILOs?5. Is the assessment workload realistic for teachers and 5. Is the assessment workload realistic for teachers and

students? students?

Selection of Assessment TasksSelection of Assessment Tasks

Constructive Alignment ofConstructive Alignment ofILOs and Assessment TasksILOs and Assessment Tasks

Subject ILO 5Subject ILO 5

Subject ILO 4Subject ILO 4

Subject ILO 3Subject ILO 3

Subject ILO 2Subject ILO 2

Subject ILO 1Subject ILO 1

AT 3AT 3

AT 2AT 2

AT 1AT 1

Assessment Tasks

Subject ILOs

Consider if : all ILOs are being addressed?Consider if : all ILOs are being addressed? there is a balanced coverage of the ILOs? there is a balanced coverage of the ILOs? the more important ILOs are given appropriate assessment emphasis.the more important ILOs are given appropriate assessment emphasis.

Common ILOs Common ILOs Possible Assessment TasksPossible Assessment Tasks

DescribeDescribe assignment, essay question examassignment, essay question examExplainExplain assignment, essay question exam, assignment, essay question exam,

oral, letter-to-a-friendoral, letter-to-a-friendIntegrateIntegrate project, assignmentproject, assignmentAnalyseAnalyse case study, assignmentcase study, assignmentApplyApply project, case study, experimentproject, case study, experimentSolve problemSolve problem case study, project, experimentcase study, project, experimentDesign, createDesign, create project, experimentproject, experimentReflectReflect reflective diary, portfolio,reflective diary, portfolio,

self-assessmentself-assessmentCommunicateCommunicate a range of oral, writing or a range of oral, writing or

listening tasks addressing the listening tasks addressing the ILOs, e.g. presentation, debate, ILOs, e.g. presentation, debate,

role play, reporting, assignment, role play, reporting, assignment, precis, paraphasing, answering precis, paraphasing, answering

questions etc.questions etc.

Assessment tasks for the sample subject ILOs Assessment tasks for the sample subject ILOs 1. 1. Explain different communicative strategiesExplain different communicative strategies..

AT: Write an assignment or answer exam questions explaining the four AT: Write an assignment or answer exam questions explaining the four most important strategies used in report writing. most important strategies used in report writing.

2. 2. Apply appropriate strategies in different modes of communication.Apply appropriate strategies in different modes of communication.

AT: Write a report on a tutorial case study discussion on a selected topic. AT: Write a report on a tutorial case study discussion on a selected topic.

3. 3. Reflect and improve own communicative strategies.Reflect and improve own communicative strategies.

AT: Self-assessment of the report, identifying strengths and areas needing AT: Self-assessment of the report, identifying strengths and areas needing improvement. Revise the report based on self-assessment. improvement. Revise the report based on self-assessment.

The three ILOs could be assessed through one assessment task – writing a report The three ILOs could be assessed through one assessment task – writing a report on a tutorial case study discussion followed by a self-assessment and revised on a tutorial case study discussion followed by a self-assessment and revised report.report.

a.a. Write a report on a tutorial group discussion; Write a report on a tutorial group discussion;

b.b. Explain the strategies used in writing the report;Explain the strategies used in writing the report;

c.c. Self-assess the report, identifying strengths and strategies to be improved.Self-assess the report, identifying strengths and strategies to be improved.

d.d. Revise the report based on c. and submit the revised report together withRevise the report based on c. and submit the revised report together with the first version.the first version.

Examination is a very commonly used assessment task Examination is a very commonly used assessment task especially for large classes. We need to consider ifespecially for large classes. We need to consider if

1.1. examinations involving answering essay type of questions examinations involving answering essay type of questions under invigilated conditions is able to assess students' under invigilated conditions is able to assess students' performance in some high level ILOs, e.g. apply, reflect, create performance in some high level ILOs, e.g. apply, reflect, create etc.etc.

2.2. there are other alternative assessments tasks which will more there are other alternative assessments tasks which will more appropriately addess those high level ILOs.appropriately addess those high level ILOs.

If we must have an invigilated “final exam”, there are far better If we must have an invigilated “final exam”, there are far better formats than the conventional written essays; for example formats than the conventional written essays; for example gobbets, critical incidents (e.g. comment on a video segement), gobbets, critical incidents (e.g. comment on a video segement), letter-to-a-friend, and so on.letter-to-a-friend, and so on.

If we must have exams, consider if the weighting (usually 50% or If we must have exams, consider if the weighting (usually 50% or more) is appropriate to reflect the relative importance of the more) is appropriate to reflect the relative importance of the ILOs being addressed.ILOs being addressed.

Exercise 4 - Designing Assessment TasksExercise 4 - Designing Assessment TasksConsider the intended learning outcomes of your subject in Consider the intended learning outcomes of your subject in

Exercise 1, suggest appropriate assessment task(s) to Exercise 1, suggest appropriate assessment task(s) to address address these ILOs. For each of the tasks, explain what the students these ILOs. For each of the tasks, explain what the students are are expected to do in completing the task. Are the students expected to do in completing the task. Are the students required required to enact the target ILO verbs? Discuss with your to enact the target ILO verbs? Discuss with your group group members.members.

Subject ILOsSubject ILOs Assessment Tasks Student activities in Assessment Tasks Student activities in completing the task completing the task

After deciding on the assessment tasks, we have to After deciding on the assessment tasks, we have to consider the following issues.consider the following issues.

1.1. Specification of the tasks Specification of the tasks Especially when continuous assessment is used, the different Especially when continuous assessment is used, the different assessment tasks making up the continuous assessment will assessment tasks making up the continuous assessment will

need to specified. need to specified.

2.2. Assessment weightingsAssessment weightingsIf assessment tasks are given different assessment weightings, If assessment tasks are given different assessment weightings, ensure that these weightings relfect the relative importance of ensure that these weightings relfect the relative importance of the ILOs. Take the four sample subject ILOs – explain, apply the ILOs. Take the four sample subject ILOs – explain, apply reflect and improve. Are they of equal reflect and improve. Are they of equal importance? If not, importance? If not, then the more important ILOs should be given heavierthen the more important ILOs should be given heavier assessment weighting.assessment weighting.

Exercise 5 – Assessment WeightingExercise 5 – Assessment Weighting

Go back to the ATs designed in Exercise 4, Go back to the ATs designed in Exercise 4, now consider now consider assessment weightings to ensure that they reflect the relative assessment weightings to ensure that they reflect the relative importance of the ILOs. Discuss with your importance of the ILOs. Discuss with your group members.group members.

Subject ILOs Assessment Tasks Student activities in Wt Subject ILOs Assessment Tasks Student activities in Wt

completing the task completing the task

Does the weighting of the ATs reflect the relative importance of the Does the weighting of the ATs reflect the relative importance of the ILOs they address?ILOs they address?

Student performance in assessment tasks should Student performance in assessment tasks should be judged against a set of clear grading criteria be judged against a set of clear grading criteria defining the quality of performance expected of defining the quality of performance expected of each of the grades.each of the grades.

Grading Criteria Grading Criteria

Grading ILOsGrading ILOs

Direct Direct Indirect Indirect(Grading the ILOs)(Grading the ILOs) (Grading the assessment tasks (Grading the assessment tasks

which are aligned with the ILOs)which are aligned with the ILOs)

Grading criteria (rubrics)Grading criteria (rubrics) Grading criteria (rubrics) Grading criteria (rubrics) of individual ILOsof individual ILOs of individual assessment tasks of individual assessment tasks

Derive final gradeDerive final grade Derive final grade Derive final grade

Example of Direct Grading Criteria of ILOsExample of Direct Grading Criteria of ILOs Marginal PassMarginal Pass Satisfactory Satisfactory Good Good Excellent Excellent DD C- C C+ C- C C+ B- B B+ A- A B- B B+ A- A GradeGrade 1.00 1.00 1.70 2.00 2.30 1.70 2.00 2.30 2.70 3.00 3.30 3.70 4.00 2.70 3.00 3.30 3.70 4.00 point/unitpoint/unit

ILOsILOs ExplainExplain Able to identify and brieflyAble to identify and briefly Able to identify a numberAble to identify a number Able to identify a fullAble to identify a full As in “Good” but As in “Good” but

write about limited points.write about limited points. relevant points with some relevant points with some range of relevant range of relevant provides views on provides views onVery little evidence of Very little evidence of details. Uses these points details. Uses these points Points with details. Points with details. possible alternative possible alternativeusing these points to using these points to to provide a fair reasoning to provide a fair reasoning Supported by relevant Supported by relevant causes and/or results causes and/or resultsprovide reasoning to provide reasoning to or causality. No evidence or causality. No evidence literature. Points are literature. Points are under under

changing changing why they are inter- why they are inter- of a comprehensive of a comprehensive organized to provide a organized to provide a conditions. conditions. Able to Able to

related. related. overview of reasoning overview of reasoning comprehensive and comprehensive and link current link current or causality.or causality. cohesive reasoning or cohesive reasoning or reasoning to reasoning to

causality. causality. situations in real- situations in real- life professional life professional

contexts.contexts.

ReflectReflect Able to use Able to use availableavailable Able to use available Able to use available Able to use available Able to use available As in “Good”. Able As in “Good”. Able information to self- information to self- information to self- information to self- information to self- information to self- to generalize self-to generalize self-

evaluate and identify evaluate and identify evaluate and identify evaluate and identify evaluate and identify evaluate and identify evaluation to beyond evaluation to beyond limited aspects of own more aspects of own limited aspects of own more aspects of own the full range of owm the full range of owm existing context. existing context. strengths and weaknesses strengths and weaknesses strengths and weaknesses strengths and weaknesses strengthes and weak- strengthes and weak- Suggest ways of Suggest ways of in a general sense. No in a general sense. Little in a general sense. No in a general sense. Little nesses. Self-evaluation nesses. Self-evaluation improving perform-improving perform- evidence of suggestions application of theory in evidence of suggestions application of theory in is based on theory. is based on theory. ance to real-life ance to real-life of ways to improve self-evaluation and limited of ways to improve self-evaluation and limited Increasingly able to Increasingly able to professional professional performance. No evidence suggestions of ways to performance. No evidence suggestions of ways to suggest ways tosuggest ways to contest. contest. of theory being used in improve performance. of theory being used in improve performance. improve performance improve performance

self-evaluation. self-evaluation. in a specific context. in a specific context.

Example of Indirect Grading Criteria of Assessment Tasks Example of Indirect Grading Criteria of Assessment Tasks in a Portfolioin a Portfolio

Marginal PassMarginal Pass Satisfactory Satisfactory Good Good Excellent Excellent DD C- C C+ C- C C+ B- B B+ A- A B- B B+ A- A

1.001.00 1.70 2.00 2.30 1.70 2.00 2.30 2.70 3.00 3.30 3.70 4.00 2.70 3.00 3.30 3.70 4.00

The pieces of evidence The pieces of evidence The evidence is relevant, The evidence is relevant, The evidence presents a As in “B” but with The evidence presents a As in “B” but with are relevant and are relevant and accurate and covers accurate and covers good appreciation of higher degree of good appreciation of higher degree of accurate, but are accurate, but are several aspects of the several aspects of the the general thrust of the originality and the general thrust of the originality and isolated, addressing isolated, addressing subject. Little evidence of subject. Little evidence of subject. Good coverage evidence of inter- subject. Good coverage evidence of inter- one aspect of the one aspect of the an overall view of the an overall view of the with relevant and nalization into with relevant and nalization into subject. Demonstration subject. Demonstration subject. Demonstrates accurate support. A clear personalized model subject. Demonstrates accurate support. A clear personalized model of understanding in a of understanding in a declarative understanding view of how various of practice. Gooddeclarative understanding view of how various of practice. Good minimally acceptable minimally acceptable of a reasonable amount ofof a reasonable amount of aspects of the subject evidence of reflect- aspects of the subject evidence of reflect- way. Poor coverage, no way. Poor coverage, no content. Able to discuss content. Able to discuss integrate to form a ion on own integrate to form a ion on own originality, weak originality, weak content meaningfully. thrust or purpose. performance basedcontent meaningfully. thrust or purpose. performance based justification of portfolio Good coverage but little Good evidence of on theory.justification of portfolio Good coverage but little Good evidence of on theory. items. Inappropriate items. Inappropriate Application or integration. application of subject Generalizes subjectApplication or integration. application of subject Generalizes subject self-evaluation.self-evaluation. Fair justification of items.Fair justification of items. Content to practice. content to new and Content to practice. content to new and

Attempted realistic self- Portfolio items well unfamiliar real- Attempted realistic self- Portfolio items well unfamiliar real- evaluation evaluation justified. Realistic justified. Realistic life contexts. life contexts.

self-evaluation. self-evaluation.

Departments need to decide whether to use Departments need to decide whether to use direct or indirect grading of the ILOs.direct or indirect grading of the ILOs.

A holistic way of deriving a final grade A holistic way of deriving a final grade from multiple grades from multiple grades

ILOsILOs AT1AT1 AT2AT2 AT3AT3 AT4AT4 GradeGrade

1. Relate1. Relate √√ √√ A – if all ILOs well A – if all ILOs well achieved but 1 & 2 achieved but 1 & 2

2. Apply2. Apply √√ √√ excellent excellent

3. Identify3. Identify √√ B – if all are well achieved B – if all are well achieved

4. Explain4. Explain √√ C – if 3, 4 & 5 are well C – if 3, 4 & 5 are well achievedachieved

5. Communicate5. Communicate √√ D – if marginal or only D – if marginal or only 2 ILOs are achieved 2 ILOs are achieved

An example of a holistic way of deriving final gradeAn example of a holistic way of deriving final grade

Curriculum and Instruction: A subject in a course for Ed. Psychlsts. Curriculum and Instruction: A subject in a course for Ed. Psychlsts.

Grading will beGrading will be based on your attaining the following ILOsbased on your attaining the following ILOs

1. 1. Apply the principles of good teaching and assessment to chosen contexts.Apply the principles of good teaching and assessment to chosen contexts.2. 2. Relate selected aspects of curriculum design and management to the educational Relate selected aspects of curriculum design and management to the educational

system in Hong Kong.system in Hong Kong.3. 3. Apply the content and experiences in this subject to enhance your effectiveness as Apply the content and experiences in this subject to enhance your effectiveness as

an educational psychologist.an educational psychologist.4. Show examples of your reflective decision-making as an educational psychologist.4. Show examples of your reflective decision-making as an educational psychologist.

Final grades will depend on how well you can demonstrate that you have met all the Final grades will depend on how well you can demonstrate that you have met all the ILOs:ILOs:

A Awarded if you have clearly met all the ILOs, provide evidence of original and A Awarded if you have clearly met all the ILOs, provide evidence of original and creative thinking, perhaps going beyond established practice.creative thinking, perhaps going beyond established practice.B Awarded when all ILOs have been met very well and effectively.B Awarded when all ILOs have been met very well and effectively.C Awarded when the ILOs have been addressed satisfactorily, or where the evidence C Awarded when the ILOs have been addressed satisfactorily, or where the evidence

is strong in some ILOs, weaker but acceptable in others.is strong in some ILOs, weaker but acceptable in others.F Less than C, work plagiarised, not submitted.F Less than C, work plagiarised, not submitted.

Example of Holistic Grading of AExample of Holistic Grading of Assessment Tasks ssessment Tasks in a Portfolioin a Portfolio

MarginalMarginal AdequateAdequate Good Good ExcellentExcellent DD C- C C+ C- C C+ B- B B+ A- A B- B B+ A- A

1.00 1.70 2.00 2.30 1.00 1.70 2.00 2.30 2.70 3.00 3.30 3.70 4.00 2.70 3.00 3.30 3.70 4.00

The pieces of evidence The pieces of evidence The evidence is relevant, The evidence presents a As in “B” but with The evidence is relevant, The evidence presents a As in “B” but with are relevant and accurate and covers good appreciation of higher degree of are relevant and accurate and covers good appreciation of higher degree of accurate, but are several aspects of the the general thrust of the originality andaccurate, but are several aspects of the the general thrust of the originality and isolated, addressing course. Little evidence of course. Good coverage evidence of inter-isolated, addressing course. Little evidence of course. Good coverage evidence of inter- one aspect of the an overall view of the with relevant and nalization intoone aspect of the an overall view of the with relevant and nalization into course. Demonstration course. Demonstrates accurate support. A clear personalized model course. Demonstration course. Demonstrates accurate support. A clear personalized model of understanding in a declarative understanding view of how various of practice. Goodof understanding in a declarative understanding view of how various of practice. Good minimally acceptable of a reasonable amount of aspects of the course evidence of reflect-minimally acceptable of a reasonable amount of aspects of the course evidence of reflect- way. Poor coverage, no content. Able to discuss integrate to form a ion on own way. Poor coverage, no content. Able to discuss integrate to form a ion on own originality, weak content meaningfully. Thrust or purpose. Performance basedoriginality, weak content meaningfully. Thrust or purpose. Performance based justification of portfolio Good coverage but little Good evidence of on theory.justification of portfolio Good coverage but little Good evidence of on theory. items. Application or integration. application of course Generalizes courseitems. Application or integration. application of course Generalizes course

Fair justification of items. Content to practice. content to new and Fair justification of items. Content to practice. content to new and Portfolio items well Unfamiliar real-Portfolio items well Unfamiliar real- justified.justified. life contexts. life contexts.

Exercise 6 – Developing Grading CriteriaExercise 6 – Developing Grading Criteria

Consider the intended learning outcomes and assessment Consider the intended learning outcomes and assessment tasks you have identified in Exercise 4, suggest tasks you have identified in Exercise 4, suggest

appropriate appropriate grading criteria addressing either (a) the grading criteria addressing either (a) the

ILOs or (b) the assessment tasks.ILOs or (b) the assessment tasks. Discuss with your group Discuss with your group members.members.

Subject ILO and AT: Subject ILO and AT:

GradeGrade Grading criteria Grading criteria

Help our students understand the grading Help our students understand the grading criteriacriteria

. Remind students of the ILOs to be assessed. . Remind students of the ILOs to be assessed.

. Explain to students the standards expected of each grade.. Explain to students the standards expected of each grade.

. . Illustrate with previous assessment tasks onIllustrate with previous assessment tasks on

How the grading criteria have been applied. How the grading criteria have been applied. How grades have been awarded.How grades have been awarded. How a better grade could have been achieved.How a better grade could have been achieved. . Clarify any problems that students may have.. Clarify any problems that students may have.

After you have completed the pre-workshop After you have completed the pre-workshop exercises, exercises,

1.1. select one group member to share your select one group member to share your ideas at the workshop, and ideas at the workshop, and 2.2. jot down and issues that you would like to jot down and issues that you would like to

bring up for discussion at the workshop. bring up for discussion at the workshop.