Assessment – Preparing, Delivering,
Assessing
Damian Gordon
Last Week: Instructional Design
The Joy of Teaching
Then Jesus took his disciples up to the mountain and gathered them around him, he taught them saying, "Blessed are the poor in spirit, for theirs is the kingdom of heaven. Blessed are the meek. Blessed are they that mourn. Blessed are you when you suffer. Be glad and rejoice, for your reward is great in heaven."
Then Simon Peter said, "Are we supposed to know this?" And Andrew said, "Will we have a test on this?" And Philip said, "I don't have any paper." And Bartholomew said, "Do we have to turn this in?" And John said, "The other disciples didn't have to learn this." And Matthew said, "May I go to the bathroom?" Then one of the Pharisees who was present asked to see Jesus' lesson
plan and inquired of Jesus, "Where are your objectives and your required competencies in the cognitive domain?"
And Jesus wept.
Overview Part 1 (Why?)
Purpose of assessment Formative -v- summative assessment
Part 2 (What?) Learning outcomes Constructive alignment
Part 3 (How?) Types of assessment Grading of assessment
Part 4 (Who?) Self / peer assessment
Assessment
How do you get a new belt in Karate, how accurate does your throws, blocks and punches have to be ? 40% 50% 75% 90% 100%
Assessment
How do you get a new belt in Karate, how accurate does your throws, blocks and punches have to be ? 40% 50% 75% 90% 100%
>>>>>>> How should it be assessed? <<<<<<<<
Assessment
To qualify as a doctor, what average result do you need to get to pass ? 40% 50% 75% 90% 100%
Assessment
To qualify as a doctor, what average result do you need to get to pass ? 40% 50% 75% 90% 100%
>>>>>>> How should it be assessed? <<<<<<<<
What is the purpose of assessment ?
John Biggs
Born in Tasmania (5th Generation)
an educational psychologist and novelist
developed the SOLO Taxonomy and the model of constructive alignment
Discusses “Backwash”
Purpose of assessment I
Reasons for assessment Select students Control / motivate students Direct teaching / learning Identify strengths / weaknesses of teaching /
learning
Purpose of assessment II
Backwash Students learn what they think they will be
assessed on Assessment determines what and how students
learn more than the curriculum does This must be capitalised on and employed
Michael Scriven
Born 1928 in Britain Degree in mathematics
and a doctorate in philosophy
made significant contributions in the fields of philosophy and evaluation
Coined the terms “formative evaluation” and “summative evaluation” in 1967
Purpose of assessment III
Types of assessment Formative assessment
Used for feedback throughout the course Should not (all) contribute to marking Useful for monitoring learning progress
Summative assessment End of course Used to grade students
Peter Knight
Born in Britain Died in 2007 Educational research
and assessment expert Author of “Children’s
Understanding of People in the Past”
Author of “Being A Teacher In Higher Education”
Purpose of assessment IV
Reason for assessment Authorities agree that assessment has two main
roles. It warrants or certifies achievement…….It also influences learning, partly by signaling what is really valued…
Peter Knight, SEFI Annual Conference, (2001)
…and one other expert(s) we
should listen to
Purpose of assessment V
Student’s views
1. “Helps me find out what I know and don’t know” (87.3%)2. “Guides my learning” (75.5%)3. “Helps me link theory to practice” (84.3%)4. “Helps lecturers evaluate student knowledge” (81.4%)5. “Enables lecturers to evaluate their teaching” (75.5%)6. “Used by lecturers to rank students” (44.1%)
Formative -v- Summative Assessment
“When the chef tastes the sauce it is formative assessment, when the customer tastes it, it is summative” Robert Stake (possibly)
Formative Assessment Students’ Top 5 reasons for assessment all
formative Formative assessment cannot be separated
from teaching The more effective the teaching method, the
better its ability to provide formative feedback
1. “Helps me find out what I know and don’t know” (87.3%)
2. “Guides my learning” (75.5%)3. “Helps me link theory to practice”
(84.3%)4. “Helps lecturers evaluate student
knowledge” (81.4%)5. “Enables lecturers to evaluate
their teaching” (75.5%)6. “Used by lecturers to rank
students” (44.1%)
Admitting / recognising error is crucial for learning Error is used to instruct, not punish Student should be able to take the formative role on
himself / herself in the future
Summative assessment
To report a grade!
One line answer
Why assess? We assess to help students learn (formative)
and to judge how much they learned (summative)
Overview Part 1 (Why?)
Purpose of assessment Formative -v- summative assessment
Part 2 (What?) Learning outcomes Constructive alignment
Part 3 (How?) Types of assessment Grading of assessment
Part 4 (Who?) Self / peer assessment
Learning Outcomes
Without a clear understanding of the programme goals students are likely to become disorientated and waste time trying to discover what it is we want them to learn
Learning outcomes consist of three parts: the student action the content the standard required to meet the objective
Industry-Oriented Learning Outcomes
Graduates to be equipped with core competencies Who will be able to become productive in a short
time with training Graduates to be equipped with the necessary
skills to adapt to changing needs as appropriate and acquire new skills quickly
Layers of Learning Outcomes
SessionModule
Programme
Constructive Alignment
LEARNING OUTCOMES
TEACHING& LEARNING
ASSESSMENTMETHODS
Constructive Alignment I
Learning outcomes, teaching methods and assessment are all inter-dependant
Only by truly integrating these components together, do we get efficient student learning
Meaning is not imposed or transmitted by direct instruction - it is created by the student's own learning activities
Constructive Alignment II
A. Knowledge and understanding
1. Software Engineering;
2. Problem Solving and Critical Analysis;
3. A range of programming languages and environments;
4. Basic principles of computer hardware, data transmission and local area networking;
5. Operating systems,
6. Business context;
7. Engineering practice;
Teaching/learning methods The knowledge required for the basic topics is
obtained via lectures, exercises, practicals, assignments and project work. Appropriate IT and other software packages are taught.
Practical demonstrators and project supervisors advise students, and feedback is provided on all continually assessed work.
As the course progresses students are expected to show greater initiative.
Assessment Most knowledge is tested through a combination of
practicals, assignments and formal examinations. Students write reports on many assignments, and
also make oral presentations of their work.
Constructive Alignment III
B. Intellectual Skills
1. Select and apply appropriate computer based methods;
2. Analyse and solve problems;
3. Transfer appropriate knowledge and methods from one topic to another;
4. Plan, conduct and write a report on a project or assessment;
5. Prepare an oral presentation.
Teaching/learning methods Appropriate software, mathematical, scientific and IT
skills and tools are taught in lectures, and problems to be solved are given as projects or assignments.
Project planning is part of the Part 4 project, and written and oral presentations are required for various assignments and projects.
Assessment Skills 1-3 are assessed partly by examination, though
sometimes also by project or assignment work. Skills 4 and 5 are assessed as part of project work.
Constructive Alignment IV
C. Practical Skills
1. To design, implement and test well structured and documented computer programs using a wide variety of programming languages, with a focus on object-oriented development,
2. To develop applications using the main web programming technologies;
3. Database manipulation and administration;
4. Research into a computer science problem;
5. Present work in both written and oral form.
Teaching/learning methods Software tools are introduced in lectures
and their use is assessed by examinations and assignments.
Programming assignments are set, and students may write programs to solve other projects.
Practicals and projects are used to teach about skill 3, and projects are used for skills 4, 5, 6 and 7.
Assessment Skills 1 to 4 are tested by assignments and
projects, is addition to coursework. Skills 1, 2 and 3 are assessed in practicals.
Skills 4 and 5 are assessed through project work.
Constructive Alignment V
D. Transferrable Skills
1. use software tools.
2. acquire, manipulate and process data.
3. use creativity and innovation.
4. solve problems.
5. communicate scientific ideas.
6. give oral presentations.
7. work as part of a team.
8. use information resources.
9. manage time.
Teaching/learning methods and strategies Software tools are taught partly in lectures, mainly
through practical sessions and assignments. Data skills are acquired in laboratory and projects. Creativity and innovation and problems solving are
experienced through projects, as are team working, time management and presentations.
Use of information resources, such as the library and IT methods is experienced through projects and assignments.
Assessment Some skills, like the use of software tools and ability
to communicate orally and in written form are directly assessed, in assignments or projects
3D Alignment Horizontal Vertical Constructive
O'Leary, C., Lawless, D., Gordon, D., Carroll, D., Mtenzi, F., and Collins, M., “3D Alignment in the Adaptive Software Engineering Curriculum” 36th Annual Frontiers in Education Conference, San Diego, USA, October 28th - 30th 2006
we recall Bloom’s Taxonomy
Blooms Taxonomy I
6 Levels (Cognitive Domains) Knowledge Comprehension Application Analysis Synthesis Evaluation
Blooms Taxonomy II
Knowledge list name identify show define recognize recall state
Year 1
Blooms Taxonomy III Comprehension
summarize explain put into your own words interpret describe compare paraphrase differentiate demonstrate visualize find more information about restate
Years 1, 2
Blooms Taxonomy IV Application
solve illustrate calculate use interpret relate manipulate apply classify modify put into practice
Years 2, 3
Blooms Taxonomy V
Analysis analyze organize deduce choose contrast compare distinguish
Year 3
Blooms Taxonomy VI Synthesis
design hypothesize support schematize write report discuss plan devise compare create construct
Years 3, 4
Blooms Taxonomy VII
Evaluation Evaluate Choose Estimate Judge Defend Criticize Justify
Year 4
Types of Knowledge
Types of knowledge I
Declarative
Procedural
Conditional
Functioning
Types of knowledge II
Declarative Knowing something Facts – recorded in books / delivered in lectures Accrues from research, not from personal
experience Verifiable, usable
Types of knowledge III
Procedural Skill based knowledge Sequences of actions Having the right competencies
Types of knowledge IV
Conditional Subsumes declarative and procedural Knowing when to do something, what to do, why
to do it… Turns procedural knowledge into functioning
knowledge
Types of knowledge V
Functional Performance of understanding Knowledge within the experience of the learner
who can put declarative knowledge to work by solving problems
Types of knowledge VI
Curricula at third level generally focus on declarative, something reflected in forms of assessment
What is required for professional competence is functioning knowledge
Assessment should reflect this.
SOLO Taxonomy
Levels of understanding I
SOLO Taxonomy Structure of Observable
Learning Outcome
PrestructuralUnistructuralMultistructural Relational
Extended
abstract
Quantitative Qualitative
Levels of understanding II
Prestructural understanding Learner misses the point No observable understanding
Levels of understanding III
Unistructural understanding Simple naming Terminology Focus on one conceptual issue in a complex task Low level understanding
Levels of understanding IV
Multistructural understanding Disorganised collection of items Knowledge telling
Levels of understanding V
Relational understanding Using a concept that integrates a collection of
data Understanding how to apply the concept to a
familiar data set or to a problem
Levels of understanding VI
Extended abstract Relating to existing principle so that unseen
problems can be handled Questioning and going beyond existing principles
Assessment Criteria
Assessment criteria I
State the Type of knowledge Level of understanding
required to meet a particular learning outcome
Assessment criteria II
“the standard required to meet the outcome” Criteria are developed by analyzing the learning
outcomes and identifying the specific characteristics that contribute to the overall assessment
How many criteria are needed that contain neither irrelevant or miss important areas?
Assessment criteria III
Criteria serve two functions For formative feedback, learners can
See how they performed on the various components of the assessment
Identify strengths and weaknesses For summative feedback,
A contract is in place between teacher and learner
Assessment criteria IV
Questions to answer when setting an assessment What learning outcomes do you intend to assess? How does your approach to assessment align with these
outcomes? What are the assessment criteria? What other skills are assumed e.g. skills from previous
years? How do you ensure that your standards are consistent?
Formative Feedback I Feedback helps students to improve and prevent them from
making the same mistakes again. It is useless if feedback comes back too late in the semester Students sometimes wait on the feedback before they feel
capable of going onto the next bit Grades do not equal feedback. Examinations are usually a summative form of assessment and
therefore not used for feedback.
Formative Feedback II
For students, high quality feedback consists of following three things:
1. A clear criteria against which to judge the comments.
2. Comments that are detailed and related to specific aspects of their work.
3. Comments that are improvement focussed.
Formative Feedback III
Students’ views
Agree Disagree Don’t know
Check understanding of course material
81.4% 12.7% 5.9%
Help students improve their work
99.0% 1.0% 0.0%
To explain the grade 83.3% 14.7% 2.0%
Formative Feedback IV
75.5% agree that they compare the grade received to the feedback
Does the feedback reflect the grade given? “I feel lecturers should consult students about their grades
not just tell them the grade. Recently I got some assignments back and the comments were ‘excellent, good conclusion, good reflections etc.’ Nothing bad was written on it, but I only received 50%. I can’t understand why it was so bad if all the comments were so positive.”
Formative Feedback V
Nearly a quarter of the students surveyed indicated that they would not approach the lecturer for clarification of feedback: “I think it would be helpful if lecturers took specific time to
speak with all their students to discuss exam results etc. At present lecturers leave it up to the student to come forward. This doesn’t work because students who haven’t done well are less likely to come forward to speak to their lecturer.”
Formative Feedback VI
1. Feedback should be provided by the assessor to the assessed student
2. High level feedback to the class can be useful
3. Feedback should be correlated to the assessment criteria
4. Asking the student to self-assess against criteria can prove useful
One line answer
What to assess? We assess all learning outcomes so that we can
guarantee that a student who passes a module will know (declarative) or be able to do (functioning) everything listed in the outcomes to a certain level.
Overview Part 1 (Why?)
Purpose of assessment Formative -v- summative assessment
Part 2 (What?) Learning outcomes Constructive alignment
Part 3 (How?) Types of assessment Grading of assessment
Part 4 (Who?) Self / peer assessment
General approaches to assessment I
Norm referenced assessment (NRA) Judgements about people Measurement model
Criterion referenced assessment (CRA) Judgements about performance Standards model
General approaches to assessment II
Measurement model Grading on a curve Marking by counting bottom up and averaging Assessment separated from teaching Multistructural, not relational or extended abstract thinking Verbatim responses gain marks Easily fit into most structures
General approaches to assessment III
Standards model Standards or criteria set as objectives Most students should be able to reach standards at an
acceptable level Different performances can reflect the same standards Performances can be judged against the criteria Holistic assessment – functional and declarative
knowledge working together
Types of Assessment
Examinations Normally unseen by students prior to the day they
are attempted Sample a range of student learning and test the
student’s own work Encourage students to memorise information
rather than attempting to understand it as a component of their overall course
Types of Assessment
Take home assignment Students given a period of time ranging from
weeks to months Gives time to reinforce theory delivered in class Allows students develop own model, through trial
and error Requires students to work without assistance for
the majority of the time
Types of Assessment
Laboratory Work Immediate feedback Can observe process Identify strengths and weaknesses
Types of Assessment
Class Presentations Opportunity to assess on both structure and
content The criteria should be made clear to the students Communication skills are key transferrable skills
Types of Assessment
Essays / Research Assess, Analyse, Evaluate ......... Two approaches to grading
Analytic Holistic
Types of Assessment
Online Tests Multiple choice questions Complete objectivity in marking the test Largely used to test factual material and the understanding
of concepts Used for testing large groups Time consuming to set No credit for partial information May encourage guessing
Types of Assessment
Problem Based Learning An instructional method that challenges students to learn to
learn Work cooperatively in groups to seek solutions to real
world problems Used to engage students' curiosity and initiate learning the
subject matter Prepares students to think critically and analytically, and to
find and use appropriate learning resources
Types of Assessment
Reflective Journal Develop an awareness of processes Write in your reflective journal as soon as
possible after an event
Assessment for Learning
Assessment for Learning Students know at the outset of a unit of study what
they are expected to learn. 1. At the beginning of the unit, the teacher will work with the
student to understand what she or he already knows about the topic as well as to identify any gaps or misconceptions (initial/diagnostic assessment).
2. As the unit progresses, the teacher and student work together to assess the student’s knowledge, what she or he needs to learn to improve and extend this knowledge, and how the student can best get to that point (formative assessment).
Assessment for learning occurs at all stages of the learning process.
Assessment for Learning
According to The UK Assessment Reform Group (1999) improving learning through assessment depends of five factors:
1. The provision of effective feedback to the students.2. The active involvement of students in their own learning.3. Adjusting teaching to take account of the results of
assessment.4. A recognition of the profound influence assessment has
on the motivation and self esteem of the students, both of which are crucial influences in learning.
5. The need for students to be able to assess themselves and understand how to improve.
Assessment for Learning
• How do we do this ?
• Sharing learning goals with students.
• Involving students in self assessment.
• Providing feedback which leads to students recognizing and taking the next steps.
• Being confident that every student can improve.
Assessment for Learning
• What barriers are there ?
• A tendency for teachers to assess quantity and presentation of work rather than quality of learning.
• Greater attention given to marking and grading, much of it tending to lower self esteem of students, rather than providing advice for improvement.
• A strong emphasis on comparing students with each other, which demoralizes the less successful learners.
Primary UsersAssessment of
Learning Policy makers Program planners Supervisors
Assessment for
Learning Students Teachers Parents
Typical UsesAssessment of
Learning Certify competence Sort students
according to achievement
Assessment for
Learning Help students see
and hit the target Help teachers
identify and respond to student needs
Teacher’s RoleAssessment of
Learning Follow test administration
procedures Use results to help
students reach standards
Assessment for
Learning Inform students of
targets Build assessments Adjust instruction Involve students in
assessment
Student’s RoleAssessment of
Learning Study to meet standards Take the test Strive for the highest
score possible Avoid failure
Assessment for
Learning Strive to understand the
target Act on classroom
assessment to improve Encourage success
The Assessment Goal
To create and maintain a balanced assessment system that includes high quality assessments of and for student learning.
Question
Assessment of learning = Summative Learning ??
Assessment for learning = Formative Learning ??
Question
Teacher-centred = Summative Learning ??
Student-centered = Formative Learning ??
Student Centred Assessment
Multi-choice Questions As we discussed previously from a teacher-
centred perspective multi-choice questions have little to recommend them, but from a student-centred point-of-view multi-choice questions are telling the student “Here are N questions and I expect you to know the answers to all of them, if you find yourself guessing a lot of answers, you really don’t know the material as well as I want you to.”
Overview Part 1 (Why?)
Purpose of assessment Formative -v- summative assessment
Part 2 (What?) Learning outcomes Constructive alignment
Part 3 (How?) Types of assessment Grading of assessment
Part 4 (Who?) Self / peer assessment
Involving Students in Assessment I Ultimately the goal of any university course is for students to be
able to make judgments on their own and others work The more time students spend on higher-level abilities like
analysing and evaluating, the better they will get at assessment One approach to getting students involved in their assessment
by having them develop their own assignment questions If students are concerned with the subject’s assessment they will
be encouraged to engage in their subjects in deeper and more meaningful ways
Involving Students in Assessment II
Peer assessment Students comment on and judge their colleagues
work Vital role to play in formative assessment Can also be used as a component in a summative
assessment package studies have found student ratings of their colleagues
to be both reliable and valid
Involving Students in Assessment III
Peer assessment – advantages1. Helps students to become more autonomous, responsible
and involved2. Encourages students to critically analyse work done by
others, rather than simply seeing a mark3. Helps clarify assessment criteria4. Gives students a wider range of feedback5. More closely parallels possible career situations where
judgment is made by a group6. Reduces the marking load on the lecturer
Involving Students in Assessment IV
Peer assessment – disadvantages1. Students may lack the ability to evaluate each other. 2. Students may not take it seriously, allowing friendships,
entertainment value, etc. to influence their marking. 3. Students may not like peer marking because of the
possibility of being discriminated against, being misunderstood, etc.
4. Without lecturer intervention, students may misinform each other.
Involving Students in Assessment V
Involving Students in Assessment VI
Self assessment the involvement of the students in identifying standards
and/or criteria to apply to their work making judgements about the extent to which they have
met these criteria and standards this form of assessment is commonly a supplement to
teacher assessment of students, but in some cases it may replace it.
BSc Computer Science I
SUMMATIVE
FORMATIVE
“diagnostic use of assessment to provide feedback to teachers and students over the course of instruction”
“generally takes place after a period of
instruction and requires making a judgment about
the learning that has occurred e.g. by grading
a test”
Categories of Assessment
BSc Computer Science II
SUMMATIVE
FORMATIVE
Laboratory Work
Assignments
Assignments
Examination
Categories of Assessment
Learning Outcomes
Programme Learning
Outcomes
Develop and
Evaluate
Learning & Teaching Strategies
Assessment Criteria
Assessment Methods
Outline of Module Content
Module Learning
Outcomes
Module
Aims
Programme Aims and Content
One line summary
Assessment is the senior partner in the three way relationship also involving learning outcomes and teaching and learning strategy, because it heavily influences how learners approach a course and what they will ultimately know and do.
Have a look at…
John Biggs, Teaching for Quality Learning at University, Open University Press, UK, 2003
Top Related