Magnificent Methods

‘Which’ Questioning Strategy? version 8 Excellent *** good ** weak *poor !!

Questioning Strategy(Students anticipate these strategies in advance)

Participation

Rate

Teacher’s feedback

Student’sfeedback

Thinking time

Studentcomfort

Q&A: volunteers answerStudents volunteer to answer questions

Q&A: nominees answer Students nominated by the teacher answer questions

Buzz groups: volunteers answerStudents work in small groups to answer a thought provoking question. Teacher asks each group in turn to contribute part of the answer. A volunteer answers for their group.

Buzz groups: nominees answer As above, but after the discussion the teacher nominates the student in each group who will contribute that group’s answer(s).

Assertive QuestioningGroups work on a thought-provoking question. The teacher nominates who will give their group’s answer. The class are asked to discuss the various group answers, to agree a ‘class answer’. Only then does the teacher ‘give away’ the right answer.

Pair checkingTeacher asks a question, then students work alone to answer it. Pairs then compare their answers, giving their partner one good point and one way their answer could be improved while the teacher observes. The teacher then gives the correct answer. Pairs now suggest another improvement to their partner’s answer.

Participation Rate: The proportion of students who are engaged in trying to answer the question is high. There are few ‘passengers’. Teacher’s feedback: The teacher gets representative feedback on the quality of students’ reasoning and understanding in the class.Student’s feedback: The students get feedback on the quality of their understanding, e.g. by discussing their understandings with others: ’dialogue’Thinking time: A given student is likely to spend a good deal of time thinking productively about the question, and the quality of their answerStudent comfort: Students are not ‘put on the spot’ and they are unlikely to feel humiliated by the teacher or by others in the class.

‘Which’ Questioning Strategy?The precise questioning strategy you adopt can make a big difference to the proportion of students who participate, both mentally and verbally, in your lessons. Here are some alternative strategies. They are evaluated overleaf against important characteristics. You can combine these strategies, or develop your own.

Question and answer: volunteers answerStudents volunteer to answer questions posed verbally by the teacherThis is usually done with ‘hands up’but sometimes students call out their answer. If more than one student volunteers the teacher chooses who will answer. The ‘thinking time’ between asking a question and the answer being delivered is usually less than a second here. There is often low ‘participation’ rate because students learn that if they do not answer, they will not be asked to contribute. If some students call out the answer they reduce the thinking time of the others.

Question and Answer: nominees answer Students nominated by the teacher answer questions asked by the teacher. “Pose, pause, pounce” That is the teacher poses the question, pauses for thought, and then ‘pounces’ on an individual to answer. (Optionally, the teacher can choose student who appear not to be attending to answer questions.)

Buzz groups: volunteers answerStudents work in small groups, or pairs, to answer a thought-provoking question, or do a calculation or similar task. The teacher asks each group in turn to contribute part of the answer. E.g. “Can you give me one advantage of using a laser printer?….Can this group give me another?” A volunteer answers for their group. They are called ‘buzz groups’ because of the buzz of conversation created while they work.

Buzz groups: nominees answerAs immediately above, but the teacher nominates the student in each group who will contribute that group’s answer(s). The teacher only chooses which student will give the group’s answer after the group discussion. All members of the group are then likely to attend, and to try to understand the group’s answer, as any of them may be asked to explain it. (Optionally, the teacher can choose students who do not appear to be attending to explain their group’s answer.)

Assertive QuestioningBuzz groups work on a thought provoking question. The teacher asks individuals to give their group’s answer. These individuals are nominated by the teacher. The teacher gets a number of answers without giving the correct answer away. The teacher then encourages the class to discuss their various answers, and to agree, and justify a ‘class answer’. Minority views are allowed, but the aim is consensus. Only when the class has agreed its answer does the teacher ‘give away’ the right answer. (See Geoff’s separate handout or ‘Teaching Today’.)

Pair checkingTeacher asks a question, and students work alone to answer it. Pairs then compare their answers and each individual says something positive about their partner’s answer and one thing that would improve it. The teacher now gives the correct answer. Pairs suggest another improvement to their partner’s answer. This can be done with pairs combining into fours to compare their answers. The teacher listens to some student conversations throughout.

‘Which’ Questioning Strategy? version 4 Excellent *** good ** weak *poor !!

Teaching Strategy

Students usually anticipate these in advance

Participation

Rate

Teacher’s feedback

Student’sfeedback

Thinking time

Studentcomfort

Mini whiteboardsStudents have A4 whiteboards (laminated card) and dry-wipe pens.

1. The teacher asks a question, and students write their answers.

2. The teacher waits until all or most of students have an answer. Optionally, students check their neighbours’ answers.

3. The teacher then asks students to hold their answers up, all at the same time. Students then look round to see what their classmates have written.

4. The teacher surveys all the boards to see what answers were given.

5. The teacher clarifies any misunderstandings.

Participation Rate: The proportion of students who are engaged in trying to answer the question is high. There are few ‘passengers’. Teacher’s feedback: The teacher gets representative feedback on the quality of students’ reasoning and understanding in the class.Student’s feedback: The students get feedback on the quality of their understanding. Ideally the student improves their understanding as a result.Thinking time: A given student is likely to spend a good deal of time thinking productively about the question, and the quality of their answerStudent comfort: Students are not ‘put on the spot’ and they are unlikely to feel humiliated by the teacher or by others in the class.

Getting Interactivity Interactivity is a vital component of good teaching. Researchers found teachers who were in the top one percent of all teachers nationally for six years running in terms of value added. They did not use conventional questioning strategies, but used something like 'assertive questioning' (below) instead. Assertive questioning is part of ‘whole class interactive teaching’ (WCIT). Professor John Hattie has synthesised and compared over 300,000 experiments to improve student attainment, he finds WCIT is one of the very best teaching methods raising students attainment by almost two grades compared to conventional methods. See chapter 15 of 'Evidence Based Teaching' Geoff Petty.

Assertive Questioning1. Buzz groups work on a thought provoking question.

2. The teacher monitors this work asking: ‘does everyone have an answer, ask me for a hint if not’

‘does anyone need more time?’ If a group does not respond to this offer of help they are ‘fair game’ for the next stage. The teacher does not give the answer away if they do help a group.

3. The teacher nominates individuals to give their group’s answer, and to justify it: “why do you think that?”. The teacher thanks the student for their answer, but does not evaluate it. They might ask supplementary questions such as 'Does anybody agree with that answer?' 'Has anybody got anything different?'.

4. The teacher gets a response from each group in this way, or at least a number of groups, and then points out any inconsistencies between the groups’ answers if any. (If there aren’t any, perhaps the question could have been more challenging, though in early practice easy questions are helpful)

5. The aim now is to get the whole class to agree their ‘class answer(s)’. The teacher encourages the class to discuss and evaluate their various answers, and to agree, and to justify their ‘class answer’. Minority views are allowed, but the aim is consensus.

6. Only when the class has agreed its answer does the teacher ‘give away’ the right answer, or evaluate and comment on the answers given.

This method works whether there are right answers or whether different interpretations and answers are likely e.g. in a critical appraisal of a painting.(See the diagram below and ‘Whole Class Interactive Teaching’, chapter 24 in ‘Teaching Today’.)

We will learn best if we all work towards a “blame-free” classroom: It’s okay if you don’t fully understand a concept first time, learning takes time. what counts is whether you understand the question or task, and its answer eventually, not whether you get it right first time I ask challenging questions so it is not humiliating to make a mistake. We all make mistakes when we learn. Indeed that is part

of how we learn. If we don’t make mistakes the work is too easy for us to learn at our maximum rate. Mistakes are useful because they tell us where we can improve. If you make a mistake, bet your life half the class has made it too. It’s good for learning to say ‘I don’t understand’ and to ask for clarification. You should never ridicule another student for their mistakes, even in a joking way because you wouldn’t like it if you were

ridiculed, and because it stops us learning. You will only learn from mistakes if you find out how to do it without mistakes next time, and really understand this. Let’s help each other! The helper learns at least as much as the helped.

Ground-rules like these are best established very early on, and are best developed Socratically by asking students for their ideas for class ground-rules. “We all want to enjoy ourselves and we all want to learn well, so what should be your ground-rules?” Note it is ‘your’ not ‘my’ ground-rules!

Student demonstration

This is an excellent method to test and develop students’ understanding of a simple skill such as mathematics problem solving, punctuation, scientific reasoning, detection of imagery in a poem etc. It is very similar to assertive questioning and is used routinely in Eastern European countries and in Pacific Rim countries such as Taiwan and Singapore, which routinely achieve top ranking in international comparisons.

The method is used after a teacher demonstration of a practical or intellectual skill. The aim is to check and correct understanding of a skill before all students practice it. It is initially a bit daunting for students but they will greatly enjoy the method if you introduce them to it properly. Use pair explaining first (see page ???” this prepares them for student demonstration very well. After they have learned to use peer explaining well, set them tasks to do in pairs followed by a student demonstration, perhaps asking for volunteers. Then move on to pair work followed by you nominating the student to demonstrate. Give them fair warning in any changes.

The basic procedure for student demonstration is:

1. You set a task: “working in pairs, factorise 6x2 - 6x - 8” “Okay, in pairs, punctuate this paragraph” “Working by yourself this time, can you see any personification or metaphors in the third or fourth verse of the

poem?”

2. Students work on the task. This can be done in pairs initially, but after a bit of practice they do tasks individually, perhaps checking each other’s completed answers in pairs.

They strive to get the answer, with any justification such as necessary reasoning or working etc. If students are in pairs they make sure that either of them can provide this justification.

3. You monitor the work. You check attention to task and occasionally ask: ‘Can everyone do this one?’ ‘Can you all explain your answer?’

Students who can’t answer the question are required to own up and get help at this stage, otherwise they are ‘fair game’ for the next stage.

4. You choose a student to demonstrate their answer to the rest of the class. If students are in pairs you choose one student at random to give the pair’s answer. The student gives their answer on the board, explaining each step and its justification to the class. You ask questions to clarify, but do not yet evaluate the answer.

“Why didn’t you use 6 and 1 as the factors of 6?”

“Why did you choose a full stop and not a comma?” “So how did you choose between personification and metaphor?”

5. You ask for a ‘class answer’. You ask the class if they agree with the student’s answer and its justification, or whether either could be improved. The aim is not to criticise the student’s answer, but for the class to agree a ‘class answer’. The student who did the demonstration becomes the class scribe, writing up any changes the class agrees to. You again fascilitate without evaluating the answers or the arguments.

“Why do you think it should be plus four and not minus?” “How many think it should be a comma? Why?” “So why exactly is it not a metaphor?”

6. You comment on the class answer. Praise any useful contributions and confirm any correct reasoning, and correct any weak reasoning.

7. The process is repeated with another task, after sufficient practice the students can do stage 2 as individuals rather than in pairs.

Students are often initially resistant to doing a demonstration if they are not used to it. So you could make use of volunteers to begin with, but try to move on to students nominated by you as soon as you can. They will be more confident of answers that they have produced in pairs, than answers produced in isolation so when you first start nominating students do it after pair work.

All the arguments in favour of assertive questioning apply also to this method. You might like to compare student demonstrations with other methods you use in similar situations, using the criteria we used earlier in this chapter.

Participation

Rate

Teacher’s feedback

Studentcomfort

Thinking time

Student demonstrati

onMy own

method(s)

Using mini white-boards

This is another brilliant way to get answers, and so feedback. It is possible to buy a class set of A3, A4, or A5 whiteboards, each complete with a dry-wipe pen and eraser in a sealable plastic wallet for just over a pound each. Or you can make your own sets by laminating card. Some people argue that pale cream boards are better than white as this assists some dyslexic students with no disadvantage to other students.

The idea was first invented in Primary schools but they are used everywhere now, including university, though there is hi-tech version

To use the mini white-board.It starts off very like students demonstration:

1. You set a task:It works well to start off with students working in pairs, and with later tasks working individually. Tell them if you are going to do this so they work towards their own independence: Working in pairs to begin with, “factorise 6x2 - 6x - 8” By yourself this time, Punctuate this sentence: “Some people such as farmers work outside.” “Working by yourself this time, write down any phrase from verses three or four that involves

personification”2. Students work on the task and write their answer on their board. Once students have their answer 3. You check what is on the boards.You ask students to hold up their boards all at the same time so that you can see their answers. Students will crane their necks to see everyone else’s answers!

4. You ask students about their answers.Students need to know not just whether they have the right answer, but whether they arrived at the answer using the correct reasoning. So you can ask students ‘why’ they have the answer they have:

That’s right Phil, why did you decide to put the comma where you did?

Alternatively you can withhold your evaluation of the answers to spark a class discussion:“Okay, some of you have a comma after ‘people’ and others haven't. What do we think?”

Clearly you can get students to call out their answers and so dispense with the board. “Okay, what is the first word in your sentence with a comma after it?”, but this will only work for very simple questions. It is not that easy to get everyone to call out at the same time and it can be noisy with some classes.

For more on mini-white board products: http://www.easyteach.co.uk/dry.html

For more on high-tech interactivity: http://www.einstruction.com/

Snowballing solutions to a challenging maths problemThis creates real dialogue, participation, and understanding. It needs to be done with supportive ground rules, where errors are accepted without blame, and are seen as an opportunity to learn. In recent studies with underachieving youngsters in low performing schools this method greatly increased students’ interest and enjoyment of maths, and improved their average standardised test results from 45% to 79%! However this degree of improvement took time, about 6 years, but expect improvement of a sort quite quickly.

Explain the no-blame ground-rules above. Then give students a challenging maths question, one they can attempt or get started on but which they often won’t be able to finish by themselves.

1. Individuals write down their own answer, or their attempt, working alone.2. Students then share what they have written in groups of three or four. Each student presents their attempt justifying it to the

others. The others give constructive criticism about the pros and cons of each approach after it has been presented.3. Each group now decides which method is best and why, and presents it to the whole class4. There is class discussion on which methods are best and why, this can be done

assertive questioning style, or student demonstration style as described above.

www.edutopia.org/math-underachieving- mathnext-rutgers-newark

An alternative is to use ‘snowballing’.

1. Individuals write down their own answer, or their attempt, working alone.2. Students pair up and show each other their work. They give constructive criticism to each other, and decide on their best

method3. Pairs form fours, which look at each pair’s method and again give pros and cons and decide on the best method. Leave time

for this dialogue. 4. The teacher chooses individuals by name from each group of four to present their solution/work to the class, and to justify it

to the class.5. The class as a whole now scrutinizes each group’s work, and decides which method(s) are best and why. This is now

assertive questioning, or student demonstration.Two Methods from ‘Teach like a Champion’ by Doug Lemov

No Opt-Out

This technique deals with a student who doesn’t know the answer, or who gets the answer wrong.

Here is a bit of dialogue showing the No Opt-Out technique being used to teach students how to calculate percentages. The teacher has already explained and demonstrated the process, and is now getting the class to calculate 7% of 320, with her guiding and writing on the board:

Teacher: What’s the first thing we do with this one? William?William: Don’t knowTeacher: Carl?Carl: We need to divide 320 by 100.Teacher: Good, why?Carl: Because dividing by 100 gets us 1 percent.Teacher: So what’s this first thing we do William?William: Divide by 100Teacher: Yes. And why?William: So we know what 1% is.

So the teacher goes back to the student who initially “failed”, to ask the question again.

Another example, this time a weak, Level 2 catering course.

Teacher: Why do we store food in a refrigerator? Harry?Harry: To keep it cool.Teacher: Yes, and why do we want it to cool?Harry: Tastes better.Teacher: Well, partly. Who can help Harry?Alex: Germs don’t like it cold.Teacher: What is the advantage of that? Susan?Susan: Stops food poisoning.Teacher: Why?Susan: Because the bugs don’t grow in the cold.….(remember it was Harry who gave the initial unsatisfactory answer)….Teacher: Germs don’t grow in the cold. Okay Harry, so why do we use fridges? Harry: To keep food cold so we don’t get food poisoning.Teacher: And how does the cold stop food poisoning?Harry: Because it stops the germs growing. The cold does.Teacher: Excellent.

No Opt out in diagrammatic form:

Teacher indicates answer is correct

Jo answers correctly

Teacher asks Jo the same question again

Teacher keeps asking questions of the class until a satisfactory answer, and its explanation are obtained.

Teacher asks other students:‘Who can help Jo?’Pete contributes, but his answer is incompleteCan anyone add to Pete’s point?Why (is that the answer)

Student Jo can’t answer a question, or answers incorrectly

Further ReadingDoug Lemov (2010) ‘Teach Like a Champion’. Jossey-Bass

Champion Teachers make use of whole class interactive teaching and interactive dialogue, see:Geoff Petty (2009) ‘Teaching Today’ 4th Ed. Nelson Thornes - see chapter 24 on Whole class Interactive TeachingGeoff Petty (2009) ‘Evidence Based Teaching’ 2nd Ed. Nelson Thornes – See chapter 9 on Whole Class Interactive Teaching, and chapter 15 on dialogue, questioning, and the self correcting

Q1. What are the advantages and disadvantages of this technique?

Q2. Could it be used for questions where there is no one right answer, e.g. interpreting a poem?

3. Prepare an agreed statement, and a question about this technique

Right is Right

Q1. What are the advantages and disadvantages of this technique?Q2. Could it be used for questions where there is no one right answer, e.g. interpreting a poem?3. Prepare an agreed statement, and a question about this technique

This technique can be used in conjunction with the No Opt Out technique, or on its own.

The idea is that the teacher questions the class until it comes up with a near perfect answer given in scholarly language. Earlier answers are used as stepping-stones, and are gradually improved to arrive at the near perfect answer. Here is some text from Lemov’s book, which he also provides in video form, which shows the technique in action in a maths class:

Armstrong: We're going to do a couple of things with volume today. Then we're going to practice volume and then surface area. Can someone give me a definition for volume to get us started? Mark?

Mark: Volume is length times width times height

Armstrong: You're telling me how we're going to solve for volume. If you say “length times width times height” you're giving me a calculation. What I want to know – and you probably know this too, Mark – is what volume is. What is that amount? Yeritza?

Yeritza: Volume is the amount of square cubes that takes up something.

Armstrong: Okay but I want to refine what you said – “the amount of cubes”. What should we say? What's the technical definition instead of just cubes? What were you going to say Wes?

Wes: The amount of cubic inches that a rectangle of prism or a three-dimensional figure takes up.(Many teachers would have accepted this answer, or a previous one.)

Armstrong: Right, any three-dimensional figure. But I don't want to just say cubic inches because it's not necessarily inches. It could be feet; it could be centimetres ; it could be yards…..

Wes: cubic units.

Armstrong: (writing on the overhead) so the amount of cubic units that an object takes up… and Donte,I know you know the other word. What's the other word for “takes up”?

Donte: Occupies

Armstrong: Yes. Occupies. Volume is the amount of cubic units that an object occupies.

Above text from Doug Lemov (2010) ‘Teach Like a Champion’. Jossey-Bass

Right is Right technique in diagrammatic form:

Checking and Correcting in lessons (Excerpts from ‘Evidence Based Teaching’

Assessment for Learning methods aim to give the student feedback information which answers three key questions:

A near perfect answer is given

(if not the teacher provides it with explanation)

An improved, but still imperfect answer is given

Teacher accepts any strengths, but explains one inadequacy in the answer, and asks the class for an improvement

Teacher asks a question

An imperfect answer is given

Some Assessment for Learning methods

The One-Minute Paper: Individuals improve their summary of a lesson

goals

mission

medal

Sadler’s three questions

• What are the goals?• How far have I got in the direction of the goals? (medal)• How can I close the gap? (mission)

This method is very popular indeed in universities where there is a lot of evidence that it works, Chizmar (1998). However it can be used at any

academic level. Again this method works best when set as a goal at the beginning of the class.

1. Students write a one-minute paper, just before the end of the class, writing a paragraph or two on these two questions:

i. What is the most important thing you have learned in this class?

ii. What is the muddiest point still remaining at the conclusion of this class?

2. Students hand in their answers or post them in a box

3. You read the responses and address the muddiest points in the next class.

If the papers are anonymous you get a very honest response, if not you can respond to personal difficulties. Some computing teachers use e-mail for

this, and can then respond individually to students who have any particularly muddy points.

The one-minute summary

This is a useful variation of the previous method.

1. Students write a one or two minute summary of what has just been presented. This could be in response to a question that summarises

the key points to the lesson as in the previous method. Again, the one minute is timed accurately.

2. You give a model, for example your own key points 3. Students improve their own or each other’s summaries using these key points4. (Optional) You collect in the papers and comment during the next class

Alternatively it could proceed rather like ‘Peer explaining topics’, with pairs checking each other’s summaries before seeing the model.

Pilot and navigator: Students instruct/teach other to do a task

This works well for students working on computers in pairs but can be used in other contexts. Students are paired up, one takes the role of

‘navigator’, and the other is the ‘pilot’. The ‘navigator’ tells the ‘pilot’ what to do and why. E.g. “Okay, we need to insert the diagram now, so choose

‘insert’ from the menu.” The pilot does this, and is corrected by the navigator if necessary. The navigator is not allowed to ‘take the controls’.

Students take turns in the roles. It’s harder to explain clearly than it is to do it, so navigators often learn more than their pilots.

This method can be used for maths problems, punctuation and for practicals etc. ‘Pilots’ are allowed to argue, but must explain why they disagree.

Peer assessment with model answersThis is a strategy which Gibbs found almost doubled attainment on a university engineering course (quoted in Biggs 2003). This strategy is

particularly useful if you are setting less work for students than you would like because you can’t keep up with marking.

Students do a worksheet of questions and put their name on it

They hand these in, and you give them out to other students to mark. Students do not know who is marking their work. Codes or pseudonyms

can be used to make work anonymous, but there is more peer pressure if the student’s name is on their work.

Students mark their peer’s work using ‘model answers’ or ‘worked solutions’ including a mark scheme provided by you. If you worry students

might not mark responsibly you could ask markers to initial their marking.

You collect all the marked work and then hand it back to the student who did it. Students each keep the model e.g. the worked solutions.

Most students will probably check the quality of the peer’s marking, but you needn’t.

You ask them what issues came up? What judgments were hard to make?

In the case Gibbs reports, the teacher did not even take down the marks that the students obtained. The average mark on the unit rose from around

45% to around 75% as a result of this strategy! (You could of course collect marks at this stage if you prefer.)

An alternative pair strategy is for the pair to work together to mark their own scripts, again using a model.

Q1. What are the advantages and disadvantages of this technique?

Q2. Could it be used for questions where there is no one right answer, e.g. interpreting a poem?

3. Prepare an agreed statement, and a question about this technique

Embedding formative assessment: Dylan Wiliam

What is Formative Assessment? It is probably best described by the diagram in the front of this handout entitled ‘How to create high quality learning’. Formative assessment involves setting a task or asking a question that requires students to express their understanding, and crucially, then using the student’s answer or work as feedback, to improve this understanding. This improvement might be brought about by the teacher, the student, or a peer.

Some diagnostic questions on Formative Assessment

Which of these statements are true in the context of formative assessment?

A. formative assessment is any assessment that occurs before a summative assessmentB. A formative test is not really a type of test, it is what happens as a result of the testC. Feedback can lead to a lowering of effort, to abandoning a goal, or to reduced aspirationD. Formative assessment requires a test, quiz, or other formal excercise.E. If the teacher allows some students not to participate, they are widening the achievement gap

Some methods that help to embed formative assessment

1 Japanese catch upSay you have 14 lessons to teach a topic or unit:

1. 12 lessons are used to teach the topic2. At the end of the 12th lesson you give students a short diagnostic test or quiz3. The test/quiz papers are not marked by the teacher, who looks them over to discover what students find difficult4. Lessons 13 and 14 are used for remedial activity on the difficulties noticed in ‘3’ above.

15% or 1/6 of time is used for remedial work to fix weaknesses discovered by the test

85% or 5/6 of time is used for teaching, ending in a short test

2 Using exemplar work to clarify success criteria and to improve work

1. A task is set, and students work alone to produce draft work2. The teacher collects papers, and secretly awards a provisional grade to each student. No comments are made on the work. The teacher chooses the three

best pieces of work (exemplars) and photocopies these.3. Students get their own work back, unmarked, along with copies of the three exemplars.4. Students work in groups to use the exemplars to decide on assessment criteria for the task.5. The exemplar work is handed back to the teacher6. Students redraft their own work, and resubmit. (They are not allowed to simply copy the exemplars)

Improve draft work, and resubmit

Decide on assessment criteria

Study exemplars

Do Draft work

3 Peer assessment of draft creative work

1. Students are given a design brief and working alone generate 5 or 6 rough ideas2. Students decide on their best idea3. Students swap their draft ideas with those of another student, and choose the best idea presented by the other student4. Work is handed back to the owner, and discussion follows if choices are different.

4 Students write their own assessment questions

After a topic has been completed, students write assessment questions, along with some means of assessing answers to their questions such as:

assessment criteria, mark scheme worked examples

The students don’t do each other’s questions, but the teacher can grade each student’s questions and assessment guidance.

5 Peer assessment to improve answers without a model or criteria

1. Students work individually on a practice exam test or quiz2. Students work in groups of 3 or 4 and share their unmarked papers3. Students try to create the best composite answer from their individual papers4. Groups share their answers with the rest of the class

6 Class discussion of high-quality diagnostic questions to improve understanding

The teacher devises a number of diagnostic questions, using the misconceptions of past students to create the ‘distractors’. Questions are multi-choice in style, however all, non, or any of the suggested answers may be true or false. So if there are 4 alternative answers given, there are 16 combinations of true/false. If there are 6 answers, then there are 64 true/false combinations. This makes it unlikely students will get the right answer for the wrong reason.

An example of a diagnostic question which has 64 possible responses is shown in the box below:

The diagnostic questions are often used towards the end of teaching a topic, in the following way:

1. The questions are displayed one at a time, and students work first individually, and then in pairs to produce their answer.

2. Students share answers in the class, giving reasons for their choices, and class discussion is used to help the class as a whole to agree the correct responses. The teacher uses interactive questioning such as ‘assertive questioning’ (See Teaching Today or Evidence Based Teaching by Geoff Petty)

3. Misconceptions noticed by the teacher during this discussion are used to improve or to add to the distractors in the diagnostic questions for the next time they are used.

4. Teaching teams share their banks of diagnostic questions

7 C3B4M

While students are working in class they are asked to consult three peers before asking the teacher for help: ( ‘see three before me’ = C3B4M)

8 Traffic-light cups

Each student is given 3 cardboard cups: red, amber, and green. As the lesson progresses, each student shows the level of their understanding by displaying the appropriate coloured cup on top on the desk in front of them:

Which statements below are true?

A. Area is Length x height for a rectangleB. 2 metres squared = 2 metres x 2 metresC. For a square, area is twice the length of one sideD. Area is measured in the units of length, such as centimetres or metresE. For a rectangle, the area is always a bigger number, than that for its perimeterF. Area is the two dimensional space occupied by a shape, in square units

Red: I don’t understand what is going on

Amber: I sort of get it, but I am not confident

Green: I understand it well, and could explain it to others.

The teacher can say to a student displaying an amber cup ‘what don’t you understand?’, and then ask a student displaying a green cup to answer the student’s question.

When the teacher judges it necessary, because there are some amber or red cups being displayed, she says:

“Okay, red cups over here with me, those with an amber cup find a student with a green cup and ask them for help”

9 All student response techniques

Students are invited to give reasons for agreeing, and are asked to improve answers they

Class show agreement or not with each answer by:

Thumbs up: ‘agree’

Teacher gets three different answers, displaying them if necessary

Teacher asks a question

10 Cooperative Learning to increase student accountability

Most learning tasks are rewarded by an individual mark for each student, or for a mark for a group product. There is another method that works four times better which is hardly used.

1. Group students so they are not in friendship groups. One way to achieve this simply is to use random groups. Number round the classroom 1, 2, 3, 4, 1, 2, etc to create 4 random groups when all the 1s, get together to form one group, all the 2s to form another and so on.

2. Set a task or tasks for these groups to help them learn something. Explain that this learning will be tested by a short test, and that students will take this individually and alone. The mark achieved by the group will then be either (teacher chooses):

The same as the lowest mark achieved by any individual in that group, or The same as the average mark of the individuals in that group The same as the average of the two lowest marks in that group

3. Students do the learning task, then take the short quiz or test.4. Groups get their Group Mark.

This method strongly encourages peer helping. It needs to be done in a spirit of fun - not too competitively. If one student gets a poor mark, you can blame the rest of the group for not explaining/helping well enough!

11 Students decide questions for the teacher

Individual students in a group suggest questions for the teacher, and answer these themselves if they can. Any questions they cannot answer can’t be that dumb.

12 Peer evaluation of homework

The teacher checks that homework has been done. If a student has not done the homework, then they don’t join in with this activity, instead they do the homework sitting alone.

The rest of the students are put in groups, are given a mark scheme, and mark the work of those in their group. Using post-its they write ‘ two stars and a wish’ on each piece of work. The ‘stars’ are for positive attributes of the work, the ‘wish’ is a weakness expressed as a forward looking target for improvement.

This encourages students to do their homework as those who did not do it feel left out.

More examples of Formative Teaching MethodsDownload 'Formative teaching' from http://www.geoffpetty.com/activelearning.html for more methods. Chapter 19 of 'Evidence Based Teaching' has the best collection of methods.

The One-Minute Paper: Individuals improve their summary of a lesson

This method is very popular indeed in universities where there is a lot of evidence that it works, Chizmar (1998). However it can be used at any academic level. Again this method works best when set as a goal at the beginning of the class.

4. Students write a one-minute paper, just before the end of the class, writing a paragraph or two on these two questions:i. What is the most important thing you have learned in this class?

ii. What is the muddiest point still remaining at the conclusion of this class?5. Students hand in their answers or post them in a box6. You read the responses and address the muddiest points in the next class.

If the papers are anonymous you get a very honest response, if not you can respond to personal difficulties. Some computing teachers use e-mail for this, and can then respond individually to students who have any particularly muddy points.

The one-minute summary

This is a useful variation of the previous method.

5. Students write a one or two minute summary of what has just been presented. This could be in response to a question that summarises the key points to the lesson as in the previous method. Again, the one minute is timed accurately.

6. You give a model, for example your own key points 7. Students improve their own or each other’s summaries using these key points8. (Optional) You collect in the papers and comment during the next class

Alternatively it could proceed rather like ‘Peer explaining topics’, with pairs checking each other’s summaries before seeing the model.

Pilot and navigator: Students instruct/teach other to do a task

This works well for students working on computers in pairs but can be used in other contexts. Students are paired up, one takes the role of ‘navigator’, and the other is the ‘pilot’. The ‘navigator’ tells the ‘pilot’ what to do and why. E.g. “Okay, we need to insert the diagram now, so choose ‘insert’ from the menu.” The pilot does this, and is corrected by the navigator if necessary. The navigator is not allowed to ‘take the controls’. Students take turns in the roles. It’s harder to explain clearly than it is to do it, so navigators often learn more than their pilots. This method can be used for maths problems, punctuation and for practicals etc. ‘Pilots’ are allowed to argue, but must explain why they disagree.

Peer assessment with model answers

This is a strategy which Gibbs found almost doubled attainment on a university engineering course (quoted in Biggs 2003). This strategy is particularly useful if you are setting less work for students than you would like because you can’t keep up with marking.

Students do a worksheet of questions and put their name on it

They hand these in, and you give them out to other students to mark. Students do not know who is marking their work. Codes or pseudonyms can be used to make work anonymous, but there is more peer pressure if the student’s name is on their work.

Students mark their peer’s work using ‘model answers’ or ‘worked solutions’ including a mark scheme provided by you. If you worry students might not mark responsibly you could ask markers to initial their marking.

You collect all the marked work and then hand it back to the student who did it. Students each keep the model e.g. the worked solutions. Most students will probably check the quality of the peer’s marking, but you needn’t.

You ask them what issues came up? What judgments were hard to make?

In the case Gibbs reports, the teacher did not even take down the marks that the students obtained. The average mark on the unit rose from around 45% to around 75% as a result of this strategy! (You could of course collect marks at this stage if you prefer.)

An alternative pair strategy is for the pair to work together to mark their own scripts, again using a model.

Spoof Assessment

Each member of the class assesses the same piece(s) of anonymous work. This might have been created by the teacher and have deliberate mistakes in it, or it might be genuine past student’s work with the name removed.

The first time spoof assessment is used students are not given assessment criteria, so they just use their gut feeling to mark the work. After students have marked the work, there is a class discussion and the teacher explains where marks were gained and lost etc.

Spoof assessment with two pieces of work. The same two anonymous pieces of work are provided for all students to assess. One piece looks good, but is bad, e.g. it is long and impressive, but doesn’t answer the question. One looks bad but is good, e.g. it is very short, but answers the question very well. Students use their own intuitive criteria to mark the work. Then there is class discussion on the marks.

Later, students are given assessment criteria, mark schemes, model answers, worked solutions, exemplars, assessment criteria etc to help them mark.

Students learn a lot from looking at good quality work.

Peer Explaining

Peers explaining of model answers

This is a variant of the above, and was devised and researched by Carroll. He found that this method enabled students to learn the skill faster while making less errors, even though more stages are involved than the usual method (which is to use only 1 and 6 below).

1. The teacher demonstrates ‘how to do it’ on the board, explaining and ‘thinking out loud’ in the usual way. E.g. how to use tangents to determine an unknown angle , how to use apostrophes, how to write a care plan from a scenario etc

2. Students are arranged in pairs, not necessarily with friends.3. Students are given questions with their model answers, fully worked. The pairs have two different sets of these. It’s best if each set contains a variety of

different types of questions.4. Students study their own worked examples alone, preparing for the next stage (say 5 minutes)5. Each student explains their set of worked examples to their partner, pointing out what was done and why, and why the method and working is sound.

Peer explaining of worked examples

6. Students then practice doing some by themselves in the usual way.

The idea behind this method is that, if teachers go straight from 1 to 6 this is too big a leap for many students. It goes straight from knowledge to application on Bloom’s taxonomy. Consequently weak students are trying to comprehend the method at the same time as trying to apply it which is too much for them. Weak students often report that they understand the teacher demonstration, yet are unable to “do one by themselves.” This strategy provides an ‘extra rung on the ladder’ (strengthening comprehension on Bloom’s taxonomy) which makes students conceptualise the method by requiring them to express it in their own words. Once students are used to peer explaining they can be encouraged to explain to small groups, or to the class as a whole. “John, can you explain your solution to question 8 on the board?”

Comparing Self, Peer and Spoof Assessment (Using the ‘Similarities and Differences’ teaching method)

DifferentState separately the unique characteristics of self, peer, and spoof assessment when compared with each other.

Note their strengths and weaknesses especially.

SameState the characteristics that self, peer, and spoof assessment share.

Note especially the shared strengths and weaknesses as a teaching strategy compared to teacher assessment

Self Assessment

Students assessing their own work against model answers, worked solutions, exemplars, assessment criteria, learning objectives or their own criteria etc

Peer Assessment

Students assessing a peer’s work against model answers, worked solutions, exemplars, assessment criteria, learning objectives, or their own criteria etc

Spoof Assessment

Each member of the class assesses an anonymous piece of work. Alternatively they could be given two anonymous pieces of work to spoof assess, one better than the other but not obviously so. Students can use their own intuitive criteria. Alternatively they could use model answers, worked solutions, exemplars, assessment criteria etc.

Where could you make use of this Similarities and Differences teaching method?

Some ‘find faults and fix’ methods

These are just a few example methods from Geoff’s book ‘Evidence Based Teaching’.

1. Self assessment with traffic lights

Students complete a quiz or test on the work they have done over the last few weeks

They self-mark their paper using a ‘model’ provided by you.

They are provided with a list of concepts, skills, and subtopics that appeared in the test, and are asked to mark each with ticks or coloured blobs as:

Green if they can understand how to do them (ignoring careless slips)

Red if they do not understand how to do them

Amber if they are not sure

The teacher looks through these self-assessments. If there are lots of red blobs next to a particular topic, then this topic is reviewed.

There is a class discussion to get more detail on the difficulties.

Optionally, the greens help the ambers while you help the reds for some or all of the topics in turn.

Students write action plans to respond to their individual weaknesses. E.g. “I need to remember to square root my answer when I use pythagoras’ theorem”

This action plan could be checked by teacher or by a peer, (preferably not a close friend). For example, students could be asked to explain how to do the questions they

have been working on to their peer in the next lesson. (See peer explaining above).

How did you do on: Name:

RedIt’s holding me up

AmberNot sure

GreenIt’s not holding me up

sine

cosine

2. Doing corrections and correcting draft work

This doesn’t sound much fun does it! However getting questions right you initially got wrong ensures you improve understanding and unlearn misconceptions. It can also

make students feel more positive about their performance if they eventually ‘get it right’. These methods also make students more careful in their first attempts as they

often wish to avoid corrective work. However, errors due to simple slips can usually be ignored, it’s fundamental errors that require correction.

Students complete an exercise, worksheet, quiz, test, etc. This method can also sometimes be used on a summative assignment, or for coursework in draft form

etc.

Students’ work is marked using self or peer assessment if you can. (You can check this later if necessary)

Students work in small groups to help each other with what they got wrong. You can help with this of course. Encourage learners to use any errors to diagnose

and correct misconceptions. “Why do you think you/he got that wrong?”

(You could try choosing the groups yourself, arranging for one relatively strong student in each group.)

Students correct or improve their work. One way of checking this is to give students the exercise or quiz etc. again the next lesson, and ask them to do only those

questions they got wrong last time. Another is to ask students to explain the right answer to a question they got wrong last time to a peer.

This strategy, like most teaching strategies, can be overused. Students may find it too dispiriting if you ask them to correct all their work, and they may well not be able to

keep up! However, the method can also be underused. Students sometimes need to have another go at something if they are really to understand how to do it properly. If

groups are supportive this can be greatly enjoyed.

3. Peer helping

This method acts as a filter to discover what students find most difficult and to fix the rest.

Students mark their own and a peer’s work together, using model answers or assessment criteria

Where they both have difficulty the pair tries to work out why, and to put this right. Where they can’t do this they ask help from other pairs or from you. Where

they can’t do this they write the problem on the board.

You discuss the problems on the board with the class

The students then improve their own work with help from their partner.

4. Mastery test

This method is suitable for checking the learning of vital knowledge and skills, it can be used in resource based learning to assess whether a student is ready to progress to

the next unit.

After teaching a topic and after class practice and homework etc you confirm learning with a simple three to five minute quiz or test, requiring the reproduction

of vital key points and skills:

recall questions on key facts

a number of simple calculations to do

a practical activity

some simple past paper questions, etc.

Students mark their own papers against a model you give them. The questions need to be easy enough for students to understand the model answers, and to

be able to mark their own paper.

Students note the questions they got wrong, and why. In maths it may help if they do these questions again, covering the answers while they do so.

There is a minimum acceptable mark of say 70-80%, (it was an easy test on vital material). Students who get less than this take a retest when they are ready,

perhaps a few days later, however, they only do those questions like those they got wrong the first time. Students also mark this re-test themselves and the

cycle repeats.

John Biggs found that this method tended to encourage surface learning. Arguably this could be overcome if other tasks and assessments required a deep approach. It

might also help if questions ask ‘why’ not just ‘what’. Watch out for surface approaches even so. See ‘Teaching Today’ Geoff Petty for a fuller account of these pitfalls.

5. Marking Key

This method is commonly used by teachers of languages. The teacher marks written work by writing codes in the margin to show that an error has been made on that line

somewhere. However the exact point where the error occurred is not shown.

A typical code might be:

S for a spelling mistake

T for a mistake with tenses

SVA for a subject verb agreement issue

P for a punctuation error

/ for a word that is not needed

The student looks at the marking and then corrects their work using the code.

Why is the student not told the detail of their error and where it took place? Because the teacher hopes to develop in the student an understanding of what they did

wrong, and this is best achieved by making the student hunt carefully for their error and think about what they have written. This will develop the ability to proof read

errors.