David McConnellMarine, Earth, and Atmospheric Sciences and Office of Faculty DevelopmentNCSU

Clicker Pedagogy in Larger Classes: How

Using Clickers Can Improve Learning

Office of Faculty Development

Seven Principles of Good Practice in Undergraduate Education

1. Encourages student-faculty contact

2. Develops cooperation among students

3. Encourages active learning

4. Provides prompt feedback

5. Emphasizes time on task

6. Communicates high expectations

7. Respects diverse talents and ways of learning

Chickering & Gamson, AAHE Bulletin, 1987, p. 3-7

Can clickers promote these practices?

When you ask a question in class, what proportion of your students will raise their hand or otherwise indicate that they have an answer?

0%

0%

0%

0%

A. More than 50%

B. About 30%

C. About 10%

D. 0-10%

• Instructor grading of short answer and essay questions

• On-going assessment through student dialog in small classes

• Computer grading of multiple choice questions using bubble-sheets

Understanding Student Learning

Learning assessm

ent systems

More instructor understanding

of learning

Less instructor understanding

of learning

Clicker Pedagogy:

Peer Instruction & Conceptests

It’s the message, not the medium

Clickers in Effective Pedagogy

Harvard students in a traditional calculus-based introductory physics class scored an average of 70% on the pre-test. Predict the average post-test score.

72%

78%

84%

90%

0% 0%0%0%

A. 72%B. 78%C. 84%D. 90%

Crouch, C.H., Mazur, E., 2001, American Journal of Physics, v. 69, #9, p.970-977

What we know about student learning

1. Students learn key concepts better when they have opportunities to actively monitor their understanding.

2. Knowledge is socially constructed and people learn best in supportive social settings when working with peers.

3. Students become better learners when we challenge them to answer questions that require the use of higher order thinking skills.

Peer instruction (& Conceptests)

Development of technique by Eric Mazur, Harvard

Short lecture (10-20 minutes)

Conceptest – conceptual multiple choice question

Individual students signal answers

Student groups may discuss answers (peer instruction)

Explanation of correct answer

Mazur, E., 1997, Peer instruction: A user’s manual: Prentice Hall, 253p.

Pedagogy for Feedback Devices

U. Of Massachusetts model skips individual student answers

Examine the map and answer the question that follows. How many plates are present?

Example: Earth Science Conceptest

McConnell, D.A., et al., 2006, Journal of Geoscience Education, v. 54, #1, p.61-68.

Results when using physical

models: (56%; 84%)

Geology conceptest database : http://serc.carleton.edu/introgeo/

interactive/conctest.html

a. 3 b. 4 c. 5 d. 6a. 3 (26%; 0%)

b. 4 (19%; 18%)

c. 5 (44%; 75%)

d. 6 (11%; 7%)

Individual responses

Post-discussion responses

Student Performance on Conceptests

• About a third of questions in an Earth Science class were asked twice.

• An average of 45% of students responded correctly on the first attempt and 63% answered correctly after peer instruction.

Five questions had fewer correct answers after peer instruction.

Range of student scores for a semester of conceptests

Student Performance on Conceptests

• Nobody has ever averaged above mid-80s for semester

• Most students miss at least a third of questions asked

Crouch, C.H., Mazur, E., 2001, American Journal of Physics, v. 69, #9, p.970-977

60%

70%

80%

90%

100%

1990 1991 1993 1994 1995 1996 1997

FCI pretest score

FCI score gain on post-test

Traditional Class

Peer Instruction Classes

n = 117 - 216

12

3

2

3

4

41 Began PI

Refined conceptests

Changed text

Open ended reading questions

Mazur’s results

Impact of Alternative Pedagogy

Dori, Y.J. and J. Belcher, J. 2004. Journal of the Learning Sciences 14(2).

Impact of Alternative Pedagogy

Poulis et al., (1998)• Results from 5,000+ physics students – increase in

pass rates (55% 80%) in classes that used audience paced feedback (clickers + student discussion)

Dori & Belcher (2004)• Compared pre- and post-test scores for traditional and

technology-enabled physics classes – learning gains were greater (27% vs. 52%) for technology enhanced class

Poulis, C., Massen, C., Robens, E., & Gilbert, M. 1998. American Journal of Physics, v.66 #5, p. 439-441.

Clicker Pedagogy:

Why this works

It’s the message, not the medium

Clickers in Effective Pedagogy

Students completed a short reading assignment. Population A studied the passage twice (7 minutes each time). Population B studied the passage once and then took a recall test. Two days later, both groups were tested on their recall of information. Predict the result.

0%

0%

0% A. Population A scored higher on the test.

B. Population B scored higher on the test.

C. There was no difference in test score.

Roediger & Karpicke, 2006, Perspectives in Psychological Science, v. 1, p.181-210.

Test Enhanced Learning

Roediger & Karpicke, 2006, Perspectives in Psychological Science, v. 1, p.181-210.

120 students complete a reading assignment (~250 words)

• Population A studied the passage twice (7 minutes each time)

• Population B studied the passage once and then took a test

• Both populations then tested at 5 minute, 2 day, and 1 week intervals

A B A B A B

• Population B retained more knowledge after 2 days, 1 week

Peer Learning Assessment

Smith et al., 2009, Science, v. 323, January 2, p.122-124.

U. Of Colorado Genetics Course Conceptest Responses

Individual responses

Peer instruction responses

Follow up individual responses

Conceptest posed

All Correct Correct after peer instruction

The Value of Peer Instruction

Experimental Group: Students took physics test individually, then again as a pair.

Control Group: Students took test individually.

Singh, C., 2005. American Journal of Physics, v.73 #5, p. 446-451. .

Proportion of pairs of students who both got the question wrong on the first test but correct on “paired” test: 29%

Mean score on second exam for experimental group: 74%

Mean score on second exam for control group: 64%

Students in both groups answered similar questions on a second exam two weeks later.

The Value of Peer Instruction

Students taught key concepts using one of four methods. Student learning assessed by proportion of correct answers to open ended questions on same concepts on final exam

Crouch, C.H., Fagen, A.P., Callan, J.P., & Mazur, E., 2004. American Journal of Physics, v.72 #6, p. 835-838.

No demonstration

Observation of demonstration w/explanation

Prediction prior to demo with a conceptest

Prediction prior to demonstration using discussion & a later conceptest

% correct answers

61

70*

77*

82*

Teaching method

n = 158-297; * = statistically significant result vs. no demonstration

Importance of Student Reflection

Dunning et al., 2003. Current directions in psychological science, v.12 #3, p.83-87

The weakest students often do not realize that they do not understand key concepts

Doubly cursed: Students who can’t answer questions correctly can’t self-diagnose their lack of ability

The Value of Peer Reflection

Experimental Group: Three 2-minute pauses per lecture, student discussion of lecture content with peer.

Control Group: No pauses for discussion in lecture.

Ruhl, Hughes, and Schloss., 1987. Teacher Education and Special Education, v.10 #1, p.14-18

All students completed a free recall exercise at end of lecture and delayed multiple choice test 12 days later.

Exp. Group – mean number of facts recalled : 22.97*Cont. Group – mean number of facts recalled : 16.63

Exp. Group – MC test average score : 84.39*Cont. Group – MC test average score : 76.28

*statistically significant gain

Value of Attendance

Moore et al., 2003, American Biology Teacher, v. 5, p.325-329.

Emphasized attendance, showed data graph weeklyAverage attendance 70%Average grade 73%

Verbal encouragement to attend (less emphasis)Average attendance 59%Average grade 64%

Science classes at University of Minnesota

Clicker Pedagogy:

. . . and it makes me feel good

It’s the message, not the medium

Clickers in Effective Pedagogy

Instructor Satisfaction Survey

1 2 3 4 5

Helped determine students understanding

More contact, communication with students

More cooperation among students

Less lecture, more student discussion

Prompt feedback on student learning

Easier to emphasize critical concepts

Emphasize high expectations

More opportunity for diverse skill sets

Class was more enjoyable than previous

Would recommend CPS for other classes

Strongly Agree - Agree - Neutral - Disagree - Strongly Disagree

(n = 35)

Student Satisfaction vs. Class Level

n = 1597

Student Satifaction Survey Scores

1 2 3 4 5

Made class more enjoyable

Would recommend use at UA

Increased my willingness to ask questions

Improved my performance in class

Increased my interaction with other students

Increased desire to come to class

Reinforced important concepts

Helped gauge level of understanding

Average Score(1=Strongly Agree, 5=Strongly Disagree)

Lower (<200)

Upper (>200)

Graduate

Impact on Students

Total Students Surveyed: 1327

Pedagogy Technology Other Total

What do you like best about the use of conceptests and clickers?

# of Responses 950 93 52 1095

% of Responses 86.8% 8.5% 4.7%  100%

What do you like least about the use of conceptests and clickers?

# of Responses 162 621 64 847

% of Responses 19.1% 73.3% 7.6%  100%

Student Comment Matrix

Benefits of Technology & Pedagogy

What did you like best about the use of the conceptests and CPS?

“That it really helped me participate more in class.” Natural Science Biology

“CPS forced me to review class materials. This helped to reinforce my memory/knowledge.” Microbiology

“I was able to gage my knowledge level to others in class.” Emergency Management.

“It makes you want to learn more and enjoy the class more.” Basic Mathematics II

“I knew what I had to study.” Human Diversity

“I actually had to figure out problems to answer them, so I understood it better.” Principles of Chemistry

“Not having to talk to participate.” Government and Politics

Drawbacks of Technology & Pedagogy

What did you like least about the use of the conceptests and CPS?

“We got into discussion groups, but didn’t discuss.” Human Diversity

“Questions sometimes are a little difficult to understand.” Criminal Case Management

“The stress of missing questions.” Statistics

“It takes time during class that we could be using to take notes.” Human Diversity

“Some questions didn’t give enough time.” Macroeconomics

“Attendance and having to come to class all of the time.” Government and Politics

“Don’t make questions all at the end of lecture. Throw some in the middle of the lecture.” Macroeconomics

Suggestions for Using Clickers

Some suggestions from our experience:

Use the clickers everyday to insure students bring them with them to class and value their use

Make questions sufficiently challenging to them worth asking – aim for correct response rates 50-70%

Avoid grading headaches – low stakes assessment, consider using for participation points

Bigger is best – use in lower level, gen. ed. classes

Less value if class already uses active pedagogy strategies or if students already participate fully

Seven Principles of Good Practice

1. Encourages student-faculty contact

2. Develops cooperation among students

3. Encourages active learning

4. Provides prompt feedback

5. Emphasizes time on task

6. Communicates high expectations

7. Respects diverse talents and ways of learning

Chickering & Gamson, AAHE Bulletin, 1987, p. 3-7

Clicker Pedagogy:

* Writing Good Questions *

It’s the message, not the medium

Clickers in Effective Pedagogy

Teaching and learning goals can be ordered using Bloom’s Taxonomy

Knowledge

Comprehension

Application

Analysis

Synthesis

Evaluation

memorization and

recall

understanding

using knowledge

taking apart

information

reorganizing

information

making judgements

Assessment with Clickers

Degrees of correctness

Right/Wrong answers

More complex questions call for more sophisticated guides and responses

Introductory Exercise 

Examine the six questions on page 2 of the handout. Assume you are a student in classes where these questions would be appropriate.

Rank the questions from the easiest to most challenging based on the character of the question and nature of knowledge needed to answer it correctly.

Classify using Bloom’s Taxonomy

A. Which one of the following values approximates best to the volume of a sphere with radius 5m? a) 2000m³ b) 1000m³ c) 500m³ d) 250m³ e) 125m³

B. How successful were recent income tax cuts in spurring economic growth?

C. What is the capital of Maine?

D. How would you restructure the school day to reflect children’s developmental needs?

E. Contrast the floor of the Atlantic Ocean with the shape of a bathtub.

F. Which statements in the President’s State of the Union address were based on facts and which were based on assumptions?

Which is a synthesis question?

A B C D E F

0% 0% 0%0%0%0%

A. A

B. B

C. C

D. D

E. E

F. F

Which is an application question?

A B C D E F

0% 0% 0%0%0%0%

A. A

B. B

C. C

D. D

E. E

F. F

Bloom’s Taxonomy

KnowledgeWhich one of the following persons is the author of "Das Kapital"?

a) Mannheim b) Marx c) Weber d) Engels e) Michels

ComprehensionFill in the blank to complete the analogy. The yolk is to the egg as the ____________ is to Earth.

a) crust b) mantle c) core d) asthenosphere

Bloom’s Taxonomy

ComprehensionIn the landscape below, how would the amount of rainfall change at location X if the mountain eroded down to the dashed line?a. Rainfall would increaseb. Rainfall would decreasec. Rainfall would stay the same

Bloom’s Taxonomy

AnalysisRead carefully through the paragraph below, and decide which of the options a, b, c, or d is correct. Rising saturated air undergoes: i) adiabatic cooling as air contracts due to decreasing pressure with increasing altitude; and, ii) warming due to the latent heat of condensation as water vapor is converted to liquid water droplets.

a. The word “contracts” should be replaced by “expands”.b. The word “condensation” should be replaced by “evaporation”.c. The word “warming” should be replaced by “cooling”.d. The word “altitude” should be replaced by “elevation”.

Bloom’s Taxonomy

a) The assertion and the reason are both correct, and the reason is valid.

b) The assertion and the reason are both correct, but the reason is invalid.

c) The assertion is correct but the reason is incorrect.d) The assertion is incorrect but the reason is correct.e) Both the assertion and the reason are incorrect.

EvaluationJudge the sentence in italics according to the criteria given below: "The United States took part in the Gulf War against Iraq BECAUSE of the lack of civil liberties imposed on the Kurds by Saddam Hussein's regime.“

Any Questions?

Clickers and Effective Pedagogy