CPRWhat’s in it for you?

December 11th, 2006Teaching Excellence CenterSofia Carlos-Cuellar and Tim Zauche

What is Calibrated Peer Review?• It is a web-based free program that

allows you to – 1. incorporate more writing in the

classroom– 2. improve critical thinking skills, self

reflection– 3. Increase communication abilities– 4. Provides another way for students to

express their knowledge

Why CPR?

• Do you use writing in your classroom currently?

• How long do you expect it would take to grade?

• Do your students critique their peers work as well as their own?

• How well constructed are your assignments?

Why do we use CPR?• What are some learning goals you

have for your class?– 1. Increase knowledge retention– 2. Improve written communication– 3. Be able to critique writing

MAJORS USING CPR at TA&M

•Accounting•Ag. Economics•Animal Science•Archeology•Biochemistry•Biology•Botany•Bus. Admin.•Chemistry•English

•Ed. Psychology•Film•French•Kinesiology•Learn. Comm.•Math•Microbiology•Nutrition•Physics•Poultry Science

PsychologyPsychology Secondary Ed.Secondary Ed. Vet Integr. Bio. Sci.Vet Integr. Bio. Sci. Wildlife & Fish. Sci.Wildlife & Fish. Sci. ZoologyZoology

How does CPR help exactly?• Excerpts from: Arlene A. Russell, UCLA

19th BCCE, Purdue, ILShows how CPR has been used in various disciplines to improve learning

Economics, Biology, General Chemistry, Business Writing, Engineering

Writing-to Learn Economics - CSUN• Comparison of intact classes

• Ten case-study assignments

• Three sections (same instructor) of Economics 200; 2 CPR-based, 1 traditional with essays but no evaluation component

• Repeated over two semesters

Chapman, K.S. COBAE Faculty Report (1999-2000)

Average score by quintile on midterm(Economics 200, Fall 1999)

0

10

20

30

40

50

60

70

80

q1 q2 q3 q4 q5

quintile

Avera

ge s

co

re o

n m

idte

rm

CPR

no CPR

Writing-to-Learn Biology - CSUF•One class, one instructor

•Ten topics, 5 taught through CPR, alternately with 5 taught through lecture and simulation

Pelaez, N.J. “Problem-based Writing with Peer Review Improves Academic Performance in Physiology” Advan Physiol Educ, Sep 2002; 26: 174 - 184.

Mean multiple choice test scores were significantly better for topics taught using CPR compared with those taught using traditional instruction.

72 + 2% versus 65 + 2%, P < 0.001, Paired t-test.

0%

20%

40%

60%

80%

100%

500 600 700 800 900 1000

Total points for BIO310

Sco

re o

n m

ult

iple

ch

oic

e q

ues

tio

ns

CPR concepts

Lecture Concepts

Linear (CPR concepts)

Linear (Lecture Concepts)

Mean essay exam scores were significantly better for topics taught using CPR compared with those taught using traditional instruction.

123 + 4 versus 114 + 3, P = 0.004, Paired t-test.

0

25

50

75

100

125

150

500 600 700 800 900 1000

Total points for BIO310

To

tal s

co

re o

n e

xa

m e

ss

ay

qu

es

tio

ns

CPR conceptsLecture Concepts( Linear)CPR concepts( Linear)Lecture Concepts

Learning-to-Write Business Writing - CSUN

• Pre-post comparison of three experimental instructional groups– CPR peer review, In-class group

review, Traditional instructor review– All three methods resulted in

statistically similar gains in writing.

Plutsky, Susan, and B.A. Wilson. 2004. Comparison of three methods for teaching and evaluating writing: A quasi-experimental study. Delta Pi Epsilon 46(1): 50-61.

Taking the CPR tour

http://cpr.molsci.ucla.edu

UWP Learning Goals• Students need to be better

communicators, specifically in writing

• Students need to learn to critique their own writing

• Students need to learn specific concepts better

Why is writing important?

Synthesis

Analysis

Application

Use of information Use of information to solve problems; to solve problems; transfer of abstract transfer of abstract or theoretical ideas or theoretical ideas

to practical to practical situations.situations.

Interpretation Identification of Identification of connections and connections and

relationshipsrelationshipsTranslation

Restatement in your Restatement in your own words; own words;

paraphrase; summaryparaphrase; summary

RecallVerbatim information; Verbatim information; memorization with no memorization with no

evidence of evidence of understandingunderstanding

Evaluation

Bloom’s Taxonomy – categorizingBloom’s Taxonomy – categorizing level of abstraction of questionslevel of abstraction of questions

First Year’s Results from A&M Class

SALG* Results on How Students View CPR (*Student Assessment of Learning Gains)

0

10

20

30

40

50

CPR isEnjoyable

Helps usLearn

Chemistry

HelpsImproveWriting

HelpsImprove

Critiquing

OthersShould Do

CPR

Term - Chem Class

Perc

en

t o

f S

tud

en

ts W

ho

A

gre

e

F'02-102

S'03-101

This was This was notnot good…… Everyone but Leader stopped doing CPR. good…… Everyone but Leader stopped doing CPR.She knew that scientific writing/critiquing were invaluable skills, so She knew that scientific writing/critiquing were invaluable skills, so She kept trying new approaches to improve student attitude.She kept trying new approaches to improve student attitude.

SALG* Results on How Students View CPR (*Student Assessment of Learning Gains)

0

10

20

30

40

50

60

70

80

CPR is Enjoyable Helps us LearnChemistry

Helps Improve Writing Helps Improve Critiquing Others Should Do CPR

Survey Questions

Per

cent

of S

tude

nts

Who

A

gree

F'02-102

S'03-101

S'04-101

F'04-102

S'05-101

F'05-102

S'06-101

What she saw was a significant increase in student acceptanceWhat she saw was a significant increase in student acceptance and understanding of CPR over time.and understanding of CPR over time.

QuantitativeQuantitative ResultsResults

QualitativeQualitative Results – OverviewResults – Overview

Percentage of Positive Comments per Semester

0.010.020.030.040.050.060.070.0

Fall 2002 Spring2003

Spring2004

Fall 2004 Spring2005

Fall 2005 Spring2006

Semester

Perc

en

tag

e (%

)

Ways to effect Student Opinion• Be upfront with them

– Let them know you are experimenting to help them learn better

• Write or edit your own assignments– Everyone’s teaching style is different

• Be tough but understanding– Give extensions when warranted– Monitor the assignment progress & email

reminders

• Do your own homework– Participate as a student

Difficulties of using CPR

• Zauche needed to be clearer of his assignment goals– What to put in an introduction section– He is now a better teacher and writer

• About 10-15 hours for a mediocre assignment

• Adding and getting usernames to students

Improvements Made Over Time

• Prepared more thorough instructions and a supportive website• Wrote most of my own assignments• Told my students upfront in the syllabus that the class was a

writing-intensive class and included my teaching philosophy• Invited students to let me review their essays before submission• Increased importance of the text entry • Increased CPR’s worth from 3-5% to 12% of class grade• Took classroom time to demonstrate CPR• Used Bloom’s Taxonomy to show importance of critiquing

What classes have used CPR at UWP?• Chem 105 (Sofia)

– Spring and Fall 06 (4 assignments)

• Chem 124 (Tim)– Fall 05, Spring 06, Fall 06

• Chem 273 Inorganic Chemistry (Tim)– Fall 06 Lab reports by section

• Abstract, Introduction, Results/Discussion

• Other Faculty– Education (LoGuidice) Fall 06– Biology (Sharon and Jeff) Spring 07– Ag (Reddy) Fall 06

Comments from Chemistry Classes• Students are aware CPR is more work

• Students are aware that they are learning more

• Are the students learning the concepts better? – Final Exam results to be compiled

• Are the students writing better?– Hard to assess currently

What have we accomplished?• Determined the level of difficulty in

creating CPR assignments

• Determined the ability to use CPR at UWP

• Better local understanding of how to create a CPR assignment

• Held a training session and are exploring the ideas of having a CPR professional hold a training session here

Support

• Assessment Activity Fund

• Teaching Excellence Center– Presented a workshop to 6 faculty in Aug

• Faculty Professional Development– Presented at the

• Biennial Conference on Chemical Education