The

21st Century Educator

Simon Bates simon.bates@ubc.ca

@simonpbates bit.ly/batestalks

Students as partners in learning & teaching

The

21st Century EducatorStudents as partners in learning & teaching

“Anatomy” of skills and values

Example driver of change: technology

Case study example

Deeper engagement with assessment,

learning

CC BY-NC-SA 2.0 https://flic.kr/p/2ZdABF

Technology - scale and pace

Slide credit: Eric Grimson (MIT)

Technology - scale and pace

Slide credit: Eric Grimson (MIT)

Technology - reach and unbundling

https://www.youtube.com/watch?v=QWEq3xifCDw

Technology - disruptions

Graph extracted from http://vikparuchuri.com/blog/on-the-automated-scoring-of-essays/

Technology - implications

Changing the

of many aspects of life, …and learning is included

what, where, when, how, from whom and with whom

So what are the

we need to embrace, develop and refine?

skills, values and habits

This work is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License.

Case study - students as partners

“How can I get my students to engage more effectively with formative assessment opportunities in the course?”

a"web&based"MCQ"repository"created"by"students"

Ins$tu$ons((signing(up(per(year:(!

2009:! ! !22(2010:! ! !66(2011:! ! !204(2012:! ! !266(2013!(Jan,Jun):! !214(

Growing(content(repository:(!Courses:! !2,500(Logins/month:! !75,000(Ques<ons:! !600,000(Answers:! !12,000,000(

Answers (20,000,000)

Questions (1,000,000)

Student'ownership'over'learning'resource'

Student'familiarity'with'social'so7ware'

Leveraging'student'energy'and'crea9vity'

Badges' Points' Leaderboards'

Selected results & analysis

Engagement - how do students use the system?

Benefits - what is the impact on learning?

Question quality - how good is what students produce?

Relevant publications:

Scaffolding student engagement via online peer learning - European Journal of Physics 35 (4), 045002 (2014)

Student-Generated Content: Enhancing learning through sharing multiple-choice questions. International Journal of Science Education, 1-15 (2014).

Assessing the quality of a student-generated question repository - Phys Rev ST PER (2014) 10, 020105

Student-generated assessment - Education in Chemistry (2013) 13 1

Typical implementation

Minimum participation requirements for each of two assessment exercises (PW1, PW2)

Write 1 Answer 5 Rate / comment 3

5% course credit

Physics 101, Energy & Waves Winter Semester: 3 sections, ~800 students

Not so typical implementationTOCCLT'.' -

Qest\I\sct

How 1o...

q/t4osTer

Submit ond onswer questionson topics in lhe torget region,just obove lhe physicsyou hove olreody mostered.

Dislroclers

) l*"*"ò

rNrrltngIh s region corrk:ins llte

plrysics knowledgo ond colrco¡lsyou connol leorn yel

becouse the louncalionscrrc nol in plocc

PHYSICS TOPICS IN YOUR TARG T REGION

buo'5ot't1 'Den¡¡lu

B¿:o\v\vì 1 JF:rcq - YOUR CHOSEN TOPIC

à..\auC'q^ x@bôo'osro

pn ro, d logroo, \o 9tS

COMMON MISCONCEPÎIONS AND ERRORS(Sec hHÞ:/jÞhy¡.udofo¡.cdu/CJP/trêconcoÞllo¡t.pdt

tor o llil ol common mhconcepllons)

¿K N(,qht %cce Açxs no\ exi¡þ, c.rrrçr\J On oblec'-t \5 õÞYJrnr cìü?ÀI in rtuiò. ü

lrh'brot4ont eprç,e .I¡1enÀs crqden-s,\{^ c>Ç c.,þ¡ec\ , cìo+ .\*¿n:r

oÇ Çtu,8,fr- btrrqon! $crCe c¿c\s Àoc¡:'cr

fr¿lt{e- r *hcrn ú?

Trr¿e. $ììfifflsz hrrr: ¿lÕoolr-g lnnS

(**= tcpo þ/rnl {otr-* qrÇ bq.\\:. 0" Ooz

Physics knowledgeond conccpluol underslonding

you lrove olrcodyconslrucleC ¡n your heod

¡,<'t.lerJ rt)\^,¿z! Þ.rrp \r- \rr't\¡ ,l?

Chcck lhol yovr on3Íêr ¡t rêo!ànoblê ðnd potrlble

\'f<,,cc\

Photo by Seth Casteelhttp://www.littlefriendsphoto.comPermission to use agreed

Writing original questions is a demanding activity

Extensive scaffolding exercises

Revisited in subsequent tutorials

Copyrig

ht2013Graham

Fow

ell/The

Hitm

an,re-prod

uced

with

permission,Edu

ca&o

nInChemistry,Vo

l50No1(201

3)

Engagement with PeerWise

Number Multiplier Number MultiplierQuestions 1105 [1.7] 998 [1.6]

Answers 11393 [17.2] 11807 [18.7]

Comments 4901 [7.4] 5509 [8.7]

PW 1 PW 2

Engagement with PeerWise

Engagement with PeerWise

Engagement with PeerWise

Engagement with PeerWise

Generally, students did

• Participate beyond minimum requirements • Engage in community learning, correcting errors • Create problems, not exercises • Provide positive feedback

Engagement with PeerWise

Generally, students did not

• Contribute trivial or irrelevant questions • Obviously plagiarize • Participate much beyond assessment periods • Leave it to the last minute (sort of….)

Engagement with PeerWise

Correlation with learning

Correlation with learning

Quartiles Q4 – top 25%

Q3 – upper middle

Q2 – lower middle

Q1 – bottom 25%

22 students did not take the FCI

Correlation with learning

Correlation with learning

1st year Chemistry N=172 University of Edinburgh

Question/Explanation Quality

Bloom’s Taxonomy of levels in the cognitive domain

Score Level Description

1 Remember Factual knowledge, trivial plugging in of numbers

2 Understand Basic understanding of content

3 Apply Implement, calculate / determine. Typically one-stage problem

4 Analyze Typical multi-step problem; requires identification of strategy

Evaluate Compare &assess various option possibilities; often conceptual

Synthesize Ideas and topics from disparate course sections combined. Significantly challenging problem.

Text

1 2 3 4 1 2 3 40

20

40

60

Num

ber o

f que

stio

ns

Assessment 1 Assessment 2

Bloom's Taxonomy: Question Quality

Textp>0.05, NS

0%

5%

10%

15%

20%25%

30%

35%

40%

45%

50%

1 2 3 4 5 6

Taxonomic Category

Per

cent

age

of S

ubm

itte

d Q

uest

ions

First semester N = 350

Second semester N = 252

\

Question/Explanation Quality

Description of explanation quality

Score Level Description

0 Missing No explanation provided or explanation incoherent/irrelevant

1 Inadequate Wrong reasoning and/or answer; trivial or flippant

2 MinimalCorrect answer but with insufficient explanation/justification/ Some aspects may be unclear/incorrect/confused.

3 Good Clear and detailed exposition of correct method & answer.

4 ExcellentThorough description of relevant physics and solution strategy. Plausibility of all answers considered. Beyond normal expectation for a correct solution

0 1 2 3 4 0 1 2 3 40

20

40

60

Num

ber o

f que

stio

ns

Assessment 1 Assessment 2

Explanation Quality

Results (UoE 2010-11)

2 successive years of the same course (N=150, 350)

‘High quality’ questions: 78%, 79%

Over 90% (most likely) correct, and 3/5 of those wrong were identified by students.

69% (2010) and 55% (2011) rated 3 or 4 for explanations

Only 2% (2010) and 4% (2011) rated 1/ 6 for taxonomic level.

Bottomley & Denny Biochem and Mol Biol Educ. 39(5) 352-361 (2011)

107 Year 2 biochem students 56 / 35 / 9 % of questions in lowest 3 levels.

Momsen et al CBE-Life Sci Educ 9, 436-440 (2010)

“9,713 assessment items submitted by 50 instructors in the United States reported that 93% of the questions asked on examinations in introductory biology courses were at the lowest two levels of the revised Bloom’s taxonomy”

Comparison with literature

Resources

Student-facing system http://peerwise.cs.auckland.ac.nz/

All the research studies referenced and scaffolding materials referred

to are accessible through the PeerWise community site http://

www.peerwise-community.org/