A New Classroom Practices Observation Protocol ... x 11 Raw CPOP form A New Classroom Practices...
-
Upload
nguyenkhanh -
Category
Documents
-
view
228 -
download
6
Transcript of A New Classroom Practices Observation Protocol ... x 11 Raw CPOP form A New Classroom Practices...
8.5 x 11 Raw CPOP form
A New Classroom Practices Observation Protocol. Francis Jones <[email protected]> Science
Education Initiative
Instructional mode
D Delivery (lecture, probs., Stu. q’ns, etc. … see “Happening”) E Experiment / Simulation / Demo M Media: Video, Anim’n, photo (other than usual ppt) S Socratic (continuous question posing) Q Question to students, not Socratic (open, simple clicker, etc.) A Active students (eg clicker sequence, worksheets, etc.) P Presentation by student(s) T Test or quiz (include groups if two-stage or TBL, etc.)
Instructor is doing …
s Static or low key - Talking m moving, interacting with screen, etc. - Talking w Wandering around class - Talking R Real time writing (board, doc. projector, etc.) H Helping or guiding student work (eg circulating) L Listening or marking (during presentations etc. ) O One-on-one: focus on individual (class may be listening) N Nothing – waiting for activity, etc.
Happening or covering
E Explain new content, knowledge or procedure S Summarizing or synthesizing R Review, revisit, refer to prior content OR knowledge framework. P Practice/Apply; problem, thinking, analysis, etc. C Case study or example(real world; may be with “P”) F Follow up, feedback on work or thinking. B Brainstorming or novel thinking M Motivational (“here’s why we’re doing this”, etc.) H Humor or just friendly A Administration (assign hmwrk, return tests, etc) O Other – explain in comments.
Students Doing
I Individual (listening or doing) C Contributing (e.g. explain in turns, etc.) Q Student asks question P Pairs (or peer instruction) G Groups (note group size in comments) W Whole class (“shout out”, discussion, etc)
Blooms Taxonomy level (updated – more action oriented) R/U remembering / understanding A/A applying / analyzing E/C evaluating / creating Judge based on verbs that are in use (instructor or activity). Blooms Taxonomy level (original from 1956) K / C knowledge / comprehension A / A application / analysis S / E synthesis / evaluation
Eng = Engagement observations: Here’s how to do this … 1) Select N students to observer. 2) Enter “X/N” = count of #students engaged (or not “disengaged”) in what’s going on. Key: “Key Flag” helps identify key messages for feedback to the instructor.
Observation Codes
Potential purposes • Characterize courses: - Student experiences, instructor actions, others • Course Transformations Impact Assessments - After transformation; pre-post if possible • Compare courses … eg. all 1xx; core vs. elective; etc. - Busy / idle students; balance of motivation, basics, context; theory / practical; etc. • Professional and peer development for instructors: - Based on code patterns and observer feedback.
Precedent • EOS – “engagement observation protocol” (E. Lane) - Included in CPOP • UBC - Physics feedback forms (P. Newbury and C. Heiner) • RTOP – Reformed Teaching Observations Protocol • TDOP1 – Teaching Dimensions Observation Protocol1 - Foundation of CPOP - Explored in Math (W. Code)
Trials to date: EOAS courses:
1xx courses: 4 2xx courses: 5 3xx courses: 7 Eosc222: 12 classes, Jan-March
Procedure and data:
• Observation form stabile at version 10.
• Forms printed on LiveScribe paper.
• Audio records are keyed to coding on forms.
• Coded forms transferred to spreadsheet template, including color and aggregation. eosc326, Nov 16th, 2011
DSQEAMP TsmwRHLONESRPC FBMHAO I CQPGWRUAAEC
0 10%
2 20%
4 60%
6 75%
8 0%
10 0%
DSQEAMP T s mwRH L ONESRPC FBMHAO I CQPGWRUAAEC Eng
12 83%
14 69%
16 50%
18 50%
20 0%
DSQEAMP T s mwRH L ONESRPC FBMHAO I CQPGWRUAAEC Eng
22 65%
24 0%
26 0%
DSQEAMP T s mwRH L ONESRPC FBMHAO I CQPGWRUAAEC Eng
28 0%
30 94%
32 0%
34 0%
36 0%
38 0%
40 0%
DSQEAMP T s mwRH L ONESRPC FBMHAO I CQPGWRUAAEC Eng
42 0%
44 0%
46 0%
48 0%
50 0%
EngInst Mode Inst. do. Happen/CoverStu. doingBlooms
eosc326, Nov 18th (active Friday), 2011
DSQEAMP TsmwRHLONESRPC FBMHAO I CQPGWRUAAEC
0
2
4
6
8
10
DSQEAMP T s mwRH L ON ESRPC FBMHAO I CQPGWRUAAEC
12
14
16
18
20
DSQEAMP T s mwRH L ON ESRPC FBMHAO I CQPGWRUAAEC
22
24
26
DSQEAMP T s mwRH L ON ESRPC FBMHAO I CQPGWRUAAEC
28
30
32
34
36
38
40
DSQEAMP T s mwRH L ON ESRPC FBMHAO I CQPGWRUAAEC
42
44
46
48
50
DSQEAMP T s mwRH L ON ESRPC FBMHAO I CQPGWRUAAEC
52
54
56
58
60
DSQEAMP T s mwRH L ON ESRPC FBMHAO I CQPGWRUAAEC
Sumsact
lect
Inst Mode Inst. do. Happen/CoverStu. doingBlooms
Mo
de
Inst
ruct
or
Ha
pp
en
ing
Co
veri
ng
Stu
de
nts
Blo
om
s
Lecture “Active Friday”
Mo
de
Inst
ruct
or
Hap
pe
nin
g C
ove
rin
g
Stu
de
nts
Blo
om
s
T
ime
T
ime
Completed raw data form
Transcribed raw data: 2 classes of eosc326
Data
Characteristics and participation in SEI
Some features to notice:
• The “Not eosc” courses were active & application oriented, but did not use groups or peers much.
• “No SEI”: These involved more delivery, more telling, and more solo time in class.
• “SEI applied” tended to be more active, applying, in groups, and there is also a wider variety.
Data recorded at 2-minute intervals in 1st, 2nd and 3rd yr courses.
0%
50%
100%
eosc
114
eosc
112
eosc
221
eosc
373
eosc
326a
eosc
322
eosc
222a
eosc
222b
eosc
332
eosc
340
eosc
110
eosc
116
eosc
270
eosc
333
eosc
326l
atsc
201
envr
200a
envr
200b
envr
300
S,Q,A
D,E,M
0%
50%
100%
eosc
114
eosc
112
eosc
221
eosc
373
eosc
326a
eosc
322
eosc
222a
eosc
222b
eosc
332
eosc
340
eosc
110
eosc
116
eosc
270
eosc
333
eosc
326l
atsc
201
envr
200a
envr
200b
envr
300
P,C,B
E,S,R,M,H,A
0%
50%
100%
eosc
114
eosc
112
eosc
221
eosc
373
eosc
326a
eosc
322
eosc
222a
eosc
222b
eosc
332
eosc
340
eosc
110
eosc
116
eosc
270
eosc
333
eosc
326l
atsc
201
envr
200a
envr
200b
envr
300
P,G,W
I,C,Q
EOS; SEI applied
Not EOSC
EOS; No SEI
Instructional mode:Active vs Delivery
What happens in class:Applying vs Telling etc.
What students are doing:Pairs/groups vs Solo
Six weeks of EOSC222 classes: What analysis possibilities?
Address questions by combining codes:
Variations versus time:– More group work in Feb.
– “Active” tends to follows basics.
Sort by Bloom’s score:
Score = 1*R+2*U+3*A+4*A+5*E+6*U
- Higher Blooms … more active.
0%
50%
100%
active
deliver
0%
50%
100%
active
deliver
Lower Bloom’s Higher Bloom’s
0%
50%
100%
pairs/grps
solo
Students: solo vs. pairs or groups
Instructional mode
Some Patterns – 19 different classesSorted by Blooms score (see codes, below left)
low Bloom:– mostly 1st, 2nd yr. (BUT see exceptions)– high "telling" instr. Mode– high "explain or review" cover
high Bloom:– mostly 2nd, 3rd yr. (BUT see exceptions)– more students working together– exception is eosc222 poster session
eosc112– high blooms, but…– low engagement
eosc114– low blooms, but…– strong engagement
envr300– high blooms, and..– most diverse instructor doing– All guided group work
eosc340– moderate blooms– very low "explaining“– diverse instr. & stu. doing– engage starts low stays high throughout
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
eo
sc2
22
a
en
vr2
00
b
eo
sc1
14
eo
sc3
32
eo
sc2
70
eo
sc3
33
eo
sc1
16
eo
sc1
10
eo
sc3
22
atsc
20
1
eo
sc2
21
eo
sc3
73
eo
sc3
40
eo
sc3
26
l
eo
sc3
26
a
eo
sc2
22
b
eo
sc1
12
en
vr3
00
en
vr2
00
a
Blooms
C
E
A
A
U
R
0
2
4
6
8
10
12
14
16
18
20
eo
sc2
22
a
en
vr2
00
b
eo
sc1
14
eo
sc3
32
eo
sc2
70
eo
sc3
33
eo
sc1
16
eo
sc1
10
eo
sc3
22
atsc
20
1
eo
sc2
21
eo
sc3
73
eo
sc3
40
eo
sc3
26
l
eo
sc3
26
a
eo
sc2
22
b
eo
sc1
12
en
vr3
00
en
vr2
00
a
Happen / cover
R
S
E
0
2
4
6
8
10
12
14
eo
sc2
22
a
en
vr2
00
b
eo
sc1
14
eo
sc3
32
eo
sc2
70
eo
sc3
33
eo
sc1
16
eo
sc1
10
eo
sc3
22
atsc
20
1
eo
sc2
21
eo
sc3
73
eo
sc3
40
eo
sc3
26
l
eo
sc3
26
a
eo
sc2
22
b
eo
sc1
12
en
vr3
00
en
vr2
00
a
Stu. doing
W
G
P
0
2
4
6
8
10
12
14
16
18
20
eo
sc2
22
a
en
vr2
00
b
eo
sc1
14
eo
sc3
32
eo
sc2
70
eo
sc3
33
eo
sc1
16
eo
sc1
10
eo
sc3
22
atsc
20
1
eo
sc2
21
eo
sc3
73
eo
sc3
40
eo
sc3
26
l
eo
sc3
26
a
eo
sc2
22
b
eo
sc1
12
en
vr3
00
en
vr2
00
a
Instr. mode
M
E
D
0%
20%
40%
60%
80%
100%
0 10 20 30 40 50 60
Pe
rce
nta
ge o
f sa
mp
le w
ho
w
ere
"e
nga
ged
"
Minutes into the class
Engagement: 4 courses, 1 class each
eosc340
eosc114
eosc110
eosc112
Engagement: 4 courses, 1 class each
Comparing two science electives:
D67%S
0%
Q11%
E0%
A22%
M0%
P0%
T0%
Instr. mode, 373
D30%
S7%
Q13%
E0%
A50%
M0%
P0%
T0%
Instr. mode, 340
I46%
C6%
Q8%
P8%
G24%
W8%
Stu. doing 340
I78%
C7%
Q0%
P15%
G0% W
0%
Stu. doing 373
Instructional
mode
Students
doing
EOSC 340
Climate change
EOSC 373
Oceanography II
NOTES and caveats: - Each course was observed only once, and no information is used relating to what students do
outside of class. - Also, the observation protocol has not yet been validated by having several people conduct simultaneous observations. - There may be other shortcomings; the project
is definitely in only the “development” stages.
ABSTRACT: A new observation procedure is being developed aimed at helping characterize instructional practices, student actions, "Bloom's levels" and other classroom characteristics, on a timeline of the class. The intention is to develop a procedure that is useful for characterizing any type of class, rather than to make judgments about "quality" in any sense. Preliminary examples of trials are shown with some possibilities for analysis; including comparisons of several classes from one course, & single classes from 16 different courses.