Post on 20-May-2015
description
1
Working with SETU
Stuart Palmer
Institute of Teaching and Learning
2
A context-free SETU comment
“I found that Stuart seemed to be complacent about this unit, his lectures where not worth turning up too, as it involved sitting there listening to him talk from the unit guide, which I did by myself at home anyway.”
3
Student evaluation of teaching (SET)
Work by Ramsden and others in the 1980s (re-)established a link between student perceptions of their learning environment and:
> the level of their engagement with their
studies; (hence, generally,)> the ‘quality’ of their learning/outcomes
4
Student evaluation of teaching
Reflecting the contemporary understanding of the multi-faceted nature of quality and the finding that it is the entirety of the university experience that contributes to the student ‘course experience’, virtually all authors examining the value of the SET process recognise that SET data are only one of many sources of information that should be called upon when evaluating teaching
5
Evaluation of teaching
Other valuable data sources include:> objective measures of student learning (such
as unit marks)> reflective (self) assessment of teaching
performance (what worked, what didn’t)> peer assessment of teaching> student focus groups
6
Evaluation of teaching
> etc. …maybe Facebook?
7
Interpreting SETU
USA Today has come out with a new survey - apparently, three out of every four people make up 75% of the population
David Letterman
8
Interpreting SETU
> a reasonable number of responses;> a reasonable overall response rate; and > a reasonably equal rate of responses
between campuses
…provide some confidence that the survey sample is representative
9
Consistency
> a small standard deviation on a mean item rating estimate indicates that most respondents were in agreement
> a large positive difference between the proportion estimates of agreement and disagreement indicates that substantially more students agreed
> if the written open-ended comments reveal common, repeated themes, these are worthy of further investigation
> etc.
10
Discrepancy
> a small number of ratings that are different from the majority can influence the reported mean estimate
> a small number of adverse comments that seem at odds with the majority of written feedback are probably not worth losing a lot of sleep over
> a rating in one item that is significantly different from the others in the SETU results for the same unit might be an indicator worth investigating further
> etc.
11
Interpreting SETU – in general
SETU is only one measure of the ‘quality’ of teaching
SETU is a pretty blunt instrument
Used in an rational manner, the quantitative and qualitative results from SETU can be an aid in the evaluation of teaching and learning
12
Interpreting SETU – in general
The larger the number of responses, the more reliable the data
Trends (as long as they are real) can be informative
13
Mean SETU responses 2003-2004
2.5
3
3.5
4
4.5
1 2 5 6 17 18
SETU question
Mea
n ra
ting
2003(n=51, 17%)
2004(n=73, 42%)
2005(n=47, 32%)
2006(n=32, 37%)
1 The teaching staff of this unit stimulated my interest in further learning.2 The teaching staff of this unit motivated me to do my best work.5 This unit was well taught.6 I had a clear idea of what was expected me in this unit.17 I would recommend this unit to other students.18 The use of on-line technologies in this unit enhanced my learning experience.
14
Mean SETU responses 2003-2005
1 The teaching staff of this unit stimulated my interest in further learning.2 The teaching staff of this unit motivated me to do my best work.5 This unit was well taught.6 I had a clear idea of what was expected me in this unit.17 I would recommend this unit to other students.18 The use of on-line technologies in this unit enhanced my learning experience.
2.5
3
3.5
4
4.5
1 2 5 6 17 18
SETU question
Mea
n ra
ting
2003(n=51, 17%)
2004(n=73, 42%)
2005(n=47, 32%)
2006(n=32, 37%)
15
Mean SETU responses 2003-2005
1 The teaching staff of this unit stimulated my interest in further learning.2 The teaching staff of this unit motivated me to do my best work.5 This unit was well taught.6 I had a clear idea of what was expected me in this unit.17 I would recommend this unit to other students.18 The use of on-line technologies in this unit enhanced my learning experience.
2.5
3
3.5
4
4.5
1 2 5 6 17 18
SETU question
Mea
n ra
ting
2003(n=51, 17%)
2004(n=73, 42%)
2005(n=47, 32%)
2006(n=32, 37%)
16
Mean SETU responses 2003-2005
1 The teaching staff of this unit stimulated my interest in further learning.2 The teaching staff of this unit motivated me to do my best work.5 This unit was well taught.6 I had a clear idea of what was expected me in this unit.17 I would recommend this unit to other students.18 The use of on-line technologies in this unit enhanced my learning experience.
2.5
3
3.5
4
4.5
1 2 5 6 17 18
SETU question
Mea
n ra
ting
2003(n=51, 17%)
2004(n=73, 42%)
2005(n=47, 32%)
2006(n=32, 37%)
17
Mean SETU responses 2003-2005
1 The teaching staff of this unit stimulated my interest in further learning.2 The teaching staff of this unit motivated me to do my best work.5 This unit was well taught.6 I had a clear idea of what was expected me in this unit.17 I would recommend this unit to other students.18 The use of on-line technologies in this unit enhanced my learning experience.
2.5
3
3.5
4
4.5
1 2 5 6 17 18
SETU question
Mea
n ra
ting
2003(n=51, 17%)
2004(n=73, 42%)
2005(n=47, 32%)
2006(n=32, 37%)
18
Mean SETU responses 2003-2005
1 The teaching staff of this unit stimulated my interest in further learning.2 The teaching staff of this unit motivated me to do my best work.5 This unit was well taught.6 I had a clear idea of what was expected me in this unit.17 I would recommend this unit to other students.18 The use of on-line technologies in this unit enhanced my learning experience.
2.5
3
3.5
4
4.5
1 2 5 6 17 18
SETU question
Mea
n ra
ting
2003(n=51, 17%)
2004(n=73, 42%)
2005(n=47, 32%)
2006(n=32, 37%)
19
Mean SETU responses 2003-2006
2.5
3
3.5
4
4.5
1 2 5 6 17 18
SETU question
Mea
n ra
ting
2003(n=51, 17%)
2004(n=73, 42%)
2005(n=47, 32%)
2006(n=32, 37%)
1 The teaching staff of this unit stimulated my interest in further learning.2 The teaching staff of this unit motivated me to do my best work.5 This unit was well taught.6 I had a clear idea of what was expected me in this unit.17 I would recommend this unit to other students.18 The use of on-line technologies in this unit enhanced my learning experience.
20
SETU teaching comments
“I found that Stuart seemed to be complacent about this unit, his lectures where not worth turning up too, as it involved sitting there listening to him talk from the unit guide, which I did by myself at home anyway.”
21
SETU teaching comments
> An exceptionally hardworking teacher. Always responding promptly to enquiries. Thank you Stuart for your efforts.
> Excellent teaching of course work> Excellent teaching, great feedback, showed interest
in progress. Best I have come across.> Good lecturer.> Great course content and well delivered
22
SETU teaching comments
> It was good how Stuart used videos to illustrate things in the real world. His lectures are well structured as all the unit material is covered in lecturers which are always informative. It was nice to get the reminder email about assignment1 and 3, this shows he has compassion towards the students.
> The lectures were good and didnt drag on too much.> The teaching support offered as part of this unit was
excellent.
23
SETU teaching comments
> This unit was very well taught. I could not fault any part of it.
> Very easy to listen to, and teaches well> Very promptuos lecturer> Well structured class.> very interesting to listen to and knows how to teach.
was very helpful to me when i had questions> very well taught, easily understood, madde the unit
enjoyable.
24
SETU teaching comments
> Yes stuart palmer is dedicated lecturer who is willing to help and puts an effort into helping students. His lectures cover alot of content with plenty of examples.
> Good uses of informations and extra material to help get a point across
> sometimes it feels very long to has class for 2 hours.
25
2
2.5
3
3.5
4
4.5
5
Well t
augh
t
Mat
erial
s
Workl
oad
Require
ments
Feedba
ck
Libra
ry
Recom
men
d
Online
OK
O/l enh
ance
d
Faculty
School
Prereq
Similar
Unit
26
2
2.5
3
3.5
4
4.5
5
Well t
augh
t
Mat
erial
s
Workl
oad
Require
ments
Feedba
ck
Libra
ry
Recom
men
d
Online
OK
O/l enh
ance
d
Faculty
School
Prereq
Similar
Unit
27
2
2.5
3
3.5
4
4.5
5
Well t
augh
t
Mat
erial
s
Workl
oad
Require
ments
Feedba
ck
Libra
ry
Recom
men
d
Online
OK
O/l enh
ance
d
Faculty
School
Prereq
Similar
Unit
28
2
2.5
3
3.5
4
4.5
5
Well t
augh
t
Mat
erial
s
Workl
oad
Require
ments
Feedba
ck
Libra
ry
Recom
men
d
Online
OK
O/l enh
ance
d
Faculty
School
Prereq
Similar
Unit
29
2
2.5
3
3.5
4
4.5
5
Well
taug
ht
Mate
rials
Wor
kload
Requir
emen
ts
Feedb
ack
Libra
ry
Recom
men
d
Online
OK
O/l enh
ance
d
Faculty
School
Prereq
Similar
Unit
30
2
2.5
3
3.5
4
4.5
5
Well
taug
ht
Mate
rials
Wor
kload
Requir
emen
ts
Feedb
ack
Libra
ry
Recom
men
d
Online
OK
O/l enh
ance
d
Faculty
School
Prereq
Similar
Unit
31
2
2.5
3
3.5
4
4.5
5
Well
taug
ht
Mate
rials
Wor
kload
Requir
emen
ts
Feedb
ack
Libra
ry
Recom
men
d
Online
OK
O/l enh
ance
d
Faculty
School
Prereq
Similar
Unit
32
Well
taug
ht
Materia
ls
Wor
kload
Requir
emen
ts
Feedb
ack
Libra
ry
Recom
mend
O/l enh
ance
d1.5
2
2.5
3
3.5
4
2005
2006
2007
33
Well
taug
ht
Materia
ls
Wor
kload
Requir
emen
ts
Feedb
ack
Libra
ry
Recom
mend
O/l enh
ance
d1.5
2
2.5
3
3.5
4
2005
2006
2007
34
Well
taug
ht
Materia
ls
Wor
kload
Requir
emen
ts
Feedb
ack
Libra
ry
Recom
mend
O/l enh
ance
d1.5
2
2.5
3
3.5
4
2005
2006
2007
35
Using SETU
The circumstances, context and influences on a particular unit are individual in nature
Standard, boilerplate responses probably aren’t that useful
SETU might offer some insights of value
36
Online PD materialswww.deakin.edu.au/itl/pd/tl-modules/scholarly