Baseline Data Evaluation Report
description
Transcript of Baseline Data Evaluation Report
DER-NSW 2010 reporting: Implications from the 2010 data collection
Dr Sarah K. Howard [email protected]
Thursday, May 5th 2011
Evaluation RQs
• How does the DER-NSW program influence teacher pedagogy?
• What is the influence of the DER-NSW program on students’ understanding, skills, and attitudes?
• What are the consequences of the DER-NSW program on students’ educational outcomes?
Response rate
• Student survey A: Total started survey, 17,079 Total completed survey: 13,760 (80.6%)
• Student survey B: Total started survey, 18,659 Total completed survey: 14,692 (78.7%)
• Teacher survey: Total started survey, 4,627 Total completed survey: 3,637 (78.6%)
Distribution across subject areas
Other 16%
English 13%
Mathematics 11%
Science 15%
HSIE 13%
PDHPE 6%
Creative Arts 7%
TAS 17%
Languages 2%
Student gender distribution
Male 47% Female
53%
Aboriginal or Torres Straight Islander origin
25183
1698 400 382
0
5000
10000
15000
20000
25000
30000
No Yes, Aboriginal Yes, Torres Strait Islander
Yes, both Aboriginal and Torres Strait
Islander
.8%
Years in an Australian school
371 689 1373
15436
0
2000
4000
6000
8000
10000
12000
14000
16000
18000
less than 1 year 1 to less than 3 years
3 to 7 years more than 7 years
3.3%
Student access to computers at home
No 5%
Yes 95%
Do you have access to a computer at home?
5%
92%
3%
Is this home computer connected to the internet?
Not connected
Yes connected
I don't know if it is connected
RQ #1, Pedagogy
• Teachers who felt using ICT was ‘fun’ rather than ‘important’ showed: – Positive conceptions of students’ creativity & organization – Beliefs that students showed more understanding when using ICT in the
classroom – They were more likely to use email
• We do see a significant relationship between teachers’ primary subject area and their use of ICT in the classroom – Mathematics is consistently lower on shifts towards student-centred
practices and ICT use – Other subject areas were less consistent
RQ #1, Pedagogy
Note. ‘0’ = Never, ‘4’ = 1-3 times a week, ‘8’ = Many times a day
5.68 6.02 6.15 6.36 6.53 6.67 6.83 6.95
0.00
1.00
2.00
3.00
4.00
5.00
6.00
7.00
8.00
How often do you use a computer when teaching?
Mathematics Languages PDHPE English Creative Arts Science HSIE TAS
Frequency of ICT used in teaching, by subject area
Belief that working with ICT is fun and confidence using laptops
2.94 3.13 3.06 3.23 3.14 3.14 3.17 3.21 2.67 2.77 2.79 2.83 2.87 2.87 2.89 3.07
0 0.5
1 1.5
2 2.5
3 3.5
I think working and teaching with computers is fun I feel confident that I can effectively use the laptop in my classes.
Note. ‘1’ = Strongly disagree, ‘4’ = Strongly agree
RQ #1, Pedagogy
Teachers' beliefs of the importance of working with ICT
3.47
3.56
3.59
3.6
3.6
3.62
3.63
3.65
3.22
3.38
3.37
3.35
3.49
3.43
3.49
3.43
3 3.1 3.2 3.3 3.4 3.5 3.6 3.7
Mathematics
English
PDHPE
Languages
TAS
Science
Creative Arts
HSIE
It is important for students to work with computers It is important to me to work with a computer
Note. ‘1’ = Strongly disagree, ‘4’ = Strongly agree
RQ #1, Pedagogy
2.26 2.36 2.41 2.45 2.53 2.61 2.64 2.71
0.00
0.50
1.00
1.50
2.00
2.50
3.00
3.50
4.00
Mathematics
Creative Arts
Science
English
PDHPE
HSIE
Languages
TAS
Teaching is becoming more student-centred with use of the laptops
Note. ‘1’ = Strongly disagree, ‘4’ = Strongly agree
RQ #1, Pedagogy
• “I’m a bit of a believer of you still need to write your maths solutions in your book because you can’t solve an equation, you can’t write your solution to an equation on the laptop unless you’re really familiar with the equation editor in the program, in the software, but that takes time…I see other subjects using that as a great tool for their learning for English or history and so on. So I think for Maths, yeah, we’re a little bit lower on the scale of usefulness.” (School 1, Teacher 1)
• “I think what I like about the laptops is that gives you a different medium for production in that you do your writing exercises but I know a lot of teachers up in the faculty have been doing things like creating your own posters and TV type advertisements, book trailers, things like that and so it’s getting them to demonstrate their knowledge in different ways and I think that that’s a good thing.” (School 3, Teacher 1)
RQ #1, Pedagogy
RQ #1 & 2, Engagement
2.58
2.65
2.31
2.37
2.62
3.05
3.23
2.94
3.00
3.29
0.00 1.00 2.00 3.00 4.00
Students'work is more creative when they use computers
Using a computer helps students to be more organised
Students' writing seems to improve when they use computers
I feel students understand what they are doing in class better when they
use computers
The more we use computers in class, the more it seems students
enjoy school
Student Teacher
Note. ‘1’ = Strongly disagree, ‘4’ = Strongly agree
Teacher and students' beliefs about ICT and learning outcomes
• Students’ perceptions of the importance of ICT in certain subject areas reflected teachers’ perceptions – e.g. importance in Mathematics
• Students reported more positive perceptions of ICT in learning than teachers: – They reported more use of ICTs in school (could be personal
or in combined classes) – They showed a more positive conception of engagement in
learning when using ICT • Students without or with limited access to computers
at home showed consistently less confidence performing ICT tasks and lower engagement with ICT
RQ #2, Engagement
RQ #2, Engagement
2.50 2.54 2.76 2.88
3.03 3.04 3.07 3.12
0.00
0.50
1.00
1.50
2.00
2.50
3.00
3.50
Students' perceptions of the importance of ICT in the subject areas
Note. ‘1’ = Strongly disagree, ‘4’ = Strongly agree
• “It really depends what I get but I’d say I’d use it at least maybe three times, four times a week…Yeah but it really depends what lessons I have because you know in maths I can’t use it at all because she won’t let us…she says it won’t help us at all…write it down like usual…[So what might be a typical assignment you’d have that you might do at home on your, on your laptops?] Research assignments…I’ve normally got them for history…We’re, we’re doing one on Romans so basically things on Romans, things on like fantasy or myths as well.” (School 4, Student 2)
RQ #1 & 2, Pedagogy
Students' confidence performing ICT practices*
0% 10% 20% 30% 40% 50% 60% 70% 80% 90%
100% D
ownl
oad
imag
es
Cre
ate
Pow
erP
oint
R
esea
rch
info
rmat
ion
Em
ail
Edi
t writ
ten
wor
k W
rite
first
dra
ft O
rgan
ize
wor
k Ta
ke n
otes
D
ownl
oad
pict
ures
C
reat
e m
ultim
edia
S
ocia
l net
wor
k P
ost c
omm
ents
to
Ent
er d
ata
to
Plo
t a g
raph
C
reat
e sp
read
shee
t B
log
Cre
ate
mov
ie
Cre
ate
web
page
P
odca
st
Cre
ate
com
pute
r
*The figure is calculated on how many students felt they could perform this task by themselves or with help
RQ #2, Engagement
RQ #2, Engagement
3.01
3.22
3.40
3.40
3.41
3.43
3.45
3.49
3.54
3.44
3.68
3.79
3.83
3.80
3.85
3.79
3.86
3.87
0.00 0.50 1.00 1.50 2.00 2.50 3.00 3.50 4.00 4.50
Create webpage
Write first draft
Create PowerPoint
Download images
Edit written work
Enter data to spreadsheet
Research information
Download pictures
Access to a computer at home No computer access at home
Students' confidence performing ICT practices & home access
RQ #2, Engagement
3.00
3.23
3.03
3.11
3.14
3.37
3.13
3.23
2.80 2.90 3.00 3.10 3.20 3.30 3.40 3.50
It is very important to me to work with a computer
I think working with a computer is fun
I use a computer because I am very interested
Time goes by quickly when I am working with a computer
Access to a computer at home No computer access at home
Students' engagement with technology & home access
Note. ‘1’ = Strongly disagree, ‘4’ = Strongly agree
• “But I think it comes from the fact that they may have been exposed to computers but they just use them for games and things… they have no access or they don’t access the other programs…Because they’re not familiar with some of the programs and they’re asking you how to – something basic that most kids would know how to do ... And half of them say at the start, ‘Well how do I do that’…Or even they just sort of go, ‘I’d rather not, can I just write it’… And even adding attachments to an email, like a lot of kids will say, ‘How do I attach the file ...’ and I’m always surprised by that.” (School 3, Teacher Focus Group)
RQ #2, Engagement
• “I do believe they’re good because like their – instead of going to computer labs we can stay in class and they’re, they’re just – they’re good to have for research and so anything that you need for like, preparation for school… or something like that and that’s why they’re good but, and it’s just that we were flattered to have them because they’re our own laptops and I guess they’re rather expensive so they’re fun to have…Well they can help with like our assignments because I’ve got like three assignments on here that I would have to do at school and send home for my computer which doesn’t work so it helps me.” (School 4, Student Focus Group)
RQ #2, Engagement
• Case study teachers felt it was too soon to predict if students’ performance on tests would improve
• Some students felt their grades had improved with use of the laptop in class – They felt more engaged in learning – Enjoyed exploration tasks
• Issues seem to exist around having students perform tasks on the laptops, when testing will not be on the laptops
RQ #3, Assessment
For 2011 • RQ#1: Further investigation into differences
between teachers’ personal & pedagogical conceptions of ICT practices in the subject areas (impact on learning & students’ perceptions of ICT)
• RQ#2: Deeper examination of the relationship between students’ home access & engagement with ICT practices, e.g. updated questionnaire items on access
• Investigate the interaction between leadership and ICT up-take, e.g. email communication at the school level