Progress Report: EIA Phase IV Studies - eiabd.com … · · 2016-08-24Progress Report: EIA Phase...
Transcript of Progress Report: EIA Phase IV Studies - eiabd.com … · · 2016-08-24Progress Report: EIA Phase...
2
CONTENTS
Headlines ................................................................................................................... 3
Programme Activities and Timeline Update ............................................................... 4
Quasi-Experimental Study Sample and Preliminary Data Analysis ............................ 5
Qualitatative Study Sample .................................................................................... 123
Appendix 1: QE study data analysis tables for English Language Proficiency Study ......... 155
Appendix 2: QE study data analysis tables for Classroom Practice Study ............. 277
Appendix 3: Qualitative Study Field Protocol & Guidance Notes ........................... 333
English in Action (2015). Progress Report: EIA Phase IV Research Studies. Technical report, English in Action, Dhaka, Bangladesh. Author: Tom Power, The Open University Acknowledgements We would like to thank the researchers from the Institute of Education and Research (IER), University of Dhaka and also assessors from Trinity College London (www.trinitycollege.com/) for their support in the collection and entry of data for this study.We also acknowledge the EIA RME team for support during the planning, fieldwork, data entry and cleaning. For more information about English in Action, visit: http://www.eiabd.com/ © English in Action, 2015 Published by English in Action (EIA) in Dhaka, Bangladesh. English in Action House 1, Road 80, Gulshan 2 Dhaka-1212, Bangladesh. Phone: 88-02 8822161, 8822234 Fax: 88-02 8822663 Email: [email protected]
3
Headlines
1. The first round of qualitative fieldwork has been completed and is in the process of being written up. Quantitative pre-test fieldwork and initial data analysis has been completed.
2. Field Operations for Teacher Development and Research Activities are all proceeding in line with the schedule shared in the proposal, July 2015. No significant amendments to the planned timeline are anticipated at this time.
3. Qualitative study field operations had to be carried out under significantly enhanced security protocols, due to recent shootings of foreign nationals. This and the end-of year exams have constrained access to schools,limiting those available for sampling and significantly increasing the time required for finalising fieldwork logistics.None-the-less, the first round of fieldwork has been completed on schedule.
4. The actual quantitative pre-test sample achieved was very close to the planned
sample. Comparison of control and treatment groups show no statistically significant differences on any major indicator overall. There are some minor statistically significant differences when samples are broken down further (e.g. to divisional level; with secondary male students, but not female or overall secondary students), but these are probably of little educational significance.
5. Pre-test descriptive statistics show English Language Proficiency levels similar to the
2009-2010 baseline.
6. Pre-test descriptive statistics on Classroom Practice show high levels of student talk-time and English language use by teachers and students; however, these do not seem to be associated withhigh level learning outcomes (see 4). Teacher talk is high (54% primary; 62% secondary), with presenting being the most common talk-type. There are low-levels of student-student talk (e.g. pair or group talk) and students spend only around a third of the lessonpracticing the four skills (35% in primary; 28% in secondary) other than listening to teacher.
7. We are considering whether the second round of the qualitative study fieldwork might
be enhanced to allow more insight into the quality of classroom practices, as well as the English-language and socio-economic contexts, of the qualitative study schools. These would be in addition the primary focus on the experiences of support in school.
4
Programme Activities and Timeline Update The implementation of field operations for the teacher development programme and associated research activities (quantitative and qualitative) have occurred in-line with timeline submitted in the proposal (July 2015) (see fig 1below). Pre-test fieldwork was completed by 31’ July, as per budget milestone. Data entry, cleaning and analysis took longer than anticipated, with initial analysis of pre-test data being completed by the end of November, 2015. Qualitative fieldwork was completed as planned, in October and November 2015. Fig1.The planned timeline
A spate of shootings of foreign nationals in Bangladesh in the fourth quarter of 2015 meant field operations for the qualitative study had to take place under enhanced security protocols, which severely constrained freedom of movement, including some international staff having to travel under armed police guard during fieldwork (see fig 2). Security and travel restrictions severely constrained the schools that could be considered for the qualitative sample and absorbed substantial time and human capacity in planning.
Figure 2. Armed Police Escort for Field Visit to Schools, Qualitative Study, November 2015.
Remaining teacher development activities, qualitative fieldwork and quantitative post-test field activities are anticipated to take place as per the timeline, without significant revisions.
5
Quasi-Experimental Study Sample and Preliminary Data Analysis Sample
The sample design was to sample some 240 teachers and 2,400 of their students, from 120 schools, with the same teachers and students taking part in both pre- and post- test. To compensate for student attrition, it was planned to test an additional 1,200 students at pre-test.
Fig. 2 The planned sample
Pre-intervention Post-intervention
Primary Schools Teachers Students Schools Teachers Students
Intervention group
30 60 900 30 60 600+
Control group 30 60 900 30 60 600+
Secondary
Intervention group
30 60 900 30 60 600+
Control group 30 60 900 30 60 600+
Total 120 240 3,600 120 240 2,400+ Fig 3. The actual pre-test sample achieved1
Actual Pre-test Sample
Primary Schools Teachers Students
Intervention group
30 59 888
Control group 30 59 880
Secondary
Intervention group
30 59 892
Control group 27 54 823
Total 117 231 3, 483
PreliminaryData Analysis
Task 1: Statistical comparison of control and treatment groups The primary statistical analysis was to compare the control and treatment groups, checking for any statistically significant differences between them. Detailed tables are provided in Appendix 1.
English Language Proficiency: There were no statistically significant differences (P<0.05) found between control and treatment groups, for:
Primary Students
Primary Student by gender
Primary Teachers
Primary Teachers by gender
Secondary Students
Female Secondary students
Secondary School Teachers
1 From initial data-analysis after cleaning.
6
Secondary School Teachers by gender There are small but statistically significant differences (P<0.05) between control and treatment groups English language proficiency, when considering:
Male secondary students (but not female, or secondary students overall). The pass rate is lower for male secondary students in the treatment group (75%) compared to control group (80%) (See Table 7. Secondary School Students ELP disaggregated by Gender, Appendix 1)
Individual divisions apart from Dhaka (primary & secondary), Syhlet (Primary) and Barishal (secondary) all show small but statistically significant differences between control and treatment groups for students, although there are no statistically significant differences over-all2. (See Table 3. Primary Students ELP disaggregated
by Division and Table 8. Secondary Students ELP disaggregated by Division, Appendix 1).
For teachers’ ELP, the sample size is too small when disaggregated by division, to allow for tests of significant differences.
Classroom Practices There were no statistically significant differences (P<0.05) between control and treatment groups in either primary or secondary classrooms, for:
Proportions of Teacher Talk in English / Bangla
Proportions of Student Talk in English / Bangla
Proportions of type of activity in classrooms (Teacher Talk, Student talk, Student Reading, Writing, Listening; Other activity).
Types of teacher talk (Presenting, Organising, Giving Feedback, Asking Questions), or the proportion of English / Bangla used within most types of teacher talk.
Most types of student talk (Single, Pair, Group, Chorus), or the proportion of English / Bangla used within each type of teacher talk. (Instances of pair and group work were very few for statistical purposes).
In primary classrooms, there were no statistically significant differences identified between control and treatment groups on any indicator. In secondary classrooms, there were some statistically significant differences (P<0.05) between control and treatment group classroom practices:
Treatment group secondary teachers used more English when giving feedback and asking questions, compared to control group secondary teachers (See Appendix 2, Table 10). (The proportion of these types of teacher talk was not statistically different (Table 9), but the balance of language used in these two types of teacher talk was statistically significant. However, there was no difference in the overall balance of talk in English / Bangla by secondary teachers in control or treatment groups (Table 8), just in the use of English and Bangla for these two types of talk).
Secondary student’s participation in choral speaking was higher in the control group (33%) than the treatment group (18%).(See Appendix 2, Table 11).
2 Previous EIA studies have often shown different patterns from one division to another… the UNICEF deprivation study found
deprivation to be relatively evenly distributed at divisional level, but increasingly varied at district and upazila level and below. This might suggest that the smaller sample sizes at divisional levels are prone to sampling effects that average out at the national level. See UNICEF (2010). A case for geographic targeting and basic social service to mitigate inequalities in Bangladesh. Technical report.
7
Summary of Statistical Analysis comparing Control and Treatment Pre-test data. The two groups are very well matched; for almost all indicators, there are no statistically significant differences between the two groups. However, when sample sizes are reduced, for example when looking only at particular divisions, some statistically significant differences emerge.
Task 2: Descriptive Statistics Preliminary draft analysis of descriptive statistics is provided below.
English Language Proficiency
Figure 4. Primary Students GESE Scores
Over half of the pre-test primary students did not achieve level 1 or above on the GESE scale. A third of pre-test primary students achieved level 1, and one-in-ten students achieved a level 2 or 3. A very small number of students (less than half a percent) achieved levels 4 or 5.
Figure 5. Secondary Students GESE Scores
Over a quarter of secondary students did not achieve level 1 or above on the GESE scale. Almost a third achieved level one, and a further third achieved levels 2-4. Few (around 1-in-20) students achieved above level 4.
57
33
6 40 0
0
10
20
30
40
50
60
0 1 2 3 4 5
Per
cen
tage
of
Stu
den
ts a
chie
vin
g gr
ade
GESE Grade
Primary Student GESE scores
28%35%
15%11%
5%2% 2% 1% 1% 0%
0%
10%
20%
30%
40%
0 1 2 3 4 5 6 7 8 9
Per
cen
tage
of
stu
den
ts
ach
ievi
ng
grad
e
GESE Grades
Secondary Student GESE Scores
8
Figure 6: Primary Teachers GESE Scores
Almost two-thirds of primary teachers achieved grades 1-3; a third achieved grades 4-6; very few (a little over one-in-twenty) achieved levels 8 or 9. A very few (one-in-twenty) failed to achieve grade 1. Figure 7. Secondary Teachers
Over a third of secondary teachers achieved grades 1-3. Almost half of secondary teachers achieved grades 4-6, and a tenth achieved grades 7-8. Very few (one-in-twenty) achieved grade 9 or 10.
2%
21%
15%
23%
12% 11% 10%
4%2%
0%
5%
10%
15%
20%
25%
0 1 2 3 4 5 6 7 8Per
cen
tage
of
teac
her
s ac
hie
vin
g gr
ade
GESE Grade
Primary Teacher GESE Scores
1%3%
8%
25%
18%15% 15%
7%
3% 2% 3%
0%
5%
10%
15%
20%
25%
30%
0 1 2 3 4 5 6 7 8 9 10
Per
cen
tage
of
teac
her
s ac
hie
vin
g gr
ade
GESE Grades
Secondary Teacher GESE Scores
9
Classroom Practices: Primary
Figure 8: Primary Students’ Use of English and Bangla
Figure 9: Primary Teachers’ Use of English & Bangla
Figure 10: Primary Classroom Activity
13%
87%
Primary Student Talk
Bangla English
30%
70%
Primary Teacher Talk
Bangla English
53%
23%
5%
5%
1%
13%
Primary Classroom Activity
teacher talk student talk ss reading ss writing ss listening other
10
Figure 11: Primary Teacher Talk by Type
Figure 12: Primary Student Talk by Type
43
23
17
17
Primary Teacher Talk Types
presenting organising feedback questions
40
46
50
Primary student talk types
individual pair group choral
11
Classroom Practices: Secondary
Figure 12: Secondary Students’ Use of English and Bangla
Figure 13: Secondary Teachers’ Use of English & Bangla
Figure 14: Secondary Classroom Activity
22%
78%
Secondary Student Talk
Bangla English
38%
62%
Secondary Teacher Talk
Bangla English
62%18%
3%
3%
3%
11%
Secondary Classroom Activity
teacher talk student talk ss reading ss writing ss listening other
12
Figure 15:Secondary Teacher Talk by Type
Figure 16: Secondary Student Talk by Type
53
15
12
20
Secondary Teacher Talk Types
presenting organising feedback questions
60
4.5
9.5
26
Secondary Student Talk Types
individual pair group choral
13
Qualitative Study Sample
Planned Sample 1. Total: 8 schools: 16 teachers & 8 head teachers. 2. 4 Control schools, 4 Treatment schools 3. Each phase: 2 'high' and 2 'low' performing schools
High performing ~ 75th percentile (+/- 5 points) for student talk and student learning outcomes
Low performing ~ 25th percentile (+/- 5 points) for student talk and student learning outcomes
School Pre-test performance
High Low
Experimental study
control 1 primary; 1 secondary
1 primary; 1 secondary
treatment 1 primary; 1 secondary
1 primary; 1 secondary
Design for Achieving Sample For student talk, we decided to count ‘% of lesson time with student talk in English’ as a variable representing both our key student talk indicators: student talk-time and % of student talk in English. Student talk in English was calculated for classes and then schools, to give a ‘classroom practice’ (CP) indicator for the school. Schools were then ranked by this indicator, and assigned a percentile position. Student GESE scores were calculated for classes and then schools, and schools were then ranked by this school GESE indicator, and assigned a percentile position. Our ideal sample would have been to find identify schools such that:
High performing = 70th-80th percentile in both CP and GESE indicators
Low performing = 20th-30th percentile, in both CP and GESE However, the percentile ranges had to be widened substantially, as we also had to identify at least two schools that were sufficiently co-located to allow a field team to visit them on consecutive days. The sampling was further constrained by limited access to schools during the revision and examinations period, and by travel limitations imposed upon the team as a result of enhanced security protocols due to recent shootings of foreign nationals.Although we had intended to select based upon GESE and CP indicators, where the practical constraints prevented this, we have privileged the selection of the student learning outcomes indicator (GESE) over the classroom practice indicator (CP), in selecting schools for the study.
14
Actual Sample
Phase Performance (GESE& CP)
Treatment (EIA) School
Control (Non-EIA) School
Primary Hi Hathazari-CTG South BuircharAlamia G.P.S 01913630564 GESE: 81st Percentile CP: 72nd Percentile
Hathazari-CTG North Buirchar Rashida G.P.S 01554326046 GESE: 72nd Percentile CP: 19thPercentile
Lo Khulna Binapani GPS 01966017427 GESE: 44th Percentile CP: 14th Percentile
Khulna Krishi College PS 01199340814 GESE: 4th Percentile CP: 42nd Percentile
Secondary Hi Rajshahi Govt PN Girls High School, GESE: 98th Percentile CP: 95th Percentile
Rajshahi Satellite Town High, GESE: 62nd Percentile CP: 22nd Percentile
Lo Khulna Shibrampur High School, GESE: 9th Percentile CP: 36th Percentile
Dhaka Algi High School, GESE: 25th Percentile CP: 4th Percentile
15
Appendix 1: QE study data analysis tables for English Language Proficiency Study Data Analysis by Dr.Nai Li, the Open University.
Primary Schools: Comparison of Control & Treatment Groups
Table 1. Primary School Students ELP
Group Treatment or Control group * GESEgrade
GESEgrade
Total 0 1 2 3 4 5
Group Treatment or Control group
1.00 Treatment group
486 316 42 36 4 4 888
54.7% 35.6% 4.7% 4.1% .5% .5% 100.0%
2.00 Control group
514 269 63 28 3 3 880
58.4% 30.6% 7.2% 3.2% .3% .3% 100.0%
Total 1000 585 105 64 7 7 1768
56.6% 33.1% 5.9% 3.6% .4% .4% 100.0%
There is no significant difference between the treatment and control group. P=.075.
16
Table 2. Primary School Students ELP disaggregated by Gender
Group Treatment or Control group * GESEgrade * Gender
Gender
GESEgrade
Total 0 1 2 3 4 5
Female Group Treatment or Control group
1.00 Treatment group
273 173 26 17 1 1 491
55.6% 35.2% 5.3% 3.5% .2% .2% 100.0%
2.00 Control group
285 149 38 16 0 0 488
58.4% 30.5% 7.8% 3.3% 0.0% 0.0% 100.0%
Total 558 322 64 33 1 1 979
57.0% 32.9% 6.5% 3.4% .1% .1% 100.0%
Male Group Treatment or Control group
1.00 Treatment group
213 143 16 19 3 3 397
53.7% 36.0% 4.0% 4.8% .8% .8% 100.0%
2.00 Control group
229 120 25 12 3 3 392
58.4% 30.6% 6.4% 3.1% .8% .8% 100.0%
Total 442 263 41 31 6 6 789
56.0% 33.3% 5.2% 3.9% .8% .8% 100.0%
Total Group Treatment or Control group
1.00 Treatment group
486 316 42 36 4 4 888
54.7% 35.6% 4.7% 4.1% .5% .5% 100.0%
2.00 Control group
514 269 63 28 3 3 880
58.4% 30.6% 7.2% 3.2% .3% .3% 100.0%
Total 1000 585 105 64 7 7 1768
56.6% 33.1% 5.9% 3.6% .4% .4% 100.0%
There is no significant difference in term of gender difference between the treatment and control group. P=.276; P=.295
17
Table 3. Primary Students ELP disaggregated by Division
Group Treatment or Control group * GESEgrade * Division Crosstabulation
Division
GESEgrade
Total 0 1 2 3 4 5
Barishal Group Treatment or Control group
1.00 Treatment group
28 27 1 56
50.0% 48.2% 1.8% 100.0%
2.00 Control group
47 5 0 52
90.4% 9.6% 0.0% 100.0%
Total 75 32 1 108
69.4% 29.6% .9% 100.0%
Chittagong Group Treatment or Control group
1.00 Treatment group
46 95 7 1 149
30.9% 63.8% 4.7% .7% 100.0%
2.00 Control group
70 79 1 0 150
46.7% 52.7% .7% 0.0% 100.0%
Total 116 174 8 1 299
38.8% 58.2% 2.7% .3% 100.0%
Dhaka Group Treatment or Control group
1.00 Treatment group
120 71 18 25 2 3 239
50.2% 29.7% 7.5% 10.5% .8% 1.3% 100.0%
2.00 Control group
111 66 40 16 2 2 237
46.8% 27.8% 16.9% 6.8% .8% .8% 100.0%
Total 231 137 58 41 4 5 476
48.5% 28.8% 12.2% 8.6% .8% 1.1% 100.0%
Khulna Group Treatment or Control group
1.00 Treatment group
96 24 120
80.0% 20.0% 100.0%
2.00 Control group
108 5 113
95.6% 4.4% 100.0%
Total 204 29 233
87.6% 12.4% 100.0%
18
Rajshahi Group Treatment or Control group
1.00 Treatment group
50 53 9 6 2 120
41.7% 44.2% 7.5% 5.0% 1.7% 100.0%
2.00 Control group
25 64 20 10 1 120
20.8% 53.3% 16.7% 8.3% .8% 100.0%
Total 75 117 29 16 3 240
31.3% 48.8% 12.1% 6.7% 1.3% 100.0%
Rangpur Group Treatment or Control group
1.00 Treatment group
71 38 7 4 1 121
58.7% 31.4% 5.8% 3.3% .8% 100.0%
2.00 Control group
85 36 0 1 0 122
69.7% 29.5% 0.0% .8% 0.0% 100.0%
Total 156 74 7 5 1 243
64.2% 30.5% 2.9% 2.1% .4% 100.0%
Sylhet Group Treatment or Control group
1.00 Treatment group
75 8 0 0 0 83
90.4% 9.6% 0.0% 0.0% 0.0% 100.0%
2.00 Control group
68 14 2 1 1 86
79.1% 16.3% 2.3% 1.2% 1.2% 100.0%
Total 143 22 2 1 1 169
84.6% 13.0% 1.2% .6% .6% 100.0%
Total Group Treatment or Control group
1.00 Treatment group
486 316 42 36 4 4 888
54.7% 35.6% 4.7% 4.1% .5% .5% 100.0%
2.00 Control group
514 269 63 28 3 3 880
58.4% 30.6% 7.2% 3.2% .3% .3% 100.0%
Total 1000 585 105 64 7 7 1768
56.6% 33.1% 5.9% 3.6% .4% .4% 100.0%
There are significant differences between the treatment and control groups in most of divisions, except Dhaka, Sylhet
19
Table 4. Primary Teachers’ ELP
Group Treatment or Control group * GESEGrade012
GESEGrade012 Total
0 1 2 3 4 5 6 7 8
Group Treatment or Control group
1.00 Treatment group
0 10 8 18 7 7 8 1 0 59
0.0% 16.9% 13.6% 30.5% 11.9% 11.9% 13.6% 1.7% 0.0% 100.0%
2.00 Control group
2 15 10 9 7 6 4 4 2 59
3.4% 25.4% 16.9% 15.3% 11.9% 10.2% 6.8% 6.8% 3.4% 100.0%
Total 2 25 18 27 14 13 12 5 2 118
1.7% 21.2% 15.3% 22.9% 11.9% 11.0% 10.2% 4.2% 1.7% 100.0%
There is no significant difference between the two groups.P=.178
20
Table 5. Primary Teachers’ ELP disaggregated by Gender
Group Treatment or Control group * GESEGrade012 * Gender
Gender
GESEGrade012
Total 0 1 2 3 4 5 6 7 8
Female Group Treatment or Control group
1.00 Treatment group
0 6 6 9 5 4 5 0 0 35
0.0% 17.1% 17.1% 25.7% 14.3% 11.4% 14.3% 0.0% 0.0% 100.0%
2.00 Control group
1 13 8 6 4 5 1 1 1 40
2.5% 32.5% 20.0% 15.0% 10.0% 12.5% 2.5% 2.5% 2.5% 100.0%
Total 1 19 14 15 9 9 6 1 1 75
1.3% 25.3% 18.7% 20.0% 12.0% 12.0% 8.0% 1.3% 1.3% 100.0%
Male Group Treatment or Control group
1.00 Treatment group
0 4 2 9 2 3 3 1 0 24
0.0% 16.7% 8.3% 37.5% 8.3% 12.5% 12.5% 4.2% 0.0% 100.0%
2.00 Control group
1 2 2 3 3 1 3 3 1 19
5.3% 10.5% 10.5% 15.8% 15.8% 5.3% 15.8% 15.8% 5.3% 100.0%
Total 1 6 4 12 5 4 6 4 1 43
2.3% 14.0% 9.3% 27.9% 11.6% 9.3% 14.0% 9.3% 2.3% 100.0%
Total Group Treatment or Control group
1.00 Treatment group
0 10 8 18 7 7 8 1 0 59
0.0% 16.9% 13.6% 30.5% 11.9% 11.9% 13.6% 1.7% 0.0% 100.0%
2.00 Control group
2 15 10 9 7 6 4 4 2 59
3.4% 25.4% 16.9% 15.3% 11.9% 10.2% 6.8% 6.8% 3.4% 100.0%
Total 2 25 18 27 14 13 12 5 2 118
1.7% 21.2% 15.3% 22.9% 11.9% 11.0% 10.2% 4.2% 1.7% 100.0%
There is no significant difference between the two groups by gender. P=.337, p=.496
21
Secondary Schools: Comparison of Control & Treatment Groups
Table 6. Secondary School Students ELP
Group Treatment or Control group * GESEgrade
GESEgrade
Total 0 1 10 2 3 4 5 6 7 8 9
Group Treatment or Control group
1.00 Treatment group
253 326 1 129 97 46 13 14 7 6 0 892
28.4% 36.5% .1% 14.5% 10.9% 5.2% 1.5% 1.6% .8% .7% 0.0% 100.0%
2.00 Control group
219 278 4 134 99 46 19 13 5 2 4 823
26.6% 33.8% .5% 16.3% 12.0% 5.6% 2.3% 1.6% .6% .2% .5% 100.0%
Total 472 604 5 263 196 92 32 27 12 8 4 1715
27.5% 35.2% .3% 15.3% 11.4% 5.4% 1.9% 1.6% .7% .5% .2% 100.0%
There is no significant difference between the two groups. P=.228
22
Table 7. Secondary School Students ELP disaggregated by Gender
Group Treatment or Control group * GESEgrade * Gender
Gender
GESEgrade
Total 0 1 10 2 3 4 5 6 7 8 9
Female Group Treatment or Control group
1.00 Treatment group
163 169 1 72 61 31 10 9 6 4 526
31.0% 32.1% .2% 13.7% 11.6% 5.9% 1.9% 1.7% 1.1% .8% 100.0%
2.00 Control group
144 148 0 69 50 24 7 3 0 0 445
32.4% 33.3% 0.0% 15.5% 11.2% 5.4% 1.6% .7% 0.0% 0.0% 100.0%
Total 307 317 1 141 111 55 17 12 6 4 971
31.6% 32.6% .1% 14.5% 11.4% 5.7% 1.8% 1.2% .6% .4% 100.0%
Male Group Treatment or Control group
1.00 Treatment group
90 157 0 57 36 15 3 5 1 2 0 366
24.6% 42.9% 0.0% 15.6% 9.8% 4.1% .8% 1.4% .3% .5% 0.0% 100.0%
2.00 Control group
75 130 4 65 49 22 12 10 5 2 4 378
19.8% 34.4% 1.1% 17.2% 13.0% 5.8% 3.2% 2.6% 1.3% .5% 1.1% 100.0%
Total 165 287 4 122 85 37 15 15 6 4 4 744
22.2% 38.6% .5% 16.4% 11.4% 5.0% 2.0% 2.0% .8% .5% .5% 100.0%
Total Group Treatment or Control group
1.00 Treatment group
253 326 1 129 97 46 13 14 7 6 0 892
28.4% 36.5% .1% 14.5% 10.9% 5.2% 1.5% 1.6% .8% .7% 0.0% 100.0%
2.00 Control group
219 278 4 134 99 46 19 13 5 2 4 823
26.6% 33.8% .5% 16.3% 12.0% 5.6% 2.3% 1.6% .6% .2% .5% 100.0%
Total 472 604 5 263 196 92 32 27 12 8 4 1715
27.5% 35.2% .3% 15.3% 11.4% 5.4% 1.9% 1.6% .7% .5% .2% 100.0%
There is significant difference within the male group. P=.005
23
Table 8. Secondary Students ELP disaggregated by Division
Group Treatment or Control group * GESEgrade * Division Crosstabulation
Division
GESEgrade
Total 0 1 10 2 3 4 5 6 7 8 9
Barishal Group Treatment or Control group
1.00 Treatment group
11 48 21 7 1 88
12.5% 54.5% 23.9% 8.0% 1.1% 100.0%
2.00 Control group
30 56 17 8 0 111
27.0% 50.5% 15.3% 7.2% 0.0% 100.0%
Total 41 104 38 15 1 199
20.6% 52.3% 19.1% 7.5% .5% 100.0%
Chittagong Group Treatment or Control group
1.00 Treatment group
34 57 28 31 3 0 1 154
22.1% 37.0% 18.2% 20.1% 1.9% 0.0% .6% 100.0%
2.00 Control group
3 36 40 45 18 4 3 149
2.0% 24.2% 26.8% 30.2% 12.1% 2.7% 2.0% 100.0%
Total 37 93 68 76 21 4 4 303
12.2% 30.7% 22.4% 25.1% 6.9% 1.3% 1.3% 100.0%
Dhaka Group Treatment or Control group
1.00 Treatment group
85 75 42 25 14 5 6 4 4 260
32.7% 28.8% 16.2% 9.6% 5.4% 1.9% 2.3% 1.5% 1.5% 100.0%
2.00 Control group
69 54 20 18 14 1 0 0 0 176
39.2% 30.7% 11.4% 10.2% 8.0% .6% 0.0% 0.0% 0.0% 100.0%
Total 154 129 62 43 28 6 6 4 4 436
35.3% 29.6% 14.2% 9.9% 6.4% 1.4% 1.4% .9% .9% 100.0%
Khulna Group Treatment or Control group
1.00 Treatment group
49 37 6 16 11 119
41.2% 31.1% 5.0% 13.4% 9.2% 100.0%
2.00 Control group
73 37 3 0 0 113
64.6% 32.7% 2.7% 0.0% 0.0% 100.0%
Total 122 74 9 16 11 232
52.6% 31.9% 3.9% 6.9% 4.7% 100.0%
24
Rajshahi Group Treatment or Control group
1.00 Treatment group
13 29 1 3 8 17 8 6 3 2 0 90
14.4% 32.2% 1.1% 3.3% 8.9% 18.9% 8.9% 6.7% 3.3% 2.2% 0.0% 100.0%
2.00 Control group
9 25 4 14 6 4 8 9 5 2 4 90
10.0% 27.8% 4.4% 15.6% 6.7% 4.4% 8.9% 10.0% 5.6% 2.2% 4.4% 100.0%
Total 22 54 5 17 14 21 16 15 8 4 4 180
12.2% 30.0% 2.8% 9.4% 7.8% 11.7% 8.9% 8.3% 4.4% 2.2% 2.2% 100.0%
Rangpur Group Treatment or Control group
1.00 Treatment group
24 63 23 10 0 0 1 121
19.8% 52.1% 19.0% 8.3% 0.0% 0.0% .8% 100.0%
2.00 Control group
28 46 26 19 4 3 0 126
22.2% 36.5% 20.6% 15.1% 3.2% 2.4% 0.0% 100.0%
Total 52 109 49 29 4 3 1 247
21.1% 44.1% 19.8% 11.7% 1.6% 1.2% .4% 100.0%
Sylhet Group Treatment or Control group
1.00 Treatment group
37 17 6 0 0 0 0 60
61.7% 28.3% 10.0% 0.0% 0.0% 0.0% 0.0% 100.0%
2.00 Control group
7 24 14 3 6 3 1 58
12.1% 41.4% 24.1% 5.2% 10.3% 5.2% 1.7% 100.0%
Total 44 41 20 3 6 3 1 118
37.3% 34.7% 16.9% 2.5% 5.1% 2.5% .8% 100.0%
Total Group Treatment or Control group
1.00 Treatment group
253 326 1 129 97 46 13 14 7 6 0 892
28.4% 36.5% .1% 14.5% 10.9% 5.2% 1.5% 1.6% .8% .7% 0.0% 100.0%
2.00 Control group
219 278 4 134 99 46 19 13 5 2 4 823
26.6% 33.8% .5% 16.3% 12.0% 5.6% 2.3% 1.6% .6% .2% .5% 100.0%
Total 472 604 5 263 196 92 32 27 12 8 4 1715
27.5% 35.2% .3% 15.3% 11.4% 5.4% 1.9% 1.6% .7% .5% .2% 100.0%
There are significant differences between the two groups in most of divisions, except Barishal, Dhaka
25
Table 9. Secondary Teachers’ ELP
Group Treatment or Control group * GESEGrade012 Crosstabulation
GESEGrade012
Total 1 10 2 3 4 5 6 7 8 9
Group Treatment or Control group
1.00 Treatment group
1 2 3 4 20 10 6 7 3 2 1 59
1.7% 3.4% 5.1% 6.8% 33.9% 16.9% 10.2% 11.9% 5.1% 3.4% 1.7% 100.0%
2.00 Control group
0 2 1 5 8 10 11 10 5 1 1 54
0.0% 3.7% 1.9% 9.3% 14.8% 18.5% 20.4% 18.5% 9.3% 1.9% 1.9% 100.0%
Total 1 4 4 9 28 20 17 17 8 3 2 113
.9% 3.5% 3.5% 8.0% 24.8% 17.7% 15.0% 15.0% 7.1% 2.7% 1.8% 100.0%
There is no significant difference between the two groups.
26
Table 10. Secondary Teachers’ ELP disaggregated by Gender
Group Treatment or Control group * GESEGrade012 * Gender Crosstabulation
Gender
GESEGrade012
Total 1 10 2 3 4 5 6 7 8 9
Female Group Treatment or Control group
1.00 Treatment group
2 1 2 6 0 2 1 2 16
12.5% 6.3% 12.5% 37.5% 0.0% 12.5% 6.3% 12.5% 100.0%
2.00 Control group
1 0 3 3 2 2 0 0 11
9.1% 0.0% 27.3% 27.3% 18.2% 18.2% 0.0% 0.0% 100.0%
Total 3 1 5 9 2 4 1 2 27
11.1% 3.7% 18.5% 33.3% 7.4% 14.8% 3.7% 7.4% 100.0%
Male Group Treatment or Control group
1.00 Treatment group
1 0 2 2 14 10 4 6 1 2 1 43
2.3% 0.0% 4.7% 4.7% 32.6% 23.3% 9.3% 14.0% 2.3% 4.7% 2.3% 100.0%
2.00 Control group
0 1 1 2 5 8 9 10 5 1 1 43
0.0% 2.3% 2.3% 4.7% 11.6% 18.6% 20.9% 23.3% 11.6% 2.3% 2.3% 100.0%
Total 1 1 3 4 19 18 13 16 6 3 2 86
1.2% 1.2% 3.5% 4.7% 22.1% 20.9% 15.1% 18.6% 7.0% 3.5% 2.3% 100.0%
Total Group Treatment or Control group
1.00 Treatment group
1 2 3 4 20 10 6 7 3 2 1 59
1.7% 3.4% 5.1% 6.8% 33.9% 16.9% 10.2% 11.9% 5.1% 3.4% 1.7% 100.0%
2.00 Control group
0 2 1 5 8 10 11 10 5 1 1 54
0.0% 3.7% 1.9% 9.3% 14.8% 18.5% 20.4% 18.5% 9.3% 1.9% 1.9% 100.0%
Total 1 4 4 9 28 20 17 17 8 3 2 113
.9% 3.5% 3.5% 8.0% 24.8% 17.7% 15.0% 15.0% 7.1% 2.7% 1.8% 100.0%
There is no significant difference between the two groups by gender. P=.446, p=.238
27
Appendix 2: QE study data analysis tables for Classroom Practice Study
Primary Schools: Comparison of Control & Treatment Groups
Table 1. Primary Classrooms: Types of Activity Treatment or Control group Teacher talking Student talking Student reading Student
writing Student listen audio
Other activity
Treatment group % 54% 24% 5% 5% 1% 12%
N 53 53 53 53 53 53
Control group % 53% 23% 5% 5% 1% 13%
N 52 52 52 52 52 52
There is no statistically significant difference between the two groups in terms of type of activity in a classroom.
Table 2: Primary Classrooms: % of Talk in Bangla and English, by Teachers and Students
Group Treatment or Control group
Teacher Talk
Bangla
Teacher Talk
English
Student Talk
Bangla
Student Talk
English
1.00 Treatment group
% 26.98% 73.02% 10.28% 89.72%
2.00 Control group % 33.89% 66.11% 15.50% 84.50%
Total % 30.49% 69.51% 12.95% 87.05%
There is no statistically significant difference between the two groups.
28
Table 3: Primary Classrooms: Types of Teacher Talk Treatment or Control group Presenting Organising Giving
feedback Asking
question
Treatment group % 42% 25% 17% 17%
N 53 53 53 53
Control group % 43% 21% 18% 18%
N 52 52 52 52
There is no statistically significant difference between the two groups.
Table 4: Primary Classrooms: Language of Teacher Talk by Type Presenting Organising Giving feedback Asking question
Treatment or Control group English Bangla English Bangla English Bangla English Bangla
Treatment group % 79% 21% 65% 35% 78% 22% 68% 32%
N 53 53 48 48 47 47 46 46
Control group % 73% 27% 52% 48% 70% 30% 67% 33%
N 51 51 45 45 41 41 46 46
There is no statistically significant difference between the two groups.
29
Table 5: Primary Classrooms: Types of Student Talk Single In pair Group Choiring
Treatment or Control group English Bangla English Bangla English Bangla English Bangla
Treatment group % 88% 12% 88% 13% 96% 4% 93% 7%
N 43 43 8 8 9 9 48 48
Control group % 81% 19% 67% 33% 80% 20% 88% 12%
N 42 42 9 9 9 9 44 44
There is no statistically significant difference between the two groups.
Table 6: Primary Classrooms: Language of Student Talk by Type Single In pair Group Choiring
Treatment or Control group English Bangla English Bangla English Bangla English Bangla
Treatment group % 88% 12% 88% 13% 96% 4% 93% 7%
N 43 43 8 8 9 9 48 48
Control group % 81% 19% 67% 33% 80% 20% 88% 12%
N 42 42 9 9 9 9 44 44
There is no statistically significant difference between the two groups.
30
Secondary Schools: Comparison of Control & Treatment Groups
Table 7. Secondary Classrooms: Types of Activity Treatment or Control group Total of
Teacher talking time
Total of Student talking time
Total of Student reading time
Total of Student writing time
Total of Student listen audio time
Total of Other activity time
Treatment group % 62% 18% 4% 4% 2% 11%
N 51 51 51 51 51 51
Control group % 62% 18% 3% 3% 4% 10%
N 36 36 36 36 36 36
There is no statistically significant difference between the two groups in terms of type of activity in a classroom.
Table 8: Secondary Classrooms: % of Talk in Bangla and English, by Teachers and Students
Group Treatment or Control group
Teacher Talk
Bangla
Teacher Talk
English
Student Talk
Bangla
Student Talk
English
1.00 Treatment group
% 32.28% 67.72% 21.60% 78.40%
2.00 Control group % 43.67% 56.33% 23.13% 76.87%
Total % 37.88% 62.12% 22.36% 77.64%
There is no statistically significant difference between the two groups.
31
Table 9: Secondary Classrooms: Types of Teacher Talk Treatment or Control group Presenting Organising Giving
feedback Asking
question
Treatment group % 52% 16% 13% 19%
N 51 51 51 51
Control group % 54% 14% 11% 21%
N 36 36 36 36
There is no statistically significant difference between the two groups.
Table 10: Secondary Classrooms: Language of Teacher Talk by Type Presenting Organising Giving feedback Asking question
Treatment or Control group
English Bangla English Bangla English Bangla English Bangla
Treatment group % 70% 30% 56% 44% 74% 26% 70% 30%
N 51 51 38 38 44 44 49 49
Control group % 60% 40% 49% 51% 59% 41% 51% 49%
N 35 35 29 29 28 28 33 33
There is no statistically significant difference between the two groups for the language used when Presenting and Organising. There are statistically significant differences between the two groups, for the language used when Giving Feedback and Asking Questions.
32
Table 11: Secondary Classrooms: Types of Student Talk
There is no statistically significant difference between the two groups. Note: the numbers of cases of pair or group work were very small.
Table 12: Secondary Classrooms: Language of Student Talk by Type Single In pair Group Choiring
Treatment or Control group
English Bangla English Bangla English Bangla English Bangla
Treatment group % 83% 17% 86% 14% 40% 60% 87% 13%
N 46 46 7 7 16 16 24 24
Control group % 76% 24% 88% 13% 95% 5% 72% 28%
N 30 30 4 4 5 5 24 24
There is no statistically significant difference between the two groups.
Note: the numbers of cases of pair or group work were very small.
Treatment or Control group Single In pair Group Choiring
Treatment group Mean 64% 5% 13% 18%
N 48 7 16 24
Control group Mean 56% 4% 6% 33%
N 35 4 5 24
33
Appendix 3: Qualitative Study Field Protocol & Guidance Notes
Guidance Notes on semi-structured interviews Extract
‘A semi-structured interview is a verbal interchange where one person, the interviewer, attempts to elicit information from another person by asking questions. Although the interviewer prepares a list of predetermined questions, semi-structured interviews unfold in a conversational manner, offering participants the chance to explore issues they feel are important …
…Semi-structured interviews (sometimes referred to as informal, conversational, or ‘soft’ interviews) are about talking with people but in ways that are self-conscious, orderly, and partially structured. Krueger and Casey (2000) explain that it is …
‘…about listening. It is about being open to hear what people have to say. It is about being nonjudgemental. It is about creating a comfortable environment for people to share. It is about being careful and systematic with the things people tell you.’
…Dunn (2005:80) explains:
“Structured interviews follow a predetermined and standardized list of questions. The questions are always asked in almost the same way and the same order.
At the other end of the continuum are unstructured forms of interviewing such as oral histories… The conversation in these interviews is actually directed by the informant rather than by the set questions.
In the middle of this continuum are semi-structured interviews. This form of interviewing has some degree of predetermined order but still ensures flexibility in the way issues are addressed by the informant”.
When conducting semi-structured interviews or focus groups it is possible to take notes or to audio/video record the discussion. I usually audio(tape) the proceedings. This allows me to focus fully on the interaction instead of feeling pressure to get the participants worlds recorded in my notebook (see Valentine, 2005).
Directly after the interviews I document the general tone of the conversation, the key themes that emerged and anything that particularly impressed me or surprised me in the conversation. Taking these notes, in a sense, is a form of data analysis… It is advantageous to transcribe interviews and focus groups as soon as possible after conducting them… Hearing the taped conversation whilst it is still fresh in your mind makes transcription much easier …’
34
Field Protocol General School Information 1. name, location and designation of the school
2. size of the school
• number of students on roll • student numbers by class, disaggregated by gender • teacher numbers (disaggregated by gender) • for each teacher being interviewed: Which classes do they teach? Which
subjects do they teach? (Primary: do all teachers teach all subjects). • does the school operate shifts? (How many, what times)? • number of classrooms?
3. other general information about the school.
4. How many Assistant Teachers, other than the two teachers included in the study, can and do teach English?
5. Professional development context
• existing or previous involvement of the school and its teachers in professional development offered by the Upazila (or external agencies in secondary) in last 3 years.
• are there any professional development / teaching quality improvement activities happening in the school (other than EIA)?
• does the head teacher have a role in professional development / teaching quality improvement activities in the school? Elaborate...
6. Does the school have links to local neighbouring schools (including primary-
secondary links)
Other additional information Photograph the school, particularly any additional resources, such as staff rooms, libraries, computer rooms, or any other unusual / interesting features.
35
SSI with Teachers Before the lesson observation and interview, where possible: • Record the teachers’ name and gender.
• Record the class (year) being observed (e.g. Class 7)
• If possible, photograph the classroom, teacher, and any resources (EIA or otherwise) to support English language teaching
• You may find it helpful to sketch the class (position of benches / tables / teachers desk / board / resources / students seating by gender)
During lesson observation record any observations you feel appropriate to support the discussion to follow. In particular: • Teacher behaviours related to communicative practices
• Use of activities, practices or materials that may have been promoted by Teacher Development Programmes (e.g. EIA, SBT, TQI etc).
• Novel or unexpected teaching and learning practices
• Student responses; extent and equity of participation
Time-stamp your notes (e.g. actual time, or lesson time) regularly. Discretely photograph or audio or video record any interesting practices during the lesson, if appropriate and possible without causing distraction / disruption.
SSI with EIA teachers, on New Classroom Activities 1. Open Ended
1. How do you feel the lesson went? (Was there anything in particular you were pleased with, or that you might do differently next time?)
2. How would you describe the students’ reaction to the lesson?
3. If I’d come here to see your lesson a year or two ago and come back now, how do you think those lessons would compare? (If there are important changes, what or who most helped you make them?)
4. If I come back again in a year or two’s time, what differences would you hope I’d see
a. in your lesson?
b. in you or your students?
What might be the most important thing in helping you make that change? 2. Directed to Themes (Note: Areas to discuss, not questions to ask!)
1. SBTD activities in the classroom
• What effect would you say any professional development or training programmes have had on what you or your students do in the classroom?
36
– Can tell me about any specific examples from the lesson we’ve just seen, or from your teaching in the last week or so?
– If so, what was there anything that helped you make that change / learn how to do this?
– What do you think you learned from making that change?
• Is there anything that makes it hard to introduce ideas from training programmes into your classroom practice, and what kinds of things might help overcome these difficulties?
2. SBTD activities outside the classroom
• Apart from attending training courses or upazila meetings, is there anything you do at home or school, that helps you get new ideas for teaching or learning? (observations, books, materials, online / offline resources etc. etc.).
• Can you give any specific examples of something you learned like this… can you tell me how you learned it…
• If another teacher in your school wanted to improve their English teaching, apart from going on courses, would you have any advice for them, based on your experience?
3. SBTD activities with peers in school
• Are there any ways in which other teachers from this school have helped you think or act differently in the classroom?
• How do the kind of routines and relationships that exist in the school support or restrict this kind of learning between teachers
4. The Role of the Head Teacher or EO’s in promoting / monitoring SBTD in school
Can you think of anything that the head teacher or your local education officers have done in your school, that help you and other teachers improve teaching and learning in your lessons?
37
SSI with Head Teacher
Head Teacher Interview A. Open Ended
1. Check and update the general information (see p4).
2. What do you think is the most important role of the head teacher – and what do you spend most of your time doing?
3. What are you trying to achieve for the school?
B. Directed to Themes
4. Do you have a particular approach (strategy) that you hope to bring about the
improvement of your teaching staff? Probe through asking:
• How particular staff chosen to participate in or lead development activities? • Do individual teachers make their own choices, are is participation required? • What is the head teacher’s role and how do they fulfil it?
5. Explore HT views related to question 5 (p4) – the professional development context
of the school.
• What kinds of things are happening in the school, to improve classroom practices… how does the HT promote or monitor such changes?
• Is there anything that happens in school, to help teachers learn about new approaches, or learn from each other? What’s the head teacher’s role in this?
• Are there any particular teachers that have done something to improve teaching and learning in the school… what did they do, and why did they do it? How did students / other teachers / the HT / the community respond?
6. What are the biggest challenges to improving teaching and learning in the school?
Is there anything helping or hindering you or your teachers, in overcoming these?
7. Can you give any specific examples of ways in which your Education Officer has promoted or monitored changes in teaching and learning in your school X, in the last 3 months?
38
SSI with Education Officer
A. Open Ended
1. What do you think is the most important role of the Education Officer – and what do you spend most of your time doing?
2. What are you trying to achieve for the schools in your Upazila?
3. Do you have any particular hopes or challenges in relation to school X (that we are studying).
B. Directed to Themes 4. What’s your strategy for helping teachers in school X improve teaching and learning
in their classrooms?
5. How do you encourage teachers to put what they learn on training courses or in local meetings, into practice in the classroom? How do you know the extent to which this is happening in reality?
6. How would you describe the head teachers role, in relation to improving quality of teaching and learning in their school, or helping teachers put their training into practice?
7. How do you see your role, in relation to the head teachers role? Practically, what mechanisms do you have to support, guide or monitor head teachers, and how do those work.
8. Can you give any specific examples of ways in which you have promoted or monitored changes in teaching and learning in school X, in the last 3 months?