Can High Achievement be Attributed to Better Teaching?

- Results of the TIMSS Video Study Bogota, Columbia, November 2006

Frederick K.S. LeungThe University of Hong Kong

East Asian students have consistently out-performed their counterparts around the world in international comparisons of mathematics achievement.

Can the high achievement be explained by better teaching in the East Asian classroom?

This presentation reports some of the results of the TIMSS Video Study in an attempt to portray the mathematics teaching in the East Asian classroom

Implications of the findings will also be discussed

IntroductionIntroduction

TIMSS 1999 Video Study (Math)TIMSS 1999 Video Study (Math)

Goal:Describe and compare eighth-grade mathematics

teaching across seven countries (Australia, Czech Republic, Hong Kong SAR, Japan*, Netherlands, Switzerland, United States)

* The 1995 Japanese data were re-analyzed using the 1999 methodology in some of the analysis

Sampling and Data Collection

National probability sample of 8th-grade math lessons: a Video Survey

One lesson per teacherSampled across the school yearStandardized camera procedures638 lessons, from 50 (Japan) – 140

(Switzerland)

Data Coding and AnalysisAn international team developed codes to apply to the

video data.Fluently bilingual coders in the international video

coding team applied 45 codes in seven coding passes to each of the videotaped lessons.

Three marks (i.e., the in-point, out-point, and category) were evaluated and included in the measures of reliability.

If, after numerous attempts, reliability measures fell below the minimum acceptable standard, the code was dropped from the study.

The Mathematics Quality Analysis GroupA specialist group in mathematics and teaching

mathematics at the post-secondary level reviewed a randomly selected subset of 120 lessons (20 lessons from each country except Japan).

The international video coding team created expanded lesson tables for each lesson in this subset.

The tables included details about the classroom interaction, the nature of the math problems worked on, mathematical generalizations, and other relevant information.

The tables were “country-blind,” with all indicators that might reveal the country removed.

Instructional Practices in East Asia as Instructional Practices in East Asia as Portrayed by the Analysis of the CodesPortrayed by the Analysis of the Codes

1. Dominance of teacher talk In all countries in the study, the teachers did a

lot of talking, and considerably more than their students

Hong Kong and Japan differ considerably in the amount of teacher talk

Average Number of Teacher and Student Words Per Lesson

5536 54525798

5148 53605902

810 824 640 766 1016 1018

0

1000

2000

3000

4000

5000

6000

7000

AU CZ HK JP NL US

Average number of teacher words Average number of student words

Ratio of teacher and student talkHong Kong and Japanese teachers spoke much

more relative to their students “Hong Kong SAR eighth-grade mathematics

teachers spoke significantly more words relative to their students (16:1) than did teachers in Australia (9:1), the Czech Republic (9:1), and the United States (8:1).” (p. 109, Chapter 5)

When we factor in the relatively large class size (about 40), the reticence of East Asian students is striking

Average Number of Teacher Words to Every One Student Word Per Lesson

9 9

16

13

108

0

4

8

12

16

20

AU CZ HK JP NL US

Num

ber

of Tea

cher

Word

s Per

1 S

tuden

t W

ord

2. More opportunities to learn new content75% of lesson time in the East Asian

classroom spent on dealing with new contentCorresponding figures for other countries

ranged between 42% (Czech Republic) and 63% (Switzerland)

Inference: East Asian students learn more mathematics than students in other countries?

Average percentage of lesson time devoted to various purposes

26 2037

1625 24 25

30

22

39

6032 39

23

36 58

24 24

3734

53

0

20

40

60

80

100

AU CZ HK J P NL SW US

Practice New Introduce New Review

3. Mathematics problems worked on more complex

Procedural complexity of problems: “the number of steps it takes to solve a problem using a common solution method” (p.70)

Japanese students worked on procedurally more complex problems

Problems Hong Kong students worked on not particularly complex, although the percentage (63%) of low complexity problems is relatively small

Average percentage of problems at each level of procedural complexity

7764 63

17

69 65 67

1625 29

45

2522

27

8 11 8

39

6 12 6

0

20

40

60

80

100

AU CZ HK JP NL SW US

Per

cen

t o

f P

rob

lem

s

Low Complexity Moderate Complexity High Complexity

Problem complexity (cont’d)

Another measure of problem complexity: length of time students spent working on the problem (more or less than 45 seconds)

Conclusion: East Asian students have more opportunities to work on procedurally more complex problems which required a longer duration to solve

Average percentage of problems that were worked on longer more than 45 s

61

7374

98

78

5955

0

20

40

60

80

100

AU CZ HK J P NL SW US

4. Problems unrelated to real-life

Majority of problems in the East Asian classroom were expressed in mathematical language and symbols, and set in contexts unrelated to real life

Similar to classrooms in Czech Republic, and differ markedly from classrooms in the Netherlands

Average Percentage of Problems Per Lesson Set Up With a Real Life Connection or With Mathematical Language or Symbols Only

7281 83 89

40

71 69

27 15 15 9

42

25 22

0

20

40

60

80

100

AU CZ HK J P NL SW US

Set-up contained a real life connection

Set-up used mathematical language or symbols only

5. More proof

Problems East Asian students worked on involved more proof

The emphasis is particularly marked in JapanThe practice in Hong Kong more in line with

Switzerland

Percentage of problems that contained at least one proof

512

39

11

‡‡‡0

20

40

60

80

100

AU CZ HK J P NL SW US

Instructional practices as portrayed by the analysis of the codes

• Dominance of teacher talk• Students have more opportunities to learn

new content• Students solve problems that are more

complex and are unrelated to real-life• More proof

Quality of Content as judged by the Quality of Content as judged by the Math Quality Analysis GroupMath Quality Analysis Group

(based on the same data set)(based on the same data set)

Japanese not in the analysis

“Readers are urged to be cautious in their interpretations of these results because the sub-sample, due to its relatively small size, might not be representative of the entire sample or of eighth-grade mathematics lessons in each country.” (p. 190, Appendix D)

1. Relatively advanced content

“the ratings for countries with the most advanced (5) to the most elementary (1) content in the sub-sample of lessons, were the Czech Republic and Hong Kong SAR (3.7), Switzerland (3.0), the Netherlands (2.9), the United States (2.7), and Australia (2.5)” (p. 191, Appendix D)

Percentage of Lessons in Sub-sample at each Content Level

10 15 15

45

15

20 1525

30

20 40

30

404535

3035

20

100 05

40

15 20 2050 0 0

0

20

40

60

80

100

AU CZ HK NL SW US

Per

cent of

Sub-sam

pled L

esso

ns

AdvancedModerate/AdvancedModerateElementary/ModerateElementary

2. More deductive reasoning

Deduction reasoning = “deriving conclusions from stated assumptions using a logical chain of inferences.”

The reasoning did not need to include a formal proof, only a logical chain of inferences with some explanation.

Percentage of Lessons in Sub-sample that Contained Deductive Reasoning

05

1510 10

50

20

40

60

80

100

AU CZ HK NL SW US

Per

cent of

Sub-sam

pled L

esso

ns

3. More coherent

Coherence was defined by the group as the

(implicit and explicit) interrelation of all

mathematical components of the lesson.

Percentage of Lessons in Sub-sample Rated at Each Level of Coherence

1515 10

10

3510

30

20

15

20

10

105

20

000 05 500

20

15

30

5560 65

90

30

0

20

40

60

80

100

AU CZ HK NL SW US

Per

cent of

Sub-sam

pled L

esso

ns

Thematic

Moderately thematic

Mixed

Moderately fragmented

Fragmented

4. More fully developed presentationPresentation = “the extent to which the lesson included some

development of the mathematical concepts or procedures”.Development required that mathematical reasons or

justifications were given for the mathematical results presented or used.

Presentation ratings took into account the quality of mathematical arguments.

Higher ratings meant that sound mathematical reasons were provided by the teacher (or students) for concepts and procedures.

Mathematical errors made by the teacher reduced the ratings.

Percentage of Lessons in Sub-sample Rated at Each Level of Presentation

20

20

35

5

2030

40

20

10

30

55

20

45

25

40

1510 00 1010

15

30

40

5501520

10

0

20

40

60

80

100

AU CZ HK NL SW US

Per

cent

of Su

b-sa

mpl

ed L

esso

ns

Fully developed

Substantially developed

Moderately developed

Partially developed

Undeveloped

5. Students more likely to be engagedStudent engagement = “the likelihood that students would be

actively engaged in meaningful mathematics during the lesson”.

A rating of very unlikely (1) indicated a lesson in which students were asked to work on few of the problems and those problems did not appear to stimulate reflection on math concepts or procedures.

A rating of very likely (5) indicated a lesson in which students were expected to work actively on, and make progress solving, problems that appeared to raise interesting mathematical questions for them and then to discuss their solutions with the class.

Percentage of Lessons in Sub-sample Rated at Each Level of Student Engagement

3010

30

10

4530

20

30

25

55

30

20

40

15

15100 05 105

30

30

15

01010 10

35

10

0

20

40

60

80

100

AU CZ HK NL SW US

Per

cent

of Su

b-sa

mpl

ed L

esso

ns

Very likely

Probable

Possible

Doubtful

Very unlikely

6. Overall quality

Overall quality judgment:

“the opportunities that the lesson provided for students to construct important mathematical understandings” (p. 199, Appendix D)

“the relative standing of Hong Kong SAR was consistently high ….” (p. 200, Appendix D)

Percentage of Lessons in Sub-sample Rated at Each Level of Overall Quality

20

20

15

15

1530

20 25

2040

45

20

35

25

40

101505

2510

15

35

30

05515

3015

0

20

40

60

80

100

AU CZ HK NL SW US

Per

cent

of Su

b-sa

mpl

ed L

esso

ns

High

Moderately high

Moderate

Moderately low

Low

General Ratings for Each Overall Dimension of Content Quality of Lessons

AUAUAU

AU

CZCZ

CZCZHKHKHK

HK

NLNLNL

NL

SWSW

SW

SW

USUSUS

US

0.0

1.0

2.0

3.0

4.0

5.0

Coherence Presentation Studentengagement

Overall quality

AU

CZ

HK

NL

SW

US

Quality of the Content as judged by Quality of the Content as judged by the Math Quality Analysis Groupthe Math Quality Analysis GroupRelatively advanced contentMore deductive reasoningMore coherentMore fully developed presentationStudents are more engaged, andOverall quality is high

DiscussionDiscussionSome characteristics of the East Asian classroom

found in this study (large class size, dominance of teacher talk, reticence of students, abstract problems unrelated to real-life) seem to be at odds with modern theories of learning

Despite the rhetoric of constructivism and student-centred learning to the contrary, the findings show that meaningful learning can still take place in a teacher directed classroom with a large class size

Teacher dominance with a lot of teacher talk does not necessarily lead to passive, receptive learning

Much depends on the content of the teacher talk and how it is delivered, and whether the talk can stimulate students to be engaged in mathematics

The data in this study suggest that the kind of teacher talk in the East Asian classroom was able to direct students to be engaged in the lesson

Indeed, a well-taught teacher-dominated lesson may better provide the mathematical coherence which students need in their construction of mathematical knowledge rather more effectively than many student-led approaches.

Mathematics content covered in East Asian classrooms

East Asian students learned more new content than their counterparts in the West

The content was more complex and advancedThere were more proofs and more use of

mathematical language

Proof and the use of maths language In many countries, mathematical language is considered

too alien and proof too abstract for school studentsBoth are deemed to be too difficult for school students

and are thus excluded from the curriculaHowever, both have traditionally been regarded as

distinctive features of mathematics, and it seems that they are still judged to be so in the East Asian classroom

Neither was stressed in TIMSS and PISA

A firm foundation in mathematics laid for East Asian students through emphasis on mathematical language and proof that enables these students to do well in the less abstract tasks in the international tests?

“In a milieu which seems to believe that the most effective way to enhance understanding and raise attainment levels is through an improved pedagogy, the clear indication that the high achievement of East Asian students is related to the high quality of the mathematics content to which they are exposed, should act as a sharp reminder that without quality content, quality learning will not take place - no matter how ingenious the teaching method.”

Expectation on studentsEast Asian teachers have higher expectations of their

students on the kind of mathematics to be learnedThe level of expected mathematics achievement in

many Western countries seems to be decliningMathematics is considered by students and teachers

alike as a difficult subjectMajority of student population not expected to learn

more advanced mathematics, and are not even expected to do well in elementary mathematics

The low student achievement becomes a self-fulfilling prophecy

Teacher competenceEast Asia teachers are sufficiently competent in

mathematics to deliver complex and advanced content (Ma, 1999, Leung and Park, 2002)?

More coherent and better developed presentation may be attributed to the mathematical and pedagogical competence of the teachers

Ma (1999): competence in mathematics and pedagogy are intrinsically related: without a profound understanding of mathematics, it is not possible to invoke the appropriate pedagogy.

Scholar teacher In East Asian or “Confucian Heritage” Culture

(Biggs, 1996), the ideal of the “scholar teacher” is that of an expert or a learned figure in the subject matter

Teaching skills are also important, but teachers will not be respected if they are not expert in the area they teach

This image of the scholar-teacher may provide incentives for East Asian teachers to strive to attain high levels of competence in the subject matter as well as in pedagogy

ConclusionConclusionNo simple casual relation between classroom teaching

and student achievement can be drawn, but East Asian teachers did teach differently from their counterparts in the West

Classroom practices are deeply rooted in the underlying cultural values of the classroom and the wider society

Simple transplant of educational practice from high achieving countries to low achieving ones would not work

One cannot transplant the practice without transplanting the culture as well

Conclusion (cont’d)We should identify not only the superficial

differences in educational practice, but the intricate relationship between the educational practice and the underlying culture of other countries

Through identifying the commonality and differences of both the educational practices and the underlying cultures, we may then determine how much can or cannot be borrowed from another culture.

My e-mail address:

frederickleung@hku.hk