DOCUMENT RESUME AUTHOR Frase, Mary; And Others · DOCUMENT RESUME. ED 410 098 SE 060 370. AUTHOR...

DOCUMENT RESUME

ED 410 098 SE 060 370

AUTHOR Frase, Mary; And OthersTITLE Pursuing Excellence: A Study of U.S. Fourth-Grade

Mathematics and Science Achievement in InternationalContext. Initial Findings from the Third InternationalMathematics and Science Study.

INSTITUTION Office of Educational Research and Improvement (ED),Washington, DC.

REPORT NO NCES-97-255; ORAD-97-1018PUB DATE Jun 97NOTE 67p.; Identical texts published separately under each report

number. Also published as a component of "AttainingExcellence: A TIMSS Resource Kit," see SE 060 930.

AVAILABLE FROM TIMSS Project Officer, National Center for EducationStatistics, Office of Educational Research and Improvement,U.S. Department of Education, 555 New Jersey Avenue, N.W.,Washington, DC 20208-5574.

PUB TYPE Reports Evaluative (142) Reports Research (143)EDRS PRICE MF01/PC03 Plus Postage.DESCRIPTORS *Achievement; *Grade 4; Intermediate Grades; *International

Studies; Science Instruction; *Scientific LiteracyIDENTIFIERS *Third International Mathematics and Science Study

ABSTRACTThe Third International Mathematics and Science Study

(TIMSS) is the largest, most comprehensive, and rigorous international studyof schools and student achievement ever conducted. During the 1995 schoolyear, the study tested the mathematics and science knowledge of ahalf-million students from 41 nations at five different grade levels. Thisreport presents findings from the tests, questionnaires, and curriculumanalysis performed at the fourth grade level in 26 countries and is thesecond of three major TIMSS reports in the "Pursuing Excellence" series.Chapters are titled "Achievement" and "Contexts of Learning". Findingsinclude: (1) in mathematics and science, U.S. fourth graders perform abovethe international average of the 26 countries; (2) in mathematics contentareas, U.S. fourth graders exceed the international average in five of thesix areas assessed; (3) in science content areas, U.S. fourth graders'performance exceeds the international average in all four of the areasassessed; (4) the international standing of U.S. fourth graders is strongerthan that of U.S. eighth graders in both mathematics and science; (5) incomparison with their international counterparts, U.S. students performbetter in science than in mathematics; and (6) preliminary analysis shedslittle light on factors regarding contexts of learning which might accountfor the differences observed. (JRH)

********************************************************************************

Reproductions supplied by EDRS are the best that can be madefrom the original document.

********************************************************************************

MAYOONGA CgMTIEGI VD% EDUCAVOC)M §4AITO45TOC4

AP

U.S. DEPARTMENT OF EDUCATIONOffice of Educational Research and Improvement

EDUCATIONAL RESOURCES INFORMATIONCENTER (ERIC)

This document has been reproduced asreceived from the person or organizationoriginating it.

Minor changes have been made toimprove reproduction quality.

Points of view or opinions stated in thisdocument do not necessarily representofficial OERI position or policy.

ASUMG EXCELL NCE

A STUDY OF U.S0 FOURTH- RADEMATHERWATRCS AND SCRENCE Ac zvamEmr

am- INTERNATRONAL CON 2 T

D

r)

JJ

ONOVO&L Fomoomog non 'rTHOEID ONVEIRMTOONA KAVNIERM MO SCOIENCE VUDV

Ogrreff OF EDUcATIoNaL RE4EARCH AND ORPGICNIENIIK7S DEPAWRIEW ©V EDuctWOCM

BEST COPY AVAILABLE 1

2

PURSUING EXCELLENCE

A STUDY OF U.S. FOURTH-GRADE

MATHEMATICS AND SCIENCE ACHIEVEMENT

IN INTERNATIONAL CONTEXT

Contributors:

Mary Frase Lois PeakPamela Jakworth William SchmidtLeslie Martin Larry SuterMartin Orland Sayuri TakahiraEugene Owen Trevor Williams

INITIAL FINDINGS FROM THETHIRD INTERNATIONAL MATHEMATICS AND SCIENCE STUDY

3

U.S. Department of EducationRichard W. RileySecretary

Office of Educational Researchand ImprovementRamon C. CortinesActing Assistant Secretary

National Center forEducation StatisticsPascal D. Forgione, Jr.Commissioner

Data Development and LongitudinalStudies GroupMary J. FraseActing Associate Commissioner

International Activities ProgramEugene OwenDirector

National Center for Education StatisticsThe National Center for Education Statistics (NCES) is the primary federal entity forcollecting, analyzing, and reporting data related to education in the United Statesand other nations. It fulfills a congressional mandate to collect, collate, analyze, andreport full and complete statistics on the condition of education in the United States;conduct and publish reports and specialized analyses of the meaning and signifi-cance of such statistics; assist state and local education agencies in improving theirstatistical systems; and review and report on education activities in foreign countries.

NCES activities are designed to address high-priority education data needs; provideconsistent, reliable, complete, and accurate indicators of education status andtrends; and report timely, useful, and high quality data to the U.S. Department ofEducation, the Congress, the states, other education policymakers, practitioners,data users, and the general public.

We strive to make our products available in a variety of formats and in language thatis appropriate to a variety of audiences. You, as our customer, are the best judge ofour success in communicating information effectively. If you have any comments orsuggestions about this or any other NCES product or report, we would like to hearfrom you. Please direct your comments to:

Lois PeakTIMSS Project OfficerNational Center for Education StatisticsOffice of Educational Research and ImprovementU.S. Department of Education555 New Jersey Avenue, NWWashington, DC 20208-5574Contact: (202) 219-1333e-mail: [email protected]

Suggested Citation:U.S. Department of Education. National Center for Education Statistics,

Pursuing Excellence: A Study of U.S. Fourth-GradeMathematics and Science Achievement in International Context,

NCES 97-255, Washington, DC: U.S. Government Printing Office, 1997.

June 1997

Available for downloading at http://www.ed.gov/NCES/timss

O

COMMISSIONER'SSTATEMENT

The Third International Mathematics and Science Study (TIMSS) is the largest, mostcomprehensive, and most rigorous international study of schools and studentachievement ever conducted. This report, Pursuing Excellence: A Study of U.S. Fourth-Grade Mathematics and Science Achievement in International Context summarizes data onfourth-grade students in 26 countries. It is the second of three major TIMSS reportsby the National Center for Education Statistics (NCES) in its Pursuing Excellenceseries. The first report, outlining U.S. comparative eighth-grade results, was releasedlast fall, while the third report, investigating student achievement in the last year ofschooling, will be published early next year. Together, these three studies will painta more complete picture than we have ever had about how U.S. schooling practicesand achievement in mathematics and science compare with the rest of the world.The information is intended to help U.S. educators, parents, policymakers, and oth-ers evaluate the strengths and weaknesses of our schools from an international per-spective.

The scope of TIMSS is unprecedented in the annals of education research. Theinternational project involved the testing of more than a half-million students inmathematics and science at three grade levels in 41 countries. In contrast to previ-ous international comparative studies, TIMSS also goes beyond the traditional"horserace" data on student performance to analyze the content of textbooks andcurricula in participating countries and to administer written questionnaires toteachers and students regarding their academic practices both inside and outside theclassroom. A particularly innovative technique used at the eighth grade was thevideotaping of a random sample of actual mathematics classrooms in the UnitedStates, Germany, and Japan, in order to better understand key similarities and dif-ferences in instructional practices across these three countries.

This wealth of data is being analyzed and published by NCES and others around theworld. Both individually and collectively, the various TIMSS reports constitute impor-tant tools that can improve the quality of primary and secondary education for allstudents. That is why the Center has been working cooperatively with other parts ofthe U.S. Department of Education to produce a multi-media resource kit designedfor educators and those interested in using TIMSS data to improve teaching, curric-ula, and student achievement in states and local communities.

The TIMSS data provide a reference point with which we can begin to clarify whatwe mean by "world-class" education. They give us tools by which we can benchmarknot only the performance of our students but also the way in which we deliverinstruction. Most important, they allow the U.S. to learn unique lessons from othermembers of the world community so that we may better pursue the goal of an excel-lent education for all students.

O

NCES is releasing the information in a variety of new forms, including CD-ROM,videotape, and the World Wide Web (http://www.ed.gov/NCES/timss). We inviteeveryone who is dedicated to enhancing the quality of our nation's mathematics andscience education to make the fullest possible use of this rich resource.

6L41-Z Jeue'PtePascal D. Forgione, Jr.Commissioner of Education Statistics

O

6

TABLE OF CONTENTS

COMMISSIONER'S STATEMENT 3

LIST OF FIGURES 7

EXECUTIVE SUMMARY 9

PREFACE 12

Overview 13

Study design 14

The TIMSS research team 15

Organization of this report 16

CHAPTER I: ACHIEVEMENT 18

Key points 18

How well do U.S. fourth graders do in mathematics and science? 19

Some special notes on the test scores 20

Which countries outperform U.S. fourth graders in both subjects? 22

Which countries do U.S. fourth graders outperform in both subjects? 22

How do our fourth graders compare with our major economic partners? 24

How do our best fourth graders compare with others' best? 24

How do our fourth graders score in the different content areasof mathematics and science? 26

Is there a gender gap in mathematics and science at fourth grade? 27

Has U.S. fourth-grade international standing improved over time? 27

How does the performance of U.S. fourth graders compare with that of U.S.eighth graders in mathematics and science? 31

How does the performance of U.S. fourth graders compare with that of U.S.eighth graders in mathematics and science content areas? 33

Summary 34

CHAPTER 2: CONTEXTS OF LEARNING 36

Key points 36

What factors might contribute to the finding that U.S. fourth graders scoreabove the international average in both mathematics and science? 38

How does the U.S. fourth-grade mathematics and science curriculum differfrom the international average? 38

Do U.S. fourth graders spend more time in class studying mathematicsand science? 39

Is homework more common in the U.S. than in other countries? 40

How does the structure of U.S. mathematics and science instructiondiffer from that in other TIMSS nations? 41

Is U.S. classroom organization different from that of other countries? 41

Are calculators and computers more common in U.S. fourth-gradeclassrooms than in other TIMSS countries? 42

Are U.S. fourth-grade teachers better trained than their colleagues inother TIMSS countries? 42

Do U.S. teachers experience fewer professional challenges than do teachersin other TIMSS countries? 43

O

Do U.S. students have more educational resources in their homes thando typical students in other TIMSS countries) 43

Do U.S. fourth graders have more positive attitudes toward mathematicsand science than do students in other countries? 45

Is heavy television watching less common among U.S. fourth graders thanamong students in other countries) 45

Summary 45

What factors might contribute to the finding that the internationalstanding of U.S. fourth graders is stronger than that of U.S. eighth graders ? 47

Is the U.S. fourth-grade curriculum more focused than that of the eighthgrade? 47

Are U.S. fourth graders assigned relatively more mathematics homeworkthan are U.S. eighth graders? 48

Is U.S. fourth-grade mathematics and science instruction different fromthat in the eighth grade? 48

Is U.S. fourth-grade mathematics classroom organization different fromthat in the eighth grade? 49,

Are calculators used more commonly in U.S. fourth-grade mathematicsclasses than in eighth-grade classes? 49

Are U.S. fourth-grade teachers more experienced than eighth-gradeteachers? 49

Do U.S. fourth-grade students have more educational resources in theirhomes than do eighth graders? 49

Do U.S. fourth-grade students watch relatively less TV than eighth-grade students) 50

Do U.S. fourth graders like mathematics and science more than do U.S.eighth graders) 50

Summary 50

What factors might contribute to the finding that the internationalstanding of U.S. students is stronger in science than in mathematics? 51

Is the U.S. curriculum more focused in science than in mathematics? 51

Do U.S. students spend relatively more time in class studying sciencethan mathematics? 53

Are class sizes different in science than in mathematics) 53Are U.S. science teachers more experienced than mathematics teachers? 53Are U.S. student attitudes more positive toward science than they are

toward mathematics? 53Summary 54

CONCLUSIONS 56WORKS CITED 58

APPENDIX I: Additional TIMSS Reports 59APPENDIX 2: Advisors to the U.S. TIMSS Study 65APPENDIX 3: National Average Scores and Standard Errors 66APPENDIX 4: Summary of National Deviations from International Study Guidelines 67

O

LIST OF FIGURES

FIGURE I: Nations' Average Mathematics Performance Comparedwith the U.S 20

FIGURE 2: Nations' Average Science Performance Comparedwith the U.S. 21

FIGURE 3: Average Achievement of Nations Meeting, and Not Meeting,International Guidelines 23

FIGURE 4: Percentage of Students from Selected Nations Scoringamong the Top Ten Percent of Fourth Graders inthe 26 TIMSS Countries 25

FIGURE 5: National Averages in Mathematics Content Areas 28

FIGURE 6: National Averages in Science Content Areas 30

FIGURE 7: U.S. Mathematics and Science Performance at a Glance 32

FIGURE 8: Teachers' Report on Average Hours of Mathematicsand Science Instruction per Week 40

FIGURE 9: Percentage of Students Whose Teachers Assign VariousAmounts of Mathematics Homework 41

FIGURE 10: Percentage of Mathematics Students Whose TeachersReport Various Circumstances Limit Their Teaching`Quite a Lot' or 'A Great Deal' 44

FIGURE I I: Percentage of Fourth- and Eighth-Grade StudentsWhose Teachers Report Teaching Various Numbersof Mathematics and Science Topics 52

EXECUTIVESUMMARY

PREFACE

The Third International Mathematicsand Science Study (TIMSS) is thelargest, most comprehensive, and mostrigorous international comparison ofeducation ever undertaken. During the1995 school year, the study tested themathematics and science knowledge ofa half-million students from 41 nationsat five different grade levels. This reportpresents findings from the tests, ques-tionnaires, and curriculum analysis per-formed at the fourth grade. Twenty-sixnations participated in the fourth-gradeassessment.

TIMSS' information not only com-pares achievement, but also providesinsights into how life in U.S. schoolsdiffers from that in other nations.

This report on fourth-grade studentsis the second of a series of three pub-lic-audience reports titled PursuingExcellence. The first report presentedfindings on student achievement ateighth grade. The third report will bereleased in the spring of 1998, andwill present findings from the twelfthgrade. Additional reports will provideinformation on various other topics.

TIMSS is a fair and accurate comparisonof mathematics and science achieve-ment in the participating nations. It isnot a comparison of "all of our studentswith other nations' best students," acharge that some critics have leveled atprevious international comparisons.The students who participated in TIMSSwere randomly selected to represent allstudents in their respective nations, withthe exception of a few nations which areclearly noted in this report. The entireassessment process was scrutinized byinternational technical review commit-

tees to ensure its adherence to estab-lished standards. Those nations in whichirregularities arose are clearly noted inthis and other TIMSS reports.

ACHIEVEMENT

One of our national goals is to be "firstin the world in mathematics and scienceachievement by the year 2000," as Presi-dent Bush and 50 governors declared in1989. In fourth-grade science achieve-ment, we are close to this mark. Fourthgraders in only one countryKoreaoutperform U.S. students in this subject.

In mathematics, U.S. fourth gradersperform above the internationalaverage of the 26 TIMSS countries.U.S. students are outperformed bythose in 7 countries and outperformthose in 12 countries. Among ourmajor economic partners who partic-ipated in the study, our students'scores are below those of Japan, notsignificantly different from those ofCanada, and are significantly higherthan those of England.

In science, U.S. fourth graders alsoperform above the internationalaverage of the 26 TIMSS countries.U.S. students are outperformed bystudents in only one countryKorea.U.S. students outperform those in 19countries. Among our major eco-nomic partners who participated inthe study, our students' scores are notsignificantly different from those offourth graders in Japan. Our studentsoutperform those in England andCanada.

In mathematics content areas, ourfourth graders exceed the interna-tional average in five of the six areas

O10

assessed. These five areas are: wholenumbers; fractions and proportional-ity; data representation, analysis, andprobability; geometry; and patterns,relations, and functions. In one con-tent area, the U.S. average is lowerthan the international averagemea-surement, estimation, and numbersense.

In science content areas, our fourthgraders' performance exceeds theinternational average in all four ofthe areas assessed. In three of thesecontent areasearth science; life sci-ence; and environmental issues andthe nature of scienceU.S. fourthgrade students are significantly out-performed by only one or two othernations. In physical science, fiveother nations perform significantlybetter than the U.S.

If an international talent search wereto select the top ten percent of allfourth-grade students in the 26 coun-tries, in mathematics 9 percent ofU.S. fourth-grade students would beincluded. In science, 16 percentwould be included.

The international standing of U.S.fourth graders is stronger than thatof U.S. eighth graders in both mathe-matics and science.

In comparison with their internation-al counterparts, U.S. students per-form better in science than in mathe-matics at both the fourth and eighthgrades.

CONTEXTS OF LEARNING

It is too early in the process of dataanalysis to provide strong evidence tosuggest factors that may be related tothe patterns of achievementdescribed here. No single factor orcombination of factors emerges asparticularly important.

On most background factors studied,there is no difference between theU.S. and the international average,or the differences are small. There-fore, these factors are unlikely to bestrongly associated with our interna-tional standing.

On those background factors onwhich there is a difference betweenthe U.S. and the international aver-age, the factor is not shared withmost high-performing countries.Therefore, these factors are alsounlikely to be strongly associatedwith our international standing.

In general, preliminary analyses shedlittle light on factors which mightaccount for the differences betweenour performance in mathematicsand science, and our performance atthe fourth and eighth grades. Fur-ther analyses are needed to providemore definitive insights on these sub-jects.

CONCLUSION

This report presents initial findingsfrom TIMSS for fourth-grade mathemat-ics and science, and evidence from earlyanalyses concerning the context of U.S.education achievement. Adequateunderstanding of our nation's educa-tion in an international perspectivemust await findings from the twelfth-grade data and deeper analysis of data atall grade levels.

TIMSS is not an answer book, but a toolto examine our own national education-al strengths and weaknesses in an inter-national perspective. All countries,including the U.S., have something tolearn from other nations, and havesomething from which other countriescan learn. These TIMSS findings will bean important source of information toguide our nation in the pursuit of excel-lence into the next century.

12

PREFACE

BEST COPY AVAILABLE

13

OVERVIEW

The Third International Mathematicsand Science Study is the largest andmost comprehensive comparative inter-national study of education that has everbeen undertaken. A half-million stu-dents from 41 countries were tested in30 different languages at five differentgrade levels to compare their mathe-matics and science achievement. Inten-sive studies of students, teachers,schools, curriculum, instruction, andpolicy issues were also carried out tounderstand the educational context inwhich learning takes place.

Twenty-six countries tested fourth-gradestudents and made their data availablefor presentation. Of these, 17 met orcame close to meeting all of the qualitycontrol requirements for sampling anddata collection. The other 9 countriesexperienced difficulties of various types.All deviations from international qualitycontrol requirements are described inAppendix 4. The 9 countries withinwhich difficulties arose are shown inparentheses both below and in figurescontained in this report.

COUNTRIES PARTICIPATINGIN FOURTH-GRADE TIMSS

(Australia) Japan

(Austria) Korea

Canada (Kuwait)Cyprus (Latvia)Czech Republic (Netherlands)England New ZealandGreece NorwayHong Kong Portugal(Hungary) ScotlandIceland SingaporeIran, Islamic (Slovenia)

Republic (Thailand)Ireland United States(Israel)

Seventeen other countries participatedin one or more aspects of TIMSS but notin the fourth-grade study. These coun-tries are Belgium (Flemish), Belgium(French), Bulgaria, Colombia, Den-mark, France, Germany, Lithuania,Mexico, the Philippines, Romania, Russ-ian Federation, Slovak Republic, SouthAfrica, Spain, Sweden, and Switzerland.

TIMSS is an important study for thoseinterested in U.S. education. In 1983,the National Commission on Excellencein Education pointed to our nation's lowperformance in international studies asevidence that we were A Nation at Risk.In 1989, President Bush and the gover-nors of all 50 states adopted the Nation-al Goals for Education, one of which wasthat "by the year 2000, the U.S. will befirst in the world in mathematics and sci-ence achievement."

Mathematics and science experts haveissued major calls for reform in theteaching of their subjects. The NationalCouncil of Teachers of Mathematicspublished Curriculum and EvaluationStandards in 1989 and Professional Stan-dards for Teaching Mathematics in 1991. In1993, the American Association for theAdvancement of Science followed suitwith Benchmarks for Science Literacy, andin 1996, the National Academy of Sci-ences published National Science Educa-tion Standards.

TIMSS helps us measure progresstoward our national goal of improvingour children's academic performance inmathematics and science. But TIMSS ismuch more than a scorecard for themathematics and science events in the"education Olympics." It is a diagnostictool to help us examine our nation'sprogress toward improvement of mathe-matics and science education. It was

14

designed to look behind the scorecardto illuminate how our education policiesand practices compare with those of theworld community.

This report draws from the results of thefourth-grade part of the TIMSS study tosummarize the initial findings concern-ing mathematics and science achieve-ment and schooling at that grade level.It is the second of three reports in thePursuing Excellence series. The first reportpresented initial findings on the eighthgrade and was released in November1996. The third report, to be publishedin spring 1998, will treat findings con-cerning students in the twelfth grade.Other reports on selected topics will bepublished over the next several years.Much more will be learned as furtheranalysis of the TIMSS data continues.

TIMSS is a fair and accurate comparisonof mathematics and science achieve-ment in the participating nations. It isnot a comparison of "all of our studentswith other nations' best students," acharge that some critics have leveled atprevious international comparisons. Inmost of the countries that participatedin TIMSS, virtually all children attendelementary school, and the studentswho took the TIMSS test were randomlyselected to represent all students intheir respective nations. The entireassessment process was scrutinized byinternational technical review commit-tees to ensure its adherence to estab-lished standards. Those nations in whichirregularities arose are clearly noted inthis and other TIMSS reports.

At each step of its development, TIMSSused careful quality control procedures.An international curriculum analysis wascarried out prior to the development ofthe assessments to ensure that the tests

reflect the mathematics and science cur-ricula of the variety of TIMSS countriesand do not overemphasize what istaught in only a few. International mon-itors carefully checked the test transla-tions and visited many classrooms whilethe tests were being administered tomake sure that the instructions wereproperly followed. The raw data fromeach country were scrutinized to be surethat no anomalies existed, and all analy-ses were double checked. Finally, thisreport has been written and carefullyreviewed to avoid overgeneralizationand inaccuracy.

STUDY DESIGN

TIMSS is the third comparison of math-ematics and science achievement car-ried out by the International Associationfor the Evaluation of EducationalAchievement (IEA). Previous IEA stud-ies of mathematics and science wereconducted for each subject separately atvarious times during the 1960s, 1970s,and 1980s. TIMSS is the first IEA studythat has assessed both mathematics andscience at the same time. Comparativestudies of other subjects, including read-ing literacy (1992)' and computers ineducation (1993)2 have also been pub-lished by the IEA.

TIMSS was designed to focus on stu-dents at three different stages of school-ing: midway through elementary school,midway through lower secondaryschool, and at the end of upper sec-ondary school. Because countriesaround the world set different ages atwhich children should begin and com-plete school, decisions about which stu-dents should be tested needed to takeboth age and grade level into account.

15

The populations tested are listed below.Participation in Population 2 was

required of all TIMSS nations, but par-ticipation in Populations 1 and 3 wasoptional.

Population 1 Those studentsenrolled in the pair of adjacentgrades that contained the most 9-year -olds. (Grades 3 and 4 in the U.S.and most of the world. Grades 2 and3 or 4 and 5 in some nations.)

Population 2 Those students inthe pair of adjacent grades that con-tained the most 13-year-olds at thetime of testing. (Grades 7 and 8 inthe U.S. and most of the world.Grades 6 and 7 in a few nations.)

Population 3 Those students intheir final year of secondary school,whatever their age. (Grade 12 in theU.S. and most nations. Grades 9-13 insome nations.)

In all participating countries, studentsin both public and private schools wereadministered the TIMSS test. In all but afew of the TIMSS countries, virtually allPopulation 1 and 2 children areenrolled in school and were thereforeeligible to take the test. Testingoccurred 2 to 3 months before the endof the 1994-95 school year. Students withspecial needs and disabilities that wouldmake it difficult for them to take the testwere excused from the assessment. Ineach country, the test was translated intothe primary language or languages ofinstruction. All testing in the U.S. wasdone in the English language.

Countries participating in TIMSS col-lected information primarily throughassessments and questionnaires. Addi-tional information on the content of

textbooks and curriculum guides wasalso collected in a separate series of cur-riculum analyses. The 26 countries par-ticipating in the Population 1 part of theTIMSS study engaged in three types ofdata collection:

Mathematics and science assess-ments One and one-half hours inlength, the assessments includedboth multiple-choice and free-response items. A smaller number ofstudents also completed hands-onperformance assessments that will bereported later.

School, teacher, and student question-naires Students answered ques-tions about their mathematics and sci-ence studies and beliefs. Teachersanswered questions on their beliefsabout mathematics and science andon teaching practices. School admin-istrators answered questions aboutschool policies and practices.

Curriculum analysis This explorato-ry study compared mathematics andscience curriculum guides and text-books. It studied subject-matter con-tent, sequencing of topics, and expec-tations for student performance.

THE TIMSS RESEARCH TEAM

TIMSS was conducted by the IEA, whichis a Netherlands-based organization ofministries of education and researchinstitutions in its member countries.The IEA delegated responsibility foroverall coordination and managementof the TIMSS study to Professor AlbertBeaton at the TIMSS InternationalStudy Center, located at Boston College.Each of the IEA member nations that

made the decision to participate inTIMSS paid for and carried out the datacollection in its own country accordingto the international guidelines. Thecosts of the international coordinationwere paid by the National Center forEducation Statistics (NCES) of the U.S.Department of Education, the NationalScience Foundation (NSF), and theCanadian Government.

TIMSS in the United States was also fund-ed by NCES and NSF. Professor WilliamSchmidt of Michigan State University wasthe U.S. National Research Coordinator.Policy decisions on the study were madeby the U.S. National Coordinating Com-mittee. Lois Peak of NCES monitored theinternational and U.S. TIMSS projects.The U.S. data collection was carried outby Westat, a private survey research firm.Trevor Williams and Nancy Caldwell wereWestat project co-directors. The manyadvisors to the study are listed in Appen-dix 2.

The U.S. TIMSS team also includes theapproximately 4,000 third and 7,000fourth graders who took the assessment,and their principals and teachers in 190schools nationwide. Their cooperationhas made this report possible.

ORGANIZATION OF THIS REPORT

This report summarizes early findingsfrom the fourth-grade data. Both third-and fourth-grade students took theTIMSS test as part of Population 1, butthis initial report focuses on findings forthe fourth grade. Future reports basedon a more complete and extensiveanalysis of the data will provide deeperunderstanding and investigate relation-ships among the findings from the dif-ferent parts of the study. Extensive doc-umentation of the data collectionmethodologies and statistical analysesused in all participating countries isavailable in technical reports3 and quali-ty control reports4 published by theTIMSS International Study Center atBoston College. Similar technical detailfor the United States will soon be avail-able from NCES.

This report is based on the comparativedata for 26 countries published by theTIMSS International Study Center atBoston College. The report's purpose isto highlight initial findings concerningthe place of the United States amongthe participating nations.

Chapter 1 draws from the results of thestudent assessments to describe U.S. stu-dent achievement in mathematics andscience in comparison with their inter-national counterparts.

Chapter 2 summarizes the evidencefrom initial analyses to attempt toaddress the key findings illuminated inChapter 1.

17

CHAPTER 1:ACHIEVEMENT

KEY POINTS:

U.S. fourth graders score above average in

both mathematics and science compared

with the 26 nations in the TIMSS fourth-grade assessment.

U.S. students' international standing is

stronger at the fourth grade than it is atthe eighth grade in both mathematics and

science.

U.S. students' international standing is

stronger in science than it is inmathematics at both the fourth and eighthgrades.

In mathematics, 9 percent of U.S. fourth

graders would rank among the world's topten percent. In science, 16 percent of U.S.

fourth graders would rank among theworld's top ten percent.

There is no significant gender gap in

fourth-grade mathematics achievement.However, in some content areas of fourth-grade science, U.S. boys outperform U.S.

girls. 1 8

BEST COPY AVAILABLE

In the past, the mathematics and scienceachievement of U.S. students has been acause for concern. International studiesof these subjects conducted over the past30 years show that our students have notperformed as well as we might expect incomparison with their peers in othernations, especially in mathematics.

The patterns of academic achievement,however, vary widely. In a recent IEAstudy of reading literacy,5 U.S. fourthgraders were second only to Finland,and U.S. eighth graders ranked amongthe top nations. Data from the eighth-grade TIMSS study,' released in Novem-ber 1996, showed that U.S. eighthgraders score above the internationalaverage of the 41 TIMSS countries in sci-ence but below the international aver-age in mathematics.

HOW WELL DO U.S. FOURTHGRADERS DO IN MATHEMATICSAND SCIENCE?

Compared with their international coun-terparts, U.S. fourth graders performabove the international average of the26 TIMSS countries in both mathemat-ics and science. In science, our studentsare outperformed by only one coun-tryKorea.

Figures 1 and 2 on pages 20 and 21 showhow U.S. students perform in these sub-jects.

Tempting though it may be, it is not cor-rect to report U.S. scores by rank alone,as would be the case if one were to sayU.S. fourth graders are "number x inthe world in mathematics." This isbecause the process of calculating eachcountry's score from the sample of stu-

dents who took the test produces onlyan estimate of the country's score, notthe true score itself. This estimate has amargin of error which is expressed as a"plus or minus" interval around the esti-mated score. In TIMSS, we can say with95 percent confidence that comparisonsof other countries' scores to those of theU.S. are accurate plus or minus about 15points, depending on the design of thesample in the other countries. Compar-isons of the U.S. with the internationalaverage are accurate plus or minusabout 6 points. (Appendix 3 contains alist of country means and standarderrors.) Because a precise score cannotbe determined with perfect accuracy, tofairly compare the U.S. with other coun-tries, nations have been groupedaccording to whether their performanceis higher than, not significantly differentfrom, or lower than the U.S.

In mathematics, fourth-grade studentsin 7 countries outperform our fourthgraders (Singapore, Korea, Japan, HongKong, the Netherlands, the CzechRepublic, and Austria). Student perfor-mance in 6 countries is not significantlydifferent from ours (Slovenia, Ireland,Hungary, Australia, Canada, and Israel).U.S. fourth graders outperform theircounterparts in 12 nations (Latvia, Scot-land, England, Cyprus, Norway, NewZealand, Greece, Thailand, Portugal,Iceland, Islamic Republic of Iran, andKuwait).

In science, students in only one coun-tryKoreaoutperform U.S. fourthgraders. Student performance in 5countries is not significantly differentfrom ours (Japan, Austria, Australia, theNetherlands, and the Czech Republic),and U.S. fourth graders outperformtheir counterparts in 19 nations (Eng-land, Canada, Singapore, Slovenia, Ire-

19

19

FIGURE 1:NATIONS' AVERAGE MATHEMATICS PERFORMANCE COMPARED WITH THE U.S.

NATIONS WITH AVERAGE SCORESSIGNIFICANTLY HIGHER THAN THE U.S.

NATION AVERAGE

SINGAPORE 625

KOREA 611

JAPAN 597

HONG KONG 587

(NETHERLANDS) 577

CZECH REPUBLIC 567

(AUSTRIA) 559

NATIONS WITH AVERAGE SCORES NOTSIGNIFICANTLY DIFFERENT FROM THE U.S.

NATION AVERAGE

(SLOVENIA) 552

IRELAND 550

(HUNGARY) 548

(AUSTRALIA) 546

UNITED STATES 545

CANADA 532

(ISRAEL) 531

NATIONS WITH AVERAGE SCORESSIGNIFICANTLY LOWER THAN THE U.S

NATION

(LATVIA (LSS))

AVERAGE

525

SCOTLAND 0 520

ENGLAND *° 513

CYPRUS 502

NORWAY 502

NEW ZEALAND 499

GREECE 492

(THAILAND) 490

PORTUGAL 475

ICELAND 474

IRAN, ISLAMIC REPUBLIC 429

(KU WAIT) 400

INTERNATIONAL AVERAGE = 529

SOURCE: Mullis et al. (1997) Mathematics Achievementin the Primary School Years. Table 1.1. Boston College:

Chestnut Hill, MA.

NOTES:I. Nations not meeting international guidelines are shown in parentheses.2. Nations in which more than ten percent of the population was excluded from testing are shown with a Latvia is

designated US because only Latvian-speaking schools were tested, which represents less than 65 percent of thepopulation.

3. Nations in which a participation rate of 75 percent of the schools and students combined was achieved only afterreplacements for refusals were substituted are shown with a °.

4.The international average is the average of the national averages of the 26 nations.

land, Scotland, Hong Kong, Hungary,New Zealand, Norway, Latvia, Israel, Ice-land, Greece, Portugal, Cyprus, Thai-land, Islamic Republic of Iran, andKuwait).

SOME SPECIAL NOTES ON THETEST SCORES

TIMSS is a fair comparison of achieve-ment for several reasons. The test was

jointly developed and carefully reviewedby the participating countries to ensurethat the items reflected curriculum top-ics considered important in all countries,and did not over-emphasize the curricu-lum content taught in only a few. Inter-national monitors carefully reviewednations' adherence to guidelines toensure that significant numbers of stu-dents were not excluded from the testingprocess for any reason. Those nationsthat did exclude more than ten percent

ori

v.

FIGURE 2:NATIONS' AVERAGE SCIENCE PERFORMANCE COMPARED WITH THE U.S.

NATIONS WITH AVERAGE SCORESSIGNIFICANTLY HIGHER THAN THE U.S.

NATION AVERAGE

KOREA 597

NATIONS WITH AVERAGE SCORES NOTSIGNIFICANTLY DIFFERENT FROM THE U.S.

NATION AVERAGE

JAPAN 574

UNITED STATES 565

(AUSTRIA) 565

(AUSTRALIA) 562

(NETHERLANDS) 557

CZECH REPUBLIC 557

INTERNATIONAL AVERAGE = 524

SOURCE: Martin et al. (1997) Science Achievement inthe Primary School Years. Table 1.1. Boston College:

Chestnut Hill, MA.

NATIONS WITH AVERAGE SCORESSIGNIFICANTLY LOWER THAN THE U.S

NATION AVERAGE

ENGLAND 0 551

CANADA 549

SINGAPORE 547

(SLOVENIA) 546

IRELAND 539

SCOTLAND 0 536

HONG KONG 533

(HUNGARY) 532

NEW ZEALAND 531

NORWAY 530

(LATVIA (LSS)) 512

(ISRAEL) 505

ICELAND 505

GREECE 497

PORTUGAL 480

CYPRUS 475

(THAILAND) 473


(KUWAIT) 401

NOTES:I. Nations not meeting international guidelines are shown in parentheses.2. Nations in which more than ten percent of the population was excluded from testing are shown with a*. Latvia is

designated LSS because only Latvian-speaking schools were tested, which represents less than 65 percent of the

population.3. Nations in which a participation rate of 75 percent of the schools and students combined was achieved only after

replacements for refusals were substituted are shown with a °.4.The international average is the average of the national averages of the 26 nations.

of their students are clearly noted in thisand other TIMSS reports. Therefore, wecan be sure that the TIMSS scores in thisreport are a fair comparison of virtuallyall students at the appropriate grade inthe various countries.

TIMSS required participating nations toadhere to extremely high technical stan-dards at all stages of participation in theproject. Of the 26 nations that partici-pated at the fourth grade, 17 met or

BEST COPY AVAILABLE

came close to meeting all technical stan-dards for the study. The remaining 9nations, however, experienced difficul-ties of various types. In some countries,the problems arose because a sizableproportion of schools, teachers, or stu-dents declined to participate. In others,the selection of schools or classroomswas not carried out according to inter-national specifications. In still others,students were slightly older than theinternational target age. The names of

21

those nations in which major difficultiesarose are shown in parentheses in thefigures in this report, and Appendix 4describes the problems each encoun-tered. Because of the problems, thesame amount of confidence cannot beattached to the scores of these 9 coun-tries as to the other 17.

When the international average is calcu-lated only from the 17 countries thatmet the international specifications, themathematics and science scores of U.S.fourth graders are still above the inter-national average for these 17 countries.Figure 3 on page 23 shows our standingin comparison with these 17 nations.

What do the test scores mean? Due tothe complex nature of the design, scor-ing, and analysis of the TIMSS test, ascore of 600 does not mean either 600items, or 60 percent, correct. Instead,this score indicates where the perfor-mance would fall if all fourth-gradescores were arranged along a scale run-ning from 0 to 1,000.

In mathematics, the international aver-age score is 529. A score of 658 or abovewould put a student in the top ten per-cent of all mathematics students in the26 TIMSS countries, and a score of 601would put a student in the top quarter.

In science, the international averagescore is 524. A score of 660 or higherwould put a student in the top ten per-cent of all science students, and a scoreof 607 would put a student in the topquarter.

WHICH COUNTRIESOUTPERFORM U.S. FOURTHGRADERS IN BOTH SUBJECTS?

We can say with confidence that 5 coun-tries outperform the U.S. in mathemat-ics at the fourth grade (Singapore,Korea, Japan, Hong Kong, and theCzech Republic). The Netherlands andAustria also outperform us in fourth-grade mathematics, but due to devia-tions in their administration of TIMSS,we have less confidence in their scores.Only one nation outperforms the U.S.in science (Korea). Therefore, at thefourth grade, only Korea outperformsthe U.S. in both mathematics and sci-ence. Four nations that outperform usin mathematics are not significantly dif-ferent from us in science (Japan, Aus-tria, the Netherlands, and the CzechRepublic). Two nations that outperformthe U.S. in mathematics score lowerthan the U.S. in science (Singapore andHong Kong).

WHICH COUNTRIES DO U.S.FOURTH GRADERS OUTPERFORMIN BOTH SUBJECTS?

We can say with confidence that in bothmathematics and science, U.S. fourthgraders outperform their counterpartsin 9 countries. These are:

Seven European countries Ice-land, England, Scotland, Norway,Greece, Cyprus, and Portugal.

Islamic Republic of Iran and NewZealand.

U.S. fourth graders also outperformLatvia, Kuwait, and Thailand in bothsubjects, but due to deviations in their

O

FIGURE 3:AVERAGE ACHIEVEMENT OF NATIONS MEETING, AND NOT MEETING, INTERNATIONAL GUIDELINES

COUNTRIES COMPLYINGWITH SPECIFICATIONS

NATION MATH SCIENCEAVERAGE AVERAGE

CANADA 532 549

CYPRUS 502 475

CZECH REPUBLIC 567 557

ENGLAND *° 513 551

GREECE 492 497

HONG KONG 587 533

ICELAND 474 505

IRAN, ISLAMIC REPUBLIC 429 416

IRELAND 550 539

JAPAN 597 574

KOREA 611 597

NEW ZEALAND 499 531

NORWAY 502 530

PORTUGAL 475 480

SCOTLAND ° 520 536

SINGAPORE 625 547

UNITED STATES 545 565

MATHEMATICS INTERNATIONAL AVERAGE = 531

SCIENCE INTERNATIONAL AVERAGE = 528

SOURCE: Mullis et al. (1997) Mathematics Achievementin the Primary School Years. Table 1.1. Boston College:

Chestnut Hill, MA and Martin et al. (1997) ScienceAchievement in the Primary School Years. Table 1.1. Boston

College: Chestnut Hill, MA.

COUNTRIES WITHLOW PARTICIPATION RATES


AUSTRALIA 546 562

AUSTRIA 559 565

LATVIA (LSS) 525 512

NETHERLANDS 577 557

COUNTRIES TESTINGOLDER-THAN-SPECIFIED STUDENTS


SLOVENIA 552 546

COUNTRIES WITH NON-STANDARDSELECTION OF CLASSROOMS


HUNGARY 548 532

COUNTRIES WITH NON-STANDARDSELECTION OF CLASSROOMS AND OTHER

DEPARTURES FROM GUIDELINES


ISRAEL 531 505

KUWAIT 400 401

THAILAND 490 473

NOTES:I. Nations in which more than ten percent of the population was excluded from testing are shown with a *. Latvia is

designated LSS because only Latvian-speaking schools were tested, which represents less than 65 percent of the

population.2. Nations in which a participation rate of 75 percent of the schools and students combined was achieved only after

replacements for refusals were substituted are shown with a 0.3.The international average is the average of the national averages of the 17 nations meeting international guidelines.

23

23BEST COPY AVAI BL

administration of TIMSS, we have lessconfidence in their scores.

HOW DO OUR FOURTH GRADERSCOMPARE WITH OUR MAJOR ECO-NOMIC PARTNERS?

The "Group of Seven" or G-7 countriesare major U.S. economic and politicalallies. The other six nations in thisgroup are Canada, France, Germany,Italy, Japan, and the United Kingdom.Italy did not administer the TIMSS test,and France and Germany did not partic-ipate in the testing of fourth-grade stu-dents. Thus the U.S. can only be com-pared with the United Kingdom, Cana-da, and Japan. The United Kingdom ismade up of England, Scotland, North-ern Ireland, and Wales; however, the lat-ter two did not participate in TIMSS.England and Scotland both have thesame international standing in compari-son with the U.S. Therefore, in this sec-tion, we describe our standing in rela-tion to England.

Except for Japanese students' scores inmathematics, U.S. fourth graders' math-ematics and science scores are similar toor higher than those of the other threeparticipating G-7 nations. Japanesefourth graders outperform their U.S.counterparts in mathematics. U.S.fourth graders' mathematics scores arenot significantly different from those ofCanada and are higher than those ofEngland. In science, U.S. fourthgraders' scores are not significantly dif-ferent than those of Japan and are sig-nificantly higher than those of Canadaand England.

HOW DO OUR BEST FOURTHGRADERS COMPARE WITHOTHERS' BEST?

Comparisons of averages tell us how typ-ical students perform, but they do nottell us about the performance of ournation's best students, including thosewho are likely to continue to study math-ematics and science in secondary schooland eventually become the next genera-tion of mathematicians, scientists, doc-tors, and engineers. If an internationaltalent search were to select the top tenpercent of all fourth-grade students inthe 26 TIMSS countries combined, whatpercentage of U.S. students would beincluded?

In mathematics, 9 percent of U.S.fourth graders are in the world's top tenpercent. This is well below the 39 per-cent of Singaporean students, 26 per-cent of Korean students, and 23 percentof Japanese students who would be cho-sen in the international mathematics tal-ent search. In science, 16 percent ofU.S. fourth graders would rank amongthe world's top ten percent. No countryhas significantly more of their studentsin the top ten percent, and 21 nationshave a smaller percentage. Figure 4 onpage 25 shows results for selected coun-tries.

If the international talent search were tolower its standards so as to choose thetop half of all students in the 26 TIMSScountries, in mathematics 56 percent ofU.S. fourth graders would be included.This compares with 85 percent in Korea,82 percent in Singapore, and 79 percentin Japan. In science, 63 percent of U.S.fourth graders would be in the top halfof the students in the TIMSS countries,compared with 81 percent of students inKorea and 68 percent in Japan.

24

FIGURE 4:PERCENTAGE OF STUDENTS FROM SELECTED NATIONS SCORING AMONG THE TOP TEN PERCENT OF

FOURTH GRADERS IN THE 26 TIMSS COUNTRIES

SINGAPORE

KOREA

JAPAN

HONG KONG

CZECHREPUBLIC

(AUSTRALIA)

(HUNGARY)

UNITED STATES

CANADA

ENGLAND

11

17

4 23

11

oti

0

13

16

18

26

MATHEMATICS

SCIENCE

5 10 15 20 25 30

PERCENTAGE OF STUDENTS

NOTE: Nations not meeting international guidelines are shown in parentheses.

35 40

SOURCE: Mullis et al. (1997) Mathematics Achievement in the Primary School Years. Table 1.4. Boston College: ChestnutHill, MA and Martin et al. (1997) Science Achievement in the Primary School Years. Table 1.4. Boston College: Chestnut

Hill, MA.

O 25

HOW DO OUR FOURTHGRADERS SCORE IN THEDIFFERENT CONTENT AREAS OFMATHEMATICS AND SCIENCE?

Representing student achievement inmathematics and science as a total scoreis a useful way to summarize achieve-ment. Mathematics and science, howev-er, contain very different content areaswhich are emphasized and sequenceddifferently in curricula around theworld. Based on national priorities,some content areas are emphasizedmore than others at a particular gradelevel.

The TIMSS fourth-grade mathematicstest included sets of items designed tosample students' ability to do work inthe following areas:

Whole Numbers (place value; ordering;comparing; problem-solving usingaddition, subtraction, and multiplica-tion).

Fractions and Proportionality (recogni-tion and work with fractions and dec-imals; word problems).

Measurement, Estimation, and NumberSense (common measures of size,time, temperature; rounding andestimation).

Data Representation, Analysis, and Prob-ability (use of data in charts, tables,and graphs; basic concepts underly-ing probability).

Geometry (visualization of two- andthree-dimensional forms; basic termsand properties; equivalence of fig-ures; coordinate points on grids).

Patterns, Relations, and Functions (pat-terns of numbers and shapes; repre-

sentation of simple numerical situa-tions; relationships of sequences ofnumbers).

In five of the six TIMSS mathematicscontent areas, the scores of U.S. fourthgraders are above the international aver-ages for those content areas. U.S.fourth-graders' performance is abovethe international average in whole num-bers; fractions and proportionality; datarepresentation, analysis and probability;geometry; and patterns, relations, andfunctions. In only one content area isthe U.S. average below the internationalaveragemeasurement, estimation andnumber sense. Figure 5 on pages 28-29shows these results.

In science, the TIMSS fourth-grade testsampled students' ability to do work inthe following subjects:

Earth Science (earth features; earthprocesses; earth in the solar system).

Life Science (structure; diversity; classi-fication; processes; cycles; and inter-actions of plants and animals).

Physical Science (matter; energy andphysical processes; forces andmotion; physical and chemicalchanges).

Environmental Issues and the Nature ofScience (environmental and resourceissues; nature of scientific knowledge;interaction of science and technol-ogy).

U.S. fourth graders score above theinternational average in all four sciencecontent areas. In three of these contentareasearth science; life science; andenvironmental issues and the nature ofscienceU.S. fourth-grade students areoutperformed by one or two other

26

26

nations. In physical science, U.S. stu-dents are outperformed by 5 othernations. Figure 6 on pages 30-31 showsthese results.

IS THERE A GENDER GAP INMATHEMATICS AND SCIENCE ATTHE FOURTH GRADE?

Policymakers in the U.S. and othercountries have made great efforts inrecent years to make mathematics andscience more accessible to girls and toencourage gender equity in these sub-jects. Overall, at the fourth grade, moreTIMSS countries have gender equity inmathematics than in science.

The U.S. is one of 22 TIMSS nations inwhich there is no significant gender gapin fourth-grade mathematics achieve-ment. For the overall science score, theU.S. is one of ten countries where a gen-der gap exists. Examining boys' andgirls' scores in the various science con-tent areas, U.S. boys significantly out-perform U.S. girls in the content areasof earth science and physical science.There is no significant differencebetween U.S. boys' and girls' scores inlife science and in environmental issuesand the nature of science.

HAS U.S. FOURTH-GRADEINTERNATIONAL STANDINGIMPROVED OVER TIME?

International comparisons over time aredifficult. The first international studiesof mathematics and science achieve-ment were conducted in the 1960s, andthere have been other assessments ineach subject since that time. However,most assessments have focused on mid-

dle-school students and students in thefinal year of high school. Assessments ofstudents in the elementary schoolgrades have been conducted less fre-quently. Prior to TIMSS, only one inter-national assessment of elementary-school children was undertaken inmathematics, although there were threeprior assessments in science.

However, each assessment was done a lit-tle differently. A different set of nationsparticipated, different topics in mathe-matics and science were included in thetests, the age and type of students sam-pled in each country changed slightly,and even the borders and names ofsome of the nations have changed.These and other factors complicatecomparisons over time and require thatany conclusions that are drawn be nec-essarily tentative.

Among the various international studiesconducted over the past 30 years, onlythe International Assessment of Educa-tional Progress (IAEP) tested the math-ematics achievement of elementary-school students. In the 1991 IAEP assess-ment of 9-year-olds in 14 nations,' U.S.students scored below the internationalaverage in mathematics. However, as wehave seen, in the 1995 TIMSS studyreported here, U.S. fourth gradersscored above the international averageof 26 nations in this subject. Compar-isons over time are difficult, so cautionshould be exercised in assuming therehas been significant improvement in ourfourth graders' international standing inmathematics, but it is a possibility.

Three previous international scienceassessments of elementary-school stu-dents were conducted in the 1960s,1980s, and early 1990s. The U.S. scoredabove the international average in twoof these three studies. In the other study,

27

FIGURE 5:NATIONAL AVERAGES IN MATHEMATICS CONTENT AREAS

WHOLE NUMBERS FRACTIONS MEASUREMENT,AND PROPORTIONALITY ESTIMATION,

AND NUMBER SENSE

NATIONS WITH AVERAGESCORES SIGNIFICANTLYHIGHER THAN THE U.S.

NATION PERCENT CORRECT

KOREA

SINGAPORE

JAPAN

HONG KONG(HUNGARY)(NETHERLANDS)

CZECH REPUBLIC

(AUSTRIA)

(SLOVENIA)

88

83

82

79

76

75

75

74

74

NATIONS WITHAVERAGE SCORES NOT

SIGNIFICANTLYDIFFERENT FROM THE U.S.


(ISRAEL)

UNITED STATES

IRELAND

CANADA(LATVIA (LSS))

71

71

70

68

68

NATIONS WITH AVERAGESCORES SIGNIFICANTLYLOWER THAN THE U.S.


(AUSTRALIA)

CYPRUS

GREECE

SCOTLAND °NORWAY

ENGLAND *0

(THAILAND)

PORTUGAL

NEW ZEALAND

ICELAND

IRAN, ISLAMIC REPUBLIC

(KUWAIT)

67

656261

61

58

58

57

57

56

51

36

()Represents International Average



SINGAPORE

HONG KONGJAPANKOREA

(NETHERLANDS)

IRELAND

74

6665

65

60

58




CZECH REPUBLIC

(AUSTRIA)

(AUSTRALIA)

UNITED STATES

(SLOVENIA)

(HUNGARY)

CYPRUS

(ISRAEL)

CANADA

53

51

51

51

50494(S)48

48

48



SCOTLAND °ENGLAND *0(LATVIA (LSS))

(THAILAND)

GREECE

NEW ZEALAND

NORWAY

PORTUGAL

ICELAND

IRAN, ISLAMIC REPUBLIC

(KUWAIT)

46

45

4444

42

41

38

38

36

32

25



JAPAN 72

KOREA 72

(NETHERLANDS) 70(AUSTRIA) 69HONG KONG 69CZECH REPUBLIC 68SINGAPORE 67(HUNGARY) 64(SLOVENIA) 64(LATVIA (LSS)) 60(AUSTRALIA) 60




IRELAND 56NORWAY 56CANADA 54(ISRAEL) 54SCOTLAND ° 53

UNITED STATES 53ENGLAND '° 52



PORTUGAL 49NEW ZEALAND 49GREECE 48CYPRUS 48ICELAND 44(THAILAND) 44


(KUWAIT) 35

NOTES:I. Nations not meeting international guidelines are shown in parentheses.2. Nations in which more than ten percent of the population was excluded from testing are shown with a *. Latvia is

designated LSS because only Latvian-speaking schools were tested, which represents less than 65 percent of thepopulation.

23

FIGURE 5 (CONTINUED):

NATIONAL AVERAGES IN MATHEMATICS CONTENT AREAS

DATA REPRESENTATION,ANALYSIS, AND PROBABILITY



SINGAPORE 81

KOREA 80

JAPAN 79




HONG KONG 76

(NETHERLANDS) 75

UNITED STATES 73



IRELAND 69

CANADA 68

(AUSTRALIA) 67

CZECH REPUBLIC 67

(AUSTRIA) 66

SCOTLAND ° 66

ENGLAND ''D 64

(ISRAEL) 64

(SLOVENIA) 64/NEW ZEALAND 61

(HUNGARY) 60

NORWAY 59

ICELAND 58

(THAILAND) 56

(LATVIA (LSS)) 54

CYPRUS 52

GREECE 50

PORTUGAL 43

(KUWAIT) 26


GEOMETRY



HONG KONG(AUSTRALIA)

74

74




ENGLAND *° 74

SCOTLAND ° 72

JAPAN 72

SINGAPORE 72

KOREA 72

CANADA 72

(SLOVENIA) 72

(NETHERLANDS) 71

UNITED STATES 71

CZECH REPUBLIC 71



(AUSTRIA) 67

(LATVIA (LSS)) 67

IRELAND 66

NEW ZEALAND 66

(HUNGARY) 66iaICELAND 63 \--=

(ISRAEL) 62

NORWAY 58

GREECE 53

(THAILAND) 53

CYPRUS 53

PORTUGAL 52


(KUWAIT) 36

PATTERNS, RELATIONS,AND FUNCTIONS



KOREA 83

JAPAN 76

SINGAPORE 76

HONG KONG 73




(HUNGARY) 69

(SLOVENIA) 68

CZECH REPUBLIC 67

UNITED STATES 66

(LATVIA (LSS)) 65

(NETHERLANDS) 65

(AUSTRIA) 64

(AUSTRALIA) 64

IRELAND 64

CANADA 62



(ISRAEL) 60(SCOTLAND ° 57

CYPRUS 55ENGLAND *° 55

NEW ZEALAND 52

NORWAY 50

(THAILAND) 50

ICELAND 48

PORTUGAL 47

GREECE 47


(KUWAIT) 33

NOTES (continued):3. Nations in which a participation rate of 75 percent of the schools and students combined was achieved only after

replacements for refusal were substituted are shown with a *.4.The international average is the average of the national averages of the 26 nations.5.The placement of England in Geometry may appear out of place; however, statistically its placement is correct.

SOURCE: Mullis et al. (1997) Mathematics Achievement in the Primary School Years. Table 2.1. Boston College: Chestnut

Hill, MA.

29

FIGURE 6:NATIONAL AVERAGES IN SCIENCE CONTENT AREAS

EARTH SCIENCE



KOREA 72

JAPAN 66




(SLOVENIA) 64

CZECH REPUBLIC 64UNITED STATES 64(HUNGARY) 62(AUSTRIA) 62CANADA 62ENGLAND *° 61

(NETHERLANDS) 61

(AUSTRALIA) 61



HONG KONG 61

IRELAND 60NORWAY 60SINGAPORE 58SCOTLAND ° 58NEW ZEALAND 57(LATVIA (LSS)) 57--ICELAND 55GREECE 52(ISRAEL) 51

PORTUGAL 50(THAILAND) 48CYPRUS 48IRAN, ISLAMIC REPUBLIC 38(KUWAIT) 36

7

LIFE SCIENCE



KOREA 76




JAPAN 73

(NETHERLANDS) 73

(AUSTRALIA) 72

(AUSTRIA) 72

CZECH REPUBLIC 71

UNITED STATES 71

SINGAPORE 70



CANADA 68ENGLAND *0 68(SLOVENIA) 68HONG KONG 68NORWAY 67NEW ZEALAND 66IRELAND 66(HUNGARY) 66SCOTLAND 0 654(ISRAEL) 61

GREECE 61

(LATVIA (LSS)) 60ICELAND 60CYPRUS 55PORTUGAL 54(THAILAND) 52(KUWAIT) 45IRAN, ISLAMIC REPUBLIC 44

®30

PHYSICAL SCIENCE



KOREA 75JAPAN 70(NETHERLANDS) 65SINGAPORE 64(AUSTRIA) 64




(AUSTRALIA) 63CZECH REPUBLIC 62(SLOVENIA) 61

CANADA 61

UNITED STATES 60ENGLAND *0 60HONG KONG 60(HUNGARY) 59



SCOTLAND ° 57IRELAND 57NEW ZEALAND 57

(ISRAEL) 55NORWAY 55

(LATVIA (LSS)) 54ICELAND 52CYPRUS 50PORTUGAL 49GREECE 49(THAILAND) 46IRAN, ISLAMIC REPUBLIC 40(KUWAIT) 37



KOREA 70




UNITED STATES

(AUSTRALIA)

JAPAN

65

63

62



(NETHERLANDS)

ENGLAND *0

CANADACZECH REPUBLIC

IRELAND

(SLOVENIA)

NEW ZEALAND

(AUSTRIA)

SINGAPORE

SCOTLAND

NORWAY

(ISRAEL)

(HUNGARY)

HONG KONG(THAILAND)

ICELAND

(LATVIA (LSS))

GREECE

CYPRUS

PORTUGAL

61

56

56

56

55

5454

54

53

53

53

514050

50

48

47

46

43

42

39


(KUWAIT) 25

()Represents International Average

NOTES:I. Nations not meeting

international guidelinesare shown in parentheses.

2. Nations in which morethan ten percent of thepopulation was excludedfrom testing are shownwith a *. Latvia is designat-ed LSS because only Lat-

vian-speaking schools

were tested, which repre-sents less than 65 percentof the population.

3. Nations in which a partici-pation rate of 75 percentof the schools and stu-dents combined wasachieved only afterreplacements for refusalswere substituted areshown with a °.

4.The international averageis the average of thenational averages of the

26 nations.5.The placement of Hong

Kong in Earth Science mayappear out of place; how-ever, statistically its place-

ment is correct.

SOURCE: Martin et al.(1997) Science Achievement

in the Primary School Years.

Table 2.1. Boston College:Chestnut Hill, MA.

REST COPY AVAI LABLE

the U.S. was not different than the inter-national average. Moreover, in all threeprevious studies, only a few nations out-performed the U.S. In the 1960s study,one nation out of 11 (Japan); in the1980s study, 5 nations out of 14 (Japan,Korea, Finland, Sweden, and Hun-gary)8, and in the 1991 study, onenation out of 13 (Korea) outperformedthe U.S.9 Taken together with theTIMSS findings reported here, itappears that U.S. students in the middleyears of elementary school perform rea-sonably well in science in comparisonwith their peers in other nations. It isnot clear whether this relative interna-tional standing has changed over time.

HOW DOES THE PERFORMANCEOF U.S. FOURTH GRADERSCOMPARE WITH THAT OF U.S.EIGHTH GRADERS INMATHEMATICS AND SCIENCE?

In both mathematics and science, ourinternational standing is higher atfourth grade than it is at eighth grade.Figure 7 on page 32 provides a quickoverview of mathematics and scienceperformance at each grade level, incomparison with all of the countriesparticipating at each grade level.

In mathematics, our fourth-grade stu-dents score above the internationalaverage, while our eighth-grade stu-dents score below the internationalaverage. In science, U.S. students scoreabove the international average at bothgrade levels. However, only one nationoutperforms us at the fourth grade,while 9 nations outperform us at theeighth grade.

31

BEST COPY AVM BLE

FIGURE 7:U.S. MATHEMATICS AND SCIENCE PERFORMANCE AT A GLANCE

How Do U.S. Students Compare withthe International Average in...?

At Grade 4?(26 nations)

At Grade 8?(41 nations)

Mathematics overall Above Below

Science overall Above Above

Mathematics Content Areas:

Data representation, analysis, and probability Above AboveGeometry Above BelowWhole numbers Above xFractions and proportionality Above xPatterns, relations, and functions Above xMeasurement, estimation, and number sense Below xFractions and number sense x SameAlgebra x SameMeasurement x BelowProportionality x Below

Science Content Areas

Earth science Above AboveLife science Above AboveEnvironmental issues and the nature of science Above AbovePhysical science Above xChemistry x SamePhysics x Same

What Percentage of U.S. Students Would Bein the International Top Ten Percent In...? At Grade 4? At Grade 8?

Mathematics 9% 5%

Science 16% 13%

KEY:Above = U.S. average performance higher than the average of participating nations at that grade.Below = U.S. average performance lower than the average of participating nations at that grade.Same = U.S. average performance not significantly different than the average of participating nations at

that grade.x = Separate content area score not reported for this grade level.

SOURCE: Mullis et al. (1997) Mathematics Achievement in the Primary School Years. Boston College,Chestnut Hill, MA; Martin et al. (1997) Science Achievement in the Primary School Years. Boston College,Chestnut Hill, MA; and U.S. Department of Education, National Center for Education Statistics.(1996).Pursuing Excellence: A Study of U.S. Eighth-Grade Mathematics and Science Teaching Learning Curriculum, andAchievement in International Context NCES 97-198,Washington, DC: Government Printing Office.

32 3 2

This pattern in relative internationalstanding is also evident when one takesinto account the fact that 41 nations par-ticipated in eighth-grade TIMSS, where-as 26 nations participated in the fourth-grade study. Comparisons of U.S. totalperformance with the internationalaverage for the 26 nations that partici-pated in both the fourth-grade andeighth-grade TIMSS studies confirm thisobservation. Among these 26 nations inmathematics, U.S. fourth graders scoreabove the international average and areoutperformed by 7 nations, whereasU.S. eighth graders score below theinternational average and are outper-formed by 13 nations. In science, U.S.fourth graders score above the interna-tional average and are outperformed byonly one other nation. U.S. eighthgraders' scores are not significantly dif-ferent from the international averageand are lower than 8 other nations.

While the U.S. international standing ishigher at fourth grade than at eighthgrade, most other countries which par-ticipated in TIMSS at both grades (19 inboth mathematics and science) have asimilar standing relative to the interna-tional average at both grade levels. Fivecountries have a lower relative standingat eighth grade than at fourth grade inone subject, and 4 countries (the U.S.,Scotland, Ireland, and Latvia) have alower relative standing at eighth gradein both subjects. Only one countrytheU.S. in mathematicsfalls from abovethe international average at fourthgrade to below the international averageat eighth grade.

Another way of looking at the U.S. per-formance at fourth and eighth grade isto see how many countries comparemore favorably to the U.S. at eighthgrade than they did at fourth grade. Of

the 25 other countries that participatedat both grade levels, all perform as wellor better relative to the U.S. at eighthgrade than they do at fourth grade. Thatis, no country compares less favorably tothe U.S. in eighth grade than it does infourth grade, and most compare morefavorably. In both subjects, most of thecountries (5 out of 6 in mathematics and4 out of 5 in science) with average scoressimilar to the U.S. in fourth grade havescores in eighth grade that are signifi-candy higher than the U.S. Likewise,many of the countries (8 of 12 in mathe-matics and 9 of 19 in science) whosescores are below the U.S. in fourth gradehave eighth-grade scores that are similarto the U.S., and in science, 3 countries(Singapore, Slovenia, and Hungary)have fourth-grade scores below the U.S.and eighth-grade scores above the U.S.

HOW DOES THE PERFORMANCEOF U.S. FOURTH GRADERS COM-PARE WITH THAT OF U.S. EIGHTHGRADERS IN MATHEMATICS ANDSCIENCE CONTENT AREAS?

The picture for the content areas ofmathematics and science is somewhatmore complicated. Figure 7 on page 32displays the results by content area. Inmathematics, the fourth-grade andeighth-grade tests have only two contentareas in which scores are reported forboth grade levels (data representation,analysis, and probability; and geome-try). U.S. students' scores in data repre-sentation, analysis and probability aresignificantly higher than the interna-tional average at both grade levels. U.S.students' scores in geometry are abovethe international average at the fourthgrade and below the average at theeighth grade.

33

With regard to the other mathematicscontent areas, U.S. fourth gradersexceed the international average inthree content areas (whole numbers;fractions and proportionality; and pat-terns, relations, and functions) but arebelow it in one (measurement, estima-tion, and number sense). Eighthgraders are not different from the inter-national average in two content areas(fractions and number sense; and alge-bra) and are below the internationalaverage in two areas (measurement andproportionality).

In science, the fourth-grade and eighth-grade tests have three content areas forwhich scores are reported at both gradelevels. U.S. fourth-graders and eighth-graders' scores are higher than theinternational average in all three ofthese content areas (earth science; lifescience; and environmental issues andthe nature of science). U.S. fourth-graders' scores are above the interna-tional average in physical science,whereas eighth-graders' scores are notdifferent than the international averagein physics and chemistry.

SUMMARY

The foregoing discussion of TIMSSachievement findings highlights threeimportant patterns:

U.S. fourth gradersinternational averagematics and science.

are above thein both mathe-

The international standing of U.S.fourth graders is stronger than thatof eighth graders in both subjects.

U.S. students perform better in sci-ence than in mathematics at both thefourth and eighth grades in compari-son with their international counter-parts.

The next chapter explores the initial evi-dence from TIMSS concerning variousfactors that may contribute to these pat-terns.

34

CHAPTER 2:CONTEXTS OF

LEARNING

KEY POINTS:

It is too early in the process of dataanalysis to provide strong evidence tosuggest factors that may be related to thepatterns of achievement described here.No single factor or combination of factorsemerges as particularly important.

On most background factors studied, thereis no difference between the U.S. and theinternational average, or the differences aresmall. Therefore, these factors are unlikelyto be strongly associated with ourinternational standing.

On those background factors on whichthere is a difference between the U.S. andthe international average, the factor is notshared with most high-performingcountries. Therefore, these factors are alsounlikely to be strongly associated with ourinternational standing.

In general, preliminary analyses shed littlelight on factors which might account forthe differences between our performancein mathematics and science, and ourperformance at the fourth and eighthgrades. Further analyses are needed toprovide more definitive insights on thesesubjects.

35 BEST COPY AVM aSLE

In Chapter 1, we found that U.S. fourthgraders score above the internationalaverage in both mathematics and sci-ence, and that in science, our fourth-graders' average score is exceeded byonly one other country. This chapterexamines the early data available fromTIMSS about the educational contextwithin which our students learn for evi-dence that might contribute to the threemajor findings that:

U.S. fourth graders score above theinternational average in both mathe-matics and science.

The international standing of U.S.fourth graders is stronger than thatof eighth graders in both subjects.

Our students' international standingis better in science than in mathe-matics at both fourth and eighthgrades.

The evidence presented in this chapteris not conclusive for two reasons. Thefirst is that TIMSS questionnaire analy-ses are still in their early stages, andtherefore much more information willbe available later concerning familybackground, teacher beliefs about math-ematics and science, and many othertopics that experts believe are associatedwith student performance in these sub-jects. The second reason is that fourth-grade TIMSS lacks evidence that wasavailable at the eighth grade from video-tape and case studies from several othercountries with which to supplement thequestionnaires. For this reason, we donot have detailed information aboutclassroom instruction, teacher training,students' daily lives, and other key topicsin a variety of countries.

Furthermore, the TIMSS study wasdesigned in such a way that more infor-

mation was collected about mathematicsthan about science, and about theeighth grade than about the fourth ortwelfth grade. This complicates compar-isons between mathematics and scienceand across grade levels.

International studies have matured tothe point that it is widely recognizedthat there is no "magic tonic" or singlefactor that is always present in everyhigh-performing country and absent inevery low-performing country. Indeed,education is a vast system of many inter-related parts. No single factor or easilyidentifiable set of factors is clearlyresponsible for high achievement. Fur-thermore, every characteristic of a high-performing country does not necessarily"cause" its high achievement.

Data on some factors such as studenteconomic status, ethnicity, and othersthat are known to be associated with dif-ferences in achievement between stu-dents in the U.S. are not available forother countries. However, factors suchas these cannot be an explanation forthe differences in the internationalstanding of the U.S. at the fourth gradein comparison with the eighth grade, orin mathematics in comparison with sci-ence, because the same students weretested in both subjects, and there is littledifference between the economic statusand ethnicity of fourth- and eighth-grade students in the U.S. This is alsothe case in other countries.

Definitive determination of which fac-tors contribute to higher achievement isbeyond the reach of initial analyses suchas those reported here. However, suchpreliminary evidence can be helpful fortwo reasons. First, it can demonstratewhich factors do not appear to bestrongly related to differences in studentperformance. Second, it can indicate

36

directions in which further researchmight look for explanations of studentperformance. Therefore, let us reviewthe currently available initial evidenceconcerning factors that might con-tribute to these three major findings.

WHAT FACTORS MIGHTCONTRIBUTE TO THE FINDINGTHAT U.S. FOURTH GRADERSSCORE ABOVE THEINTERNATIONAL AVERAGE INBOTH MATHEMATICS ANDSCIENCE?

Logically, if we are to suggest that anyparticular factor may be related to theU.S.' relatively high international stand-ing in mathematics and science atfourth grade, then that same factorshould be one on which the U.S. is sig-nificantly different from the interna-tional average in both mathematics andscience. Furthermore, the factor shouldusually be found in other high-perform-ing countries and not in low-performingones.

If the pattern of evidence for a given fac-tor does not fit both of these criteriasimultaneously, logically, it is not likelyto contribute strongly to the relativelyhigh international standing of the U.S.at the fourth grade. Using these criteriafor judging the evidence, the followingsection examines the factors for whichTIMSS data are currently available to seehow the educational context in whichU.S. fourth graders learn differs fromthe international average.

HOW DOES THE U.S. FOURTH-GRADE MATHEMATICS ANDSCIENCE CURRICULUM DIFFERFROM THE INTERNATIONALAVERAGE?

When considering curriculum, a distinc-tion should be made between the offi-cially intended curriculum and whatteachers actually teach. National, state,and local authorities, as well as publish-ers, set forth the officially intended cur-riculum in both curriculum guidelinesand textbooks. Depending on the coun-try, teachers make decisions about whatto teach based more or less closely onthe officially intended curriculum. Whatteachers actually teach their students issometimes called the "implemented cur-riculum." Both the officially intendedcurriculum and the implemented cur-riculum must be considered when dis-cussing a nation's goals for learning.

U.S. curriculum is not determined at thenational level, as it is in most TIMSScountries. In 18 of the 26 countries thatparticipated in TIMSS at the fourthgrade, curriculum is primarily deter-mined at the national level. It is primar-ily determined at the state level in onecountry, Canada. Decisions about cur-riculum are not centralized in theremaining 7 countries, one of which isthe U.S. In many countries where cur-riculum decision-making is not central-ized, decisions are made at the districtor local level.

TIMSS studied the "intended curricu-lum," as set forth by state and localauthorities in the approximately 40countries that participated in the fourth-grade curriculum analysis. Experts ineach country were provided withdetailed lists of topics and asked tojudge which topics were intended to be

3 7

taught at various grades. Examples ofdetailed topics in mathematics are"common fractions" and "whole num-ber operations." Examples in scienceare "weather and climate" and "electric-ity." The number of topics intended tobe taught is defined here as a measureof curricular focus. Nations in whichfewer topics are intended to be taughtare considered to have a more focusedcurriculum. Some experts believe thathaving fewer topics in the intended cur-riculum may facilitate higher studentachievement by allowing the teaching ofeach topic in more depth.

When the judgments of the U.S. expertswere compared with the internationalaverage of the approximately 40 coun-tries, it was found that the number oftopics intended for coverage in the U.S.at the fourth grade was above the inter-national average in mathematics, andsomewhat below the international aver-age in science. In grades one, two, andthree, the number of topics intendedfor coverage in the U.S. is above theinternational average in both mathe-matics and science. Thus, the evidenceis mixed: the U.S. fourth-grade intendedcurriculum is more focused in scienceand less focused in mathematics thanthe international average, but both areless focused compared with the interna-tional average at grades one, two, andthree. Therefore, greater curricularfocus (or fewer intended topics) couldnot be a strong factor contributing toour above-average fourth-grade perfor-mance in both subjects.

At this time, we are unable to saywhether or not the U.S. fourth-gradeimplemented curriculum is morefocused than the international average.We can only compare expert judgmentsof the officially intended curriculum

because teacher reports from the TIMSSquestionnaires of what they actuallyteach are not yet available for any coun-try besides the U.S.

DO U.S. FOURTH GRADERSSPEND MORE TIME IN CLASSSTUDYING MATHEMATICS ANDSCIENCE?

Our fourth-grade students spend moretime in class per week learning mathe-matics and science than do their averageinternational counterparts. U.S. fourthgraders receive an average of 4.2 hoursof instruction per week in mathematics.This is 18 minutes more per week thanthe international average of 3.9 hours.When science is taught as a separate sub-ject, U.S. students receive an average of2.7 hours of instruction per week, whichis 48 minutes more than the interna-tional average of 1.9 hours per week. Ofthe countries in which fourth-grade sci-ence is taught as a separate subject, onlyPortugal has significantly more hours ofinstruction.

However, caution should be exercised inconsidering class time as a factor thatmight contribute to U.S. fourth graders'above-average performance in mathe-matics, given that 4 of the 7 nations thatoutperform us in mathematics spendless time in class per week than the U.S.and also less than the international aver-age. Figure 8 on page 40 shows the com-parison of U.S. weekly hours of instruc-tion with the international averages.

38

IS HOMEWORK MORE COMMONIN THE U.S. THAN IN OTHERCOUNTRIES?

U.S. fourth-grade teachers assign aboutthe same amount of mathematics home-work as teachers in most other coun-tries. When teachers were asked howoften they assign mathematics home-work, the most common response in themajority of TIMSS countries was "threeor more times a week." Teachers of 71percent of U.S. fourth-grade studentsrespond in this way, a figure slightlyabove the international average of 66percent. Teachers of another 22 percentof U.S. students report that they assignhomework once or twice a week. Figure9 on page 41 shows these results. Thepercentage of U.S. fourth graders

Y 4.5he

31 4.0 3 9cca.

03.0

an 2.5

Lo- 2.0

0 1.5

to 1,0

0.5

receiving mathematics homework threeor more times per week is lower than 4of the 7 countries that outperform us inmathematics, and higher than the other3 countries that outperform us.

When asked, "If you assign homework,how many minutes do you usuallyassign?," teachers of most U.S. fourthgraders respond "30 minutes or less,"which is also the case in most otherTIMSS countries. TIMSS did not askteachers about the amount of sciencehomework that they assign fourth-gradestudents.

FIGURE 8:TEACHERS' REPORT ON AVERAGE HOURS

OF MATHEMATICS AND SCIENCE INSTRUCTION PER WEEK

4.2

3.5

0.0

U.S.

INTERNATIONAL

AVERAGE

MATHEMATICS SCIENCE (AS SEPARATE SUBJECT)

SOURCE: Mullis et al. (1997) Mathematics Achievement in the Primary School Years.Table 5.4. Boston College: Chestnut Hill, MA and Martin et al. (1997) ScienceAchievement in the Primary School Years. Table 5.5. Boston College: Chestnut Hill, MA.

O39

HOW DOES THE STRUCTURE OFU.S. MATHEMATICS AND SCIENCEINSTRUCTION DIFFER FROMTHAT IN OTHER TIMSS NATIONS?

U.S. fourth graders usually study mathe-matics and science from the sameteacher, and this is typical of fourth-grade students in most other TIMSScountries. While three Asian countrieshave large class sizes of between 36 and43 students, U.S. average class size (24)is close to the international average (25)for the other countries. The three Asiancountries with large class sizes outper-form the U.S. in math, but only one ofthese countries outperforms us in sci-ence.

80

70

60z50

,W^

z 30UC

jr. 20

40

10

IS U.S. CLASSROOMORGANIZATION DIFFERENT THANTHAT OF OTHER COUNTRIES?

Information about teaching collected inthe fourth-grade TIMSS study is basedon teacher questionnaire reports anddoes not provide as rich or detailed apicture as the TIMSS videotape studythat was conducted in eighth-grademathematics classrooms. However, ini-tial analyses of the questionnaire datasuggest that organization for instructionin U.S. fourth-grade classrooms is simi-lar to that in other countries.

In most TIMSS countries, teachers offourth graders report that their twomost common patterns for organizinginstruction used in most or every lesson

FIGURE 9:PERCENTAGE OF STUDENTS WHOSE TEACHERS ASSIGN

VARIOUS AMOUNTS OF MATHEMATICS HOMEWORK

71 3-5 TIMES PER WEEK

1-2 TIMES PER WEEK

E0 LESS THAN ONCE PER WEEK

III NEVER

U.S. INTERNATIONAL AVERAGE

SOURCE: Mullis et al. (1997) Mathematics Achievement in the Primary School Years.Table 5.19. Boston College: Chestnut Hill, MA.

41

40

in both mathematics and science is toteach the class as a whole, and to havethe students work individually with assis-tance from the teacher. These two pat-terns are also the most common pat-terns in the U.S. in fourth-grade mathe-matics. Science in the U.S. is slightly dif-ferent from mathematics because thesecond most common pattern is to havestudents work together as a class withstudents responding to each other,rather than individually with assistancefrom the teacher.

Teachers sometimes organize studentsto work in pairs, or in small groups withteacher assistance. The U.S. is close tothe international average in the fre-quency with which teachers have stu-dents work in pairs or small groups withteacher assistance. Teachers of aboutone-fifth of students in the U.S. reportthat they use this pattern in most orevery mathematics lesson and teachersof one-fourth of students use it in mostor every science lesson.

ARE CALCULATORS ANDCOMPUTERS MORE COMMON INU.S. FOURTH-GRADECLASSROOMS THAN IN OTHERTIMSS COUNTRIES?

U.S. fourth graders use calculators andcomputers in mathematics class morefrequently than do students in mostother TIMSS countries. Use of calcula-tors in U.S. fourth-grade mathematicsclasses is about twice the internationalaverage. In the U.S., teachers of 39 per-cent of the students report having stu-dents use calculators in their mathemat-ics classes at least once or twice a weekcompared with the international aver-age of 18 percent. Internationally, the

teachers of two-thirds of the TIMSS stu-dents report that they never or hardlyever have students use calculators intheir mathematics classes comparedwith the teachers of one-third of U.S.students. In 6 of the 7 nations thatoutscore the U.S. in mathematics, teach-ers of 85 percent or more of the stu-dents report that students never use cal-culators in class.

Among the 26 countries that participat-ed in fourth-grade TIMSS, teachers inthe U.S. and Canada are among themost likely to report that students usecomputers in at least some mathematicslessons. Teachers of 37 percent of theU.S. students report that computers areused in at least some lessons, in compar-ison with 13 percent of students interna-tionally. Teachers in 5 of the 7 countriesthat outscore the U.S. in mathematicsreport that they never or almost neverhave students use computers in mathe-matics lessons.

In all countries, fourth-grade studentswere not allowed to use calculators orcomputers when taking the TIMSS test.

ARE U.S. FOURTH-GRADETEACHERS BETTER TRAINED THANTHEIR COLLEAGUES IN OTHERTIMSS COUNTRIES?

The profile of a typical U.S. teacher offourth graders is similar to that of teach-ers in most other TIMSS countries: awoman at least 40 years old with morethan 10 years of teaching experience.However, teachers of U.S. fourth gradershave more university training than theircounterparts in most TIMSS countries.This is because the U.S. is one of only 10of the 26 nations that participated in

TIMSS at the fourth grade that requireselementary school teachers to earn a uni-versity degree before being certified toteach. This university degree typicallyrequires 4 years. Thirteen TIMSS coun-tries require a degree from a non-univer-sity teacher-training institution, whichusually requires 3 years to complete.Three nations certify teachers fromeither universities or teacher traininginstitutions. Among the 7 countries thatoutperform the U.S. in fourth-grademathematics, 3 countries require a uni-versity degree and the other 4 countriesrequire a degree from a 3-year teacher-training institution. Almost all TIMSScountries, including the U.S., requiresome period of teaching or practiceexperience, as well as an evaluation orexamination of the teacher before fullteacher certification is granted.

U.S. fourth-grade teachers meet withother teachers to discuss curriculumabout as frequently as their colleagues inother TIMSS countries. Mathematicsteachers of about 60 percent of U.S. stu-dents report that they meet at least oncea week with other teachers in their sub-ject area to discuss and plan curriculumor teaching approaches; the same is truein science. About 40 percent say theymeet less frequently. Teachers in manyother TIMSS countries report meetingwith similar frequency.

DO U.S. TEACHERS EXPERIENCEFEWER PROFESSIONALCHALLENGES THAN DO TEACHERSIN OTHER TIMSS COUNTRIES?

U.S. teachers' perceptions of their pro-fessional challenges are similar to butpossibly less limiting than teachers inmost of the 26 TIMSS countries. Teach-

ers of fourth graders in most TIMSScountries, including the U.S., most fre-quently cite varying academic abilities ofstudents, and a high student-teacherratio as factors that limit how they teachtheir class either "quite a lot" or "a greatdeal." The next most frequent chal-lenges are shortages of equipment foruse in demonstrations and other exer-cises, and disruptive students. However,teachers of a smaller percentage of U.S.students report that these factors limithow they teach their classes than theinternational average. Figure 10 on page44 shows that the U.S. is below the inter-national average with respect to all fourof these challenges, including classroomdiscipline. It is important to remember,however, that this information repre-sents teachers' subjective impressionsrather than actual measures of the class-room environment in the various coun-tries.

DO U.S. STUDENTS HAVE MOREEDUCATIONAL RESOURCES INTHEIR HOMES THAN DO TYPICALSTUDENTS IN OTHER TIMSSCOUNTRIES?

Experts believe that homes that havebooks, a desk in a quiet place for study,a dictionary, a computer, and other edu-cational resources provide an environ-ment that fosters academic develop-ment. U.S. fourth graders have moreeducational resources in their homesthan the average student in other TIMSScountries, but this is not true of mostother high-performing countries. Aboutone-half of U.S. fourth graders reportthat they have three key resources intheir home: a dictionary, their own studydesk, and a computer, compared withabout one-third of students in the 26

42

TIMSS countries. A smaller percentageof fourth graders in 4 of the 7 countriesthat outperform us in mathematics andthe one country that outperforms theU.S. in science report having all three ofthese educational resources in theirhomes than in the U.S. Of the other 3countries that outperform the U.S. inmathematics, in one country, the per-centage is about the same as the U.S., inanother country the percentage is larg-er, and in another nation the studentswere not asked this question.

Fifty-six percent of U.S. fourth gradersreport having at least 100 books in theirhomes, which is greater than the inter-national average of 42 percent. A small-er percentage of students in 3 of the 7nations that outperform us in mathe-matics report having more than 100books, and about the same percentage

report having at least this many books inthe 3 other nations. One nation did notask its students this question.

Computers are more common in thehomes of U.S. fourth graders than theinternational average, but the U.S. doesnot lead the world in this respect. In theU.S., 56 percent of fourth graders reporthaving a computer at home, comparedwith the international average of 49 per-cent. In 5 countries, 75 percent or moreof fourth graders report having a com-puter at home: Scotland, England, theNetherlands, Iceland, and Ireland. Onlyone of these nations (the Netherlands)outscores the U.S. in mathematics andnone in science. In 4 of the 7 nationsthat outscore the U.S. in mathematics,fewer students report having a comput-er at home than do students in the U.S.

FIGURE 10:PERCENTAGE OF MATHEMATICS STUDENTS WHOSE TEACHERS REPORT

VARIOUS CIRCUMSTANCES LIMIT THEIR TEACHING

'QUITE A LOT' OR 'A GREAT DEAL'

DIFFERENT ACADEMIC ABILITIES

HIGH STUDENT/TEACHER RATIO

EQUIPMENT SHORTAGE

DISRUPTIVE STUDENTS

61

si INTERNATIONAL AVERAGE

U.S. AVERAGE

10 20 30 40 50 60 70 80

PERCENTAGE OF STUDENTS

SOURCE: Mullis et al. (1997) Mathematics Achievement in the Primary School Years.Figure 5.4. Boston College: Chestnut Hill, MA.

® 43

DO U.S. FOURTH GRADERS HAVEMORE POSITIVE ATTITUDESTOWARD MATHEMATICS ANDSCIENCE THAN DO STUDENTS INOTHER COUNTRIES?

More U.S. fourth graders believe it isimportant to do well in math and scienceand have confidence about their perfor-mance in these subjects than their inter-national counterparts.

Over 90 percent of students in mostTIMSS countries believe that it is impor-tant to do well in mathematics and sci-ence. U.S. students are even more likelyto agree that it is important to do well inthese subjects. In the U.S., virtually allfourth graders think it is important todo well: 98 percent believe it is impor-tant to do well in mathematics, and 97percent believe it is important to do wellin science.

A large majority of fourth graders in theU.S. have confidence that they are doingwell in mathematics and science, andthis is the case in other countries as well.Ninety-one percent of U.S. fourthgraders either agree or strongly agreethat they are doing well in mathematicsand science. In both mathematics andscience, U.S. fourth graders are morelikely than the international average tostrongly agree that they are doing well inthese subjects.

Fourth graders in the U.S. report theylike both mathematics and science, andthe percentage who report this is aboutthe same as the average of all TIMSS stu-dents. In the U.S., 84 percent of fourthgraders report that they either "like"mathematics or like mathematics "a lot."In science, this proportion is 85 percent.

IS HEAVY TELEVISION WATCHINGLESS COMMON AMONG U.S.FOURTH GRADERS THAN AMONGSTUDENTS IN OTHERCOUNTRIES?

More U.S. fourth graders watch largeamounts of TV than the internationalaverage. Thirty-two percent of U.S. stu-dents report watching three hours ormore of television on a normal schoolday. This is higher than the internation-al average of 25 percent of students whoreport watching this much. In 4 of the 7nations that outperform us in mathe-matics, the percentage of students whoreport watching three or more hours oftelevision per night is smaller than theinternational average. In one nation,the percentage is about the same as theinternational average, and in one it islarger. One nation did not ask this ques-tion of its students.

SUMMARY

Let us return to the question with whichthis section started: "What factors mightcontribute to the finding that U.S.fourth graders score above the interna-tional average in both mathematics andscience?" We have examined the earlyevidence from the TIMSS question-naires about how various factors relatedto our fourth-grade education in gener-al, and our mathematics and scienceeducation in particular, differ from theinternational average. We have alsoexamined whether or not these differ-ences are also characteristic of mosthigh-performing countries. It is unlikelythat any of the factors described in thischapter, when considered in isolation,contribute strongly to U.S. performancefor two reasons. First, on most of the

44

background factors studied, there is nodifference between the U.S. and theinternational average, or the differencesare small. Second, on those factors onwhich there is a difference between theU.S. and the international average, thefactor is not shared with most high-per-forming countries.

Rather than considering single factors inisolation, it is possible that the combinedeffect of several factors creates an educa-tional environment that nurtures U.S.above-average performance. Cautionshould be exercised, however, in assum-ing that this is the case because most ofthe characteristics of U.S. fourth-gradeeducation also characterize U.S. eighth-grade education, and at the eighthgrade, U.S. students score below theinternational average in mathematics.

There may be other important factorsabout U.S. education that contribute toour above-average performance at thefourth-grade level that were not mea-sured by the TIMSS study. There alsoare likely to be factors that were mea-sured by the TIMSS study but have notyet been fully analyzed that will providemore information about what con-tributes to our students' above-averageperformance at this grade level.

Even though differences from the inter-national average may not appear to havea strong relationship to our internation-al performance, understanding such dif-ferences helps us view the context ofU.S. education in comparative perspec-tive. Therefore, let us summarize the ini-tial questionnaire findings about howU.S. fourth-grade mathematics and sci-ence education differs from the interna-tional average.

Curriculum is not determined at thenational level in the U.S., as it is inmost TIMSS countries.

Students in the U.S. spend more classtime per week studying mathematicsand science than the internationalaverage.

Teachers in the U.S. assign about thesame amount of mathematics home-work as teachers in other TIMSScountries.

Students in the U.S. use calculatorsand computers in mathematics classmore often than do students in mostother TIMSS countries.

Teachers in the U.S. have more uni-versity training than do their col-leagues in many other TIMSS coun-tries.

U.S. teachers' professional chal-lenges are perceived to be similar to,but possibly less limiting than, thoseexperienced by teachers in otherTIMSS countries.

Students in the U.S. have more edu-cational resources in their homesthan the international average.

More U.S. students believe it is

important to do well in mathematicsand science, and have more confi-dence about their performance inthese subjects, than their averageinternational counterpart.

More U.S. fourth graders watch largeamounts of TV than their interna-tional counterparts.

O 45

WHAT FACTORS MIGHTCONTRIBUTE TO THE FINDINGTHAT THE INTERNATIONALSTANDING OF U.S. FOURTHGRADERS IS STRONGER THANTHAT OF U.S. EIGHTHGRADERS?

In Chapter 1, we have seen that in bothmathematics and science our students'international standing is higher at thefourth grade than it is at the eighthgrade. In mathematics, among the 26nations that participated at the fourthgrade, our students' total test scores areabove the international average whereasour eighth-graders' scores are below theinternational average of the 41 nationsthat participated at this grade level. Inscience, our students score above theinternational average for the 26 nationsat the fourth grade and the 41 nations atthe eighth grade. However, comparingour performance with the 26 nationsthat participated at both grade levels,whereas one nation outperforms us atthe fourth grade, 8 nations outperformus at the eighth grade. The pattern ofU.S. younger students performing rela-tively better compared with their inter-national peers is not unique to TIMSS. Itis a pattern that has been observed inmost previous international assessmentsin mathematics, science, and reading.m

Later analyses will allow more in-depthstudy of our students' better interna-tional standing at the fourth grade thanat the eighth grade. At the time of theseinitial analyses, identical data are notavailable for some factors at both thefourth and eighth grades, and for bothmathematics and science. Further dataanalysis will allow expanded investiga-tion of these questions. In addition,some of the explanations may lie with

factors that occur prior to the fourthgrade or occur in the grades betweenfourth and eighth, for which there arelittle TIMSS data.

The following section examines the ini-tial evidence about factors that mightcontribute to the international standingof our fourth graders being strongerthan that of our eighth graders. Logical-ly, for a factor to be strongly related toour better international performance atthe fourth grade, it should be present indifferent amounts at the fourth andeighth grades relative to the interna-tional average, and this differenceshould be found in both subjects.

For example, to consider the amount ofhomework that U.S. students areassigned as an important factor con-tributing to differences in performancebetween the two grades, two pieces ofevidence are desirable. First, theamount of homework fourth gradersreceive should be higher relative to thethe international average at the fourthgrade than it is at the eighth grade. Sec-ond, this should be the case in bothmathematics and science. Using thesecriteria for judging the evidence, the fol-lowing section examines the earlyTIMSS data available at this time.

IS THE U.S. FOURTH-GRADECURRICULUM MORE FOCUSEDTHAN THAT OF THE EIGHTHGRADE?

Comparing expert judgments of theofficially intended mathematics curricu-lum, there does not appear to be muchdifference between the fourth gradeand the eighth grade. The number ofmathematics topics judged as intended

46

by experts is substantially above theinternational average at both grade lev-els. In science, the number of topicsexperts judged to be intended is slightlybelow the international average at thefourth grade and somewhat above at theeighth grade. If focus is defined as asmaller number of intended topics, theevidence concerning the intended cur-riculum is mixed: in mathematics, thereis little difference in the amount offocus relative to the international aver-age between fourth and eighth grade. Inscience, the fourth-grade curriculummay be more focused than the eighth-grade curriculum, relative to the otherTIMSS countries. If the full span ofgrades from first to eighth is examined,the U.S. intended curriculum containsmore topics than the international aver-age at every grade in both subjects,except for fourth-grade science.

To learn about the implemented cur-riculum, or what is actually taught,teachers in the 26 nations that partici-pated in fourth-grade TIMSS were givenlists of broad categories of subject mat-ter, referred to here as "topic areas."Examples of topic areas in mathematicsare "whole numbers," and "fractionsand decimals," and in science are "earthfeatures" and "matter." Teachers wereasked to check off those topic areas thatthey had already taught their fourthgraders that year or planned to teachthem before the school year ended inorder to provide an estimate of teachingcoverage for the whole year. At this time,only U.S. data are available on theimplemented curriculum. Therefore,we can compare the differencesbetween the fourth and eighth grades inthe U.S., but we cannot tell how relative-ly focused the U.S. curriculum is com-pared to other countries.

In mathematics, on average, U.S. fourthgraders are taught 14 out of 20 topicareas, and eighth graders are taught 16out of 21 topic areas. In science, on aver-age, U.S. fourth graders are taught 16out of 22 topic areas, and eighth gradersare taught 14 out of 22 topic areas.

ARE U.S. FOURTH GRADERSASSIGNED RELATIVELY MOREMATHEMATICS HOMEWORKTHAN ARE U.S. EIGHTHGRADERS?

U.S. fourth graders are less frequentlyassigned mathematics homework thanare eighth graders, but the frequency ofhomework is higher than the interna-tional average in both grades. Accordingto teacher reports, 71 percent of U.S.fourth graders and 86 percent of U.S.eighth graders are assigned mathemat-ics homework three or more times perweek. These percentages of students aregreater than the international averagefor the fourth and eighth grades.

IS U.S. FOURTH-GRADEMATHEMATICS AND SCIENCEINSTRUCTION DIFFERENT FROMTHAT IN THE EIGHTH GRADE?

U.S. fourth graders usually study mathe-matics and science from the sameteacher, and eighth graders usually studyit from different teachers, which is alsotrue for the majority of students in mostother TIMSS countries.

Mathematics class sizes in U.S. fourthand eighth grades are approximatelyequal and are not far from the interna-tional average for both grade levels.

Except for a few Asian countries withvery large classes, U.S. class sizes arequite similar to the average of the rest ofthe countries at both grade levels.

IS U.S. FOURTH-GRADEMATHEMATICS CLASSROOMORGANIZATION DIFFERENTFROM THAT IN THE EIGHTHGRADE?

Teachers of fourth- and eighth-gradestudents report similar patterns of orga-nization in their mathematics classes,and at both grade levels these are com-parable with the international averages.At both grade levels, in most or every les-son, the most common pattern of math-ematics instruction is for the teacher toteach the whole class, or to have stu-dents work individually with assistancefrom the teacher during at least somepart of most lessons.

ARE CALCULATORS USED MORECOMMONLY IN U.S. FOURTH-GRADE MATHEMATICS CLASSESTHAN IN EIGHTH-GRADECLASSES?

U.S. fourth graders use calculators lessfrequently in mathematics classes thando U.S. eighth graders. Teachers of 39percent of U.S. fourth graders reportthat students use calculators in mathe-matics class at least once or twice a week,in comparison with 82 percent of eighthgraders. At both grade levels, teachers ofU.S. students are more likely to reportthat they have students who use calcula-tors in class than the internationalaverage.

ARE U.S. FOURTH-GRADETEACHERS MORE EXPERIENCEDTHAN EIGHTH-GRADE TEACHERS?

In mathematics, there is little differencebetween the fourth and eighth grades inthe percentage of students whose teach-ers have more than 10 years of experi-ence. In science, more fourth-gradeteachers than eighth-grade teachershave this much experience. In mathe-matics, teachers of 62 percent of stu-dents at both grade levels have morethan 10 years of teaching experience. Inscience, teachers of 62 percent offourth-grade students and 52 percent ofeighth-grade students have more than10 years of experience. These percent-ages are close to the international aver-age for fourth- and eighth-grade mathe-matics and fourth-grade science. For sci-ence, the percentage of U.S. eighth-grade students whose teachers havemore than ten years of experience is lessthan the international average.

DO U.S. FOURTH-GRADESTUDENTS HAVE MOREEDUCATIONAL RESOURCES INTHEIR HOMES THAN DO EIGHTHGRADERS?

Fewer U.S. fourth graders have theirown study desk, a computer, and a dic-tionary at home than do eighth graders,but the percentage of U.S. students whohave all three of these study aids athome is larger than the internationalaverage at both grade levels. At thefourth grade, 49 percent of U.S. stu-dents have all three study aids, and atthe eighth grade, 56 percent report hav-ing all three.

49 48

The percentage of students who reporthaving a computer at home, and thepercentage who report having at leastthree bookcases filled with books (or atleast 200 books) at home is about thesame for both grade levels, and both ofthese percentages are above the interna-tional averages.

DO U.S. FOURTH-GRADESTUDENTS WATCH RELATIVELYLESS TV THAN EIGHTH-GRADESTUDENTS?

U.S. students watch more hours of TV inboth fourth and eighth grades than theinternational average.

DO U.S. FOURTH GRADERS LIKEMATHEMATICS AND SCIENCEMORE THAN DO U.S. EIGHTHGRADERS?

A majority of U.S. fourth and eighthgraders report that they like both math-ematics and science, but more fourthgraders report liking these subjects thando eighth graders. In mathematics, 84percent of fourth graders say that they"like" mathematics or "like it a lot," com-pared with 71 percent of eighth graders.In science, 85 percent of U.S. fourthgraders say that they like the subject orlike it a lot compared with 71 percent ofeighth graders. These percentages areclose to the international averages forfourth- and eighth-grade science andfourth-grade mathematics. For mathe-matics, the percentage of U.S. eighthgraders who like mathematics or like it alot is above the international average.

SUMMARY

None of the factors among those con-sidered appear to strongly contribute tothe better international standing of U.S.fourth graders in comparison with U.S.eighth graders in both subjects. Amongthe factors for which we have compara-ble data at both grades, some factorsexhibit little or no difference betweenfourth and eighth grades or the patternof evidence is mixed. In fact, each of thedifferences in the U.S. between thefourth and eighth grades listed below istypical of most other TIMSS countriesand is also typical of the internationalaverages for these two grade levels. Forsome factors, such as homework, in-classcalculator use, and possession of studyaids in the home, U.S. fourth gradershave less of these factors than do U.S.eighth graders. However, compared withthe international average on these fac-tors, there is no difference in the U.S.standing between the fourth and eighthgrades. Therefore, evidence concerningwhy the international standing of U.S.fourth graders is stronger than that ofeighth graders is inconclusive andincomplete. Further analyses will possi-bly shed more light on this question.

To review the initial findings presentedhere concerning the differencesbetween fourth grade and eighth grade:

While U.S. fourth graders are less fre-quently assigned mathematics home-work than are eighth graders, bothare above the international average.

In the U.S., as well as other TIMSScountries, fourth graders usuallystudy mathematics and science fromthe same teacher, and eighth gradersstudy it from different teachers. U.S.eighth graders are less likely to have

science teachers with more than tenyears of experience than are U.S.fourth graders.

U.S. fourth graders use calculators inmathematics classes less frequentlythan do eighth graders and both areabove the international average.

While fewer U.S. fourth graders havetheir own study desk, computer, anddictionary at home than eighthgraders, the percentages are higherthan the international average atboth grades.

U.S. fourth graders report that theylike mathematics and science morethan eighth graders, but only U.S.eighth graders in mathematicsexceed the international average.

None of these factors, either in isolationor in combination, appear to contributeheavily to the relative differencebetween U.S. fourth- and eighth-gradeperformance.

WHAT FACTORS MIGHTCONTRIBUTE TO THE FINDINGTHAT THE INTERNATIONALSTANDING OF U.S. STUDENTSIS STRONGER IN SCIENCETHAN IN MATHEMATICS?

In Chapter 1, we have seen that at boththe fourth and eighth grades, our inter-national standing is stronger in sciencethan it is in mathematics. In science, ourstudents are above the internationalaverage at both grade levels. Among the26 nations that participated in TIMSS atboth the fourth and eighth grades, onlyone nation outperforms us in science at

the fourth grade, and 8 nations outper-form us at the eighth grade. In mathe-matics, our students are above the inter-national average at the fourth grade andbelow it at the eighth grade. Sevennations outperform us in mathematicsat the fourth grade and 13 nations out-perform us at the eighth grade.

Logically, if we are to suggest that anyparticular factor may be related to ourstudents' stronger international stand-ing in science than in mathematics, thatfactor should be present in differentamounts in mathematics and science,relative to the international average.This difference should also be found atboth the fourth and eighth grades.

Further analyses will allow more in-depth investigation of these questions.Based on the data available at this time,it is not always possible to compare back-ground characteristics of U.S. eighth-grade education to the internationalaverage. Currently available TIMSS dataallow within-U.S. comparison of differ-ences between fourth-grade mathemat-ics and science for only a few back-ground factors. At the time of these ini-tial analyses, comparable analyses of thedata for the eighth grade have not beenconducted. Later analyses will allow formuch deeper investigation of these sub-jects.

IS THE U.S. CURRICULUM MOREFOCUSED IN SCIENCE THAN INMATHEMATICS?

In mathematics, according to expertjudgment concerning the number oftopics officially intended in mathemat-ics, the U.S. intended curriculumincludes more topics than the interna-

5 0

tional average at both the grades, fourthand eighth. In science, the number oftopics in the U.S. intended curriculumis somewhat below the internationalaverage at grade 4, and close to theinternational average at grade 8. If allgrade levels between one and eight areexamined, science appears closer to theinternational average than mathematics.Thus, the U.S. intended curriculum maybe more focused in science than inmathematics.

When asked what they actually teach,teachers of 73 percent of U.S. fourth-grade students report teaching fewerthan 17 mathematics topic areas peryear. In science, the comparable figure

Ct

80

70

60

50

O-w 40

30

Ua.w 20

10

is 50 percent. At the eighth grade, math-ematics teachers of 55 percent of U.S.students teach less than 17 topics peryear, in comparison to science teachersof 75 percent of U.S. students. Figure11, below, shows these findings. At thistime, only U.S. data are available on theimplemented curriculum. Therefore,we cannot compare these U.S. findingsto those of other countries.

In summary, evidence concerning cur-riculum focus is mixed. Relative toother countries, the U.S. intended cur-riculum is more focused in science thanin mathematics at both the fourth andeighth grades.

FIGURE 11:PERCENTAGE OF FOURTH- AND EIGHTH-GRADE STUDENTS WHOSE TEACHERS REPORT

TEACHING VARIOUS NUMBERS OF MATHEMATICS AND SCIENCE TOPICS

4th Grade

4

0 I

1 TO 8 9 TO 16

69 MATHEMATICS

1:1 SCIENCE

8th Grade

17 TO 22 1 T08

NUMBER OF TOPIC AREAS

SOURCE: Third International Mathematics and Science Study teacher questionnaires; unpublished tabulationsby Michigan State University and Westat

9 TO 16 17 TO 22

51

DO U.S. STUDENTS SPENDRELATIVELY MORE TIME IN CLASSSTUDYING SCIENCE THANMATHEMATICS?

As described above, U.S. fourth gradersspend more time in class per week study-ing mathematics than studying science.In mathematics, U.S. fourth gradersreceive 4.2 hours per week of instructionand 2.7 hours per week in science. Inboth subjects, U.S. fourth gradersreceive more instruction per week thanthe international average. However, rel-ative to the international average, U.S.fourth graders receive considerablymore additional science instruction thanmathematics instruction. U.S. fourthgraders receive 48 minutes more scienceinstruction per week than the interna-tional average, and 18 minutes moremathematics instruction per week thanthe international average.

Initial analyses do not provide compara-ble information for the eighth grade inboth subjects, nor do they provide anestimate of total amount of class timeper year, taking into account the num-ber of weeks in the school year.

ARE-CLASS SIZES DIFFERENT INSCIENCE THAN IN MATHEMATICS?

At the fourth grade, there is usually nodifference between class sizes in mathe-matics and science because most stu-dents in the U.S. and in the majority ofTIMSS countries study mathematics andscience in the same class from the sameteacher.

ARE U.S. SCIENCE TEACHERSMORE EXPERIENCED THANMATHEMATICS TEACHERS?

Evidence concerning teacher experi-ence is mixed. Both absolutely, andcompared with the international aver-age, U.S. eighth graders are less likely tohave teachers with more than 10 years ofexperience in science than in mathe-matics. In the U.S., 52 percent of eighth-grade students have science teacherswith more than 10 years of experience,compared with the international aver-age of 62 percent. In mathematics, 62percent of U.S. students have teacherswith more than 10 years of experience,which is not significantly different fromthe international average.

At the fourth grade, in the U.S. andmost countries, mathematics teachershave the same amount of teaching expe-rience as science teachers because thesame teacher teaches both subjects.

ARE U.S. STUDENT ATTITUDESMORE POSITIVE TOWARDSCIENCE THAN THEY ARETOWARD MATHEMATICS?

Approximately equal percentages ofU.S. students have positive attitudestoward mathematics as have positive atti-tudes toward science. In both subjects,at both the fourth and eighth grades, amajority of students report that they likethese subjects and feel that they usuallydo well in them. In both subjects, thepercentage of U.S. students who expresspositive attitudes is similar to the inter-national average except in eighth-grademathematics where the percentage ofU.S. students who like the subject isabove the international average.

52

SUMMARY

Evidence concerning why the interna-tional standing of U.S. students isstronger in science than in mathematicsis inconclusive and incomplete. Furtheranalyses will shed more light on thisquestion. For most of the factors forwhich data are currently available, thereis no difference between mathematicsand science, or evidence from thefourth and eighth grades is mixed. Theonly factor among those reviewed herethat exhibits a difference between math-ematics and science is:

Compared with the internationalaverage, U.S. fourth graders receiveconsiderably more additional instruc-tion per week in science than inmathematics.

However, it is not clear if this difference isalso characteristic of the eighth grade, orif it represents more total class time peryear. Therefore, caution should be usedin assuming that it contributes strongly toU.S. students' stronger performance inscience than in mathematics.

54 53

CONCLUSIONS

BEST COPY AVAILABLE

54

This report has presented highlightsfrom initial analyses of the academicperformance of U.S. fourth graders incomparison with countries that partici-pated in the TIMSS fourth-grade study.The report has also presented the evi-dence available from early analyses con-cerning why U.S. students performabove the international average at thefourth grade, and why their comparativeinternational standing is stronger at thefourth grade than at the eighth grade,and stronger in science than in mathe-matics. Adequate understanding of theanswers to these questions must awaitdeeper analysis.

TIMSS does not suggest any single fac-tor or combination of factors that leadsto high academic performance in everycountry. If anything, TIMSS suggeststhat there may be multiple recipes forexcellence and that different combina-tions of factors may contribute to highachievement in different countries.There are no educational characteristicsthat are present in every high-perform-ing TIMSS country.

Although the evidence presented in thisreport does not point to any factors thatare strongly related to high achieve-

ment, the evidence does suggest thatsome factors commonly thought to berelated are not necessarily so. For exam-ple, more time in class, more home-work, less television, and smaller classsizes have often been thought to bestrongly related to higher achievement.The TIMSS evidence presented hereshows that these factors are not neces-sarily characteristic of most high-per-forming countries, and also that theycannot explain the difference betweenour nation's relative international stand-ing at the fourth and eighth grades, andin science and mathematics.

International comparisons havematured to the point where we nolonger search for single factors thatalways produce world-class perfor-mance. Instead, we need to use thesefindings as an objective assessment ofthe strengths and weaknesses character-istic of each specific national educationsystem. All countries, including the U.S.,have something to learn from othernations, and have something fromwhich other countries can learn. TIMSSallows us to examine our own nationaleducational strengths and weaknesses inthe mirror of other nations.

55

WORKSCITED

1. El ley, W.B. (1992). How in the WorldDo Students Read? The Hague, Nether-lands: International Association for theEvaluation of Educational Achieve-ment.

2. Pelgrum, H. and Plomp, T. (1993).International TEA Computers in EducationStudy. New York: Pergamon Press.

3. Martin, M. and Kelly, D. (1996).Third International Mathematics and Sci-ence Study: Technical Report, Volume 1:Design and Development. Chestnut Hill,MA: Boston College.

4. Martin, M. and Mullis, I. V. S. (1996).Third International Mathematics and Sci-ence Study: Quality Assurance in Data Col-lection. Chestnut Hill, MA: Boston Col-lege.

5. U.S. Department of Education,National Center for Education Statis-tics. (1996). Reading Literacy in the Unit-ed States: Findings from the IEA ReadingLiteracy Study. NCES 96-258. Washing-ton, DC: Government Printing Office.

6. U.S. Department of Education,National Center for Education Statis-tics. (1996). Pursuing Excellence: A Studyof U.S. Eighth-Grade Mathematics and Sci-ence Teaching, Learning, Curriculum, andAchievement in International Context.NCES 97-198, Washington, DC: Govern-ment Printing Office.

7. Lapointe, A., Mead, N., and Askew, J.(1992). Learning Mathematics. Prince-ton, NJ: Educational Testing Service.

8. U.S. Department of Education,National Center for Education Statis-tics. (1992). International Mathematicsand Science Assessments: What Have WeLearned? NCES 92-011. Washington,DC: Government Printing Office.

9. Lapointe, A., Askew, J., and Mead, N.(1992). Learning Science. Princeton, NJ:Educational Testing Service.

10. U.S. Department of Education,National Center for Education Statis-tics. (1994). Understanding the Perfor-mance of U.S. Students on InternationalAssessments. NCES 94-240. Washington,DC: Government Printing Office.

58 )

APPENDIX 1ADDITIONAL TIMSS REPORTS

WHERE CAN I FIND A GOODSUMMARY OF TIMSS FINDINGSTHAT PUTS U.S. EDUCATION INCOMPARATIVE PERSPECTIVE?

Pursuing Excellence: A Study of U.S. Eighth-Grade Mathematics and Science Teaching,Learning, Curriculum, and Achievement inInternational Context, November 1996This report draws from the assessments,surveys, video, and case studies ofTIMSS to summarize the most impor-tant findings concerning U.S. achieve-ment and schooling in the eighth grade.Paperback, 80 pp. $9.50.

To order, contact: U.S. GovernmentBookstore Superintendent of Docu-ments, U.S. Government PrintingOffice, Washington, DC 20402; Tele-phone: (202) 512-1800; Fax: (202) 512-2250; E-mail: [email protected];Internet: http://www.access.gpo.gov/su_docs. GPO #065-000-00959-5. Alsomay be downloaded from http://www.ed.gov/NCES/timss.

Pursuing Excellence: A Study of U.S. Fourth.Grade Mathematics and Science Achieve-ment in International ContextThisreport summarizes the most importantfindings concerning U.S. achievementand schooling in the fourth grade.Paperback.

To order, contact: U.S. GovernmentBookstore Superintendent of Docu-ments, U.S. Government PrintingOffice, Washington, DC 20402; Tele-phone: (202) 512-1800; Fax: (202) 512-2250; E-mail: [email protected];Internet: http://www.access.gpo.gov/su_docs. NCES 97-255. Also may bedownloaded from: http://www.ed.gov/NCES/timss.

TIMSS: A Video Report, February 1997This video summarizes the TIMSS' keyfindings concerning U.S. eighth-gradeeducation and includes the views ofbusiness leaders, policymakers, educa-tors, and researchers on the study'simplications for America's schools. 13minutes. $20.

To order, contact: U.S. GovernmentBookstore Superintendent of Docu-ments, U.S. Government PrintingOffice, Washington, DC 20402; Tele-phone: (202) 512-1800; Fax: (202) 512-2250; E-mail: [email protected];Internet: http://www.access.gpo.gov/su_docs. GPO #065-000-01003-8.

Highlights of Results from TIMSS, Novem-ber 1996Glossy brochure, 8 pp.

To order, contact: TIMSS InternationalStudy Center, Center for the Study ofTesting, Evaluation, and EducationalPolicy (CSTEEP), Campion Hall Room323, School of Education, Boston Col-lege, Chestnut Hill, MA 02167; Tele-phone: (617) 552-4521; Fax: (617) 552-8419; E-mail: [email protected].

Web SitesThere are several web sitesdevoted to TIMSS. For general informa-tion about the study as well as directaccess to many TIMSS publications,please see:

http://www.ed.gov/NCES/timsshttp://wwwcsteep.bc.edu/timsshttp://uttou2.to.utwente.n1/http://ustimss.msu.edu/

57

BEST COPY AVAILARi F

WHERE CAN I FIND A DETAILEDINTERNATIONAL COMPARISONOF EIGHTH-GRADE STUDENTS?

Mathematics Achievement in the MiddleSchool Years: lEA's Third InternationalMathematics and Science Study (TIMSS),November 1996This report focuseson mathematics achievement in 41 coun-tries at the two grades with the largestproportion of 13-year-oldsthe seventhand eighth grades in most countries.The report includes selected back-ground information about students andteachers. Paperback, 176 pp. + 60 pp.Appendix, $30.

To order, contact: TIMSS InternationalStudy Center, Center for the Study ofTesting, Evaluation, and EducationalPolicy (CSTEEP), Campion Hall Room323, School of Education, Boston Col-lege, Chestnut Hill, MA 02167; Tele-phone: (617) 552-4521; Fax: (617) 552-8419; E-mail: [email protected]. Also can bedownloaded from: http://wwwcsteep.bc.edu/TIMSS1/TIMSSPublications.html#International.

Science Achievement in the Middle SchoolYears: lEA's Third International Mathemat-ics and Science Study (TIMSS), November1996This report focuses on scienceachievement in 41 countries at the twogrades with the largest proportion of 13-year- olds the seventh and eighthgrades in most countries. The reportincludes selected background informa-tion about students and teachers. Paper-back, 168 pp. + 62 pp. Appendix, $30.

To order, contact: TIMSS InternationalStudy Center, Center for the Study ofTesting, Evaluation, and EducationalPolicy (CSTEEP), Campion Hall Room323, School of Education, Boston Col-

lege, Chestnut Hill, MA 02167; Tele-phone: (617) 552-4521; Fax: (617) 552-8419; E-mail: [email protected]. Also can bedownloaded from: http://wwwcsteep.bc.edu/TIMSS1/TIMSSPublications.html#International

WHERE CAN I FIND A DETAILEDINTERNATIONAL COMPARISONOF FOURTH-GRADE STUDENTS?

Mathematics Achievement in the Elemen-tary School Years: LEA's Third InternationalMathematics and Science Study (TIMSS),June 1997This report focuses onmathematics achievement in 26 coun-tries at the two grades with the largestproportion of 9-year-oldsthe third andfourth grades in most countries. Thereport includes selected backgroundinformation about students and teach-ers. Paperback. $20 (+ $7 shipping andhandling, if international).

To order, contact: TIMSS InternationalStudy Center, Center for the Study ofTesting, Evaluation, and EducationalPolicy (CSTEEP), Campion Hall Room323, School of Education, Boston Col-lege, Chestnut Hill, MA 02167; Tele-phone: (617) 552-4521; Fax: (617) 552-8419; E-mail: [email protected]. Also can bedownloaded from: http://wwwcsteep.bc.edu/TIMSS1/TIMSSPublications.html#International

Science Achievement in the ElementarySchool Years: lEA's Third InternationalMathematics and Science Study (TIMSS),June 1997This report focuses on sci-ence achievement in 26 countries at thetwo grades with the largest proportionof 9-year-oldsthe third and fourthgrades in most countries. The reportincludes selected background informa-

O53

don about students and teachers. Paper-back. $20 (+ $7 shipping and handling, ifinternational).

To order, contact: TIMSS InternationalStudy Center, Center for the Study ofTesting, Evaluation, and EducationalPolicy (CSTEEP), Campion Hall Room323, School of Education, Boston Col-lege, Chestnut Hill, MA 02167; Tele-phone: (617) 552-4521; Fax: (617) 552-8419; E-mail: [email protected]. Also can bedownloaded from: http://wwwcsteep.bc.edu/TIMSS1/TIMSSPublications.html#International.

HOW CAN I GET A FIRST HANDGLIMPSE OF ACTUAL CLASSROOMLESSONS IN THE UNITED STATES,GERMANY, AND JAPAN?

VHS VIDEO Examples from the Eighth-Grade Mathematics Lessons in the UnitedStates, Japan, and GermanyActualfootage of eighth-grade mathematicsclasses in Germany, the U.S., and Japanlets viewers see firsthand an abbreviatedgeometry and algebra lesson in each ofthree countries: Germany, Japan, andthe United States. 72 minutes.

To order, contact: National Center forEducation Statistics, 555 New Jersey Ave.,Suite #402A, NW, Washington, DC20208; Telephone: (202) 219-1333; Fax:(202) 219-1736; E-mail: [email protected].

CD-ROM Video Examples from the TIMSSVideotape Classroom Study: Eighth-GradeMathematics in Germany, Japan, and theUnited StatesActual footage of eighth-grade mathematics classes lets viewerssee first hand an abbreviated geometry

and algebra lesson in each of threecountries: Germany, Japan, and theUnited States.

Minimum System Requirements:IBM PC or 100 percent compatible, MSWindows ® (Windows 95 ® recom-mended), Pentium 8 90, 16 mb of RAM,256 color SVGA or better, Double-speedor higher CD-ROM drive, Sound Card,orMacintosh ® PowerPC 100 ® or 100 per-cent compatible System 7.5.3, 16 mb ofRAM, 256-color or better, Netscape Nav-igator 3.0 with MPG plug-in Double-speed or higher CD-ROM drive.

To order, contact: National EducationData Resource Center, c/o PinkertonComputer Consultants, Inc., 1900 N.Beauregard St., Suite 200, Alexandria, VA22311-1722; Telephone: (703) 845-3151;Fax: (703) 820-7465; E-mail: [email protected]; Internet address: http://www.ed.gov/pubs/ncesprograms/elementary/others/ndrc.html

WHERE CAN I FIND OUT WHATTIMSS HAS LEARNED ABOUTCURRICULUM?

A Splintered Vision: An Analysis of U.S.Mathematics and Science Curricula,1997This book enunciates the argu-ment that mathematics and science cur-ricula in U.S. schools suffer from a lackof focus. The authors contend that intheir effort to canvas as many topics aspossible, both teachers and textbookpublishers fail to delve into the mostimportant subjects with sufficientdepth. 176 pp. Hardback ISBN: 0 -7923-4440-5, $87; Paperback ISBN: 0 -7923-4441-3, $49.

59

To order, contact: Kluwer AcademicPublishers Group, Order Department,P.O. Box 358, Accord Station, Hingham,MA 02018-0358; Telephone: (617) 871-6600; Fax (617) 871-6528; E-mail: [email protected]; Internet: http://www.wkap.nl or http://ustimss.msu.edu/publicat.htm.

Many Visions, Many Aims: Volume 1, ACross-National Exploration of CurricularIntentions in School Mathematics, 1997An analysis of mathematics curriculumguides and textbooks in 50 countries.This report looks at the sequence andthe topics covered from kindergartenthrough the end of secondary school,analyzed in a comparative framework.286 pp. Hardback ISBN: 0-7923-4436-7,$120; Paperback ISBN: 0-7923-4437-5,$55.

To order, contact: Kluwer AcademicPublishers Group, Order Department,P.O. Box 358, Accord Station, Hingham,MA 02018-0358; Telephone: (617) 871-6600; Fax (617) 871-6528. E-mail: [email protected]; Internet: http://www.wkap.nl or http://ustimss.msu.edu/publicat.htm.

Characterizing Pedagogical Flow: An Inves-tigation of Mathematics and Science Teach-ing in Six Countries, 1996Describes theresults of the Study of Mathematics andScience Opportunity (SMSO) survey,which investigated curriculum contentand instructional methods in France,Japan, Norway, Spain, Switzerland, andthe United States using case studies ineach participating country. 229 pp.Hardback ISBN: 0792342720, $110;Paperback ISBN: 0792342739, $49.

To order, contact: Kluwer AcademicPublishers Group, Order Department,

P.O. Box 358, Accord Station, Hingham,MA 02018-0358; Telephone: (617) 871-6600; Fax (617) 871-6528. E-mail: [email protected]; Internet: http://www.wkap.nl or http://ustimss.msu.edu/publicat.htm.

TIMSS Monograph Series No. 3 Mathemat-ics Textbooks: A Comparative Study ofGrade 8 Texts, 1995Geoffrey Howson,Emeritus Professor of MathematicalCurriculum Studies at the University ofSouthhampton, England, examineseight mathematics textbooks for 13-year -olds for their pedagogical andphilosophical similarities and differ-ences. Texts are from the United States,the Netherlands, the United Kingdom,Norway, Spain, France, Switzerland, andJapan. Paperback, 96 pp. ISBN: 1-895766 -03-6. $16.95.

To order, contact: Pacific EducationalPress, Faculty of Education, Universityof British Columbia, Vancouver, CanadaV6T 1Z4; Telephone: (604) 822-5385;Fax: (604) 822-6603; E-mail: cedwards®interchange.ubc.ca.

WHERE CAN I FIND OUT MOREABOUT THE METHODOLOGY OFTIMSS?

Third International Mathematics and Sci-ence Study: Quality Assurance in Data Col-lection, 1996A report on the qualityassurance program that ensured thecomparability of results across partici-pating countries. The program empha-sized instrument translation and adapta-tion, sampling response rates, testadministration and data collection, thereliability of the coding process, and theintegrity of the database. 93 pp. + 91 pp.Appendix.

62 60

To order, contact: TIMSS InternationalStudy Center, Center for the Study ofTesting, Evaluation, and EducationalPolicy (CSTEEP), Campion Hall Room323, School of Education, Boston Col-lege, Chestnut Hill, MA 02167; Tele-phone: (617) 5524521; Fax (617) 552-8419; E-mail: [email protected]; Also, may bedownloaded from: http://wwwcsteep.bc.edu/TIMSS1/TIMSSPublications.html #International.

Third International Mathematics and Sci-ence Study: Technical Report, Volume 1Design and Development, 1996Thisreport describes the study, design, anddevelopment of TIMSS up to, but notincluding, the operational stage of maindata collection. Paperback, 149 pp. + 40pp. Appendix.

To order, contact: TIMSS InternationalStudy Center, Center for the Study ofTesting, Evaluation, and EducationalPolicy (CSTEEP), Campion Hall Room323, School of Education, Boston Col-lege, Chestnut Hill, MA 02167; Tele-phone: (617) 552-4521; Fax: (617) 552-8419; E-mail: [email protected]; Also, may bedownloaded from: http://wwwcsteep.bc.edu/TIMSS1/TIMSSPublications.html#International.

TIMSS Monograph Series No. 1 CurriculumFrameworks for Mathematics and Science,1993This monograph explains thestudy's foci and its key first step - thedevelopment of the curriculum frame-works that served as the guide fordesigning the study's achievement tests.The frameworks are included in theappendices. Paperback, 102 pp. ISBN:0-88865-090-6. $16.95.

To order, contact: Pacific EducationalPress, Faculty of Education, Universityof British Columbia, Vancouver, CanadaV6T 1Z4. Telephone: (604) 822-5385.Fax: (604) 822-6603. E-mail: [email protected].

TIMSS Monograph Series No. 2 ResearchQuestions and Study Design, 1996Thismonograph presents the study'sresearch objectives along with discus-sions that include: the impact of priorstudies on the design of TIMSS; how theresearch questions were derived fromTIMSS' conceptual framework; and howthe research questions and test itemswere tailored to meet the contexts of theparticipating countries. Paperback, 112pp. ISBN: 1-895766-02-8. $17.95.

To order, contact: Pacific EducationalPress, Faculty of Education, Universityof British Columbia, Vancouver, CanadaV6T 1Z4; Telephone: (604) 822-5385;Fax: (604) 822-6603; E-mail: [email protected].

WHERE CAN I READ THE ACTUALTEST ITEMS GIVEN TO STUDENTS?

TIMSS Mathematics Items Released Set forPopulation 2 (Seventh and eighth grades)All publicly released items used to assessseventh- and eighth-grade students inthe TIMSS study. Paperback, 142 pp.$20 (+ $5 shipping and handling, if inter-national).

TIMSS Science Items Released Set for Popu-lation 2 (Seventh and eighth grades)Allpublicly released items used to assessseventh- and eighth-grade students inthe TIMSS study. Paperback, 127 pp.

63 61

$20 (+ $5 shipping and handling, if inter-national).

TIMSS Mathematics Items Released Set forPopulation 1 (Third and fourth grades)All publicly released items used to assessthird- and fourth-grade students in theTIMSS study. Paperback. $20 (+ $5 ship-ping and handling, if international).

TIMSS Science Items Released Set for Popu-lation 1 (Third and fourth grades)Allpublicly released items used to assessthird- and fourth-grade students in theTIMSS study. Paperback. $20 (+ $5 ship-ping and handling, if international).

To order, contact: TIMSS InternationalStudy Center, Center for the Study ofTesting, Evaluation, and EducationalPolicy (CSTEEP), Campion Hall Room323, School of Education, Boston Col-lege, Chestnut Hill, MA 02167; Tele-phone: (617) 552-4521; Fax: (617) 552-8419; E-mail: [email protected]; Also, can bedownloaded from: http://wwwcsteep.bc.edu/TIMSS1/TIMSSPublications.html#International.

HOW CAN I FIND OUT MOREABOUT EDUCATION IN VARIOUSTIMSS COUNTRIES?

National Contexts for Mathematics and Sci-ence Education: An Encyclopedia of theEducation Systems Participating in TIMSS,1997Each participating country's edu-cation system is discussed in a separatechapter, considering geographic andeconomic influences, school gover-nance, teacher education, curriculum,and other factors. Hardback, 423 pp.$75.

To order, contact: Pacific EducationalPress, Faculty of Education, Universityof British Columbia, Vancouver, CanadaV6T 1Z4; Telephone: (604) 822-5385;Fax: (604) 822-6603; E-mail: [email protected]

O62

APPENDIX 2ADVISORS TO THE U.S. TIMSS STUDY

William SchmidtChairU.S. TIMSS National ResearchCoordinatorMichigan State University

Gordon AmbachCouncil of Chief State School Officers

Deborah BallUniversity of Michigan

Audrey ChampagneSUNY at Albany

David CohenUniversity of Michigan

John DosseyIllinois State University

Emerson ElliottNational Council for Accreditation ofTeacher Education

Sheldon GlashowHarvard University

Larry HedgesUniversity of Chicago

Henry HeikkinenUniversity of Northern Colorado

Jeremy KilpatrickUniversity of Georgia

Mary LindquistColumbus College

Marcia LinnUniversity of California at Berkeley

Robert LinnUniversity of Colorado

Paul SallyUniversity of Chicago

Richard ShavelsonStanford University

Bruce SpencerNorthwestern University

Elizabeth StageUniversity of California

James TaylorGlobal M

Kenneth TraversUniversity of Illinois

Paul WilliamsUniversity of Wisconsin

O

APPENDIX 3NATIONAL AVERAGE SCORES AND STANDARD ERRORS

The 95 percent "plus or minus" confidence interval around each' nation's scoreis two times the standard error.

NATION

MATHEMATICS SCIENCE

AVERAGE STANDARDERROR

AVERAGE STANDARDERROR

(AUSTRALIA) 546 3.1 562 2.9(AUSTRIA) 559 3.1 565 3.3CANADA 532 3.3 549 3.0CYPRUS 502 3.1 475 3.3CZECH REPUBLIC 567 3.3 557 3.1ENGLAND 513 3.2 551 3.3GREECE 492 4.4 497 4,1HONG KONG 587 4.3 533 3.7(HUNGARY) 548 3.7 532 3.4ICELAND 474 2.7 505 3.3IRAN, ISLAMIC REP. 429 4.0 416 3.9IRELAND 550 3.4 539 3.3(ISRAEL) 531 3.5 505 3.6JAPAN 597 2.1 574 1.8KOREA 611 2.1 597 1.9(KUWAIT) 400 2.8 401 3.1(LATVIA (LSS)) 525 4.8 512 4.9(NETHERLANDS) 577 3.4 557 3.1NEW ZEALAND 499 4.3 531 4.9NORWAY 502 3.0 530 3.6PORTUGAL 475 3.5 480 4.0SCOTLAND 520 3.9 536 4,2SINGAPORE 625 5.3 547 5.0(SLOVENIA) 552 3.2 546 3.3(THAILAND) 490 4.7 473 4.9UNITED STATES 545 3.0 565 3.1

MATHEMATICS INTERNATIONAL AVERAGE = 529

SCIENCE INTERNATIONAL AVERAGE = 524

Note: Nations not meeting international guidelines are shown in parentheses.

Source: Mullis et al. (1997) Mathematics Achievement in the Primary School Years. Table 1.1. Boston College: Chestnut

Hill, MA and Martin et al. (1997) Science Achievement in the Primary School Years. Table 1.1. Boston College: ChestnutHill, MA.

64

APPENDIX 4

SUMMARY OF NATIONAL DEVIATIONS FROMINTERNATIONAL STUDY GUIDELINES

Eleven of the 26 TIMSS countries expe-rienced a more or less serious deviationfrom international guidelines for execu-tion of the study. In 9 countries, theTIMSS International Study Center con-sidered the deviations to be sufficientlyserious to raise questions about the con-fidence to be placed in their scores.These 9 nations with major difficultiesare noted with an asterisk.

*AustraliaParticipation rate did notmeet the international criterion of 75percent of schools and students com-bined. Participation rate was 69 percentafter replacements for refusals were sub-stituted.

*AustriaParticipation rate did notmeet either the international criterionof at least 50 percent participation byschools before replacement or 75 per-cent of schools and students combined.The initial participation rate was 49 per-cent for schools before replacement.Participation rate was 69 percent afterreplacements for refusals were substitut-ed.

EnglandMore than the internationalcriterion of ten percent of schools andstudents were excused from the test forvarious reasons, with resulting coverageof 88 percent of the desired population.Participation rate of 83 percent ofschools and students combined wasachieved only after replacements forrefusals were substituted.

*HungaryInternational guidelines forsampling procedures at the classroomlevel were not followed.

*IsraelInternational guidelines forsampling procedures at the classroomlevel were not followed. The test wasadministered only in the Hebrew-speak-ing public school system. Participationrate did not meet either the internation-al criterion of at least 50 percent partici-pation by schools before replacement or75 percent of schools and students com-bined. Israel tested only the fourthgrade, in contrast to other nations thattested the two adjacent grades contain-ing the most 9-year olds. Participationrate was 38 percent both before andafter replacements for refusals were sub-stituted.

*KuwaitInternational guidelines forsampling procedures at the classroomlevel were not followed. In contrast toother nations that tested two adjacentgrades, Kuwait tested the fifth grade,which contained relatively few 9-yearolds.

*Latvia (LSS)Test administered onlyin Latvian-speaking schools, with result-ing coverage of 60 percent of thedesired population. Because coveragefell below the international 65 percentpopulation-coverage criterion, Latvia isdesignated (LSS) for Latvian-speakingschools.

*Netherlands--Participation rate didnot meet either the international criteri-on of at least 50 percent participation byschools before replacement or 75 per-cent of schools and students combined.The initial participation rate beforereplacement was 29 percent. Participa-tion rate was 59 percent after replace-ments for refusals were substituted.

65

ScotlandParticipation rate of 76 per-cent of schools and students combinedwas achieved only after replacements forrefusals were substituted.

*SloveniaStudents tested were olderthan those in other countries becauseSlovenia did not test the two grades withthe most 9-year olds.

*ThailandInternational guidelines forsampling procedures at the classroomlevel were not followed. The sampleincluded a high percentage of older stu-dents.

*U.S. Government Printing Office: 1997 - 428-502/70501

66

HavoomaL Cgavn mca EpucaToom gworsocg©G3 ool EVCAVOCMAL RIEglEACICH GJMn DD ORPKNEMMV

O[;pAAVNIEN4 CDV EIDUCAVOCDN

1

MOM DT-41g.

BEST COPY AVAILABLE]

ter

(9/92)

U.S. DEPARTMENT OF EDUCATIONOffice of Educational Research and improvement (OERI)

Educational Resources Information Center (ERIC)

NOTICE

REPRODUCTION BASIS

IC

This document is covered by a signed "Reproduction Release(Blanket)" form (on file within the ERIC system), encompassing allor classes of documents from its source organization and, therefore,does not require a "Specific Document" Release form.

This document is Federally-funded, or carries its own permission toreproduce, or is otherwise in the public domain and, therefore, maybe reproduced by ERIC without a signed Reproduction Releaseform (either "Specific Document" or "Blanket").

DOCUMENT RESUME AUTHOR Frase, Mary; And Others · DOCUMENT RESUME. ED 410 098 SE 060 370. AUTHOR...

Documents

Transcript of DOCUMENT RESUME AUTHOR Frase, Mary; And Others · DOCUMENT RESUME. ED 410 098 SE 060 370. AUTHOR...