Benchmarking to International Standards TIMSS & PIRLS: A Bridge to School Improvement

20
Benchmarking to International Standards TIMSS & PIRLS: A Bridge to School Improvement Ina V.S. Mullis and Michael O. Martin, Directors TIMSS & PIRLS International Study Center Boston College Institute of Education Sciences May 30, 2008 This paper is intended to promote the exchange of ideas among researchers and policy makers. The views expressed in it are part of ongoing research and analysis and do not necessarily reflect the position of the National Center for Education Statistics, the Institute of Education Sciences, or the U.S. Department of Education.

description

Benchmarking to International Standards TIMSS & PIRLS: A Bridge to School Improvement. Ina V.S. Mullis and Michael O. Martin, Directors TIMSS & PIRLS International Study Center Boston College Institute of Education Sciences May 30, 2008. - PowerPoint PPT Presentation

Transcript of Benchmarking to International Standards TIMSS & PIRLS: A Bridge to School Improvement

Page 1: Benchmarking  to International Standards TIMSS & PIRLS:  A Bridge to School  Improvement

Benchmarking to International Standards

TIMSS & PIRLS: A Bridge to School Improvement

Benchmarking to International Standards

TIMSS & PIRLS: A Bridge to School Improvement

Ina V.S. Mullis and Michael O. Martin, DirectorsTIMSS & PIRLS International Study Center

Boston College

Institute of Education SciencesMay 30, 2008

Ina V.S. Mullis and Michael O. Martin, DirectorsTIMSS & PIRLS International Study Center

Boston College

Institute of Education SciencesMay 30, 2008

This paper is intended to promote the exchange of ideas among researchers and policy makers.  The views expressed in it are part of ongoing research and analysis and do not necessarily reflect the position of the National Center for Education Statistics, the Institute of Education Sciences, or the U.S. Department of Education.

Page 2: Benchmarking  to International Standards TIMSS & PIRLS:  A Bridge to School  Improvement

2

What Are TIMSS and PIRLS?What Are TIMSS and PIRLS?

International Assessments of Educational Achievement

TIMSS

– Mathematics and science

– Fourth and eighth grades

PIRLS

– Reading

– Fourth grade

Page 3: Benchmarking  to International Standards TIMSS & PIRLS:  A Bridge to School  Improvement

3

Why TIMSS and PIRLS?Why TIMSS and PIRLS?

Provide comparative information about educational achievement across countries in relation to the approaches used to improve teaching and learning in mathematics, science, and reading

– Monitor curricular implementation and effectiveness

– Identify most promising instructional practices

– Measure progress in achievement over time and across grades

Page 4: Benchmarking  to International Standards TIMSS & PIRLS:  A Bridge to School  Improvement

4

Trends Over TimeTrends Over Time

Grade 4

– TIMSS in 1995, 2003, 2007, 2011…

– PIRLS in 2001, 2006, 2011…

Grade 8

– TIMSS in 1995, 1999, 2003, 2007, 2011…

Grade 12

– TIMSS Advanced in 1995, 2008…

Page 5: Benchmarking  to International Standards TIMSS & PIRLS:  A Bridge to School  Improvement

5

TIMSS Benchmarking U.S. States and DistrictsTIMSS Benchmarking U.S. States and Districts

1995 First in the World Consortium, Colorado, Illinois, Minnesota

1997 Missouri and Oregon

1999 13 states and 14 districts and consortia

2003 Indiana

2007 Massachusetts and Minnesota

Page 6: Benchmarking  to International Standards TIMSS & PIRLS:  A Bridge to School  Improvement

6IEA’s Curriculum Model: An Educational Perspective on Educational Issues

IEA’s Curriculum Model: An Educational Perspective on Educational Issues

Page 7: Benchmarking  to International Standards TIMSS & PIRLS:  A Bridge to School  Improvement

7

What Data Are Collected?What Data Are Collected?

Intended Curriculum

– Participating countries, curriculum experts

– Routinely published in Encyclopedia

Implemented Curriculum

– Teachers and principals of participating students

– The students themselves

Attained Curriculum

– The mathematics, science, and reading tests

– Routinely published in International Reports

Page 8: Benchmarking  to International Standards TIMSS & PIRLS:  A Bridge to School  Improvement

8• 13 above

average, U.S.

3 about average

9 below average

• Wide range of achievement – across and within countries

Page 9: Benchmarking  to International Standards TIMSS & PIRLS:  A Bridge to School  Improvement

9

• 13 above average7 about average, U.S.25 below average

• Wide range of achievement – across and within countries

• Range between 5th and 95th percentiles approximately 270-300

• Average in Singapore same as 95th percentile in lowest countries

Page 10: Benchmarking  to International Standards TIMSS & PIRLS:  A Bridge to School  Improvement

10

Widespread ParticipationWidespread Participation

TIMSS 2007 Grade 8

– 50 Countries

– 242,000 Students

TIMSS 2007 Grade 4

– 37 Countries

– 183,000 Students

PIRLS 2006 Grade 4

– 40 Countries

– 215,000 Students

Page 11: Benchmarking  to International Standards TIMSS & PIRLS:  A Bridge to School  Improvement

11

Comprehensive Content CoverageComprehensive Content Coverage

Mathematics Framework

Grade 4 – 179 items (192 score points)

– Number (50%)

– Geometric shapes and measures (35%)

– Data display (15%)

Grade 8 – 215 items (238 score points)

– Number (30%)

– Algebra (30%)

– Geometry (20%)

– Data and chance (20%)

Page 12: Benchmarking  to International Standards TIMSS & PIRLS:  A Bridge to School  Improvement

12

Comprehensive Content CoverageComprehensive Content Coverage

Science Framework

Grade 4 – 174 items (194 score points)

– Life science (45%)

– Physical science (35%)

– Earth science (20%)

Grade 8 – 212 items (240 score points)

– Biology (35%)

– Chemistry (20%)

– Physics (25%)

– Earth science (20%)

Page 13: Benchmarking  to International Standards TIMSS & PIRLS:  A Bridge to School  Improvement

13

Mathematics and Science Cognitive Domains – Grades 4 & 8Mathematics and Science Cognitive Domains – Grades 4 & 8

Reasoning (20-30%)

– Analyze, integrate/synthesize, generalize, hypothesize/predict, draw conclusions, justify, solve non-routine problems and conduct investigations

Applying (35-40%)

– Compare/contrast, model, represent, use relationships and concepts to solve problems

Knowing (30-40%)

– Recall, recognize, classify, define, measure

Page 14: Benchmarking  to International Standards TIMSS & PIRLS:  A Bridge to School  Improvement

14

Comprehensive CoverageComprehensive Coverage

PIRLS Framework – Grade 4

Purposes for Reading

– Reading for Literary Experience (50%)

– Reading to Acquire and Use Information (50%)

Processes of Comprehension

– Retrieval and Straightforward Inferencing (50%)

– Interpreting, Integrating & Evaluating (50%)

Assessment:

10 passages – 126 items (167 score points)

Page 15: Benchmarking  to International Standards TIMSS & PIRLS:  A Bridge to School  Improvement

15

Comprehensive Background Information About Contexts for Teaching and Learning

Comprehensive Background Information About Contexts for Teaching and Learning

Questionnaires:

• Country – intended curriculum

• Student – home and classroom experience

• Teacher – implemented curriculum, education and preparation, instructional practices

• School – climate, resources, composition

• Parent (PIRLS) – early literacy activities, home resources, parental attitudes toward reading

Page 16: Benchmarking  to International Standards TIMSS & PIRLS:  A Bridge to School  Improvement

16

Process for Updating FrameworksProcess for Updating Frameworks

Goal: Include topics regarded as important in a significant number of countries

• Recommendations by subject matter experts

– e.g., committees, NSF initiative for TIMSS 2003

• Questionnaires completed by participating countries

– Topics included in their curricula

– Desirability of assessing particular topics

• Iterative process of reviewing and revising successive drafts

Page 17: Benchmarking  to International Standards TIMSS & PIRLS:  A Bridge to School  Improvement

17

Aspects of Comparative ValidityAspects of Comparative Validity

• Are target populations comparable?

• Was sampling conducted properly?

• Are translations comparable?

• Were the tests administered appropriately?

• Was scoring done correctly?

• Are the data comparable?

Page 18: Benchmarking  to International Standards TIMSS & PIRLS:  A Bridge to School  Improvement

18

In Conclusion…In Conclusion…

• Much has been learned about effective education since the pioneering days

• Different countries use different approaches but an effective educational system always requires enormous effort

– High percentages of students completing high school, and taking advanced courses

– A rigorous and progressive curriculum

– Well-prepared teachers

– Resources for facilities and materials

– Students ready to learn and encouraged by society

Page 19: Benchmarking  to International Standards TIMSS & PIRLS:  A Bridge to School  Improvement

19

Page 20: Benchmarking  to International Standards TIMSS & PIRLS:  A Bridge to School  Improvement

Benchmarking to International Standards

TIMSS & PIRLS: A Bridge to School Improvement

Benchmarking to International Standards

TIMSS & PIRLS: A Bridge to School Improvement

Ina V.S. Mullis and Michael O. Martin, DirectorsTIMSS & PIRLS International Study Center

Boston College

Institute of Education SciencesMay 30, 2008

Ina V.S. Mullis and Michael O. Martin, DirectorsTIMSS & PIRLS International Study Center

Boston College

Institute of Education SciencesMay 30, 2008