Designing an Assessment System for the Race to the Top
description
Transcript of Designing an Assessment System for the Race to the Top
Designing an Assessment Designing an Assessment System for the Race to System for the Race to
the Topthe Top
Edward HaertelSchool of EducationStanford University
RTTT Public Meeting, General AssessmentDenver, ColoradoDecember 1, 2009
1
What’s Different This What’s Different This Time?Time?In the absence of well-defined and demanding standards, education in the United States has gravitated toward de facto national minimum expectations, with curricula focusing on low-level reading and arithmetic skills and on small amounts of factual material in other content areas. Most current assessment methods reinforce the emphasis on these low-level skills and processing bits of information rather than on problem solving and critical thinking. The adoption of world-class standards would force the Nation to confront today's educational performance expectations that are simply too low.
NCEST Report, 1992
2
New IdeasNew IdeasCommon Standards
◦fewer, clearer, higher?Aligned assessments of individual
growthExplicit attention to range of
assessment types / formats, content sampling, item release plans, etc.
21st century technology◦computer-based testing, longitudinal growth
models, Evidence-Centered Design, …
3
Old IdeasOld IdeasIf assessments are aligned to
standards, then growth in test scores will track intended schooling outcomes
Essentially all valued schooling outcomes can be captured by test performances of◦students working alone◦without “smart workplace” resources◦answering clear questions with clear
answers
4
Old IdeasOld IdeasOne assessment system will “do it all”
◦track progress of students / schools / …
◦inform decisions about graduation / college readiness /
career readiness / college course placement◦support improvement
in teaching / learning / instructional programs◦determine effectiveness
of schools / principals / teachers
5
6
Theories of Action are, if anything, less clear than in 1992
A Theory of Action is a chain of logical propositions that “connects the dots,” explaining step by step, in commonsense terms, how the assessment system, accountability system, and related reforms will lead to improved learning outcomes
Rethinking Old Ideas To Rethinking Old Ideas To Make New Ideas WorkMake New Ideas WorkAligning tests and curricula to
common standards is not sufficient to create a coherent system
7
Content Standards
Curriculum &
Instruction
Assessments
?
Rethinking Old Ideas To Rethinking Old Ideas To Make New Ideas WorkMake New Ideas WorkAligning tests and curricula to
common standards is not sufficient to create a coherent system
8
Content Standards
Curriculum &
Instruction
Assessments
?
11th Grade History /11th Grade History /Social StudiesSocial Studies
US11.11.2.Discuss the significant domestic policy speeches of Truman, Eisenhower, Kennedy, Johnson, Nixon, Carter, Reagan, Bush, and Clinton (e.g., education, civil rights, economic policy, environmental policy).
9
Item Testing US11.11.2Item Testing US11.11.2
10
High School BiologyHigh School BiologyBI6.fStudents know at each link in a food web some energy is stored in newly made structures but much energy is dissipated into the environment as heat. This dissipation may be represented in an energy pyramid.
11
Item Testing BI6.fItem Testing BI6.f
12
Rethinking AlignmentRethinking AlignmentIt is not enough for each item to be
somehow connected to one or more standards
Assessments, standards, and curriculum and instruction must all be considered together ◦Curriculum-embedded formative
assessments◦Portfolios incorporating well-specified,
externally scorable components13
Rethinking Old Ideas To Rethinking Old Ideas To Make New Ideas WorkMake New Ideas WorkTesting need not imply that
students work alone, without references or resources, on small tasks with one right answer◦Well-structured problems
(Frederiksen, 1984)◦Content-lean, process-constrained tasks
(Baxter & Glaser, 1998)◦Sequestered Problem Solving
(Bransford & Schwartz, 1999)
14
Rethinking What It Means Rethinking What It Means to Take A Testto Take A TestA comprehensive assessment system
must also include◦ill-structured problems with multiple solutions
that are better or worse, not right or wrong◦content-rich, process-open tasks requiring
flexible, adaptive application of coherent knowledge structures to solve novel problems
◦tests of preparation for future learning – measuring skill in approaching new problems, “knowing with” as well as “knowing how” or “knowing that”
15
Rethinking Old Ideas To Rethinking Old Ideas To Make New Ideas WorkMake New Ideas WorkA single form of testing cannot
satisfy the full set of RTTT design requirements◦Sole reliance on external, on-demand
tests will fall short, even if constructed-response items are included
◦Sole reliance on external, on-demand tests will fall short, even if interim / benchmark assessments are included
16
Rethinking Multiple Rethinking Multiple IndicatorsIndicatorsA comprehensive assessment system
must use multiple forms of evidence◦curriculum-embedded assessments◦extended projects◦a range of student performances including
finding, organizing, and evaluating information analyzing and synthesizing data planning and carrying out investigations solving unfamiliar, open-ended problems written reports and oral presentations
17
Rethinking Multiple Rethinking Multiple IndicatorsIndicatorsDifferent forms of assessment,
organized in different ways, may be required for◦improving instructional decision
making in the classroom◦tracking student growth and
predicting college or career readiness
◦accountability
18
Rethinking Multiple Rethinking Multiple IndicatorsIndicatorsStandards, assessments, curriculum,
and instruction must be aligned and coherent◦horizontal links across content areas◦vertical links across grade levels
Assessment at the classroom or school level may employ matrix sampling◦different students answer different
questions to provide better group-level information
◦some assessment system components may not yield individual scores
19
SummarySummaryWe must RETHINK Alignment
◦not just test-to-standards, but instead a coherent, simultaneous consideration of standards, assessments, curriculum and instruction, and I might add, teacher preparation
20
SummarySummaryWe must RETHINK what it means to
take a test◦not just sequestered problem solving,
with tidy questions and routine problems, but also complex tasks with multiple solutions judged by multiple
criteria sometimes worked out over an extended time sometimes involving collaboration sometimes requiring students to figure out
what information they need and where to find it
21
SummarySummaryWe must RETHINK multiple
indicators◦not just a broader mix of item formats◦not just interim plus end-of-year tests◦some portion of classroom assessment
must be sufficiently documented and structured so as to be incorporated into comprehensive assessment systems to meet multiple needs
22
23
Thank you