Race to the Top Assessment Program: Public Hearings
description
Transcript of Race to the Top Assessment Program: Public Hearings
Race to the Top Assessment Race to the Top Assessment Program: Public HearingsProgram: Public Hearings
Some Thoughts on Grades 3-8 and High Some Thoughts on Grades 3-8 and High
School AssessmentsSchool Assessments
December 1, 2009December 1, 2009Denver, ColoradoDenver, Colorado
Presenter:Presenter:Lloyd Bond, The Carnegie FoundationLloyd Bond, The Carnegie Foundation
Issues about which there is Issues about which there is consensus consensus
Intended usesIntended uses
Content, Formats, & Test AdministrationContent, Formats, & Test Administration
ValidationValidation
ReportingReporting
Consensus IssuesConsensus IssuesIntended usesIntended uses
Monitoring student progressMonitoring student progress Teacher, principal, and school accountabilityTeacher, principal, and school accountability College and work readiness (?)College and work readiness (?)
Content, Formats, & Test AdministrationContent, Formats, & Test Administration Multiple measures of student successMultiple measures of student success Paper & pencil, computer assisted, MC, short answer, extended Paper & pencil, computer assisted, MC, short answer, extended
response, and portfolioresponse, and portfolio Both mid-course and end-of-course assessmentsBoth mid-course and end-of-course assessments Multiple opportunities and multiple ways to demonstrate Multiple opportunities and multiple ways to demonstrate
masterymastery (See “My Child Doesn’t Test Well” (See “My Child Doesn’t Test Well” Carnegie Perspectives Web Site)Carnegie Perspectives Web Site)
Instructionally useful information to teachers and principalsInstructionally useful information to teachers and principals
Consensus Issues (cont’d)Consensus Issues (cont’d)
ValidationValidation Appropriate evidence claims of content fidelity and Appropriate evidence claims of content fidelity and
college/work readiness (?)college/work readiness (?)
ReportingReporting Reporting of assessment information (including Reporting of assessment information (including
assessment architecture, standards, and results) to assessment architecture, standards, and results) to the various non-technical stakeholders should be as the various non-technical stakeholders should be as straightforward as possible, understandable to literate straightforward as possible, understandable to literate adultsadults
Thoughts on Unresolved Thoughts on Unresolved and/or Contentious Issuesand/or Contentious Issues
Broad & Shallow vs. Narrow & DeepBroad & Shallow vs. Narrow & DeepConsequential Validity - “Teaching to the Consequential Validity - “Teaching to the
Test”Test”The concepts of college and work The concepts of college and work
“readiness”“readiness”Familiar vs.aspirational assessmentFamiliar vs.aspirational assessment
Broad & Shallow vs... Narrow & Broad & Shallow vs... Narrow & DeepDeep
The delicate balance between (1) learning The delicate balance between (1) learning a lot of things marginally and (2) learning a a lot of things marginally and (2) learning a modest number of important things wellmodest number of important things well
One of the distinguishing curricular One of the distinguishing curricular features of countries that consistently do features of countries that consistently do well on TIMMS and other assessments is well on TIMMS and other assessments is that they error on the side of “(relatively) that they error on the side of “(relatively) narrow and deep”narrow and deep”
Thoughts on the “what” of Thoughts on the “what” of assessmentassessment
The Grade 3-8 through High School assessments should move increasingly and continuously from:
1) assessment of declarative knowledge to assessment of procedural and conceptual knowledge
2) assessment of procedures and skills to assessment of problem solving and critical thinking
Thoughts on ValidityThoughts on ValidityGathering evidence of content & predictive Gathering evidence of content & predictive
validity should be reasonably validity should be reasonably straightforwardstraightforward
Gathering evidence regarding Gathering evidence regarding consequential validity can be problematicconsequential validity can be problematic
Consequential Validity of the Consequential Validity of the AssessmentsAssessments
Testing what is taught vs. teaching what is Testing what is taught vs. teaching what is tested tested
Using the RTT assessments for Using the RTT assessments for accountability purposes guarantees that accountability purposes guarantees that attempts to “teach to the test” will occurattempts to “teach to the test” will occur
We must therefore make the assessments We must therefore make the assessments worth teaching toworth teaching to
(See “Teaching to the Test” (See “Teaching to the Test” Carnegie Carnegie Perspectives Perspectives Web Site)Web Site)
College & Work ReadinessCollege & Work Readiness College and work “readiness” are as yet ill-College and work “readiness” are as yet ill-
defined and illusive conceptsdefined and illusive concepts ““Ready” for majoring in physical education at Ready” for majoring in physical education at
Podunk U or ready for majoring in electrical Podunk U or ready for majoring in electrical engineering at Cat Techengineering at Cat Tech
Ready for a cashier at McDonald’s or ready for Ready for a cashier at McDonald’s or ready for office manager trainee at Merrill Lynchoffice manager trainee at Merrill Lynch
It is doubtful whether a single test can serve this It is doubtful whether a single test can serve this multiple purposesmultiple purposes
Measuring important skills and Measuring important skills and abilitiesabilities
““We do not yet know how to measure all of We do not yet know how to measure all of the important things. Ergo, what we CAN the important things. Ergo, what we CAN measure becomes all important”measure becomes all important”
Aspirational AssessmentsAspirational Assessments In addition to the usual and familiar, the In addition to the usual and familiar, the
RTT assessment program should RTT assessment program should encourage and support consortia that also encourage and support consortia that also move the profession forwardmove the profession forward
Toward the Assessment of Toward the Assessment of “Understanding”“Understanding”
Students can “know” things and know how to do Students can “know” things and know how to do things without genuine understandingthings without genuine understanding
““Study to remember, and you will forget; study to Study to remember, and you will forget; study to understand, and you will remember.” understand, and you will remember.” ~Anonymous~Anonymous
Teach for regurgitation, and students will forget; Teach for regurgitation, and students will forget; teach for understanding, and they will teach for understanding, and they will remember.remember.
““The Assessment of Understanding” The Assessment of Understanding” Carnegie Carnegie Perspectives” Perspectives” Web SiteWeb Site
Final ThoughtsFinal Thoughts
Beware of the assessment tail wagging the Beware of the assessment tail wagging the content dog content dog
Don’t reinvent the wheel. Encourage consortia Don’t reinvent the wheel. Encourage consortia to take advantage of what has already been to take advantage of what has already been done and what we already know (e.g., College done and what we already know (e.g., College Board Accuplacer) Board Accuplacer)
Encourage involvement of higher education Encourage involvement of higher education faculty and industry personnel in the high school faculty and industry personnel in the high school assessmentassessment