Smarter Balanced: On Track and Moving Forward

Presentation delivered as part of the series:Spotlight on Assessment, Evaluation, and Research

Ready, Set, Go: Smarter Balanced Assessment System in ConnecticutMarch 31, 2014

Joe Willhoft, Executive Director

Agenda

• How is Smarter Balanced Different?

• Progress in Developing the Assessment SystemField Test

• Addressing Key Issues:TechnologyStudent PrivacySetting Performance Standards (“cut scores”)Sustainability

About Smarter Balanced

Smarter Balanced Assessment Consortium

• 24 states & territories (22 governing, 1 advisory, 1 affiliate)

• K-12 & Higher Education Leads in each state

A Balanced Assessment System

Common Core State Standards specify

K-12 expectatio

ns for college and

career readiness

All students

leave high

school college

and career ready

Teachers and schools have information and tools

they need to improve

teaching and learning Interim

assessments Flexible, open,

used for actionable feedback

Summative assessments

Benchmarked to college and

career readiness

Teacher resources for

formative assessment

practicesto improve instruction

Describe Explain

Interpret

Level One(Recall)

Level Three(Strategic Thinking)

(Extended Thinking)

Level Two(Skill/

Concept)

Design

Synthesize

Connect

Apply Concepts

Critique

Analyze

Create

Prove

Arrange

Calculate

Draw

Repeat Tabulate

Recognize

Memorize

Identify

Who, What, When, Where, Why

List

Name

Use

Illustrate

Measure

Define

RecallMatch

Graph

Classify

Cause/Effect

Estimate

Compare

Relate

Infer

Categorize

Organize

Interpret

Predict

Modify

Summarize

ShowConstruct

Develop a Logical ArgumentAssessRevise

Apprise

Hypothesize

InvestigateCritique

Compare

Formulate Draw ConclusionsExplain

Differentiate

Use Concepts to SolveNon-Routine Problems

Level Four

Source: Webb, Norman L. and others, “Web Alignment Tool” 24 July 2005. Wisconsin Center of Educational Research, University of Wisconsin-Madison, 2 Feb 2006 6

Assessing the Common Core

Smarter Balanced assessments move beyond basic skills and recall to assess higher-order skills such as critical thinking and problem solving

Educator Involvement

Item Development & Review• 500+ educators from member states

Digital Library of Formative Assessment Tools & Practices• State Leadership Teams: 7 to 12 member steering committees• State Network of Educators: 70 to 150 members per state identifying and

reviewing materials• 1,500 educators involved across consortium

Teacher Ambassadors• 125 teachers drawn from leadership of NEA and AFT• Trained on Smarter Balanced so that they can keep their fellow teachers

informed• Contributing to Digital Library

Accessibility & Accommodations

8

Note: For detailed information, see the Smarter Balanced Usability, Accessibility, and Accommodations Guidelines

Test Development Update

Assessment Development

Cognitive Labs and

Small-scale Trials (2012-

13)

Pilot Test (Spring 2013)

Field Test Item

Development Complete

(December 2013)

Field Test (March 18 to June 6, 2014

Preliminary Standard-

setting (September

2014)

Operational Assessment

Launch (2014-15)

Pilot Test: Approximately 650,000 students at more than 4,000 schoolsField Test: Expect more than 3 million students at more than 20,000 schools

Field Testing: A Practice Run of New

Assessments• “Tests the tests” to ensure fair and

accurate assessments for all students in spring 2015

• Gives teachers and schools chance to practice test administration procedures and gauge tech readiness

• Gives students opportunity to experience the new assessments

• No individual or school scores/results will be produced

• Testing Window: March 25 to June 6

• All 22 Governing States and US Virgin Islands

• At least 3 million students (research sample = 1.9M)

– Most states: 10% of students in English, 10% in math

– Several states (CA, CT, ID, MT, SD) testing most or all students

– More than 20,000 schools in more than 4,000 districts

• 21,000 questions/performance tasks in English and math, grades 3-8 and 11

Field Test Scope

Resources to Help Schools Prepare• Practice Test (will be updated May 2014)

Both subject areas, grades 3 through 8 and 11 Approx. 23 items & 1 performance task in each subject Uses same software as operational test Most accessibility and accommodation features

• Training Test Quick Introduction to Smarter Balanced interface, item types, and

resources 14 items combined in English and math (all item types

represented) Grade bands (3 – 5, 6 – 8, and high school) All accessibility and accommodation features

• Communications Toolkit of Easily Customizable Resources: Parent Notification Letter (in English and Spanish) and Robo-call

Script Parent Q&A PowerPoint Presentation (suitable for use at PTA meetings) Newsletter Article Web Buttons to drive traffic to Practice and Training Test

• Visit http://sbac.portal.airast.org/

Addressing Key Issues

Technology Requirements Responsive to School Needs

• Online “Readiness Tool” allows schools and districts to evaluate technology readiness

• Standards have been established for new and existing hardware

• Schools do NOT need one-to-one computers Technology standards maximize access to online testing

(support for older machines and operating systems, small file size to reduce bandwidth requirements)

School with 600 students could test online with a single 30-computer lab

Smarter Balanced Tech Readiness Calculator allows schools to estimate number of testing days and associated bandwidth

• Pencil-and-paper option available for three-year transition period

Safeguarding Student Privacy

• States endorsed principle that each state will retain control of its student data.– Smarter Balanced will share no student-level

information with the federal government.– Smarter Balanced will not provide data to third

parties (e.g. researchers) without state approval.– PARCC and Smarter chiefs recently sent letter to

Secretary Duncan affirming that consortia will not change state reporting practices.

• Each state has entered data privacy and security agreements to safeguard student information.

Setting Performance Standards

• Content Standards (Common Core) define what students must know and be able to do. Performance Standards define how well or how thoroughly students have mastered the content standards.

• Currently, each state sets its own performance standards.

• Under Smarter Balanced, seeking consensus among 22 states!

• Four Performance Levels: Level 3 = on track to college/career readiness at a particular grade.

• Standards set through a deliberative process involving primarily K-12 teachers and higher education faculty. Opportunities for parent involvement as well.

• Standard setters will be able to reference outside benchmarks (e.g. NAEP, SAT, ACT)

Standard Setting Calendar

• Chiefs vote on standard setting designApril 29-30

• Distributed Standard SettingMid September

• In-Person Standard SettingLate September

• Chiefs endorse recommended Cut ScoresEarly October

• States ratify (e.g. state board approval)• Seek participation commitments from colleges and

universities for 2014-15 11th gradersFall 2014

A Smarter Future

Sustaining Smarter Balanced

• Federal grant funding consortium expires 9/30/14

• Working with UCLA Graduate School of Education and Information Studies to serve as host and partner for a sustainable Smarter Balanced.

• Key principles: Retain state led governance of the

Consortium (only minor changes to governance structure envisioned).

Shared state ownership of the item pool, digital library, and other IP.

Smarter Balanced will perform services necessary to maintain quality and comparability of the assessment system; states and their vendors will manage test administration.

Future Work

• Continuing the work of the Proficiency Based Learning Task Force

• Investing in supports for mathematic reasoning

• Validating AI scoring and promoting the value of handscoring

• Commission additional formative modules

• Commission additional items and tasks

• Move summative items and tasks from summative to interim

• Validate achievement standards and the assessment itself

Questions/Discussion