Unit 2: California State Assessment System

Assessment Literacy Module

California Department of EducationTom Torlakson, State Superintendent of Public Instruction

2

Welcome to Unit 2

The purpose of this unit is to increase your knowledge of the components of California’s new state assessment system.

3

Learning Objectives for Unit 2

By the end of this unit, participants will be able to:

Describe plans for aligning components of the CA CCSS and new assessment system that are unique to English learners and students with disabilities

Explain the purpose and characteristics of the new assessment system for California, the Measurement of Academic Performance and Progress (MAPP)

4

Standardized Testing and Reporting Program

A brief review of California’s Standardized Testing and Reporting (STAR) Program (ending in the 2013–14 school year) provides a backdrop for a new CCSS-aligned statewide assessment system.

5

Standardized Testing and Reporting Program

STAR Program assessments were first implemented in 1998 and are aligned to the 1997 academic content standards. The majority of test questions are multiple-choice, with some writing samples. The STAR Program consists of four components:

The California Standards Tests (CSTs)

The California Modified Assessment (CMA)

The California Alternate Performance Assessment (CAPA)

The Standards-based Tests in Spanish (STS)

6

Standardized Testing and Reporting Program

STAR tests were designed to produce (1) school-, district-, county-, and state-level results that allow the state to monitor each school’s progress toward established state performance targets; and (2) results that allow the federal government to monitor the annual yearly progress of schools and local educational agencies (LEAs) toward meeting the accountability targets of the federal Elementary and Secondary Education Act (ESEA), also known as No Child Left Behind (NCLB).

Additional information is available on the STAR Program Resources Web page at http://www.cde.ca.gov/ta/tg/sr/resources.asp.

7

New State Assessment System

The new statewide assessment system was informed by the efforts of the State Superintendent of Public Instruction (SSPI), with input from a statewide work group, through a publication titled, Recommendations for Transitioning California to a Future Assessment System (2013).

• The new system is being developed by a multi-state assessment consortium with capabilities to meet those recommendations.

8

New State Assessment System

The urgent calls for change were taken into consideration when the SSPI’s recommendations were developed and sent to the California State Legislature and the Governor for consideration in the re-authorization of the STAR Program.

SSPI Torlakson (2013) specifically recommended an assessment system:

Whose primary purpose would be to model and promote high-quality teaching and student learning activities

That would include a variety of assessment approaches and item types

That would be aligned with college and career readiness, address 21st Century Skills, and have meaning for students

9

New State Assessment System

Read the following quotes. Notice the repeated references to 21st Century Learning Skills.

“The California Common Core State Standards ask students to acquire a deeper knowledge of the subjects they study and be able to perform more complex tasks using what they have learned. It is critical that we have assessments that measure their progress toward these goals (iv).

"The concept is simple but powerful: If our assessments require students to use problem solving and critical thinking skills to perform well, those same skills are much more likely to be taught in our classrooms day in and day out. The goals we set for our assessment system have profound implications for our students and our schools (iv).”

–Torlakson 2013

10

New State Assessment System

"At the heart of the recommendations is a clear vision and commitment to establishing a bold and innovative assessment system that includes a variety of assessment approaches and item types that model and promote high-quality teaching and student learning and sets a course to ensure that all California students are well prepared to enter college and careers in today's competitive economy (x).

"The CA CCSS call for a more integrated approach to delivering instruction across all subjects. The CA CCSS are designed to be robust and relevant to the real world…They require student collaboration; fluency with multimedia and technology; and the development of strong complex reasoning, problem solving, and communication skills. High-level skills such as these transcend subjects and demand a reexamination of the state's existing system of professional learning, curriculum development, assessment, and accountability (1)."

11

New State Assessment System

Respond to the following:

1. What is one critical difference between the 1997 California academic content standards and the CA CCSS?

2. Summarize the purpose and characteristics of a CCSS-based assessment system in the overall instructional cycle as described in Recommendations for Transitioning California to a Future Assessment System.

12

New State Assessment System

The entire report is available through the Web link below:

Recommendations for Transitioning California to a Future Assessment System: A Report by State Superintendent of Public Instruction Tom Torlaksonhttp://www.cde.ca.gov/ta/tg/sa/documents/suptrecrpt2013.pdf#search=Transition%20Report&view=FitH&pagemode=none

13

MAPP Legislation

Assembly Bill 484 outlines a new CCSS-based assessment system for California, known as the Measurement of Academic Performance and Progress (MAPP). The text of the legislation emphasizes that the purpose of assessment is to promote quality instruction and learning that align with the CA CCSS, CCR Anchor Standards, and 21st Century Skills.

“It is the intent of the Legislature in enacting this chapter to provide a system of assessments of pupils that has the primary purposes of assisting teachers, administrators, and pupils and their parents; improving teaching and learning; and promoting high-quality teaching and learning using a variety of assessment approaches and item types.”

“California should adopt a coordinated and consolidated testing system that will develop statewide academically rigorous content and performance standards that reflect the knowledge and complex skills that pupils will need in order to succeed in the information-based, global economy of the 21st century.”

—Assembly Bill 484 2013

14

MAPP Legislation

The full text and status of the legislation is available through the Web link below:

Assembly Bill 484 http://leginfo.legislature.ca.gov/faces/billNavClient.xhtml?

bill_id=201320140AB484

15

MAPP Legislation

As you watch this video, consider what 21st Century Skills the job applicant is missing that are important to the potential employer:

“What's wrong with this picture?"http://www.youtube.com/watch?feature=player_embedded&v=dY2mRM4i6tY

1. What makes this video simultaneously humorous and realistic?

2. What implications does this video have for you and how you will support 21st Century Skills and implementation of the CA CCSS?

3. How do components of MAPP, based on Recommendations for Transitioning California to a Future Assessment System, support implementation of the CA CCSS?

16

Smarter Balanced Assessments

California is a governing state in the Smarter Balanced Assessment Consortium (SBAC). The goal of SBAC is to develop a balanced, coherent student assessment program that is aligned to the CCSS and measures student progress toward college and career readiness.

Beginning in 2014–15, Smarter Balanced assessments will be administered to students annually in grades 3–8 in English language arts (ELA) and mathematics, and once in grade 11 as required by federal law. Unlike STAR assessments, they will incorporate formative, interim, and summative assessments designed to support instruction and learning.

Smarter Balanced Overview

17

Smarter Balanced Assessments

Smarter Balanced Overview

Map of participating Smarter Balanced states:http://www.smarterbalanced.org/about/member-states/

“Smarter Balanced is guided by the belief that a balanced, high-quality assessment system-including formative, interim, and summative components-can improve teaching and learning by providing information and tools for teachers and schools to help students succeed. Timely and meaningful assessment information can offer specific information about areas of performance so that teachers can follow up with targeted instruction, students can better target their own efforts, and administrators and policymakers can more fully understand what students know and can do, in order to guide curriculum and professional development decisions.”

—Smarter Balanced Assessment Consortium 2012

18

Computer Adaptive Technology

Moving away from fixed-form testing, Smarter Balanced assessments will capitalize on the precision and efficiency of computer adaptive testing (CAT) for both the mandatory summative assessment and the optional interim assessments (SBAC 2012).

19

Computer Adaptive Technology

Read the excerpt below and note key pieces of information on how the Smarter Balanced assessments will differ from the STAR Program assessments.

“Smarter Balanced assessments make use of computer adaptive technology, which is more precise and efficient than fixed-form testing. Teachers, principals, and parents can receive results from computerized assessments in weeks, not months. Faster results mean that teachers can use the information from optional interim assessments throughout the school year to differentiate instruction and better meet the unique needs of their students.”

–Smarter Balanced 2013

20

Computer Adaptive Technology

Computer Adaptive Testing (CAT): Computer automatically scores responses and adjusts subsequent questions according to the ability demonstrated by the test taker on the prior questions. • Provides a more accurate measurement of student achievement,

particularly for high- and low-performing students.

• Every student receives a unique set of test questions, reducing the need to test students on particular content simultaneously.

• Enables schools to group and assess students in a configuration that matches their specific capacity.

For additional information, visit the Smarter Balanced Computer Adaptive Testing Web page at http://www.smarterbalanced.org/smarter-balanced-assessments/computer-adaptive-testing/.

21

Computer Adaptive Technology

Fixed-form testing: All students see the same set or sets of questions that can be varied through the use of different test forms or versions. • Most commonly represented by the traditional paper-and-pencil test

but can also be administered on a computer (i.e., computer-based). • A computer can randomize questions or present them in different

orders for different test takers.

22

A Balanced Assessment System

This graphic illustrates the components of the Smarter Balanced assessment system. Notice the multiple assessment points included.

23

A Balanced Assessment System

“The 'Balanced' in Smarter Balanced refers both to our model of an interconnected system of summative, interim, and formative components, as well as a balance between technologically advanced computer adaptive testing (CAT) and extended, thematically related performance tasks."

—Smarter Balanced Preliminary Test Blueprints 2012

Read about possible outcomes from the perspectives of students, teachers, parents, and policy makers:

What Will Smarter Balanced Assessments Mean for Me?

What Will Smarter Balanced Assessments Mean for Me?

Handout

24

A Balanced Assessment System

1. How does the purpose of California’s proposed assessment system as described by Smarter Balanced differ from the STAR system?

2. Describe benefits you anticipate for teachers, students, and parents as the Smarter Balanced computer adaptive assessment system is implemented in California.

25

Coherent System Components

This table provides additional detail on each of the Smarter Balanced assessment system components. How does this system differ from the STAR Program?

Type of Assessment

When Administered Format/Structure Assessment Methods Purpose/Use

Summative Last 12 weeks of school year

Two parts: Computer Adaptive and Performance Assessment

Selected Response;Constructed Response; Performance Tasks; Technology- Enhanced

Assess student achievement growth as part of program evaluation and school, district, and state accountability

Interim Locally determined intervals

Two parts: Computer Adaptive and Performance Assessment

Selected Response;Constructed Response;Performance Tasks; Technology- Enhanced

Progress monitoring;identifying strengths and needs in relation to the CA CCSS

Formative Tools and Practices

Daily classroom use

Digital, on-demand library of practices, tools, and resources

Not Applicable Progress monitoring;identifying individual student and student group strengths and needs in relation to the CA CCSS; Professional learning for teachers

Components of the Smarter Balanced Assessment System

26

A Balanced Assessment System

Read more about Smarter Balanced and computer adaptive testing in the document titled, Computer Adaptive Testing:

Join the CDE Smarter Balanced electronic mailing list: subscribe-sbac@mlist.cde.ca.gov

Computer Adaptive Testing

Handout

27

Assessment Methods and Item Types

Smarter Balanced assessments will be using several methods and item types to gather information about student learning. The descriptions on the next several slides are applicable not only to Smarter Balanced, but also to any statewide assessment system as well as to district-, school-, and classroom-level assessments.

Assessment Methods (optional)

Handout

28

Assessment Methods and Item Types

Selected-response (SR) items prompt students to select one or more correct responses from a set of choices. Carefully constructed and reviewed SR items allow students to demonstrate their use of complex thinking skills, such as developing comparisons or contrasts; identifying cause and effect; identifying patterns or conflicting points of view; or categorizing, summarizing, or interpreting information.

Selected-Response

29

Assessment Methods and Item Types

Example SR items are available on the Smarter Balanced Web site:

Selected-Response

Grades 6–8 Math: Expressions and equations item 1 http://sampleitems.smarterbalanced.org/itempreview/ModernShell.aspx?config=SBAC\Content\EEProgressions1.json

High School ELA: Diamonds in the Sky 1 http://sampleitems.smarterbalanced.org/itempreview/ModernShell.aspx?config=SBAC\Content\Diamonds1.json

30

Assessment Methods and Item Types

Constructed-response (CR) tasks prompt students to generate a text or numerical response in order to collect evidence about their knowledge or understanding of a given assessment target. Smarter Balanced assessments use both short and extended CR items. Short CR items may require test-takers to enter a single word, phrase, sentence, number, or set of numbers, whereas extended CR items require more elaborated answers and explanations of reasoning.

Constructed-Response

31

Assessment Methods and Item Types

CR tasks allow students to demonstrate complex thinking skills by formulating comparisons or contrasts; proposing causes and effects; identifying patterns or conflicting points of view; categorizing, summarizing, or interpreting information; and developing generalizations, explanations, justifications, or evidence-based conclusions (Darling-Hammond & Pecheone 2010). These complex thinking skills support development of the 4 Cs and college and career readiness.

Constructed-Response

32

Assessment Methods and Item TypesConstructed-Response

Example CR items are available on the Smarter Balanced Web site:

Grades 3–5 Math: Rectangle 2 http://sampleitems.smarterbalanced.org/itempreview/ModernShell.aspx?config=SBAC\Content\Garden2.json

Grades 6–8 Writing: Cell Phones http://sampleitems.smarterbalanced.org/itempreview/ModernShell.aspx?config=SBAC\Content\Writing1a.json

33

Assessment Methods and Item Types

Technology-enhanced (TE) items use specialized computer interactions for collecting response data that go beyond traditional SR or CR items. TE tasks require students to use technology skills to respond to the item prompt. For example, students may be required to draw a figure with a mouse, stylus, or finger; drag and drop text; highlight text; or reorder text.

Technology-Enhanced Items

34

Assessment Methods and Item TypesTechnology-Enhanced Items

Example TE items are available on the Smarter Balanced Web site:

Grades 3–5 ELA: Grandma Ruth 2http://sampleitems.smarterbalanced.org/itempreview/sbac/ELA.htm

High School Math: Water Tankhttp://sampleitems.smarterbalanced.org/itempreview/ModernShell.aspx?config=SBAC\Content\WaterTank.json

35

Assessment Methods and Item Types

Performance tasks (PTs) measure a student’s ability to integrate knowledge and skills across multiple standards—a key component of college and career readiness. PTs are used to measure capacities such as depth of understanding, research skills, and complex analysis, which cannot be adequately assessed with SR or CR items. These item types challenge students to apply their knowledge and skills to respond to complex real-world problems.

Performance Tasks

36

Assessment Methods and Item Types

Smarter Balanced interim and summative components of the assessment system will include PTs at each grade level tested. Additional extended tasks will be available in the Digital Library of Formative Assessments as part of the Smarter Balanced exemplar instructional modules and inventory of currently available resources.

Performance Tasks

Smarter Balanced has released some sample performance tasks focused on assessing critical thinking, problem solving, and communication skills described in MAPP.

37

Assessment Methods and Item TypesPerformance Tasks

Grades 3–5 ELA: Animal Defenses http://www.smarterbalanced.org/wordpress/wp-content/uploads/2012/09/performance-tasks/animal-performance.pdf

Grades 3–5 Math: Planting Tulipshttp://www.smarterbalanced.org/wordpress/wp-content/uploads/2012/09/performance-tasks/tulips.pdf

Grades 6–8 ELA: Garden of Learning http://www.smarterbalanced.org/wordpress/wp-content/uploads/2012/09/performance-tasks/garden.pdf

38

Assessment Methods and Item TypesPerformance Tasks

Grades 6-8 Math: Taking a Field Trip http://www.smarterbalanced.org/wordpress/wp-content/uploads/2012/09/performance-tasks/fieldtrip.pdf

High School ELA: Nuclear Powerhttp://www.smarterbalanced.org/wordpress/wp-content/uploads/2012/09/performance-tasks/nuclear.pdf

High School Math: Thermometer Cricketshttp://www.smarterbalanced.org/wordpress/wp-content/uploads/2012/09/performance-tasks/crickets.pdf

39

Assessment Methods and Item Types

1. Which of the essential 21st Century Learning Skills are required for the performance tasks that you reviewed?

A. Critical thinking and problem solvingB. CommunicationC. CollaborationD. CreativityE. Fluency with multi-media and/or technology

2. What are some of the things you noticed as you reviewed the performance task items?

3. Which assessment item type would you select to gather information on students’ ability to integrate knowledge and skills across standards and/or across content areas?

4.   Which would you choose to gather information on students’ ability to recall information? Describe your rationale.

40

Assessment Methods and Item Types

Additional sample items available on the Smarter Balanced Sample Items and Performance Tasks Web page athttp://www.smarterbalanced.org/sample-items-and-performance-tasks/.

Learn more about California’s role in the development of the Smarter Balanced Digital Library of formative assessment practices and tools:http://www.cde.ca.gov/ta/tg/sa/diglib.asp

41

Smarter Balanced Preliminary Blueprints

Preliminary blueprints for the Smarter Balanced summative assessment (released November 2012) provide detailed information on the CA CCSS content to be assessed and the item types to be used for assessment.

“For those responsible at the instructional level, the test blueprint provides a guide to the relative importance of competing content demands and suggests how the content is intended to be demonstrated…”

“The blueprints provide information on the Smarter Balanced assessment claims, assessment targets, item types, and depth of knowledge.”

—Smarter Balanced General Item Specifications April 2012

42

Smarter Balanced Preliminary Blueprints

Understand the following definitions:

Assessment claims: Broad statements of the assessment system's learning outcomes.

Assessment targets: Description of the expectations of what will be assessed by the items and tasks.

43

Smarter Balanced Preliminary BlueprintsAssessment Targets

Smarter Balanced assessment targets comprise an essential component of a quality assurance approach to item development referred to as evidence-centered design (ECD). Central to this approach is the idea of collecting evidence from a student’s response to an item or task that supports a claim about the extent to which a student has developed the knowledge, skill, and ability that is contained in a content standard (SBAC 2013).

44

Smarter Balanced Preliminary BlueprintsAssessment Targets

“Clear and rigorous prioritized assessment targets translate the grade-level Common Core standards into content frameworks along a learning continuum.”

—Measured Progress/ETS Collaborative 2012

"These summative assessment targets (evidence) at each grade level represent the prioritized content for summative assessment...”

—Smarter Balanced General Item Specifications April 2012

Additional information about the ECD approach is available on the SBAC Item Writing and Review Web page at http://www.smarterbalanced.org/smarter-balanced-assessments/item-writing-and-review/.

45

Smarter Balanced Preliminary BlueprintsDepth of Knowledge

The Smarter Balanced assessment system relies on the depth of knowledge (DOK) performance levels to aid in creating and scoring ELA and mathematics items. More basic levels of knowledge, such as recall, are assessed with SR items, while the deeper levels of knowledge, such as reasoning, are most often assessed with CR items or PTs.

The CA CCSS “…require high-level cognitive demand, such as requiring students to demonstrate deeper conceptual understanding through the application of content knowledge and skills to new situations and sustained tasks. For each assessment target in ELA and mathematics, the depth(s) of knowledge that the student needs to bring to the item/task has been identified.”

—Smarter Balanced General Item Specifications April 2012

46

Smarter Balanced Preliminary BlueprintsDepth of Knowledge

Level 1: Recall and Reproduction Requires eliciting information such as a fact, definition, term, or a simple procedure, as well as performing a simple algorithm or applying a formula.

Level 2: Basic Skills and Concepts Requires the engagement of some mental processing beyond a recall of information.

Level 3: Strategic Thinking and ReasoningRequires reasoning, planning, using evidence, and explanations of thinking.

Level 4: Extended ThinkingRequires complex reasoning, planning, developing, and thinking most likely over an extended period of time.

47

Smarter Balanced Preliminary Blueprints

Explore the Smarter Balanced Assessment Consortium Preliminary Test Blueprints in greater depth: http://www.smarterbalanced.org/wordpress/wp-content/uploads/2011/12/Smarter-Balanced-Preliminary-Test-Blueprints.pdf

48

Smarter Balanced Practice Tests

Smarter Balanced practice tests contain sample questions aligned to the CCSS for ELA and mathematics in grades 3–8 and 11. Practice tests are not computer adaptive, but allow users to experience some of the features of online testing and to get a beginning understanding of how Smarter Balanced will assess students’ mastery of the CCSS (SBAC 2013).

For additional information about Smarter Balanced pilot and practice tests, visit the following Web pages:http://www.smarterbalanced.org/pilot-test/ http://sbac.portal.airast.org/Practice_Test/default.html

49

English Learners and Students with Disabilities in California’s Assessment System

MAPP legislation specifies inclusive practices and checks for fairness when assessing English learners and students with disabilities.

“The department shall develop a three-year plan of activities, with the approval of the state board, supporting the continuous improvement of the assessments developed … Activities may include, but not necessarily be limited to, a variety of internal and external studies such as validity studies, alignment studies, and studies evaluating test fairness, testing accommodations, testing policies, and reporting procedures, and consequential validity studies specific to pupil populations such as English learners and pupils with disabilities.”

‒Assembly Bill 484

50

English Language Assessment

Students in kindergarten through grade twelve whose home language is not English are required to take an annual English language proficiency assessment based on English Language Development (ELD) standards. California’s current assessment is the California English Language Development Test (CELDT) and is based on the 1999 ELD standards.

The English Learner Proficiency Assessment for California (ELPAC) will be first administered in 2015–16 and will be aligned to the 2012 ELD Standards for California Public Schools and the CA CCSS. The system’s goal is to maximize assessment information on language development to support English learners’ attainment of the CA CCSS.

The 2012 ELD Standards for California Public Schools are available on the CDE English Language Development Standards Web page at http://www.cde.ca.gov/sp/el/er/eldstandards.asp.

51

Implementation TimelineA tentative development process and timeline have been established for the ELPAC, culminating in the implementation of a new system in 2015–16. Steps in the process are expected to include:

The California English Language Development Standards Implementation Plan is available on the CDE English Language Development Standards Web page at http://www.cde.ca.gov/sp/el/er/eldstandards.asp.

52

Assessment Accessibility

Access by Design

Smarter Balanced assessment items are designed to be accessible to a broad spectrum of students, including English learners, students with learning disabilities, students with vision needs who require braille and/or tactile supports, students who require audio representation, and students who communicate in American Sign Language.

53

Assessment Accessibility

Access by DesignUnder STAR Program testing, test variations, accommodations, and modifications available to students are specified on the Testing Variations, Accommodations and Modifications Matrix (http://www.cde.ca.gov/ta/tg/sr/).

Developers of the new assessments are using Smarter Balanced General Accessibility Guidelines to support implementation of assessment inclusion strategies from an innovative approach called Access by Design:http://www.smarterbalanced.org/wordpress/wp-content/uploads/2012/05/TaskItemSpecifications/Guidelines/AccessibilityandAccommodations/GeneralAccessibilityGuidelines.pdf

54

Assessment Accessibility

Access by DesignAccess by Design is based on the principals of Universal Design for Learning (UDL) and allows students to interact with the content of the assessment as they apply their knowledge and skills so that their responses accurately reflect their abilities. The computer adaptive system provides support to a wide range of students by matching prompts to their ability levels and through technologies such as font style, size changes, and sound volume adjustments.

Testing variations, accommodations, and modifications will be available as specified in Individual Education Plans (IEPs) or 504 Plans.

Additional information on UDL is available on the CAST Web site at http://www.cast.org/udl/.

55

Students with Significant Cognitive Disabilities

In the STAR Program, students with the most significant cognitive disabilities participate in the CAPA, an assessment aligned with ELA, mathematics, and science content standards from the late 1990s.

To address the needs of this student population under the new system, California joined the National Center and State Collaborative (NCSC), a state consortium working to develop a new CCSS-aligned assessment which will replace the CAPA.

56

Students with Significant Cognitive Disabilities

For more information on alternate assessments, visit the NCSC Web site at http://www.ncscpartners.org/.

The NCSC alternate assessments are designed to reach a wide range of students with significant cognitive disabilities and will assess knowledge in grades 3–8 and once in high school. These alternate assessments will align with the assessment systems being developed by Smarter Balanced and will be ready for use by the 2014–15 school year.

57

Summary of Unit 2

California is transitioning into a new assessment system, MAPP, which is designed to address 21st Century Learning Skills and the skills and knowledge students need to be college and career ready through the CA CCSS and CCR Anchor Standards.

As a member of Smarter Balanced, California is participating in the development of CA CCSS-based summative and interim assessments for students in grades 3–8 and 11. Computer adaptive assessments will use a variety of assessment methods, including selected response, constructed response, and performance tasks.

Smarter Balanced will also develop a Digital Library of Formative Assessment Tools and Practices for classroom use that will be fully aligned to the summative and interim assessments.

58

Summary of Unit 2

English learners and students with disabilities will have access to Smarter Balanced assessments through a variety of universal-design principles integrated into the system.

English learners’ access to the CA CCSS will be maximized through alignment of the new ELPAC to the new California ELD standards.

Students with significant cognitive disabilities will participate in a CA CCSS-aligned assessment system.

59

Summary of Unit 2

Check your understanding of the ideas in Unit 2:

Describe the components of the Smarter Balanced assessment system.

How will access to the CA CCSS and CA CCSS-aligned assessments be maximized for English learners and students with disabilities?