RESEARCH ANDEVALUATION

Test Coordinator Meeting

December 18, 2014

RESEARCH ANDEVALUATION

Agenda

• Welcome/Introductions• Technology Update• Smarter Balanced Interim Assessments

• ICA-Interim Comprehensive Assessment• IAB-Interim Assessment Blocks

• Washington Comprehensive Assessment Program – WCAP• Questions???• Closing Remarks

Time: 4 to 6 p.m.

Location: CAB Auditorium

RESEARCH ANDEVALUATION

Welcome and Introductions

Research and Evaluation• Pat Cummings

Director of Research & Evaluation

• Linda MedfordCoordinator Student Assessment

• Lindsay FranklinAssessment Specialist

Special GuestThea Turnley-Student System Specialist

Curriculum and Instruction• Roy Tatlonghari

Instructional Facilitator

• Nicolle WamboldInstructional Facilitator

Technology• Glenn Desseau

Manger Technology

• Field Technicians

Technology Update

Software deployments are occurring between 6 a.m. – 9 a.m., depending on site schedules.

Remember:• Connect all new COWs to power and data every night.• Connect all laptops in new COWs to power and data every night.• Contact HELP at 253.571.4357 or HELP@Tacoma.k12.wa.us for

assistance.

Technology Update

Elementary Schools Middle Schools High Schools

Number of Schools with SBAC Software Deployed 26

Elementary Schools 22

Middle Schools 3

High Schools 1

Total Number of Managed Cart Laptops 1,708

Laptops with Software 700

Success Rate 41%

Percentage of district machines with SBAC software 25%

Total number to date 3000

SBAC Interim -Comprehensive and Blocks

Interim Assessments Timeline Interim Assessments will be available Jan 6 (more information to come after the

Test Administration Manual is completed and the secure browser is installed) Jan 6 – Comprehensive Interim Jan 27 – Interim Assessment Blocks

Initial release will include fixed forms; adaptive forms will be available as the item pool allows

Interim AssessmentsInterim Comprehen

sive Assessment

(ICA)

Interim Assessment

Blocks (IAB)

Comprehensive (ICA) are almost identical to the end-of-year Smarter Balanced assessments.  They take as long to administer (total of 7 to 8 hours), and report scores as composite scale scores, achievement levels and ‘claims” 

Blocks (IAB) are short tests organized by “claims” Examples or current thinking on test design: ELAMath (Grades 3-5)Math (Grades 6-7)Math (Grade 8)Math High School

There are two types of Interim Assessments: Comprehensive Blocks

Interim Comprehensive Assessments (ICAs)

• Interim Comprehensive Assessments (ICAs) use the same blueprints

as the summative assessments and assess the same standards. They

will be provided as fixed forms and will become adaptive when item

counts are adequate.

• The ICAs include the same item types and formats, including

performance tasks, as the summative assessments, and yield results

on the same vertical scale. The ICAs yield overall scale scores,

overall performance level designations, and claim-level information.

Interim Comprehensive Assessment (ICA)

ICA Reporting

• ICA reporting is the same as for the summative assessment:

Overall scale score with error band endpoints and

achievement level per content area/subject.

Claim score reporting is based on three classifications related

to the overall scale score cut point between levels 2 and 3.

Interim Comprehensive Assessment (ICA)

Interim Assessment Blocks (IABs)

• Interim Assessment Blocks (IABs) focus on smaller sets of

targets and therefore provide more detailed information for

instructional purposes.

• The blocks are available either as fixed forms or with the use of

a computer adaptive algorithm. The IABs yield overall

information for each block.

Interim Assessment Blocks (IAB)

IAB Reporting

• Individual student scores are available for each block.

• Reporting for each block is based on three classifications related to

the overall scale score cut point between levels 2 and 3:

1. Below Standard

2. At/Near Standard

3. Above Standard

Interim Assessment Blocks (IAB)

IABs for ELA Grades 3 – 5

IABs for ELA Grade 3 – 5

1 Read Literary Texts

2 Read Informational Texts

3 Edit/Revise

4 Brief Writes

5 Listen/Interpret

6 Research

7 Narrative Performance Task* Gr 4 and 5 only in 2014-2015

8 Opinion Performance Task* N/A in 2014-2015

9 Informational Performance Task* Gr 3 only in 2014-2015

* This is a full Performance Task as students experience in the summative assessment

IABs for Mathematics Grade 3

IABs for Mathematics Grade 3

1 Operations and Algebraic Thinking

2 Numbers and Operations in Base 10 Not available in 2014-2015

3 Fractions

4 Measurement and Data

5 Mathematics Performance Task*

* This is a full Performance Task as students experience in the summative assessment

IABs for Mathematics Grade 4

IABs for Mathematics Grade 4

1 Operations and Algebraic Thinking

2 Numbers and Operations in Base 10

3 Fractions

4 Geometry- Not available in 2014-2015

4 Measurement and Data

5 Mathematics Performance Task*

* This is a full Performance Task as students experience in the summative assessment

IABs for Mathematics Grade 5

IABs for Mathematics Grade 5

1 Operations and Algebraic Thinking

2 Numbers and Operations in Base 10

3 Fractions

4 Geometry

5 Measurement and Data - Not available in 2014-2015

6 Mathematics Performance Task* - Not available in 2014-2015

* This is a full Performance Task as students experience in the summative assessment

IABs for ELA Grades 6-8

IABs for ELA Grades 6-8

1 Read Literary Texts

2 Read Informational Texts

3 Edit/Revise

4 Brief Writes

5 Listen/Interpret

6 Research

7 Narrative Performance Task* Not available in 2014-2015

8 Opinion Performance Task* Not available in 2014-2015

9 Informational Performance Task*

* This is a full Performance Task as students experience in the summative assessment

IABs for Mathematics Grades 6-8

IABs for Mathematics Grades 6-7

1 Ratios and Proportional Relationships

2 Number System

3 Expressions and Equations

4 Geometry - Not available at Gr 7 in 2014-2015

5 Statistics and Probability - Not available in 2014-2015

6 Mathematics Performance Task*

IABs for Mathematics Grade 8

1Expressions and Equations – I (and Proportionality) - Available with #2 as a single Expressions & Equations Block in 2014-2015

2 Expressions and Equations - II

3 Functions

4 Geometry

5 Mathematics Performance Task*

* This is a full Performance Task as students experience in the summative assessment

IABs for ELA High School

IABs for ELA High School

1 Read Literary Texts

2 Read Informational Texts

3 Edit/Revise

4 Brief Writes

5 Listen/Interpret

6 Research

7 Explanatory Performance Task* Not available in 2014-2015

8 Argument Performance Task*

* This is a full Performance Task as students experience in the summative assessment

IABs for Mathematics High School

IABs for Mathematics High School

Algebra and Functions

1 Linear Functions

2 Quadratics

3 Exponentials – Not available in 2014-2015

4 Polynomials – Not available in 2014-2015

5 Radicals – Not available in 2014-2015

6 Rationals – Not available in 2014-2015

7 Trigonometric – Not available in 2014-2015Geometry

8 Transformations – Not available in 2014-2015

9 Right Triangle Ratios

10 Three-Dimensional Geometry – Not available in 2014-2015

11 Proofs – Not available in 2014-2015

12 Circles – Not available in 2014-2015

13 Applications – Not available in 2014-2015

IABs for Mathematics High School

IABs for Mathematics High School

Other

14 Interpreting Categorical and Quantitative Data – Not available in 2014-2015

15 Probability – Not available in 2014-2015

16 Making Inferences and Justifying Conclusions – Not available in 2014-2015

17 Mathematics Performance Task*

* This is a full Performance Task as students experience in the summative assessment

Preparing for New Test Scores

Smarter Balanced assessments measure the full range of the Common Core State Standards.

They are designed to let teachers and parents know whether students are on track to be

college- and career-ready by the time they graduate.

Because the new standards set higher expectations for students--and the new tests are

designed to assess student performance against these higher expectations--our definition of

grade level performance is higher than it used to be.

As a result, it’s likely that fewer students will meet grade level standards, especially for the

first few years. Results should improve as students have additional years of instruction

aligned to the new standards and become better equipped to meet the challenges they present.

This does not mean that our students are “doing worse” than they did last year. Rather, the

scores represent a “new baseline” that provides a more accurate indicator for educators,

students, and parents as they work to meet the rigorous demands of college and career

readiness.

Option #1 for SBAC Interim Assessments TBD

Comprehensive (ICA)

Available first week January

Almost identical to the end-of-year Smarter Balanced assessments

except is fix and not adaptive

Total of 7 to 8 hours (most likely longer)

Teacher have to score items (how many and what training is need

not yet available)

Report scores as composite scale scores, achievement levels and

‘claims’

Teacher training materials regarding this option not yet available

Option #2 for SBAC Interim Assessments TBD

Blocks (IAB)

Available last week of January

Short tests organized by “claims” (not all claims are available…

information it trickling out)

Test are 1 to 2 hours for each subject (ELA/Math)

Teacher have to score items (which items and what training is need

not yet available)

Report scores as met standard only

Assessment map (what specific items are tested) is not available

Whether the Block will include a performance activity is not available

Teacher training materials regarding this option not yet available

Option #3 for SBAC Interim Assessments TBD

Practice Test (Kent Model... must do all three options) Option is available now. Includes a classroom teaching activity (for each grade and subject.. 30 to 60 minutes) & a

performance task (computer based- 60 minutes) that is connected to the activity. Is followed with the complete online practice test that teaches all the tools (approximately

one hour).

Comments: Do not get scores back Worked very well last year in other districts Can do anytime Can be controlled and monitor from district (we send schedule and copies of performance

tasks) Provides exposure to both tools (online) and performance activities Training is activities are easier to manage and can be consolidated Less stressful for teachers and students The Portal has yet to link the practice test so we do not know if students will log in as

guest or through their student account After the WCAP Portal test is available we will review and determine if  more scripted

teacher training materials are necessary to supplement the activity… this will be in a model similar to Emily Bannon’s Digital Library tutorial.

Washington Comprehensive Assessment Program – WCAP Portal

Located on the Web:

http://wa.portal.airast.org/

Student and FamiliesComing Soon

Test Administrators

Test Coordinators

Technology Coordinators

WCAP - Test Administrators

General Information – Teacher Fact Sheet

• User Guides and Manuals

• Online Reporting System – ORS User Guide – coming soon

• Guidelines on Tools, Supports, Accommodations for State Assessments

• Test Information Distribution Engine – TIDE – coming soon

• Smarter Balanced Usability, Accessibility, and Accommodations Guidelines – coming soon

• Smarter Balanced Accessibility and Accommodations Implementation Guide – coming soon

• Smarter Balanced Test Administration Manual TAM – coming soon

WCAP - Test Administrators (Continued)

Assessment Resources

• Grades 3-5

• Grades 6-8

• High School

Classroom Activities for ELA and Math

Smarter Balanced Usability, Accessibility, and Accommodations Guidelines – coming soon

Smarter Balanced Accessibility and Accommodations Implementation Guide – coming soon

Modules Test Information Distribution Engine (TIDE) Training Module – coming soon

ORS Training Module – coming soon

Smarter Balanced Performance Task Training Module

Smarter Balanced Accessibility and Accommodations Training Module – coming soon

Embedded Universal Tools and Online Features Module – coming soon

Test Administrator Online System Interface Module [PPTX]

Student Testing Interface Training Module – coming soon

WCAP - Test Administrators (Continued)

WCAP – Test Coordinators

General Information – Smarter Balanced Field Test Report

• User Guides and Manuals

• Online Reporting System – ORS User Guide – coming soon

• Guidelines on Tools, Supports, Accommodations for State Assessments

• Test Information Distribution Engine – TIDE – coming soon

• Smarter Balanced Usability, Accessibility, and Accommodations Guidelines – coming soon

• Smarter Balanced Accessibility and Accommodations Implementation Guide – coming soon

• Smarter Balanced Test Administration Manual TAM – coming soon

Assessment Resources

• Grades 3-5 – Grades 6-8 – High School

• Classroom Activities for ELA and Math

• Smarter Balanced Usability, Accessibility, and Accommodations Guidelines – coming soon

• Smarter Balanced Accessibility and Accommodations Implementation Guide – coming soon

• Achievement Level Descriptors (ALD) Glossary of Terms [PDF]

• Smarter Balanced Adaptive Software[PDF]

• Smarter Balanced Infographics Chart

WCAP – Test Coordinators (Continued)

Modules

Test Information Distribution Engine (TIDE) Training Module – coming soon

ORS Training Module – coming soon

Smarter Balanced Performance Task Training Module

Smarter Balanced Accessibility and Accommodations Training Module – coming soon

Embedded Universal Tools and Online Features Module – coming soon

Test Administrator Online System Interface Module [PPTX]

Student Testing Interface Training Module – coming soon

WCAP – Test Coordinators (Continued)

WCAP – Technology Coordinators

General Information• What is a CAT? [PPTX]

• Smarter Balanced Performance Task Training Module [PPTX]

Technical Information

• Online System Requirements Guidelines [DOCX]

• Braille Requirements Documents for Online Systems – coming soon

• System Support Sunset Document – coming soon

• Technical Specifications for the Online System [DOCX]

• Technology Requirements for the Online Systems Training Module[PPTX]

• What is a CAT? [PPTX]

• Smarter Balanced Performance Task Training Module [PPTX]

• Secure Browser Installation Guide[DOCX]

• Smarter Balanced Accessibility and Accommodations Training Module – coming soon

• Embedded Universal Tools and Online Features Module – coming soon

• Student Testing Interface Training Module – coming soon

WCAP – Technology Coordinators (Continued)

Test Information Distribution Engine - TIDE

TIDE Tasks• Upload/edit staff records and roles • Load accommodations and test settings• Managing student rosters (only your enrolled students)• Additional Materials Orders – coming soon

CEDARS tasks NOT TIDE TASKS • Adding/deleting students• Editing student demographics or biographic info• Flagging exempted students (home/private/F1 if enrolled)

TIDE (Continued)

What OSPI will do? Load initial staff records to

TIDE DAC=DC; Designees=DA Emails will arrive as TIDE

goes live 12/19 (unless TIDE is delayed by testing)

Load unique student records to TIDE

Provide list in WAMS of shared and/or dual primary SSIDs for review

TIDE (Continued)

What to do right now?

Prep local security settings for emails

• Separate email coming from OSPI

Alert teachers that emails will be generated

• 3 day expiration on temp password

• Can use ‘forgot password’ without security questions

after temp password expires

Upload staff records with roles

TIDE (Continued)

Digital Library – Training Video

Information on SBAC Field Test

• Plan for training well in advance of testing.• Evaluate how well the training is going with teachers and

administrators, and revise or regroup as needed.• Test equipment early and check all laptops that will be used for

testing at least a few weeks before online test administration. • Be sure computers are completely shut off and restarted once a

week and also restarted the morning of testing before the test session.

• Make sure there are enough headphones for test sessions.• Trouble-shoot any technical issues with the practice and training

tests.• The teacher/proctor needs to be very active and watch students

testing.

Students’ Advice for Teachers Preparing Students for the SBAC

• Be calm know that your class is going to do well.

• Have them do a couple practice sessions so they know what to do when they actually take the test.

• They always need to walk around the room or classroom so they can see if the student needs help because sometimes a student is raising their hand and the teacher does not see.

• Do not get your kids worried about the test. Tell them it is just like any other test. Remember to tell them to relax and just do their best. Also, tell them to check their work multiple times so they can check each question and make sure they got it correct.

• Make sure students know that there is a spell check and dictionary available when they are working on the essay.

• Encourage the student to go slow, take their time, and read every question closely.

• I would tell them to take time to prepare the class for the test because I feel like I did not have enough time to learn and review all the things that were in the test.

• Advice I would give to my teacher who is preparing this next year is kind of nothing. The teachers did a great job!

Grade 3 (128) Grade 4 (73) Grade 5 (208) Grade 6 (151) Grade 7 (82) Grade 8 (66) Grade 11 (37)

Very hard 0.0234375 0.0684931506849315

0.0528846153846154

0.0596026490066225

0.0365853658536585

0.106060606060606

0.216216216216216

Kind of hard 0.1171875 0.219178082191781

0.254807692307692

0.198675496688742

0.146341463414634

0.227272727272727

0.108108108108108

In between easy and hard

0.609375 0.602739726027397

0.543269230769231

0.516556291390729

0.573170731707317

0.515151515151515

0.432432432432432

Kind of easy 0.1953125 0.10958904109589

0.115384615384615

0.185430463576159

0.219512195121951

0.136363636363636

0.108108108108108

Very easy 0.0546875 NaN 0.0336538461538462

0.0397350993377483

0.024390243902439

0.0151515151515152

0.135135135135135

5.0%

15.0%

25.0%

35.0%

45.0%

55.0%

65.0%

75.0%

85.0%

95.0%

Modal response was “in between easy and hard.” Grade 8 and 11 students reported more hard and very hard responses than younger grade levels.

“In general, I thought the Smarter Balanced Assessment was” by Grade Level

Strike-through

Digital Notepad

Mark for Review

Zoom

Highlighter

Spell Check

5.0 15.0 25.0 35.0 45.0 55.0 65.0 75.0 85.0 95.0

Strike-through

Math Tools

Digital Notepad

English Glossary

Mark for Review

English Dictionary

Zoom Calculator High-lighter

Writing Tools

Spell Check

Used it and help-ful

28.8 31.7 33.7 35.4 36.8 41.1 44.1 44.6 46.2 46.7 54.6

Used it but not helpful

16 10.98 17.33 14.69 13.98 14.93 15.07 10.61 14.08 11.01 7.89

Tried it but confusing

5.47 3.35 5.2 3.23 5.24 4.13 3.73 3.18 2.89 4.64 4.14

Didn’t use - didn’t need

31.2 28.51 37.2 31.13 33.87 26.93 30.8 16.31 29.08 21.75 14.84

Didn't use - didn’t know how

3.87 2.95 1.6 1.62 3.23 1.07 1.6 1.33 0.66 3.71 3.07

Didn’t see it on test

14.67 22.49 4.93 13.88 6.85 11.87 4.67 24.01 7.11 12.2 15.51

28.831.733.735.436.8

41.144.144.646.246.7

54.6

The spell check, writing tools, highlighter, calculator, zoom, and English dictionary were used by at least 40% of students taking the field test. The other tools were used to a lesser extent.

“Please tell us how you felt about each of the online tools”(Percent that used and found each tool helpful in green)

RESEARCH ANDEVALUATION

??????????????

Questions

RESEARCH AND EVALUATION

Pat Cummings, Directorpcummin@tacoma.k12.wa.us253-571-1280

Linda Medford, Coordinator Student Assessmentlmedfor@tacoma.k12.wa.us253-571-1193

Lindsay Franklin, Assessment Specialistlfrankl@tacoma.k12.wa.us253-571-1190