Assessment Literacy Series

Quick Start ©

1

-Orientation Module-

1. Develop high-quality performance measures for

use within educator effectiveness systems.

2. Increase the assessment literacy of participants,

thus resulting in more accurate measures of

student achievement.

3. Gain a deeper understanding of academic

content standards.

2

Quick Start

- Program Objectives-

3

Quick Start

“A Flexible System “

Create assessments best

suited for educators

New

Establish the quality of current

assessments

Existing Augment current assessments with

customized items/tasks

Upgrade

Quick Start

Process Components

Des

ign

• Purpose Statement

• Targeted Content Standards

• Test Blueprint

Bu

ild

• Items/Tasks

• Scoring Keys Rubrics, & Checklists

• Test Forms

Rev

iew

• Item/Task Reviews

• Alignment Reviews

• Data Reviews

• Refinements

4

Training Material

Modularized into three (3) phases

Organized into four (4) subfolders

• Videos: PowerPoint Presentations, Videos, etc.

• Guides: Handouts, Rules of Thumb, Models, etc.

• Templates: Forms, Spreadsheets, Calculators, etc.

• Stuff: Research, Technical Papers, Websites, etc.

Delivered via online learning platform: Homeroom

http://homeroom.ria2001.org/login.php

5

Helpful Resources

What is Assessment Literacy?

6

The skills, knowledge, and concepts

associated with sound assessment

practices, including the critical review

of quality evidence.

Assessment Life Cycle Establish Purpose

Select Content

Standards

Build Test Blueprint

Develop Items/Tasks

Develop Scoring Key-

Rubrics

Create Test

Forms & Guidelines

Conduct Quality Reviews

Examine Assessment

Results/Data

Apply Refinements

7

Principles of Well-developed

Measures Measures must:

• Be built to achieve the designated purpose.

• Produce results that are used for the intended purpose.

• Align to targeted content standards.

• Contain a balance between depth and breadth of targeted content.

• Be standardized, rigorous, and objective.

• Be sensitive to time constraints and standards of fairness.

• Be valid and reliable.

8

STEP #1-Purpose Statement

• Explain the measure’s intent, what they will measure, and how

the results will be used.

STEP #2-Targeted Content Standards

• Select content standards associated with each performance

measure.

STEP #3-Test Blueprints

• Identify the number of items assigned to each targeted content

standard and the distribution of cognitive levels.

9

Assessment Life Cycle

-Procedural Steps-

STEP #4-Item/Task Development

• Create questions/tasks for test-takers in order to ascertain test

takers knowledge, skills, and concept mastery of the targeted

content standards.

STEP #5-Scoring Keys & Rubrics

• Provide answers for multiple choice items and detailed scoring

rubrics or checklists for constructed response tasks.

STEP #6-Test Forms

• Organize and sequence items/tasks, add item tags, and develop

administration guidelines that provide detailed instructions for

how to prepare, administer, and collect results.

10

Procedural Steps (cont.)

STEP #7-Item/Task Reviews

• Review operational forms, scoring key, rubric, checklist, and

administration guidelines to ensure pre-established quality

expectations have been met.

STEP #8-Alignment Reviews

• Determine the overall rigor and alignment at the item/task-

level and for the overall operational form.

STEP #9-Data Reviews

• Review data to evaluate items/tasks and overall assessment

performance.

11

Procedural Steps (cont.)

STEP #10-Refinements

• Use data to improve the assessment’s technical quality,

alignment characteristics and performance level

expectations.

• Identify items/tasks needing replacement or refinement

based upon student results, teacher feedback, etc.

• Establish the number of replacement items/tasks and the

number of new items to “field test.”

• Correct shortcomings identified in the Performance Measure

Rubric.

12

Procedural Steps (cont.)

Quick Start program meets the foundational needs

of educators by using performance measure data to

guide instruction and educator effectiveness systems.

Quick Start program provides a step-by-step

framework to guide educators in either creating new

assessments or refining existing assessments that better

match classroom instruction.

13

Program Summary