Blueprinting Your Way to High-Quality Assessment

Today’s Goal

To create a test blueprint and revise an assessment.

Test Blueprint

DefineReview

Confirm Framework

Using a Blueprint Microsoft Excel Worksheet

Class Year

Learning Target DOK Level 1-4

Method-SR, CR, PT

DOK 1: Recall or Produce

DOK 2: Basic Application

DOK 3: Strategic Thinking

DOK 4: Extended Thinking

Total Points

%

1 0 #DIV/0!2 0 #DIV/0!3 0 #DIV/0!4 0 #DIV/0!5 0 #DIV/0!6 0 #DIV/0!7 0 #DIV/0!8 0 #DIV/0!9 0 #DIV/0!10 0 #DIV/0!

Total Points 0 0 0 0 0% #DIV/0! #DIV/0! #DIV/0! #DIV/0!

Teacher AssessmentsLearning Target

Assessment Blueprint

Building a Test Blueprint

Ideally - - - The blueprint is created before the assessment.

Blueprinting an Assessment

Realistically- - - We can improve our assessments by creating a blueprint from a current assessment and making adjustments.

The Standards

Why are you giving this assessment? What are you measuring?1

Assessments for Student Growth:

Need to assess priority targets (essential skills)

Must align all items to standards

Need to allow for stretch (low-high ability)

Need to be valid & reliable

©2013. Battelle for Kids. All Rights Reserved.

Identify Learning Priorities

Longevity

Leverage

Levels

Learning Priorities

Ainsworth, L. (2003). Power standards: Identifying the standards that matter the most. Englewood, CO: Lead and Learn Press.

©2013. Battelle for Kids. All Rights Reserved.

Learning Priorities

Introduced early Reinforced often Multiple opportunities

for learning

Learning Priorities

Understanding Student Expectations

Grades 3-11 ELA & Math Evidence Tables

Performance Level Descriptors PLD

Science Grades 5 & 8HS Physical Science

Social StudiesGrades 4 & 6HS American History & Government

Test Specifications

Other Subjects Content StandardsAnchor Standards – College & Career Readiness

Using a Blueprint Microsoft Excel Worksheet

Depth of Knowledge

How deeply do the students need to engage with the content?

4 Common Misconceptions about DOK

Misconception #1All kids can’t do this.

• Work with others first; DOK 3-4 are not meant to only be done independently.

• Speaking and listening support deeper thinking.

• Plan questioning and formative checks from DOK 1-4 over the course of a lesson/unit.

-Plan for differentiation and scaffolding

Hess

4.NF.2 Compare two fractions with different numerators and different denominators, e.g., by creating common denominators or numerators, or by comparing to a benchmark fraction such as ½.

Farmer Fred’s fields are worth twelve hundred dollars total. Each field’s value is based on its size. What fraction of the total value is each field worth? How much is each field worth? Show and explain all of your mathematical thinking.

NYC DOE

Misconception #2Bloom’s = Webb’s DOK

Hess

It’s what comes after the verb that indicates the complexity of the task.

Description Tasks• Describe the information contained in graphics

or data tables in the text; or describe the rule for rounding a number.

• Describe how the two characters are alike and different.

• Describe the data or text evidence that supports your solution, reasoning, or conclusion.

• Describe varying perspectives on global climate change using supporting evidence, and identify the most significant effects it might have on the planet in 100 years. Hess

Analysis Tasks• Analyze

Washington’s Farewell Address to explain why it is considered a seminal U.S. document.

• Analyze each statement to decide whether it is fact or opinion.

• Analyze the data. What do you see as other possible outcomes?

• Analyze primary and secondary sources to differentiate between historical fact and historical interpretation.

Misconception #3Webb’s DOK model is a taxonomy.

• Bloom’s is a taxonomy, intended to be a hierarchy.

• Webb’s model is not a taxonomy.

-It differentiates levels of engagement with content

-DOK 3-4 are not better than DOK 1-2

Hess

Misconception #4DOK is about difficulty.

• DOK is about complexity, not difficulty

• What mental processing must occur?

Hess

Level 1 Level 2

Level 3 Level 4

DOK

What it is…

DOK Level 1Recall or Reproduction

• Recall a fact, term, principle, concept, or perform a routine procedure.

• Follow a well-known rule, procedure, or formula.

• The answer is either right or wrong.

Hess, Karin 2013 CRM

DOK Level 2Basic Application & Reasoning

• Use of information • Select appropriate procedures • Usually multiple steps • Routine problems • Apply 2+ concepts• Make limited decisions• One right answer

Hess, Karin 2013 CRM

DOK Level 3Strategic Thinking

• Requires reasoning • Developing a plan to approach a problem• Non-routine• Involves making a claim and providing

supporting evidence = “Back it up, Jack!”• Often more than one possible approach or

answer

Hess, Karin 2013 CRM

DOK Level 4Extended Thinking – Real World

• Requires time to research, problem solve, and process multiple conditions of a task

• An original investigation or application to real world

• Requires significant conceptual understanding and application of skills across disciplines

• Often more than one possible approach or answer

• Multiple sourcesHess, Karin 2013 CRM

DOK is about complexity—not difficulty!

• The intended student learning outcome determines the DOK level. What mental processing must occur?

• While verbs may appear to point to a DOK level, it is what comes after the verb that is the best indicator of the rigor/DOK level.

• The DOK level represents a “ceiling.”  

Karin K. Hess, Ed.D., Senior Associate Center for Assessment, Dover, NH khess@nciea.org

Using a BlueprintMicrosoft Excel Worksheet

The Assessment

Does the assessment provide the BEST evidence of student learning? How do I create an assessment that will allow all students to demonstrate growth?

2

Assessment Methods

Selected Response

Constructed Response

Performance Assessment

Assessment Methods

Selected Response

• Multiple Choice

• Matching• True/False

Constructed Response

• Short Answer

• Extended Response

• Essay

Performance

• Product• Visual• Verbal• Physical

Multiple Assessment Methods

Multiple Assessment Methods

Students Demonstrate Learning through the Written Word

Students Demonstrate Learning through Performance

Selected Response

Constructed Response

Product Performance

Physical/ Verbal/VisualPerformance

Learning is measured

through correct or incorrect responses.

Learning is measured through rubrics orscoring guides. These methods may involve

a variety of responses and/or processes.

Matching Level-Method

Selected Response

Constructed Response

Performance

Level 1 – Knowledge/Recall

Strong Good Poor

Level 2 – Application

Good Strong Good

Level 3 – Strategic Thinking

Poor Strong Strong

Level 4 – Extended Thinking

Poor Good Strong

Learning Expectations Assessment

Learning ExpectationsTarget DOK

LevelMethod

DOK 1Recall or

Reproduce

DOK 2Basic

Application of Skills/ Concepts

 

DOK 3Strategic Thinking

DOK 4Extended Thinking

Total Points

%

Describe how characters in a story respond to major events or challenges.

2 SR & CR     points %

Ask and answer questions to demonstrate understanding of key details in a text.

1 SR       points %

Determine the lesson or moral of a story.

2 CR     points %

Identify difference in point of view of different characters.

3 P     points %

Compare and contrast two or more versions of the same story.

3 P       points %

Total Points     points points points 0 points    

%     % % % 0%    

Assessment Methods1. You can use more than one method for each standards.

2. Choose an appropriate method to meet the demands of the standard and are realistic for number of students and time constraints.

Stretch

Assessments for the purpose of

demonstrating growth need stretch to allow both high and low-

achieving students to show evidence of

learning.

Two ways to achieve this:

Difficulty DOK

Who was the 1st president of the United States?

Who was the 32nd president of the United States?

Identify 1 possible source of water contamination?

Using data on water quality in two locations, conduct a comparative analysis to quantify the quality of the water.

Hess’ Cognitive Rigor Matrix

Bloom’s Taxonomy Webb’s Depth of Knowledge

Karin Hess – Cognitive Matrix

August 2007Kentucky Department of Education

Other Options to Demonstrate StretchInclude items that assess content from

adjacent grade levels

(IF you TEACH this content!)

common core institute

• 1st Grade: Describe characters, settings, and major events in a story using key details.

• 2nd Grade: Describe how characters in a story respond to major events and challenges.

• 3rd Grade: Describe characters in a story (e.g., their traits, motivations, or feelings), and explain how their actions contribute to the sequence of events.

Creating Stretch Across Grade Levels

Example of Stretch

DOK 1: Given a topic, identify two arguments to support or oppose the topic

DOK 2: Identify relevant evidence in a text

DOK 3: Analyze two different texts on the same topic to determine the effectiveness of both arguments. Justify your conclusions.

DOK 4: Research a topic and write an argumentative essay

Writing:Write arguments to support claims in an analysis of substantive topics or texts, using valid reasoning and relevant and sufficient evidence.

The poster seeks to protect America and aid Britain in the struggle against

A. ItalyB. GermanyC. JapanD. China

Low complexity – DOK 1

What type of propaganda is used in this poster?

A. Name CallingB. TransferC. FearD. Testimonial

Higher complexity – DOK 2

Justify your answer using evidence from the poster and your knowledge of propaganda.

Compare these 3 WWII propaganda posters. Explain the type of propaganda used for each. Use your knowledge of the types of propaganda and specific details from the posters to decide with poster you think would be most effective.

High complexity – DOK 3

Other Options to Demonstrate StretchInclude items that assess grade-level

content at a greater or lesser difficulty/lexile

newsela.com

Learning Expectations Assessment

Learning ExpectationsTarget DOK

LevelMethod

DOK 1Recall or

Reproduce

DOK 2Basic

Application of Skills/ Concepts

 

DOK 3Strategic Thinking

DOK 4Extended Thinking

Total Points

%

Describe how characters in a story respond to major events or challenges.

2 SR & CR 2 points 3 points     5 points 20%

Ask and answer questions to demonstrate understanding of key details in a text.

1 SR 3 points       3 points 12%

Determine the lesson or moral of a story.

2 CR 1 point 2 points     3 points 12%

Identify difference in point of view of different characters.

3 P   2 points 4 points   6 points 24%

Compare and contrast two or more versions of the same story.

3 P     8 points   8 points 32%

Total Points     6 points 7 points 12 points 0 points    

%     24% 28% 48% 0%    

Levels of Cognitive Complexity1. Ensure your assessment items meet the depth of knowledge of the standard.

2. Provide an opportunity for stretch in the assessment so all student can show growth.

High Quality Assessment Items

Not all test questions are created equal . . .

Here are a few things to consider . .

The Blueprint

Use the blueprint to align each assessment item with the standard and DoK level of the question in order to evaluate the weight and rigor of the assessment.

3

Reflect on Weight & RigorLearning Expectations Assessment

Learning ExpectationsTarget DOK

LevelMethod

DOK 1Recall or

Reproduce

DOK 2Basic

Application of Skills/ Concepts

 

DOK 3Strategic Thinking

DOK 4Extended Thinking

Total Points %

1. Understand a fraction is part of a whole.

1   sr 4       4 points40 % 

2A & 2B. Represent a fraction on a number line by defining the interval from 0 to 1 at lengths 1/b.

 2  sr 1 2      3 points 30  %

3A. Understand two fractions are equivalent if they are at the same point on a number line.

 1  sr 1       1  points 10  %

3C. Express whole numbers as fractions.

 1  sr  1       1  points 10  %

3D. Compare two fractions and record the results of the comparison as >, =, <. Recognize comparisons are valid only when the two fractions compare to the same whole.

 2  sr1  

    1 points 10  %

Total Points     7 points 3 points points  points     

%     70  % 30  %  % %    

Weight

Rigor

The Power of the BlueprintClass Year

Learning Target DOK Level 1-

Method-SR, CR,

DOK 1: Recall or Produce

DOK 2: Basic Application

DOK 3: Strategic

DOK 4: Extended

Total Points

%

1 CCSS.Math.Content.3.NF.A.1 Understand a fraction 1/b as the quantity formed by 1 part when a whole i s partitioned into b equal parts ; understand a fraction a/b as the quantity formed by a parts of s i ze 1/b. (A fraction is a part of a whole)

1 sr4 4 40%

(2A) CCSS.Math.Content.3.NF.A.2a Represent a fraction 1/b on a number l ine diagram by defining the interva l from 0 to 1 as the whole and partitioning i t into b equal parts . Recognize that each part has s i ze 1/b and that the endpoint of the part based at 0 locates the number 1/b on the number l ine.

2 sr

3 3 30%(2B) CCSS.Math.Content.3.NF.A.2b Represent a fraction a/b on a number l ine diagram by marking off a lengths 1/b from 0. Recognize that the resulting interva l has s ize a/b and that i ts endpoint locates the number a/b on the number l ine.

2 sr0 0%

(3A) CCSS.Math.Content.3.NF.A.3a Understand two fractions as equiva lent (equal ) i f they are the same s ize, or the same point on a number l ine.

1 sr1 1 10%

(3B) CCSS.Math.Content.3.NF.A.3b Recognize and generate s imple equiva lent fractions , e.g., 1/2 = 2/4, 4/6 = 2/3. Explain why the fractions are equiva lent, e.g., by using a visua l fraction model .

2 sr0 0%

(3C) CCSS.Math.Content.3.NF.A.3c Express whole numbers as fractions , and recognize fractions that are equiva lent to whole numbers . Examples : Express 3 in the form 3 = 3/1; recognize that 6/1 = 6; locate 4/4 and 1 at the same point of a number l ine diagram.

1 sr

1 1 10%(3D) CCSS.Math.Content.3.NF.A.3d Compare two fractions with the same numerator or the same denominator by reasoning about their s i ze. Recognize that comparisons are va l id only when the two fractions refer to the same whole. Record the resul ts of comparisons with the symbols >, =, or <, and justify the conclus ions , e.g., by using a visua l fraction model .

2 sr

1 1 10%

Assessment BlueprintTeacher - 3rd Grade Fraction Test Sample

Learning Target Assessments

Questions 4, 5, 6, 7

Questions1, 2, 3

Question 8 - recognize

Question 10 -

recognize

Question 9 - Poorly written

Your Turn• Blueprint your own assessment• Critically analyze the depth of the standards

and the depth of the assessment items• Make adjustments for next year

Does this assessment provide the BEST

evidence of student learning?

education.ohio.govShannon PenceNW Assessment Literacyspence@auglaizeesc.org

www.livebinders.comSearch – Ohio Assessment Literacy

What we learn today doesn’t make what we did yesterday wrong, but it can make tomorrow better. – Jan Chappuis