CFA Seminar 2 1 1 Common Formative Assessments: The Power of Assessments For Learning Somers Public...

CFA Seminar 2CFA Seminar 2 11

Common Formative Assessments:

The Power of AssessmentsFor Learning

Common Formative Assessments:

The Power of AssessmentsFor Learning

Somers Public SchoolsSession #2

Somers Public SchoolsSession #2


Essential QuestionEssential Question

How do Common Formative Assessments connect to other

powerful instruction and assessment practices?

How do Common Formative Assessments connect to other

powerful instruction and assessment practices?


Guiding QuestionsGuiding Questions

What are Common Formative Assessments?

What are the components of a quality common formative assessment?

What are the benefits of using common formative assessments to both teachers and students?

What are Common Formative Assessments?

What are the components of a quality common formative assessment?

What are the benefits of using common formative assessments to both teachers and students?


Our Learning ObjectiveOur Learning Objective

Improve our assessment literacy through deeper understanding of the assessment-design process.

Improve our assessment literacy through deeper understanding of the assessment-design process.


Part 2: Laying The Standards Foundation

Part 2: Laying The Standards Foundation

Overview of 10 Design Steps Steps 1-6

Overview of 10 Design Steps Steps 1-6


Assessment of Only Highest Priority Standards

Assessment of Only Highest Priority Standards

“It is critical that all of the assessed standards be truly significant. From an instructional perspective, it is better for tests to measure a handful of powerful skills accurately than it is for tests to do an inaccurate job of measuring many skills.”

“It is critical that all of the assessed standards be truly significant. From an instructional perspective, it is better for tests to measure a handful of powerful skills accurately than it is for tests to do an inaccurate job of measuring many skills.”

W. J. Popham, Test Better, Teach Better, 2003, p. 143


Laying The Standards Foundation:

Steps 1-6

Laying The Standards Foundation:

Steps 1-6Step 1: Choose Important TopicStep 2: Identify Matching Priority StandardsStep 3: “Unwrap” Matching Priority StandardsStep 4: Create Graphic OrganizerStep 5: Determine the Big IdeasStep 6: Write the Essential Questions

Step 1: Choose Important TopicStep 2: Identify Matching Priority StandardsStep 3: “Unwrap” Matching Priority StandardsStep 4: Create Graphic OrganizerStep 5: Determine the Big IdeasStep 6: Write the Essential Questions


Step 1: Choose an Important Topic to Assess

Step 1: Choose an Important Topic to Assess

With colleagues, identify a topic in your selected content area that is essential for students to understand.

Topic examples: sound-letter relationships, making text connections, persuasive writing, informational text, plants, cells, subtraction with regrouping, linear equations, fraction-decimal equivalency, U.S. Constitution, etc.

With colleagues, identify a topic in your selected content area that is essential for students to understand.

Topic examples: sound-letter relationships, making text connections, persuasive writing, informational text, plants, cells, subtraction with regrouping, linear equations, fraction-decimal equivalency, U.S. Constitution, etc.


Step 2: Identify Matching Priority Standards


Review the Priority Standards for your individual grade or course.

Identify those essential standards that match your chosen topic.

Limit the number you select so you can develop a sufficient number of assessment items for each one.

Review the Priority Standards for your individual grade or course.

Identify those essential standards that match your chosen topic.

Limit the number you select so you can develop a sufficient number of assessment items for each one.




5.2.3 Recognize main ideas presented in texts and provide evidence that supports those ideas.

5.2.4 Draw inferences, conclusions, or generalizations about text and support them with textual evidence and prior knowledge.

5.2.5 Contrast facts, supported inferences, and opinions in text.





Step 3: “Unwrap” Selected Priority Standards

Step 3: “Unwrap” Selected Priority Standards

Identify the key concepts (important nouns or noun phrases) by underlining them.

Identify the skills (verbs) by making them bold.

Identify the key concepts (important nouns or noun phrases) by underlining them.

Identify the skills (verbs) by making them bold.


Step 3: “Unwrap” Matching Priority Standards

Step 3: “Unwrap” Matching Priority Standards








Step 4 : Create Graphic Organizer

Step 4 : Create Graphic Organizer

Enables you to represent each of the “unwrapped” concepts and skills in a way that makes sense to you in bulleted list, concept map, or outline

Reveals “at a glance” all the learning targets (concepts and skills)

Referencing graphic organizer sharply focuses instruction and assessment.

Enables you to represent each of the “unwrapped” concepts and skills in a way that makes sense to you in bulleted list, concept map, or outline

Reveals “at a glance” all the learning targets (concepts and skills)

Referencing graphic organizer sharply focuses instruction and assessment.


Step 5: Determine Big Ideas

Step 5: Determine Big Ideas

Big Ideas are what you want your students to discover on their own as a result of instruction and learning activities.

Big Ideas represent the main ideas, conclusions, or generalizations about the “unwrapped” concepts and skills in a focused instructional unit of study.

Should be drawn from your curriculum documents

Big Ideas are what you want your students to discover on their own as a result of instruction and learning activities.

Big Ideas represent the main ideas, conclusions, or generalizations about the “unwrapped” concepts and skills in a focused instructional unit of study.

Should be drawn from your curriculum documents


Big Idea DefinitionsBig Idea DefinitionsBig Idea

An open-ended, enduring idea that may apply to more than one area of study (broad)

Student-worded statement arising from an integrated understanding of the “unwrapped” concepts and skills just studied (topical)

The student’s answer or response to a related Essential Question!

Big IdeaAn open-ended, enduring idea that

may apply to more than one area of study (broad)

Student-worded statement arising from an integrated understanding of the “unwrapped” concepts and skills just studied (topical)

The student’s answer or response to a related Essential Question!


Broader Big Ideas

Broader Big Ideas are the generalizations derived from one area of study that connect to and can be found in several subject matter areas.

People can justify their conclusions with observable data.


Topical Big Ideas

Topical Big Ideas relate primarily to the inherent understanding in a particular course of study or section of the standards.

The position of a digit determines its value in a number.


Examples of Topical Big Ideas

Examples of Topical Big Ideas

Objects can be compared, classified, and sorted by their different attributes.

Knowing who one is writing for is essential to engaging the readers.

Mathematical formulas and estimates both provide shortcuts for determining needed mathematical information.

Objects can be compared, classified, and sorted by their different attributes.

Knowing who one is writing for is essential to engaging the readers.

Mathematical formulas and estimates both provide shortcuts for determining needed mathematical information.


Step 5 Activity: Determine the Big Ideas

Step 5 Activity: Determine the Big Ideas

Review your “unwrapped” concepts on graphic organizer.

Decide the main or essential understandings you want students to realize on their own by the time they take the common formative (post-) assessment (draw from curriculum documents).

Transfer to the CFA Planning Template

Review your “unwrapped” concepts on graphic organizer.

Decide the main or essential understandings you want students to realize on their own by the time they take the common formative (post-) assessment (draw from curriculum documents).

Transfer to the CFA Planning Template


Questions, Not Statements

Questions, Not Statements

Will stimulate student curiosity to

find the answers

Will stimulate student curiosity to

find the answers


Guidelines for Writing Essential Questions

Guidelines for Writing Essential Questions

Can you write engaging questions that lead your students to discover the Big Ideas on their own?

Can you make your Essential Questions open-ended?

Can you write questions that take students beyond who, what, where, and when to how and why?

Use curriculum document questions and rewrite in “kid-friendly” terms.

Can you write engaging questions that lead your students to discover the Big Ideas on their own?

Can you make your Essential Questions open-ended?

Can you write questions that take students beyond who, what, where, and when to how and why?

Use curriculum document questions and rewrite in “kid-friendly” terms.


Essential Questions to Guide Instruction & Assessment

From “Unwrapping” the Standards, Larry Ainsworth, 2003.

Essential Questions to Guide Instruction & Assessment

From “Unwrapping” the Standards, Larry Ainsworth, 2003.

What are literary devices? Why do authors use them?

(Literary devices enhance and deepen fiction’s impact upon the reader.)

What are literary devices? Why do authors use them?

(Literary devices enhance and deepen fiction’s impact upon the reader.)

This is an example of a “one-two punch” question.

The Big Idea in parentheses is an appropriate response to the second question.


Examples of Topical Big Ideas With Corresponding Essential QuestionsExamples of Topical Big Ideas With Corresponding Essential Questions How is one group of objects different

from another? (Objects can be compared, classified, and sorted by their different attributes.)

How can an author “capture” an audience? (Knowing who one is writing for is essential to engaging the readers.)

Why learn mathematical formulas? How do estimation and formulas work together? (Mathematical formulas and estimates both provide shortcuts for determining needed mathematical information.)

How is one group of objects different from another? (Objects can be compared, classified, and sorted by their different attributes.)

How can an author “capture” an audience? (Knowing who one is writing for is essential to engaging the readers.)

Why learn mathematical formulas? How do estimation and formulas work together? (Mathematical formulas and estimates both provide shortcuts for determining needed mathematical information.)


Step 6 Example: Reading Comprehension Essential Questions

With Corresponding Big Ideas

Step 6 Example: Reading Comprehension Essential Questions

With Corresponding Big Ideas

1. Are main ideas by themselves enough for us to believe them? (Main ideas must be supported with evidence from text and supporting details.)

2. What are conclusions and generalizations? How do we arrive at them? (We draw conclusions and make generalizations from what we read and from our own experiences.)

3. Facts, opinions, inferences! What’s the difference, and why should we know? (Knowing the differences between facts, opinions, and inferences helps you make your own decisions about what you read.)

1. Are main ideas by themselves enough for us to believe them? (Main ideas must be supported with evidence from text and supporting details.)

2. What are conclusions and generalizations? How do we arrive at them? (We draw conclusions and make generalizations from what we read and from our own experiences.)

3. Facts, opinions, inferences! What’s the difference, and why should we know? (Knowing the differences between facts, opinions, and inferences helps you make your own decisions about what you read.)


Step 6 Activity: Write the Essential Questions

Step 6 Activity: Write the Essential Questions

Look again at your Big Ideas. What questions could you ask students

that would lead them to discover your Big Ideas (draw from curriculum documents)?

Write your Essential Questions on the CFA Planning Template

Can you include any “one-two punch” questions?

Check: Do your Big Ideas answer or respond to your Essential Questions?

Look again at your Big Ideas. What questions could you ask students

that would lead them to discover your Big Ideas (draw from curriculum documents)?

Write your Essential Questions on the CFA Planning Template

Can you include any “one-two punch” questions?

Check: Do your Big Ideas answer or respond to your Essential Questions?


Using Essential Questions To Establish Student Learning

Goals

Using Essential Questions To Establish Student Learning

Goals Teachers post in the classroom

the Essential Questions and share them with students at the beginning of an instructional unit.

Establish student learning goal – to be able to answer or respond to the Essential Questions with student-worded Big Ideas by end of instructional unit!

Teachers post in the classroom the Essential Questions and share them with students at the beginning of an instructional unit.

Establish student learning goal – to be able to answer or respond to the Essential Questions with student-worded Big Ideas by end of instructional unit!


Benefits of Essential Questions


Teachers use as instructional filter for selecting lessons and activities that advance student understanding toward Big Ideas

Students develop their understanding of the “unwrapped” concepts and skills as they move through instruction and activities.

Teachers use as instructional filter for selecting lessons and activities that advance student understanding toward Big Ideas

Students develop their understanding of the “unwrapped” concepts and skills as they move through instruction and activities.




Promote students’ integrated understanding of “unwrapped” concepts and skills in their Big Idea responses.

The third section of the common formative assessment requires students to respond to the Essential Questions with the Big Ideas stated in their own words and supported by explanation.

Promote students’ integrated understanding of “unwrapped” concepts and skills in their Big Idea responses.

The third section of the common formative assessment requires students to respond to the Essential Questions with the Big Ideas stated in their own words and supported by explanation.


Break Time!Break Time!


Writing Assessment Items to Get at Identified Standards/Skills

Writing Assessment Items to Get at Identified Standards/Skills

Step 7: Write Selected-Response Items.

Step 8: Write Constructed-Response Items (extended- or short-).

Step 7: Write Selected-Response Items.

Step 8: Write Constructed-Response Items (extended- or short-).


First Know Your PurposeFirst Know Your Purpose

The purpose of assessment, in general, is to find out what your students know and are able to do with regard to the standards you are teaching.

The purpose of Common Formative Assessments, in particular, is to evaluate your students’ understanding of the Priority Standards.

The purpose of assessment, in general, is to find out what your students know and are able to do with regard to the standards you are teaching.

The purpose of Common Formative Assessments, in particular, is to evaluate your students’ understanding of the Priority Standards.


Then Determine Evidence Needed To Make

Inference

Then Determine Evidence Needed To Make

Inference Once the essential purpose of

assessment is identified, educators must ask the critical question:

“What kinds of assessments will provide the best evidence as to whether students have met this singular purpose?”

Once the essential purpose of assessment is identified, educators must ask the critical question:

“What kinds of assessments will provide the best evidence as to whether students have met this singular purpose?”


The Two Major Assessment

Formats

The Two Major Assessment

Formats

1. Selected-Response 2. Constructed-

Response

1. Selected-Response 2. Constructed-

Response


Types of Selected-Response

Types of Selected-Response

Requires students to select one response from a provided list or provide very brief answer

Types include: multiple-choice, True/False, matching, short answer or fill-in from provided list

Requires students to select one response from a provided list or provide very brief answer

Types include: multiple-choice, True/False, matching, short answer or fill-in from provided list


Selected-Response: Benefits and Drawback

Selected-Response: Benefits and Drawback

Benefits: Student answers can be quickly and objectively scored as correct or incorrect. Can be used to efficiently assess students’ knowledge of factual information, main concepts, and basic skills.

Drawback: Tends to promote recall or memorization of factual information, rather than evidence of higher-level understanding—unless items deliberately designed to do so.

Benefits: Student answers can be quickly and objectively scored as correct or incorrect. Can be used to efficiently assess students’ knowledge of factual information, main concepts, and basic skills.

Drawback: Tends to promote recall or memorization of factual information, rather than evidence of higher-level understanding—unless items deliberately designed to do so.


Reasons FOR Selected-Response

Reasons FOR Selected-Response

Better content domain sampling Higher reliability Greater efficiency Objectivity Measurability for higher-level

thinking Mechanical scoring

Better content domain sampling Higher reliability Greater efficiency Objectivity Measurability for higher-level

thinking Mechanical scoringT. M. Haladyna, Writing Test Items to Evaluate

Higher Order Thinking, 1997, pp. 65-66


Reasons AGAINST Selected-Response Reasons AGAINST

Selected-Response Emphasis on learning of isolated

facts (“Multiple choice testing can lead to multiple choice teaching.”)

Inappropriate for some purposes (writing and creative thinking)

Lack of student writing (unless part of assessment design)

Emphasis on learning of isolated facts (“Multiple choice testing can lead to multiple choice teaching.”)

Inappropriate for some purposes (writing and creative thinking)

Lack of student writing (unless part of assessment design)

T.M. Haladyna, Writing Test Items to Evaluate Higher Order Thinking, 1997, pp. 66-67


Multiple-Choice To Assess

Higher Level Thinking?

Multiple-Choice To Assess

Higher Level Thinking?Common myth is that multiple choice items ONLY assess lower-level thinking skills (recall of information, etc.) and therefore will not be appropriate for evaluating students’ higher-level thinking skills.

Common myth is that multiple choice items ONLY assess lower-level thinking skills (recall of information, etc.) and therefore will not be appropriate for evaluating students’ higher-level thinking skills.


Dispelling the MythDispelling the Myth “Research has not yet been done to

prove this point (that multiple choice items can only assess lower-level thinking). If you want to measure understanding and some types of mental skills and abilities, multiple-choice formats can be written to measure these behaviors effectively.”

“Research has not yet been done to prove this point (that multiple choice items can only assess lower-level thinking). If you want to measure understanding and some types of mental skills and abilities, multiple-choice formats can be written to measure these behaviors effectively.”

T. M. Haladyna, Writing Test Items to Evaluate Higher Order Thinking, 1997, p. 98


No Research Support Favoring One Assessment

Format

No Research Support Favoring One Assessment

Format“There is simply no body of assembled

research indicating that one of these item-types is superior to the other.”

Challenge: Can you write selected-response items to assess students’ higher-level thinking skills as indicated by rigor in “unwrapped” skills?

“There is simply no body of assembled research indicating that one of these item-types is superior to the other.”

Challenge: Can you write selected-response items to assess students’ higher-level thinking skills as indicated by rigor in “unwrapped” skills?



Constructed-ResponseConstructed-Response

Includes short-response (words, phrases, single sentences, computation, etc.) and extended-response (multiple sentences or paragraphs)

Requires students to organize and use knowledge and skills to answer a question or complete a task

Includes short-response (words, phrases, single sentences, computation, etc.) and extended-response (multiple sentences or paragraphs)

Requires students to organize and use knowledge and skills to answer a question or complete a task


Constructed-ResponseConstructed-Response More likely to reveal whether or not

students have gained integrated understanding with regard to standards they are learning

Requires scoring guide (rubric) to evaluate degree of student proficiency

More likely to reveal whether or not students have gained integrated understanding with regard to standards they are learning

Requires scoring guide (rubric) to evaluate degree of student proficiency


Constructed-Response ItemsBenefit and Drawbacks

Constructed-Response ItemsBenefit and Drawbacks

Benefit: Provide teachers with more valid inferences about student understanding than those derived from selected-response items

Drawbacks: Take longer to score; can have errors in design; dependent on student writing proficiency; challenge to score accurately

Benefit: Provide teachers with more valid inferences about student understanding than those derived from selected-response items

Drawbacks: Take longer to score; can have errors in design; dependent on student writing proficiency; challenge to score accurately


More Valid Inferences from Constructed Response

More Valid Inferences from Constructed Response

“Because a student really needs to understand something in order to construct a response based on that understanding, students’ responses to these sort of items will better contribute to valid inferences (about their actual levels of mastery) than will students’ answers to selected-response items.”

“Because a student really needs to understand something in order to construct a response based on that understanding, students’ responses to these sort of items will better contribute to valid inferences (about their actual levels of mastery) than will students’ answers to selected-response items.”



How Many Items to Write?

How Many Items to Write?

“A good guideline for making decisions regarding percentage of importance for each learning target is that the percentage of instructional time and percentage of assessment items should be roughly equal.”

“A good guideline for making decisions regarding percentage of importance for each learning target is that the percentage of instructional time and percentage of assessment items should be roughly equal.”R. J. Stiggins, J. Arter, J. Chappuis, and S. Chappuis, Assessment for Learning: An Action Guide For School Leaders, 2004, p. 114.


The Best GuidelineThe Best Guideline

Remember the purpose of your assessment.

Then ask, “How many total items do I need in order to be able to make an accurate inference as to what students know and can do?”

Limit the total number of items so that student papers can be quickly scored and the results used right away.

Remember the purpose of your assessment.

Then ask, “How many total items do I need in order to be able to make an accurate inference as to what students know and can do?”

Limit the total number of items so that student papers can be quickly scored and the results used right away.


A Multiple-Measure Assessment

A Multiple-Measure Assessment

Our model of common formative assessment includes: One or more selected-response types One extended-response item Student Big Idea responses to Essential

Questions Using a “multiple-measure”

assessment enables educators to make more accurate inferences.

Our model of common formative assessment includes: One or more selected-response types One extended-response item Student Big Idea responses to Essential

Questions Using a “multiple-measure”

assessment enables educators to make more accurate inferences.


Writing First-DraftAssessment ItemsWriting First-DraftAssessment Items

• Specific Item Writing Guidelines• Step 7: Write Selected-Response Items• Step 8: Write Constructed-Response

Items

• Specific Item Writing Guidelines• Step 7: Write Selected-Response Items• Step 8: Write Constructed-Response

Items


General Guidelines for Effective Item WritingGeneral Guidelines for Effective Item Writing

Reflect higher-order instructional objectives

Students should not be able to answer solely from memory—must apply their knowledge, not just recall it

Reflect higher-order instructional objectives

Students should not be able to answer solely from memory—must apply their knowledge, not just recall it

W. J. Popham, Test Better, Teach Better, ASCD, 2003.


General Guidelines for Effective Item WritingGeneral Guidelines for Effective Item Writing

Questions should be based—in part—on new material (i.e. be able to read or interpret graph they have never seen before)

Be brief and clear—goal is to “test mastery of material, not students’ ability to figure out what you’re asking”

Questions should be based—in part—on new material (i.e. be able to read or interpret graph they have never seen before)

Be brief and clear—goal is to “test mastery of material, not students’ ability to figure out what you’re asking”

W. J. Popham, Test Better, Teach Better, 2003.

R. J. Stiggins, Student-Centered Classroom Assessment, 2nd ed., 1997, p. 134.


Five Roadblocks to Effective Item-Writing

Five Roadblocks to Effective Item-Writing

1. Unclear directions2. Ambiguous statements3. Unintentional clues4. Complex phrasing5. Difficult vocabulary

1. Unclear directions2. Ambiguous statements3. Unintentional clues4. Complex phrasing5. Difficult vocabulary



Criteria for Writing Selected-Response Items

Criteria for Writing Selected-Response Items

1. Write clearly in a “sharply focused manner.”

2. Ask a question with only one best answer.

3. Write items consistent with grade-level reading expectations.

4. Eliminate clues leading to correct answer.

5. Make response options brief.

1. Write clearly in a “sharply focused manner.”

2. Ask a question with only one best answer.

3. Write items consistent with grade-level reading expectations.

4. Eliminate clues leading to correct answer.

5. Make response options brief.Adapted from R. J. Stiggins, Student-Centered Assessment, 2001.


Step 7 Activity, Section 1: Write Selected-Response Items


Choose the particular types (multiple-choice, true-false, matching, fill-in from provided list) that will best meet your purpose.

Continue writing items while referencing the specific criteria provided. (Note: You may wish to review the two or three multiple-choice items you have already written to decide whether or not they meet the specific guidelines.)

Choose the particular types (multiple-choice, true-false, matching, fill-in from provided list) that will best meet your purpose.

Continue writing items while referencing the specific criteria provided. (Note: You may wish to review the two or three multiple-choice items you have already written to decide whether or not they meet the specific guidelines.)




Remember to design items to match level of rigor in skills!

For multiple-choice, first write the stem; next write the correct/best answer; then write the distracters.

Remember to design items to match level of rigor in skills!

For multiple-choice, first write the stem; next write the correct/best answer; then write the distracters.


Resources for Common Formative Assessment

Items

Resources for Common Formative Assessment

Items Textbook questions* (that meet

criteria for well-written items) Assessment or evaluation

components* of text series State website for released state

exam questions

Textbook questions* (that meet criteria for well-written items)

Assessment or evaluation components* of text series

State website for released state exam questions

*Check for permission to duplicate any copyrighted material.


Online Question BanksOnline Question Banks

NAEP (The Nation’s Report Card) http://nces.ed.gov/nationsreportcard/ITMRLS/

State websites with released assessment items

“Banks” of teacher-created common formative assessments following this model

NAEP (The Nation’s Report Card) http://nces.ed.gov/nationsreportcard/ITMRLS/

State websites with released assessment items

“Banks” of teacher-created common formative assessments following this model

http://nces.ed.gov/nationsreportcard/ITMRLS/







Step 8 Activity, Section 2: Write Constructed-Response

Items

Step 8 Activity, Section 2: Write Constructed-Response

Items Refer to your “unwrapped” concepts, skills,

and Bloom’s Taxonomy levels on your template.

Decide what types of constructed-response items you will write (one extended-response or a few short-response).

Practice writing items referencing the specific criteria provided.

Design items to match level of rigor in skills!

Refer to your “unwrapped” concepts, skills, and Bloom’s Taxonomy levels on your template.

Decide what types of constructed-response items you will write (one extended-response or a few short-response).

Practice writing items referencing the specific criteria provided.

Design items to match level of rigor in skills!

CFA Seminar 2 1 1 Common Formative Assessments: The Power of Assessments For Learning Somers Public...

Documents

Transcript of CFA Seminar 2 1 1 Common Formative Assessments: The Power of Assessments For Learning Somers Public...