Writing Constructed Response Items

Professional Development to Practice

The contents of this presentation were developed under a grant from the US Department of Education to the Missouri Department of Elementary and Secondary Education (#H323A120018). However, these contents do not necessarily represent the policy of the US Department of Education, and you should not assume endorsement by the Federal Government. Professional Development to Practice


Writing Constructed

Response Items

Common Formative Assessment


KEY:

Core Training Modules

Follow-up Training Modules

Precursors to Training

Once teams determine an EP to focus on, they can choose one or multiples of these focused modules. Each of the EP modules in this section will include: implementation guidance with tools and troubleshooting, and using data to determine effectiveness.

The contentsof this presentation weredeveloped under a grant fromtheUSDepartment of Education to theMissouri Department of Elementaryand Secondary Education(#H323A120018). However, thesecontents do not necessarily representthe policy of the US Department ofEducation, and youshould not assumeendorsement by the FederalGovernment.

Collaborative Data Teams (CDT)

Foundational Processes

AgendasCommunicationNormsRoles

Overview and Purpose

Collaborative Teams

Activity: Wrap Up/Overview of

Next Steps

Follow-Up Based on Data: Coaching and Revisiting PD

School-Based Implementation Coaching

Overview and Purpose of

Coaching for supporting

school-wide implementation

Critical skills of coaching

Coaching in Practice


Next Steps

Follow-Up Based on Data: Coaching and Revisiting PD

Collaborative Work Training

Follow-up to Training

Getting Started

Wrap Up Activity

Focus AreasIntroduction to Missouri Collaborative Work

Use Getting Started Guide to determine starting point and scope of learning

Data-Based Decision Making (DBDM)

Overview and Purpose of DBDM

Data Team Process Steps Sequence and Examples

1. Collect and Chart Data2. Analyze and Prioritize3. SMART Goal4. Instructional Decision Making5. Determine Results Indicators6. Ongoing Monitoring

Developing Meaningful

Learning Targets

Quality Assessment

Design

Performance Events

Constructed Response

Items

Selected Response

Items

Common Formative Assessment (CFA)

Overview and Purpose of CFA

AdvancedProcesses

Consensus Collaborative SkillsProtocols

Overview and Purpose of EP

Effective Teaching/ Learning Practices (EP)

Spaced versus

Massed

Feedback

Assessment Capable Learners

Reciprocal Teaching

Spaced versus

Massed

Assessment Capable Learners

September 2013


Next Steps


Next Steps


Next Steps


Writing Constructed

Response Items

Developing Meaningful Learning

Targets

Quality Assessment Design

Common Formative Assessment (CFA)

Overview and Purpose of CFA


What are Constructed-Response (CR) Questions?

Formulate own answersConstruct own responsesClosed and Open-Ended CRs


What do you think?

What is the difference between an open-ended constructed response item and a closed constructed response item?

Jana Scott, University of MO-Columbia, Revised Presentation 2013.


Constructed Response Questions

CLOSED CRs Measure knowledge and comprehension Elicit a right or wrong answerNo tolerance for diversityUsually allow for only one way to arrive at a correct answer

OPEN-ENDED CRs •Measure higher-level cognitive processes such as application, inference, analysis and synthesis•Elicit diverse answers•Mild to high tolerance for diversity•Usually allow for more than one way to arrive at an answer


Degrees of Open-Endedness

Closed CR Minimal

Open-Ended CR

Medium Open-Ended CR

Extended Open-Ended CR


Closed 0-1 Minimal Open Ended 0-2 Medium Open Ended 0-3 Extended Open-Ended 0-4

Closed CR items have two distinct levels of performance….right or wrong or acceptable and not acceptable.

The range of responses yielded by minimal constructed response items necessitates the use of three distinct levels of performance.

The range of responses yielded by medium constructed response items necessitates the use for four distinct levels of performance.

The wide range of responses yielded by extended constructed response items necessitates the use for five distinct levels of performance.(Sometimes referred called Extended Responses.)

Example: What is the setting of the story Little Red Riding Hood (LRRH)?

Example: What is the setting of the story LRRH? How do you know?

Example: What is the setting? Explain the impact on the story if the setting was changed?. Use two specific examples from the story to support your answer.

Example: Change the setting of LRRH to a city. Explain how the plot, characters, and major story events are impacted and how they are different than the original story.

Usual time for completion is about 2-3 minutes.




Jana Scott, University of MO-Columbia, 2012.

Constructed Response Fall on a Spectrum of Open-Endedness (Handout)


Inquiry Activity What characteristics do open-

ended constructed response (OCR) questions possess?

Samples in handout packet.


OCR QuestionsApplication Complex cognitive processesElicit diverse answersMultiple ways to arrive at an

answerSupport or justification, More than one component piece.


OCR Questions Making an inference Interpreting information Logical responses based on

givensClear concise answer Completed anywhere from 2-15

minutes (depending on type


Different Names for Same Type of Item (Handout)

Missouri Terms Smarter Balanced Math

Smarter Balanced ELA

Assessment For Learning

Leadership and Learning Center

Closed Constructed Response Items 0-1

Referred to as fill-in-the-blank or short answer questions

Not referenced Referred to as short answer written response

Does not consider Fill-in the Blank to be CR Items

Referred to as fill-in the blank and short answer questions

Minimal Open-Ended Constructed Response 0-2

Medium Open-Ended Constructed Response Item 0-3

Referred to as brief constructed response questions

Usually part of the computer-adaptive section

Referred to as 2 Point CRs or 3-Point CRs

Referred to as short written response items

Referred to as short constructed response

Extended Open-Ended Constructed Response Items 0-4

Performance Events or Tasks

Any CR Item that is part of the Performance Task will be referred to as Extended Response (ER) items.

Performance tasks are a series of scaffolded CRs (called ERs) related to a theme.

Not referenced

Part II of Performance Task is a Writing Prompt

Referred to as extended written response items

Referred to as Performance Task

Referred to as extended response

Referred to as extended response


1. Forces students to demonstrate an in-depth understanding of content 2. Requires the use of complex cognitive processes or higher level thinking skills 3. Elicits diverse answers 4. Allows for multiple points of view and interpretations. 5. Requires support, justification, evidence, and/or explanation of reasoning 6. Requires the student to refer to specific places in the text that justify their conclusions

or claims 7. Requires students to “inference or interpret information 8. Is worded in a clear, concise manner 9. Prompts the student to do everything required by the rubric for the highest rating 10. Avoids teaching in the item stem (giving clues in the item stem) 11. If support is elicited, there is ample information (in story, graph, etc.) for students to use as support. 12. Is written in the simplest language possible.

Critique Form for OCRs (Handout) Except Mathematics


1. The problem allows for multiple solution processes or may be solved in more than one way. (This does not necessarily mean that there are multiple answers; instead there are multiple ways to arrive at an answer.)

2. The problem allows for application of knowledge and/or is set in a real world scenario. 3. The solution process or how one goes about finding an answer is not apparent. 4. The problem involves multiple steps. 5. The problem contains enough information to find a solution and have clarity; but not too

much to diminish the problem-solving approach to learning. 6. The problem is written is such a way to clearly elicit the desired evidence of a student’s

problem-solving approach. 7. When feasible, the problem promotes connections between and among concepts. 8. When feasible, the problem requires supporting reasoning/data and/or justification/

explanation. 9. When feasible, the problem allows for conjecture. 10. The problem accurately assesses the learning target. 11. The problem is appropriate in terms of grade-level difficulty, cognitive complexity, and

reading-level. 12. The content and terms used are mathematically accurate. 13. The problem asks students to show a process, communicate a process, and/or explain a

problem-solving approach. 14. The wording used is clear, concise, and to the point.

Critique Form for Higher-Level Math CRs (Handout)


Constructed Response Items from Smarter

Balanced Assessment Consortium


The table below shows the number of students in each third-grade class at Lincoln School.

There are 105 fourth-grade students at Lincoln School. How many more fourth-grade students than third-grade students are at Lincoln School? Show or explain how you found your answer.

The table below shows the number of students in each third-grade class at Lincoln School.

There are 105 fourth-grade students at Lincoln School. How many more fourth-grade students than third-grade students are at Lincoln School? Show or explain how you found your answer.

Students in Third-Grade

Class Number of Students

Mrs. Roy 24

Mr. Grant 21

Mr. Harrison 22

Ms. Mack 25


Scaffolded Set of

CRs Grade 7


Scaffolded Set of CRs Grade 8


Constructed ResponseExtended Response

Ms. McCrary wants to make a rabbit pen in a section of her lawn. Her plan for the rabbit pen includes the following:•It will be in the shape of a rectangle.•It will take 24 feet of fence material to make.•Each side will be longer than 1 foot.•The length and width will measure whole feet.

Part ADraw 3 different rectangles that can each represent Ms. McCrary’s rabbit pen. Be sure to use all 24 feet of fence material for each pen.

Use the grid below. Click the places where you want the corners of your rectangle to be. Draw one rectangle at a time. If you make a mistake, click on your rectangle to delete it. Continue as many times as necessary.

Use your keyboard to type the length and width of each rabbit pen you draw. Then type the area of each rabbit pen. Be sure to select the correct unit for each answer.

[Students will input length, width, and area for each rabbit pen. Students will choose unit from drop down menu.]

Pen 1: Length: (feet, square feet) Width: (feet, square feet) Area: (feet, square feet)

Part BMs. McCrary wants her rabbit to have more than 60 square feet of ground area inside the pen. She finds that if she uses the side of her house as one of the sides of the rabbit pen, she can make the rabbit pen larger.•Draw another rectangular rabbit pen. •Use all 24 feet of fencing for 3 sides of the pen.•Use one side of the house for the other side of the pen. •Make sure the ground area inside the pen is greater than 60 square feet.Use the grid below. Click the places where you want the corners of your rectangle to be. If you make a mistake, click on your rectangle to delete it.



Use your keyboard to type the length and width of each rabbit pen you draw. Then type the area of each rabbit pen. Be sure to select the correct unit for each answer.

Length: (feet, square feet) Width: (feet, square feet) Area: (feet, square feet)


Grade 4 OCR


Grade 4 ELA


Handout Packet


Items That Are Not Considered to Be Quality

Speculate the Reasons! 1. Create a pamphlet to advertise a country. Include the

country’s flag, anthem, imports, exports, etc.2. Write an essay to explain what the country of Canada

is like. 3. Create a PowerPoint Presentation about the novel Red

Badge of Courage. 4. Draw a chart showing the three branches of

government. 5. Draw a picture of a cell. Be sure to add labels. 6. Write five events that happened in the story. 7. Draw a picture of the food pyramid. 8. Explain the causes of the Civil War.


What are two major flaws in this OCR question?

How might it be repaired? In class, we learned that when supply

goes down, cost goes up. Explain what would happen to the price of hula-hoops if the largest supplier in the United States went out of business. Use four details to support your answer.


What are the flaws in these questions?Then, describe how to repair the

questions. 1. Did you like the story? Explain why or why not

using three details and/or examples from the story.

2. Explain, what you think will happen to John (the main character) after the end of the story.

3. Predict what will happen to Mary if she disobeys her mother? Use two details and/or examples from the story in your answer.

4. What are two historical reasons to explain the present-day appearance of our national flag?


Possible Stimulus Materials ELA Mathematics Other

Readings • video clips • audio clips • research topic/issue/ problem • graphs, charts, other visuals • etc.

Graphs Pictures Models Tables Figures Scenarios Data Bases Video Clips Maps Spreadsheets Photos Research Reports

Any items used in ELA and Mathematics.

Any items used in your subject area.


Locate Stimulus Materials • National Archives• Smithsonian Institute• Marco Polo website • Internet search for charts, graphs,

maps, excerpts from documents• Digital video clips from U-tube • Magazine articles • Newspaper articles • Reading A to Z • Project Gutenberg


Complete the graphic organizer to show the social, political, and economic impact of the Monroe Doctrine.

Mon

roe

Doc

trin

e

Social

Political

Economic

Before After


Complete the graphic organizer to show two cause-effect relationships contained in the article.

Cause Effect


Steps in Writing an

Open-Ended CR

See Handout


1. Work in a small group.

2. Choose any brochure from the center of your table to use as stimulus material.

3. Write an open-ended constructed response question related to any content area.

4. Be ready to share.

Apply Your Knowledge


Scoring Open-Ended

Constructed

Response Items


Scoring a Closed CR

Generic Rubric for Closed CR 1 PointThe response demonstrates the knowledge necessary to complete the prompted purpose and contains the correct or acceptable answer. O Points The response demonstrates little to no evidence of knowledge that is appropriate to the intent of the prompted purpose and does not contain the correct answer.


Requires a different approach than just grading responses right or wrong

OCR questions elicit responses that fall on a wide spectrum of possibilities

Criterion referenced scoring guide or rubric is used.

Generic or Task Specific

Response is rated based in its ability to meet specified criteria

Scoring Responses to OCRs


GenericGeneric Rubric for a Minimal CR

2 Points The response demonstrates proficient evidence of the knowledge necessary to complete the prompted purpose. The student’s strategy and execution meet the content (including concepts, technique, representations and connections), thinking processes and qualitative demands of the task. Minor omissions may exist, but do not detract from the correctness of the response. 1 PointThe response demonstrates partial knowledge necessary to complete the prompted purpose. It may contain overlooked issues, misleading assumptions, and/or errors in execution. Evidence in the response demonstrates that that the student can revise the work to accomplish the task with the help of written feedback. The student does not need dialogue or additional instructions. O Points The response lacks any evidence of knowledge that is appropriate to the intent of the prompted purpose.


2 Points: The response contains a main idea and two details as support.

1 Point: The response contains the main idea and one detail as support.

0 Points: The response shows little to no knowledge.

Task Specific


Steps in Writing an

Open-Ended CR

See Handout


Steps in Writing a Scoring Guide

See Handout


2 Points: The response contains an acceptable/correct answer and a valid solution process. The response shows an understanding of the process needed to find the answer.

1 Point: The response contains an acceptable/correct answer. OR 1 Point: The response contains a valid solution process but had minor computational errors. The response shows an understanding of the process needed to find the answer. 0 Points: The response shows severe misunderstanding.

Scoring Guide for an OCR Math Problem


Sample Scoring Guide for a Mathematics OCR from SBAC


Rubric for a 2-Point ELA OCR Item


44

Rubric for a 3-Point ELA OCR Item


Generic Rubric for Extended CR


Generic Rubric for Medium CR


Generic Rubrics for Minimal and Closed CRs


Work with a small group to write a scoring rubric for these two questions. Look at the steps in your handout packet.

1.What is the main idea of the article? Use two details and examples from the article to explain why you believe this to be the main idea.

2.Evaluate Sally’s experimental design. Identify two things Sally could have done differently to make her results more valid. Give reasoning for your suggestions.

Application Activity


Who wants to ask the

first question?

Qs and As


Out of all the things you learned about OCRs, what point stands out as being the “shining star”?

Reflection


Practice ProfileMissouri Collaborative Work Practice Profile

Foundations present in the implementation of each essential function: Commitment to the success of all students and to improving the quality of instruction. Common Formati ve Assessment

Essential Functions Exemplary Proficiency Ideal Implementation

Proficient

Close to Proficient

(Skill is emerging, but not yet to ideal

proficiency. Coaching is

recommended.)

Far from Proficient (Follow-up

professional development and coaching

is critical.)

Evidence

1

Educators develop clear and meaningful learning goals to guide instruction and student learning.

All of the following criteria are met. Learning goal is clearly connected to a big idea/essential

learning in the domain Learning goal develops deep understanding of

underlying concepts and/or acquisition of skills Learning goal clearly engages higher order thinking

processes Learning goal is clearly manageable and can be

accomplished in the course of a lesson or unit (may be several periods)

Learning target is clearly explained to students Connections between current learning goal and prior

learning are clearly made

At least 6 of the criteria are met.


Less than 4 of the criteria are met.

Common Formative Assessment Development & Implementation Template.

2

Educators establish clear and measureable student success criteria in a rubric, scoring guide, or checklist.

All of the following criteria are met. Success criteria are clearly and effectively aligned to

learning goals Success criteria clearly and effectively relate to what

students will say, do, make or write to show evidence of learning

Success criteria clearly and effectively reflect ways for students to indicate their current status relative to the learning goals

Success criteria are communicated in language student can fully understand

Success criteria are frequently referred to during the learning process.



Less than 2 of the criteria are met.

Common Formative Assessment Development & Implementation Template.


Implementation Fidelity

Yes Partially No If partially or no, please

explain. 1. Common formative assessment is linked to selected

learning standards.

2. Learning goal engages higher order thinking processes.

3. Learning goal can be accomplished in the course of a unit.

4. Learning target is written in language that students can clearly understand.

5. Learning target is clearly explained to students. 6. Success criteria are written in language that

students can clearly understand in a rubric or checklist.

7. Students receive feedback based on learning goal and their assessment results.

8. The quality of assessment items for measuring mastery is reviewed and items revised as needed.

Total

Writing Constructed Response Items

Documents

Transcript of Writing Constructed Response Items