Item Development Presentation

1

June 11, 2012

Item Developer’s TasksTo develop high-quality, standards-based items

that will adhere to the Florida Department of Education Item Specification and will be used on three formative district test forms to evaluate students’ strengths and weaknesses on the Next Generation Sunshine State Standards (NGSSS) in Social Studies

2

3

DOK I

DOK II

DOK III

DOK Sample Question 1

4

What was the immediate cause of the outbreak of World War I (1914)?

A. Formation of secret alliances.B. Competition over Asian and African colonies.C. Accumulation of arms and weapons.D. Assassination of Archduke Ferdinand of Austria.

5

Use the table to answer the question. 1860

North South

Industrial Agricultural

More cities Mainly rural

Many factories Few Factories

Slavery widely opposed Slavery widely supported

A. The South had more large cities than the North.B. Sectionalism was based on social and economic differences.C. The South was well prepared to win a war against the North.D. Agriculture was no longer important to the United States economy.

Which conclusion related to the Civil War can be drawn from the information in this chart?

DOK Sample Question 2

6

DOK Sample Question 3

Which late 19th-century event is portrayed in this cartoon?

A.economic tariffs on European goodsB.the incident leading to war with SpainC.colonial expansion into Latin AmericaD.the capture of the Philippine Islands

Cognitive Complexity Levels

7

Low Complexity Low-complexity items require students to recall, observe, question or represent basic facts. For a low complexity item, the student would be expected to demonstrate simple skills or abilities. A low-complexity item requires only basic understanding – often verbatim recall or simple understanding of a term or process.

Moderate Complexity Moderate-complexity items require two steps: identification and subsequent processing of that information. Students are expected to make inferences and may encounter items that include words such as classify, organize and compare. Depending on the objective a particular moderate-level item, students may also be required to explain, describe, or interpret.

High Complexity High-complexity items make heavy demands on student thinking. Students may be asked to explain, generalize, or make multiple connections. High complexity items require several steps involving abstract reasoning and planning.

Recall and Reproduction

Skills & Concepts

Strategic/ Extended Thinking

8

Terminology Item - the entire multiple choice question. Stem – the first, sentence-like portion of the

multiple-choice question. Alternatives – all possible multiple-choice

response options. Key response – correct answer. Distracters - the wrong answers. They are

called distracters because they should be written to be plausible and, ideally, reflect common misconceptions, thus distracting students with only superfluous knowledge of the tested material.

9

ExampleWhat is the process of elimination in a multiple choice question? (stem)

  A. Skipping the entire question (distracter)*B. Eliminating the wrong answers (key)  C. Eliminating all extreme answers

(distracter)  D. Just guessing (distracter) 

10

Example (poor)In World War II the Allied strategy, agreed upon

by the U.S. and Great Britain was toA. concentrate on defending Japan before turning to

Germany.B. fight only against Japan, leaving the Russians to

fight Germany alone.C. take a passive role and limit operations to

reacting to Axis moves.D.concentrate on defeating Germany first before

turning on Japan.11

Example (poor)Use a direct question (not an incomplete sentence) in the stemIn World War II the Allied strategy, agreed upon

by the U.S. and Great Britain was toA. concentrate on defending Japan before turning to

Germany.B. fight only against Japan, leaving the Russians to

fight Germany alone.C. take a passive role and limit operations to

reacting to Axis moves.D.concentrate on defeating Germany first before

turning on Japan.12

Example (better)What was the Allied strategy, agreed upon by

the U.S. and Great Britain, in World War II?A. Concentrate on defending Japan before turning to

GermanyB. Fight only against Japan, leaving the Russians to

fight Germany aloneC. Take a passive role and limit operations to

reacting to Axis movesD.Concentrate on defeating Germany first before

turning on Japan13

Example (poor)Which of the following events abolished the

institution of slavery in the United States?

A.Ratification of the 13th amendment in 1865B.Creation of the Freedmen’s Bureau of 1865C.Compromise of 1850D.Emancipation Proclamation of 1863

14

ExampleArrange numerical alternatives (such as dates) in ascending order.

Which of the following events abolished the institution

of slavery in the United States?

A.Ratification of the 13th amendment in 1865B.Creation of the Freedmen’s Bureau of 1865C.Compromise of 1850D.Emancipation Proclamation of 1863

15

Example (better)

Which of the following events formally abolished the

institution of slavery in the United States?

A.Compromise of 1850B.Emancipation Proclamation of 1863C.Creation of the Freedmen’s Bureau of 1865D.Ratification of the 13th amendment in 1865

16

Example (poor)

Which of the following groups of citizens of the United States were not allowed to cast their votes in elections for President of the United States until the 23rd Amendment was passed?

17

ExampleDo not use negative words, such as “not” or “except” in a stem.

Which of the following groups of citizens of the United States were not allowed to cast their votes in elections for President of the United States until the 23rd Amendment was passed?

18

Example (better)

Passage of the 23rd Amendment gave presidential election voting privilege to which group of citizens?

19

Example (poor)

Which of the following is an example of a success of the

“right-to-life” movement ?

A.It created a constitutional amendment banning abortions.

B.Achieving restrictions on abortionsC.Getting the Stewart v. Freeman case to the

Supreme CourtD.All of these choices are correct.

20

ExampleDo not use “all of the above” or “none of the above”

Which of the following is an example of a success of

the “right-to-life” movement ?

A.It created a constitutional amendment banning abortions.

B.Achieving restrictions on abortionsC.Getting the Stewart v. Freeman case to the

Supreme CourtD.All of these choices are correct.

21

Example (better)

Which of the following is an example of a success of the “right-to-life” movement ?

A.Creating a constitutional amendment banning abortions

B.Achieving restrictions on abortions C.Getting the Stewart v. Freeman case to the

Supreme Court D.Reversing Roe v. Wade

22

Writing Good ItemsPart I. Emphasis on the Stem

Write questions that examine important concepts; avoid questions addressing minute details.

Make the stem clear and unambiguous.Use simple grammar and sentence structure.Do not use vocabulary more advanced than

required.Include in the stem any word(s) that

otherwise would be repeated in each alternative.

23

Example (poor)How many grievances against the king

were named in the Declaration of

Independence?

24

Example How many grievances against the king were named in the Declaration of Independence?

Write questions that examine important concepts; avoid questions addressing minute details.

25

Example (better)What major grievance against the king

wasnamed in the Declaration of

Independence?

26

Example (poor)As the level of fertility approaches its

nadir, what is the most likely ramification for

the citizenry of a developing nation?

27

Example As the level of fertility approaches its nadir, what is the most likely ramification for the citizenry of a developing nation?

Make the stem clear and unambiguous.

Do not use vocabulary more advanced than required.

28

Example (better)

What would be the likely result of a decline in

fertility in a developing nation?

29

Example (poor)I believe that it must be the policy of the

United States to support free peoples who are resisting attempted subjugation by armed minorities or outside pressures.

This statement was usedA.to support the Baruch Plan. B.to support the Monroe Doctrine. C.to support the Good Neighbor Policy. D.to support the Truman Doctrine.

30

ExampleI believe that it must be the policy of the United

States to support free peoples who are resisting attempted subjugation by armed minorities or outside pressures.

This statement was usedA.to support the Baruch Plan. B.to support the Monroe Doctrine. C.to support the Good Neighbor Policy. D.to support the Truman Doctrine.

Include in the stem any word(s) that otherwise would be repeated in each alternative.

31

Example (better)I believe that it must be the policy of the United

States to support free peoples who are resisting attempted subjugation by armed minorities or outside pressures.

This statement was used to support which of the following?A.The Baruch PlanB.The Monroe DoctrineC.The Good Neighbor PolicyD.The Truman Doctrine

32

Writing Good ItemsPart II. Emphasis on Alternatives

Do not let any portion of the stem provide unintended clues.

Do not let the length of alternatives provide unintended clues.

Make all distracters plausible (use common errors).Make the alternatives mutually exclusive.Keep the grammar of each alternative consistent.Be careful to keep items independent, so that the

answer to one question does not provide a clue to solving another.

Make the position of the key response (correct answer) random.

33

Example (poor)What was one of the most important domestic economic results of World War II?

A. Complete employment for minority AmericansB. The end of discrimination for African

AmericansC. Change in social status for womenD. The swift ending of the Great Depression

34

ExampleWhat was one of the most important domestic economic results of World War II?

A. Complete employment for minority AmericansB. The end of discrimination for African

AmericansC. Change in social status for womenD. The swift ending of the Great Depression

Do not let any portion of the stem provide unintended clues.

35

Example (better)What was one of the most important domestic results of World War II?

A. Complete employment for minority AmericansB. The end of discrimination for African

AmericansC. Change in social status for womenD. The swift ending of the Great Depression

36

Example (poor)“Reaganomics” was based upon which of the following views?

A. The government must cut taxes to stimulate investment, which would, in turn, increase employment and tax

revenue.B. The government must commit to large expenditures.C. The government must institute wage and price controls.D. The government must increase taxes on corporations and individuals.

37

Example“Reaganomics” was based upon which of the following views?

A. The government must cut taxes to stimulate investment, which would, in turn, increase employment and tax

revenue.B. The government must commit to large expenditures.C. The government must institute wage and price controls.D. The government must increase taxes on corporations and individuals.

Do not let the length of alternatives provide unintended clues.

38

Example (better)“Reaganomics” was based upon which of the following views?

A. The government must cut taxes to stimulate investment

into the economy.B. The government must commit to large expenditures.C. The government must institute wage and price

controls.D. The government must increase taxes on

corporations and individuals.

39

Example (poor)

The English based their claim to territorial rights in North America on the expeditions of which of their explorers?

A. Henry HudsonB. Ferdinand MagellanC. Sir Walter RaleighD. Amerigo Vespucci

40

Example

The English based their claim to territorial rights in North America on the expeditions of which of their explorers?

A. Henry HudsonB. Ferdinand MagellanC. Sir Walter RaleighD. Amerigo Vespucci

Make all distracters plausible (use common errors).

41

Example (better)

The English based their claim to territorial rights in North America on the expeditions of which of their explorers?

A. Henry HudsonB. Francis DrakeC. Sir Walter RaleighD. John Rolfe

42

Example (poor)Which of the following was a circumstance of the election of 1789?

A. Southern Federalists supported Thomas Jefferson.

B. The Federalist Party did not unite behind Adams.

C. George Washington took an active role.D. The Federalist party divided over the support

of Adams.

43

ExampleWhich of the following was a circumstance of the election of 1789?

A. Southern Federalists supported Thomas Jefferson.B. The Federalist Party did not unite behind Adams.C. George Washington took an active role.D. The Federalist party divided over the support of

Adams.

Make the alternatives mutually exclusive.

44

Example (better)Which of the following was a circumstance of the election of 1789?

A. Southern Federalists supported Thomas Jefferson.

B. The Federalist Party united behind Adams.C. George Washington took an active role.D. The Federalist party divided over the support

of Adams.

45

Example (poor)Why did President Kennedy decide to remove Diem from the presidency of South Vietnam?

A. Diem launched attacks on the country's Buddhists.

B. Massacring a large number of Viet CongC. Refusing to allow American soldiers to engage

in combatD. Shooting his own brother for treason

46

ExampleWhy did President Kennedy decide to remove Diem from the presidency of South Vietnam?A. Diem launched attacks on the country's

Buddhists.B. Massacring a large number of Viet CongC. Refusing to allow American soldiers to engage

in combatD. Shooting his own brother for treasonKeep the grammar of each alternative consistent.

47

Example (better)Why did President Kennedy decide to remove Diem from the presidency of South Vietnam?

A. Diem launched attacks on the country's Buddhists.

B. Diem massacred a large number of Viet Cong.C. Diem refused to allow American soldiers to

engage in combat.D. Diem had his own brother shot for treason.

48

Example Which of the following events occurred as a direct result of the Townshend Acts?

A. The Committees of Correspondence were formed.

B. The First Continental Congress was convened.

C. The principle of "salutary neglect" ended.

D. The colonists resented mercantilist economic policy.

What was the purpose of the colonial Committees of Correspondence created as a result of the Townshend Acts?

A. Keep colonial intellectuals in contact with each other

B. Publicize grievances against England

C. Improve the writing skills of young gentlemen

D. Correspond with English radicals who supported the American causeBe careful to keep items independent, so that the answer

to one question does not provide a clue to solving another.

49

ExampleWhich of the following events occurred as a direct result of the Townshend Acts?

A. The Committees of Correspondence were formed.

B. The First Continental Congress was convened.

C. The principle of "salutary neglect" ended.

D. The colonists resented mercantilist economic policy.

What was the purpose of the colonial Committees of Correspondence created in response to the Townshend Acts?

A. Keep colonial intellectuals in contact with each other

B. Publicize grievances against England

C. Improve the writing skills of young gentlemen

D. Correspond with English radicals who supported

the American cause

50

51

Practice ExerciseAs a group you will develop one multiple choice question on the following benchmark associated with your course.

Use the template to guide you in creating the question..