FCAT 2.0 and FL EOC Assessments Test Item Specifications

Nancy E. Brito, NBCT, Accountability/Instructional Specialist Division of Quality Assurance

Educational Data Warehouse, Accountability, and School Improvement 1

Let’s Begin with The End in Mind

2

Vision

envisions a dynamic collaborative multicultural

community where education and lifelong learning are

valued and supported, and all learners reach their highest

potential and succeed in the global economy.

The School District of Palm Beach County

Mission

The School District of Palm Beach is committed to providing a world-class education with excellence

and equity to empower each student to reach his or her highest potential with the most effective staff to foster the knowledge, skills, and ethics required for

responsible and productive careers.

Graduation Requirement

5

What is the purpose of the Test Item

Specifications? 6

The Test Item Specifications is a resource that defines the

content and format of the test and test items for item writers and reviewers.

7

The Test Item Specifications is a resource that defines the

content and format of the test and test items for item writers and reviewers.

indicates the alignment of test items with the NGSSS.

8

The Test Item Specifications is a resource that defines the

content and format of the test and test items for item writers and reviewers.

indicates the alignment of test items with the NGSSS.

serves to provide all stakeholders with information about the scope and function of the FCAT 2.0/ EOC assessments.

9

The Test Item Specifications is a resource that defines the

content and format of the test and test items for item writers and reviewers.

indicates the alignment of test items with the NGSSS.

serves to provide all stakeholders with information about the scope and function of the FCAT 2.0/ EOC assessments.

provides general and grade-specific guidelines for the development of all test items used in the FCAT 2.0/EOC Assessments.

10

The Test Item Specifications is a resource that defines the

content and format of the test and test items for item writers and reviewers.

indicates the alignment of test items with the NGSSS.

serves to provide all stakeholders with information about the scope and function of the FCAT 2.0/ EOC assessments.

provides general and grade-specific guidelines for the development of all test items used in the FCAT 2.0/EOC Assessments.

11

How should we use the Test Item Specifications?

12

How should we use the Test Item Specifications?

To ask questions during instruction that are of the same cognitive rigor as FCAT 2.0/EOC Assessments test items

13

How should we use the Test Item Specifications?

To ask questions during instruction that are of the same cognitive rigor as FCAT 2.0/EOC Assessments test items

To develop assessment questions that are of the same cognitive rigor as FCAT 2.0/EOC Assessments test items

14

How should we use the Test Item Specifications?

To analyze and evaluate assessment items that align to the NGSSS and adhere to the Test Item Specifications

15

On writing Assessment Items…

A test item must focus the attention of the examinee on the principle or construct upon which the item is based. Ideally, students who answer a test item incorrectly will do so because their mastery of the principle or construct in focus was inadequate or incomplete.

http://scoring.msu.edu/writitem.html 16

On writing Assessment Items…

Any characteristics of a test item which distract the examinee from the major point or focus of the item, reduces the effectiveness of that item. Any item answered correctly or incorrectly because of extraneous factors in the item, results in misleading feedback to both examinee and examiner.

http://scoring.msu.edu/writitem.html 17

A well designed, and aligned classroom assessment provides information on the strengths and challenges of students.

What do my students know? What are my students able to do?

How am I doing? 18

19

Let’s explore… The Test Item

Specifications

LOOK

20

Contents of the Item “Specs” FCAT 2.0

Reading (R) Grades 3-5 FCAT 2.0 Reading Grades 6-8 FCAT 2.0 Reading Grades 9-10 FCAT 2.0 Reading

Mathematics (M) Grades 3-5 FCAT 2.0 Mathematics Grades 6-8 FCAT 2.0 Mathematics

Science (S) Grade 5 FCAT 2.0 Science Grade 8 FCAT 2.0 Science

21

Contents of the Item “Specs” Florida End-of-Course (EOC)

Algebra 1 EOC Assessment (A) Biology 1 EOC Assessment (B)

Geometry EOC Assessment (G) Civics EOC Assessment (C)

U.S. History (U)

22

Introduction

Scope of Document General and grade-specific guidelines

and criteria Criteria used to develop items Cognitive complexity & item difficulty Test design Benchmark explanations, content limits,

item types, and attributes

23

Overall Considerations Describes the guidelines that apply to all test items.

Pages 2-3

Scenarios

24

Criteria for FCAT 2.0 Reading Text Pages 3-7

Types Sources Characteristic Content Modifications

Text Features Diversity Reading Level Length of Text

25

Item Style and Format General Guidelines Scope of Items Universal Design Context Dependent

Multiple Choice Gridded Response Fill-in Response Use of Graphics

Criteria for FCAT 2.0 and

EOC Assessment Test Items R – pages 10-11

M – pages 4-9 S – pages 5-9 B – pages 4-8 C – pages 3-6 U – pages 3-6

26

Use of Graphics in Reading Page 6

Should be included whenever possible May include photographs, website features,

illustrations, maps, charts, graphs, advertisements, and schedules

Should include appropriate labels, legends, keys, and/or captions on maps, graphs, tables, text boxes, and other graphic stimuli

Should add to understanding of text Should reflect multicultural diversity, and

avoid gender stereotyping 27

Use of Graphics in Mathematics Grades 3-8, Algebra 1, and Geometry

Page 4 Used to provide necessary and

supplemental information Levels of graphics are dependent upon

benchmarks assessed Used as appropriate to benchmark All artwork must be high quality Clip art is not acceptable under any

conditions

28

Use of Graphics in Science Grades 3-8 and Biology Page 5

Always preceded by text Should be titled, and titles should be in

all caps, boldfaced, and centered May have titles placed above or below

the illustrations or pictures. Used to provide necessary and

supplemental information

29

Use of Graphics in Civics and U. S. History Page 3

Provide necessary and supplemental information

May include political cartoons, maps, photographs, diagrams, illustrations, charts, or tables

30

Item Style and Format • General Guidelines

R - pages 10-11 M - pages 4-9 S – pages 5-9 A – pages 4-9 B - pages 4-8 G - pages 4-9 C – pages 3-4 U – pages 3-4

31

Item Style and Format • Item Type Specific Guidelines

MC

GR

FR

R - pages 10-11 M - pages 4-9 S – pages 5-9 A – pages 4-9 B - pages 4-8 G - pages 4-9 C – pages 3-4 U – pages 3-4

32

R – pages 12-16

M – pages 10-15

S – pages 10-16

A – pages 10-16

Item Difficulty, Cognitive Complexity Classification & Universal Design

B – pages 09-16

G – pages 10-16

C – pages 7-13

U – pages 7-13

33

Item Difficulty

34

Cognitive Complexity

LOW Items are based on recall

and recognition of previously learned

concepts and principles

MODERATE Items require the use of

informal methods in reasoning and problem-solving strategies from

various domains

HIGH Items require the ability to solve non-routine or unique problems using

abstract reasoning, analysis, judgment, or

creative thought

• Considers the cognitive demand of the test item

35

Examples of Reading

Activities -

Cognitive Classification

Levels

Page 15

36

Examples of Mathematics

Activities -

Cognitive Classification

Levels

Page 14

37

Examples of Science

Activities -

Cognitive Classification

Levels

Grades 5, 8 Page 15

Biology Page 14

38

Examples of Civics

Activities -

Cognitive Classification

Levels

Page 12

39

Examples of U. S. History

Activities -

Cognitive Classification

Levels

Page 12

40

Cognitive Complexity Classification Reading Page 16

41

Cognitive Complexity Classification Mathematics

Page 15

42

Cognitive Complexity Classification Science S - Page 16

B - Page 15

43

Cognitive Complexity Classification Civics and U. S. History C – page 12

U – page 12

44

Examples of Complexity

Maria’s dog Spot almost knocked her down as she entered the door. Maria quickly noticed that all bowls were empty and that she’d forgotten to leave food out. Spot could hardly wait for Maria to fill the bowls as he hurriedly ate. Later when Maria returned to the kitchen, she couldn’t help but notice the smile on Spot’s face as he slept.

45

Examples of Complexity

Low- What was the name of Maria’s dog? (Read the Lines)

Moderate- What was Spot’s problem?

(Read Between the Lines)

High- Why was Spot MOST LIKELY smiling in his sleep?

(Read beyond the Lines)

46

Reduction of wordiness Avoidance of ambiguity Selection of reader-friendly

construction and terminology Consistently applied concept

names and graphic conventions

Universal Design

47

R – page 18

M – page 18

S – page 18

A – page 18

Benchmark Classification System

B – page 17

G – page 18

C – page 15

U – page 15

48

Benchmark Classification System Page 18

49

Benchmark Classification System Page 18

50

Benchmark Classification System Page 17

51

Benchmark Classification System Page 15

52

Benchmark Classification System Page 15

53

R – pages 22

M – pages 53

S – pages 34

A – pages 53

Benchmark Specifications and Sample Items

B – pages 37

G – page 53

C – page 17

U – page 17

54

• Strand/Reporting Category or Body of Knowledge

• Standard

• Big Idea/Supporting Idea

• Benchmark

• Item Type

• Sample Items

• Benchmark Clarification

• Content Focus

• Content Limits

• Text Attributes

• Stimulus Attributes

• Distractor/Response Attributes

• Prior Knowledge

Benchmark Specifications

55

Definitions of Benchmark Specifications

R - page 19 M – pages 19-20 S – pages 19-20 A – pages 19-20 B - page 18 G - pages 19 C – page 16 U – pages 16-17

56

Reading Benchmark Specifications

57

Benchmarks Contain Multiple Facets

58

Content Assessed - Appendix B Geometry

59

Content Assessed - Appendix B U. S. History

60

Content Assessed - Appendix B Reading

61

Example Individual Benchmark Specification for SC.5.P.10.1

62

Example Individual Benchmark Specification for SC.5.P.10.1

63

Example Individual Benchmark Specification for SC.5.P.10.1

64

Example Individual Benchmark Specification for SC.5.P.10.1

65

Reporting Categories

R – Appendix C

M – Appendix C

A – Appendix C

G – Appendix C

C – Appendix D

U – Appendix D 66

Reporting Categories: Reading

67

Reporting Categories: Mathematics

68

Reporting Categories: Science

69

Reporting Categories: U.S. History

70

Reporting Category Standard Percentage

Late Nineteenth and Early Twentieth Century (1860-1910)

Standard 2: Civil War and Reconstruction 15%

Standard 3: Industrial Revolution 17%

Global Military, Political, and Economic Challenges (18490-1940)

Standard 4: World Affairs through WWI 17%

Standard 5: Roaring Twenties and the Great Depression 17%

The United States and the Defense of the International Peace (194040-2012)

Standard 6: World War II and Post-World War II 17%

Standard 7: Modern United States: Global Leadership and Domestic Issues 15%

Reporting Categories: Civics

71

Reporting Category Percentage Origins and Purposes of Law Government 25%

Roles, Rights, and Responsibilities of Citizens 25%

Government Policies and Political Processes 25%

Organization and Function of Government 25%

Glossary

R – Appendix D

M – Appendix D

S – Appendix C

A – Appendix D

B – Appendix C

G – Appendix D

C – Appendix C

U – Appendix C

72

Test Design Summary

R – Appendix F

M – Appendix F

S – Appendix E

A – Appendix F

B – Appendix E

G – Appendix F

C - Appendix D

U - Appendix D

73

Test Design Summary Item Types and Numbers

74

Test Design Summary Duration of Test in Minutes

75

Test Design Summary Reading Passage Types and

Percentages

76

Test Design Summary EOC Course Codes

77

Other Important Information

R – Appendix G: Passages

M – Appendix G: Reference Sheets

S – Appendix D: Periodic Table

C – Appendix E: Course Description

U – Appendix E: Course Description

78

Example: Appendix G

79

Example: Appendix G

80

Example: Appendix D

81

Example: Appendix E

82

Where can educators find the FCAT 2.0 and EOC Test Item

Specifications ?

http://fcat.fldoe.org/fcat2/itemspecs.asp

http://fcat.fldoe.org/eoc/itemspecs.asp 83

Need Assistance? Contact us!

Dr. Carla Kendall Online Assessment Manager/ Mathematics Test Developer

(561) 969-5822, PX45822

Senella Johnson , ED. S. Reading /Writing Test Development Specialist

(561) 969-5842, PX45842

Nancy E. Brito, NBCT Accountability/ Instructional Specialist

(561) 357-7521, PX47521

Web Site:

FLDOE SharePoint:

84