TEST SPECIFICATIONS and BLUEPRINTS...include the ability of liquids to take the shape of their...

SCIENCETEST SPECIFICATIONS

25

and BLUEPRINTS

2008-2009

BenchmarkGrade

It is the policy of the State Board of Education and a priority of the Oregon Department of Education that there will be no discrimination or harassment on the grounds of race, color, sex, marital status, religion, national origin, age or handicap in any educational programs, activities, or employment. Persons having questions about equal opportunity and nondiscrimination should contact the State Superintendent of Public Instruction at the Oregon Department of Education.

Developed by the Office of Assessment and Information Services Oregon Department of Education

255 Capitol Street NE Salem, Oregon 97310-0203

(503) 947-5600

Susan Castillo State Superintendent of Public Instruction

Ken Hermens Language Arts Assessment Specialist

Doug Kosty Assistant Superintendent

Leslie Phillips Science, and Social Sciences Assessment Specialist

Tony Alpert Director, Assessment and Evaluation

James Leigh Mathematics Assessment Specialist

Steve Slater Manager, Scoring, Psychometrics and Validity

Dianna Carrizales Extended Assessment Specialist

Kathleen Vanderwall Manager, Test Design and Administration

Sheila Somerville Electronic Publishing Specialist

All or any part of this document may be photocopied for educational purposes without permission from the Oregon Department of Education and distributed for the cost of reproduction.

Science Test Specifications Oregon Department of Education and Test Blueprints Office of Assessment and Information Services

1

Science Test Specifications Background

This document explains the Oregon Department of Education’s statewide assessment program called OAKS (Oregon Assessment of Knowledge and Skills) and the specifications used when the state tests are designed. Test specifications such as these are used to establish the guidelines by which test content may be selected and test items written. They lead to a “test blueprint” that lays out for the test item writers, typically Oregon teachers, item format and number of questions to be written in each category.

Oregon educators further contribute to the test development and alignment process by serving on advisory committees called Content and Assessment Panels. Stakeholders in these committees are involved in each phase of the development of these specifications to assure that they accurately and clearly explain the overall design of the test and describe the specific content that might appear on the test to measure the knowledge and skills described in the content standards.

Oregon’s knowledge and skills test questions use a multiple choice format, with each item having a single correct answer and three incorrect answers. A computer scans the “fill-in-the-bubble” answer sheets or electronically collects scores. The results are scored against the answer key to produce a raw score. The raw score is converted to a scale score called a Rasch unit or RIT score. Students receive a scale score based on the number of questions answered correctly compared to the total number of questions on the form—taking into account the difficulty of the questions. Students are not penalized for guessing.

The content of these specifications reflects the skill expectations outlined in the Content Standards adopted by the State Board of Education for implementation beginning in the 2002-2003 school year. These standards were developed, in part, to correlate to the

skills assessed on the science portion of the National Assessment of Educational Progress and align with the National Science Standards. As a result, Oregon uses similar terminology in its descriptions of the science score reporting categories and subcategories (listed later in this document).

Statewide and Local Assessments

Statewide assessments are multiple choice tests of knowledge and skills that are developed and scored by the state. Local assessments include performance assessments that may be scored using statewide scoring guides that are administered and scored at the local level (see appendix). Local assessments are not included in state accountability reports, e.g. AYP reports.

Paper/Pencil Administration On Oregon’s science knowledge and skills paper/pencil tests there is one wide range test given at:

• grade 5-Benchmark 2 (content instructed at grades 4 and 5), • grade 8-Benchmark 3 (instructed at grades 6, 7 and 8), and • grade 10-High School level (instructed at grades 9 and 10)

Paper and pencil forms are typically administered only to students or schools qualifying for a waiver of the electronically administered state tests. The paper and pencil form consists of a single form and opportunity and this form is used for special versions including Braille, Large-print, Spanish/English side-by-side and Russian/English side-side forms.

Electronic Administration On the science knowledge and skills OAKS online tests, there are multiple opportunities to participate in fully-adaptive testing.

Science, Benchmark 2/Grade 5


2

In this format, the accuracy of the student responses to questions determines the next item or set of items the student will see. Students are allowed to preview test questions if a set of questions link to a specific graphic or stimulus. Having the tests fully adaptive allows for more precision in measurement and less frustration for the students.

Electronic administration of the science test for each grade tested includes up to three test opportunities in English or English-Spanish formats. An online practice test of sample items is available for students who may need practice using a scrollbar to view the full length of stimuli presented with item sets or modules.



3

Criteria for Science Test Questions Various types of science questions will be included in the test. Selections will align with three Score Reporting Categories (SRC):

- Physical Science (SRC 2) Understand structures and properties of matter and changes that occur in the physical world;

- Life Science (SRC 3) Understand structure, functions, and interactions of living organisms and the environment; and

- Earth and Space Science (SRC 4) Understand physical properties of the Earth, how those properties change and Earth’s relationship to other celestial bodies.

The Test Items

• Each test item will measure only one Score Reporting Category. The number of items in a test form will measure each of the three reporting categories equally (33%).

• Each Score Reporting Category will have items with a range of difficulty levels. This range of difficulty will be approximately the same across reporting categories.

• Test items will be in the form of questions or sentences that require completion.

• Test items may be worded in the negative (“Which of these is NOT . . . “), but this structure will be used rarely and only when it offers substantial advantages for the item construction.

• Each item will have four answer choices. Answer choices on paper tests will be arranged one of three ways beneath the test item: vertically, horizontally, or in two columns (i.e., A and B in the left column, C and D in the right column). On TESA, answer choices are listed below the stem or to the right of a stimulus

• Neither “None of the above” nor “All of the above” will be used as an answer choice. “There is not enough information to tell” is an allowed answer choice, but infrequently used.

• Test items are designed to be appropriate for students in the assigned benchmark in terms of reading level, interests, and experience.

• Test items may contain extraneous information.

• Students will be told in the test directions to choose the best answer from among the choices.

• Test items will be stated in the clearest manner possible.

• Test items will be free of age, gender, ethnic, religious or disability stereotypes or bias.

• Shading will be minimized. It will be used only to make a figure’s size, shape or dimensions clear, and not solely for artistic effect.

• Side-by-side Spanish and Russian test items are used on paper/pencil tests; stacked Spanish/English items are used on electronic tests.



4

Criteria for Science Modules

• A portion of each test will be incorporated into modules. A module is defined as a stimulus containing scientific information, accompanied by two to four knowledge and skills questions or items. The stimulus for each module will vary in length, format and character. It could be one or a combination of any of the following: data table, diagram, chart, drawing, photo or reading text.

• The remaining items on each test will be stand-alone knowledge and skills items. Students will not refer to additional material when answering those questions.

• Each Knowledge and Skills test item within a module will measure only one Score Reporting Category (SRC)-Physical Science, Life Science, or Earth and Space Science. Within a module, though, items may measure more that one SRC.

• Each stimulus will be free of age, gender, and other bias, as evaluated by the ODE Assessment Sensitivity Panel.

• Although the stimulus for each module will provide scientific information, students will be required to draw on prior knowledge to answer many of the items. In other words, there may not be sufficient information in the stimulus to answer all the test questions associated with it. The stimulus may simply provide a context for some test questions.

• Each module will be preceded by a bold-faced title on the paper tests, which will serve to set the accompanying items apart from other modules in the test form.

• Each stimulus and their corresponding items and answer choices will appear on the same page or facing page of the test booklet.

The following pages contain a more detailed examination of the test content for science. The first column lists the content and benchmark standards assessed for that particular score reporting category. The second column lists the specific eligible content, testable content, and gives a more detailed explanation of how the standard would be assessed. Finally, the third column gives some sample items that are very similar to the type of questions asked on a test related to that eligible content.




5

PHYSICAL SCIENCE Score Reporting Category 2MATTER

Common Curriculum Goal: Understand structure and properties of matter.

Content Standard Understand structure and properties of matter.

Benchmark 2 Standard Identify substances as they exist in different states of matter.

National Science Education Standards Pgs. 123-127

Eligible Content Distinguish among solids, liquids, and gases. 2.1.51.1

Identify unique properties of each state of matter. 2.1.51.2

Explanation

Matter can exist in different states – solid, liquid and gas. There are differences and similarities among the three states of matter. These include the ability of liquids to take the shape of their container, of gases to fill their container and to exert pressure on the walls of that container, and of solids to maintain their original shape.

Sample Question

“Takes up space” describes

A. solids only.

B. liquids only.

C. gases only.

D. solids, liquids and gases. *



6

PHYSICAL SCIENCE Score Reporting Category 2 MATTER

Common Curriculum Goal: Understand chemical and physical changes.

Content Standard

Describe and analyze chemical and physical changes.

Benchmark 2 Standard Describe the ability of matter to change state by heating and cooling.


Eligible Content

Recognize that heating and cooling cause changes in states of matter. 2.2.51.1

Identify changes in states of matter seen in the environment. 2.2.51.2

Explanation

Substances such as water can change state, but are still the same substance. Heating and cooling causes changes in the properties of materials, and many kinds of changes occur faster under hotter conditions. Energy, in this case thermal energy, has effects that go far beyond simply changing the temperature of substances.

Sample Question

When water is heated, the temperature at which bubbles of gas form is called

A. the freezing point.

B. the melting point.

C. the boiling point. *

D. the dew point.



7

PHYSICAL SCIENCE Score Reporting Category 2FORCE

Common Curriculum Goal: Understand fundamental forces, their forms, and their effects on motion.

Content Standard

Describe fundamental forces and the motions resulting from them.

Benchmark 2 Standard Describe and compare the motion of objects.


Eligible Content

Recognize and describe the motion of an object in terms of one or more forces acting on it. 2.3.51.1

Explanation Changes in speed or direction of the motion of any object are caused by forces. The greater the force, the greater the change in motion. Objects of greater mass require more force to change their velocity.

Sample Question

Predict the outcome of rolling a 5-pound iron ball on these surfaces:

- a flat grassy path, - a flat gravel road, and - a flat cement road.

A. The ball will roll farthest on the cement. *

B. The ball will roll farthest on the gravel.

C. The ball will roll farthest on the grassy path.

D. The ball, if rolled with equal force, will roll equal distances on each of the surfaces.



8

PHYSICAL SCIENCE Score Reporting Category 2 FORCE

Common Curriculum Goal: Understand fundamental forces, their forms, and their effects on motion.

Content Standard

Describe fundamental forces and the motions resulting from them.

Benchmark 2 Standard Identify examples of magnetism and gravity exerting force on an object.


Eligible Content

Recognize that magnets attract and repel each other and other materials. 2.3.52.1

Recognize that things on or near Earth are pulled toward it by Earth’s gravity. 2.3.52.2

Explanation

Forces can act at a distance. Magnetic force creates attraction or repulsion between magnetically charged substances. This magnetic force diminishes with distance. Gravitational force attracts objects and, like magnetism, also diminishes with distance.

Sample Question

An inflated balloon would be attracted by

A. gravity. *

B. magnetism.

C. both gravity and magnetism.

D. neither gravity nor magnetism.



9

PHYSICAL SCIENCE Score Reporting Category 2 ENERGY

Common Curriculum Goal: Understand energy, its transformations, and interactions with matter.

Content Standard

Explain and analyze the interaction of energy and matter.

Benchmark 2 Standard Identify forms of various types of energy and their effects on matter.

National Science Education Standards 123-127

Eligible Content

Identify various forms of energy including heat, light, sound, and electricity. 2.4.51.1

Explanation

Energy comes in many forms including heat, light, sound, and electricity. The sun is the main source of energy for life systems on Earth.

Sample Question

PAPER CLIP ELECTROMAGNETS

Rachel made four electromagnets by winding coils of copper wire around a nail. She connected each end of the wire to a battery to form an electromagnet which she used to pick up paper clips.

In this experiment, what kind of energy is changed directly into magnetic energy?

A. Heat energy B. Electrical energy * C. Chemical energy D. Light energy



10

PHYSICAL SCIENCE Score Reporting Category 2 ENERGY

Common Curriculum Goal: Understand energy, its transformations, and interactions with matter.

Content Standard

Explain and analyze the interaction of energy and matter.

Benchmark 2 Standard Describe examples of energy transfer.


Eligible Content

Identify the direction of heat transfer on a diagram showing objects at different temperatures. 2.4.52.1

Identify ways to produce heat including light, burning, electricity, friction, and as a byproduct of mechanical and electrical machines. 2.4.52.2

Identify examples of energy transfer in the environment. 2.4.52.3

Explanation

Heat can be produced by light, combustion, electricity, friction, and as a byproduct of machines. Heat can be transferred from warm areas to cool areas through conduction, convection, and radiation and will transfer until both areas achieve the same temperature.

Sample Question

To heat water using a renewable energy source, you would use

A. natural gas.

B. oil.

C. solar energy. *

D. coal.



11

LIFE SCIENCE Score Reporting Category 3 ORGANISMS

Common Curriculum Goal: Understand the characteristics, structure, and functions of organisms.

Content Standard

Describe the characteristics, structure, and functions of organisms.

Benchmark 2 Standard Group or classify organisms based on a variety of characteristics.


Eligible Content

Classify a variety of living things into groups using various characteristics. 3.1.51.1

Explanation

All organisms have characteristics, structures, and functions which allow them to be grouped and classified according to similarities and differences.

Sample Question

These animals can be classified together based on all of the following characteristics EXCEPT

A. cold-blooded animals.

B. vertebrates.

C. animals that live both in and out of water.

D. herbivores. *



12



Content Standard


Benchmark 2 Standard Describe the function of organ systems.


Eligible Content

Classify organs by the system to which they belong. 3.1.52.1

Explanation

Specialized cells perform specialized functions in multi cellular organisms. Groups of specialized cells cooperate to form a tissue, such as a muscle. Different tissues are in turn grouped together to form larger functional units, called organs. Each type of cell, tissue, and organ has distinct structure and set of functions that serve the organism as a whole.

Sample Question

The lung would best be described as an organ in which body system?

A. Circulatory

B. Skeletal

C. Excretory

D. Respiratory *



13



Content Standard


Benchmark 2 Standard Describe basic plant and animal structures and their functions.


Eligible Content

Associate specific structures with their functions in the survival of the organism. 3.1.53.1

Explanation

Each plant or animal has different structures that serve different functions in growth, survival, and reproduction. For example, humans have distinct body structures for walking, holding, seeing, and talking.

Sample Questions

The long neck of a giraffe best serves which purpose?

A. Protection from predators

B. Gathering food *

C. Increasing speed

D. Protection from heat

Feathers help penguins in the same way that fur helps bears, by making it easier for them to

A. walk.

B. swim.

C. find food.

D. keep warm. *



14

LIFE SCIENCE Score Reporting Category 3HEREDITY

Common Curriculum Goal: Understand the transmission of traits in living things.

Content Standard

Understand the transmission of traits in living things.

Benchmark 2 Standard Describe the life cycle of an organism.


Eligible Content

Describe the life cycle of common organisms. 3.2.51.1

Recognize that organisms are produced by living organisms of similar kind, and do not appear spontaneously from inanimate materials. 3.2.51.2

Explanation

Plants and animals have life cycles that, while they differ in the details, all include being born, developing into adults, reproducing and, eventually, dying. Some characteristics of plants and animals are inherited from parents; others result from an interaction with the environment and cannot be passed on to the next generation.

Sample Question

Some animals, such as caterpillars, go through changes in form as they become adults. This is called

A. larvae.

B. metamorphosis. *

C. molting.

D. heredity.



15

LIFE SCIENCE Score Reporting Category 3DIVERSITY/INTERDEPENDENCE

Common Curriculum Goal: Understand the relationships among living things and between living things and their environments.

Content Standard

Explain and analyze the interdependence of organisms in their natural environment.

Benchmark 2 Standard Describe the relationship between characteristics of specific habitats and the organisms that live there.


Eligible Content

Use drawings or models to represent a series of food chains for specific habitats. 3.3.51.1

Identify the producers, consumers, and decomposers in a given habitat. 3.3.51.2

Recognize how all animals depend upon plants whether or not they eat the plants directly. 3.3.51.3

Explain the relationship between animal behavior and species survival. 3.3.51.4

Describe the living and nonliving resources in a specific habitat and the adaptations of organisms to that habitat. 3.3.51.5

Explanation

Populations of organisms can be categorized by the function they serve in an ecosystem. Plants and some microorganism are producers – they make their own food. All animals, including humans, are consumers, which obtain food by eating other organisms. Decomposers, primarily bacteria and fungi, are consumers that use waste materials and dead organisms for food. Food webs identify the relationships among producers, consumers, and decomposers in an ecosystem.

Sample Question

According to the food web, what do small birds eat?

A. Grasses and rabbits.

B. Grasses and grasshoppers. *

C. Bacteria and fungi.

D. Hawks and foxes.



Content Standard

Explain and analyze the interdependence of organisms in their natural environment.

Benchmark 2 Standard Describe the relationship between characteristics of specific habitats and the organisms that live there.


Eligible Content

Use drawings or models to represent a series of food chains for specific habitats. 3.3.51.1

Identify the producers, consumers, and decomposers in a given habitat. 3.3.51.2

Recognize how all animals depend upon plants whether or not they eat the plants directly. 3.3.51.3

Explain the relationship between animal behavior and species survival. 3.3.51.4

Describe the living and nonliving resources in a specific habitat and the adaptations of organisms to that habitat. 3.3.51.5

Explanation

Populations of organisms can be categorized by the function they serve in an ecosystem. Plants and some microorganism are producers – they make their own food. All animals, including humans, are consumers, which obtain food by eating other organisms. Decomposers, primarily bacteria and fungi, are consumers that use waste materials and dead organisms for food. Food webs identify the relationships among producers, consumers, and decomposers in an ecosystem.

Sample Question

According to the food web, what do small birds eat?

Grasses and rabbits.

Grasses and grasshoppers. *

Bacteria and fungi.

Hawks and foxes.



16



Content Standard

Describe and analyze diversity of species, natural selection, and adaptation.

Benchmark 2 Standard Describe how adaptations help a species survive.


Eligible Content

Describe changes to the environment that have caused the population of some species to change. 3.4.51.1

Identify conditions that might cause a species to become endangered or extinct. 3.4.51.2

Explanation

Species acquire many of their unique characteristics through biological adaptation, which involves the selection of naturally occurring variations in populations. Biological adaptations include changes in structures, behaviors, and physiology that enhance survival and reproductive success in a particular environment.

Sample Question

Which of the following best describes a condition that might cause a bird to become extinct? A. Additional food sources

B. Additional predators*

C. Additional nesting sites

D. Additional rainfall



17

EARTH AND SPACE SCIENCE Score Reporting Category 4THE DYNAMIC EARTH

Common Curriculum Goal: Understand the properties and limited availability of the materials which make up the Earth.

Content Standard

Identify the structure of the Earth system and the availability and use of the materials that make up that system.

Benchmark 2 Standard Identify properties and uses of Earth materials.


Eligible Content

Recognize that Earth materials are used in different ways based on differences in their physical and chemical properties. 4.1.51.1

Recognize that soils vary in color, texture, components, reaction to water, and ability to support the growth of plants. 4.1.51.2

Recognize that the supply of many resources is limited, and that resources can be extended through recycling and decreased use. 4.1.51.3

Recognize that discarded products contribute to the problem of waste disposal. 4.1.51.4

Explanation

The materials which make up the Earth have many different physical and chemical properties. Materials such as soil vary in color, texture, components, reaction to water, and the ability to support the growth of plants. These differences in physical and chemical properties of materials make them suitable for various uses (e.g. building materials, fuels, mediums for plant growth). The supply of many of these materials is limited and their use can be extended through conservation and recycling. The consumption and disposal of these materials can contribute to environmental problems.

Sample Question

How can police detectives use the dirt on a suspect’s shoes as evidence that the suspect was in a certain location?

A. Shoes react differently to different types of soils.

B. Footprints are often left in the soft dirt.

C. Each shoe leaves a unique footprint.

D. Soils differ from place to place in color and texture. *



18


Common Curriculum Goal: Understand changes occurring within the lithosphere, hydrosphere, and atmosphere of the Earth.

Content Standard

Explain and analyze changes occurring within the lithosphere, hydrosphere, and atmosphere of the Earth.

Benchmark 2 Standard Describe patterns of seasonal weather.


Eligible Content

Describe weather in measurable quantities including temperature, wind direction, wind speed, and precipitation. 4.2.51.1

Interpret data over a period of time and use information to describe changes in weather from day to day, week to week, and season to season. 4.2.52.2

Explanation

Weather changes from day to day and over the seasons. Weather can be described by measurable quantities, such as temperature, wind direction and speed, and precipitation. These quantities can be analyzed over periods of time.

Sample Question

Suppose you recorded these times of sunrise.

May 1 5:02 May 9 4:54 May 17 4:42 May 25 4:37 June 1 4:32

Study the table above and predict approximately what time the sun rose on May 15.

A. 4:30

B. 4:40

C. 4:45*

D. 4:55



19


Common Curriculum Goal: Understand changes occurring within the lithosphere, hydrosphere, and atmosphere of the Earth.

Content Standard

Explain and analyze changes occurring within the lithosphere, hydrosphere, and atmosphere of the Earth.

Benchmark 2 Standard Identify causes of Earth surfaces changes.


Eligible Content

Identify effects of wind and water on Earth materials using appropriate models. 4.2.52.1 Identify effects of rapid changes on Earth’s surface features including earthquakes and volcanoes. 4.2.52.2

Explanation

The surface of the Earth changes. Some changes are due to slow destructive processes such as erosion and weathering. Some changes are due to rapid constructive and destructive processes such as landslides, volcanic eruptions, and earthquakes. These changes all affect the topography of the Earth’s surface.

Sample Question

Which of the following is least likely to be an effect of volcanic eruption?

A. Mudslides

B. Flooding

C. Lava or ash

D. Thunderstorms *



20

EARTH AND SPACE SCIENCE Score Reporting Category 4 THE EARTH IN SPACE

Common Curriculum Goal: Understand the Earth’s place in the solar system and the universe.

Content Standard

Explain relationships among the Earth, sun, moon, and the solar system.

Benchmark 2 Standard Describe the Earth’s places in the solar system and the patterns of movements of objects within the solar system using pictorial models.


Eligible Content

Describe Earth’s position and movement in the solar system. 4.3.51.1

Recognize that the rotation of the Earth on its axis every 24 hours produces the night-and-day cycle. 4.3.51.2

Explanation

The Earth is the third planet from the sun in a system that includes the moon, the sun, eight other planets and their moons, and smaller objects, such as asteroids and comets. Most objects in the solar system are in regular and predictable motion, such as the Earth’s elliptical orbit around the sun. Those motions explain such phenomena as the day, the year, phases of the moon, and eclipses.

Sample Questions

Suppose four spaceships are launched from Earth today. It would take longest for a spaceship to get to which location?

A. Mercury

B. Moon

C. Jupiter

D. Pluto *



21

Test Blueprint Introduction

The blueprints used to construct Knowledge and Skills Tests for Science prescribe the:

• Score Reporting Categories (SRC) included on each test,

• The cognitive demand and difficulty level of items as distributed on a test form,

• the number and percentages of test items from each SRC included on each test, and

• the total number and percentages of operational and field test items included for each test.

Teachers and other educators have historically played a vital role in the development of these specifications and blueprints by serving on Content and Assessment Panels and other review groups. These groups have advised the Department as content and benchmark standards have been developed, and have helped establish priorities on which standards to assess and the weighting of the strands within each content area assessment. Alignment of Test Items to Benchmark Standards

Test items are carefully aligned to benchmark standards at the appropriate grade level through a rigorous process at two points in the test item development process: 1. At item development workshops, item writers are provided

with adopted content standards to which they must write test items; during a peer review process, this alignment is verified by another benchmark level item developer.

2. Alignment of items to the benchmark standards is further verified during a review by members of a Content and Assessment Panel, who ensure items not only match the

benchmark standards, but also verify overall quality and appropriateness. Reviewers either accept items as a strong match to the targeted standards, edit items to achieve a strong match, or reject items which do not strongly match the standards.

The Appendix to this document includes additional evidence describing procedures ensuring alignment during item, development, including descriptions of Item Development and the Life of an Item.

Content Coverage

Prior to item writing activities, item databases are reviewed to determine the extent that the available items represent the emphasis and content in the standards. If any benchmark standards are underrepresented in the item pool, they are identified and targeted specifically for additional item development. This assures that every year, the proportion of items in the item pools is comparable in both emphasis and content to the content standards. The table on the following page describes the emphasis given to each content strand; this emphasis is reflected in both the item pools and administered tests (progressive and paper and pencil).

For electronic administration, all tests and the item pools from which they are constructed follow the weighting of each content strand as reflected in the table on page 15. Items within a strand, or SRC, are selected to provide a range of difficulty so that the progressive nature of the test is maintained as students of varied ability levels are presented with items most appropriate to their ability from that pool. Although a student



22

may not see an item addressing every one of the assessable standards in a single test event, the item pool contains multiple items for each content standard at a variety of difficulty levels. In addition, the adaptive algorithm specifically considers alignment criteria when drawing test items. As a result, we accomplish the dual purpose of creating a test form that is appropriately developed for each student and it meets the criteria set forth for alignment (e.g., balance of representation, depth of knowledge). For paper and pencil tests, the item set will include items aligned with as many of that benchmark level’s standards as possible. Within a three-year period, paper and pencil test forms will sample items from all adopted standards at that benchmark level. In order to report subscores, or strand scores, no less than six items will be used. The shorter online progressive tests will be used to report total test scores only; no subscores will be given for these tests.

Additional Test Design Criteria Each item assesses only one SRC at one grade. Each item assesses only one sub-SRC at one grade. Online-progressive test opportunities provide a range and breadth of items within each SRC and sub-SRC. Test pools attempt to provide a minimum of one item at each difficulty level for each piece of testable content. Test pools range in size from 400 to 530 or more items. Correct key placement is controlled on paper/pencil tests to restrict a repeat key to 3 or less in sequence. Key placement cannot be controlled for online-progressive assessments, so to ensure more random correct keys, item writers are instructed to rotate the correct key for their items during item authoring. Modules, or items that have a stimulus and have 2-5 questions, are designed to be presented together on both paper and on-line tests. Each module may include items from different SRCs or sub-SRCs, but each item only assesses one sub-SRC. Between one third and one half of the items from each SRC in each test pool, are Modules. English test blueprints provide the criteria for all Spanish/English and Russian/English tests. Test pools and paper/pencil tests are designed to match the English test opportunities.



23

Weighting of Science Score Reporting Categories The chart below shows the score reporting categories for each of the benchmark levels and the percentage of questions on a given test form, whether online or paper and pencil, that assess each score reporting category. For example, at grade 5, 33% of the items on a test assess Life Science, which equals about 17 items on a 50-item test. The second chart is an expanded view of the criteria for test weighting.

Percent of Questions on Test Science Score Reporting Category Benchmark 2;

Grade 5 Benchmark 3;

Grade 8 HS;

Grades 9-12

Physical Science 33% 33% 33%

Life Science 33% 33% 33%

Earth and Space Science 33% 33% 33%



24

Science Test Blueprint- Benchmark 2/Grade 5 (OAKS online, Paper/Pencil) Content Coverage and Weighting

Score Reporting Categories

Testable content codes

% of questions assessed per test P/P and OAKS online

Number of KS items P/P Test

Number of KS items on OAKS Online

2008-09 OAKS Online Test Pool size

Physical Science

2.1.51.(1,2) 2.2.51.(1,2) 2.3.51.1 2.3.52.(1,2) 2.4.51.1 2.4.52.(1-3)

33% 20

12-14 155

Life Science 3.1.51.1 3.1.52.1 3.1.53.1 3.2.51.(1,2) 3.3.51.(1-5) 3.4.51.(1,2)

33% 20

12-14 162

Earth and Space Science

4.1.51.(1-4) 4.2.51.(1,2) 4.2.52.(1,2) 4.3.51.1 4.3.51.2

33% 20

12-14 141

Operational Item Total

- 60

40 458

Field Test Item Total

- NA

6 400

Total items on Test

100% 60

46



25

Cognitive Demand and Item Distribution by Difficulty The science test pools and paper/pencil tests are designed so that a range of Depth of Knowledge items and a range of difficulty items are included for each student test opportunity. Although a range of Depth of Knowledge is included, the tests are not yet designed with an established goal (see Appendix 4, Depth of Knowledge Distribution Goal). The difficulty distribution goals for test pools and paper/pencil tests are outlined in the chart below. The three Depth of Knowledge levels used to qualify Oregon’s multiple choice test items are: • Recall: Item requires a student to recall a fact,

information or procedure. • Skill/Concept: Item requires a student to use skill or

concept, including thinking that requires two or more steps.

• Strategic Thinking: Item requires a student to use reason, develop a plan or use a sequence of steps.

Paper/pencil items are arranged so students are first introduced to items of medium or easy difficulty and then given more difficult items. This pattern is repeated throughout the test. Online progressive tests provide students with questions at the beginning of the test at or about the mean RIT level and as the student responds, makes adjustments by selecting appropriate items for each student based upon their correct and incorrect responses.

Difficulty Criteria for Benchmark 2/Grade 5 level

Grade 5 Science

Distribution on Test

Difficulty

204-214 33%

215-229 33%

230-244 33%

Depth of Knowledge

Recall

Skill/Concept

Strategic ThinkingSee Appendix 4

RIT Range 204-244

Mean RIT 222



26

Achievement Level Descriptors Achievement level descriptors describe what students know and can do based on their performance on statewide knowledge and skills tests in the various content areas. These may be used by educators to target instruction and inform parents and students of the expectations for students to be considered proficient at a particular grade level. The Achievement Level Descriptors are based on a sampling of a larger set of testable content outlined in the Oregon Content Standards. Results for individual students are only one indicator of student ability as measured at the time of testing. These statements give a general description of what most students know and can do within a particular band of achievement and are presented in the order of the way they are reported rather than by importance or test emphasis. Students who score at or within a particular level of achievement possess the bulk of the abilities described at that level and generally have mastered the skills described in the preceding achievement levels. Achievement Level Descriptors for each subject area were developed by groups of parents, educators, and business people who worked with state officials to establish the minimum scores required for Exceeds, Meets, Nearly Meets and Does Not Yet Meet.



27

Exceeds Student scores at this level indicate a very strong academic performance based on the benchmark level knowledge and skills outlined in the state content standards for Science. These students can consistently explain and describe the fundamental properties of matter, force and energy and the basic structures, functions and interactions of living organisms in the environment. They can consistently describe Earth’s properties and correctly explain Earth’s relationship in space.

Students who EXCEED the Benchmark 2 (Grade 5) science standard identify and thoroughly describe substances and their properties as they exist in different states of matter. They thoroughly describe how matter is affected by heating and cooling. They identify changes and recognize effects of changes in states of matter in the environment. Students accurately describe and compare the motion of objects in terms of one or more forces acting upon them. They recognize and apply principles of magnetism and gravity. They identify various types of energy and describe their effect on matter. They identify and describe ways energy produces and transfers heat. Students analyze characteristics that separate organisms into different classifications. Students understand how an organ functions in a system and how different systems interact within an organism. Students evaluate how elements in a food chain or food web inter-relate. Students analyze how an animal’s behavior affects its ability to survive as a species. Students understand how organisms adapt to survive based on living/non-living resources in their habitat. Students analyze environmental changes as they impact adaptation or survival of a species. Students apply their knowledge of properties of Earth materials. They describe weather in measurable quantities and interpret complex data to begin to analyze weather patterns. They describe causes and effects of changes on Earth’s surface features. They describe and apply their knowledge of the Earth’s place and movement in the solar system. They apply principles of conservation and appropriate waste disposal.



28

Meets Student scores at this level indicate a solid academic performance based on the benchmark level knowledge and skills outlined in the state content standards for Science. These students can explain and describe most fundamental properties of matter, force and energy and the basic structures, functions and interactions of living organisms in the environment. They can describe most of Earth’s properties and can explain Earth’s relationship in space.

Students who MEET the Benchmark 2 (Grade 5) science standard identify and describe substances and their properties as they exist in different states of matter. They describe how matter is affected by heating and cooling. They identify changes in states of matter in the environment. Students recognize and describe the motion of objects in terms of one or more forces acting upon it. Students recognize basic principles of magnetism and gravity. They identify various types of energy and their effect on matter. They identify ways energy produces and transfers heat. Students identify and classify living things into groups based on observable characteristics and their life cycle. (e.g. mammals, insects, fish, birds, reptiles and plants.) Students identify major organs by the system to which they belong and understand that an organism’s structure and function go hand-in-hand with its survival. Students understand components of food chains and webs. Students will recognize that how an animal interacts within its habitat affects its species’ survival. Students recognize adaptations that organisms need to survive and that conditions in an environment cause species to either change, become endangered, or become extinct. Students recognize most properties of Earth materials and begin to apply this knowledge. They describe weather in measurable quantities including temperature, wind direction, wind speed, and precipitation. They interpret data in weather over time. They identify causes and effects of Earth’s surface changes. They describe Earth’s place and movement in the solar system. They apply principles of conservation and appropriate waste disposal.



29

Nearly Meets Student scores at this level indicate an incomplete grasp of the benchmark level knowledge and skills outlined in the state content standards for Science. These students can partially explain and describe the fundamental properties of matter, force and energy and the basic structures, functions and interactions of living organisms in the environment. They can partially identify Earth’s properties and can recognize some of Earth’s relationship in space.

Students who NEARLY MEET the Benchmark 2 (Grade 5) standard identify some substances and their properties as they exist in different states of matter. Students partially describe how matter is affected by heating and cooling. They identify some changes in states of matter in the environment. Students recognize some of the basic principles of magnetism and gravity. They identify some types of energy and their effects on matter. They identify some ways energy produces and transfers heat.

Students recognize that living things belong in certain groups based on observable characteristics and their life cycle. (e.g. mammals, insects, fish, birds, reptiles and plants) Students recognize some body organs. They sometimes recognize that an organism’s structure and function go hand-in-hand with its survival. Students recognize some elements of food chains and webs. Students have some recognition of how an animal interacts within its habitat and how that interaction affects its survival. Students may recognize that organisms adapt to survive based on the living and non-living resources in its habitat and may recognize that conditions in an environment cause species to either change, become endangered, or become extinct.

Students recognize some properties of Earth materials. They know that resources are limited and that discarded products contribute to the problem of waste disposal. They interpret basic data over a period of time. They sometimes use information to identify changes in weather. They identify some changes of Earth’s surface features. They partially identify Earth’s place and movement in the solar system.



30

Does Not Yet Meet Student scores at this level indicate a minimal and/or inaccurate grasp of the benchmark level knowledge and skills outlined in the state content standards for Science. These students inconsistently explain and/or minimally describe the fundamental properties of matter, force and energy and the basic structures, functions and interactions of living organisms in the environment. They can minimally identify Earth’s properties and Earth’s relationship in space.

Students who DO NOT YET MEET the Benchmark 2 science standard have limited knowledge about substances as they exist in different states of matter and students have very basic knowledge about how matter is affected by heating and cooling. They identify some changes in states of matter in the environment. Students have limited understanding of magnetic principles, the motion of objects, and Earth’s gravity. They identify some types of energy and have limited knowledge of energy transfer. Students correctly classify some living things into groups based on characteristics and may identify some organ systems and plant and animal structures. They recognize that organisms are produced by living organisms of a similar kind. They sometimes recognize that some animals depend upon plants for survival. Students identify a few properties of Earth materials. They know that resources are limited and that discarded products contribute to the problem of waste disposal. They identify data collected over a period of time and may use information to identify changes in weather. They identify at least one rapid change on Earth’s surface.



31

Appendix

The Appendix of this document includes ancillary materials provided to students to complete science testing; and additional assessment documents that deal with test construction, design and assessment. Included in this section are:

Appendix 1: Periodic Table-included for testing at Grade 8 and High School

Appendix 2: Scientific Inquiry Test Specification for local assessment of work samples

Appendix 3: Achievement Standards Summary for all subjects

Appendix 4: Depth of Knowledge Distribution Goal

Appendix 5: Item Development

Appendix 6: Life of an Item

13 14 15 16 17

18

La57

Ac89

1.008

1

4.003

2

20.18

10

14.01

7

39.95

18

35.45

1719.00

9

16.00

8

83.80

36

131.3

54

(222)

86

12.01

6C

10.81B5

26.98Al13

28.09Si14

30.97P15

32.07S16

6.941Li3

9.012Be

4

22.99Na

11

24.31Mg

12

39.10K19

40.08Ca

20

44.96Sc21

47.88Ti22

50.94V23

52.00Cr24

54.94Mn

25

55.85Fe26

58.93Co27

58.69Ni28

63.55Cu

29

65.39Zn30

69.72Ga31

72.58Ge32

74.92As33

78.96Se34

79.90

35

85.47Rb

37

87.62Sr38

88.91Y39

91.22Zr40

92.91Nb41

95.94Mo

42

132.9Cs

55

137.3Ba

56

178.5Hf72

180.9Ta73

183.9W74

(223)Fr87

(226) (227)

138.9

Ra88

(98)

43

126.9I53

101.1Ru

44

102.9Rh45

106.4Pd46

107.9Ag47

112.4Cd48

186.2Re

75

190.2Os

76

192.2Ir77

195.1Pt78

197.0Au79

200.5Hg

80

204.4Tl81

207.2Pb82

208.9Bi83

(209)Po84

(210)At85

114.8In49

118.7Sn50

121.8Sb51

127.6Te52

Br

1

54

2

3 6 7 1211

175.0Lu71

140.1Ce

58

(232.0)Th90

(231)Pa91

(238)U92

140.9Pr59

144.2Nd60

(257)

103

(147)

61

(237)

93

(242)

94

(243)

95

(247)

96

(247)

97

(249)

98

(254)

99

(253)

100

(256)

101

(254)

102

(150.4)Sm

62

152.0Eu

63

157.3Gd

64

158.9Tb65

162.5Dy66

164.9Ho67

167.3Er68

168.9Tm

69

173.0Yb70

(262)

107

(257)

104

(260)

105

(263)

106

(265)

108

(266)

109

(271)

110

(272)

111

(277)

112

Rf

Cf Es Fm MdPu Am Cm Bk LrNo

Bh

Np

Hs Uub(289)

114

Uuq(293)

116

Uuh(?)

118

UuoUuuDs

Pm

MtSgDb

Tc

9 108

June 2008

Appendix 1 Science, Benchmark 2/Grade 5



1

SCIENTIFIC INQUIRY FORMING THE QUESTION/HYPOTHESIS Score Reporting Category 5

Common Curriculum Goal: Formulate and express scientific questions or hypotheses to be investigated.

Content Standard

Make observations. Formulate and express scientific questions or hypotheses to be investigated based on the observations.

Benchmark 2 Standard

Make observations. Ask questions or form hypotheses based on those observations, which can be explored through scientific investigations.


Eligible Content

See 2002-2009 Scientific Inquiry Benchmark 2 Scoring Guide, Level 4.

Explanation

Students should be able to identify a question, formulate a testable hypothesis and demonstrate logical connections between the scientific concepts guiding a hypothesis and the design of the experiment.

Locally Assessed

Teachers are expected to provide instruction and classroom assessment in all four dimensions of the scoring guide and scores from these assessments* must be reported to school district work sample management. Scientific Inquiry Anchor Papers and Support Materials are available at http://www.ode.state.or.us/search/page/?id=518 *Forming the Question/Hypothesis scores are not required to be reported for Benchmark 2 work sample management.




2

SCIENTIFIC INQUIRY DESIGNING THE INVESTIGATION Score Reporting Category 5

Common Curriculum Goal: Design safe and ethical scientific investigations to address questions or hypotheses.

Content Standard

Design scientific investigations to address and explain questions or hypotheses.


Design a simple scientific investigation to answer questions or test hypotheses.


Eligible Content


Explanation

Students must be able to design and construct a scientific investigation using proper equipment, procedures and safety precautions. They should provide clarification of ideas that guide the inquiry and obtain scientific knowledge from sources other than the actual investigation. Their investigation should include clarification of the question, method, controls, variables and provide organization of the data.

Locally Assessed

Teachers are expected to provide instruction and classroom assessment in all four dimensions of the scoring guide and scores from these assessments must be reported to school district work sample management. Scientific Inquiry Anchor Papers and Support Materials are available at http://www.ode.state.or.us/search/page/?id=518




3

SCIENTIFIC INQUIRY COLLECTING AND PRESENTING DATA Score Reporting Category 5

Common Curriculum Goal: Conduct procedures to collect, organize, and display scientific data.

Content Standard

Collect, organize, and display scientific data.


Collect, organize, and summarize data from investigations.


Eligible Content


Explanation

Students summarize varied sources of evidence including their own data and published reports and evaluate the reports of others using scientific criteria.

Locally Assessed





4

SCIENTIFIC INQUIRY ANALYZING AND INTERPRETING RESULTS Score Reporting Category 5

Common Curriculum Goal: Analyze scientific information to develop and present conclusions.

Content Standard

Analyze scientific information to develop and present conclusions.


Summarize, analyze, and interpret data from investigations.


Eligible Content


Explanation

Students are able to analyze data, taking steps to limit observer and sample bias using mathematical statistics and probability concepts. They are able to formulate and revise explanations and models based on evidence and logical argument while preserving significant information.

Locally Assessed





5

These standards were adopted by the State Board of Education April 26, 2001. Students were first accountable for meeting these standards in the Spring 2003 statewide assessment.

Instruction in the Common Curriculum Goals of Unifying Concepts and Processes, History and Nature of Science, Science in Personal and Social Perspectives, and Science and Technology is required in all Oregon school districts; however, they are not included in the state assessment except as specifically indicated in

the eligible content in Earth/Space Science, Life Science, or Physical Science.

UNIFYING CONCEPTS AND PROCESSES: Understand and apply major concepts and processes common to all sciences.

COMMON CURRICULUM GOALS: • Understand that any collection of things that have an influence on one another can be thought of as a system. • Understand that a model is a tentative scheme or structure with explanatory power. • Understand that both patterns of change and stability are important in the natural world. • Understand that changes in scale influence the characteristics, properties and relationships within a system.

HISTORY AND NATURE OF SCIENCE: Understand science as a human endeavor, the nature of scientific knowledge, and the history

of science as it relates to and clarifies scientific inquiries.

COMMON CURRICULUM GOALS: • Understand that science is a human endeavor practiced by individuals from many different cultures. • Understand that scientific knowledge is subject to change based on new findings and results of scientific observation and experimentation. • Understand that scientific knowledge distinguishes itself through the use of empirical standards, logical arguments, and skepticism.

SCIENCE IN PERSONAL AND SOCIAL PERSPECTIVES: Understand that science provides a basis for understanding and acting on personal and social issues.

COMMON CURRICULUM GOALS: • Describe the role of science and technology in local, national, and global issues. • Describe how daily choices of individuals, taken together, affect global resource cycles, ecosystems, and natural resource supplies. • Explain risks and benefits in personal and community health from a science perspective.

SCIENCE AND TECHNOLOGY: Understand the interconnections among science, technology, and society.

COMMON CURRICULUM GOALS: • Understand the relationship that exists between science and technology. • Understand the process of technological design to solve problems and meet needs.


Grade 3 MEET EXCEEDReading/Literature 204 218

Mathematics 205 217

Writing, Speaking, Science, Social Sciences No state test

Grade 4 MEET EXCEED

Reading/Literature 211 223

Writing• Composite score• Minimum score in each trait• Conventions score

32 to 39* (out of 48)3 (out of 6)Not doubled

•••

40 to 48 (out of 48)4 (out of 6)Not doubled

•••

Voice and Word Choice are not included in the achievement standard.* A composite score of 28 to 31 points conditionally meets the standard. The school district may determine that the standard has been met IF that student has met all writing local performance requirements.

Mathematics 212 225

Speaking, Science, and Social Sciences No state test

Grade 5 MEET EXCEED


Mathematics 218 229

Science 225 238

Social Sciences # 215 225

# Optional state test; may be required by districts or schools.

Writing, Speaking No state test

2008-09 Achievement Standards SummaryThe charts below show the achievement standards (requirements to meet and exceed) for Oregon’s Assessments of Knowledge and Skills (OAKS) by content area and grade or benchmark level. All students are required to take reading/literature and mathematics assessments in grades 3–8 and 10; in writing in grades 4, 7, and 10; and Science in grades 5, 8, and 10. Social Sciences assessments are optional; howev-er, they may be required by some districts or schools. For detailed assessment information, refer to the 2008-09 Test Administration Manual (www.ode.state.or.us/go/TAM). It provides timelines, options, and procedures that ensure both test reliability and validity from classroom to classroom, teacher to teacher, school to school, and district to district.


Writing• Composite score• Minimum score in each trait• Conventions score

40 to 49* (out of 60)3 (out of 6)Doubled

•••

50 to 60* (out of 60)4 (out of 6)Doubled

•••

Voice and Word Choice are not included in the achievement standard.* A composite score of 35 to 39 points conditionally meets the standard. The school district may determine that the standard has been met IF that student has met all writing local performance requirements.

Mathematics 226 238

Speaking, Science, and Social Sciences No state test

Grade 8 MEET EXCEED


Mathematics 230 241

Science 234 246

Social Sciences # 231 241

# Optional state test; may be required by districts or schools.

Writing, Speaking No state test


Mathematics 221 232

Writing, Speaking, Science, Social Sciences No state test

High School Achievement Standards1

Oregon Assessment of Knowledge and Skills (OAKS) is one option to provide evidence of proficiency in

Essential Skills .for Purposes of

Accountability and High School Diploma 2

(Beginning for Class of 2012)Subject Area Meets Exceeds Notes Essential Skill

Reading/ Literature 236 248

Content of the 2008-2009 OAKS Reading/Literature Assessment is based on the Grade Level Content Standards adopted in 2002-2003.

Read and comprehend a variety of text

Writing• Composite score• Minimum score allowed

in any trait• Conventions score

• 40 to 49 (out of 60)• 3 (out of 6)

• Doubled

• 50 to 60• 4 (out of 6)

• Doubled

• A composite score of 35 to 39 conditionally meets the standard. The district may determine that the standard has been met `if that student has met all writing local performance requirements.

• Scores on Voice and Word Choice traits are not included in the achievement standard.

Write clearly and accurately

Mathematics 236 246 Content of the 2008-09 OAKS Mathematics test is based on the Grade Level Content Standards adopted in 2002.

Apply mathematics in a variety of settings

Science 240 249 Content of the 2008-09 OAKS Science is based on the Content Standards adopted in 2001.

Social Sciences 239 249 Optional state assessment; the Social Sciences Assessment may be required by districts or schools.

1 In future years, Achievement Standards may change for the purposes of accountability and earning a high school diploma. 2 For purposes of demonstrating mastery of Essential Skills, students must meet the achievement standards in effect during their 8th grade year. However, students may use achievement standards adopted in their 9th through 12th grade years that are equal to or lower than the achievement standards approved as of March 1 of the students’ 8th grade year. In addition, students may demonstrate proficiency in the Essential Skills using additional assessment options adopted in their 9th through 12th grade years.

ACHIEVEMENT STANDARDS

Oregon Standards Newspaper — 2008–2009 School Year — Oregon Department of Education Page 6A

Achievement Standards for Work Samples

Oregon Standards Newspaper — 2008–2009 School Year — Oregon Department of Education Page 7A

A Look at Work Samples As Required Local Performance Assessments(Grades 3 – 8 and High School)

Local performance assessments evaluate the application of students’ knowledge and skills. OAR 581-22-0615 Assessment of Essential Skills requires students to complete one or more local performance assessments per year (in grades 3-8) and at least once in high school for each skill area assessed. The table below outlines the achievement standards for work samples scored with an official state scoring guide and used as a local performance assessment.

Skill Area (Official State

Scoring Guide)

GradeAchievement Standard

for Purpose of Local Performance Assessment

Notes about Work Samples

Meets(out of 6)

Exceeds(out of 6)

Writing Grade 3 3 4 Grade 3 students are not held to a standard in Sentence Fluency.

Grades 4-8 and High School 4 5 Voice and Word Choice may be scored but are not required traits. Exemplars reflect expectations at each grade level.

Speaking Grade 3 3 4 Grade 3 students are not held to a standard in Language.

Grades 4-8 and High School 4 5 Exemplars reflect expectations at each grade level.

Mathematics Problem Solving Grades 3-8 and High School 4 5 Exemplars reflect expectations at each grade level.

Scientific InquiryBenchmark 2 (Grades 3-5)

Benchmark 3 (Grades 6-8) and High School

4 5 A separate official scoring guide exists for each of the three grade-bands (3-5, 6-8, and high school).

OptionalIn addition, one social science analysis work sample that is administered in accordance with local policies

may be used as a local performance assessment for grades 3 through 8 and for high school.

Social Science Analysis

Benchmark 3 (Grades 6-8) and High School

4 5 A separate official scoring guide exists for 6-8 and for high school. Support for grades 3-5 will be released by ODE in August 2009.

Related Web Links: Official State Scoring Guides: http://www.ode.state.or.us/search/page/?id=32Exemplars of scored work samples are currently found on subject-specific assessment pages linked from: http://www.ode.state.or.us/search/page/?id=1307

A Look at Work Samples As a Local Assessment Option for Assessing Essential Skills for the Oregon Diploma

The graduating class of 2012 will be required to demonstrate proficiency in the first set of essential skills: read and comprehend a variety of text, write clearly and accurately, speak clearly and coherently, and apply mathematics in a variety of settings in order to receive a high school diploma. Work samples are one assessment option that high school students may use to demonstrate they are proficient in the es-sential skills. Regarding demonstration of proficiency in the essential skills, districts must:

• provide students with instruction in and multiple assessment opportunities to demonstrate proficiency in the Essential Skills for the purpose of earning a high school or modified diploma • allow students to use assessment options adopted in a student’s 9th through 12th grade years• allow students to use achievement standards adopted in their 9th through 12th grade years that are equal to or lower than the achievement standards approved as of March 1 of the students’ 8th grade year.

At the high school level, students may use work samples to fulfill both the local performance assessment requirement and the Essential Skills requirement. By March 1, 2009, ODE will issue criteria to help districts establish other local assessments that are comparable. This information will be included as an addendum to the 2008-09 Test Administration Manual.The table below describes the achievement standard for work samples scored for the purpose of demonstrating proficiency in the essential skills with regard to conferring a high school diploma.

Essential Skill Scoring GuideAchievement Standard for

Purpose of Conferring High School Diploma

(Cut Scores)Number and Types of Work Samples

Write clearly and accurately

Official Writing Scoring Guide 4 out of 6 in each required trait to meet the standard;5 or 6 to exceed

3 total work samples: one expository, one persuasive, and one narrative sample

Speak clearly and coherently

Official Speaking Scoring Guide

4 out of 6 in each required trait to meet the standard;5 or 6 to exceed

3 total work samples: one expository, one persuasive, and one unrehearsed

Apply mathematics in a variety of settings

Official Mathematics Problem Solving Scoring Guide

4 out of 6 in each required strand to meet the standard plus accuracy;5 or 6 to exceed

2 total work samples: Each from a different area limited to: geometry, algebraic relationships, statistics, or probability


1

Depth of Knowledge Distribution Goal Oregon recognizes the importance of Depth of Knowledge as part of test specifications. To that end, we are implementing a strategy to overtly incorporate a test design process that includes the three dimensions of content, difficulty, and depth of knowledge.

The first step in the process began in Fall, 2007 when our content panels were asked for their determination as to what the appropriate allocation of Depth of Knowledge should be given the content standards.

Secondly, we will analyze the gap between the Depth of Knowledge available in our current item pools against the content panels’ recommendations.

The third step will involve engaging item writers to write items to fill in the critical gaps. These items would then be reviewed through our standard processes and would be field tested beginning in 2009.

By 2011, items that meet technical specifications would be available for operational testing. We would anticipate being able to include Depth of Knowledge as an explicit part of the test specifications at this time. The three Depth of Knowledge levels to be addressed are: • Recall: Item requires a student to recall a fact, information or procedure.

• Skill/Concept: Item requires a student to use a skill or concept, including thinking that requires two or more steps.

• Strategic Thinking: Item requires a student to use reason, develop a plan or use a sequence of steps.

Depth of Knowledge/Difficulty Distribution: Science

Grade 5 Distribution on Test

Grade 8 Distribution on Test

HS Distribution on Test

Difficulty Difficulty Difficulty

204-214 33% 213-226 33% 217-230 33%

215-229 33% 227-239 33% 231-243 33%

230-244 33% 240-253 33% 244-257 33%

Depth of Knowledge Depth of Knowledge Depth of Knowledge

Recall Recall Recall

Skill/Concept Skill/Concept Skill/Concept

Strategic Thinking

To be established by 2010-11

Strategic Thinking


Strategic Thinking


RIT Range 204-244 RIT Range 213-253 RIT Range 217-257

Mean RIT 222 Mean RIT 231

Mean RIT 235


Science Test Specifications 1 Oregon Department of Education and Test Blueprints Office of Assessment and Information Services

ITEM DEVELOPMENT

Oregon’s item development procedures are consistent with industry practice and take approximately two years, including writing, review, and field-testing. Just as the development of Oregon’s content and performance standards was an open, consensus-driven process, the development of test items and prompts to measure those constructs is grounded in a similar philosophy.

Item Writing

For the Knowledge and Skills (multiple-choice) tests and Performance Assessments, most item writing takes place during three to five-day item writing workshops, in which more than 350 Oregon teachers across the four main content areas write and review up to 10,000 items.

Item writers are typically Oregon teachers who have received training in item construction, are familiar with test specifications, and have demonstrated skill in writing items that pass content panel reviews. Experienced and successful writers are invited to write items annually. Item writers receive professional development compensation for their time and related expenses.

All items are written to measure specific subdomains of the content standards at a variety of specified levels of cognitive complexity. Cognitive complexity is represented by the

following classification, developed from Bloom’s (1956) educational taxonomy:1

• Recall: Recall, label, or locate information; define or describe facts or processes.

• Skill/Concept (Basic Application): Use information or conceptual knowledge, often requiring two or more steps; summarize, classify, or explain information or processes; make predictions or generalizations; solve problems.

• Strategic thinking: Analyze, critique, compare or contrast; create new information; or organize presented information.

• Extended thinking: Make connections and extensions (Writing Performance Assessment and local work sample assessments).

During the item writing workshop, writers draft items and conduct peer reviews of each other’s items. Examples of items are provided, and facilitators provide process guidance and additional review. Writers and reviewers evaluate the strength and clarity of the match between the drafted item and the standard it measures. All items are worked out or solved multiple times by multiple reviewers who verify that distractors

1 Bloom, B. S. (ed.), Engelhart, M. D., Furst, E. J., Hill, W. H., & Krathwohl, D. R. (1956). Taxonomy of educational objectives: Handbook I: Cognitive domain. New York: David McKay.



are plausible, that answers are correct, and that each item has only a single correct answer.

Item writing and passage selection are guided by the following principles for each of the item types.

Multiple-choice items should

• have one correct response option.

• contain plausible distractors that represent feasible misunderstandings of the content.

• provide options that are grammatically parallel in structure and length.

• represent the range of cognitive complexities and include challenging items for students performing at all levels.

• be appropriate for students in the assigned grade in terms of reading level, vocabulary, interest, and experience.

• arrange numerical answers in either ascending or descending order, except in items involving the ordering of numbers.

• be embedded in a real-world context.

• not provide answers or hints to other items in the set or test.

• be in the form of questions or sentences that require completion.

• use clear language and not be worded in the negative unless doing so provides substantial advantages in item construction.

• be free of absolute wording, such as “always” and “never,” and have qualifying words (e.g., least, most, except) printed in small caps for emphasis.

• reflect the diversity of students in Oregon and not involve death, violence, drug and alcohol abuse, criminal activities, or the occult.

• be free of ethnic, gender, political, and religious bias.

Selected passages should

• represent literary (fiction), informative (nonfiction), and practical selections (e.g., nontraditional pieces, including tables, charts, glossaries, indexes).

• have a definite beginning, middle, and end and a sense of completeness.

• be of high interest and appropriate readability for the grade level.

• be of appropriate length for the grade level—

- Grade 3, 300 words or less (150–250 on average) - Grade 4, 400 words or less (250–350 on average) - Grade 5, 500 words or less (350–450 on average) - Grade 6, 600 words or less (500–600 on average) - Grade 7, 700 words or less (600–700 on average) - Grade 8, 800 words or less (700–800 on average) - Grade 10 (HS), 1,000 words or less (800–900 on

average)



• not involve death, violence, drug and alcohol abuse, criminal activities, or the occult.


• not represent material that is so widely anthologized or taught that students may have already studied the selection.

• not provide answers or hints to other items in the test.

• include real-world texts (from consumer, workplace, or such public documents as letters to the editor, newspaper and magazine articles, thesaurus entries) to the extent possible.

• reflect the diversity of students in Oregon.

• include material by writers from Oregon or the Pacific Northwest.

Essay questions (writing prompts) should

• be of high interest and readability appropriate to the

grade level.

• reflect the diversity of students in Oregon.

• represent the range of cognitive complexities and include challenging items for students performing at all levels.

• be appropriate for students in the assigned grade, in terms of reading level, vocabulary, interest, and experience.

• be embedded in a real-world context in which students may have experience.

• not involve death, violence, drug and alcohol abuse, criminal activities, or the occult.


Following item writing workshops, items are entered into the ODE’s Item Management System (ITS). In the item manage-ment system, each item is given a unique item identification number to facilitate the monitoring and tracking of changes to and usage of the item throughout the review process and each item’s history. ITS provides authorized users with access to each item’s alignment and attributes, field-test results and use, response rationales, and previous versions.

Although item writing workshops still occur annually, ODE has recently moved toward distributed item writing in which consistently strong item writers write additional items throughout the year. Items still go through the review process described.

Committee/Panel Review

All items generated for use on Oregon statewide assessments must pass a series of rigorous reviews before they can be used in field and operational tests. Items are accepted, rejected, or modified by the Content and Assessment Panel to make sure they represent the constructs embodied in grade-specific content standards and test specifications. Only the items that measure the grade-level expectations are carried forward to the field-test stage. In addition to judgments of content relevance, the panels appraise the technical quality of items, looking for items that are free from such flaws as (a) inappropriate readability level, (b) ambiguity, (c) keyed answers and distractors, (d) unclear instructions, and (e) factual inaccuracy.



ODE Content and Assessment Panels exist for each of the content areas for which statewide tests are given: English/Language Arts (this panel reviews Writing and Reading/Literature assessment items), Mathematics, Science, and Social Sciences.

Members of Content and Assessment Committees reflect the demographics of Oregon’s student population. The following table describes the structure of these panels.

Structure of ODE Content and Assessment Panels/Committees

Panel/Committee Number of Members Meeting Frequency Who Nominates Members?

Assessment Policy Advisory Committee 15–20 4–6 times a year School districts, COSA, OSBA, OEA, ESDs, and OPTA

Sensitivity Panel 15–20 4–6 times a year School districts, OEA, ESDs (application process)

English/Language Arts Content and Assessment Panel 35 6–8 times a year School districts, OEA, ESDs, and self-

nominate (application process) Mathematics Content and Assessment Panel 35 6–8 times a year School districts, OEA, ESDs, and self-

nominate (application process) Science Content and Assessment Panel 35 6–8 times a year School districts, OEA, ESDs, and self-

nominate (application process) Social Sciences Content and Assessment Panel 25 6–8 times a year School districts, OEA, ESDs, and self-

nominate (application process)

Note. Oregon’s Accommodations and Modifications Review Panel and English Language Proficiency Content and Assessment Panel are not described here. Source: http://www.ode.state.or.us/teachlearn/testing/dev/panels/structurecapanels.doc



Each panel has approximately 20–30 members who serve three-year terms with one-third of the members rotating off the panel each year and replaced by new representatives.

Panel members commit to 6–8 school days of service with an additional 3 or 4 days during the summer. Although committee members are not compensated for their service, they do receive travel reimbursement for committee travel of more than 70 miles, and substitute teachers are provided for service during the school year.

Most members of these panels are classroom teachers, with some representation from state and district curriculum personnel. Criteria for panel selection include the following:

• Knowledge of Oregon’s content standards and expertise in the subject area and its eligible content

• Teaching experience at the benchmark to which the individual will be assigned

• Geographical location to ensure that all regions of Oregon are represented

• Gender and ethnic diversity to ensure that the panel represents the diversity of Oregon’s student population

Current item writers are not allowed to serve on item review committees.

The panels for each content area use the following review process:

• Three content panel members review each item independently and complete an Item Review Form (IRF) (figure 1) using a pre-assigned reviewer ID.

• Then, the three content panel members review the item collectively, and item reviewers make a recommendation for each item on the IRF to either (a) accept the item as written, (b) accept the item with revisions, or (c) reject the item (sometimes an alternate question is offered that entails a simple revision).

• When all three reviewers agree that an item should be accepted or rejected, no further discussion is needed. If one or more of the reviewers feel that an item should be revised, then they attempt to reach a consensus and produce a “master copy” of their recommendation. The same is true if one or two of the reviewers reject an item that another reviewer finds acceptable with or without revisions.

• In most cases, recommendations are followed and revisions are made, or items are eliminated. The ODE content specialist can override the recommendation, but this occurs rarely and only for compelling reasons.



Figure 1. Sample Oregon Content and Assessment Panel Item Review Form

The panels perform specific checks on items to confirm that

• the SRC and subcategory match.

• the key is correct.

• alternate valid interpretations making the distractors correct do not exist.

• items are grade-level appropriate in content and reading levels.

• the item is of overall high quality (wording and grammar, graphic quality, curricular importance, etc).

• the identified level of difficulty (i.e., easy, medium, hard) is correct.

• Reading/Literature passages are appropriate in content and reading levels.

• the level of cognitive complexity (i.e., recall, skill/concept,or strategic thinking) is confirmed by ODE and content panels, (consistent with bullets on page 1 of this document) andis appropriate to the item and correctly identified.

Following the Content and Assessment Panels and according to panel feedback, Oregon assessment specialists edit and revise



items in ITS. Before the reviewed items are approved for inclusion within operational tests as field-test items, they are presented to a sensitivity panel. Sensitivity panels review items in two-day meetings, four to six times a year. These panels review items for bias, controversial content, and overly emotional issues.

In general, sensitivity panels ensure that items

• present racial, ethnic, and cultural groups in a positive light.

• do not contain controversial, offensive, or potentially upsetting content.

• avoid content familiar only to specific groups of students because of race or ethnicity, class, or geographic location.

• aid in the elimination of stereotypes.

• avoid words or phrases that have multiple meanings.

Following the sensitivity panels and according to panel feedback, Oregon assessment specialists edit and revise items

in ITS. Next, ODE assessment specialists identify items from this pool for inclusion within operational field tests.

ITEM USE AND RELEASE

Approximately every three years, ODE releases one sample test for each content area and grade level comprising items used on previous test forms. These items are no longer secure and are taken out of the pool of eligible test items. ODE is reviewing its item-release policies for 2009.

Released items are provided in the form of practice tests. Practice tests for reading/literature, math, social sciences, and science are available on ODE’s Web site at

http://www.ode.state.or.us/search/results/?id=226.

Sample Writing prompts are also available at

http://www.ode.state.or.us/teachlearn/subjects/elarts/writing/assessment/ usingsampleprompts.pdf.


The complete two-year Lifecycle of a Knowledge and Skills ItemMathematics, Reading/Literature, Science, Social Sciences

PAGE1

Phase I Item Writing Phase II Item Review Phase III Field Testing

Bank of POTENTIAL items Bank of REVIEWED items

testitem

testitem

testitem

a. Assessment Specialist identif iesitems to be field tested.a. Assessment Staff plans for Item Writing

Workshop-site selection, invitation to itemwriters, construction of database software tofacilitate tracking and test construction.

b. Item Writing Workshop: Over 350Oregon teachers write more than 10,000items in four content areas over five days.Teachers receive professional developmenttraining on item development. Items arewritten explicitly to measure Oregonacademic content standards.

c. Teachers review items written bytheir peers.

Go to next phase

e. Assessment Specialist editsand revises items according to feedback.

d. At the workshop, Assessment staffenter items into a data base.

b. Four Content and Assessment Panels,consisting of Oregon teachers, review testitems with respect to content validity and gradeappropriateness.

d. Sensitivity Panel reviews itemsin two day meetings, four times a year.

a. Assessment Specialist sorts andorganizes items for review.

c. Assessment Specialist edits and revisesitems according to feedback.

b. Field test items are embeddedin an operational test.

c. Students complete operationaltests with embedded field test items.

Go to next phase Go to next phase

Contractor

Bank of FIELD items

d. Tests are processed by acontractor.

Appendix 6

a. Assessment staff analyze data todetermine if the item "behaves" asexpected.

The complete two-year Lifecycle of a Knowledge and Skills ItemMathematics, Reading/Literature, Science, Social Sciences

PAGE 2

Phase IV Data Analysis ofField Tested Items

Phase VTest Construction

Phase VI Data Analysis ofOperational Test Items

testitem

a. Tests sent to contractor forprinting distribution or web delivery.

freq mean

PBIS Pval

freq mean

PBIS Pval

b. Assessment Specialist reviews howwell the item performed in the field test.

c. Assessment staff calibrate thedifficulty of field tested items.

a. Assessment Specialist selects itemsfor the operational test.

b. Assessment Specialist balances itemswith the Score Reporting Category (SRC)(such as Geometry in Mathematics or WordMeaning in Reading/Literature) and rangedifficulty according to test specifications.

c. Assessment staff construct tests andfinalize Administration Manual.

b. Students complete the operationaltest and receive instant scores whenusing web delivery.

c. Assessment staff analyze itemstatistics to verify the item performsas expected.

Go to next phase

Go to next phase

Bank of CALIBRATED items

anchoritem

fieldtesteditem

d. Assessment staff and contractorsproof-read tests for errors.

FINAL OPERATIONAL

TESTS

d. Assessment staff analyze itemstatistics to make sure items are notbiased against a particular subgroup(e.g., students with disabilities, ethnicgroups, or gender).

e. Item performance tables describehow well each item performs.

Appendix 6

Oregon Department of Education

255 Capitol St NE, Salem, Oregon 97310 (503) 947-5600

TEST SPECIFICATIONS and BLUEPRINTS...include the ability of liquids to take the shape of their...

Documents

Transcript of TEST SPECIFICATIONS and BLUEPRINTS...include the ability of liquids to take the shape of their...