Test Creation Assignment Megan Gibiser The College of...
Transcript of Test Creation Assignment Megan Gibiser The College of...
Running Head: TEST CREATION ASSIGNMENT 1
Test Creation Assignment
Megan Gibiser
The College of William and Mary
CRIN 550-01
TEST CREATION ASSIGNMENT 2
Part I: Overview and Description
Course Description and Major Goals
The Kindergarten science curriculum as per the Virginia Standards of Learning (SOLs),
builds a foundation for scientific inquiry. Through topics such as magnets, physical properties,
life processes, and shadows, students will conduct investigations to understand that scientific
knowledge is based on evidence and use their five senses to gather information. The standards
emphasize the development of important science skills such as questioning, measuring,
classifying, and sorting to help students learn about the world around them. Additionally, the
curriculum introduces students to the idea of change over time and presents fundamental
scientific concepts that can be expanded upon in the later grades.
Specific Unit Description
This unit focuses on the life processes and specific needs of plants. It emphasizes that
plants have several basic needs including air, light, space, water, and nutrients to survive.
Through experimentation and observation, students will understand that there are consequences
if these life needs are not met. This unit also highlights the fact that plants change as they grow
throughout their life cycle which may include changes in size, leaf growth, or flowering.
Description and Explanation of Intended Learning Outcomes (ILO)
This assessment is based on the Virginia Science SOL, K.7 which states that the student
will investigate and understand basic needs and life processes of plants and animals. Although
the SOL references both plants and animals, only the four Intended Learning Outcomes (ILOs)
that refer to plants were used in developing this assessment. These ILOs came directly from the
Essential Knowledge, Skills, and Processes section of the SOL Curriculum Framework and were
TEST CREATION ASSIGNMENT 3
unpacked to determine the explicit, implied, and conditional content, as well as the cognitive
level on Bloom’s Taxonomy. To see a full description of the ILOs, refer to Appendix A.
In addition to this summative assessment, both a pre-assessment and formative
assessments will be used to gauge students’ understanding. At the start of the unit, the class will
complete a KWL chart to elicit prior knowledge about plants. During instruction, formative
assessments in the form of worksheets and entries in the students’ science journal will be
evaluated. The teacher will also complete informal observations during class discussions to
determine if instruction needs to be modified to improve student learning before the final unit
test.
Description of Classroom and School Context
This unit of study is intended for a Kindergarten class at Matthew Whaley Elementary
School. The class consists of 22 students, 11 boys and 11 girls. The students come from a range
of cultural and economic backgrounds as there are two African American students, four Hispanic
and ESL students who speak Spanish as their primary language, three biracial students, and
thirteen white students. Several students qualify for free or reduced lunch. Additionally, there
are two students with Individualized Education Programs (IEPs), but they are only pulled out
once a week for individual instruction. Although most students read at the instructional level for
Kindergarten, two students are reading above grade level. Lesson plans are generally developed
by individual teachers, but resources and ideas are shared amongst the Kindergarten team.
Students remain with the same teacher throughout the day except during centers in which they
meet with resource teachers.
The diversity of this classroom was taken into consideration when developing the present
assessment. To accommodate students with various reading abilities, the test will be orally
TEST CREATION ASSIGNMENT 4
administered by the teacher who will read each question one by one to the class. Also, to assist
the four ESL students, the select response items incorporated pictures whenever possible,
underlined key words, and avoided any culturally biased language. One of the supply response
items asks students for visuals as well as written responses to allow students to represent their
learning in multiple ways. For the two written responses, students will dictate what they wrote to
the teacher, so writing ability is not a factor.
Purpose of Test and Test Results
This test will be used as a summative assessment following a two week unit on plant life
cycles. The results will reflect whether students understood the concepts (rooted in the ILOs)
that were taught during class. It will also help the teacher reflect upon the effectiveness of her
instruction and make adjustments for future teaching. Students’ test results will be one variable
to consider when assigning 3rd
quarter report card grades for science.
Part II: Design Elements of Assessment
Intended Learning Outcomes
For a full list of the ILOs, see Unpacking the Curriculum (Appendix A).
Table of Specifications
A Table of Specifications (ToS) aligns test questions to the ILOS and places appropriate
weight on content based on how much it was emphasized during instruction (Gareis & Grant,
2008). A ToS also aligns content to the level of cognitive demand on Bloom’s Taxonomy. To
see a full ToS for this assessment, refer to Appendix B.
Construct Validity
Since this assessment follows the framework of the ILOs and measures what it intends to
measure, the test is high in construct validity. Students are only assessed on the scientific
TEST CREATION ASSIGNMENT 5
knowledge learned during the unit. Other content areas such as reading and writing are not
assessed since the questions are read aloud and the students dictate their written responses to the
teacher. The test has high face validity as well because similar items are grouped (select
response first and then supply response), and it looks and reads like a science assessment.
Content Validity
This test is also high in content validity because it adequately samples the ILOs that the
unit is based on. All four ILOs are referenced, and the number of questions on the assessment
accurately reflects the level of emphasis given to topics during instruction as displayed in the
ToS. For example, describing changes in a plants life cycle was given strong emphasis so there
are four questions to assess it, while comparing young and adult plants was given low emphasis
so it is only assessed on one supply response item. Overall, the test is representative of students’
understanding of plant’s needs and their life cycles.
Rationale of Assessment Item Types
This assessment consists of seven select response items and three supply response items.
Since Kindergarten students generally take a longer time to complete supply response due to
their developing writing skills, more emphasis was placed on select response. However, some
supply response items were necessary to assess students at the higher levels of cognitive demand.
For example, to reach the analysis and evaluation levels on Bloom’s taxonomy, students needed
to develop original answers.
The true/false and multiple choice question types were used to assess the knowledge and
comprehension levels of Bloom’s taxonomy including recall of vocabulary and understanding of
basic processes. Item eight was incorporated to assess students understanding of the entire plant
life cycle which was greatly emphasized during instruction. However, each response was only
TEST CREATION ASSIGNMENT 6
worth .5, to prevent students from losing many points if they mixed up part of the order. Item
nine asks for a short written response and item ten elicits a picture and a written response since
they are both accessing the analysis and evaluation levels of Bloom’s Taxonomy. Not only
would this have been difficult to assess with select response, but the supply response provides
greater insight into students’ thinking.
Potential Reliability Threats
There are some threats to reliability that may be found within this assessment. For
example, the majority of this assessment is true/ false, multiple choice, or sequencing, so it is
possible that students may guess at the answers. This may make it difficult to distinguish
students who knew the material from those who did not. However, every effort was made to
follow proper rules for item construction. For example, the true/ false questions are objective, do
not contain negatives, and focus on one central idea. Likewise, the multiple choice questions all
provided four plausible choices and did not give hints to the answer. One test item does ask
students to circle two answer choices. Although this change is bolded and will be emphasized
while reading the test, it could potentially confuse students and decrease reliability.
After the assessment was constructed, cooperating teacher Debbie Zanca proofread the
test. As a veteran teacher she understands what question types would be developmentally
appropriate for Kindergarten and reviewed the questions for grammatical and spelling mistakes.
This greatly reduces the possibility of error within the test items. Other attempts to avoid
systematic error include reading the test orally to students, providing visuals for reference,
avoiding culturally biased language, and allowing students to dictate their written responses to
the teacher. Despite these attempts to avoid systematic error, random error is always possible.
TEST CREATION ASSIGNMENT 7
For example, there could be a fire drill during the test or noise in the hall that would distract
students and decrease overall reliability.
Discussion of Predictive Validity
Since Kindergarteners do not take high-stakes tests such as the SOL, there is not an
immediate concern for predictive validity. However, since this assessment is based directly on
the ILOs, students’ performance on this test would be a good predictor of how they would fare
on a similar standardized assessment. Individual students’ scores should be consistent between
the two tests or other factors such as content and construct validity would need to be reassessed
(Gareis & Grant, 2008). It is important to note that performance on one assessment may not
always be predictive of another assessment with similar content if there is random error such as
differences in the testing environments.
Scoring and Grading Procedures
This assessment will be worth a total of 18 points. The select response questions (except
for item one) are worth one point each. Item one is worth two points since there are two correct
responses. Students will get one point for each need identified. The supply response items vary
in weight. Item eight is worth two points (each correct ordinal number is worth .5 points), item
nine is worth two points, and item ten is worth six points as specified in the supply response
rubrics (See Appendix E).
In order to remain objective when grading, the teacher will follow the detailed criteria in
the rubrics and highlight the criteria that the student meets. The teacher will score the students’
tests anonymously by covering up the students’ names. This will limit unintended bias based on
past experiences with that student (Gareis & Grant, 2008). Also, the teacher will review scored
responses at a later time to improve intra-rater reliability, and ensure that mood or time of day
TEST CREATION ASSIGNMENT 8
did not affect the scoring procedures. Finally, the teacher will score each supply response item
for all students before moving on to the next question. This will help to get a general picture of
the students’ responses and be more consistent when scoring. Afterward, the teacher will record
the total number of points received for each student. Since Kindergarteners do not receive letter
grades, the test will not affect their final grades for science, but it will provide information on
student learning that can dictate the effectiveness of instruction and be shared with parents
during conferences.
Part III: Test and Answer Key
Plant Life Cycles Unit Test
See Appendix C
Test Answer Key
See Appendix D
TEST CREATION ASSIGNMENT 9
References
Gareis, C. R., & Grant, L. W. (2008). Teacher-Made Assessments: How to Connect Curriculum,
Instruction, and Student Learning. Larchmont, NY: Eye on Education, Inc.
Running Head: TEST CREATION ASSIGNMENT 10
Appendix A: Unpacking the Curriculum
Intended Learning Outcome
Underline the content and circle the word(s) that provides an
indication of the cognitive level(s)
Content
List the explicit, implied, and conditional content
(p. 58-59)
Cognitive Level on Blooms’
Taxonomy Provide the cognitive level(s) in
terms of Bloom’s Taxonomy (p. 61)
The student will describe the life needs of plants.
The life needs are nutrients, water, air, light, and a
place that has adequate space for them to grow.
Explicit – the life needs of plants (nutrients, water,
air, light, adequate space to grow)
Implied – what a plant is, what nutrients, air, light,
and space are, what a life need is
Conditional – none
“describe”= knowledge
(remembering specific terms)
The student will predict what will happen to
animals and plants if life needs are not met.
Explicit – what will happen to plants and animals
Implied – what the life needs of plants are, cause
and effect, how to make predictions
Conditional – if life needs are not met
“predict”= evaluation
(making judgments)
The student will describe some simple changes
that plants undergo during their life cycles. This
may include size, presence of leaves and branches,
fruits, and seeds.
Explicit – simple changes that plants undergo
during their life cycles
Implied – what is a life cycle, what are leaves,
branches, fruit, and seeds, define: change
Conditional – none
“describe”= comprehension
(need to understand the meaning
of content)
The student will compare and contrast young plants
and animals with their parents, using pictures and/or
live organisms.
Explicit – compare and contrast young plants and
animals with their parents
Implied – how to compare and contrast, difference
between parents and young
Conditional – using pictures and/ or live organisms
“compare and contrast”=
analysis (understand relationships)
TEST CREATION ASSIGNMENT 11
Appendix B: Table of Specifications
Content
Bloom’s Taxonomy
Knowledge Comprehension Application Analysis Synthesis Evaluation
The student will describe the life needs
of plants. The life needs are nutrients,
water, air, light, and a place that has
adequate space for them to grow.
X
1, 3, 7
S
The student will predict what will
happen to animals and plants if life
needs are not met.
X
6, 10
M
The student will describe some simple
changes that plants undergo during their
life cycles. This may include size,
presence of leaves and branches, fruits,
and seeds.
X
2, 4, 5, 8
S
The student will compare and contrast
young plants and animals with their
parents, using pictures and/or live
organisms.
X
9
L
Indicate the emphasis of the instructional content and importance for each item. S= Strong M=Moderate L=Low
Running Head: TEST CREATION ASSIGNMENT 12
Appendix C: Plant Life Cycles Unit Test
Note: The teacher will read the directions, and all words in questions 1-6. In questions 7-10, the
teacher is prompted on what sections to read to the students.
Name ___________________________________________________
Directions: Circle the correct response(s).
1. Circle 2 things that plants need to survive.
2. A plant begins its life cycle as a __________ .
A. flower
B. seed
C. soil
D. fruit
3. A plant needs air to grow.
True
False
TEST CREATION ASSIGNMENT 13
4. Which of the following pictures shows a seedling?
A.
B.
C.
D.
5. The arrow points to the plant’s ____________.
A. stem
B. roots
C. flower
D. leaves
TEST CREATION ASSIGNMENT 14
6. What will happen if a plant does not have enough space?
A. It will grow big and strong.
B. It will lose its leaves.
C. It will grow fruit.
D. It will die.
7. Plants get nutrients from the soil.
True
False
8. The teacher reads: Put the stages of the plant life cycle in order by writing
the numbers 1 to 4 in the boxes below. Write the number 1 below the first stage,
the number 2 below the second stage, and so on.
TEST CREATION ASSIGNMENT 15
9. The teacher reads: Write one way that the adult plant is different from the
young plant.
Write your response below.
__________________________
__________________________
__________________________
__________________________
Adult Plant Young Plant
TEST CREATION ASSIGNMENT 16
10. The teacher reads: Predict what will happen to this seedling if it is kept in a
dark closet and not watered. Draw a colored picture of what the plant will look like
in ten days. Then write a sentence explaining what will happen to the plant and why.
The plant looks like this today. The plant will look like this in 10 days.
Draw your picture here.
Write your sentence below.
_________________________________
_________________________________
_________________________________
_________________________________
_________________________________
TEST CREATION ASSIGNMENT 17
Appendix D: Test Answer Key
Name ___________________________________________________
Directions: Circle the correct response(s).
1. Circle 2 things that plants need to survive.
2. A plant begins its life cycle as a __________.
A. flower
B. seed
C. soil
D. fruit
3. A plant needs air to grow.
True
False
TEST CREATION ASSIGNMENT 18
4. Which of the following pictures shows a seedling?
A.
B.
C.
D.
5. The arrow points to the plant’s ____________.
A. stem
B. roots
C. flower
D. leaves
TEST CREATION ASSIGNMENT 19
6. What will happen if a plant does not have enough space?
A. It will grow healthy and strong.
B. It will lose its leaves.
C. It will grow fruit.
D. It will not grow.
7. Plants get nutrients from the soil.
True
False
8. The teacher reads: Put the stages of the plant life cycle in order by writing
the numbers 1 to 4 in the boxes below. Write the number 1 below the first stage,
the number 2 below the second stage, and so on.
1 2 3 4
TEST CREATION ASSIGNMENT 20
9. The teacher reads: Write at least one way that the adult plant is different
from the young plant.
Write your response below. (SAMPLE RESPONSE)
The adult tree is bigger and has fruit.
10. The teacher reads: Predict what will happen to this seedling if it is kept in a
dark closet and not watered. Draw a colored picture of what the plant will look like
in ten days. Then write a sentence explaining what will happen to the plant and why.
The plant looks like this today. The plant will look like this in 10 days.
Draw your picture here.
Write your sentence below. (SAMPLE RESPONSE)
_The seedling will die because plants___
need sunlight and water to grow.______
Adult Plant Young Plant
TEST CREATION ASSIGNMENT 21
Appendix E: Supply Response Rubrics
Holistic Rubric (To be used for item 9)
In Full Bloom (2 points)
The student identified more than one difference between
the young plant and the adult plant. (fruit, size, branches)
Budding Scientist (1 point)
The student identified one difference between the young
plant and the adult plant. (fruit, size, branches)
Needs Room to Grow (0 points)
The student did not identify a difference between the two
plants.
Analytical Rubric (To be used for item 10)
Criteria
In Full Bloom
Budding Scientist
Needs Room to Grow
Picture
The student provides a
detailed and colored
drawing of a wilted/
dead seedling in the
planter. (3pts)
The student’s drawing
shows a wilted/dead
plant, but the plant is
not a seedling or is
shown in a different
environment. (2pts)
The student’s drawing
is off topic or shows a
healthy plant. (0pts)
Sentence
The student writes a
sentence that explains
that the plant will die
because it needs light/
sunlight and water to
survive. (3pts)
The student writes that
the plant will die, but
doesn’t explain why.
(2pts)
The student’s sentence
is incorrect (says the
plant will be healthy or
specifies the wrong life
needs). (0pts)