Designing the Test and Test Questions Jason Peake.

34
Designing the Test and Test Questions Jason Peake

Transcript of Designing the Test and Test Questions Jason Peake.

Page 1: Designing the Test and Test Questions Jason Peake.

Designing the Test and Test Questions

Jason Peake

Page 2: Designing the Test and Test Questions Jason Peake.

Principles of Testing

Validity Does the test measure what it is supposed to

measure? Does the test require student performance as

described by the objective?

Page 3: Designing the Test and Test Questions Jason Peake.

Principles of Testing

Reliability Does the test generate the same results each

time it is given? Is the test affected by misinterpretation

Poorly written directions Vague questions Misleading words Not sure of response

Page 4: Designing the Test and Test Questions Jason Peake.

Why do we evaluate students?

Motivate students

Measures achievement

Identify areas for review

Check effectiveness of materials, teaching method, and teacher

Assign grades

Page 5: Designing the Test and Test Questions Jason Peake.

Agenda

1. Aligning evaluation with level of learning.

2. Developing a test blueprint and preparing to write items

3. Common pitfalls when writing test questions

4. Writing questions at various levels of performance

Page 6: Designing the Test and Test Questions Jason Peake.

Defining objectives using Bloom’s Taxonomy

CATEGORY• Evaluation• Synthesis• Analysis

• Application• Comprehension• Knowledge

SKILL• Judge• Combine• Break apart

• Apply principles• Understanding• Recall

Page 7: Designing the Test and Test Questions Jason Peake.

Sample Verbs

Knowledge Cite, define, identify, label, list, match, name,

recognize, reproduce, select, state

Comprehension Classify, convert, describe, distinguish between,

explain, extend, give examples, illustrate, interpret, paraphrase, summarize, translate

Application Apply, arrange, compute, construct, demonstrate,

discover, modify, operate, predict, prepare, produce, relate, show, solve, use

Page 8: Designing the Test and Test Questions Jason Peake.

Sample Verbs continued

Analysis Analyze, associate, determine, diagram, differentiate,

discriminate, distinguish, estimate, infer, order, outline, point out, separate, subdivide

Synthesis Combine, compile, compose, construct, create, design,

develop, devise, formulate, integrate, modify, organize, plan, propose, rearrange, reorganize, revise, rewrite, tell, write

Evaluation Appraise, assess, compare, conclude, contrast, criticize,

discriminate, evaluate, judge, justify, support, weigh

Page 9: Designing the Test and Test Questions Jason Peake.

Agenda

1. Aligning evaluation with level of learning.

2. Developing a test blueprint and preparing to write items

3. Common pitfalls when writing test questions

4. Writing questions at various levels of performance

Page 10: Designing the Test and Test Questions Jason Peake.

Test blueprint (specifications)

• Matrix of number of test questions by topic and level of objective

• Helps you plan your test• Do before writing items

Page 11: Designing the Test and Test Questions Jason Peake.

Example blueprint

Topic A Topic B Topic C Topic D Total

Knowledge 1 2 1 1 5 (20%)

Comprehension 2 1 1 2 6 (24%)

Application 2 1 1 1 5 (20%)

Analysis 1 1 2 2 6 (24%)

Synthesis 1 1 2 (8%)

Evaluation 1 1 (4%)

Total 6

(24%)

6

(24%)

6

(24%)

7

(28%)

25

(100%)

Page 12: Designing the Test and Test Questions Jason Peake.

Agenda

1. Aligning evaluation with level of learning.

2. Developing a test blueprint and preparing to write items

3. Common pitfalls when writing test questions

4. Writing questions at various levels of performance

Page 13: Designing the Test and Test Questions Jason Peake.

Preparing to write items

• Create test items while preparing class lessons• Make note of questions asked frequently by

students • Make note of misconceptions made frequently

by students during class or homework• Invite students to submit items at the end of each

class or at other times• Pull questions from formative assessments.

Page 14: Designing the Test and Test Questions Jason Peake.

Agenda

1. Aligning evaluation with level of learning.

2. Developing a test blueprint and preparing to write items

3. Common pitfalls when writing test questions

4. Writing questions at various levels of performance

Page 15: Designing the Test and Test Questions Jason Peake.

Common pitfalls

• Providing unintended cues• Using trick questions• Using distractors that are obviously

incorrect• Using NOT or EXCEPT in the question• Including all of the above, none of the

above, or only A and B as option choices

Page 16: Designing the Test and Test Questions Jason Peake.

Preparing for test-wise students

• Longest answer• “C” as correct answer• Avoid picking “always” or “never”• Opposites• Scientific sounding• Avoid simple & obvious• Related word• Guess

Page 17: Designing the Test and Test Questions Jason Peake.

Agenda

1. Aligning evaluation with level of learning.

2. Developing a test blueprint and preparing to write items

3. Common pitfalls when writing test questions

4. Writing questions at various levels of performance

Page 18: Designing the Test and Test Questions Jason Peake.

Levels of performance

Lower levels (knowledge, comprehension, and application) Recognize the principle in altered format. Match component parts with vocabulary. Identify example seen before. Recognize an example never seen before. Solve a problem using familiar format, but different specifics.

Higher levels (analysis, synthesis, evaluation) Pick out components of a situation. Problem solve by combining elements. Make judgments of value.

Page 19: Designing the Test and Test Questions Jason Peake.

Categories of Test Questions

Test can roughly be broken into 1. Objective

Multiple choice Matching Completion True false

2. Subjective Short answer and/or Essay Oral test Case Study

Page 20: Designing the Test and Test Questions Jason Peake.

Sample multiple choice items

Lower levels Which of the following is an example of erosion? (choices

include one or more used in class) Identify the definition of water erosion? Which of the following is an example of erosion? (choices

never seen in class) Here is an erosion scenario. Which of the following principles

apply?

Higher levels Which of the following solutions would be the best to apply in

the scenario described below to solve the erosion problem? Here is an erosion problem situation. Identify which principle

from this class could be used to solve it.

Page 21: Designing the Test and Test Questions Jason Peake.

Sample multiple choice distracters

A term used in the unit but referring to a different concept Cause of erosion – no till farming

An everyday term that is close but misleading in meaning Cause of erosion – hydrofoil

A common misconception or confusion students have Cause of erosion – lack of diverse soil particle size

A plausible word or phrase that is meaningless but derives from something in the stem

Cause of erosion – systemic erosion syndrome

Page 22: Designing the Test and Test Questions Jason Peake.

Matching

Appropriate for only knowledge level testingUse a reasonable number of items 7 – 12All premises in one list, all responses in another listPremises can be long, but try to keep responses shortUse more responses than premises

Page 23: Designing the Test and Test Questions Jason Peake.

Clear Directions

____ 1. List premise

____ 2. List premise

____ 3. List Premise

____ 4. List Premise

____ 5. List Premise

____ 6. List Premise

A. Response

B. Response

C. Response

D. Response

E. Response

F. Response

G. Response

Page 24: Designing the Test and Test Questions Jason Peake.

Completion Items

Most useful for lower order levels of learning

Use your own words

Be sure only one answer can apply

Keep all blanks the same length

Avoid grammatical clues

Page 25: Designing the Test and Test Questions Jason Peake.

Complete the meaning of each statement by writing the correct word in the corresponding numbere blank

1. A thermometer should be rinsed in _____________ water.

2. The NPK in fertilizers stands for _____________, _____________, and _____________

Page 26: Designing the Test and Test Questions Jason Peake.

True / False Questions

Can be used to check for knowledge and comprehension

Keep questions simple

Make entirely true or entirely false

Avoid double negative items

Page 27: Designing the Test and Test Questions Jason Peake.

Writing True / False Questions

For each of the statements below circle “T” for true and circle “F” for false. T F Swine gestation period is 33 days.

For each of the statements below circle “T” for true and circle “F” for false and explain in the blank why it is false. T F Swine gestation period is 33 days.

_________________________________

Page 28: Designing the Test and Test Questions Jason Peake.

Subjective Test Questions

Short answer and/or Essay Oral test Case Study

Page 29: Designing the Test and Test Questions Jason Peake.

Short Answer and/or Essay

May require many different behaviors:

Comparison, for or against, cause and effect, explanation, summary, analysis, describe relationships, apply rules, or discussion

Page 30: Designing the Test and Test Questions Jason Peake.

Oral Test

All students must receive the same item stated in clear concise manner designed to elicit correct answers arranged in logical order.

Page 31: Designing the Test and Test Questions Jason Peake.

Case Study

Problem is practical and realistic

Requires comprehension of prior learning and application of that learning

Page 32: Designing the Test and Test Questions Jason Peake.

Subjective Test Questions

Short answer and/or Essay Oral test Case Study

Theses subjective test questions or subjective test require a criteria to be graded. We will work on grading criteria tomorrow.

Page 33: Designing the Test and Test Questions Jason Peake.

Which Type of Question to Use?

Select the appropriate technique to test the student competence at the level of cognition required by the objective.

Avoid one type of test or test questionWe know students have different strengths Visual Audio kinesthetic

Page 34: Designing the Test and Test Questions Jason Peake.

Materials in this presentation developed in part by…

Brian ParrUniversity of Georgia

Dr. Dawn ZimmaroUniversity of Texas