8/3/2019 Construct Test Item
1/39
Evaluation of Student Learning:
Test Construction &Other Practical Strategies
8/3/2019 Construct Test Item
2/39
Guiding Principles for Evaluation
Evaluation should relate directly to instructional
objectives
Each evaluation activity should be designed to promote
student growth
The actual activity should be useful practice in itself
Feedback should be useable by the student
Multiple evaluation strategies should be provided to
master achievement of X objective/competency
Student should clearly understand the methods of
evaluation for X test or activity
8/3/2019 Construct Test Item
3/39
Questions to Ask yourself in Designing
a Test
What objectives will (should) I be testing?
What types of items will be included in the
test?
How long will the test be in terms of time andnumber of items?
How much will each objective be worth in
terms of weighting and number of items?
8/3/2019 Construct Test Item
4/39
Tests as Diagnostic Tools
Students demonstrate learning
Instructor effectiveness modify teaching
strategies or activities
8/3/2019 Construct Test Item
5/39
Different Types of Tests & Learning
Paper & Pencil/Web Testing
Limited Choice Questions (MC, T/F, Matching)
Open-Ended Questions (Short Answer, Essay)
Performance Testing Acquisition of skills that can be demonstrated
through action (e.g., music, nursing, etc.)
8/3/2019 Construct Test Item
6/39
Planning a Test
First step: Outline learning objectives or
major concepts to be covered by the test
Test should be representative of objectives
and material covered Major student complaint:Tests dont fairly
cover the material that was supposed to be
canvassed on the test.
8/3/2019 Construct Test Item
7/39
Planning a Test
Second Step: Create a test blueprint
Third Step: Create questions based on blueprint Match the question type with the appropriate level
of learning Fourth Step: For each check on the blueprint,
3-4 alternative question on ideas and itemtypes which will get at the same objective
Fifth Step: Organize questions and/or ideas byitem types
8/3/2019 Construct Test Item
8/39
Planning a Test
Sixth Step: Eliminate similar questions
Seventh Step: Reread all of the items try
doing this from the standpoint of a student
8/3/2019 Construct Test Item
9/39
Planning a Test
Eight Step: Organize questions logically
Ninth Step: Time yourself actually taking the
test and then multiply that by about 4
depending on the level of students Tenth Step: Analyze the results (item analyses)
8/3/2019 Construct Test Item
10/39
Translating Course
Objectives/Competencies into
Test Items Syllabus
Specification table- what was taught/weight areas to be tested
Creating a Test Blueprint (see handout)
Blueprint- this is the test plan, i.e., which questions test whatconcept
Plotting the objectives/competencies against some hierarchyrepresenting levels of cognitive difficulty or depth ofprocessing
8/3/2019 Construct Test Item
11/39
Thinking Skills
What level of learning corresponds to the
course content
Blooms Taxonomy of Educational Objectives
Knowledge (see handout)
Comprehension
Application
Analysis
Synthesis
Evaluation
8/3/2019 Construct Test Item
12/39
Practical Considerations
Representative sample of the course content
not random purposeful based on blueprint
Representative sample of skill or cognitive
levels across content Analyze results by level AND content area
8/3/2019 Construct Test Item
13/39
Question Arrangement on a Test
Group by question type
Common instructions will save reading time
Limit the number of times students have to
change frame of reference Patterns on test must be logical
Arrange from a content standpoint
Keep similar concepts together
Group by difficulty (easy to hard)
8/3/2019 Construct Test Item
14/39
Selecting the Right Type of evaluation
How do you know what type of question to useand when?
It depends on the skill you are testing.
Evaluation should always match as closely aspossible the actual activity youre teaching. Examples: Teaching Speech, should evaluate an oral
speech
If testing ability to write in Spanish, better give anessay.
Testing reading MC, TF
Wouldnt use MC to test creative writing
8/3/2019 Construct Test Item
15/39
Question Types verses
Cognitive Levels of Learning
Knowledge
Comprehension
Application Analysis
Synthesis
EvaluationMultiple Choice (MC)
True/False (TF)
Matching
Completion
Short Answer
MC
Short Answer
Problems
Essay
Performance
MC
Short Answer
Essay
8/3/2019 Construct Test Item
16/39
Constructing the Test
Types of Test Questions:
Multiple-Choice Items
True-False Items
Matching Items Fill-In, Completion or Short-Answer Items
Essay Questions
8/3/2019 Construct Test Item
17/39
Multiple Choice Items
Advantages:
Extremely versatile-can measure the higher level
mental processes (application, analysis, synthesis
and evaluation)
A compromise between a short answer/essay and
T/F item
Can cover a wide range of content can be sampled
by one test Disadvantages
Difficult to construct plausible alternative responses
8/3/2019 Construct Test Item
18/39
Types of Multiple Choice Items
Four Basic Types
Question Type
Incomplete Statement Type
Right Answer Type Best Answer Type
Which Type is Best?
Question Type vs. Incomplete Statement
Right Answer vs. Best Answer Type
8/3/2019 Construct Test Item
19/39
Multiple Choice Items
1. Writing the stem first:
A. Be sure the stem asks a clear question
B. Stems phrased as questions are usually easier to
writeC. Stems should not contain a lot of irrelevant info.
D. Appropriate reading level/terms
E. Be sure the stem is grammatically correct
F. Avoid negatively stated stems
8/3/2019 Construct Test Item
20/39
Multiple Choice Items
Writing the correct response
Use same terms/reading level
Avoid too many qualifiers
Assign a random position in the answer sequence Read the stem and correct response together
Generate the distractors/alternative responses
8/3/2019 Construct Test Item
21/39
Multiple Choice Items
Other Tips for Constructing MC Items: Items should have 3-4 alternatives.
Stem should present a single, clearly formulatedproblem
Simple, understandable, exclude extraneous wordsfrom both stem and alternatives
Include in the stem any word that are repeated ineach response
Avoid all of the above (can answer based on partialinformation)
Avoid none of the above
8/3/2019 Construct Test Item
22/39
Multiple Choice Items
Alternative responses/distractors should be plausible
and as homogeneous as possible
Response alternatives should not overlap
Two synonymous terms (arithmetic average/mean)
Avoid double negatives
None of the following are part of the brain except which one?
Emphasize negative wording
Each item should be independent of other items in thetest
Information in the stem of one item should NOT help answer
another item.
8/3/2019 Construct Test Item
23/39
True-False Test Items
Best suited for testing 3 kinds of info.: Knowledge level learning
Understanding of misconceptions
When there are two logical responses
Advantages: Sample a large amount of learning per unit of
student testing time
Disadvantages:
Tends to be very easy 50-50 chance of guessing
Tends to be low in reliability
8/3/2019 Construct Test Item
24/39
Tips for Constructing True/False Items
Tips for constructing True-False Items
Avoid double negatives
Avoid long or complex sentences
Include only one central idea in each statement
Exact quantitative (two, three, four) language is betterthan qualitative (some, few, many)
Avoid a pattern of answers
8/3/2019 Construct Test Item
25/39
Objective Test Item Analyses
Evaluating the Effectiveness of Items..
Why?
Scientific way to improve the quality of tests and test items
Identify poorly written items which mislead students
Identify areas (competencies) of difficulty
Item analyses provided info. on:
Item difficulty
Item discrimination
Effectiveness of alternatives in MC Tests
8/3/2019 Construct Test Item
26/39
Short-Answer Items
Two Types: (Question and Incomplete Statement)
Advantages:
Easy to construct
Excellent format for measuring who, what, when, and where
info.
Guessing in minimized
Student must know the material- rather than simply recognize
the answer
Disadvantages: Grading can be time consuming
More than one answer can be correct
8/3/2019 Construct Test Item
27/39
Short Answer Items
Tips for Constructing Short Answer Items Better to supply the term and require a definition
For numerical answers, indicate the degree ofprecision expected and the units in which they are
to be expressed. Use direct questions rather than incomplete
statements
Try to phrase items so that there is only one possible
correct response When incomplete statements are used, do not use
more than one blank within an item.
8/3/2019 Construct Test Item
28/39
Essay Questions
Types of Essay Questions
Extended Response Question
Great deal of latitude on how to respond to a question.
Example: Discuss essay and multiple-choice type tests.
Restricted Response Question
More specific, easier to score, improved reliability and validity
Example: Compare and contrast the relative advantages of
disadvantages of essay and multiple choice tests with respect
to: reliability, validity, objectivity, & usability.
8/3/2019 Construct Test Item
29/39
Essay Items
Advantages: Measures higher learning levels (synthesis,
evaluation) and is easier to construct than anobjective test item
Students are less likely to answer an essay questionby guessing
Require superior study methods
Offer students an opportunity to demonstrate their
abilities to: Organize knowledge
Express opinions
Foster creativity
8/3/2019 Construct Test Item
30/39
Essay Items
May limit the sampling of material covered Tends to reduce validity of the test
Disadvantages
Subjective unreliable nature of scoring halo effect good or bad students previous level of
performance
Written expression
Handwriting legibility
Grammatical and spelling errors
Time Consuming
8/3/2019 Construct Test Item
31/39
Essay Questions
Give students a clear idea of the scope & directionintended for the answer Might help to start the question with the description of the
required behavior (e.g., compare, analyze)
Appropriate language level for students Construct questions that require students to
demonstrate a command of background info, but do notsimply repeat that info.
If question calls for an opinion, be sure that the
emphasis is not on the opinion but on the way itspresented or argued.
Use a larger number of shorter, more specific questionsrather than one or two longer questions so that more
information can be assessed.
8/3/2019 Construct Test Item
32/39
Essay Questions
You might Give students a pair of sample answers to a question of the
type you will give on the test.
Sketch out a rubric (grading scheme) for each question beforereading the papers OR randomly select a few to read and make
up the grading scheme based on those answers Give students a writing rubric
Detach identifying information and use code numbers insteadto avoid letting personality factors influence you.
After grading all the papers on one item, reread the first few to
make sure you maintained consistent standards Be clear to student the extend to which factors other than
content (e.g., grammar, handwriting, etc.) will influence thegrade.
8/3/2019 Construct Test Item
33/39
Essay Questions
Tips for constructing Essay Questions
Provide reasonable time limits for each question
thinking and writing time
Avoid permitting students a choice of questions Will not necessarily get a representative sample of student
achievement. Only be requiring all students to answer all
questions can their achievement be compared
A definite task should be put forth to the student
Critical words: compare, contrast, analyze, evaluate, etc.
8/3/2019 Construct Test Item
34/39
Scoring Essay Items
Write an outline of the key points (use outline to design
a rubric)
Determine how many points are to be assigned to the
question as a whole and to the various parts within it.
If possible, score the test without knowledge of the
students name
Face Sheet
Score all of the answers to one question before
proceeding to the next question Consistent standard
8/3/2019 Construct Test Item
35/39
Scoring Essay Exams
If possible, score each set of answers within the
same time frame
Handwriting, spelling & Neatness
Two separate grades? Mastery of material
Other
8/3/2019 Construct Test Item
36/39
Alternative Methods of Assessment
Research/Term Papers
Research Reviews
Reports
Case Studies Portfolios
Projects
Performances
Peer evaluation Mastery
Simulations
8/3/2019 Construct Test Item
37/39
Cheating
Preventing Cheating
Reduce the pressure (multiple evaluations)
Make reasonable demands (length/content of exam)
Use alternative seating Use alternative forms
Be cautious with extra copies
8/3/2019 Construct Test Item
38/39
Using Assessment & Evaluation to
Improve Student Learning Outcomes
Providing feedback to student
Closing the assessment & evaluation loop
Maximizing student learning
8/3/2019 Construct Test Item
39/39
Questions?
Top Related