LOGO

Dinh Cong Triet

TEST DESIGNING &

TEST SPECIFICATION BUILDING

Summer training course- August 2011

10 Questions for thought

WHAT

to test?

Skills

Speaking Reading

Listening Writing

Knowledge

Language

Grammar

lexis

semantics

syntax

morphology

Reading

WHAT

to test? Reading aloud Silent reading

pronunciation

Stresses & intonation• Identify main facts & details,

• Relate cause & effect

• Identify sequence of events, predicting outcomes

• inferring meaning from contextual clues

Subskills

Listening

WHAT

to test?

• Identify main facts & details,

• Relate cause & effect

• Identify sequence of events, predicting outcomes

• inferring meaning from contextual clues

Subskills

Grammar

WHAT

to test?

Syntax (sentence structure)

Mechanics: e.g Punctuation, capital letter, comma, etc..

Rules of grammar

pronouns

nouns

verbs

subject & verb agreement

prepositions

tenses

adjectives adverbscomparatives

Speaking

WHAT

to test?

Language

Presentation skills

Content / ideas

Pronunciation

Organization

quantity quality

clarity

accuracy

appropriacy format

vocabulary Sentence complexity

syntax grammar

fluency confidence

Eye contact style

Writing

WHAT

to test?

LanguageContent / ideas

Organization

quantityquality

clarity

accuracy

appropriacy

Paraphrasing

voc Sentence complexity

syntax

Topic sentence Supportive sentences

compoundSimple complex

WHAT

to test?

Skills

Speaking Reading

Listening Writing

Knowledge

Language

Grammar

lexis

semantics

syntax

morphology

Reading

WHAT

to test? Reading aloud Silent reading

pronunciation

Stresses & intonation• Identify main facts & details,

• Relate cause & effect

• Identify sequence of events, predicting outcomes

• inferring meaning from contextual clues

Subskills

Listening

WHAT

to test?

• Identify main facts & details,

• Relate cause & effect

• Identify sequence of events, predicting outcomes

• inferring meaning from contextual clues

Subskills

Grammar

WHAT

to test?

Syntax (sentence structure)

Mechanics: e.g Punctuation, capital letter, comma, etc..

Rules of grammar

pronouns

nouns

verbs

subject & verb agreement

prepositions

tenses

adjectives adverbscomparatives

Speaking

WHAT

to test?

Language

Presentation skills

Content / ideas

Pronunciation

Organization

quantity quality

clarity

accuracy

appropriacy format

vocabulary Sentence complexity

syntax grammar

fluency confidence

Eye contact style

Writing

WHAT

to test?

LanguageContent / ideas

Organization

quantityquality

clarity

accuracy

appropriacy

Paraphrasing

voc Sentence complexity

syntax

Topic sentence Supportive sentences

compoundSimple complex

What makes test questions difficult?

THINKING LEVELS

Designing test questions in accordance with the

levels of thinking

Bloom’s taxonomy

Old version, 1956, Benjamin Bloom

New version, 1990’s , Lorin Anderson

BLOOM’S TAXONOMY.

Knowledge

Comprehension

Application

Analysis

Synthesis

Evaluation

Arrange, define, list, memorise, name, relate, recall, repeat, state

Classify, describe, discuss, explain, express, recognise, identify, report

Apply, choose, demonstrate, illustrate, solve, use, interpret

Analyse, calculate, categorise, criticise, contrast, distinguish

Arrange, collect, compose, plan, construct, create, design, develop

Argue, assess, justify, judge, rate, support, value, evaluate

Useful verbs Sample questions stems

TellListDescribeRelateLocateWriteFindStatename

• What happened after….?• How many…..?• Who was it that……..?• Can you name the …….?• Describe what happened at…..?• Who spoke to…..?• Can you tell why….?• Find the meaning of …..?• What is …….?• Which is true or false…….?

Bloom’s taxonomy – Knowledge level

Useful verbs Sample questions stems

ExplainInterpretOutlineDiscussDistinguishPredictRestateCompareTranslateDescribe

• Can you write in your own words….?

• Can you write a brief outline….?• What do you think could happen

next….?• Who do you think….?• What was the main idea…..?• Who was the key character…..?• Can you distinguish between….?• Can you provide an example of …..?• What do you mean ….?• Can you provide a definition for….?

Bloom’s taxonomy – Comprehension level

Useful verbs Sample questions stems

Solve ShowUseillustrateConstructCompleteExamineClassify

• Do you know another instance where….?

• Could this have happened in ….?• Can you group by charateristics such

as….?• What factors would you change if…..?• What questions would you ask of…..?• From the information given, can you

develop a set of instructions about……?

• Would this information be useful if you had….?

Bloom’s taxonomy – Application level

Useful verbs Sample questions stems

AnalyseExamineCompareContrastInvestigateCategoriseIdentifyExplainSeparate

• Which events could have happened….?

• What do you see as other possible outcomes…?

• How was this similar to….?• Can you explain what must have

happened when…?• What are some of the problems of ….?• Can you distinguish between….?• What were some of the motives

behind….?

Bloom’s taxonomy – Analysis level

Useful verbs Sample questions stems

CreateInventComposePredictPlanConstructDesignImaginePropose devise

• Can you design a…..to….?• Can you see a possible solution to ….?• If you had access to all resources how

would you deal with….?• What would happen if …..?• How many ways can you……?• Can you write a new recipe for a tasty

dish?• Why not compose a song about…..?

Bloom’s taxonomy – Synthesis level

Useful verbs Sample questions stems

JudgeSelectChooseDecideJustifyDebateVerifyArgueRecommenddetermine

• Is there a better solution to ….• Judge the value of….• Can you defend your position

about…..?• Do you think……is a good or bad

thing?• What changes to ……would you

recommend?• Do you believe….?• Are you a …..person?• How would you feel if….?• How effective are…..?• What do you think about…….?

Bloom’s taxonomy – Evaluation level

BLOOM’S TAXONOMY.

Skills Subskills Question types Levels

BLOOM’S TAXONOMY.

Skills Subskills Question types Levels

Skills Sub-skills Question types Question levels

Reading Identifying main facts & details (FD)

Recognizing cause & effects relationship (CE)

Inferring meaning from contextual context ( ICC)

Predicting outcomes (PO) Sequence of events (SOE) Evaluating writer’s attitude &

view

True/ false Re-arrangement Structured Open-ended (subjective

response, free writing) MCQ (wh-question,

matching, completion with options)

Cloze procedure

Listening Identifying facts & details relate cause & effect identify sequence of events Inferring meaning from

contextual clues. Predicting out comes Inferring speaker’s attitude &

emotion (sad, happy, worried) Inferring relationship between

speakers.

MCQ with pictures or without pictures.

Fill in the blanks/ completion item.

Labelling. Matching True / false. sequencing

BARRETT’S TAXONOMY.

Table of Specifications

Question levels

Bloom’s(Know,Comp, App, Ana, Syn, Eva)

Barrett’s (LC, RO, IC, Eva, Appr)

Skills & subskills

Mark allocation

Question

Sections/numbers

Time allocation

Curriculum specs

knowledge

Question types

(MCQ, T/F, gap-fill, matching, sequencing, open-ended)

LSRW

LOGO

Dinh Cong Triet