TESTING WRITING Carolina Salazar Testing and Evaluation

THE TESTING PROBLEM

REPRESENTATIVE

VALIDLY SAMPLED

VALIDLY AND RELIABLY SCORED

REPRESENTATIVE

REPRESENTATIVE TASKS

SPECIFICATION OF ALL THE POSSIBLE CONTENT

Operations

Types of text

Adressees

Length of texts

Topics Dialect Style

Expressing,

directing, describing

…

Form, letter, fax,

note…

Boss, friends, family…

1 page, 150

words…Not

specialist,

relevant…

Standard

Formal, informal

…

REPRESENTATIVE SAMPLE OF THE SPECIFIED CONTENT

Try to include a representative sample of tasks

MORE TASKS

MORE VALIDI

TY

EXAMPLES

CCSE (CAMBRIDGE CERTIFICATES IN COMMUNICATIVE SKILLS IN ENGLISH)

FIRST STAGE: READING

TOPIC: Working in a Summer Camp for Children in America

TASK 1

Write a letter to American Summer Camps.

Find out about Dates

Hours of work Type of accomodation

Ask for the application form

TASK 2

Fill in the application form.

TASK 3

You are now working in the ASCFC. You write a postcard to

an English-speaking friend

On your postcard tell your friend:

Where you are Why you are there

Two things you like about the summer camp

TASK 4

You have arranged to go out tonight with two other Helpers

at the ASCFC in Florida. You have to change your plans suddenly, and cannot meet them. Leave them a note. Apologize and explain why you

cannot meet them Suggest a different day to go out

ANALYSIS

Representative sample

of tasks?

Content validity?

Practical?

THE TESTING PROBLEM

REPRESENTATIVE

VALIDLY SAMPLED

VALIDLY AND RELIABLY SCORED

VALIDLY SAMPLED

VALID SAMPLE OF WRITING ABILITY

SET AS MANY SEPARATE TASKS AS FEASIBLE People’s performance Consistent

Offer them more opportunities to start

Desirable

Practical

TEST ONLY WRITING, NOTHING ELSE

NO YES

Creativity

Intelligence

Imagination

Writing

TEST ONLY WRITING, NOTHING ELSE

READING

ILLUSTRATIONS

SERIES OF PICTURES

GRAPHICS – CHARTS

NOTES

THE TESTING PROBLEM

REPRESENTATIVE

VALIDLY SAMPLED

VALIDLY AND RELIABLY SCORED

VALIDLY AND RELIABLY SCORED

VALID AND RELIABLE SCORING

ENSURE VALID AND RELIABLE SCORING Set tasks which can be reliably scored

Set as many tasks as posible

Restrict candidates

Give no choice of tasks

Ensure long enough samples

Create appropriate scales for scoring

HOLISTIC SCORING

“Impressionistic”

Overall impression

Purpose of the test

PRO

Very rapid

CON

Scorer reliability?

http://www.ets.org/toefl/pbt/scores/writing_score_guide/

ANALYTIC SCORING

http://www.ode.state.or.us/wma/teachlearn/testing/scoring/guides/2011-12/wriscorguide_eng_no-dates.pdf

Separate score – separate aspects

PROS

Uneven development of subskills

Scorers consider details of performance

More reliable

CONS

Time consuming

Too specific (The whole is greater than the sum of its parts)

HOLISTIC OR ANALYTIC?

PURPOSECIRCUMSTANCE

S

ACCURACY RESOURCES

DECISIONMULTIPLE SCORING

MORE CONSIDERATIONS ON

SCORING

SOME CONSIDERATIONS Calibrate the scale Collect samples of performance Cover the range of scales

Select and train scorers Native speakers Sensitive to language Experienced Training (3 stages)

Follow acceptable scoring procedures Score each task independently Several scorers Identify discrepancies

Give feedback Useful – positive Decide the content during callibration

CONCLUSIONS

CONCLUSIONS

Pay attention to validity and reliability

Give importance to writing as an independent skill

Always be meaningful and enhance real communication

SOURCES

http://www.ode.state.or.us/wma/teachlearn/testing/scoring/guides/2011-12/wriscorguide_eng_no-dates.pdf

http://www.ets.org/toefl/pbt/scores/writing_score_guide/