THE ASSESSMENT CYCLE, ASSESSMENT DESIGN AND SPECIFICATIONS

PROSET - TEMPUS 1

Prepared by Maria Verbitskaya, Angelika Kalinina, Elena Solovova

PROSET - TEMPUS 2

• Teachers want to know what parts of the course cause difficulties and require special attention.

• Teachers want to know what progress students have made.

• Students want to know their strengths and weaknesses.

• Parents want to know how their children are doing.

What makes assessment necessary?

What makes assessment necessary?

PROSET - TEMPUS 3

• Universities want to choose the best applicants for enrolment.

• Teachers want to know whether the students are coping with the programme.

• Employers want to choose the best applicant for the job.

THE ASSESSMENT CYCLE

The Assessment Cycle Designers

Producers

Organizers

AdministratorsAssessees

Scorers

Users

Validators

Decision

Script

Score report

Report

Guidelines

Form

Specifications

Components of the assessment cycle:

• designing a test and developing test specifications;• producing and working out the form of the test;• organizing a test and working out the guidelines;• test administration and organising the procedure;• preparing assessees for tests;• scoring the results of the test;• setting pass marks;• reporting outcomes of tests to stakeholders;• validation.

TYPES OF TEST AND ASSESSMENT

What forms of assessment do you use with your students?

PROSET - TEMPUS 8

• test• portfolio• project work• observation• case study• interview, oral questions• experimental work, etc.

Testing & assessment

PROSET - TEMPUS 9

Assessment

Testing

portfolios

observations

projects

exercises

class tests

national exams

What is a test?

PROSET - TEMPUS 10

Any procedure for measuring:•ability,•knowledge, •or performance. (Longman Dictionary of Language Teaching and Applied Linguistics)

What is a test?

PROSET - TEMPUS 11

“A test is a measurement instrument designed to elicit a specific sample of individual’s behaviour. … a test necessarily quantifies characteristics of individuals according to explicit procedures.” (L.F. Bachman. Fundamental Considerations in Language Testing, 1990)

Test as an instrument of assessment (measurement)

PROSET - TEMPUS 12

WHYWHATHOW

What types of tests do we use?

PROSET - TEMPUS 13

Test Purposes:• proficiency tests• achievement tests• placement tests• diagnostic tests.

What types of tests do we use?

PROSET - TEMPUS 14

• direct vs indirect;• discrete point vs integrated;• objective vs subjective;• low stakes vs high stakes;• norm-referenced vs criterion-referenced.

Norm-referenced testing: the bell curve

Numberofstudents

0 25 50 75 100

Score (points)

PROSET - TEMPUS 15

DESCRIBING TESTS AND ASSESSMENTS

The Assessment Cycle:

where does a test begin?

PROSET - TEMPUS 17

Designers

Producers

Organizers

AdministratorsAssessees

Scorers

Users

Validators

Decision

Script

Score report

Report

Guidelines

Form

Specifications

What kinds of information should be available about a test?

PROSET - TEMPUS 18

WHYWHATHOW

Test specification

PROSET - TEMPUS 19

‘the official statement about what the test tests and how it tests it … to be followed by test and item writers’ALTE Multilingual Glossary of Language Testing Terms.

CUP, 1998 •test design statement •blueprint•task and item specifications.

Test design statement

PROSET - TEMPUS 20

• the purpose, the knowledge, skills or abilities it is intended to assess;

• the resources available;• the uses of the results, the intended impact

of its use.

Blueprint

PROSET - TEMPUS 21

• content• methods• scoring.

Test tasks

PROSET - TEMPUS 22

• prompt (a reading/listening text, an essay question, a picture to describe);

• response (ticking a box, giving a short answer, describing a picture, etc.).

The building blocks of a test

PROSET - TEMPUS 23

• instructions• item• stem• options• distractors• key.

Response types

PROSET - TEMPUS 24

• selected response• constructed response– short response–extended response.

• personal response.(Brown & Hudson, 1998)

Responses: EGE

PROSET - TEMPUS 25

• selected response Выбор ответа(A1-A7, A8-A14, A15-A23)

• short responseКраткий ответ (B1, B2, B3, B4-B10, B11-B16)

• extended response Свободно конструируемый ответ (C1, C2)

Task & item specifications

PROSET - TEMPUS 26

• How many items are included in each task?• What area of knowledge, skill or ability is

assessed in each task?• What should the test-taker do (instructions)?• How are scores on each item and task

determined?

One and the same specification for all?

PROSET - TEMPUS 27

• test-takers• item writers• test validators• users of test results.

The same type of specification for

a classroom test and EGE?

PROSET - TEMPUS 28

EGE specification: •11 pages•12 parts (sections)•4 tables•an appendix.

Your classroom test specification/ plan

PROSET - TEMPUS 29

• test purpose• number of tasks and items• what these items assess• how they are scored• time limit • can you suggest anything else?