Readingtestspecifications assignment-01-ppt-141130013903-conversion-gate01 - copy

READINGTEST

SPECIFICATIONS

READING TEST SPECIFICATIONS: 4.8

• Introduction:

How can we help students to learn in an efficient, effective, attractive and accessible way? There is no simple straight ward answer to this question; depending on the specific situation, solution X will work best for person Y. It’s generally acknowledged that we can improve testing consider by making the conditions to optimal specifications and then use this acknowledge to design new test events.

We are going to introduce the concepts of test design, and examine what a test designer must know in order to create high-quality test designing, and to discuss the nature of this specifications, how it can be modeled in terms of rules, and how the rules are derived. We also will use three examples (Designs / Contexts) to introduce the different modeling concepts and use the term test designer, to describe those who learning design task to perform.

THE PURPOSE OF THE TESTThis is a reading comprehension assessment. • To measure the testees' performance based on

the curriculum during teaching and learning sessions in the whole school semester.

• This test is under local control - used to place learners into classes for the next school semester

• To discover how much the students have achieved in their L2 learning.


THE PURPOSE OF THE TEST Cont.• This test is low-stakes or summative.• However, teachers use the information to

make decisions about which classes are more suitable for the learners to be in for the next semester of school, according to their levels of performance.


• Grade 4 ESL students of Baghdad Secondary School, Iraq.

• The assessment is designed to measure the academic achievement of all test takers with diverse background of Grade 4.

• Items in the assessment are written in plain language, without unnecessarily complex syntactical structure because they are adolescent ESL learners with the kind of language needs.

• This assessment is carefully designed to provide accommodations for this type of learner’s level; teenage ESL students.

• The passage and items are accessible.


The Test Takers

Context and Modality

• The Modality: Reading skill (Comprehension). • The context: In this test design the tester chooses THREE efficient contexts

which is: • Multiple choice item 2) True and False item. 3) Matching item.

The Test design: Reading comprehension test for measuring the competence and the performance of the testees. (Achievement test).

• ACHIEVEMENT TEST Achievement test attempts to measure what an individual has learned-his or her present level of performance; in this design the testees will be examined for his/her comprehension in the reading skill.


1. Identify the puspose of the test.2. Identify the best specification.3. Select the contents of the test.4. Consider the form of the test.5. Write the test item.6. Consider the layout of the test.7. Consider the timing of the test.8. Plan the scoring of the test.

In planning a test, the Designer can proceed the following :


1.To diagnose a student’s strength, weakness and difficulties.

2.To measure achievement.3.To measure aptitude and potential.4.To identify readiness for a program.

1. IDENTIFY THE PURPOSE OF THE TEST


2. IDENTIFY THE TEST SPECIFICATIONThe test specification includes :

1. Which program objectives and student learning outcomes will be addressed.

2. Which content areas will be addressed.

3. The relative weightings, balance and coverage of items.

4. The total number of items in the test.

5. The number of questions required to address a particular element of a program or learning outcome

6. The exact item in the test.


3. SELECT THE CONTENTS OF THE TEST

Gronlund and Linn (1990), suggest that an item analysis will need to consider :

1. The suitability of the format of each item for the (learning) objective (appropriateness)

2. The ability of each item to enable students to demonstrate their performance of the (learning) objective (relevance)

3. Clarity of the task for each item

4. The straight forwardness of the task

5. The independence of each item (i.e. where the influence of other items of the tests minimal and where successful completion of one another)

6. The adequacy of coverage of each (learning) objective by the items of the test.

4. CONSIDER THE FORM OF THE TEST

The designer (tester) will need to consider whether the test will be undertaken individually, or in a group, and what form it will take. Comprehension test, for example, can be conducted if the designer feels that reading will obstruct the true purpose of the test.


5. WRITE THE TEST ITEMS

In this test we tackled three contexts:1. Multiple Choice.2. True-false Items.3. Matching.


A. CONSTRUCTING MULTIPLE CHOICE ITEMS

1. Design each item to measure a specific objective.• (Direct MCQ) Example: Q/ Which of these statements correctly summarizes

how the author of this passage feels about robots? a. Robots are old. b. Robots are confusing. c. Robots are helpful. d. Robots are dangerous.

• (MCQ Indirect question) Example:

Q/ What is the main purpose of paragraph 2?

A. to show how easy it is to make a robot. B. to tell what a robot is .C. to describe the things a robot can do. D. to explain the difference between a robot and a machine.


2. State both stem and options as simply and directly as possible.

We are sometimes tempted to make multiple choice items too wordy. A good rule is to get directly to the point.

( MCQ cloze) Example: Q/ Long ago, people ………… robots.

a. played

b. saw

c. imagined

d. broke


3. Make certain that intended answer is clearly the only correct one.

Q/ According to the passage, when was the first real robot made?

a. 1961

b. 1900

c. 2003

d. 2000 years ago


4. USE ITEM INDICES TO ACCEPT, DISCARD, OR REVISE ITEMS

Item facility (level of difficulty): Is the extent to which an item is easy or difficult for the proposed group of test-takers. IF shows how easy or difficult the particular item that’s proved in the test.

Item discrimination (discriminating power): Item discrimination is the extent to which an item differentiates between high- and low-ability test-takers. It measures how well the test items are arranged to identify the differences in the students competence.


4. USE ITEM INDICES TO ACCEPT, DISCARD, OR REVISE ITEMS. Cont.

Distractor efficiency: Distractor efficiency is one more important measure of multiple choice item’s value in a test. The efficiency of distractors is the extent to which: -

a. The distractors “lure” a sufficient number of testees especially lower-ability ones

b. Those responses are somewhat evenly distributed across all distractors.

B. TRUE-FALSE ITEMSThe true-false item typically present a declarative statement that the students must mark as either true or false. Instructors generally use true-false items to measure the recall of factual knowledge such as name, events, dates, definitions, etc. But this format has the potential to measure higher levels of cognitive ability, such as comprehension of significant ideas and their application in solving problems.

Examples:

A robot is a special kind of machine. (T or F)

A robot is a human being. (T or F)


c. MATCHINGA matching exercise typically consists of a list of questions or problems to be answered along with a list of responses. The examinee is required to make an association between each question and a response.


EXAMPLE

B AA. 1961 1. Something special is NOTB. Machine 2. Robots may be used toC. Normal 3. When was the first real robot made?D. Make things 4. Who is the first one imagined robots?E. Homer 5. Most robots just look likeF. 2000G. Tired

Match the following items in list (A) to their responses in list (B):


6. CONSIDER THE LAYOUT• The Clarity of the Instruction:

What to do (Objective)

How long to take (Time)

How many items to attempt (Quantity)

What kind of response is required How and where to enter the response (Answer placing)• The Location and Sequence of Items

The progression from the easy to the more difficult items of the test.

• The Visual Layout of the Page Minimize the unnecessary visual material or words.

7. CONSIDER THE TIMING OF THE TEST

• The timing refers to two areas:1. When the test will take place2. The time allowances to be given to the test and

its component items

8. PLAN THE SCORING OF THE TEST


SCORING SYSTEM DESIGN AND DESCRIPTIONS

Multiple-Choice Item

True and False Item

Matching Item

• 4 answer choices (only one correct answer).

• generic scoring rubric, 1 point for correct answer or 0 point for incorrect (dichotomous).

• Only 2 options (1 correct and 1 incorrect response).

• Scored dichotomously (choose whether the statement is ‘True’ or ‘False’).

• If they choose the correctly - 1 point• If they don’t - 0 point

• 5 questions and 7 options can be chosen to match with the questions.

• only 1 correct response for every item• 1 point – if they match correctly• no point - if they match it incorrectly

Multiple-Choice Item

True and False Item

SCORE CATEGORY AND DESCRIPTIONS

CORRECT ANSWER (A, B, C, D OR E)These responses represent one correct choice (1 point)

INCORRECT ANSWER (A, B, C, D OR E)These responses represent one incorrect choice (0 point)


CORRECT ANSWER (TRUE OR FALSE OPTION)These responses represent one correct choice (1 point)

INCORRECT ANSWER (TRUE OR FALSE OPTION)These responses represent one incorrect choice (0 point)


READING TEST SPECIFICATIONS: 4.8 Matching Item


CORRECT MATCHThese responses represent one correct match (1 point)

INCORRECT MATCHThese responses represent one incorrect match (0 point)

ReferencesAlderson, J. C. (1988). Testing English for specific purposes – how specific can we get? In Hughes, A. (ed.),

Testing English for University Study. London: Modern English Publications and the British Council, 16–18. Allman, C. (2005). Building Assessment Initiatives for Schools: An Accommodations Guide for Parents and

Students with Visual Impairments. American Foundation for the Blind. Available online: http://www.afb.org/Section.asp?SectionID=58&TopicID=264&DocumentID=2762.

Assessment Reform Group (1999). Assessment for Learning: 10 principles. Retrieved from http://

www.qca.org.uk/qca_4336.aspx , 12 May 2009.

Bachman, L. F. (2004). Statistical Analyses for Language Assessment. Cambridge: Cambridge University Press.

Bernstein, J. (1999). Phone Pass TM Testing: Structure and construct. Menlo Park, CA: Ordinate.

Carroll, J.B. (1961). Fundamental considerations in testing for English language proficiency of foreign students. In Allen, H.B. (Ed.), Teaching English as a Second Language. McGraw Hill, New York, pp. 364-372.

Center for Applied Linguistics. (2007). Foreign Language Assessment Directory. Retrieved 3 July 2009, from

http://www.cal.org/CALWebDB/FLAD Cohen, A. S. and Wollack, J. A. (2006). Test administration, security, scoring and reporting. In Brennan, R. L.

(ed.), Educational Measurement. 4th edition. New York: American Council on Education/Praeger, 355–386. Fulcher, G. (2010). Communicative language testing. In de Bot, K. (ed.), The Mouton Handbook of Teaching

English as a Foreign Language. The Hague: Mouton.

http://www.afb.org/Section.asp?SectionID=58&TopicID=264&DocumentID=2762

http://www.qca.org.uk/qca_4336.aspx

http://www.cal.org/CALWebDB/FLAD

Readingtestspecifications assignment-01-ppt-141130013903-conversion-gate01 - copy

Documents

Transcript of Readingtestspecifications assignment-01-ppt-141130013903-conversion-gate01 - copy