Test appraisal

05/03/2023 UNIVERISTY OF EDUCATION, LAHORE

Test Appraisal System.Concepts of Appraisal &

Procedures B


Topics of Discussion• Definition• Meaning• Assembling the Classroom Test• Important Test Characteristics• Item Analysis and its benefits• Item Analysis for Norm Referenced Test• Item Analysis for Criterion Referenced Test• Establishing Test Validity and Reliability• Preparing Test Item Bank


According to Newstrom: “It is the process of evaluating the performance of Students, sharing that information with them and searching for ways to improve their performance’’.

Definitions

• Test appraisal is the step where the Institutions management finds out how effective it has been at conduction and evaluation of students.• A “TEST Performance appraisal” is a process of evaluating a Student’s performance of a test in terms of its placement and promotion.

Meaning


Objectives of TEST Appraisal

Measuring the efficiencyMaintaining Control.

Concrete and tangible particulars about their workAssessment of performance

Mutual goals of the students, teachers & the organization.Growth & developmentIncrease harmony & enhance effectiveness

Personal development Self Satisfaction Involvement in the Work.

STUDENTS INSTITUTIONALAccording to:

Aims at:

05/03/2023

Desirable characteristics for tests:

Apart from assessment and evaluation, we must consider the following points while testing or going to develop a test appraisal system:• Validity• Reliability• Utility• Practicality• Discrimination

UNIVERISTY OF EDUCATION, LAHORE


VALIDITY• Validity refers to the degree in which our test or other

measuring device is truly measuring what we intended it to measure. The test question “1 + 1 = _____” is certainly a valid basic addition question because it is truly measuring a student’s ability to perform basic addition.

• The subjective testing inevitably associated with testing the productive skills reduces reliability but they are more valid because they test integrative knowledge of the language so they give the teacher the opportunity to see how students really use the language.

• Teacher can have a better view of their students’ competence of language.

• So the higher reliability, the less valid or all the opposite.


RELIABILITY• Reliability refers to the consistency of the

scoring of the test, both between different raters, and between the same rater on different occasions.

• Objective testing should give perfect reliability.

• However, faulty tests (ambiguous multiple choice, wrong answers on an answer sheet, etc) can reduce the reliability of even an objective test.


UTILITY, PRACTICALITY and DISCRIMINATION

• Utility: A test which provides a lot of feedback to assist in the planning of the rest of a course or future courses.

• Practicality: the efficiency of the test in physical terms. (Does it require a lot of equipment? Does it take a lot of time to set, administer or mark?)

• Discrimination: the ability of a test to discriminate between stronger and weaker students.


TEST/ Performance Appraisal Process

Issues in appraisal system

Appraisal Design

Formal and informal

Whose performance?

Who are the raters?

What problems?

How to solve?

What to evaluate?

When to evaluate?

What methods?


AdvantagesProvide a record of

performance over a period of

time.Provide an opportunity

for a manager to

meet & discuss

performance

Provide the employee

with feedback

about their performanc

e

Provide an opportunity

for an employee to

discuss issues and to clarify

expectations

Can be motivational

with the support of a good reward

and compensati

on


If not done appropriately, can

be a negative experience.

Subject to rater errors & biases.

If not done right can be a complete waste

of time.Can be stressful for

all involved

Disadvantages


What is Item Analysis ?• Item analysis is a statistical technique which

is used for selecting and rejecting the items of the test on the basis of their difficulty value and discriminated power.


Objectives of Item Analysis

• To select appropriate items for the final draft

• To obtain the information about the difficulty value(D.V) of

all the items

• To provide discriminatory power (D.I) to differentiate

between capable and less capable examinees for the items

• To provide modification to be made in some of the items

• To prepare the final draft properly ( easy to difficult items)


STEPS for Item Analysis

• Arrange the scores in descending order

• Separate two sub groups of the test papers

• Take 27% of the scores out of the highest scores and 27%

of the scores falling at bottom

• Count the number of right answer in highest group (R.H)

and count the no of right answer in lowest group (R.L)

• Count the non-response (N.R) examinees

Difficulty value (D.V)• “The difficulty value of an item is defined as the

proportion or percentage of the examinees who have

answered the item correctly” (J.P. Guilford)• The formula for difficulty value (D.V)

• D.V = (R.H + R.L)/ (N.H + N.L)• R.H – rightly answered in highest group• R.L - rightly answered in lowest group• N.H – no of examinees in highest group• N.L - no of examinees in lowest group

Discrimination index (D.I)• “Index of discrimination is that ability of an item on

the basis of which the discrimination is made between superiors and inferiors” (Blood and Budd (1972)

• The formula for discrimination index(D.I)

• D.I = (R.H - R.L)/ (N.H or N.L)• R.H – rightly answered in highest group• R.L - rightly answered in lowest group• N.H – no of examinees in highest group• N.L - no of examinees in lowest group

General guidelines for discriminating index (D.I)

According to Ebel ,

D.I Item Evaluation

≥0.40 Very good items

0.30 - 0.39 Reasonably good but subject to improvement

0.20 – 0.29 Marginal items , need improvement

<0.19 Poor items . Rejected or revised

General guidelines for difficulty value (D.V)

Low difficulty value index means, that item is high difficulty oneex: D.V=0.20 » 20% only answered correctly for that item. So that item is too difficult

High difficulty value index means, that item is easy oneex: D.V=0.80 » 80% answered correctly for that item. So that item is too easy one

Criteria for selection and rejection items

Positive discrimination index only selected

Negative and zero discrimination index items are

rejected

High and low difficulty value items are rejected


Any Queries

Test appraisal

Education

Transcript of Test appraisal