How to Perform Simple Manual Item Analysis

Dr. Belal Hijji, RN, PhD

January 18, 2012

2

Learning Outcomes

At the end of this short training session, students will be

able to:• Compute two item analysis indices of a multiple-choice

examination. • Identify item construction flaws, if any.

3

Steps for Computing Item Analysis Indices 1. After scoring the test, arrange the scores from the highest to the

lowest.

2. Divide the scores into top 50% and lower 50% of all scores.

3. For each item, tally the number of students in each group who selected each option. Record these counts on a copy of the test item

next to each response option. For example:

Item 1: Which of the following is a modifiable risk factor for coronary artery disease?

Higher 50% Lower 50%

A. Hypertension* 13 13

B. Family history 0 1

C. Age 2 1

D. Race 0 0

4

Steps for Computing Item Analysis Indices 4. Apply the following equation to calculate the difficulty index for each

item: P = (Rh + Rl)/ T. An item is easy, moderately difficult (acceptable), or difficult if its p value is ≥ .80, between .21 and .79,

and ≤ .20, respectively.

Rh is the number of students in the top 50% who selected the correct option, Rl is the number of students in the lower 50% who selected

the correct option, and T is the total number of students. For the example on the previous slide, the P-value is (13 + 13)/ 30 = .87.

5

Steps for Computing Item Analysis Indices (Continued…) 5. Calculate the discrimination index (D) for each item. This is done

by dividing the Rh by the total number of students in the top 50%. This will yield Ph. Repeat the process to calculate Pl from Rl.

Subtract Pl from Ph to obtain D. For the example on slide 3, Ph = 13/15 = .87; Pl = 13/15 = .87. The D = .87- .87 = 0, indicating that

the item’s discriminatory power is nonexistent. However, A discrimination index of .20 or higher is appropriate

6. Check each item for implausible distractors, ambiguity and miskeying. Look at the table on next slide to find out how plausible

was each distracter for item 1.

6

Computations of Item Analysis Indices

Percentages Upper 50% - Lower 50%

OptionsItem KeyABCDOmitsp

1A13 - 130 - 12 - 10 - 00 - 0.87

%.87– .870 - .06.12 - .060 - 0

d0-.06.060

As this item is easy (p = .87), its ability to discriminate between the top and lower 50% of examinees is zero.

Distracter “D” is implausible because none of the lower 50% students selected it.

7

Item 2: What may happen to a patient if rapid administration of cold blood is performed through a central venous route terminating in or near the right atrium? The number of examinees was 298.

Higher 50% Lower 50%A. Post transfusion purpura 14 14B. Cardiac arrhythmia* 141 52C. Acute intravascular haemolytic reaction 18 37D. Transfusion associated acute lung injury 13 9

Example 2

Look at this item’s analysis indices on next slide

8

Percentages Upper 50% - Lower 50%

OptionsItem KeyABCDOmitsp

2B14 - 14141 - 5218 - 3713 - 90 - 0.65

%.07 - .13.76 - .46.10 - .33.07 - .08

d-.06.30-.23-.01

Computations of Item Analysis Indices

What can you conclude when the number of examinees is large?

This is a good item because it is moderately difficult, has good discriminatory power, and all its distractors are plausible.

9

Where Do You Go From Here?

Based on the training you received, please continue working on the test until you compute the difficulty and discrimination indices for each item and tabulate the data.

10

References

• Oermann, M. and Gaberson, K. (2009).Evaluation and Testing in Nursing Education. 3rd edition. New York: Springer.