1

Statistical Lesson on Screening

Stephen E. FienbergStatistics 36-149

Fall, 2001

2

Screening Tests

What is Pr(HIV+)?

Test Results +ve -ve Totals +ve 254 7,381 7,635 -ve 6 92,359 92,365

Totals 260 99,740 100,000

True HIV Status

3

Screening Tests

What is Pr(HIV+)? Pr(HIV+)= 260/100,000=0.00260.

True HIV Status

Test Results +ve -ve Totals +ve 254 7,381 7,635 -ve 6 92,359 92,365

Totals 260 99,740 100,000

4

Error Rates for Tests

What is Pr(HIV+|Test Result +)?

True HIV StatusTest Results +ve -ve Totals

+ve 254 7,381 7,635 -ve 6 92,359 92,365

Totals 260 99,740 100,000

5

Error Rates for Tests

What is Pr(HIV+|Test Result +)?Pr(HIV+ | Test Result +) = 254/7,635 = 0.0333 ==>

False Positive Rate = Pr(HIV- | Test Result +) = 7,381/7,635 = 0.9667

True HIV Status

Test Results +ve -ve Totals +ve 254 7,381 7,635 -ve 6 92,359 92,365

Totals 260 99,740 100,000

6

Error Rates for Tests

What is Pr(HIV+|Test Result -)?

True HIV Status

Test Results +ve -ve Totals +ve 254 7,381 7,635 -ve 6 92,359 92,365

Totals 260 99,740 100,000

7

Error Rates for Tests

What is Pr(HIV+|Test Result -)?Pr(HIV+ | Test Result -) = 6/92,365 = 0.000065

False Negative Rate

True HIV Status

Test Results +ve -ve Totals +ve 254 7,381 7,635 -ve 6 92,359 92,365

Totals 260 99,740 100,000

8

Validity-Reliability of Tests

• Reliability means repeatability across different times, places, subjects, and/or experimental conditions.– Test-retest reliability-- same subject and same measurement

procedure including examiner, equipoment etc., on two occasions.

– Inter-rater reliability--different examiners/judges.

• Validity means consistent association between test and truth in general population of interest.

9

How to Measure Validity?

• Problem:– False positive and false negative error rates depend

on the mix of true HIV+ and HIV- people in the population.

– In our example, they depend on the frequency of positive and negative test results and the base rate for HIV+!

– So if the base rate for HIV+ changes our assessment of validity does.

10

Example: Airport Security

• Screening of luggage using x-ray equipment in airport security.

• What do terms Reliability (test-retest, and inter-rater) and Validity mean?

11

Sensitivity and Specificity for Tests

• Sensitivity of test= Pr(Test Result +| HIV+) = 254/260 = 0.977

• Specificity of test= Pr(Test Result -| HIV-) = 92,359/99,740 = 0.926

True HIV Status

Test Results +ve -ve Totals +ve 254 7,381 7,635 -ve 6 92,359 92,365

Totals 260 99,740 100,000

12

• Can plot rates on a graph:

Characteristics of Test

13

• To convey information on validity we use plot of “sensitivity” and “1 - specificity” for different versions of the test.

• Let’s see how to do this in an example.

Receiver Operating Characteristic Curve

14

15

16

17

18

19

20

21

22

23

ROC Analysis

• To measure the validity we connect the dots on ROC plot and compute the area under the curve:– perfect test has area 1.– “chance” test has area 0.5.

24

Back To Airport Security

• Does any of this have relevance to the issues of “Terrorist” screening and “Racial”screening for security purposes?

25

References

• Handout on Web from Rhiannon Weaver• Slides from Center for Biomedical

Informatics, UPMC:– http://www.cbmi.upmc.edu/courses/fall/intro/oct12