1Apr 21, 2023

Chapter 17Chapter 17Comparing Two ProportionsComparing Two Proportions

3

Data conditions• Binary response variables (“success/failure”)• Binary explanatory variable

Group 1 = “exposed”Group 2 = “non-exposed”

• Notation:

4

Sample Proportions

1

11ˆ

n

ap

2

22ˆ

n

ap

Sample proportion (average risk), group 1:

Sample proportion (average risk), group 2:

5

Example: WHI Estrogen Trial

Random Assignment

Group 1 n1 = 8506

Group 2 n2 = 8102

Estrogen Treatment

Placebo

Compare risks of index outcome*

*Death, MI, breast cancer, etc.

6

2-by-2 TableSuccesses Failures Total

Group 1 a1 b1 n1

Group 2 a2 b2 n2

Total m1 m2 N

1

11ˆ

n

ap

2

22ˆ

n

ap

7

WHI DataD+ D− Total

E+ 751 7755 8506

E- 623 7479 8102

Total 1374 15234 16608

8506

751ˆ1 p

07689.0

08829.0

8102

623ˆ 2 p

14

§17.3 Hypothesis TestA. H0: p1 = p2 (equivalently H0: RR = 1)

B. Test statistic (three options)

– z (large samples)– Chi-square (large samples, next chapter)

– Fisher’s exact (any size sample)

C. P-value

D. Interpret evidence against H0

19

Fisher’s Exact Test• All purpose test for testing H0: p1 = p2

• Based on exact binomial probabilities

• Calculation intensive, but easy with modern software

• Comes in original and Mid-Probability corrected forms

20

Example: Fisher’s TestData. The incidence of colonic necrosis in an exposed group is 2 of 117. The incidence in a non-exposed group is 0 of 862.

Ask: Is this difference statistically significant?

A.Hypothesis statements. Under the null hypothesis, there is no difference in risks in the two populations. Thus: H0: p1 = p2

Ha: p1 > p2 (one-sided) or Ha: p1 ≠ p2 (two-sided)

21

Fisher’s Test, ExampleB. Test statistic none per se

C. P-value. Use WinPepi > Compare2.exe > A.

D. Interpret. P-value = .014 strong (“significant”) evidence against H0

D+ D−

E+ 2 115

E− 0 862

22

§17.4 Proportion Ratio (Relative Risk)

• “Relative risk” is used to refer to the RATIO of two proportions

• Also called “risk ratio”

2

1

ˆ

ˆˆp

pRR

23

Example: RR (WHI Data)+ − Total

Estrogen + 751 7755 8506

Estrogen − 623 7479 8102

2

1

ˆ

ˆˆp

pRR

08829.08506

751ˆ1 p 07689.0

8102

623ˆ 2 p

07689.0

08829.0 15.11483.1

24

Interpretation• The RR is a risk multiplier

– RR of 1.15 suggests risk in exposed group is “1.15 times” that of non-exposed group

– This is 0.15 (15%) above the relative baseline

• When p1 = p2, RR = 1.

– Baseline RR is 1, indicating “no association”– RR of 1.15 represents a weak positive

association

25

Confidence Interval for the RR

ln ≡ natural log, base e

RRSEzRR

eˆln

21

ˆln

2211

1111ˆln

wherenanaRR

SE

To derive information about the precision of the estimate, calculate a (1– α)100% CI for the RR with this formula:

26

90% CI for RR, WHID+ D− Total

E+ 751 7755 8506

E− 623 7479 8102

1.645 ,confidence 90%For

051920.0

1382.0)1483.1ln(ˆln

81021

6231

85061

7511

ˆln

z

SE

RR

RR

)25.1 ,05.1(

2236.0,0528.00854.01382.0)051920.0)(645.1(1382.0

eee

27

WinPepi > Compare2.exe > Program B

D+ D− Total

E+ 751 7755 8506

E − 623 7479 8102

See prior slide for hand calculations

28

Confidence Interval for the RR

• Interpretation similar to other confidence intervals

• Interval intends to capture the parameter (in this case the RR parameter)

• Confidence level refers to confidence in the procedure

• CI length quantifies the precision of the estimate

29

§17.5 Systematic Error• CIs and P-values address random error

only

• In observational studies, systematic errors are more important than random error

• Consider three types of systematic errors:

– Confounding

– Information bias

– Selection bias

30

Confounding• Confounding = mixing together of the

effects of the explanatory variable with the extraneous factors.

• Example: – WHI trial found 15% increase in risk in

estrogen exposed group. – Earlier observational studies found 40% lower

in estrogen exposed groups. – Plausible explanation: Confounding by

extraneous lifestyles factors in observational studies

31

Information Bias• Information bias - mismeasurement

(misclassification) leading to overestimation or underestimation in risk

• Nondifferential misclassification (occurs to the same extent in the groups) tends to bias results toward the null or have no effect

• Differential misclassification (one groups experiences a greater degree of misclassification than the other) bias can be in either direction.

32

Nondifferential & Differential Misclassification - Examples

33

Selection Bias• Selection bias ≡ systematic error related to

manner in which study participants are selected• Example. If we shoot an arrow into the broad

side of a barn and draw a bull’s-eye where it had landed, have we identified anything that is nonrandom?

34

Sample Size & Powerfor Comparing Proportions

Three approaches: 1. n needed to estimate given effect with

margin of error m (not covered in Ch 17)

2. n needed to test H0 at given α and power

3. Power of test of H0 under given conditions

35

Sample Size Requirements for Comparing Proportions

Depends on:

r ≡ sample size ratio = n1 / n2

1−β ≡ power (acceptable type II error rate)

α ≡ significance level (type I error rate)

p1 ≡ expected proportion, group 1

p2 ≡ expected proportion in group 2, or expected effect size (e.g., RR)

36

Calculation • Formulas on pp.

396 – 402 (complex)

• In practice use WinPEPI > Compare2.exe > Sample size

37

WinPepi > Compare2 > S1