Objectives  9.1, 9.2 Inference for two-way tables

p  The hypothesis: no association

p  Expected cell counts

p  The chi-square test

p  Using software

p  Further reading:

http://onlinestatbook.com/2/chi_square/contingency.html

Independence/Association:  Sample  and  Population  p  In the previous section we defined the notion of independence and

dependence (also called association) using two-way contingency tables: p  Recall two variables are independent if the probability of one variable

conditioned on the other is the same as the marginal probabilities. §  Example: In Chapter 13 we gave an example where the gender of a person

did not change the chance of a pass or fail. This means that gender and passing are independent variables.

p  If the marginal probabilities and the conditional probabilities are not the same, then there is an association between the variables. §  Example: In Chapter 13 we have an example where the gender of a person

changed (dramatically) the chance of them wearing a dress (or not) to the Oscars. This means there is an association between dress wearing and gender. Knowledge of one variables (ie the person is a female) changes the chances of the wearing a dress or not.

p  In reality we do not ever observe the population, and the numbers in a two-way table is a sample from a population. p  In such a case, even if variables are independent, sampling variation will mean

that the marginal probability will not be the same as the conditional.

p  Example: The number of males and females in higher education is know to be equal. However, in a given class the numbers of males and females are likely to be different.

p  Thus we need to `test’ for independent between the variables given the data set. p  We do this by `predicting’ what the numbers in the tables would be

under the scenario of independence and make a comparison to what is actually observed in the data.

p  This is best explained through several examples…

p  But in reality these are only samples from the entire population. For a sample we cannot expect that even in the case of independence (no association) that the marginal probabilities and the conditional probabilities will be exactly the same. It will be different due to random variation in the sample.

p  As in everything we have done so far, we are not interested in the sample but the population itself. What can we infer about the population based on the sample?

p  Therefore we are interested in seeing whether in the population there is an association between the two variables. p  In the case that we have a two-by two table, for example, the hair and

and minodoxil example we can test if the conditional probabilities (proportion who saw an improvement using minidoxil vs placebo) are the same or not by using the test on two proportions.

p  However, this method cannot be extended to larger tables such as the student smoking example.

p  Instead we take a slightly different approach. We calculate what we expect to see if there is no dependence and compare it what we do observe. We formalize this on the next slides.

Hypothesis  test  for  association  When we have two categorical variables it is often of interest to determine whether they are associated. As always, a firm decision can

only be made by rejecting a null hypothesis using an appropriate test procedure. This is because we need to know if the apparent differences

among sample proportions are likely to have occurred just by chance

due to random sampling.

The hypotheses are

H0: the variables are not associated vs. Ha: the variables are associated.

We will use the chi-square (χ2) test to assess the null hypothesis based on how well the data fit with what H0 predicts the counts of a two-way

table to be.

To test the null hypothesis of no association between the variables, we must compare the actual (observed) counts from the sample data with the expected counts. The null hypothesis predicts that the cell proportions of the column variable within each row be the same as their overall proportions for the whole table.

Specifically, the expected count in any cell of a two-way table when H0 is true is:

Expected  cell  counts  

Do not round the expected count – it usually is not a whole number.

The expected count is a mean, not a value we would actually see.

Example  1:  Oscars  and  dresses  Male Female Total

Dress 0 215 215

No Dress 200 5 205

Total 200 220 420

Male Female Total

Dress 215 (

215420 = 0.512)

No Dress 205 (

205420 = 0.488)

Total 200 (

200420 = 0.476) 220 (

220420 = 0.524) 420

Let us return to the Oscars data. Suppose we miss all the numbers inside the table and only observe the subtotals, ie. The number who wore dresses, the number who did not wear dresses, the number of females and the number of males. Question These are the marginal numbers. Can we use these to deduce the numbers inside the boxes? Answer Only if dress and gender are independent variables. In this case the conditionals and marginal probabilities are the same.

Male Female Total

Dress 51.2% of 200 males 51.2% of 220 females 215 (

215420 = 0.512)

=102.38 =112.6

No Dress 48.8% of 200 males 48.8% of 220 females 205 (

205420 = 0.488)

=97.62 =107.38

Total 200 (

200420 = 0.476) 220 (

220420 = 0.524) 420

If there is no dependence between gender and whether they wear a dress or not, then we can use the marginal probabilities to predict the numbers.

Compare the numbers from what is observed and expected under independence, they are completely different!

p  We measure the difference between what is observed actually in the data and what we predict if they is no association:

�2 =(0� 102.8)2

102.8+

(112.6� 215)2

112.6+

(200� 97.62)2

97.72+

(107.38� 5)2

107.38= 400.44

This difference is huge! It means that our predictions are completely wrong and this is because we made the predictions under the assumption that there dress and gender are completely independent – clearly they are not – this is why there is such a big difference. Observe that the p-value is very small. This tells us we are rejecting the null hypothesis, which is that there is no dependence between gender and whether they wear a dress or not.

Example  2:  Gender  and  grades  Let us return to the gender and grade data. Suppose we miss all the numbers inside the table and only observe the subtotals, ie. The number who passed, the number who did not pass, the number of females and the number of males.

We `fill in’ the middle of the table what we expect to see if the is no dependence between gender and grades If there is no dependence between gender and grades, then we can use the marginal probabilities to predicted numbers.

Compare the numbers from what is observed and expected under independence, they are completely different!

Male Female Total

Pass 108 180 288

Fail 12 20 32

Total 120 200 320

Male Female Total

Pass 288

Fail 32

Total 120 200 320

Male Female Total

Pass 90% of the 120 males 90% of the 200 females 288 (

288320 = 0.9)

= 108 = 180

Fail 10% of 120 males 10% of the 320 females 32 (

32320 = 0.10)

=12 = 20

Total 120 (

120320 = 0.375) 200 (

200320 = 0.625) 320

p  We measure the difference between what is observed actually in the data and what we predict if they is no association:

There is no difference at all, the predictions were exactly what is expected under independence of grade and gender. Look at the p-value it is one. This tells us we cannot rejecting the null hypothesis, that there is no dependence between gender and whether they wear a dress or not.

�2 =(108� 108)2

108+

(180� 180)2

180+

(12� 12)2

12+

(5� 5)2

5= 0

Example  3:  Minidoxil  and  hair  Let us return to the Minidoxildata. Suppose we miss all the numbers inside the table and only observe the subtotals, ie. The number who saw an improvement, the number who did not, and the numbers in both groups.. Question These are the marginal numbers. Can we use these to deduce the numbers inside the boxes?

Again we can only use the marginals if there is no dependence/association between the treatment and hair growth. If this is the case, we obtain the numbers on the left.

Unlike the previous two examples, the numbers neither match or are completely different. How to interpret these differences?

Minidoxil Placebo Subtotals

Improvement 26% of 310 26% of 302 159 (

159612 = 26%)

=80.54 =78.46

No improvement 74% of 310 74% of 302 463 (

453612 = 74%)

=229.94 =223.54

Total 310 (

310612 = 50.6) 302 (

302612 = 49.3%) 612

Minidoxil Placebo Subtotals

Improvement 99 60 159

No improvement 211 242 453

Total 310 302 612

Minidoxil Placebo Subtotals

Improvement 159

No improvement 453

Total 310 302 612

p  We measure the difference between what is observed actually in the data and what we predict if they is no association:

This difference is not zero (as in Example 2 or huge as in Example 1). Can we explain this difference under just sampling variation when in fact the Minidoxil is no different to the placebo? It is hard to judge based on the difference 11.58, we need to know the distribution associated with 11.58 and from here deduce the p-value. We see that the p-value is 0.7%, in the next few slides we explain where this comes from.

�2 =(99� 80.54)2

80.54+

(60� 78.46)2

60+

(211� 229.94)2

229.94+

(223.54� 242)2

223.54= 11.58.

The  chi-­‐squared  distribution  This is what a chi-squared looks like. It tells us that if there is no association between gender and binge drinking the chi-statistics is likely to be quite small. In fact the chance of it being large are quite slim. These chance can be obtained using the critical values for the chi-squared distribution given in the chi-squared tables. Looking up chi-squared tables with 1df. We see that there is a 25% chance the chi-value will be more than 1.32 and a 5% chance it will be more than 3.84. We apply this to our chi-value.

For the chi-square test, H0 states that there is no association between the row and column variables in a two-way table. The alternative is that

these variables are related.

If H0 is true, the chi-square test statistic has approximately a chi-square distribution with (r − 1)×(c − 1) degrees of freedom.

Use the chi-square table (Table F)

to get the P-value.

The P-value for the chi-square test

is the area to the right of the test

statistic χ2 under this distribution.

pdf 0.25 0.2 0.15 0.1 0.05 0.025 0.02 0.01 0.005 0.0025 0.001 0.00051 1.32 1.64 2.07 2.71 3.84 5.02 5.41 6.63 7.88 9.14 10.83 12.12 2 2.77 3.22 3.79 4.61 5.99 7.38 7.82 9.21 10.60 11.98 13.82 15.20 3 4.11 4.64 5.32 6.25 7.81 9.35 9.84 11.34 12.84 14.32 16.27 17.73 4 5.39 5.99 6.74 7.78 9.49 11.14 11.67 13.28 14.86 16.42 18.47 20.00 5 6.63 7.29 8.12 9.24 11.07 12.83 13.39 15.09 16.75 18.39 20.51 22.11 6 7.84 8.56 9.45 10.64 12.59 14.45 15.03 16.81 18.55 20.25 22.46 24.10 7 9.04 9.80 10.75 12.02 14.07 16.01 16.62 18.48 20.28 22.04 24.32 26.02 8 10.22 11.03 12.03 13.36 15.51 17.53 18.17 20.09 21.95 23.77 26.12 27.87 9 11.39 12.24 13.29 14.68 16.92 19.02 19.68 21.67 23.59 25.46 27.88 29.67 10 12.55 13.44 14.53 15.99 18.31 20.48 21.16 23.21 25.19 27.11 29.59 31.42 11 13.70 14.63 15.77 17.28 19.68 21.92 22.62 24.72 26.76 28.73 31.26 33.14 12 14.85 15.81 16.99 18.55 21.03 23.34 24.05 26.22 28.30 30.32 32.91 34.82 13 15.98 16.98 18.20 19.81 22.36 24.74 25.47 27.69 29.82 31.88 34.53 36.48 14 17.12 18.15 19.41 21.06 23.68 26.12 26.87 29.14 31.32 33.43 36.12 38.11 15 18.25 19.31 20.60 22.31 25.00 27.49 28.26 30.58 32.80 34.95 37.70 39.72 16 19.37 20.47 21.79 23.54 26.30 28.85 29.63 32.00 34.27 36.46 39.25 41.31 17 20.49 21.61 22.98 24.77 27.59 30.19 31.00 33.41 35.72 37.95 40.79 42.88 18 21.60 22.76 24.16 25.99 28.87 31.53 32.35 34.81 37.16 39.42 42.31 44.43 19 22.72 23.90 25.33 27.20 30.14 32.85 33.69 36.19 38.58 40.88 43.82 45.97 20 23.83 25.04 26.50 28.41 31.41 34.17 35.02 37.57 40.00 42.34 45.31 47.50 21 24.93 26.17 27.66 29.62 32.67 35.48 36.34 38.93 41.40 43.78 46.80 49.01 22 26.04 27.30 28.82 30.81 33.92 36.78 37.66 40.29 42.80 45.20 48.27 50.51 23 27.14 28.43 29.98 32.01 35.17 38.08 38.97 41.64 44.18 46.62 49.73 52.00 24 28.24 29.55 31.13 33.20 36.42 39.36 40.27 42.98 45.56 48.03 51.18 53.48 25 29.34 30.68 32.28 34.38 37.65 40.65 41.57 44.31 46.93 49.44 52.62 54.95 26 30.43 31.79 33.43 35.56 38.89 41.92 42.86 45.64 48.29 50.83 54.05 56.41 27 31.53 32.91 34.57 36.74 40.11 43.19 44.14 46.96 49.64 52.22 55.48 57.86 28 32.62 34.03 35.71 37.92 41.34 44.46 45.42 48.28 50.99 53.59 56.89 59.30 29 33.71 35.14 36.85 39.09 42.56 45.72 46.69 49.59 52.34 54.97 58.30 60.73 30 34.80 36.25 37.99 40.26 43.77 46.98 47.96 50.89 53.67 56.33 59.70 62.16 40 45.62 47.27 49.24 51.81 55.76 59.34 60.44 63.69 66.77 69.70 73.40 76.09 50 56.33 58.16 60.35 63.17 67.50 71.42 72.61 76.15 79.49 82.66 86.66 89.56 60 66.98 68.97 71.34 74.40 79.08 83.30 84.58 88.38 91.95 95.34 99.61 102.70 80 88.13 90.41 93.11 96.58 101.90 106.60 108.10 112.30 116.30 120.10 124.80 128.30 100 109.10 111.70 114.70 118.50 124.30 129.60 131.10 135.80 140.20 144.30 149.40 153.20

Table  F  

df = (r−1)×(c−1)

If χ2 = 10.73 and df = 2, the P-value is between 0.0025 and 0.005.

The chi-value for the mindoxil example is 11.58. We see that it is far to the right of the chi-squared distribution. The area to the RIGHT of 11.58 is 0.7%. This matches with the Statcrunch output. This can be deduced from the tables. Since 11.58 > 3.84 (which corresponds to 5% in the chi table) it is clear that the p-value corresponding to 11.58 is a lot less than 5%. Interpretation If there is no association between treatment and hair, then about 0.7% of the time we will observe a difference in the data of 11.58 times or more.

35.14% 35.14% 35.14%

Observed % of No Relapse by Treatment

Example  4:  Treating  cocaine  addiction  74 patients addicted to cocaine were assigned at random to one of three possible treatments. The observed variable is whether or not they relapsed into addiction after their treatment. We test whether the chance of relapse is related to treatment with α = 0.05.

Overall 26/74 = 35.14% did not relapse. If this occurrence does not depend on the treatment then we would “expect” 35.14% of each group not to relapse. This is what a null hypothesis of “no association” would predict.

Expected % of No Relapse by Treatment

No Yes Total

Desipramine (25×26)/74 = 8.78

(25×48)/74 = 16.22

25

Lithium (26×26)/74 = 9.14

(26×48)/74 = 16.86

26

Placebo (23×26)/74 = 8.08

(23×48)/74 = 14.92

23

Total 26 48 74

Treating  cocaine  addiction,  cont.  

We have noted that, assuming treatment has no effect on the relapse response, each treatment group should be 35.14% “no relapse” and 64.85% “relapse”. The expected counts are computed using the margin totals from the observed (actual) counts.

Observed counts

Expected counts

No Yes Total

Desipramine 15 10 25

Lithium 7 19 26

Placebo 4 19 23

Total 26 48 74

Comparing the observed and expected counts, we can see that the two do not agree very well. But to say whether this is significant, we need to compute a test statistic and a P-value.

Row totals are the same for both tables. Column totals are the same for both tables.

No Yes Total

Desipramine 8.78 16.22 25

Lithium 9.14 16.86 26

Placebo 8.08 14.92 23

Total 26 48 74

Treating  cocaine  addiction,  cont.  Now we compute the χ2 test statistic.

Observed counts

Expected counts

No Yes Total

Desipramine 15 10 25

Lithium 7 19 26

Placebo 4 19 23

Total 26 48 74

The degrees of freedom is (3−1) ×(2−1) = 2. From Table F, we find that the area to the right of 10.73 is between 0.0025 and 0.005.

2

2 2

2 2

2 2

(obs. exp.)2exp.

(15 8.78) (10 16.22)8.78 16.22

(7 9.14) (19 16.86)9.14 16.86

(4 8.08) (19 14.92)8.08 14.92

10.73.

−

− −

− −

− −

χ =

= +

+ +

+ +

=

∑

The P-value is less than α = 0.05 so we conclude that the chance of relapse is related to the treatment. A causal effect also is indicated because the treatment was applied and then the response (relapse/no relapse) was observed. For many data sets we cannot say there is a causal effect – see HW9, Q4.

Observed % of no relapse

Cocaine addiction, cont.

The P-value is 0.0047, which is very significant. We reject the null hypothesis of no association and conclude there is a relationship between the treatment and the outcome (relapse or not).

From StatCrunch: use Stat-Tables-Contingency-with summary. The counts are to be provided in a table format.

Meaning of conditional probabilities: Cocaine addiction, cont.

Since the outcome (relapse or not) is a response variable, it is sensible to ask about its conditional distribution, given each treatment, and then to compare across treatments. For example, 60% of addicts treated with Desipramine did not relapse and 27% of addicts treated with Lithium did not relapse while only 17% of addicts treated with the placebo did not relapse.

But it is not sensible to look at the conditional proportions for treatment, given the outcome, because treatment is not a response variable in this study: it was applied to the patients (subjects). In fact, the row totals are the sizes of three samples and are not random.

P (No|Placebo) = 4

23

= 17.39% has meaning

P (Placebo|No) = 4

26

= 15.38% has no interpretation

Cocaine addiction, cont.

Using the confidence interval formula for a single proportion, we can estimate the individual proportions of “no relapse” for each treatment. For Desipramine:

For Lithium:

For Placebo:

.600 1.96 .600(.400) / 25 .600 .192(.408,.792).± × = ±

=

.269 1.96 .269(.731) / 26 .269 .170(.099,.439).± × = ±

=

.174 1.96 .174(.826) / 25 .174 .155(.019,.329).± × = ±

=

We can also compare two groups, say Desipramine and Lithium:

.600(.400) .269(.731)25 26.600 .269 1.96

.331 .257 (.074,.588).

− ± × +

= ± =

Does right/left-handedness of students vary between

genders? We will test

H0: There is no relationship between

handedness and gender,

Ha: There is some relationship,

at significance level α = 0.01.

From the class survey, we get the summary in the

table at right. In StatCrunch use: Stat-Tables-

Contingency-with data.

The results of a chi-square test show that the P-value

is 4.33%. Since this is bigger than α, we do not

reject H0. There is insufficient evidence to conclude

that handedness differs by gender.

Example  5:  Left-­‐handed  students    

Note: only 407 students responded to both questions.

Example  6:  Parental  smoking    Does parental smoking influence the smoking habits of their high school children?

High school students were asked whether they smoke and whether their parents smoke.

The percent of students who smoke is greatest when both parents smoke and least when neither parent smokes (22% vs. 14%).

If a student smokes, it is more likely that both parents do (40%) than that neither parent does (19%).

In StatCrunch, request row percents (if the column variable is a response) and column percents (if the row variable is a response). The proportion of students who smoke, among those for whom both parents smoke, is 400/1780 = 22.47%. The proportion of students for whom both parents smoke, among those who smoke, is 400/1004 = 39.84%.

The chi-square test is an overall technique for determining evidence of

a relationship between two categorical variables. There are two cases.

p  Compare category proportions for several populations. A simple

random sample is selected from each population and a single categorical

variable is observed.

p  The cocaine addiction study is an example of this. The populations are the 3

treatments. A SRS was obtained for each treatment. One response was observed.

p  Test independence of two categorical variables in a single population.

A single random sample is obtained from the populations and each

individual is classified according to the two categorical variables.

p  The handedness survey is an example of this. There was one random sample

from the population and each student responded to the two categorical questions.

We use the χ2 test to test the null hypothesis of no relationship for both.

Experimental  designs  for  two-­‐way  tables  

The chi-square statistic (χ2) is a measure of how much the observed cell counts in a two-way table diverge from the expected cell counts.

Summing over all r×c cells in the table (r and c are the number of rows and columns), the formula for the χ2 statistic is

( ) 22 observed count expected count

expected count

−χ =∑

Large values for χ2 represent strong deviations from the expected distribution under the H0 and provide evidence against H0.

How large a χ2 value is required for statistical significance will depend on its degrees of freedom df = (r−1)×(c−1).

Review:  The  chi-­‐square  test  statistic  

Beware: the denominators are expected counts. Do not include the margins in the sum.

Summary  of  testing  for  association  p  This test pertains to association between two categorical

variables only. The hypotheses are: H0: the variables are not associated vs. Ha: the variables are associated.

p  The data are summarized in a r×c table with cells containing the observed counts for each combination of categories of the two variables.

r is the number of rows (categories of 1st variable) and c is the number of columns (categories of 2nd variable).

p  The expected count for each cell in the table is computed by

p  The test statistic is

The P-value is computed from the chi-square distribution as the area to the right of the χ2 statistic, with df = (r−1)×(c−1).

( ) 22 observed count expected count

.expected count

−χ =∑

row total column totalexpected count .total sample size

×=

When  is  it  “safe”  to  use  a  χ2  test?  

Like the z-tests for proportions, the chi-square test is based on an

approximation. We can safely use the test when:

p  The samples are simple random samples (SRS).

p  All but one or two individual observed counts are 1 or more.

p  All expected counts are 5 or more, except perhaps one which

should be at least 1.

For a 2 x 2 table, all four expected counts should be 5 or more.

If the approximation is not appropriate, a statistician should be

consulted for the proper procedure.

Accompanying  problems  associated  with  this  Chapter  

p  Quiz p  Homework 9