Nonparametric Tests of Significance

Statistics for Political ScienceLevin and FoxChapter Nine

Part One

What is a parametric test?

When a test of significance requires:

1. Normality in the population (a normal distribution) or at least large samples so that the sampling distribution is normal.

2. An interval-level measure.

Nonparametric tests of significance have a list of requirements that do not include normality or the interval level of measurement.

Nonparametric tests use a concept of power (power of a test): the probability of rejecting the null hypothesis when it is false and should be rejected. In other words, the probability of accurately claiming a statistically significant relationship does exist between two variables.

Non-parametric Tests

Power varies from test to test: The more powerful the test, the more likely the null hypothesis is to be

rejected when it is false and they have more difficult requirements to satisfy.

Less powerful tests have less stringent requirements on the data and the null hypothesis may be retained when it should be rejected.

Accuracy of different tests:The more powerful the test is, the more likely it is to accurately determine

whether or not a statistically significant relationship does exist between variables.

Non-parametric Tests

Nonparametric Tests:

Two Nonparametric Tests:

The Chi-Square Test: concerned with the distinction between expected frequencies and observed frequencies.

The Median Test: A chi-square based test that also evaluates whether the scores fall above or below the median.

Some things to know about chi square:

1) It compares the distribution of one variable (DV) across the category of another variable (IV)

2) It makes comparisons across frequencies rather than mean scores.3) It is a comparison of what we expect to what we observe.

Null versus Research Hypotheses:The null hypotheses states that the populations do not differ with respect to

the frequency of occurrence of a given characteristic, whereas a research hypothesis asserts that sample difference reflects population difference in terms of the relative frequency of a given characteristic.

Nonparametric Tests:

Chi Square: Example: Political Orientation and Child Rearing

Null Hypothesis: The relative frequency or percentage of liberals who are permissive IS the same as the relative frequency of conservatives who are permissive.

Research Hypothesis: The relative frequency or percentage of liberals who are permissive is NOT the same as the relative frequency of conservatives who are permissive.

Nonparametric Tests:

Chi Square: Example: Political Orientation and Child Rearing

Expected and Observed Frequencies:The chi-square test of significance is defined by Expected and Observed Frequencies.

Expected Frequencies (fe) is the frequency we would expect to get if the hull hypothesis is true, that is there is no difference between the populations.

Observed Frequencies (fo) refers to results we actually obtain when conducting a study (may or may not vary between groups).

Only if the difference between expected and observed frequencies is large enough do we reject the null hypothesis and decide that a population difference does exist.

Nonparametric Tests:

Nonparametric Tests:Chi Square: Political Orientation and Child Rearing: Observed Frequencies

13 7

7 13

Liberals Conservatives

Political Orientation

Child-Rearing Methods

Permissive

Not Permissive

Total

Total 20 20

20

20

N = 40

Col. Marginal

Col. Marginal

Row Marginal

Row Marginal

Chi Square: Example: Political Orientation and Child RearingSince the marginals are all equal, it is easy to calculate the expected frequencies: 10 in each cell.

10 10

10 10

Liberals Conservatives

Political Orientation

Child-Rearing Methods

Permissive

Not Permissive

Total

Total 20 20

20

20

N = 40

It is unusual for a study to produce row and column marginals that are evenly split.

Chi Square: Example: Political Orientation and Child RearingCalculating expected frequencies when the marginals are not even:

15 10

5 10

Liberals Conservatives

Political Orientation

Child-Rearing Methods

Permissive

Not Permissive

Total

Total 20 20

25

15

N = 40

To determine if these frequencies depart from what is expected (null) by chance alone, we have to calculate the expected frequencies.

Row Marginal

Row Marginal

If 25 of 40 respondents are permissive, than 62.5 % of them are permissive. To then determine the expected frequency, which asserts that Libs and Cons are the same (null) we have to calculate what would be 62.5% of 20 Libs and 20 Cons (the number of each that are in the study.

15 (12.5) 10 (12.5)

5 (7.5) 10 (7.5)

Liberals Conservatives

Political OrientationChild-Rearing

Methods

Permissive

Not Permissive

Total

Total 20 20

25 (62.5%)

15

N = 40

The answer is 12.5 (62.5% of 20 or .625 x 20). We then know that the expected frequency for non permissive is 7.5 (20 – 12.5).

Calculating Expected Frequencies

ffee = = (column marginal)(row marginal)(column marginal)(row marginal)

NN

Example:

fe = (20)(25) 40

= 500 40

= 12.5

15 (12.5) 10 (12.5)

5 (7.5) 10 (7.5)

Liberals Conservatives

Political OrientationChild-Rearing

Methods

Permissive

Not Permissive

Total

Total 20 20

25 (62.5%)

15

N = 40

The answer is 12.5 (62.5% of 20 or .625 x 20). We then know that the expected frequency for non permissive is 7.5 (20 – 12.5).

Example: fe = (25)(20) 40 = 500 40 = 12.5

The Chi-Square Test Formula

e

eo

f

ff 22 )(

Where:

fo = observed frequency in any cell

fe = expected frequency in any cell

Once we have the observed and expected frequencies we can use the following formula to calculate Chi-square.

Nonparametric Tests: Chi-Square Tests

ObservedObserved ExpectedExpected SubtractSubtract SquareSquare Divide by feDivide by fe

SumSum

After obtaining fo and fe, we subtract fe from fo, square the difference, divide by the fe and then add them up.

Nonparametric Tests: Chi-Square Tests

df = (r-1)(c-1)

Where

r = the number of rows of observed frequenciesc = the number of columns of observed frequencies

Formula for Finding the Degrees of Freedom

Formula for Finding the Degrees of Freedom

Formula for Finding the Degrees of FreedomSince there are two rows and two columns of observed frequencies in our 2 x 2 table

df = (r-1)(c-1)df = (2-1)(2-1)

= (1)(1) = 1

Next Step, Table E, where we will find a list of chi-square scores that are significant at .05 and .01 levels.

Table E (.05, df = 1): 3.84Obtained X = 2.66Retain null

2

Step by Step Chi-Square Test:

Step by Step Chi-Square Test:

1) Subtract each expected frequency from its corresponding observed frequency

2) Square the difference 3) Divide by the expected frequency, and then 4) Add up these quotients for all the cells to obtain the chi-square value

Comparing Several GroupsWhen comparing more than two groups, you use essentially the same

process as when comparing 2 x 2 tables.

Nonparametric Tests:

Chi Square: Example: Political Orientation and Child Rearing: 2 x 2:

Null Hypothesis: The relative frequency of permissive, moderate, and authoritarian child-rearing methods IS the same for Protestants, Catholics, and Jews.

Research Hypothesis: The relative frequency of permissive, moderate, and authoritarian child-rearing methods is NOT the same for Protestants, Catholics, and Jews.

Nonparametric Tests:

Chi Square: Example: Political Orientation and Child RearingCalculating expected frequencies when the marginals are not even:

15 (12.5) 10 (12.5)

5 (7.5) 10 (7.5)

Liberals Conservatives

Political Orientation

Child-Rearing Methods

Permissive

Not Permissive

Total

Total 20 20

25 (62.5%)

15

N = 40

Comparing Several Groups

Chi Square: Example: Political Orientation and Child Rearing

15 (12.5) 10 (12.5)

5 (7.5) 10 (7.5)

Liberals ConservativesChild-Rearing

Methods

Permissive

Not Permissive

Total

Total 20 20

25 (62.5%)

15

N = 40

Comparing Several Groups

Correcting for Small Frequencies

Generally, chi square should be used with great care whenever some of the frequencies are below Five (5).

Though, this is not a hard and fast rule.

Yate’s Correction

HOWEVER, when working with a 2x2 table where any expected frequency is less than 10 but greater than 5, use Yate’s correction which reduces the difference between the expected and observed frequencies.

e

eo

f

ff2

2 5.

The vertical indicate that we must reduce the absolute value (ignoring minus signs) of each fo – fe by .5

Yate’s Correction

Smoking StatusNationality

American Canadian

Nonsmokers

Smokers

15 (11.67)

6 (9.33)

5 (8.33)

10 (6.67)

20

16N = 36Total 21 15

ObservedObserved ExpectedExpected SubtractSubtract Subtract .5Subtract .5 SquareSquare Divide by feDivide by fe

SumSum

Requirements for the use of Chi-Square

Requirements for the use of Chi-Square: 1. A comparison between two or more samples.

2. Nominal data must be used.

3. Samples should have been randomly selected.

4. The expected cell frequencies should not be too small.

Median Test

Median Test: Is a simply nonparametric test for determining the likelihood that two or

more random samples have been taken from populations with the same median.

It involves conducting a Chi-Square test where one of the dimensions is whether the scores fall above or below the median of the two groups combined.

Median TestMedian Test: Example: Gender and Embarrassment Experiment asked 15 men and 12 women with average singing ability to sing in front of expert

judges (sound familiar?)

Men (N = 15) Women (N = 12)

469

111215161819212324252627

123578

101314172022

Median Test

Median Test: Example: Gender and Embarrassment Experiment asked 15 men and 12 women with average singing ability to sing

in front of expert judges (sound familiar?)

Step 1: Find Median of the two samples

Mdn = (N + 1) ÷ 2= 27 + 1 ÷ 2

Mdn = 14

Median Test

Median Test: Example: Gender and Embarrassment Experiment asked 15 men and 12 women with average singing ability to sing

in front of expert judges (sound familiar?)

Step 1: Find Median of the two samples

Mdn = (N + 1) ÷ 2= 27 + 1 ÷ 2

Mdn = 14

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 271 2 3 4 5 6 7 8 9 10 11 12 13 14

Median Test

Median Test: Example: Gender and Embarrassment Experiment asked 15 men and 12 women with average singing ability to sing

in front of expert judges (sound familiar?)

Step 2: Count the Number in each sample falling above and below the median

MedianGender

Men Women

Above

Below

10

5

3

9

Median Test

Median Test: Example: Gender and Embarrassment Mdn = (N + 1) ÷ 2

= 27 + 1 ÷ 2 Mdn = 14

Men: Below Mdn = 5 Above Mdn = 10

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27

1 2 3 4 5 6 7 8 9 10 11 12 13 14

Women: Below Mdn = 9 Above Mdn = 3

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27

1 2 3 4 5 6 7 8 9 10 11 12 13 14

Median TestMedian Test: Example: Gender and Embarrassment

Mdn = 14 minutes

Men Women

469

111215161819212324252627

123578

101314172022

14: 10 above, 5 below

14: 3 above, 9 below

Step 2: Count the Number in each sample falling above and below the median

13

14N = 2715 12

MedianGender

Men Women

Above

Below

10 (7.22)

5 (7.78)

3 (5.78)

9 (6.22) Total

ObservedObserved ExpectedExpected SubtractSubtract SquareSquareSubtract .5Subtract .5 Divide by feDivide by fe

SumSum

Step 3: Calculate the Degrees of Freedom

df = (r-1)(c-1) = (2-1)(2-1) = (1)(1) = 1

We then go to Table E, which tells us that at .05 and a df = 1 chi-square must exceed 3.84 to be considered statistically significant.

Obtained X = 3.13Table E = 3.84Retain Null hypothesis

There is insufficient that men and women differ in terms of embarrassment.

2