Testing means, part II

The paired t-test

Outline of lecture

• Options in statistics– sometimes there is more than one option

• One-sample t-test: review– testing the sample mean

• The paired t-test– testing the mean difference

A digression:Options in statistics

Example

• A student wants to check the fairness of the loonie

• She flips the coin 1,000,000 times, and gets heads 501,823 times.

• Is this a fair coin?

Ho: The coin is fair (p

heads = 0.5).

Ha: The coin is not fair (p

heads ≠ 0.5).

n = 1,000,000 trialsx = 501,823 successes

Under the null hypothesis, the number of successesshould follow a binomial distribution with n=1,000,000and p=0.5

4 9 8 0 0 0 4 9 9 0 0 0 5 0 0 0 0 0 5 0 1 0 0 0 5 0 2 0 0 0

0

e+

00

2

e-0

44

e

-04

6

e-0

48

e

-04

x

Pro

ba

bili

ty

Test statistic

Binomial test

• P = 2*Pr[X≥501,823]P = 2*(Pr[X = 501,823] + Pr[X = 501,824] +

Pr[X = 501,825] + Pr[X = 501,826] + ...

+ Pr[X = 999,999] + Pr[X = 1,000,000]

Central limit theorem

The sum or mean of a large number of measurementsrandomly sampled from any population is approximately normally distributed

Binomial Distribution

Normal approximation to the binomial distribution

The binomial distribution, when number of trials n is large

and probability of success p is not close to 0 or 1, is

approximated by a normal distribution having mean np and

standard deviation

 

np 1 - p( ) .

Example

• A student wants to check the fairness of the loonie

• She flips the coin 1,000,000 times, and gets heads 501,823 times.

• Is this a fair coin?

Normal approximation

• Under the null hypothesis, data are approximately normally distributed

• Mean: np = 1,000,000 * 0.5 = 500,000• Standard deviation:

• s = 500s= n p 1− p= 1,000,000∗0.5∗1−0.5

Normal distributions

• Any normal distribution can be converted to a standard normal distribution, by

Z-score

Z=Y −ms

Z=501,823−500,000500

=3.646

From standard normal table:P = 0.0001

Z=Y −ms

Conclusion

• P = 0.0001, so we reject the null hypothesis• This is much easier than the binomial test• Can use as long as p is not close to 0 or 1

and n is large

Example

• A student wants to check the fairness of the loonie

• She flips the coin 1,000,000 times, and gets heads 500,823 times.

• Is this a fair coin?

A Third Option!

• Chi-squared goodness of fit test• Null expectation: equal number of successes

and failures• Compare to chi-squared distribution with 1

d.f.

Result Observed ExpectedHeads 501823 500000Tails 498167 500000

Test statistic: 13.3Critical value: 3.84

Coin toss example

Binomial test Normal approximation

Chi-squaredgoodness of fit test

Most accurateHard to calculateAssumes:Random sample

ApproximateEasier to calculateAssumes:Random sampleLarge np far from 0, 1

ApproximateEasier to calculateAssumes:Random sampleNo expected <1Not more than 20% less than 5

Coin toss example

Binomial test Normal approximation

Chi-squaredgoodness of fit test

in this case, n very large (1,000,000)all P < 0.05, reject null hypothesis

Normal distributions

• Any normal distribution can be converted to a standard normal distribution, by

Z-score

Z=Y −ms

t distribution

• We carry out a similar transformation on the sample mean

t=Y−ms / n

mean under Ho

estimatedstandard error

How do we use this?

• t has a Student's t distribution• Find confidence limits for the mean• Carry out one-sample t-test

t has a Student’s t distribution*

t has a Student’s t distribution*

* Under the null hypothesis

Uncertaintymakes the nulldistributionFATTER

Confidence interval for a mean

(2) = 2-tailed significance level

df = degrees of freedom, n-1

SEY = standard error of the mean

Y ±SE Y t2 , df

Confidence interval for a mean

Y±SE Y t2 ,df

95 % Confidence interval:Use α(2) = 0.05

Confidence interval for a mean

Y±SE Y t2 ,df

c % Confidence interval:Use α(2) = 1-c/100

SampleNull hypothesis

The population mean is equal to

o

One-sample t-test

Test statistic

t=Y− o

s / n

Null distributiont with n-1 dfcompare

How unusual is this test statistic?

P < 0.05 P > 0.05

Reject Ho Fail to reject Ho

The following are equivalent:

• Test statistic > critical value• P < alpha• Reject the null hypothesis• Statistically significant

Quick reference summary: One-sample t-test

• What is it for? Compares the mean of a numerical variable to a hypothesized value, μ

o

• What does it assume? Individuals are randomly sampled from a population that is normally distributed

• Test statistic: t

• Distribution under Ho: t-distribution with n-1 degrees of

freedom

• Formulae:Y = sample mean, s = sample standard deviation

t=Y− o

s / n

32

Comparing means

• Goal: to compare the mean of a numerical variable for different groups.

• Tests one categorical vs. one numerical variable

Example:gender (M, F) vs. height

33

Paired vs. 2 sample comparisons

34

Paired designs

• Data from the two groups are paired• There is a one-to-one correspondence

between the individuals in the two groups

35

More on pairs

• Each member of the pair shares much in common with the other, except for the tested categorical variable

• Example: identical twins raised in different environments

• Can use the same individual at different points in time

• Example: before, after medical treatment

36

Paired design: Examples

• Same river, upstream and downstream of a power plant

• Tattoos on both arms: how to get them off? Compare lasers to dermabrasion

37

Paired comparisons - setup

• We have many pairs

• In each pair, there is one member that has one treatment and another who has another treatment

• “Treatment” can mean “group”

38

Paired comparisons

• To compare two groups, we use the mean of the difference between the two members of each pair

39

Example: National No Smoking Day

• Data compares injuries at work on National No Smoking Day (in Britain) to the same day the week before

• Each data point is a year

40

data

Year Injur ies before NoSmoking Day

Injur ies on NoSmoking Day

1987 516 540

1988 610 620

1989 581 599

1990 586 639

1991 554 607

1992 632 603

1993 479 519

1994 583 560

1995 445 515

1996 522 556

41

Calculate differencesInjur ies before No

Smoking DayInjur ies on NoSmoking Day

Differ ence

(d)

516 540 24

610 620 10

581 599 18

586 639 53

554 607 53

632 603 -29

479 519 40

583 560 -23

445 515 70

522 556 34

42

Paired t test

• Compares the mean of the differences to a value given in the null hypothesis

• For each pair, calculate the difference.

• The paired t-test is a one-sample t-test on the differences.

43

Hypotheses

Ho: Work related injuries do not change duringNo Smoking Days (μ=0)

Ha: Work related injuries change during No Smoking Days (μ≠0)

44

Calculate differencesInjur ies before No

Smoking DayInjur ies on NoSmoking Day

Differ ence

(d)

516 540 24

610 620 10

581 599 18

586 639 53

554 607 53

632 603 -29

479 519 40

583 560 -23

445 515 70

522 556 34

45

Calculate t using d’s

 

d = 25

sd2 = 1043 .78

n =10

 

t =25 - 0

1043 .78 /10= 2.45

46

Caution!

• The number of data points in a paired t test is the number of pairs. -- Not the number of individuals

• Degrees of freedom = Number of pairs - 1

Here, df = 10-1 = 9

47

Critical value of t

So we can reject the null hypothesis: Stopping smoking increases job-related accidents in the short term.

t 0.05 2 , 9=2.26

Test statistic: t = 2.45

48

Assumptions of paired t test

• Pairs are chosen at random

• The differences have a normal distribution

It does not assume that the individual values are normally distributed, only the differences.

Quick reference summary: Paired t-test

• What is it for? To test whether the mean difference in a population equals a null hypothesized value, μ

do

• What does it assume? Pairs are randomly sampled from a population. The differences are normally distributed

• Test statistic: t

• Distribution under Ho: t-distribution with n-1 degrees of

freedom, where n is the number of pairs

• Formula:

t=d− do

SE d