Hypothesis Testing I: 1 Hypothesis Testing – Part I.

Hypothesis Testing I: 1

Hypothesis Testing – Part I


Recall:

1. We learned how to describe data

• Made no assumptions about where the data came from

• Nor about method of sampling

2. We focused on methods of sampling

• Probability samples

• Learned how to calculate probabilities

• Focus on specific probability distributions


3. We learned how to estimate unknown population parameters

• Goal: to try to understand the characteristics (parameters) of the population that gave rise to our sample data

4. Now, we’ll learn how to evaluate alternative explanations for the data we have observed

• The purpose is to test a research hypothesis


Examples of Research Hypotheses

1. Early treatment, compared to later treatment, of

patients with an evolving Ml will result in better

heart function (ejection fraction) at 24 hours.

2. The implementation of policy “A” will result in a

reduction in the inappropriate use of a particular

drug.

3. The delivery of an educational intervention to high school students will result in greater use of “safer” sex practices.

4. The average cost of a particular procedure is $X.


To answer such questions,

• we collect data

• then analyze the data

• to see if they are compatible with the research hypothesis being true.

We reason by use of “proof by contradiction”

• Proof by example won’t work.

The critic can always claim a counter

example must exist.


The Logic of Statistical Hypothesis Testing

The investigator starts

• by presuming the NULL explanation, eg:

• The treatment as NO benefit

• The new cost is the same as the old (there is NO difference between cost of new and old)

• Data are then collected and evaluated for consistency with the NULL explanation


• If the data are NOT consistent with the null explanation

• then abandon the null explanation in favor of an alternative

• Typically, it is the alternative explanation that the investigator would like to advance


If the null hypothesis is not true –

Then some alternative hypothesis must be true

This suggests some guidelines:

1. We’ll let

“Ho” represent the null hypothesis

“Ha” represent the alternative hypothesis

(called “H-naught” or “H-a”


1. The null and alternative hypotheses are typically specified so that

• The null is the one we hope to contradict

• The null and alternative are

• Mutually exclusive

• Collectively exhaustive

• Both should be specified in advance! – before the data is collected.


The Research Hypothesis Ho and Ha

Examples

1. Early treatment post MI results in better function of the heart at 24 hours (better = higher)

Define study with:

Group 1 = “Early” 1 = true mean at 24

hours

Group 2 = “Late” 2 = true mean at 24 hours

Research hypothesis says 1 > 2

This is the alternative hypothesis since it is the explanation the investigator seeks to advance.


1 2 1 2: : 0o oH H

1 2 1 2: : 0a aH H

Thus, with the alternative defined, the null is defined to be anything other than the alternative:

That is, 1 2

Thus we have:

Note that we can rewrite the hypotheses to compare the difference between group means to zero


If our research hypothesis is that “early treatment leads to different heart function at 24 hours”

then our alternative is that the means are not

equal: 12

1 2 1 2: : 0o oH H

1 2 1 2: : 0a aH H


The first case is called a one-sided alternative

• we are interested in a change in only one

direction : 1 > 2

The second case is called a two-sided alternative

• we consider change in either direction away

from equality: 1 2


Example 2:

The average cost of a particular procedure is $X

Suppose a medical insurance company wants to pay no more than $500 for a particular surgical procedure:

Let = true average cost

The research hypothesis says 500

specify this as the alternative hypothesis

Thus, Ho: ≥ 500

Ha: < 500


Hypothesis test as “proof by contradiction”

1. Assume null hypothesis is true

2. Determine a “rejection region” corresponding

to values unlikely to occur

using this assumption (Ho true)

3. Either:

“Reject” Ho if the observed data is in the

rejection region.

OR

“Fail to Reject” Ho if data in the rejection region

with assumption


500

For example we will reject Ho: > 500

when X is low enough that we believe the

true mean must be less than $500

X

Rejection Region


Steps in Constructing a Statistical Hypothesis Test

1. Identify the research question

2. State the assumptions necessary for computing probabilities

3. Specify Ho and Ha and the α-level (usually α = 0.05)

4. Specify the test statistic

5. Specify a decision rule

6. Compute the test statistic and the achieved significance( or P - value) from sample data

7. Come to a “Statistical” Decision

8. Reach a Conclusion

9. Report a confidence interval


Example:

1. Identify the research question

Suppose the mean birth weight for 1998 of all US hospital births is known to be = 3400 gm with = 710 gm, based upon national birth certificate data.

How do births at Hospital A compare?

We are asking

Is the mean birth weight at Hospital A different from the national mean?


Experiment: Collect birth weights of 100 consecutive births at Hospital A and compute our sample mean of x = 3250 gm.

2. What Assumptions must we make about our data to compute probabilities?Assume:

• a random sample of births from

• a population with known = 710 gm (known national standard deviation).

Thus, by the central limit theorem:

2 2710~ , ~ ,

100X N X N

n


3. Specify null and alternative hypotheses:

Ho: The true mean birth weight at Hospital A is

the same as the national mean.

Ha: The true mean birth weight at Hospital A is

different from the national mean

Or

Ho: = 3400

Ha: 3400


4. Compute the Test Statistic

This is where the proof by contradiction thinking comes in.

We want to know:

If it is true that = 3400 gm (Ho)

• what are the chances of observing a sample mean as far away from = 3400 as x = 3250?


Since Ha is two-sided

(greater OR less than the value for Ho),

we want to know the probability represented by the

following shaded area:

3250 3400 3550

150 150

“What are the chances of observing x as far away

from the pop. mean =3400 as the one we have (3250)

in either direction” ?


We want to compute: Pr[ 3250] Pr[ 3550]x x

We know how to do this!

We can transform the probability calculation into an

equivalent one for a standard normal:

3250 3400 3550

150 150

/

xz

n

(2)Pr[ 3250]x


When we use = 3400, as presumed by Ho , the

resulting quantity is called a

a Test Statistic

More generally

• if we let o represent the value of specified

by Ho we have

Test Statistic:

/ox

zn


5. Specify Decision Rule

What is the probability of observing a sample

mean as far away from o as the mean we have

observed?”

This probability calculation is known as:

the achieved significance

or the significance of the data

or the p-value


Our decision rule might be

Reject Ho if the achieved significance is less

than 0.05

This is equivalent to saying,

• If the probability of observing a sample mean, x,

this far or farther from o

• is less than 5%,

• then we will reject Ho in favor of Ha.


6. Compute the test statistic from the sample data:

3250 34002.11

/ 710 / 100ox

zn


-2.11 0 2.11

Pr[ 3250] Pr[ 3550]x x (2)Pr[ 3250]x

3250 3400(2)Pr

/ 710 / 100ox

n

(2)Pr[ 2.11]z

The achieved significance (or P-value)is:

(2)(.0174)

.0348


7. Statistical Decision:

• Since 0.0348 is less than 0.05 we will reject Ho.

• We are saying that x = 3250 is sufficiently

different from µo = 3400

• that it suggests that Ho is not true and should be

abandoned.

• That is, if Ho is true, the probability of a sample

mean this far away is only .0348 or 3.5% – an

unlikely outcome, so reject Ho.


8. Conclusion:

Hospital A has babies of significantly different

birth weight than the US average.

In fact, the mean birth weight at Hospital A

appears to be lower than the US mean.


9. Compute a Confidence Interval Estimate of the true mean birth weight for babies at Hospital A

We have all the ingredients to compute a confidence interval estimate:

x = 3250, = 710, n=100

since the true standard deviation is known we use:

z.975 = 1.96 for a 95% confidence interval:

x z (/n) = 3250 (1.96)(710/10) = 3250 139.2

95% CI: (3110.8 , 3389.2)


Interpretation:

The hypothesized mean o = 3400 falls outside

(above) the 95% confidence interval.

It therefore seems likely that the mean birth weight at Hospital A is less than the overall US mean.

Your confidence interval should give a consistent result with your hypothesis test.

If it doesn’t – check your work!


Comparing CI estimates and Hypothesis Testing

• When conducting a hypothesis test, with an =.05 decision rule, we are centering an interval

around the hypothesized mean (0):

• When our observed sample mean (x) falls outside this interval, we interpret this as indicating, with 0.05 likelihood of error, that our sample comes from a distribution with a different mean

0z.975n z.975n

x ??


COMMENTS:

1. The “.05 rule” alone is very uninformative

• it leads to a “reject” or “do not reject” with no information about the data.

A better approach is to report both

• the achieved significance

• confidence interval estimate

You can then interpret these, while also leaving room for your reader to interpret.


2. Don’t forget the conclusion step!

Too often, only a p-value is reported or, worse

still, only a “reject” or “do not reject” is reported.

3. Statistical significance alone gives no clues

about biology.

Keep in mind that a standard error is a function

of sample size n. This means that by increasing n,

the SE can be made smaller and smaller.


Eventually, any observation can achieve statistical significance regardless of its biological relevance.

For example is a statistically significant change in blood pressure of 1 mm Hg very useful?

If we have a very large n, say n=1000 we might find such a difference of 1mmhg statistically significant, but it may not be a biologically meaningful distinction.


4. A statistical hypothesis test uses probabilities

based only on the null hypothesis (Ho) model!

The proof by contradiction thinking asks us to:

• presume that Ho is true

• then examine the plausibility of our data in light of this assumption.

• We either reject it, or we fail to do so.

We do not prove that Ho is correct.


5. We can summarize the results of statistical hypothesis testing as follows:

Type II error

Type I error

NULL HYPOTHESIS

Actually True Actually False

Fail to

Reject

DECISION

Reject


IF Ho is true and we (incorrectly) reject Ho

• we have type I error

• we can calculate Pr[type I error] =

IF Ha is true and we (incorrectly) fail to reject Ho

• we have type II error

• we must have a specific Ha model before

we can calculate Pr[type II error] =

IF Ha is true and we (correctly) reject Ho

• This occurs with probability = (1-)

which we call the “POWER” of a test


Example 2

Does a new treatment for cancer increase the survival time from diagnosis significantly beyond 38.3 months?

A sample of 100 subjects given the new treatment had a mean survival time of 46.9 months.

Assume the data are a random sample of survival times from a N() with = 43.3 months.

(e.g., we may know the distribution of survival times from prior studies)


SOLUTION.

2. Assumptions

We have a random sample of n=100 survival times from a population with = 43.3.

Thus, 243.3~ ,

100X N

3. Specify Ho and Ha

Research hypothesis suggests an increase in survival

Ho: 38.3

Ha: > 38.3 (one sided!)


4. Specify Test Statistic:

Since = 43.3 is known, we’ll use

38.3

/ 43.3 / 100ox x

zn

5. Decision Rule

• We’ll calculate z using observed data

• compute the achieved significance (p-value)

• and compare this to 0.05

• If it is less than 0.05 we will reject Ho

• otherwise we will “fail to reject” Ho


6. Calculations – Achieved significance

Be careful! For a one-sided test, we are

concerned with a probability in only 1

direction from o!

Pr[ 46.9]x 46.9 38.3Pr

/ 43.3 / 100ox

n

Pr[ 1.986]z .0233

.0233

1.986z

38.3 46.9


7. Statistical Decision

.023 < .05 Reject Ho

8. Conclusion

It is unlikely that the improvements in survival time are due to chance. The new treatment appears to significantly improves survival.

9. Confidence Interval on True Mean survival using new treatment:

z.975 = 1.96 for a 95% confidence interval, known :

x z (/n) = 46.9 (1.96)(43.3/10) = 46.9 8.49

95% CI: (38.41, 55.39)


A note on One-sided hypothesis tests:

Quite often, we are interested in a change in only one direction:

• Does a new drug increase the proportion of patients cured?

• Does a new policy decrease the hospital length of stay (LOS)?

A test that looks at a change in only one direction seems to make sense.

However in practice this is rarely done.


• If it is possible for the change to occur in either direction

• then a test should look for the change in either direction.

For example,

• the new drug could actually decrease the proportion of patients cured,

• or the new policy could potentially result in increased LOS due to unexpected side effects.

Standard practice is to use a two-sided test!


Recap of Significance Testing So Far

The Basic Idea

1. Compute the “probability of the data”

(achieved significance) presuming Ho to be true.

• Large Probabilities are consistent with Ho

-- do not reject

• Small Probabilities are NOT consistent with Ho

-- reject


“Probability of the Data”

We want to know the probability of a sample

statistic as extreme or more extreme than the one

observed.

One Sided Alternative

0 0

Two Sided Alternative

Distribution determined

by Ho

t or z = observed sample statistic


Next we will consider a couple of examples

that parallel the situations we have discussed

so far for confidence interval estimation.

We will also focus on computer analysis for

conducting hypothesis tests


Application 1: One Population, 2 Known, Test

ofhypothesis onmean,

1. Research Question: Serum enzyme A levels are

measured in 10 patients with a sample mean of 22.

If it is known that the population variance is 45 and

if normality is assumed, are the data consistent with

a population mean of 25?

That is, we have n=10 2=45 x=22

o=25


2. Assumptions

• Random sample of serum enzyme A levels

• from a Normal distribution with 2 = 45.

• Thus,

45~ ,

10X N

Why must we assume normality of the data for this example?

Is n particularly large for Central Limit Theorem

to hold?


3. Specify Ho and Ha :

The wording “are the data consistent with”

suggests a two sided alternative:

Ho: 25

Ha: 25

4. Test Statistic

2 known suggests use of Normal or

z-transformation:

/ox

zn


5. Decision Rule

We’ll calculate the achieved significance

(p-value) and compare to 05

Reject Ho for p<.05, else fail to reject.

6. Calculations

Test Statistic:

22 251.41

/ 45 /10ox

zn


- 1.41 0 1.41

For 2-sided test:Total area is theachieved level of significance or the p-value

Pr[ 1.41] Pr[ 1.41] 0.1586p z z

Achieved Significance:

Minitab examples



• 0.1586 represents the probability of a sample mean at least as far away from 25 as 22, if in fact the true mean is 25.

• 0.1586, or 15.86% is reasonably high.

• This suggests the data are consistent with the null hypothesis

Do NOT reject Ho

.1586 > .05 (or .1586 > )


8. Conclusion

• The data are consistent with an hypothesized mean serum A level of = 25.

Note

• we have not proven Ho is true, merely that

• with the evidence of our sample, = 25 is a reasonable possibility

• and we cannot reject it.


9. 95% Confidence Interval

x z.975(/n) = 22 1.96(2.12) = (17.5, 26.5)

Note that:

o = 25 is within the interval


Using Minitab: Stat Basic Stats 1-Sample Z

Test mean: Enter o

Sigma: Enter

Select variable to test


Z-Test

Test of mu = 25.00 vs mu not = 25.00The assumed sigma = 6.71

Variable N Mean StDev SE Mean Z PSerA 10 22.00 6.29 2.12 -1.41 0.16


Application 2: One Normal Population, 2 UNknown

Test of

1. Research Question

A drug company claims a certain capsule contains 2.5 milligrams of a drug X. An independent laboratory obtained a random sample of 20 capsules and measured the amount of the drug in each. The measurements were as follows:

3.31 1.30 0.61 2.42 1.94 2.23 2.35 0.96 2.97 2.91

1.70 2.05 3.15 2.54 1.84 2.23 1.94 0.88 0.83 1.92

Is the drug company claim correct?


2. Assumptions

We have data from a random sample from a normal distribution, 2 unknown.

3. Specify Ho and Ha

Ho: o = 2.5 mg

Ha: o 2.5 mg (Two-sided)

4. Test Statistic

2 UNknown suggests use of the t-transformation:

Why must we assume normality of the data?

So the t-distribution is applicable!

1/

on

xt

s n


5. Decision Rule

We’ll calculate the achieved significance level

(two-sided) and compare this to a type I error

of 0.05

6. Calculations (x = 2.00, s = .787)

Test Statistic:

Achieved significance:

2.0 2.52.83

/ .787 / 20ox

ts n

19 19Pr[ 2.83] Pr[ 2.83] 2(.0054) .0108p t t



• 0.0108 < 0.05 p-value < type I error ()

• Therefore, REJECT Ho

8. Conclusion

• The mean amount of drug in the capsules appears to be significantly different from 2.5 mg.

• In fact, the mean is significantly less than 2.5 mg.


9. Confidence Interval Estimate

x t19; .975 (se) = 2.00 2.093(.176) = (1.66, 2.35).

• Note that The confidence interval does not include the hypothesized mean amount of 2.50


The particular test just conducted is known as a

ONE-SAMPLE t-TEST.

• We have a sample from a single population

• We are comparing our observed sample mean to some hypothesized value for the mean.


Using Minitab: Stats Basic Stats 1-Sample t

Enter o

Select variable to test


T-Test of the Mean

Test of mu = 2.500 vs mu not = 2.500

Variable N Mean StDev SE Mean T P

drugx 20 2.004 0.787 0.176 -2.82 0.011

Note that the one sample t-test provides you with estimates of the mean, standard deviation and standard error.

By checking the Confidence Interval option, you can get a confidence interval rather than a hypothesis test.

Hypothesis Testing I: 1 Hypothesis Testing – Part I.

Documents

Transcript of Hypothesis Testing I: 1 Hypothesis Testing – Part I.