Socratic Logic, Statistical Hypotheses And Significance Testing

© 2009 Max Chipulu, University of Southampton- Socratic Logic, Hypotheses and Statistical Significance

© 2003 Max Chipulu

1

Previously…

• Randomness

• Descriptive Statistics


© 2003 Max Chipulu

2

Next Four Weeks

• Introduction to Statistical Modelling

• Types of Data

• Simulation

• Discrete Probability Distributions

• The Method of Maximum Likelihood

• Continuous Probability Distributions

• Association and Correlation

• Regression Analysis

• Module Review


Socratic Logic and Statistical Hypothesis TestingSocratic Logic and Statistical Hypothesis Testing

ObjectivesObjectives

1. To introduce the steps in Statistical Modelling

2. To discuss what are statistical hypotheses and how to test them

3. To discuss the three types of II in Statistical Inference

4. To discuss the concept of Statistical Significance

• To introduce hypothesis testing using

Significance tests


Have you come across this picture before?


It is a famous painting It is a famous painting

of Socrates about to of Socrates about to

drink his poison in 399 drink his poison in 399

BC BC

It was his death sentence.It was his death sentence.


Socrates was a Wise ManSocrates was a Wise Man

‘I reflected as I walked away, well, I am certainly wiser than this man. It is only too

likely that neither of us has any knowledge to boast of, but he thinks that he knows

something which he does not know, whereas I am quite conscious of my ignorance. At any rate, it seems that I am wiser than he is to the small extent, that I do not think that I

know what I do not know.’

••For example, Plato quotes:For example, Plato quotes:


SocratesSocrates’’ CrimesCrimes

• Socrates was brilliant at argument

• He was often to be found in debate with groups of the impressionable idle young men of Athens

• But there was war on; and the Athenian establishment was nervous

• They said he was ‘corrupting the young’


SocratesSocrates’’ CrimesCrimes

• They also said he rejected the traditional gods;

that he introduced new gods!

• This was against the Athenian Law, and

Socrates was charged; it was a serious charge,

which carried the death sentence

• So Socrates had to prove that he was innocent


The Test of SocratesThe Test of Socrates’’ InnocenceInnocence

• In that Athens, it was possible to talk with

the Gods through the oracle at Delphi

• Socrates’ friend asked the oracle, ‘Is there

any man wiser than Socrates?’

• ‘No’, said the Delphic oracle


The DecisionThe Decision

• Since the Gods were not unhappy with

Socrates, then surely he was not guilty of the

charges?

• Not according to the Jury; it rejected these

arguments

• And Socrates was condemned to death


Statistical ReasoningStatistical Reasoning

But Socrates’ way of thinking remains very

valid in tackling contestable theories in

Statistics

We can see this by looking at an everyday

analogy of Socrates’ Reasoning:

Does Tobacco Smoking Cause Cancer?


How to test whether How to test whether

smoking increases smoking increases

Cancer ratesCancer rates

2. Design an 2. Design an

Experiment to test HExperiment to test H00

How to demonstrate How to demonstrate

‘‘Not guiltyNot guilty’’: Consult the : Consult the

GodsGods

Random Sample of Random Sample of

Smokers/NonSmokers/Non--smokers?smokers?

3. Collect the Data to 3. Collect the Data to

test Htest H00

Record What the Record What the

Delphi SaysDelphi Says

Tobacco does not cause Tobacco does not cause

cancercancer

1. State the Maintained 1. State the Maintained

Hypothesis or HHypothesis or H00

Socrates was not Socrates was not

Guilty of the ChargesGuilty of the Charges

Everyday ProblemEveryday ProblemThe General ProblemThe General ProblemSocratesSocrates’’ ApologyApology

Statistical ReasoningStatistical Reasoning


Previous results suggest Previous results suggest

HH00

Must be rejectedMust be rejected

7. Error type II: If 7. Error type II: If

Wrong HWrong H0 0 is acceptedis accepted✔✔ ✔✔6. Error type I: If 6. Error type I: If

Correct HCorrect H0 0 is rejectedis rejected

But HBut H00 was rejected:was rejected:

They found Socrates They found Socrates

guiltyguilty

‘‘Yes, there is a differenceYes, there is a difference’’, ,

Reject HReject H00

5. Decision: Reject or 5. Decision: Reject or

Accept Hypothesis, HAccept Hypothesis, H00

‘‘YesYes’’, hence CANNOT , hence CANNOT

reject Hreject H00

Is difference in rates Is difference in rates

between the 2 samples between the 2 samples

nonnon--zero?zero?

4. Analyse the data with 4. Analyse the data with

a Statistical Testa Statistical Test

Are the Gods happy Are the Gods happy

with Socrates?with Socrates?

Everyday ProblemEveryday ProblemThe General ProblemThe General ProblemSocratesSocrates’’ ApologyApology


It is no surprise at all that the word ‘hypothesis’ derives from Greek


Errors in Statistical Inference

• Given a null hypothesis and an alternative

hypothesis, the decision will be (1) to reject the

null hypothesis or (2) to fail to reject (i.e.

accept) the null hypothesis. Both decisions 1

and 2 could be in error:

• Error type I: correct hypothesis is rejected

• Error type II: the wrong hypothesis is accepted


Maybe even more serious, error type III is…

• “An approximate solution to the right problem is much better than an exact solution to the wrong problem”, George Box, Statistics Icon, most famous for his work on time series forecasting ‘Box-Jenkins’ method

•To ask the wrong question, i.e. propose the

wrong hypothesis in the first place:


Statistical Significance: Weird Good Vs Weird Bad

• Scenario 1

• Suppose you wake up one day and you have a headache. There is no apparent cause for this headache, e.g. it is NOT the morning after a party.

• Would you:

• A: Think ‘this is a bit weird- a random headache’-Basically ignore it, maybe take a couple of pain killers?; Or

• B: Think ‘oh my, this could be serious, I must go and consult a doctor immediately’

• Please show your hand for A or B…


Statistical Significance: Weird Good Vs Weird Bad

• Scenario 2

• Now suppose you continue to suffer a headache for several days. Let us say you have it for ten days. Would you:

• A: Basically ignore it- continue taking pain killers, or

• B: Think, ‘This must be serious, I must go and consult a doctor’?

• Please show your hand for A or B


What is the difference between Scenario 1 and 2?

• The difference is believability or rather lack of believability:

• In scenario 1, most people would believe that the headache is just ‘random’ and it will pass. Why? Well, because the probability that anyone could suffer a random headache on any given day is not small- it happens frequently.

• In scenario 2, most people would NOT believe that this a random headache because the probability of a random headache for 10 consecutive days is very, very small.

• This is statistical significance. Scenario 1 is NOT statistically significant; Scenario 2 is. How? Well let us see how this works in terms of hypothesis.


Weird Good=‘Random’ Weird Bad=‘Not Random’, i.e. Statistically Significant

• Suppose that the null hypothesis, H0 = ‘the headache is random’

• Then the alternative hypothesis is Ha = ‘the headache is NOT random’

• So when the result, i.e. headache for one day or headache for 10 days is observed, the question is can we believe this headache is random? If the probability is small and we cannot believe that the headache is random, then we must reject the null hypothesis, i.e. we conclude that the observed headache is statistically significant.

• Usually, we will reject the null hypothesis if the probability of observing the result (or something worse) under the null hypothesis is 0.05 (5%) or less.


Tests of Significance

In order to reject or accept a null hypothesis, we

must work out whether the probability of the

observed result is small under the null hypothesis.

So, always, we need a test probability distribution

that the observed result would follow under the null

hypothesis. Such a probability distribution is called a

statistical test of significance.

ALL hypothesis tests are based on a test of Significance:


Binomial Test �

Chi-square �

KS (similar to Chi-square but uses proportions) �

Wilcoxon (T-test equivalent for �

categorical data) �

Z-test (result assumed to follow the normal distribution) �

T-test: results assumed to follow t-distribution, e.g. least squares regression coefficients. �

F-test of variance, e.g. variance explained by a regression �

Most Common Examples

Not as exact as parametric tests but research shows are almost as

powerful

More exact than non-parametric tests, therefore whenever

appropriate use parametric testsAdvantages

Categorical data or quantitative data for small samples when most parametric

assumptions are violated

Quantitative data taken from large samples

Usually applicable

for

Since no assumptions are made, applications can be flexible

Strictly only appropriate if assumptions have not been

violatedApplication

We make NO assumptions about the type of probability distribution function that the observed result

might follow

We assume that the observed result follows a specific probability distribution function, e.g. the

normal distribution

Characteristic

Non-ParametricParametric

Type of Test of Significance


Further Reading

• Rae R. Newton and Kjell Erik Rudestam,

1999. Your Statistical Consultant. Sage

Publications Inc.

• Ramon E. Henkel, 1976. Tests of Significance.

Sage Publications Inc. (In Library box HA 33

QAA).

Socratic Logic, Statistical Hypotheses And Significance Testing

Business

Transcript of Socratic Logic, Statistical Hypotheses And Significance Testing