© 2009 Max Chipulu, University of Southampton- Socratic Logic, Hypotheses and Statistical Significance
© 2003 Max Chipulu
1
Previously…
• Randomness
• Descriptive Statistics
© 2009 Max Chipulu, University of Southampton- Socratic Logic, Hypotheses and Statistical Significance
© 2003 Max Chipulu
2
Next Four Weeks
• Introduction to Statistical Modelling
• Types of Data
• Simulation
• Discrete Probability Distributions
• The Method of Maximum Likelihood
• Continuous Probability Distributions
• Association and Correlation
• Regression Analysis
• Module Review
© 2009 Max Chipulu, University of Southampton- Socratic Logic, Hypotheses and Statistical Significance
Socratic Logic and Statistical Hypothesis TestingSocratic Logic and Statistical Hypothesis Testing
ObjectivesObjectives
1. To introduce the steps in Statistical Modelling
2. To discuss what are statistical hypotheses and how to test them
3. To discuss the three types of II in Statistical Inference
4. To discuss the concept of Statistical Significance
• To introduce hypothesis testing using
Significance tests
© 2009 Max Chipulu, University of Southampton- Socratic Logic, Hypotheses and Statistical Significance
Have you come across this picture before?
© 2009 Max Chipulu, University of Southampton- Socratic Logic, Hypotheses and Statistical Significance
It is a famous painting It is a famous painting
of Socrates about to of Socrates about to
drink his poison in 399 drink his poison in 399
BC BC
It was his death sentence.It was his death sentence.
© 2009 Max Chipulu, University of Southampton- Socratic Logic, Hypotheses and Statistical Significance
Socrates was a Wise ManSocrates was a Wise Man
‘I reflected as I walked away, well, I am certainly wiser than this man. It is only too
likely that neither of us has any knowledge to boast of, but he thinks that he knows
something which he does not know, whereas I am quite conscious of my ignorance. At any rate, it seems that I am wiser than he is to the small extent, that I do not think that I
know what I do not know.’
••For example, Plato quotes:For example, Plato quotes:
© 2009 Max Chipulu, University of Southampton- Socratic Logic, Hypotheses and Statistical Significance
SocratesSocrates’’ CrimesCrimes
• Socrates was brilliant at argument
• He was often to be found in debate with groups of the impressionable idle young men of Athens
• But there was war on; and the Athenian establishment was nervous
• They said he was ‘corrupting the young’
© 2009 Max Chipulu, University of Southampton- Socratic Logic, Hypotheses and Statistical Significance
SocratesSocrates’’ CrimesCrimes
• They also said he rejected the traditional gods;
that he introduced new gods!
• This was against the Athenian Law, and
Socrates was charged; it was a serious charge,
which carried the death sentence
• So Socrates had to prove that he was innocent
© 2009 Max Chipulu, University of Southampton- Socratic Logic, Hypotheses and Statistical Significance
The Test of SocratesThe Test of Socrates’’ InnocenceInnocence
• In that Athens, it was possible to talk with
the Gods through the oracle at Delphi
• Socrates’ friend asked the oracle, ‘Is there
any man wiser than Socrates?’
• ‘No’, said the Delphic oracle
© 2009 Max Chipulu, University of Southampton- Socratic Logic, Hypotheses and Statistical Significance
The DecisionThe Decision
• Since the Gods were not unhappy with
Socrates, then surely he was not guilty of the
charges?
• Not according to the Jury; it rejected these
arguments
• And Socrates was condemned to death
© 2009 Max Chipulu, University of Southampton- Socratic Logic, Hypotheses and Statistical Significance
Statistical ReasoningStatistical Reasoning
But Socrates’ way of thinking remains very
valid in tackling contestable theories in
Statistics
We can see this by looking at an everyday
analogy of Socrates’ Reasoning:
Does Tobacco Smoking Cause Cancer?
© 2009 Max Chipulu, University of Southampton- Socratic Logic, Hypotheses and Statistical Significance
How to test whether How to test whether
smoking increases smoking increases
Cancer ratesCancer rates
2. Design an 2. Design an
Experiment to test HExperiment to test H00
How to demonstrate How to demonstrate
‘‘Not guiltyNot guilty’’: Consult the : Consult the
GodsGods
Random Sample of Random Sample of
Smokers/NonSmokers/Non--smokers?smokers?
3. Collect the Data to 3. Collect the Data to
test Htest H00
Record What the Record What the
Delphi SaysDelphi Says
Tobacco does not cause Tobacco does not cause
cancercancer
1. State the Maintained 1. State the Maintained
Hypothesis or HHypothesis or H00
Socrates was not Socrates was not
Guilty of the ChargesGuilty of the Charges
Everyday ProblemEveryday ProblemThe General ProblemThe General ProblemSocratesSocrates’’ ApologyApology
Statistical ReasoningStatistical Reasoning
© 2009 Max Chipulu, University of Southampton- Socratic Logic, Hypotheses and Statistical Significance
Previous results suggest Previous results suggest
HH00
Must be rejectedMust be rejected
7. Error type II: If 7. Error type II: If
Wrong HWrong H0 0 is acceptedis accepted✔✔ ✔✔6. Error type I: If 6. Error type I: If
Correct HCorrect H0 0 is rejectedis rejected
But HBut H00 was rejected:was rejected:
They found Socrates They found Socrates
guiltyguilty
‘‘Yes, there is a differenceYes, there is a difference’’, ,
Reject HReject H00
5. Decision: Reject or 5. Decision: Reject or
Accept Hypothesis, HAccept Hypothesis, H00
‘‘YesYes’’, hence CANNOT , hence CANNOT
reject Hreject H00
Is difference in rates Is difference in rates
between the 2 samples between the 2 samples
nonnon--zero?zero?
4. Analyse the data with 4. Analyse the data with
a Statistical Testa Statistical Test
Are the Gods happy Are the Gods happy
with Socrates?with Socrates?
Everyday ProblemEveryday ProblemThe General ProblemThe General ProblemSocratesSocrates’’ ApologyApology
© 2009 Max Chipulu, University of Southampton- Socratic Logic, Hypotheses and Statistical Significance
It is no surprise at all that the word ‘hypothesis’ derives from Greek
© 2009 Max Chipulu, University of Southampton- Socratic Logic, Hypotheses and Statistical Significance
Errors in Statistical Inference
• Given a null hypothesis and an alternative
hypothesis, the decision will be (1) to reject the
null hypothesis or (2) to fail to reject (i.e.
accept) the null hypothesis. Both decisions 1
and 2 could be in error:
• Error type I: correct hypothesis is rejected
• Error type II: the wrong hypothesis is accepted
© 2009 Max Chipulu, University of Southampton- Socratic Logic, Hypotheses and Statistical Significance
Maybe even more serious, error type III is…
• “An approximate solution to the right problem is much better than an exact solution to the wrong problem”, George Box, Statistics Icon, most famous for his work on time series forecasting ‘Box-Jenkins’ method
•To ask the wrong question, i.e. propose the
wrong hypothesis in the first place:
© 2009 Max Chipulu, University of Southampton- Socratic Logic, Hypotheses and Statistical Significance
Statistical Significance: Weird Good Vs Weird Bad
• Scenario 1
• Suppose you wake up one day and you have a headache. There is no apparent cause for this headache, e.g. it is NOT the morning after a party.
• Would you:
• A: Think ‘this is a bit weird- a random headache’-Basically ignore it, maybe take a couple of pain killers?; Or
• B: Think ‘oh my, this could be serious, I must go and consult a doctor immediately’
• Please show your hand for A or B…
© 2009 Max Chipulu, University of Southampton- Socratic Logic, Hypotheses and Statistical Significance
Statistical Significance: Weird Good Vs Weird Bad
• Scenario 2
• Now suppose you continue to suffer a headache for several days. Let us say you have it for ten days. Would you:
• A: Basically ignore it- continue taking pain killers, or
• B: Think, ‘This must be serious, I must go and consult a doctor’?
• Please show your hand for A or B
© 2009 Max Chipulu, University of Southampton- Socratic Logic, Hypotheses and Statistical Significance
What is the difference between Scenario 1 and 2?
• The difference is believability or rather lack of believability:
• In scenario 1, most people would believe that the headache is just ‘random’ and it will pass. Why? Well, because the probability that anyone could suffer a random headache on any given day is not small- it happens frequently.
• In scenario 2, most people would NOT believe that this a random headache because the probability of a random headache for 10 consecutive days is very, very small.
• This is statistical significance. Scenario 1 is NOT statistically significant; Scenario 2 is. How? Well let us see how this works in terms of hypothesis.
© 2009 Max Chipulu, University of Southampton- Socratic Logic, Hypotheses and Statistical Significance
Weird Good=‘Random’ Weird Bad=‘Not Random’, i.e. Statistically Significant
• Suppose that the null hypothesis, H0 = ‘the headache is random’
• Then the alternative hypothesis is Ha = ‘the headache is NOT random’
• So when the result, i.e. headache for one day or headache for 10 days is observed, the question is can we believe this headache is random? If the probability is small and we cannot believe that the headache is random, then we must reject the null hypothesis, i.e. we conclude that the observed headache is statistically significant.
• Usually, we will reject the null hypothesis if the probability of observing the result (or something worse) under the null hypothesis is 0.05 (5%) or less.
© 2009 Max Chipulu, University of Southampton- Socratic Logic, Hypotheses and Statistical Significance
Tests of Significance
In order to reject or accept a null hypothesis, we
must work out whether the probability of the
observed result is small under the null hypothesis.
So, always, we need a test probability distribution
that the observed result would follow under the null
hypothesis. Such a probability distribution is called a
statistical test of significance.
ALL hypothesis tests are based on a test of Significance:
© 2009 Max Chipulu, University of Southampton- Socratic Logic, Hypotheses and Statistical Significance
Binomial Test �
Chi-square �
KS (similar to Chi-square but uses proportions) �
Wilcoxon (T-test equivalent for �
categorical data) �
Z-test (result assumed to follow the normal distribution) �
T-test: results assumed to follow t-distribution, e.g. least squares regression coefficients. �
F-test of variance, e.g. variance explained by a regression �
Most Common Examples
Not as exact as parametric tests but research shows are almost as
powerful
More exact than non-parametric tests, therefore whenever
appropriate use parametric testsAdvantages
Categorical data or quantitative data for small samples when most parametric
assumptions are violated
Quantitative data taken from large samples
Usually applicable
for
Since no assumptions are made, applications can be flexible
Strictly only appropriate if assumptions have not been
violatedApplication
We make NO assumptions about the type of probability distribution function that the observed result
might follow
We assume that the observed result follows a specific probability distribution function, e.g. the
normal distribution
Characteristic
Non-ParametricParametric
Type of Test of Significance
© 2009 Max Chipulu, University of Southampton- Socratic Logic, Hypotheses and Statistical Significance
Further Reading
• Rae R. Newton and Kjell Erik Rudestam,
1999. Your Statistical Consultant. Sage
Publications Inc.
• Ramon E. Henkel, 1976. Tests of Significance.
Sage Publications Inc. (In Library box HA 33
QAA).
Top Related