Practical statistics for Neuroscience miniprojects Steven Kiddle Slides & data : .
-
Upload
elaine-morris -
Category
Documents
-
view
222 -
download
0
Transcript of Practical statistics for Neuroscience miniprojects Steven Kiddle Slides & data : .
Practical statistics for Neuroscience miniprojects
Steven Kiddle
Slides & data : http://bit.ly/1Jaor2r
We are unlikely to finish all the slides
Keep them, they may be helpful for your miniproject
Lecture outline
• Taught component– How to present statistics– Hypothesis testing– Normal distribution
• Practical component– Plots– Statistical tests– Multiple testing corrections
Taught component
Why are statistics important?
• Help to make science more repeatable and objective
• Help you to interpret your results• Help you to assess the level of evidence you
have supporting a hypothesis
• A vital skill for a scientific career!
How to report statistics
• Always report:(1) Statistical software you used(2) Statistical tests you used(3) Significance level you used(4) Sample size
I checked these in 17 randomly chosen neuroscience project posters
How not to report statistics!
• I found that:(1) 16/17 didn’t report the statistical software used(2) 11/17 didn’t report the statistical tests used(3) 9/17 didn’t report the significance level used(4) 2/17 didn’t report the sample size!
Commonly used analysis methods
• Plotting:- Box plots- Line plots
• Hypothesis testing– T-test– ANOVA– Chi-squared
Hypothesis testing
• Two types of hypothesis– Null hypothesis (H0)• Usually that there are no differences between groups
or that two variables are unrelated– Example : (H0) Smoking and lung cancer are unrelated
– Alternative hypothesis (H1)• There are differences between groups, or that two
variables are related– Example : (H1) Smoking and lung cancer are associated
Significance levels
• You accept the alternative hypothesis if the chance of your data being generated under the null hypothesis (the ‘p-value’) is beneath a pre-specified significance level α– Typically α = 0.05
• You should state the significance threshold you use in your report
Multiple hypothesis testing I• Suppose you have a significance threshold of α = 0.05
• Suppose that you measure 100 variables that are NOT related to a disease
• You perform 100 hypothesis tests to compare your variables to disease state• H0 : Variable is not affected by disease state
• H1 : Variable is affected by disease state
• For how many variables do you expect to reject the null hypothesis (H0) even though its true?
Multiple hypothesis testing II
• α = 0.05 means that if the null hypothesis (H0) is true, we would expect to reject it 5% of the time
• So if H0 is true and we did 100 tests, we would expect to reject H0 5 times by chance alone• That is bad, these findings will not replicate
• How do we stop it?• Multiple testing corrections
Multiple testing corrections
• Bonferroni correction– If we want α = 0.05 , instead use α = 0.05/n
where n is the number of tests you want to use– So for 100 tests, we would use α = 0.0005,
and would only have 5% chance of any test rejecting the null hypothesis
• Benjamini-Hochberg correction– Popular alternative
Normal distribution
Tests that rely on assumptions of normality
• T-tests• ANOVA / linear models
How to check if you data is normally distributed
• Histograms• Statistical tests
• Can apply to data
• But better to apply to residuals of the models– For t-test, that means looking at the groups separately– For ANOVA, that means extracting residuals from the
model
What do you do if your data is not normally distributed?
• If sample size is really small– Nothing you can do – use test anyway
• If data is skewed– Transform data (e.g. log? square root?)
• Use non-parametric tests– Mann-whitney U instead of T-test– Spearman’s Rank Correlation
• Last resort - Remove outliers?– Systematically and preferably only if you know what causes them
How to present plots
• Label both axes– Large enough to read
• Show units• If using stars (*) for significance levels, explain
what *, **, *** means
Lunnon et al., (2012) Journal of Alzheimer’s Disease
How to present statistics I
• Say what statistical software used, e.g.– SPSS, STATA, R, MATLAB, etc
• Say what the sample size is
• Say what statistical test is being performed– T-test, ANOVA, chi-squared, etc
• Say what significance level you are using for the study– Think, is it appropriate given my sample size and number of
hypotheses being tested?
How to present statistics II
• Report p-value– And/or multiple testing corrected p-value• E.g. Q-values for Benjamini-Hochberg
• Report coefficient (β), and ideally it’s standard error for each reported statistic– This can be more informative than a p-value,
especially for small datasets
How to present statistics III
A more complete guide, tailored to SPSS and specific tests is given at:
http://statistics-help-for-students.com/
Be cautious in your interpretations• Correlation does not equal causation!
• Can you hypothesise a mechanism by which causation could occur?
Why does correlation not equal causation?
• It looks like the variables are correlated when they are not– How this happens?• By chance, especially when multiple testing is
performed but not corrected for
• Variables are truly correlated but there is either:– Reverse causation– Confounding by other variables
Confounding
Statistical software
• Excel– Point and click, quite limited
• SPSS– Point and click, a little limited
• STATA– Command line
• R, MATLAB, etc– Command line, very useful, steep learning curve
http://www.r-project.org/http://www.rstudio.com/
R introduction
Practical component - SPSS
Data is faked to show large differences, real data will not be so
clear cut
Outline
• Data• Tests for normality• Plots• T-test• ANOVA• Chi-squared• Non-parametric tests
Data
• Create folder in ‘My Documents’• Download data and save in your new folder:
Slides & data : http://bit.ly/1Jaor2r
• Open zip folder• Double click on ‘neuroscience_example.sav’ to
open SPSS
Introduction to the data
• 5 variables– a , b , c , d , e
• 2 are binary– a , b
• 3 are continuous– c , d , e
Normality checks I
• Need to check data is normally distributed when we want to apply– T-test– ANOVA– Linear regression
Normality checks II
Let’s see if the variable ‘d’ is normally distributed
Normality checks III
Can see that the data has two peaks
Rejects the null hypothesis that the data is normally distributed
Normality checks IIINow we take into account variable ‘a’, we find that ‘d’ is normally distributed when we take into account ‘a’
Plots
• Histograms (shown in normality check)– Show distribution of a continuous variable
• Boxplots– Show the distribution of a continuous variable between
groups
• Line plot/scatter plot– Shows the relationship between two continuous
variables
Generating a boxplot I
Generating a boxplot II
Generating a boxplot III
Generating a boxplot IV
Double click on plot to label axis
Labelling a plot I
Double click to change label
How to present plots
• Label both axes– Large enough to read
• Show units• If using stars (*) for significance levels, explain
what *, **, *** means
Lunnon et al., (2012) Journal of Alzheimer’s Disease
Labelling a plot II
Saving plots
Rename document and save in your folderYou can now open the document and extract the plot as an image
Boxplot exercises
• Make a few more boxplots comparing binary variables to continuous variables
• Try adding labels• Try saving• Try to interpret the boxplot– Do you see differences between the groups
Generating a line plot I
Generating a line plot II
T-test
• Compares a binary variable (yes/no) to a continuous variable
• Example null hypothesis– Mean height is the same across males and females
• Example alternative hypothesis– Mean height is different between males and
females
Performing a t-test I
Performing a t-test II
Descriptive statistics
Performing a t-test II
Levene test null hypothesis : the variance (and standard deviation) of the two groups are the same.
At the significance level 0.05 we can reject the null hypothesis.
Therefore we should use the second row (‘Equal variances not assumed’).
Performing a t-test II
T-test null hypothesis : the mean of the two groups are the same
At the significance level 0.05 we can reject the null hypothesis (p-value is less than 0.001).
I.e. the data supports the fact that the variable ‘d’ is different between the two groups.
How to report findings (option A)
• A two-sample t-test assuming unequal variances performed in SPSS showed differences in variable ‘d’ between groups 0 (N = 19) and 1 (N = 31) in variable ‘a’ at the 5% significance level (mean difference = 1.6, standard error = 0.89, p-value < 0.001 ).
How to report findings (option B)
• Materials and methods:– Statistical analysis
• Statistical analysis was performed in SPSS 20.• Group differences were analysed using two sample t-test
assuming unequal variances.• A significance level of 5% was applied to all hypothesis tests.
• Results:– Variable ‘d’ was found to differ between groups 0 (N = 19)
and 1 (N = 31) in variable ‘a’ (mean difference = 1.6, standard error = 0.89, p-value < 0.001 ).
How that might look
• Materials and methods:– Statistical analysis
• Statistical analysis was performed in SPSS 20.• Group differences were analysed using two sample t-test
assuming unequal variances.• A significance level of 5% was applied to all hypothesis tests.
• Results:– Change in blood glucose levels differed between males (N
= 19) and females (N = 31) (mean difference = 1.6 ng/ml , standard error = 0.89, p-value < 0.001 ).
How to present statistics
A more complete guide, tailored to SPSS and specific tests is given at:
http://statistics-help-for-students.com/
One-way ANOVA
• Extension of t-test idea• Compares a binary variable (yes/no) to a several
variables, continuous or nominal (including binary)• Example null hypothesis– Mean height is the same across males and females,
regardless of age• Example alternative hypotheses– Mean height is different between males and females– Mean height differs across ages
Performing ANOVA I
Performing ANOVA II
Which variables differ between variable ‘a’ group 0 and 1?
How would you perform a Bonferroni multiple testing correction?
How would you report these findings? (clue: http://statistics-help-for-students.com/ )
Chi-squared test
• Compares multiple nominal variables• Example null hypothesis– Lung cancer and smoking are unrelated
• Example alternative hypotheses– Smokers are more likely to have lung cancer
Performing chi-squared I
Performing chi-squared II
How would you report these findings? (clue: http://statistics-help-for-students.com/ )
Non-parametric tests
• If a continuous variable is not normally distributed, using parametric tests may give you misleading results– T-test and ANOVA are parametric tests
• Solution, use non-parametric tests– Such as• Mann-Whitney U• Spearman’s Rank Correlation
Mann-Whitney U
• A non-parametric equivalent of a t-test
Performing Mann-Whitney U I
Performing Mann-Whitney U II
How would you report these findings? (clue: http://statistics-help-for-students.com/ )
Spearman’s Rank Correlation
• A non-parametric equivalent of correlation, or a one-way ANOVA between a binary variable and a continuous variable
Performing Spearman’s Rank Correlation I
Performing Spearman’s Rank Correlation II
How would you report these findings? (clue: http://statistics-help-for-students.com/ )