How To Read A Medical Paper: Part 3, Assessing the Statistical Analysis
-
Upload
drlukekane -
Category
Health & Medicine
-
view
183 -
download
0
description
Transcript of How To Read A Medical Paper: Part 3, Assessing the Statistical Analysis
How To Read A Paper 1
How To Read A Paper IIIStatistics – again…
Dr Luke KaneMay 2014
How To Read A Paper 2
References
• Presentation based on T Greenhalgh’s series “How to read a paper” from the BMJ
• Greenhalgh, T. (1997) “How to read a paper” British Medical Journal. Web, accessed April-May 2014 at <http://www.bmj.com/about-bmj/resources-readers/publications/how-read-paper>
How To Read A Paper 3
What to Look For When Appraising the Statistics
• Checklist of questions to help you appraise the statistical validity of a paper
• Have the authors set the scene?– Have they determined whether their groups are
comparable? – And if necessary, adjusted for baseline differences?
• What sort of data have they got and have they used appropriate statistical tests?
• Have they used unusual tests? Why?• Is the data analysed according to the original protocol?
How To Read A Paper 4
Have the Authors: “Set the scene”?
• I.e. have they laid out clearly the characteristics of the groups?– For example, ratio of M:F, socioeconomic status,
age etc– If there are differences then this can pose a
challenge when interpreting the results• Adjustments can be made to allow for these differences
How To Read A Paper 5
Remember This?
How To Read A Paper 6
Have They Used the Appropriate Statistical Tests?
• Categorical and continous variables• Parametric and non-parametric tests– Parametric assumes the data has a particular form
of distribution– Non-parametric make no assumption about the
form – Parametric tests are more powerful than non-
parametric tests– Always use a parametric test if you can!
How To Read A Paper 7
Little Summary of Stats So Far…
The field of statistics exists because it is usually impossible to collect data from all individuals of interest (population). Our only solution is to collect data from a subset (sample) of the individuals of interest, but our real desire is to know the “truth” about the population. Quantities such as means, standard deviations and proportions are all important values and are called “parameters” when we are talking about a population. Since we usually cannot get data from the whole population, we cannot know the values of the parameters for that population. We can, however, calculate estimates of these quantities for our sample. When they are calculated from sample data, these quantities are called “statistics.” A statistic estimates a parameter.
How To Read A Paper 8
Parametric & Non-Parametric
• Parametric statistical procedures rely on assumptions about the shape of the distribution (i.e. assume a normal distribution) and about the parameters (i.e. means and standard deviations) of the assumed distribution.
• Nonparametric statistical procedures rely on no or few assumptions about the shape or parameters of the population distribution from which the sample was drawn. – Hoskin T (2012)
How To Read A Paper 9
Parametric & Nonparametric
• This helps you to decide which test to use…• Remember all of the parametric procedures
rely on an assumption of approximate normality
How To Read A Paper 10
Analysis Type Example Parametric Procedure
Nonparametric Procedure
Compare means between two distinct/independent groups
Is mean BP for patients assigned to placebo different from the mean for patients in the treatment group?
Two-sample t-test Wilcoxon rank- sum test
Compare two quantitative measurements taken from the same individual
Was there a significant change in BP between start and the 6m follow-up measurement in treatment group?
Paired t-test Wilcoxon signed- rank test
Compare means between three or more distinct/independent groups
If our experiment had 3 groups (placebo, new drug #1, new drug #2), see if BP differs among the three groups?
Analysis of Variance(ANOVA)
Kruskal-Wallis Test
Estimate the degree of association between two quantitative variables
Is systolic blood pressure associated with patient’s age?
Pearson coefficient of correlation
Spearman’s rank correlation
How To Read A Paper 11
Example
• Length of stay in hospital
• Compare M/F• Describe the
distribution• F: Right skew
– Mean 60– Median 30
• M: Normal– Mean 30– Median 30
How To Read A Paper 12
Example Continued
• Assumption of normality not reasonable• Therefore cannot use a parametric test• BUT can usea Wilcoxon Rank Sum Test
Analysis Type Example Parametric Procedure
Nonparametric Procedure
Compare means between two distinct/independent groups
Is mean BP for patients assigned to placebo different from the mean for patients in the treatment group?
Two-sample t-test
Wilcoxon rank- sum test
How To Read A Paper 13
Example Continued
• If we do a Wilcoxon Rank Sum Test– P-value is 0.63– Not less than 0.05 (which is the value that we typically compare
against)– Therefore not statistically significant– There is no difference in length of stay between genders
• If you assume the data is normally distributed and do a Two Sample T test – P-value is 0.04– Therefore statistically significant– There is a statistically significant difference between length of stay
for male and females
How To Read A Paper 14
So why can’t we just use nonparametric tests?
• Nonparametric tests are good as they make fewer assumptions about the distribution of measurements in population
• BUT– They are less statistically powerful• They need a larger sample size
– Their results are often less easy to interpet
How To Read A Paper 15
Small Sample Sizes
• More likely to have variable data• I.e. not normally distributed• So often need to use a non-parametric test
How To Read A Paper 16
Transformation
• Gives the graph a normal shape– Mathematical transformation– Such as using log, square root (√x), reciprocal (1/x)
• Some data cannot be transformed
How To Read A Paper 17
Obscure Tests
• Obscure means unclear, uncertain, hazy, vague• There are 1000s of statistical tests– There are about 12 important ones– Most of the rest should only be used in certain, specific
situations• If you are reading a paper with a standard set of data
which was collected in a standard way but the test they use is very obscure then you need to know WHY!?
• They should explain why they chose it and describe it with references
How To Read A Paper 18
Is the Data Analysed According to the Original Protocol?
• Often trials are stopped before they are finished if the new drug is doing well– This is to make sure that negative results don’t
come up later– This is very bad science– You should try to work out if they have done this
but it is very hard!– You can look at a registry to compare what they
said they would do and what they did do
How To Read A Paper 19
Paired Tests
• If you measure something twice on each subject• Each measurement if paired with another
measurement– E.g. blood pressure lying and standing– Height of a tree before and after fertiliser– IQ of younger and older brothers
• You would assume that the paired values are closer together than random values– Unless you allow for this then the results will be
significantly biased
How To Read A Paper 20
Summary
• Setting the scene– Group characteristics
• Think about the data– Continuous vs categorical– Distribution
• Parametric or non-parametric tests• Appropriate tests• Obscure tests• Transformation• Paired Tests• Protocol
How To Read A Paper 21
References• Bowers, D. (2008) Medical Statistics from Scratch: An Introduction for
Health Professionals. USA: Wiley-Interscience.• Grant, A. (2014) “Epidemiology for tropical doctors”. Lecture (S6) from
the Diploma of Tropical Medicine & Hygiene, London School of Hygiene & Tropical Medicine.
• Greenhalgh, T. (1997) “How to read a paper” British Medical Journal. Web, accessed April-May 2014 at <http://www.bmj.com/about-bmj/resources-readers/publications/how-read-paper>
• Hoskin, T (2012) Parametric and non-parametric: Demystifying the Terms. Retrieved from http://www.mayo.edu/mayo-edu-docs/center-for-translational-science-activities-documents/berd-5-6.pdf
• http://www.gla.ac.uk/sums/users/jdbmcdonald/PrePost_TTest/instruct1.html