Reporting Statistical Methodology Apr 1-05

8/2/2019 Reporting Statistical Methodology Apr 1-05

1/41

REPORTING STATISTICALMETHODOLOGY AND

ANALYSIS IN MEDICAL RESEARCH

INAS ELATTAR

Professor of Biostatistics

National Cancer Institute

Cairo University


2/41

INTRODUCTION

1. Provide the audience with a concise reference

on what should be reported.

2. Advice about important aspects of statistical

design, analysis, and presentation.


3/41

MATERIALS AND METHODS

The method section should include everything

relating to the study design, subjects and

statistical methods used


4/41

Type of study: Depends on the purpose of the

study and the research question to be

answered

Important for understanding the conclusion

that can be drawn from the study

Clearly describe the design of the study


5/41

Give numbers of observations

Sample size needed should be stated: useful to

include a justification for the number of subjects

studied

Basic observational units should be determined

Proper method of analysis depends on the basicobservational units. The analysis of correlated

observations raises difficult statistical analysis

issues


6/41

Discuss eligibility of experimental

subjects

Reasons for and methods of selecting patients

Some studies have eligibility criteria, eg: exclusion of

patients outside a specified age range, those previously

treated, and those too ill to answer questions, etc Two major goals for reporting detailed ways of selection:

1) can be repeated by other investigators, 2) provides

readers with a solid link between patients studied and

population

Eligibility constraints can introduce a sizable bias when

results are generalized to the population


7/41

Give details about randomization

Randomization enhances generalization of

results and avoids biases

Not sufficient to state that: subjects were

randomly selected, random often means

haphazard


8/41

Describe the methods for, and success

of, any blinding of observations

In studies involving comparisons between twotreatments, the most effective way to reduce bias on the

part of both patient and physician is blinding

Blinding is the hiding of certain information from patientsand physicians

Because blinding can be applied in many different ways,researchers should state exactly who was blinded towhat


9/41

Report treatment complications

Nearly every medical treatment carries some

risk of complications, side effects

If no adverse effects can be found, the report

should say so and explain what was done to find

them


10/41

Report losses to observation (dropouts

from a clinical trial)

Patients lost to follow-up, including losses or exclusions

for noncompliance

These patients might be different from the others with

respect to factors that might affect the results

Efforts to trace them should attempted

Reasons for losses should be reported in details

When sample size in the text, table or graph differs from

the original sample size, the difference should be

explained


11/41

Describe the statistical methods with

enough details to enable a

knowledgeable reader with access to

the original data to verify the reportedresults


12/41

Use of statistics in medicine can be

summarized as follows:

1) Statistics used to answer questions concerning

differences,

2) Statistics used to answer questions concerningassociations,

3) Statistics used to answer questions concerning

predictions


13/41

Researchers should always have a cleardefinition of their goal. The researchers have

to choose between different measure andtechniques:

1) Mean or median?

2) Normal or nonparametricdistribution

3) Adjustments, matching or stratifications


14/41

Choice of statistical method depends on the

question raised and type of data collected

Search for results that are significant, this

leads to conclusions that may not hold if study isrepeated, this called Fishing expedition


15/41

Authors should always report the statistical

methods used

Authors should report positives and negatives oftheir study design

Most statistical techniques depend on someassumptions, authors should indicate that theseassumptions have been checked


16/41

References for study design and

statistical methods should be standardworks when possible rather than

papers where designs or methods were

originally reported

In statistics, an original paper is usually verytechnical and it can be of great help to

methodologists


17/41

Specify any general use computer

programs used

Computer programs are sometimes found tohave errors.

Readers might wish to know about the program

used for their own use


18/41

RESULTS

The results section entails a detailed

presentation of the analyses carried out and the

outcome of the statistical analysis used to

answer the research question


19/41

Put general descriptions of statistical

methods in the methods section.When data are summarized in the

Results section,specify the statistical

methods used to analyze them

Whenever more than one procedure is used,

specify the statistical methods applied in the

Results Section


20/41

Clearly specify the units of data

Units should always be specified in text, tables,

and figures. Is time months or years? Is

quantities kilograms or grams ? Are rates per

10,000 or100,000?

Careful reporting of units prevents

misunderstanding of the results


21/41

When possible, quantify findings and

present them with appropriateindicators of error or uncertainty

Authors should use the most appropriate

measure that describes precisely their data,

such as means and standard deviations as wellas confidence intervals


22/41

Reporting test of significance (p-values) only,

should be avoided

Use standard deviation rather than standarderror of mean. There are two reasons for this

recommendations: 1)standard error is a function

of sample size, 2)standard error pertains to

groups, not individuals

Exact P values should be reported

Specify two-tail or one tail tests


23/41

Avoid sole reliance on statistical

hypothesis testing such as the use of P

values which fails to convey important

quantitative information

Report both point estimates (means,proportions,or differences between means) and

condifence intervals

Confidence intervals provide information on the

magnitude of the effect and how estimates

would vary in other samples


24/41

Restrict tables and figures to those needed to

explain the argument of the paper and to assessits support.

Use graphs as an alternative to tables with many

entries; do not duplicate data in graphs and

tables


25/41

Avoid nontechnical uses of technical terms in

statistics, such as random (which impliesrandomized device), normal, significant,correlation, and sample

Normal refers to some kind of probabilitydistribution, should not be confused withmeaning of normal patient, that is free of disease

Significant refers to the result of a formal

statistical test of significance Correlation is a technical word which refers toa specific method to measure association


26/41

Define statistical terms, abbreviations, andmost symbols


27/41

DISCUSSION

Clearly point out drawbacks of the study

Do not overemphasize the value of P values.

Test statistics depends on the sample size

Clearly identify tests of hypotheses generated

from the research question of the study and

those defined after initiation of the study

Hypotheses generated after beginning of the

study are exploratory. Should be used to answer

future research questions


28/41

CONCLUSIONS

Purpose of statistical methods is to provide astraightforward factual account of the scientific

evidence derived from research

To design suitable studies and carry outsensible statistical analyses is of utmost

importance for the assessment of results

Communication of findings in a clear and

objective manner is as crucial as design and

analyses


29/41

Reading the Literature

Checklist


30/41

Reading the Abstract

Is the topic of the study important and worth knowingabout?

What is the purpose of the study? Is the focus on adifference or a relationship? The purpose should beclearly stated; one should not have to guess.

What is the main outcome from the study: Does theoutcome describe something measured on a numericalscale of something counted on a categorical scale? Theoutcome should be clearly stated

Is the population of patients relevant to your practice

can you use these results in the care of your patients?The population in the study affects whether or not theresults can generalized.

If statistically significant, do the results have clinicalsignificance as well?


31/41

Reading the Introduction

If the article does not contain an abstract, the introduction

section should include all of the above information plus

the following information

What research has already been done on this topic and

what outcomes were reported? The study should add

new information


32/41

Reading the Methods

Is the appropriate study design used (clinical trial, cohort,case-control, cross-sectional, meta-analysis)?

Does the study cover an adequate period of time? Is the

follow-up period long enough?

Are the criteria for inclusion and exclusion of subjects

clear? How do these criteria limit the applicability of the

conclusions? The criteria also affect whether or not the

results can be generalized. Are standard measures used? Is a reference to any

unusual measurement/procedure given if needed? Are

the measures reliable/replicable?


33/41

Reading the Methods (cont)

What other outcomes (or dependent variables) and riskfactors (or independent variables) are in the study? Are

they clearly defined?

Are statistical methods outlined? Are they appropriate?(The first question is easy to check; the second may be

more difficult to answer.)

Is there a statement about power the number of

patients that are needed to find the desired outcome? A

statement about sample size is essential in a negative

study


34/41


Clinical trial

How are the subjects recruited? Are the subjects randomly assigned for the study

groups? If not:

o How are patients selected for the study to avoid selection bias?

o If historical controls are used, are methods and criteria the same

for the experimental group; are cases and controls compared onprognostic factors?

Is there a control group? If so, is it a good one?

Are appropriate therapies included?

Is the study blind? Double-blind? If not, should it be? How is compliance assured/evaluated?

If some cases are censored, is a survival method such

as Kaplan-Meier or the Cox model used?


35/41


Cohort study

How are the subjects recruited?

Are the subjects randomly selected from an eligible

pool?

How rigorously are subjects followed? How many

dropouts does the study have and who are they?

Are appropriate therapies included?

If some cases are censored, is a survival method such

as Kaplan-Meier or the Cox model used?


36/41


Case-control study


pool?

Is the control group a good one (bias-free)?

Are records reviewed independently by more than one

person (thereby increasing the reliability of data)?


37/41


Cross-sectional (survey, epidemiologic)

study

Are the questions unbiased?


pool?

What is the response rate?


38/41


Meta-analysis

How is the literature search conducted?

Are the criteria for inclusion and exclusionof studies clearly stated?

Is an effort made to reduce publicationbias (because negative studies are lesslikely to be published)?

Is there information on how many studiesare needed to change the conclusion?


39/41

Reading the Results

Do the reported findings answer the research

questions?

Are actual values reported means, standard

deviations, proportions so that the magnitudeof differences can be judged by the reader?

Are many P values reported, thus increasing the

chance that some findings are bogus?

Are groups similar on baseline measures? If not,

how did investigators deal with these differences

(confounding factors)?


40/41

Reading the Results (cont)

Are the graphs and tables, and their legends easy to

read and understand?

If the topic is a diagnostic procedure, is information on

both sensitivity and specificity (false positive rate) given?

If predictive values are given, is the dependence on

prevalence emphasized?


41/41

Reading the Conclusion and Discussion

Are the research questions adequately discussed?

Are the conclusions justified? Do the authors extrapolate

more than they should, for example, beyond the length

of time subjects were studied or to populations not

included in the study?

Are the conclusions of the study discussed in the context

of other relevant research?

Are limitations of the research addressed?

Reporting Statistical Methodology Apr 1-05

Documents

Transcript of Reporting Statistical Methodology Apr 1-05