Research methodology by almuzian
-
Upload
mohammed-almuzian -
Category
Health & Medicine
-
view
406 -
download
4
Transcript of Research methodology by almuzian
Research Methodology
How to write a research protocol (O’Brien 2002)
Definition
A protocol is a document that explicitly states the reasoning behind and structure of a research project.
Advantages1. It states the question you want to answer.
2. To plan the project in detail, before you start.
3. To see the total process of your project.
4. It acts as a guide for all personnel involved in the project.
5. It enables you to monitor the progress of the project.
6. It is necessary if you need to apply for funding or ethical approval
Components of a research protocol
A. The problem to be investigated
1. Project title; should be short and concise
2. The research problem; eg: In this study I intend to find whether the use of
a fixed functional appliance (the Herbst appliance) will result in greater
skeletal change than a removable functional appliance (the Twin Block).
If I can show that this occurs this will be an important finding for
orthodontic care.
3. Background (including the literature review); The most important feature
of the background to the project is that it should be brief and to the point.
For a research protocol the background should be no longer than two
pages of A4 paper. In this section, you should concisely review the
literature that is relevant to the problem that you are trying to solve. In
this respect, it is probably good practice to limit the number of papers
quoted to less than 20. When you write the review, you should draw
attention to the good points and the deficiencies of the studies quoted.
You should also remember that it does not always mean that if a study
has been published in a journal, it is flawless in its methodology and
conclusion. The literature review should logically lead to the statement of
the aims of the proposed project.
4. The aims; the aims of the project should be explicitly stated. These should
be confined to the intention of the project and they should arise from the
literature review.
5. The hypothesis. It is general practice that hypotheses are stated in the null
form, because they have their basis in inferential statistics. You challenge
the hypothesis of no difference. The result of statistical testing gives the
probability that the hypothesis of no difference is true.
B. Method of investigation
In a study protocol, the method should be stated in the future tense.
1. Subjects; it includes:
The population the subjects will be drawn from.
The total number and the number in any subgroups within the
investigation.
The inclusion and exclusion criteria for the subjects.
2. Design; which involve mentioning the method of randomization,
patient registration and allocation.
3. Experimental procedure; describe the method of treatment and how to
collect the data and when
4. Materials, measurements, and apparatus used; describe the parameter
that will be measured (like OJ or MR)
5. Sample size calculation;
6. The statistical methods that you are going to use.
7. Project milestones (Deadline) This section is not essential.
Nevertheless, it does provide a guide (and reminder!) for you and your
supervisor to inform if you are ahead or behind schedule with your
project.
8. Method of dissemination of findings (method of publication): Again,
this is not always essential, but it does let the reader know what you
intend to do with the results of the study. It is occasionally possible to
list the potential titles and publication strategy of the investigators.
However, this can sometimes be considered an over optimistic
approach.
9. Resources required: Finally you should make a list of all the resources
that you are likely to require to successfully complete your
investigation. If these resources have cost implications, you should
also note the potential cost of the investigation.
Advantages of RCT7. The RCT is prospective. As a result, the subjects and the data are under
some control by the investigator.
8. The treatment or intervention is randomly allocated. Therefore, the
perceptions of the investigator on the value of a particular treatment
should not influence treatment allocation that could bias the results.
9. The study is planned before the data is collected. This is the important
distinction between the RCT and the retrospective investigation, and
these results in a minimization of bias that is inherent in the retrospective
study.
RCT Study design (O’Brien 2003)1. Select aims and State objectives, objectives should be capable of
statistical description and analysis
2. Null hypothesis (always assumes there no difference)
3. Outcomes: It is important to know before you start how you are going to
measure the effect of each intervention.
4. The study population and site of the study: This is an important step
because it is important that this population is relevant to both the question
that we hope to answer and to the provision of orthodontic treatment.
5. List of inclusion and exclusion criteria for the study.
6. Power calculations
7. Define the measuring techniques and instruments
8. Define the control group
9. Ethical issues, e.g. - welfare of subjects, should time and money be spent
researching this subject, patient consent
10.Patient registration:
Patient requires treatment and is eligible
Agreement to randomize
Patient consent
11.Formal entry: Details of the patient are then entered onto a log sheet of
the trial or, more commonly, onto a computer database.
12.Randomization: the object of randomization is to allocate one or more
interventions (or control), in a manner that ensures that the samples that
you are going to compare, are similar in every respect apart from the
intervention. In most trials, a randomization list has been prepared in
advance using random numbers.
13.Allocation: The next stage is the method by which the operator finds out
which treatment the patient has been assigned to. It is essential that the
operator does not know what the assignment will be in advance and there
are several methods of concealing this. One popular method is to transfer
the randomization list to a series of sealed envelopes each containing the
allocation on a card. The clinician then opens the next envelope in the
series when the patient formally enters the trial. This method is
particularly relevant when the clinician registers his/her own patients.
However, care needs to be taken to ensure that the clinician does not
reseal the envelope having discovered that the allocation was not what
he/she was hoping! The best method of allocation is to make use of a
central registration office. In this method the treatment assignment is read
from a prepared list and given to the investigator while still on the phone,
following the registration of the patient. While this method is more
expensive and requires more preparation than using envelopes it does
provide an almost foolproof method of allocation.
14.Blinding may occur in many ways, for example, blinding the patient, the
operator, the investigator who measures the outcomes and the
statisticians. However, when we consider the nature of orthodontic
treatment it is impossible to blind treatment allocation to both the
operator and patient. As a result, the only type of blinding that we can
practice is blinding of the person who records and analyses the data. This
is important because if, for example, the evaluator knows that a group of
patients have had a new treatment then they may record outcome data in a
favourable manner. Blinding can be done by concealing the identity of
the patient and the treatment allocation using numbers, or by having the
data recorded by a different person from the one who is going to analyse
the data.
15.Monitoring progress: So now you have set up your trial, and you think
that you can just sit back and the trial will run, and all you have to do is to
collect and analyse the data. Unfortunately, this is not the case! Several
areas should be evaluated as part of this process.
The first of these is protocol compliance. You need to check that the
study protocols are being followed by the operator(s) in the study. The
easiest way of doing this for an orthodontic study is to periodically look
at the records of the patients in the study and check for protocol
deviations that are recorded.
You should also check for adverse effects.
Finally, a careful record of all study withdrawals or drop-outs should be
made, and as much baseline data as possible recorded. This will ensure
that a statistical check can be made to discover whether the drop-outs
were similar to those people who remained in the trial
16.Interim data analysis is useful if:
you need some data to present at a conference,
check that the treatments are not causing harm, which is important for the
ethics of the trial.
17.Stopping rules are defined at the start of the trial to ensure that there is a
‘safety valve’. If, for example, it becomes obvious during a trial that one
or more treatments is significantly worse or better than another, then the
trial should be stopped.
18.Data analysis: Methods of data analysis for RCTs do not markedly differ
from other orthodontic studies. However, it is important to consider the
difficult question of how to handle data from patients who dropped out of
the investigation. When this occurs we are left with several choices. The
first is to report the number of patients who withdrew from the
investigation and emphasize that the two interventions under
investigation had certain success and failure rates. Or the data analysis
should include the results of the treatments on all the patients who entered
the study, regardless of successful compliance or completion of the
treatment. This is termed an intention to treat analysis (ITT analysis).
This type of approach results in a measure of the true effectiveness of the
treatment and should be attempted wherever possible. One possible
drawback of this approach with orthodontic treatment is that we may not
have collected data on the patients who dropped out of the investigation,
as they may not have returned to the clinic. One solution to this is to
statistically impute data to compensate for the lost data. Several statistical
packages have the ability to be programmed to carry out this type of
analysis.
19.Writing up the project by following the Consolidated Standards of
Reporting Trials (CONSORT) guidelines developed to assess reporting of
randomized clinical trials in journals, (Moher 2010 and Altman 2001)
20.In all RCT, a data and monitoring board for the safety (DSMB) of the
trial is important and crucial.
21.Publication
Systematic review study design and protocol planning1. Create a rationale and aims of the study
2. Outcomes: It is important to know before you start how you are going to
measure the effect of each intervention.
3. Set inclusion and exclusion criteria
4. Conduct a search
Electronically via MEDLINE ‘’American version’’, EMBASE
‘’European version’’, Cochrane library ‘’CENTRAL’’
manually
different languages
Gray literatures.
The search can depend on the use of
Single vocabularies (specific word) and free text vocabularies (in the
whole text) to increase search sensitivity
It can be combined with Boolean operators ‘’AND, NOT, OR’’, or
truncation and wildcard by using this symbol ‘‘?’’(child? mean different
extension of the word, eg, childhood, children, child’s), proximity
operators (eg, dental near/6 anxiety, mean any article contains dental and
anxiety within 6 word closeness)
It can be filtered according to the relevant design of study (RCT or case
control)
5. Review the included studies
6. Reject non-relevant studies
7. Assess the methodology of relevant studies
8. Exclude if weak methodology
9. Extract data from the remaining studies
10.Analyse data
11.If the methodology was similar then combine them to increase the sample
size and increase the power of study
12. Summarize and draw conclusion
Classification of studiesIn general they classified into observational and experimental studies.
Research method
Key features
Case Report A small case series describing the outcome of
treatment of a few (less than 5-10) cases or
reporting potential problems with treatment.
Cross-
sectional
Data collected from sample members on one
occasion.
1. Used to study prevalence (called
descriptive cross sectional) and
relationship between variables (analytical
cross sectional). It can be used to create a
new question which need to be answer by
subsequent powerful study. It can be used
to formulate the normative for patient
(like Bolton norms).
2. Advantages: low cost, short time, can
screen large sample,
3. Disadvantages: not strong, weak external
validity,
Longitudinal Data collected from sample members on two or
more occasions.
Cases control Describes what happens to patients without
actively intervening with the treatment they
receive and they are retrospective with separate
control group.
Cohort study Describes what happens to patients without
actively intervening with the treatment they
receive and they are prospective with separate
control group.
The disadvantages are the cost and long
duration with many confounding factors that
could appear.
Clinical trial Assess whether one health care intervention is
better than another, a placebo or no treatment.
Are prospective and controlled. Allocation to
test/control groups is predetermined.
Control gp can be ethically used if the new
intervention is equipoise which means a
presumed equality between new and old
therapy.
Weakness of RCT is the cost and time as well as
weak external validity (which means the result
not represent the large population sample
because most of the sample who recruited are
either sicker or more compliant than an
individual in community).
RCT represent the intervention efficacy under
ideal conditions rather than effectiveness under
real condition.
RCT not always give the final conclusion about
certain treatment and intervention, so the
outcomes could be followed for a period of time
after applying the new intervention in a system
called ‘’post market surveillance programme’’
No control
trials Quasi-experiment or randomization
Literature
control
Comparison made to information on patients in
a published paper or growth study. Prone to
chronological and/or geographical bias.
Historical
control
Comparison made with patients treated
previously in the same unit/place. Prone to
chronological bias.
Matched
control
Comparison made with patients who are similar
in respect to one or two specific criteria. Prone
to allocation bias.
Clinical trial
Review article Traditional Summarizes information from several
review previously published papers on a specific topic.
Systematic
review
Papers are identified, critically appraised and
the results synthesized according to a defined
protocol.
Meta-
analysis
Combines the results from several different
clinical trials to obtain an overall estimate of the
effectiveness of a particular intervention.
The Cochrane
Collaboration
• launched in 1995
• Summarize the evidences and
provides a database of controlled trials
and systemic reviews
• provides guidelines on preparation of
good clinic trials and systemic
reviews
Which study design for which aim?
For a question about.....
....these are the most appropriate research
methods
Example
Therapy Clinical trial Competing interventions,
a placebo, no treatment.
Diagnosis or screening Cross-sectional survey Prevalence of orthodontic
treatment need.
Prognosis Cohort study or
longitudinal survey
Long-term stability of
mandibular incisors.
Causation Case control study or
cohort study
Factors influencing root
resorption following fixed
appliance treatment.
Summary of evidence Systematic review Orthodontic treatment for
posterior crossbites.
Validity: the extent to which the value obtained represents the object of
interest
Reproducibility: the closeness of successive measurements, errors in
measurement can be both systematic and random
Reliability: often used in a broader sense to encompass both
reproducibility and validity
Systematic error: a tendency to overestimate or underestimate a
parameter giving a biased
Random error: this is the variability due to chance. Increasing the number
of observations e.g. repeating measurements and taking an average value
is an important way of reducing random error
There are four main biases that can affect the internal validity of a study.
1. Selection bias
The term selection bias is used in different ways within the medical
literature.
It is often used in relation to bias occurring during the selection of
representative participants or to bias occurring during the selection of
participants to exposures.
2. Performance bias
Performance bias refers to systematic differences in care provided to
participants in a study.
Blinding of study participants and investigators to treatment allocation
helps minimize performance bias.
When a study is described as single blind only the participants are blind
to their group allocation.
When both participants and investigators are blind to group allocation the
study is described as double blind.
3. Measurement bias
Even when blinding to treatment groups cannot be achieved, blinding to
outcome assessment is usually possible.
In orthodontics, this can be achieved by blind assessment of study
models, radiographs and/or photographs.
This can help minimize systematic differences that may occur in how
outcomes are determined from the groups under comparison
(measurement or detection bias).
Error estimation or measurement • Estimation of reproducibility error is an important part of
experimental study
• Error estimation normally involves the replication of a certain number
of observations
• Replicated measurements should be chosen at random and after a
suitable time interval following the initial measurements
• Important to distinguish between systematic and random error
• The difference between the repeated measurements gives a measure of
the systematic error
• The SD of the difference gives a measure of the random error
• Quoting these two values allows differentiation between random and
systematic error
• The error variance should not be greater than 10% of the total variance
4. Attrition bias
It occurs when there are systematic differences between comparison
groups in withdrawals or exclusions of participants from the results of a
study.
For example, patients may drop out of a study because of side effects of
the intervention or difficulty in wearing a particular appliance.
Excluding these patients from the analysis could result in an over-
estimate of the effectiveness of the intervention.
Conversely, participants might drop out of a study due to an
improvement in the symptoms or malocclusion, e.g. overjet or
crowding, resulting in an under-estimate of treatment effect if they are
not included in the analysis.
In order to minimize attrition bias, all study participants should be
accounted for in the analysis and the analysis undertaken on an
intention-to-treat basis i.e. participants are analysed according to the
group to which they were initially allocated, regardless of whether they
dropped-out, fully complied with the treatment or ended up crossing
over to the other treatment group
Types of errors (Macfarlane, 2003)
There are two types of errors that should be taken into account when
designing a study.
A type I error is the error of wrongly rejecting the null hypothesis when
it is true. The level of significance is defined as the probability of
making a type I error and is denoted by α. In order to guard against type
I errors it is usually set to small values such as 0.05.
A type II error is the error of wrongly accepting the null hypothesis
when it is false. The probability of making a type II error is denoted by
β. The power of a hypothesis test is equal to 1 – β, and is often
expressed as a percentage, rather than a proportion. In medical research,
it is frequently set at least to 80%.
Sample size (Macfarlane, 2003)
It depends on:
1. Type I error (alpha error or statistical significance)
2. Type II error
3. Clinical significant (how does one decide what difference is worth
detecting? Sometimes this information comes from clinical experience
or similar studies in other areas, but often a small pilot study is
required).
4. SD
5. The pre-known mean difference of the object to be measured