Experimental Study Designs Dr. lamya Alnaim. Introduction The primary method for testing the...

103
Experimental Study Designs Dr. lamya Alnaim

Transcript of Experimental Study Designs Dr. lamya Alnaim. Introduction The primary method for testing the...

Page 1: Experimental Study Designs Dr. lamya Alnaim. Introduction  The primary method for testing the effectiveness of new therapies and other interventions,

Experimental Study Designs

Dr. lamya Alnaim

Page 2: Experimental Study Designs Dr. lamya Alnaim. Introduction  The primary method for testing the effectiveness of new therapies and other interventions,

Introduction The primary method for testing the

effectiveness of new therapies and other interventions, including innovative drugs.

By the 1930s, the pharmaceutical industry had adopted experimental methods.

In the 1960s, the controlled clinical trial became the standard for doing pharmaceutical research and measuring the therapeutic benefits of new drugs.

Page 3: Experimental Study Designs Dr. lamya Alnaim. Introduction  The primary method for testing the effectiveness of new therapies and other interventions,

Introduction The drug regulations of the 1960s also

reinforced the importance of controlled clinical trials by requiring that proof of effectiveness for new drugs be made through use of these research methods.

In pharmacoepidemiology, the primary use of experimental design is in performing clinical trials, most notably randomized, controlled clinical trials.

Page 4: Experimental Study Designs Dr. lamya Alnaim. Introduction  The primary method for testing the effectiveness of new therapies and other interventions,

Experimental Design An experiment is a study designed to compare

benefits of an intervention with standard treatments, or no treatment, such as a new drug therapy or prevention program, or to show cause and effect. Performed prospectively. Subjects are selected from a study population,

assigned to the various study groups, and monitored over time to determine the outcomes that occur and are produced by the new drug therapy, treatment, or intervention.

Page 5: Experimental Study Designs Dr. lamya Alnaim. Introduction  The primary method for testing the effectiveness of new therapies and other interventions,

Experimental DesignAdvantages.

Randomization tends to balance confounding variables across the various study groups, especially variables that might be associated with changes in the disease state or the outcome of the intervention under study.

Detailed information and data are collected at the beginning to develop a baseline and at specified follow-up periods throughout the study.

The investigators have control over variables such as the dose or degree of intervention.

The blinding process reduces distortion in assessment. This design is the only real test of cause–effect

relationships.

Page 6: Experimental Study Designs Dr. lamya Alnaim. Introduction  The primary method for testing the effectiveness of new therapies and other interventions,

Experimental DesignDisadvantages Subject participation criteria that may limit

generalizability of findings. Restrictive criteria for inclusion or exclusion of

subjects may produce a very homogeneous study population that restricts application of the results to patients with other characteristics.

May require years of follow-up and prolonged observation to determine treatment outcomes. Higher costs, increased likelihood that patients will be lost to follow-

up delayed treatment recommendations.

Page 7: Experimental Study Designs Dr. lamya Alnaim. Introduction  The primary method for testing the effectiveness of new therapies and other interventions,

Experimental DesignDisadvantages Large sample sizes are typically required to

demonstrate differences among study groups. Increasing the study population raises the cost of

the trial may make it difficult to locate a sufficiently large

pool of eligible patients. Ethical concerns subjects may not comply with the treatment

and assignment.

Page 8: Experimental Study Designs Dr. lamya Alnaim. Introduction  The primary method for testing the effectiveness of new therapies and other interventions,

Classification(1) Therapeutic trials

Therapeutic agents or procedures are given to patients in an attempt to cure the disease, relieve symptoms, or prolong survival

(2) Intervention trials The investigator intervenes before the disease has

developed in individuals with certain characteristics that increase their risk of developing the disease

(3) Prevention trials To determine the efficacy of a preventive agent or

procedure among people who do not have the disease but may be at greater risk for developing it.

Page 9: Experimental Study Designs Dr. lamya Alnaim. Introduction  The primary method for testing the effectiveness of new therapies and other interventions,

The basic design of a randomized, controlled clinical trial

Page 10: Experimental Study Designs Dr. lamya Alnaim. Introduction  The primary method for testing the effectiveness of new therapies and other interventions,

Clinical Drug Trials The development of the clinical trial is a

product of the application of the modern scientific method to medicine.

Experiments on human populations have inherent difficulties not found in the laboratory. laboratory experiments involve controlled

environments and the manipulation of just a small number of variables,

Control of human subjects and their environments, with a variety of other factors, is much more difficult.

Page 11: Experimental Study Designs Dr. lamya Alnaim. Introduction  The primary method for testing the effectiveness of new therapies and other interventions,

Clinical Drug Trials The basic idea is to "try out" (trial) a new drug

in clinical practice on sick patients. The goal is to determine the therapeutic

benefits of the new drug in humans after it has passed safety testing in animals as well as safety, dosage range, and effectiveness studies. (phases I and II).

Page 12: Experimental Study Designs Dr. lamya Alnaim. Introduction  The primary method for testing the effectiveness of new therapies and other interventions,

Clinical Drug TrialsThe design of drug trials varies by

certain components the specific treatments being evaluated

and compared how patients are selected for the different

study groups who knows which patients are receiving

the experimental treatment which ones are receiving other treatments

or no treatment

Page 13: Experimental Study Designs Dr. lamya Alnaim. Introduction  The primary method for testing the effectiveness of new therapies and other interventions,

Table 4-1. Components of a Clinical Drug Trial

Experimental study: A research method in which one group of subjects receives a new treatment (e.g., drug, device, surgery), and they are compared with one or more other group(s) who receive different standard treatments or placebo. These groups are studied over the same time period using the same measures of safety and effectiveness. 

Experimental group: The group of patients that receives the drug under investigation. 

Control group: The group of patients that receives a different type of treatment, either a traditional one (already approved and used in therapy) or no treatment (a placebo). 

Randomization: The process of assigning individual patients to different treatment groups in such a way that each patient has the same chance, equal to and independent of every other patient, of being selected for any particular study or treatment group. The idea is to make all study groups as equal as possible at the beginning of the experiment. 

Blinding: The process of ensuring that almost everyone involved in the drug trial is unaware of who is receiving the experimental drug and who is receiving a traditional drug treatment, or a placebo, throughout the duration of the study. In experimental studies, lack of knowledge about which patients are receiving which treatments may be limited to the patients (single-blind); patients and treating clinicians (double-blind); or patients, treating clinicians, and the scientific investigators (triple-blind). 

Placebo: An inactive form of treatment, usually an inert sugar pill, received by patients in the control group. This treatment provides the basis for a trial group (the controls) to receive no (beneficial) treatment so that a good comparison can be made with the results of the experimental group. The use of placebo in most clinical trials is declining due to ethical concerns.

Page 14: Experimental Study Designs Dr. lamya Alnaim. Introduction  The primary method for testing the effectiveness of new therapies and other interventions,

The "clinical inquiry" a simple, nonexperimental method in

which only one group of patients receives a new drug

data are collected and analyzed to see if it had any effect;

no control or comparison group is used.

Page 15: Experimental Study Designs Dr. lamya Alnaim. Introduction  The primary method for testing the effectiveness of new therapies and other interventions,

Controlled clinical trial The control group consists of participants not

from the original pool of potential study patients or patients whose past cases involved receiving other treatments in regular health care or research situations.

Page 16: Experimental Study Designs Dr. lamya Alnaim. Introduction  The primary method for testing the effectiveness of new therapies and other interventions,

The randomized, controlled clinical trial The best method for determining the true

therapeutic benefit of a new unknown treatment

A clinical trial is often performed at more than one facility to increase the number and diversity of the subjects who choose to participate in it, leading to more powerful and generalizable results.

This may also shorten the time of the study.

Page 17: Experimental Study Designs Dr. lamya Alnaim. Introduction  The primary method for testing the effectiveness of new therapies and other interventions,

Variables

Page 18: Experimental Study Designs Dr. lamya Alnaim. Introduction  The primary method for testing the effectiveness of new therapies and other interventions,

Types of variables Variable

A characteristic that changes or has different values for different individuals (EX: height, level of anxiety, temperature of room)

Independent Variable (IV)Variable that is changed or manipulated

by the experimenter Dependent Variable (DV)

Variable that used to measure the change or effect of the IV

Page 19: Experimental Study Designs Dr. lamya Alnaim. Introduction  The primary method for testing the effectiveness of new therapies and other interventions,

Types of variables Control Variables:

held constant by the experimenter to eliminate them as potential causes.

useing only research participants who have problems with anxiety or depression, this diagnosis would be a control variable.

Random Variables: allowed to vary freely to eliminate them as

potential causes. Characteristics of the research participants, as

long as they really do vary freely. Examples: age, personality type, or career goals.

Page 20: Experimental Study Designs Dr. lamya Alnaim. Introduction  The primary method for testing the effectiveness of new therapies and other interventions,

Types of variables Confounding Variables:

vary systematically with the independent variable

May be a cause. Good experimental designs

eliminate them.

Page 21: Experimental Study Designs Dr. lamya Alnaim. Introduction  The primary method for testing the effectiveness of new therapies and other interventions,

Types of variables Confounding Variables:

If we divide the research participants into two groups, the experimental group the control group

A systematic difference between these two groups, it will not be a fair test.

If those in the experimental group know they are getting a new treatment and therefore expect to get better, the expectations will be a confounding variable.

If the experimental group does improve, we will not know whether it was because of the therpay itself (the Independent Variable) or because of the participants' expectations (a Confounding Variable).

Page 22: Experimental Study Designs Dr. lamya Alnaim. Introduction  The primary method for testing the effectiveness of new therapies and other interventions,

Reliability And Validity

Page 23: Experimental Study Designs Dr. lamya Alnaim. Introduction  The primary method for testing the effectiveness of new therapies and other interventions,

Reliability And Validity 1. Reliability

Are the results of the experiment repeatable?

If the experiment were done the same way again, would it produce the same results?

Reliability is a requirement before the validity of the experiment can be established.

 

Page 24: Experimental Study Designs Dr. lamya Alnaim. Introduction  The primary method for testing the effectiveness of new therapies and other interventions,

Reliability And Validity II. Internal validity

Accuracy or truth-value Does the research design lead to true

statements? Did the independent variable cause the

effects in the dependent variable? In experimental research, this usually

means eliminating alternative hypotheses. whether the therapy really was the causal

factor in improving participants' state.

Page 25: Experimental Study Designs Dr. lamya Alnaim. Introduction  The primary method for testing the effectiveness of new therapies and other interventions,

Reliability And Validity III. External validity

Generalizability Can the results can be applied in

another setting or to another population of research participants?

Page 26: Experimental Study Designs Dr. lamya Alnaim. Introduction  The primary method for testing the effectiveness of new therapies and other interventions,

Hypotheses

Page 27: Experimental Study Designs Dr. lamya Alnaim. Introduction  The primary method for testing the effectiveness of new therapies and other interventions,

Hypotheses Hypothesis

Makes a prediction about how the manipulation of the independent variable will affect the dependent variable

ORMakes a prediction about the relationship between two variables (a correlation)

Page 28: Experimental Study Designs Dr. lamya Alnaim. Introduction  The primary method for testing the effectiveness of new therapies and other interventions,

Types of Hypotheses Experimental research methods revolve

around hypotheses, educated guesses. We start with a hypothesis about how the

results will turn out, i.e., that there is an effect and it is due to the independent variable.

This first hypothesis is the research hypothesis.

Then we hold the possibility that there is no effect of the independent variable on the dependent variable or that the differences observed are due to chance only. This hypothesis is the null hypothesis.

Page 29: Experimental Study Designs Dr. lamya Alnaim. Introduction  The primary method for testing the effectiveness of new therapies and other interventions,

Types of Hypotheses The first step in experimential research,

then, is ruling out chance. we set up an experimental design that

will allow us to reject the null hypothesis.

If we can confidently reject the null hypothesis, then we gain confidence in the research hypothesis.

At this point, another group of hypotheses comes into play, the alternative hypotheses

Page 30: Experimental Study Designs Dr. lamya Alnaim. Introduction  The primary method for testing the effectiveness of new therapies and other interventions,

Types of Hypotheses If there is an effect beyond chance, it may

be due to the independent variable or it may be due to a number of other factors, so-called extraneous variables or confounding variables.

We use experimental designs to allow us to eliminate alternative hypotheses.

The bottom line for experimental designs is this: THE PURPOSE OF EXPERIMENTAL

DESIGNSIS TO ELIMINATE ALTERNATIVE HYPOTHESES.

Page 31: Experimental Study Designs Dr. lamya Alnaim. Introduction  The primary method for testing the effectiveness of new therapies and other interventions,

Types of HypothesesNull hypothesis  states that differences are due to

chance or that there are no differences between treatments (used in statistical analysis). the null hypothesis is that the new

form of therapy is no better than either no therapy or conventional therapies.

Page 32: Experimental Study Designs Dr. lamya Alnaim. Introduction  The primary method for testing the effectiveness of new therapies and other interventions,

Types of HypothesesAlternative hypotheses  suggest that results are due to

factors other than IV. These factors, rather than the

independent variable, may cause the improvements.

Page 33: Experimental Study Designs Dr. lamya Alnaim. Introduction  The primary method for testing the effectiveness of new therapies and other interventions,

Types of Experimental Design

Page 34: Experimental Study Designs Dr. lamya Alnaim. Introduction  The primary method for testing the effectiveness of new therapies and other interventions,

Pre-experimental or faulty designs One group of subjects gets one

treatment. a pre- and post-test or just a post-

test. May eliminate chance, otherwise

eliminates no alternative hypotheses.

Page 35: Experimental Study Designs Dr. lamya Alnaim. Introduction  The primary method for testing the effectiveness of new therapies and other interventions,

I-Pre-experimental or faulty designsExample Research participants who receive the new form

of therapy are tested afterward, or participants are measured before and after the therapy. This may be useful in showing that there is

some reason to believe the new therapy works,

we cannot draw any conclusions about why there is improvement.

It should be considered a pilot test at best and followed up with a better research design.

Page 36: Experimental Study Designs Dr. lamya Alnaim. Introduction  The primary method for testing the effectiveness of new therapies and other interventions,

II-Quasi-experimental Designs Eliminate some, but not all,

alternative hypotheses. They are especially useful in applied

settings where real-life constraints make it undesirable or impossible to control the research setting.

 

Page 37: Experimental Study Designs Dr. lamya Alnaim. Introduction  The primary method for testing the effectiveness of new therapies and other interventions,

II-Quasi-experimental DesignsExample if the new therapy is being used in a mental

health center or a private practice. Rather than compromise the needs of clients to eliminate alternative hypotheses, we would be willing to allow some alternative hypotheses. This is a choice of relevance and external validity over control and internal validity.

Ideally, such a design would be paired with others to allow us to draw stronger conclusions.

Page 38: Experimental Study Designs Dr. lamya Alnaim. Introduction  The primary method for testing the effectiveness of new therapies and other interventions,

II-Quasi-experimental Designs1-Non-equivalent groups or static groups design

Two groups receive different treatments, but are not randomly assigned or matched to conditions.

Eliminates history effects but not subject effects.

Page 39: Experimental Study Designs Dr. lamya Alnaim. Introduction  The primary method for testing the effectiveness of new therapies and other interventions,

II-Quasi-experimental DesignsExample Participants may be given the choice of which

therapy to receive. The participants most likely to benefit from

the new therapy are assigned to that condition.

Intact (already-existing) group may be usedall the clients in an existing therapy group

If the group receiving the new therapy improves more than the control group, we can be somewhat more confident in the benefits of the new therapy.

Page 40: Experimental Study Designs Dr. lamya Alnaim. Introduction  The primary method for testing the effectiveness of new therapies and other interventions,

II-Quasi-experimental Designs2. Time-series design There is one group of research

participants with several baseline measures, a treatment, and at least one more measurement.

Eliminates subject effects but not history effects.

Page 41: Experimental Study Designs Dr. lamya Alnaim. Introduction  The primary method for testing the effectiveness of new therapies and other interventions,

II-Quasi-experimental DesignsExample One group of research participants is

selected for the study. Endpoint is measured each month for

several months. Then given the new therapy and measured again.

If they improve after the therapy, we are more confident the new therapy helps.

Used most often to evaluate public policy changes which affect a large group of people.

Page 42: Experimental Study Designs Dr. lamya Alnaim. Introduction  The primary method for testing the effectiveness of new therapies and other interventions,

II-Quasi-experimental Designs3. Multiple time-series design Two or more groups (not randomly

assigned) receive several pre-treatment measures and at least one post-treatment measure.

Can eliminate history and most subject effects. considered a "strong quasi-

experimental design."

Page 43: Experimental Study Designs Dr. lamya Alnaim. Introduction  The primary method for testing the effectiveness of new therapies and other interventions,

II-Quasi-experimental DesignsExample: After taking a series of

measurements at two different Clinics, all clients at one clinic are given the new therapy.

If those clients improve more than the clients at the "control" clinic we can be more confident of the new therapy.

Page 44: Experimental Study Designs Dr. lamya Alnaim. Introduction  The primary method for testing the effectiveness of new therapies and other interventions,

III-True Experimental Designs These designs attempt to eliminate

most alternative hypotheses, especially those related to time (history, maturation, and regression) and those related to make-up of the groups (selection effects).

Such control may be at the expense of making the situation too artificial.

Page 45: Experimental Study Designs Dr. lamya Alnaim. Introduction  The primary method for testing the effectiveness of new therapies and other interventions,

III-True Experimental Designs1. Randomized groups design, between-groups design

Each research participant is randomly assigned to one group and gets only one level of the indepndent variable.

There may be pre-tests and post-tests or only post-tests.

This design can eliminate selection, history, and maturation effects.

Page 46: Experimental Study Designs Dr. lamya Alnaim. Introduction  The primary method for testing the effectiveness of new therapies and other interventions,

III-True Experimental Designs2. Repeated measure design, within-subject design

Each research participant gets all levels of the IV.

Treatment orders must be counter balanced to eliminate order effects.

 

Page 47: Experimental Study Designs Dr. lamya Alnaim. Introduction  The primary method for testing the effectiveness of new therapies and other interventions,

III-True Experimental Designs3. Mixed model designs or complex designs

These designs combine randomized groups and repeated measdures designs.

For instance, there may be two IVs, one measured between groups and one measured within groups.

Page 48: Experimental Study Designs Dr. lamya Alnaim. Introduction  The primary method for testing the effectiveness of new therapies and other interventions,

Back to Validity

Page 49: Experimental Study Designs Dr. lamya Alnaim. Introduction  The primary method for testing the effectiveness of new therapies and other interventions,

Validity ThreatsSubject effect, selection effect: results are due to systematic differences in

research participants assigned to different treatments.

Example: If the research participants who receive the new form of therapy are different from those in a control One group could be healthier, more

motivated, or more experienced with psychotherapy.

Common solution: Matching or random assignment to groups

Page 50: Experimental Study Designs Dr. lamya Alnaim. Introduction  The primary method for testing the effectiveness of new therapies and other interventions,

Validity ThreatsHistory effect: results are due to events outside the experiment. Example: if there is one group of research

participants who are being measured at several points in time. Some event that is not part of the research, say

something traumatic like a natural disaster, which occurs at the same time as the treatment could affect the results.

Common solution: A control group which will be exposed to the same history but not the new form of therapy.

Page 51: Experimental Study Designs Dr. lamya Alnaim. Introduction  The primary method for testing the effectiveness of new therapies and other interventions,

Validity ThreatsMaturation effect: results are due to changes within subjects over

time, e.g., growth, warm-up, fatigue, learning to learn.

This is a problem in research that measures a DV over a period of time and especially in research with repeated exposures to the IV.

Example: If one group of participants, have improvments over time without the new form of therapy.

Common solution: A control group which is measured over the same period of time but does not receive the new therapy.

Page 52: Experimental Study Designs Dr. lamya Alnaim. Introduction  The primary method for testing the effectiveness of new therapies and other interventions,

Validity Threats

Experimenter expectancy effect, Experimenter bias:

results are due to the experimenter's actions or expectations.

A number of studies have shown that researchers tend to find the results they are looking for.

The causes for this range from overt cheating to very subtle influences on data collection and interactions with research participants.

Experimenters are not always aware of the extent of these influences.

Page 53: Experimental Study Designs Dr. lamya Alnaim. Introduction  The primary method for testing the effectiveness of new therapies and other interventions,

Validity Threats

Example If the researcher is the one to assess the

research participants‘ dependent variable, he or she may distort the assessments in the direction of the research hypothesis.

Common solution: Use independent judges or more objective measurements of the dependent variable.

Page 54: Experimental Study Designs Dr. lamya Alnaim. Introduction  The primary method for testing the effectiveness of new therapies and other interventions,

Validity ThreatsDemand characteristics, Hawthorne effect: 

Results are due to subjects' expectations of desired behavior in the research setting

Called "demand" because participants may perceive a demand to behave or report on themselves in a certain way.

Called the Hawthorne Effect after a famous series of experiments at a manufacturing plant in Hawthorne, Ohio.

Page 55: Experimental Study Designs Dr. lamya Alnaim. Introduction  The primary method for testing the effectiveness of new therapies and other interventions,

Validity Threats researchers selected a group of

factory workers and changed various conditions such as lighting to see what would increase performance.

They found that any change increased performance, suggesting that research participants were responding to the general expectation that they would perform better and to the social dynamics of being observed closely.

Page 56: Experimental Study Designs Dr. lamya Alnaim. Introduction  The primary method for testing the effectiveness of new therapies and other interventions,

Validity ThreatsExample The researcher communicates his or her

expectations to the research participants which in turn influences their responses.

Common solution: Blind and double-blind designs help avoid these problems.

Also, using a control group which is measured the same way without the treatment.

Page 57: Experimental Study Designs Dr. lamya Alnaim. Introduction  The primary method for testing the effectiveness of new therapies and other interventions,

Validity ThreatsTesting effect, reactivity: results are due to the data gathering procedures,

e.g., being influenced by the test or learning from one test administration to the next.

Example Measuring the participants' mental health could

get them thinking about their lives, thus improving them.

Improvements would then be due to the data gathering, not the therapy itself.

Common solution: Use a control group which is also measured, but without the therapy or with an alternative form of therapy.

Page 58: Experimental Study Designs Dr. lamya Alnaim. Introduction  The primary method for testing the effectiveness of new therapies and other interventions,

Validity ThreatsRegression artifact, regression-to-the-

mean results are due to extreme scores moving

toward the mean over time. Example: If a group is made up of those with

the worst mental health scores over time they are likely to improve without therapy.

This may be mistakenly attributed to the therapy.

Common solution: Use a control group which has similar charcteristics but which does not receive the new therapy

Page 59: Experimental Study Designs Dr. lamya Alnaim. Introduction  The primary method for testing the effectiveness of new therapies and other interventions,

Validity ThreatsInstrumentation results are due to an aberration in

measuring tools, either mechanical instrument or test.

Example: The dependent variable may be measured by a poor test.

Common solution: Select or develop a better measure.

Page 60: Experimental Study Designs Dr. lamya Alnaim. Introduction  The primary method for testing the effectiveness of new therapies and other interventions,

Validity ThreatsHalo effect the researcher's expectations about

certain subjects based on some subject characteristics.

Example: Judges rating the mental health of the participants (the dependent variable) may ascribe better mental health based on other characteristics.

Common solutions: Random assignment, blind judges, more objective measures.

Page 61: Experimental Study Designs Dr. lamya Alnaim. Introduction  The primary method for testing the effectiveness of new therapies and other interventions,

Validity ThreatsAttrition or mortality effect When subjects drop out of an

experiment, it can bias the results. Especially true when more subjects

drop out of one treatment condition than another.

The study is no longer a fair test. This leads to a kind of subject effect

because the subjects in the different groups are no longer equivalent.

Page 62: Experimental Study Designs Dr. lamya Alnaim. Introduction  The primary method for testing the effectiveness of new therapies and other interventions,

Validity ThreatsAttrition or mortality effect If a study consists of a new therapy group, a

conventional therapy group, and a no-therapy group. If more subjects in the new therapy group drop out

of the study, it may be because the new therapy was not appropriate for them.

This leaves only those who benefitted most, making the therapy look better than it really is.

Common solution: There is no way to force research participants to stay in the study, but if attrition looks like a problem, find out why participants dropped out.

This can sometimes give important clues about the study.

Page 63: Experimental Study Designs Dr. lamya Alnaim. Introduction  The primary method for testing the effectiveness of new therapies and other interventions,

Randomized, Controlled Clinical Trials

Page 64: Experimental Study Designs Dr. lamya Alnaim. Introduction  The primary method for testing the effectiveness of new therapies and other interventions,

Randomized, Controlled Clinical Trials

the most scientific and ethical method for investigation of new therapies

The main objective of this method is to make certain that, after the trial is over, the better (or best) of the studied treatments is identified.

The key components in a randomized clinical trial is the use of a control group and randomization of subjects into the study groups.

Not all clinical trials, however, are of this type.

Page 65: Experimental Study Designs Dr. lamya Alnaim. Introduction  The primary method for testing the effectiveness of new therapies and other interventions,

Randomized, Controlled Clinical Trials In some circumstances, the randomization

process may not be justified. This issue has arisen in recent research on

drugs for treatment of AIDS. Patients with AIDS believe that they all should be in the clinical trial and in the new investigational drug study group. No one should be denied an experimental treatment because they were randomly selected to be in the control or placebo group.

Patients should always be informed of any randomization process that is being used to influence their treatment choices.

Page 66: Experimental Study Designs Dr. lamya Alnaim. Introduction  The primary method for testing the effectiveness of new therapies and other interventions,

Randomized, Controlled Clinical Trials still are far from being a perfect method. Clinical trials answer only questions that

have been asked specifically. the experimental conditions of the trial

often differ so much from the conditions encountered in clinical practice that the trial's results may not be applicable to real-life situations.

Concerns about the quality of clinical trials.

Page 67: Experimental Study Designs Dr. lamya Alnaim. Introduction  The primary method for testing the effectiveness of new therapies and other interventions,

Randomization It maximizes the probability that the two

groups are as similar as possible in terms of background characteristics, especially factors and variables that may influence response to therapy or primary outcome measure.

In a randomized study, treatment group assignment is based on probability alone and is not influenced by the physicians' or patients' preferences.

Every patient in the original patient pool, or group, must have a chance equal to every other patient of being assigned to a particular treatment group.

Page 68: Experimental Study Designs Dr. lamya Alnaim. Introduction  The primary method for testing the effectiveness of new therapies and other interventions,

Blinding A process of keeping individuals

involved in the study unaware of assignment of subjects to different study groups.

Single-blind keeping the subjects unaware

double-blind: keeping the investigators and subjects unaware

triple-blind: data analysis is done by outside evaluators independent of the investigators.

Page 69: Experimental Study Designs Dr. lamya Alnaim. Introduction  The primary method for testing the effectiveness of new therapies and other interventions,

Blinding When a study is double-blinded and

randomization of patients was used, most people involved in the study should not know who is getting which treatment, or no treatment, and a more objective assessment of the new therapy can be made.

The blinding process can break down in studies where some patients are not receiving any treatment, and other patients are taking a very active drug with intensely noticeable effects.

Page 70: Experimental Study Designs Dr. lamya Alnaim. Introduction  The primary method for testing the effectiveness of new therapies and other interventions,

Intention-to-Treat and Interim Analyses Comparison of the entire experimental group with

the entire study group regardless of whether subjects completed the full course of their study treatments.

Loss to follow-up is an issue. Investigators often have more control over

patient involvement throughout the clinical trial because of the patient's investment in seeking a new, better treatment.

The key issue is whether the probability of loss to follow-up is related to exposures, treatments, the disease, and other study outcomes.

Page 71: Experimental Study Designs Dr. lamya Alnaim. Introduction  The primary method for testing the effectiveness of new therapies and other interventions,

Intention-to-Treat and Interim Analyses The major advantage of ITT analysis

is that it maintains the randomization scheme, as subjects do not randomly drop out of studies.

It is necessary to develop guidelines for deciding whether a trial should be modified or stopped before originally scheduled.

This decision is usually based on interim analyses.

Page 72: Experimental Study Designs Dr. lamya Alnaim. Introduction  The primary method for testing the effectiveness of new therapies and other interventions,

Intention-to-Treat and Interim AnalysesInterim analyses. To ensure that the welfare of the

participants is protected, and to provide useful new therapies to the medical community, study result should be monitored at various points by a group that is independent of the investigators conducting the trial.

If the data indicate a clearly significant benefit in terms of the primary endpoint, or if one treatment is clearly harmful, then early termination of the trial must be considered

Page 73: Experimental Study Designs Dr. lamya Alnaim. Introduction  The primary method for testing the effectiveness of new therapies and other interventions,

Intention-to-Treat and Interim Analyses The decision to terminate a study

early is based on a number of complex issues and must be made with great caution.

The first requirement for considering termination or modification is the observation of a sustained statistical association that is extreme and highly significant, virtually impossible to arise by chance alone.

Page 74: Experimental Study Designs Dr. lamya Alnaim. Introduction  The primary method for testing the effectiveness of new therapies and other interventions,

Intention-to-Treat and Interim Analyses Statistical test results should not be

used as the sole basis for the decision to stop or continue a trial.

Observing significant associations must be considered in the context of the totality of evidence, including postulated or known biological mechanisms that might explain the results, the results of other clinical trials or observational studies.

Page 75: Experimental Study Designs Dr. lamya Alnaim. Introduction  The primary method for testing the effectiveness of new therapies and other interventions,

Process Of Performing A Clinical Trial

Page 76: Experimental Study Designs Dr. lamya Alnaim. Introduction  The primary method for testing the effectiveness of new therapies and other interventions,

Process Of Performing A Clinical Trial The first step is to formulate the

major research question. usually is referred to as a

hypothesis. determines the importance of

selected independent variables, such as the types of interventions or treatments to be compared and the nature of the dependent variables, endpoints, or outcomes to be evaluated.

Page 77: Experimental Study Designs Dr. lamya Alnaim. Introduction  The primary method for testing the effectiveness of new therapies and other interventions,

Process Of Performing A Clinical Trial The number and nature of subjects for the

study as well as eligibility and exclusion requirements for subject participation in the study.

Endpoints must be determined in terms of clinical importance, which ones can be measured in a reasonable manner, and which ones can be studied in the given population with the constraints and resources available to the study. More than one endpoint can be measured

to assess treatment efficacy.

Page 78: Experimental Study Designs Dr. lamya Alnaim. Introduction  The primary method for testing the effectiveness of new therapies and other interventions,

Sample Size Determination And Data Analysis The number of subjects who should

be enrolled in the clinical trial is known as the sample size.

should be determined soon after the primary research question or outcome has been formulated.

At the conclusion of an experiment, the data are analyzed and a statistical decision is made either to accept or to reject the hypothesis.

Page 79: Experimental Study Designs Dr. lamya Alnaim. Introduction  The primary method for testing the effectiveness of new therapies and other interventions,

Sample Size Determination And Data Analysis The decision to accept or reject a

hypothesis is based on probabilities, and the study results based on these decisions may not truly reflect what is occurring in "reality.“

Reality, or "the truth," can be thought of as the results of the intervention when they are applied correctly to all possible patients (i.e., anyone with the clinical condition).

Page 80: Experimental Study Designs Dr. lamya Alnaim. Introduction  The primary method for testing the effectiveness of new therapies and other interventions,

Sample Size Determination And Data Analysis Using only a sample of the entire patient

population for the trial, the investigator hopes to use the trial's results to generalize about all patients.

The problem is that the investigator is intervening only on the study sample and cannot intervene with the whole population of patients.

There is always a risk of arriving at a mistaken conclusion because the trial sample did not represent all types of patients with the same disease

Page 81: Experimental Study Designs Dr. lamya Alnaim. Introduction  The primary method for testing the effectiveness of new therapies and other interventions,

REALITY

Outcome different Outcome not different

A B

STUDY RESULTS

Outcome different Study result valid (true positive)

Type I error (false positive)

C D

Outcome Not different

Type I I error (false negative)

Study result valid (true negative)

The relationship between clinical trial results and reality

Page 82: Experimental Study Designs Dr. lamya Alnaim. Introduction  The primary method for testing the effectiveness of new therapies and other interventions,

The relationship between clinical trial results and reality Cells A and D represent a relationship

between study results and reality that is correct. The study results are valid and truly

reflect what would happen to other patients in the population if they were given the same treatment as those in the trial sample.

Page 83: Experimental Study Designs Dr. lamya Alnaim. Introduction  The primary method for testing the effectiveness of new therapies and other interventions,

The relationship between clinical trial results and reality Cells A and D represent a relationship

between study results and reality that is correct. In cell A, the study results and

reality both indicate that the outcomes are different (e.g., the new drug is more effective than a placebo). This is also known as a true-positive result.

Page 84: Experimental Study Designs Dr. lamya Alnaim. Introduction  The primary method for testing the effectiveness of new therapies and other interventions,

The relationship between clinical trial results and reality Cells A and D represent a relationship

between study results and reality that is correct. In cell D, the study results and

reality both indicate that the outcomes are not different (e.g., the new drug is not more effective than a placebo). This is known as a true-negative result.

Page 85: Experimental Study Designs Dr. lamya Alnaim. Introduction  The primary method for testing the effectiveness of new therapies and other interventions,

The relationship between clinical trial results and reality

Two types of errors can be made in interpreting study results.

1. The results might indicate that there is a difference in outcomes or treatments when in reality there is no difference (cell B).

This is a type I error; under these circumstances, the

study results are not valid but are falsely positive.

Page 86: Experimental Study Designs Dr. lamya Alnaim. Introduction  The primary method for testing the effectiveness of new therapies and other interventions,

The relationship between clinical trial results and reality

Two types of errors can be made in interpreting study results.

2. if the study failed to find a difference in outcomes or treatments when there actually was a difference, a type II error has occurred (cell C)

under these circumstances, the study results are not valid but are falsely negative.

Page 87: Experimental Study Designs Dr. lamya Alnaim. Introduction  The primary method for testing the effectiveness of new therapies and other interventions,

The relationship between clinical trial results and reality

Falsely positive or falsely negative study results can occur because of

faulty methodology, chance occurrence, or for both

some errors can be minimized by careful attention to study design

errors due to chance can never be completely eliminated. Such errors, however, can be estimated.

Page 88: Experimental Study Designs Dr. lamya Alnaim. Introduction  The primary method for testing the effectiveness of new therapies and other interventions,

The relationship between clinical trial results and reality The notation used to describe the

likelihood of a type I error (i.e., the difference observed between treatment groups is not a true difference but due instead to chance) is the alpha level.

The notation used to describe the likelihood of a type II error (i.e., the study did not find a difference between treatment groups when there is indeed a difference) is called the beta level.

Page 89: Experimental Study Designs Dr. lamya Alnaim. Introduction  The primary method for testing the effectiveness of new therapies and other interventions,

The relationship between clinical trial results and reality Researchers specify the levels when

planning a study. an alpha level may be specified at

0.05, a 5% risk of committing a type I error, falsely concluding that the treatment groups differ when in reality they do not.

Or specify a level, or risk of committing a type II error, of 0.1, a 10% chance of missing a true difference between the treatment groups is being allowed

Page 90: Experimental Study Designs Dr. lamya Alnaim. Introduction  The primary method for testing the effectiveness of new therapies and other interventions,

The relationship between clinical trial results and reality The statistical power, or ability of a

study to detect a true difference between groups, is 1-beta . For a beta level of 0.10, the study

would have a 90% chance of detecting the true difference in outcomes between treatment groups.

Page 91: Experimental Study Designs Dr. lamya Alnaim. Introduction  The primary method for testing the effectiveness of new therapies and other interventions,

The relationship between clinical trial results and reality Once the alpha and beta levels have

been established, must specify an extremely important study parameter for determining sample size—the magnitude of the difference in outcome between treatment groups that the study will be designed to detect.

Should be selected on the basis of clinical information.

Page 92: Experimental Study Designs Dr. lamya Alnaim. Introduction  The primary method for testing the effectiveness of new therapies and other interventions,

The relationship between clinical trial results and reality In deciding on the level of outcome difference

for detection, the following may be considered The difference in outcome that would be

important to clinicians treating this type of patient

The difference that would be meaningful to a patient who may suffer the consequences of the disease,

the difference in outcome that would justify use of the more effective treatment despite greater expense or greater side effects

Page 93: Experimental Study Designs Dr. lamya Alnaim. Introduction  The primary method for testing the effectiveness of new therapies and other interventions,

Clinical Trials And The U.S. Drug Approval Process

Page 94: Experimental Study Designs Dr. lamya Alnaim. Introduction  The primary method for testing the effectiveness of new therapies and other interventions,

Patient Compliance In Clinical Drug Research

Page 95: Experimental Study Designs Dr. lamya Alnaim. Introduction  The primary method for testing the effectiveness of new therapies and other interventions,

Patient Compliance In Clinical Drug Research The failure of patients to take drugs as

prescribed, called noncompliance, has been documented, and its implications for patient care are well known to health professionals.

The impact noncompliance can have on the drug development process, from clinical trials to marketing, has not been well appreciated by pharmaceutical manufacturers and regulators.

Noncompliance occurs for a variety of reasons, and many determinants have been identified through research

Page 96: Experimental Study Designs Dr. lamya Alnaim. Introduction  The primary method for testing the effectiveness of new therapies and other interventions,

Patient Compliance In Clinical Drug Research Noncompliance by participants in

clinical trials affects the evaluation and approval of new pharmaceutical agents at two stages.

Page 97: Experimental Study Designs Dr. lamya Alnaim. Introduction  The primary method for testing the effectiveness of new therapies and other interventions,

Patient Compliance In Clinical Drug Research

First, during the pre-approval drug development phases, noncompliance may alter results of clinical investigations that determine the optimal dose for package insert labeling.

Page 98: Experimental Study Designs Dr. lamya Alnaim. Introduction  The primary method for testing the effectiveness of new therapies and other interventions,

Patient Compliance In Clinical Drug Research

Second after the drug has been approved and is marketed, noncompliance, especially if it is widespread, may lead to the emergence of unexpected problems, such as irregular dosing, overprescribing, failures to achieve patient care outcomes, and toxicities.

Page 99: Experimental Study Designs Dr. lamya Alnaim. Introduction  The primary method for testing the effectiveness of new therapies and other interventions,

Patient Compliance In Clinical Drug Research Some drug approval delays can be

attributed to noncompliance and the failure of clinical investigators to recognize it.

There may be difficulty in demonstrating a drug's effectiveness in intention-to-treat analyses, which assumes that patients follow the regimens to which they were assigned.

Noncompliance also can interfere with determination of therapeutic dosage ranges.

Page 100: Experimental Study Designs Dr. lamya Alnaim. Introduction  The primary method for testing the effectiveness of new therapies and other interventions,

Patient Compliance In Clinical Drug Research Most trials that now report the safety

and efficacy of a drug do not reflect the individual patient's response to a prescribed dosage.

They reflect a composite of the responses of patients who fail to comply, those who partially comply, and those who fully comply with the therapeutic regimen.

Page 101: Experimental Study Designs Dr. lamya Alnaim. Introduction  The primary method for testing the effectiveness of new therapies and other interventions,

Patient Compliance In Clinical Drug Research It is necessary to take into account the

impact of known compliance or noncompliance on estimates of a drug's safety and efficacy.

Giving equal weight to compliant patients who took the drug and to patients who never took the drug, took the wrong drug, or took inadequate amounts increases the probability of overestimating the therapeutic dose range.

Page 102: Experimental Study Designs Dr. lamya Alnaim. Introduction  The primary method for testing the effectiveness of new therapies and other interventions,

Patient Compliance In Clinical Drug Research Adverse reactions then could occur when

patients take the recommended dose in practice.

Additional trials may be necessary for approval, and the problem resides with patient noncompliance, not the drug product's dose or formulation.

Noncompliance also affects statistical analysis of clinical trial and postmarketing data.

Page 103: Experimental Study Designs Dr. lamya Alnaim. Introduction  The primary method for testing the effectiveness of new therapies and other interventions,

Patient Compliance In Clinical Drug Research Clinical trial research, particularly on chronic

diseases, must include new endpoints. Priority should be given to subjective, in

addition to objective, endpoints that imply a focus on risk factors and symptomatic treatments over cures. Such research needs to emphasize the psychosocial component as equally or more important than the biological component.

These areas of study are exploring the emergent quality-of-life concepts and new ethical priorities in health care.