Getting SMART About Adaptive Interventions
Transcript of Getting SMART About Adaptive Interventions
![Page 1: Getting SMART About Adaptive Interventions](https://reader030.fdocuments.us/reader030/viewer/2022012708/61a8a7919a3cd449d4355198/html5/thumbnails/1.jpg)
Getting SMART About Adaptive Interventions
Daniel Almirall1,2 Xi Lu (Lucy)1,2,3
Inbal Nahum-Shani1,2 Linda Collins2
Susan A. Murphy1,2,3,4
1Institute for Social Research, Univ of Michigan,2The Methodology Center, Penn State Univ,3Department of Statistics, Univ of Michigan,4Department of Psychiatry, Univ of Michigan
CROI, Seattle, WA - Feb-23-2015
Almirall, Xu, Nahum-Shani, Collins, Murphy Getting SMART 1 / 58
![Page 2: Getting SMART About Adaptive Interventions](https://reader030.fdocuments.us/reader030/viewer/2022012708/61a8a7919a3cd449d4355198/html5/thumbnails/2.jpg)
Adaptive InterventionsSequential Multiple Assignment Randomized Trial (SMART)
SMART Design PrinciplesTake Home Points
Adaptive Implementation Interventions
Thanks to Richard Chaisson, Susan Buchbinder andCROI organizers!
Disclosures or Conflicts of Interest
None
Almirall, Xu, Nahum-Shani, Collins, Murphy Getting SMART 2 / 58
![Page 3: Getting SMART About Adaptive Interventions](https://reader030.fdocuments.us/reader030/viewer/2022012708/61a8a7919a3cd449d4355198/html5/thumbnails/3.jpg)
Adaptive InterventionsSequential Multiple Assignment Randomized Trial (SMART)
SMART Design PrinciplesTake Home Points
Adaptive Implementation Interventions
Outline (25min/30slides + Q&A)
Adaptive InterventionsWhat? Why?
Sequential Multiple Assignment Randomized Trial (SMART)What are SMARTs?
SMART Design PrinciplesKeep it SimpleChoosing Primary and Secondary Hypotheses
Take Home Points
Adaptive Implementation Interventions
Almirall, Xu, Nahum-Shani, Collins, Murphy Getting SMART 3 / 58
![Page 4: Getting SMART About Adaptive Interventions](https://reader030.fdocuments.us/reader030/viewer/2022012708/61a8a7919a3cd449d4355198/html5/thumbnails/4.jpg)
Adaptive InterventionsSequential Multiple Assignment Randomized Trial (SMART)
SMART Design PrinciplesTake Home Points
Adaptive Implementation Interventions
What? Why?
Adaptive Interventions
Almirall, Xu, Nahum-Shani, Collins, Murphy Getting SMART 4 / 58
![Page 5: Getting SMART About Adaptive Interventions](https://reader030.fdocuments.us/reader030/viewer/2022012708/61a8a7919a3cd449d4355198/html5/thumbnails/5.jpg)
Adaptive InterventionsSequential Multiple Assignment Randomized Trial (SMART)
SMART Design PrinciplesTake Home Points
Adaptive Implementation Interventions
What? Why?
Definition: A Adaptive Intervention is
I a sequence of individually tailored decision rulesI that specify whether, how, or whenI and based on which measuresI to alter the dosage (duration, frequency or amount), type,
or deliveryI at critical decision points in the course of care.
Adaptive Interventions (AIs) help guide the type of sequentialtreatment decision making that is typical (and often needed!) ofclinical practice.
Almirall, Xu, Nahum-Shani, Collins, Murphy Getting SMART 5 / 58
![Page 6: Getting SMART About Adaptive Interventions](https://reader030.fdocuments.us/reader030/viewer/2022012708/61a8a7919a3cd449d4355198/html5/thumbnails/6.jpg)
Adaptive InterventionsSequential Multiple Assignment Randomized Trial (SMART)
SMART Design PrinciplesTake Home Points
Adaptive Implementation Interventions
What? Why?
Concrete Example of an Adaptive InterventionChild ADHD in Schools, Ages 6-12
I What does it look like from researcher’s/clinician’s POV?I What does it look like from the child’s/parent’s POV?
Almirall, Xu, Nahum-Shani, Collins, Murphy Getting SMART 6 / 58
![Page 7: Getting SMART About Adaptive Interventions](https://reader030.fdocuments.us/reader030/viewer/2022012708/61a8a7919a3cd449d4355198/html5/thumbnails/7.jpg)
Adaptive InterventionsSequential Multiple Assignment Randomized Trial (SMART)
SMART Design PrinciplesTake Home Points
Adaptive Implementation Interventions
What? Why?
What are the parts of an Adaptive Intervention?1. Critical decision points: based on time or other measures2. Treatment options at each stage3. Tailoring variables: to decide how to adapt treatment4. Decision rules: inputs tailoring variable, outputs treatments
aka: dynamic treatment regimens, adaptive txt strategies, txt algorithms,medication algorithms, stepped care, txt policies, multi-stage strategies...
Almirall, Xu, Nahum-Shani, Collins, Murphy Getting SMART 7 / 58
![Page 8: Getting SMART About Adaptive Interventions](https://reader030.fdocuments.us/reader030/viewer/2022012708/61a8a7919a3cd449d4355198/html5/thumbnails/8.jpg)
Adaptive InterventionsSequential Multiple Assignment Randomized Trial (SMART)
SMART Design PrinciplesTake Home Points
Adaptive Implementation Interventions
What? Why?
Why Adaptive Interventions?Necessary...
I Nature of chronic disorders/phenomena (substance use,mental health, autism, diabetes, cancer, HIV/AIDS)
I Waxing and waning course (multiple relapse, recurrence)I Life events, comorbidities, non-adherence may arise
I Disorders for which there is no widely effective treatment.
I Disorders for which there are widely effective treatments,but they are costly or burdensome.
I Bottom line: High heterogeneity in response to treatmentI Within person (over time) and between person
All require sequences of treatment decisions!
Almirall, Xu, Nahum-Shani, Collins, Murphy Getting SMART 8 / 58
![Page 9: Getting SMART About Adaptive Interventions](https://reader030.fdocuments.us/reader030/viewer/2022012708/61a8a7919a3cd449d4355198/html5/thumbnails/9.jpg)
Adaptive InterventionsSequential Multiple Assignment Randomized Trial (SMART)
SMART Design PrinciplesTake Home Points
Adaptive Implementation Interventions
What? Why?
Ok, so adaptive interventions are great, but......there are so many unanswered questions.
Almirall, Xu, Nahum-Shani, Collins, Murphy Getting SMART 9 / 58
![Page 10: Getting SMART About Adaptive Interventions](https://reader030.fdocuments.us/reader030/viewer/2022012708/61a8a7919a3cd449d4355198/html5/thumbnails/10.jpg)
Adaptive InterventionsSequential Multiple Assignment Randomized Trial (SMART)
SMART Design PrinciplesTake Home Points
Adaptive Implementation Interventions
What are SMARTs?
SEQUENTIAL MULTIPLE ASSIGNMENTRANDOMIZED TRIALS (SMARTs)
Almirall, Xu, Nahum-Shani, Collins, Murphy Getting SMART 10 / 58
![Page 11: Getting SMART About Adaptive Interventions](https://reader030.fdocuments.us/reader030/viewer/2022012708/61a8a7919a3cd449d4355198/html5/thumbnails/11.jpg)
Adaptive InterventionsSequential Multiple Assignment Randomized Trial (SMART)
SMART Design PrinciplesTake Home Points
Adaptive Implementation Interventions
What are SMARTs?
What is a Sequential Multiple AssignmentRandomized Trial (SMART)?
I Multi-stage trials; same participants throughoutI Each stage corresponds to a critical decision pointI At each stage, subjects randomized to set of treatment
optionsI The goal of a SMART is to inform the development of
adaptive interventions.
I will give you an example SMART, but first...
Almirall, Xu, Nahum-Shani, Collins, Murphy Getting SMART 11 / 58
![Page 12: Getting SMART About Adaptive Interventions](https://reader030.fdocuments.us/reader030/viewer/2022012708/61a8a7919a3cd449d4355198/html5/thumbnails/12.jpg)
Adaptive InterventionsSequential Multiple Assignment Randomized Trial (SMART)
SMART Design PrinciplesTake Home Points
Adaptive Implementation Interventions
What are SMARTs?
Background for an Example SMARTADHD Treatment in Children Ages 6-12
I Both medication (MED) and behavioral modification(BMOD) have been shown to be efficacious
I However, there is much debate on whether first-lineintervention should be pharmacological of behavioral,especially in younger children
I Further, there is a need for a ”rescue treatment” if the firsttreatment does not go well because 20-50% of children donot substantially improve on BMOD or MED
I So important questions for clinical practice include“What treatment do we begin with: BMOD or MED?””Among non-responders, what second treatment is best?”
Almirall, Xu, Nahum-Shani, Collins, Murphy Getting SMART 12 / 58
![Page 13: Getting SMART About Adaptive Interventions](https://reader030.fdocuments.us/reader030/viewer/2022012708/61a8a7919a3cd449d4355198/html5/thumbnails/13.jpg)
Concrete Example of a SMART: Child ADHDPI: William Pelham, PhD, Florida International University, IES-Funded GrantN = 153, 8 month study, Monthly non-response (ITB < 75% and IRS > 1 domain)
![Page 14: Getting SMART About Adaptive Interventions](https://reader030.fdocuments.us/reader030/viewer/2022012708/61a8a7919a3cd449d4355198/html5/thumbnails/14.jpg)
One of Four Adaptive Interventions Within the SMART
![Page 15: Getting SMART About Adaptive Interventions](https://reader030.fdocuments.us/reader030/viewer/2022012708/61a8a7919a3cd449d4355198/html5/thumbnails/15.jpg)
4 Embedded Adaptive Interventions in this SMART
![Page 16: Getting SMART About Adaptive Interventions](https://reader030.fdocuments.us/reader030/viewer/2022012708/61a8a7919a3cd449d4355198/html5/thumbnails/16.jpg)
Adaptive InterventionsSequential Multiple Assignment Randomized Trial (SMART)
SMART Design PrinciplesTake Home Points
Adaptive Implementation Interventions
Keep it SimpleChoosing Primary and Secondary Hypotheses
SMART DESIGN PRINCIPLES
Almirall, Xu, Nahum-Shani, Collins, Murphy Getting SMART 16 / 58
![Page 17: Getting SMART About Adaptive Interventions](https://reader030.fdocuments.us/reader030/viewer/2022012708/61a8a7919a3cd449d4355198/html5/thumbnails/17.jpg)
Adaptive InterventionsSequential Multiple Assignment Randomized Trial (SMART)
SMART Design PrinciplesTake Home Points
Adaptive Implementation Interventions
Keep it SimpleChoosing Primary and Secondary Hypotheses
SMART Design Principles
1. KISS Principle: Keep It Simple, Straightforward
2. Power for simple important primary hypotheses
3. Take Appropriate steps to develop a moredeeply-individualized (optimized) Adaptive Intervention
Almirall, Xu, Nahum-Shani, Collins, Murphy Getting SMART 17 / 58
![Page 18: Getting SMART About Adaptive Interventions](https://reader030.fdocuments.us/reader030/viewer/2022012708/61a8a7919a3cd449d4355198/html5/thumbnails/18.jpg)
Adaptive InterventionsSequential Multiple Assignment Randomized Trial (SMART)
SMART Design PrinciplesTake Home Points
Adaptive Implementation Interventions
Keep it SimpleChoosing Primary and Secondary Hypotheses
Keep It Simple, StraightforwardOverarching Principle
At each stage, or critical decision point,...I Restrict class of treatment options only by ethical,
feasibility, or strong scientific considerations
I If you do restrict randomizations, use low dimensionalsummary to restrict subsequent treatments
I Use binary responder statusI Should be easy to use in actual clinical practice
I Collect additional, auxiliary time-varying measuresI To develop a more deeply-tailored Adaptive InterventionI Think time-varying effect moderators
Almirall, Xu, Nahum-Shani, Collins, Murphy Getting SMART 18 / 58
![Page 19: Getting SMART About Adaptive Interventions](https://reader030.fdocuments.us/reader030/viewer/2022012708/61a8a7919a3cd449d4355198/html5/thumbnails/19.jpg)
Concrete Example of a SMART: Child ADHDPI: William Pelham, PhD, Florida International University, IES-Funded GrantN = 153, 8 month study, Monthly non-response (ITB < 75% and IRS > 1 domain)
![Page 20: Getting SMART About Adaptive Interventions](https://reader030.fdocuments.us/reader030/viewer/2022012708/61a8a7919a3cd449d4355198/html5/thumbnails/20.jpg)
Adaptive InterventionsSequential Multiple Assignment Randomized Trial (SMART)
SMART Design PrinciplesTake Home Points
Adaptive Implementation Interventions
Keep it SimpleChoosing Primary and Secondary Hypotheses
SMART Design: Primary and Secondary Aims
Choose a simple primary aim/question that aids developmentof an adaptive intervention. Sample size is chosen based onthe statistical power to address this aim.
Choose secondary aims/questions that further develop theAdaptive Intervention and take advantage of sequentialrandomization to eliminate confounding.
Almirall, Xu, Nahum-Shani, Collins, Murphy Getting SMART 20 / 58
![Page 21: Getting SMART About Adaptive Interventions](https://reader030.fdocuments.us/reader030/viewer/2022012708/61a8a7919a3cd449d4355198/html5/thumbnails/21.jpg)
Example Aim 1: Primary AimWhat is the effect of starting with BMOD vs MED on longitudinal outcomes?
PowerES N0.8 340.5 830.2 505ρ = 0.60α = 0.05β = 0.20
![Page 22: Getting SMART About Adaptive Interventions](https://reader030.fdocuments.us/reader030/viewer/2022012708/61a8a7919a3cd449d4355198/html5/thumbnails/22.jpg)
Example Aim 2: Primary or Secondary AimAmong non-responders, is it better to INTENSIFY vs AUGMENT?
![Page 23: Getting SMART About Adaptive Interventions](https://reader030.fdocuments.us/reader030/viewer/2022012708/61a8a7919a3cd449d4355198/html5/thumbnails/23.jpg)
Example Aim 3: Primary or Secondary AimIs there a difference between two of the embedded adaptive interventions?This could also be a Primary Aim.
Sample size calculators exist for this; see Oetting, Levy, Weiss,and Murphy 2011. Zhiguo Li at Duke. Kelley Kidwell at UMich.
![Page 24: Getting SMART About Adaptive Interventions](https://reader030.fdocuments.us/reader030/viewer/2022012708/61a8a7919a3cd449d4355198/html5/thumbnails/24.jpg)
Example Aim 4: Secondary or Tertiary AimBuild a more deeply tailored adaptive intervention (go beyond the 4 embedded adaptiveinterventions). Rarely, would this be a Primary Aim.
![Page 25: Getting SMART About Adaptive Interventions](https://reader030.fdocuments.us/reader030/viewer/2022012708/61a8a7919a3cd449d4355198/html5/thumbnails/25.jpg)
Adaptive InterventionsSequential Multiple Assignment Randomized Trial (SMART)
SMART Design PrinciplesTake Home Points
Adaptive Implementation Interventions
TAKE HOME POINTS
Almirall, Xu, Nahum-Shani, Collins, Murphy Getting SMART 25 / 58
![Page 26: Getting SMART About Adaptive Interventions](https://reader030.fdocuments.us/reader030/viewer/2022012708/61a8a7919a3cd449d4355198/html5/thumbnails/26.jpg)
Adaptive InterventionsSequential Multiple Assignment Randomized Trial (SMART)
SMART Design PrinciplesTake Home Points
Adaptive Implementation Interventions
Take Home the Following
I Adaptive Interventions individualize treatment up-front andthroughout; they are guides for clinical practice
I SMARTs are used to build better Adaptive InterventionsI Next study: RCT of SMART-optimized AI vs control
I SMARTs are not adaptive trial designs (confusing!)
I SMARTs do not have to be complicated
I SMARTs do not necessarily require larger sample sizes
Almirall, Xu, Nahum-Shani, Collins, Murphy Getting SMART 26 / 58
![Page 27: Getting SMART About Adaptive Interventions](https://reader030.fdocuments.us/reader030/viewer/2022012708/61a8a7919a3cd449d4355198/html5/thumbnails/27.jpg)
Adaptive InterventionsSequential Multiple Assignment Randomized Trial (SMART)
SMART Design PrinciplesTake Home Points
Adaptive Implementation Interventions
The Case of Adaptive Implementation Interventions
Almirall, Xu, Nahum-Shani, Collins, Murphy Getting SMART 27 / 58
![Page 28: Getting SMART About Adaptive Interventions](https://reader030.fdocuments.us/reader030/viewer/2022012708/61a8a7919a3cd449d4355198/html5/thumbnails/28.jpg)
Adaptive InterventionsSequential Multiple Assignment Randomized Trial (SMART)
SMART Design PrinciplesTake Home Points
Adaptive Implementation Interventions
Adaptive Implementation Interventions
I Evidence-based interventions sit on the “academic shelf”.
I Organizational factors (e.g., training, culture, or climate)that limit (or slow) their uptake.
I Often a problem in under-resourced, community-basedhealth centers (e.g., some HIV testing clinics)
I Organizations are also quite heterogeneous—somemay “suffer relapses” in progress toward uptake, not all willrequire the same level of intervention—possibly requiringan adaptive implementation intervention approach.
Almirall, Xu, Nahum-Shani, Collins, Murphy Getting SMART 28 / 58
![Page 29: Getting SMART About Adaptive Interventions](https://reader030.fdocuments.us/reader030/viewer/2022012708/61a8a7919a3cd449d4355198/html5/thumbnails/29.jpg)
Adaptive Implementation Intervention in Mental HealthPI: Kilbourne; Co-I: AlmirallImprove the uptake of a psychosocial intervention for mood disorders in the community
![Page 30: Getting SMART About Adaptive Interventions](https://reader030.fdocuments.us/reader030/viewer/2022012708/61a8a7919a3cd449d4355198/html5/thumbnails/30.jpg)
Adaptive InterventionsSequential Multiple Assignment Randomized Trial (SMART)
SMART Design PrinciplesTake Home Points
Adaptive Implementation Interventions
Thank you! Questions?
Email me with questions (Daniel Almirall):I Email: [email protected] Web: http://www-personal.umich.edu/∼dalmiral/
Find more papers about SMART:I http://www.lsa.stat.umich.edu/∼samurphy/ (Susan Murphy)I http://www-personal.umich.edu/ inbal (Inbal Nahum-Shani)
Almirall, Xu, Nahum-Shani, Collins, Murphy Getting SMART 30 / 58
![Page 31: Getting SMART About Adaptive Interventions](https://reader030.fdocuments.us/reader030/viewer/2022012708/61a8a7919a3cd449d4355198/html5/thumbnails/31.jpg)
Adaptive InterventionsSequential Multiple Assignment Randomized Trial (SMART)
SMART Design PrinciplesTake Home Points
Adaptive Implementation Interventions
EXTRA SLIDES[beyond this point, slides may not be coherent]
Almirall, Xu, Nahum-Shani, Collins, Murphy Getting SMART 31 / 58
![Page 32: Getting SMART About Adaptive Interventions](https://reader030.fdocuments.us/reader030/viewer/2022012708/61a8a7919a3cd449d4355198/html5/thumbnails/32.jpg)
Adaptive InterventionsSequential Multiple Assignment Randomized Trial (SMART)
SMART Design PrinciplesTake Home Points
Adaptive Implementation Interventions
GENERATING HYPOTHESES vs BUILDING vs EVALUATING
ADAPTIVE INTERVENTIONS?
Almirall, Xu, Nahum-Shani, Collins, Murphy Getting SMART 32 / 58
![Page 33: Getting SMART About Adaptive Interventions](https://reader030.fdocuments.us/reader030/viewer/2022012708/61a8a7919a3cd449d4355198/html5/thumbnails/33.jpg)
Adaptive InterventionsSequential Multiple Assignment Randomized Trial (SMART)
SMART Design PrinciplesTake Home Points
Adaptive Implementation Interventions
3 Different Research Questions/Aims⇒ 3 Different Research Designs
I Aim Type 1: When generating hypotheses about anAdaptive Intervention
I Aim Type 2: When building an Adaptive Intervention
I Aim Type 3: When evaluating a particular AdaptiveIntervention
Almirall, Xu, Nahum-Shani, Collins, Murphy Getting SMART 33 / 58
![Page 34: Getting SMART About Adaptive Interventions](https://reader030.fdocuments.us/reader030/viewer/2022012708/61a8a7919a3cd449d4355198/html5/thumbnails/34.jpg)
3 Different Research Questions/Aims⇒ 3 Different Research Designs
Some example questions:
I Aim Type 1 (Hypothesis Gen.) Example: Doesaugmenting txt for non-responders (as observed in aprevious trial) correlate with better outcomes?
I Aim Type 2 (Building) Example: What are the besttailoring variables or decision rules?
I Aim Type 3 (Evaluating) Example: Does an adaptiveintervention have a statistically and clinically signif. effectas compared to control intervention?
![Page 35: Getting SMART About Adaptive Interventions](https://reader030.fdocuments.us/reader030/viewer/2022012708/61a8a7919a3cd449d4355198/html5/thumbnails/35.jpg)
3 Different Research Questions/Aims⇒ 3 Different Research Designs
I Aim Type 1: When generating hypotheses about anAdaptive Intervention
I Aim Type 2: When building an Adaptive Intervention
I Aim Type 3: When evaluating a particular AdaptiveIntervention
Obs. Study Exp. Studye.g., analysis of e.g., e.g.,
Type Aim previous RCT SMART RCT1 Hypothesis Gen. YES ≈ ∼2 Building ≈ YES ≈3 Evaluating ∼ ≈ YES
![Page 36: Getting SMART About Adaptive Interventions](https://reader030.fdocuments.us/reader030/viewer/2022012708/61a8a7919a3cd449d4355198/html5/thumbnails/36.jpg)
Adaptive InterventionsSequential Multiple Assignment Randomized Trial (SMART)
SMART Design PrinciplesTake Home Points
Adaptive Implementation Interventions
A (potentially not-so) SMART Alternative
Almirall, Xu, Nahum-Shani, Collins, Murphy Getting SMART 36 / 58
![Page 37: Getting SMART About Adaptive Interventions](https://reader030.fdocuments.us/reader030/viewer/2022012708/61a8a7919a3cd449d4355198/html5/thumbnails/37.jpg)
Back to the idea of generating hypotheses vs buildingvs evaluating adaptive interventions...
Obs. Study Exp. Studye.g., analysis of e.g., e.g.,
Type Aim previous RCT SMART RCT1 Hypothesis Gen. YES ≈ ∼2 Building ≈ YES ≈3 Evaluating ∼ ≈ YES
Why not use multiple RCTs to build an adaptive intervention?
![Page 38: Getting SMART About Adaptive Interventions](https://reader030.fdocuments.us/reader030/viewer/2022012708/61a8a7919a3cd449d4355198/html5/thumbnails/38.jpg)
Concrete Example of a SMART: Child ADHDPI: William Pelham, PhD, Florida International University, IES-Funded GrantN = 153, 8 month study, Monthly non-response (ITB < 75% and IRS > 1 domain)
![Page 39: Getting SMART About Adaptive Interventions](https://reader030.fdocuments.us/reader030/viewer/2022012708/61a8a7919a3cd449d4355198/html5/thumbnails/39.jpg)
Adaptive InterventionsSequential Multiple Assignment Randomized Trial (SMART)
SMART Design PrinciplesTake Home Points
Adaptive Implementation Interventions
Why Not Use Multiple RCTs to Construct an AI?Three Concerns about Using Multiple Trials as an Alternative to a SMART
1. Concern 1: Delayed Therapeutic Effects
2. Concern 2: Diagnostic Effects
3. Concern 3: Cohort (Sample Selection) Effects
All three concerns emanate from the basic idea thatconstructing an adaptive intervention based on a ”local”study-to-study point of view may not be optimal.
Almirall, Xu, Nahum-Shani, Collins, Murphy Getting SMART 39 / 58
![Page 40: Getting SMART About Adaptive Interventions](https://reader030.fdocuments.us/reader030/viewer/2022012708/61a8a7919a3cd449d4355198/html5/thumbnails/40.jpg)
Adaptive InterventionsSequential Multiple Assignment Randomized Trial (SMART)
SMART Design PrinciplesTake Home Points
Adaptive Implementation Interventions
Hypothesis-generating Observational StudiesPost-hoc Analyses Useful for Building Adaptive Interventions
I Give examples of different observational study questionsthey can examine using data from a previous 2-arm RCT
I Standard observational study caveats apply:I No manipulation usually means lack of heterogeneity in txt
options (beyond what is controlled by experimentation inoriginal RCT)
I Some RCTs use samples that are too homogeneousI Confounding by observed baseline and time-varying factorsI Unobserved, unknown, unmeasured confounding by
baseline and time-varying factors
Almirall, Xu, Nahum-Shani, Collins, Murphy Getting SMART 40 / 58
![Page 41: Getting SMART About Adaptive Interventions](https://reader030.fdocuments.us/reader030/viewer/2022012708/61a8a7919a3cd449d4355198/html5/thumbnails/41.jpg)
Adaptive InterventionsSequential Multiple Assignment Randomized Trial (SMART)
SMART Design PrinciplesTake Home Points
Adaptive Implementation Interventions
Hypothesis-generating Observational StudiesPost-hoc Analyses Useful for Building Adaptive Interventions
I There exists a literature for examining the impact oftime-varying treatments in observational studies
I Marginal Structural Models (Robins, 1999; Bray, Almirall, etal., 2006) to examine the marginal impact of observedtime-varying sequences of treatment
I Structural Nested Mean Models (Robins, 1994; Almirall, etal., 2010, 2011) to examine time-varying moderators ofobserved time-varying sequences of treatment
I Marginal Mean Models (Murphy, et al., 2001): to examinethe impact of observed adaptive interventions
Almirall, Xu, Nahum-Shani, Collins, Murphy Getting SMART 41 / 58
![Page 42: Getting SMART About Adaptive Interventions](https://reader030.fdocuments.us/reader030/viewer/2022012708/61a8a7919a3cd449d4355198/html5/thumbnails/42.jpg)
Adaptive InterventionsSequential Multiple Assignment Randomized Trial (SMART)
SMART Design PrinciplesTake Home Points
Adaptive Implementation Interventions
Early precursors to SMART
I CATIE (2001) Treatment of Psychosis in Patients withAlzheimer’s
I CATIE (2001) Treatment of Psychosis in Patients withSchizophrenia
I STAR*D (2003) Treatment of Depression
Almirall, Xu, Nahum-Shani, Collins, Murphy Getting SMART 42 / 58
![Page 43: Getting SMART About Adaptive Interventions](https://reader030.fdocuments.us/reader030/viewer/2022012708/61a8a7919a3cd449d4355198/html5/thumbnails/43.jpg)
Adaptive InterventionsSequential Multiple Assignment Randomized Trial (SMART)
SMART Design PrinciplesTake Home Points
Adaptive Implementation Interventions
Other Alternatives
I Piecing Together Results from Multiple TrialsI Choose best first-line treatment on the basis of a two-arm
RCT; then choose best second-line treatment on the basisof another separate, two-arm RCT
I Concerns: delayed therapeutic effects, and cohort effects
I Observational (Non-experimental) Comparisons of AIsI Using data from longitudinal randomized trialsI May yield results that inform a SMART proposalI Understand current treatment sequencing practicesI Typical problems associated with observational studies
I Expert Opinion
Almirall, Xu, Nahum-Shani, Collins, Murphy Getting SMART 43 / 58
![Page 44: Getting SMART About Adaptive Interventions](https://reader030.fdocuments.us/reader030/viewer/2022012708/61a8a7919a3cd449d4355198/html5/thumbnails/44.jpg)
Adaptive InterventionsSequential Multiple Assignment Randomized Trial (SMART)
SMART Design PrinciplesTake Home Points
Adaptive Implementation Interventions
Why Not Use Multiple Trials to Construct an AIThree Concerns about Using Multiple Trials as an Alternative to a SMART
1. Concern 1: Delayed Therapeutic Effects
2. Concern 2: Diagnostic Effects
3. Concern 3: Cohort Effects
All three concerns emanate from the basic idea thatconstructing an adaptive intervention based on a myopic, local,study-to-study point of view may not be optimal.
Almirall, Xu, Nahum-Shani, Collins, Murphy Getting SMART 44 / 58
![Page 45: Getting SMART About Adaptive Interventions](https://reader030.fdocuments.us/reader030/viewer/2022012708/61a8a7919a3cd449d4355198/html5/thumbnails/45.jpg)
Adaptive InterventionsSequential Multiple Assignment Randomized Trial (SMART)
SMART Design PrinciplesTake Home Points
Adaptive Implementation Interventions
Why Not Use Multiple Trials to Construct an AIConcern 1: Delayed Therapeutic Effects, or Sequential Treatment Interactions
Positive Synergy Btwn First- and Second-line Treatments
Tapering off medication after 12 weeks of use may not appearbest initially, but may have enhanced long term effectivenesswhen followed by a particular augmentation, switch, ormaintenance strategy.
Tapering off medication after 12 weeks may set the child up forbetter success with any one of the second-line treatments.
Almirall, Xu, Nahum-Shani, Collins, Murphy Getting SMART 45 / 58
![Page 46: Getting SMART About Adaptive Interventions](https://reader030.fdocuments.us/reader030/viewer/2022012708/61a8a7919a3cd449d4355198/html5/thumbnails/46.jpg)
Adaptive InterventionsSequential Multiple Assignment Randomized Trial (SMART)
SMART Design PrinciplesTake Home Points
Adaptive Implementation Interventions
Why Not Use Multiple Trials to Construct an AIConcern 1: Delayed Therapeutic Effects, or Sequential Treatment Interactions
Negative Synergy Btwn First- and Second-line Treatments
Keeping the child on medication an additional 12 weeks mayproduce a higher proportion of responders at first, but may alsoresult in side effects that reduce the variety of subsequenttreatments available if s/he relapses.
The burden associated with continuing medication an additional12 weeks may be so high that non-responders will not adhereto second-line treatments.
Almirall, Xu, Nahum-Shani, Collins, Murphy Getting SMART 46 / 58
![Page 47: Getting SMART About Adaptive Interventions](https://reader030.fdocuments.us/reader030/viewer/2022012708/61a8a7919a3cd449d4355198/html5/thumbnails/47.jpg)
Adaptive InterventionsSequential Multiple Assignment Randomized Trial (SMART)
SMART Design PrinciplesTake Home Points
Adaptive Implementation Interventions
Why Not Use Multiple Trials to Construct an AIConcern 2: Diagnostic Effects
Tapering off medication after 12 weeks initial use may notproduce a higher proportion of responders at first, but may elicitsymptoms that allow you to better match subsequent treatmentto the child.
The improved matching (personalizing) on subsequenttreatments may result in a better response overall as comparedto any sequence of treatments that offered an additional 12weeks of medication after the initial 12 weeks.
Almirall, Xu, Nahum-Shani, Collins, Murphy Getting SMART 47 / 58
![Page 48: Getting SMART About Adaptive Interventions](https://reader030.fdocuments.us/reader030/viewer/2022012708/61a8a7919a3cd449d4355198/html5/thumbnails/48.jpg)
Adaptive InterventionsSequential Multiple Assignment Randomized Trial (SMART)
SMART Design PrinciplesTake Home Points
Adaptive Implementation Interventions
Why Not Use Multiple Trials to Construct an AIConcern 3: Cohort Effects
I Children enrolled in the initial and secondary trials may bedifferent.
I Children who remain in the trial(s) may be different.I Characteristics of adherent children may differ from study
to study.I Children that know they are undergoing adaptive
interventions may have different adherence patterns.
Bottom line: The population of children we are makinginferences about may simply be different from study-to-study.
Almirall, Xu, Nahum-Shani, Collins, Murphy Getting SMART 48 / 58
![Page 49: Getting SMART About Adaptive Interventions](https://reader030.fdocuments.us/reader030/viewer/2022012708/61a8a7919a3cd449d4355198/html5/thumbnails/49.jpg)
Adaptive InterventionsSequential Multiple Assignment Randomized Trial (SMART)
SMART Design PrinciplesTake Home Points
Adaptive Implementation Interventions
SMART CASE STUDIES
Almirall, Xu, Nahum-Shani, Collins, Murphy Getting SMART 49 / 58
![Page 50: Getting SMART About Adaptive Interventions](https://reader030.fdocuments.us/reader030/viewer/2022012708/61a8a7919a3cd449d4355198/html5/thumbnails/50.jpg)
Autism SMART (N = 61)PI: Kasari (UCLA). (ages 5-8; planned N = 98 but recruitment difficult, despitemulti-site. Wk12 response rates much higher than anticipated.)
![Page 51: Getting SMART About Adaptive Interventions](https://reader030.fdocuments.us/reader030/viewer/2022012708/61a8a7919a3cd449d4355198/html5/thumbnails/51.jpg)
Longitudinal Analysis of the Autism SMARTYt = Socially communicative utterances over 36 weeks
AI Estimate 95% CI(AAC,AAC+) 51.4 [45.6, 57.3](JASP,AAC) 40.7 [34.5, 46.8]
(JASP,JASP+) 39.3 [32.6, 46.0]
![Page 52: Getting SMART About Adaptive Interventions](https://reader030.fdocuments.us/reader030/viewer/2022012708/61a8a7919a3cd449d4355198/html5/thumbnails/52.jpg)
Moderators Analysis of the Autism SMART
2040
6080
100
Baseline SCU < 40 (75%)
Time (weeks)
Soc
ially
Com
mun
icat
ive
Utte
ranc
es
0 12 24 36
(AAC, AAC+)(JASP, AAC)(JASP, JASP+)
2040
6080
100
Baseline SCU ≥ 40 (25%)
Time (weeks)
Soc
ially
Com
mun
icat
ive
Utte
ranc
es
0 12 24 36
(AAC, AAC+)(JASP, AAC)(JASP, JASP+)
![Page 53: Getting SMART About Adaptive Interventions](https://reader030.fdocuments.us/reader030/viewer/2022012708/61a8a7919a3cd449d4355198/html5/thumbnails/53.jpg)
Child ADHD SMARTPI: William Pelham, PhD, Florida International UniversityN = 153, 8 month study, Monthly non-response (ITB < 75% and IRS > 1 domain)
![Page 54: Getting SMART About Adaptive Interventions](https://reader030.fdocuments.us/reader030/viewer/2022012708/61a8a7919a3cd449d4355198/html5/thumbnails/54.jpg)
Longitudinal Analysis of the ADHD SMARTYt = Classroom performance over 8 months (school year)
Time (months)
Cla
ssro
om p
erfo
rman
ce
2.1
2.2
2.3
2.4
2.5
2.6
2.7
1 2 3 4 5 6 7 8
DTRBMOD.INTBMOD.MEDMED.BMODMED.INT
AI Color(MED, MED+) Purple
(MED, MED+BMD) Blue(BMD,BMD+MED) Red
(BMD,BMD+) Green
![Page 55: Getting SMART About Adaptive Interventions](https://reader030.fdocuments.us/reader030/viewer/2022012708/61a8a7919a3cd449d4355198/html5/thumbnails/55.jpg)
Interventions for Minimally Verbal Children with AutismPI: Kasari(UCLA), Kaiser(Vanderbilt), Smith(Rochester), Lord(Cornell), Almirall(Mich)
![Page 56: Getting SMART About Adaptive Interventions](https://reader030.fdocuments.us/reader030/viewer/2022012708/61a8a7919a3cd449d4355198/html5/thumbnails/56.jpg)
Interventions for Minimally Verbal Children with AutismPI: Kasari(UCLA), Kaiser(Vanderbilt), Smith(Rochester), Lord(Cornell), Almirall(Mich)
Non-Responders
(Parent training no
feasible) JASP (joint
attention and social play) Continue JASP
JASP + Parent Training R
DTT (discrete trials training)
Continue DTT
DTT + Parent Training
Responders
(Blended txt
unnecessary)
R
Non-Responders
(Parent training not
feasible)
Responders
(Blended txt
unnecessary)
R
JASP + DTT
Continue JASP
R
JASP + DTT
Continue DTT
R
![Page 57: Getting SMART About Adaptive Interventions](https://reader030.fdocuments.us/reader030/viewer/2022012708/61a8a7919a3cd449d4355198/html5/thumbnails/57.jpg)
Adaptive Implementation Intervention in Mental HealthPI: Kilbourne; Co-I: Almirall (Aim is to improve the uptake of a psychosocial interventionfor mood disorders)
![Page 58: Getting SMART About Adaptive Interventions](https://reader030.fdocuments.us/reader030/viewer/2022012708/61a8a7919a3cd449d4355198/html5/thumbnails/58.jpg)
Treatment for Alcohol DependencePI: Oslin, University of Pennsylvania
Early Trigger for NR: 2+ HDD CBI
CBI + Naltrexone
R
Late Trigger for NR: 5+ HDD
CBI
CBI + Naltrexone
Non-Response R
Non-Response R
Naltrexone
TDM + Naltrexone
8 Week Response R
Naltrexone
TDM + Naltrexone
8 Week Response R