UNPACKING THE BLACK BOX ENGINEERING MORE POTENT BEHAVIORAL INTERVENTIONS TO IMPROVE PUBLIC ...
description
Transcript of UNPACKING THE BLACK BOX ENGINEERING MORE POTENT BEHAVIORAL INTERVENTIONS TO IMPROVE PUBLIC ...
![Page 1: UNPACKING THE BLACK BOX ENGINEERING MORE POTENT BEHAVIORAL INTERVENTIONS TO IMPROVE PUBLIC HEALTH](https://reader031.fdocuments.us/reader031/viewer/2022013012/56816957550346895de1055a/html5/thumbnails/1.jpg)
UNPACKING THE BLACK BOXENGINEERING MORE POTENT BEHAVIORALINTERVENTIONS TO IMPROVE PUBLIC HEALTH
Linda M. Collins, Ph.D.
![Page 2: UNPACKING THE BLACK BOX ENGINEERING MORE POTENT BEHAVIORAL INTERVENTIONS TO IMPROVE PUBLIC HEALTH](https://reader031.fdocuments.us/reader031/viewer/2022013012/56816957550346895de1055a/html5/thumbnails/2.jpg)
Outline What is a behavioral intervention? What is the black box, and why unpack it? What is optimization? A few examples Concluding remarks
![Page 3: UNPACKING THE BLACK BOX ENGINEERING MORE POTENT BEHAVIORAL INTERVENTIONS TO IMPROVE PUBLIC HEALTH](https://reader031.fdocuments.us/reader031/viewer/2022013012/56816957550346895de1055a/html5/thumbnails/3.jpg)
What is a behavioral intervention?
Definition: A program aimed at modifying behavior for the purpose of preventing/treating disease, promoting health, and/or enhancing well-being.
![Page 4: UNPACKING THE BLACK BOX ENGINEERING MORE POTENT BEHAVIORAL INTERVENTIONS TO IMPROVE PUBLIC HEALTH](https://reader031.fdocuments.us/reader031/viewer/2022013012/56816957550346895de1055a/html5/thumbnails/4.jpg)
What is a behavioral intervention?
A few examples from PSU (out of many):
PROSPER (Greenberg, Bierman, Feinberg, Welsh, Perkins, Mincemoyer, Corbin)
Keepin’ it REAL (Hecht, Miller-Day, Graham)
Project ACT (Turrisi)
Active MOMS (Downs)
Interventions for at-risk caregivers (Zarit)
Head Start REDI (Bierman, Domitrovich)
HealthWise (Caldwell, Smith)
RESERVE (Kolanowski, Fick)
![Page 5: UNPACKING THE BLACK BOX ENGINEERING MORE POTENT BEHAVIORAL INTERVENTIONS TO IMPROVE PUBLIC HEALTH](https://reader031.fdocuments.us/reader031/viewer/2022013012/56816957550346895de1055a/html5/thumbnails/5.jpg)
What is a behavioral intervention?
Most behavioral interventions are made up of multiple components.
Some components may be pharmaceutical or medical.
![Page 6: UNPACKING THE BLACK BOX ENGINEERING MORE POTENT BEHAVIORAL INTERVENTIONS TO IMPROVE PUBLIC HEALTH](https://reader031.fdocuments.us/reader031/viewer/2022013012/56816957550346895de1055a/html5/thumbnails/6.jpg)
Outline What is a behavioral intervention? What is the black box, and why unpack it? What is optimization? A few examples Concluding remarks
![Page 7: UNPACKING THE BLACK BOX ENGINEERING MORE POTENT BEHAVIORAL INTERVENTIONS TO IMPROVE PUBLIC HEALTH](https://reader031.fdocuments.us/reader031/viewer/2022013012/56816957550346895de1055a/html5/thumbnails/7.jpg)
Treatment package approach
component
component
component
RCT
component component
![Page 8: UNPACKING THE BLACK BOX ENGINEERING MORE POTENT BEHAVIORAL INTERVENTIONS TO IMPROVE PUBLIC HEALTH](https://reader031.fdocuments.us/reader031/viewer/2022013012/56816957550346895de1055a/html5/thumbnails/8.jpg)
What is the black box, and why unpack it?
If RCT finds significant effect, it is UNKNOWN Which components are making positive
contribution to overall effect Whether all the components are really needed Whether a component’s contribution offsets its
cost How to make the intervention more effective,
efficient, cost-effective
![Page 9: UNPACKING THE BLACK BOX ENGINEERING MORE POTENT BEHAVIORAL INTERVENTIONS TO IMPROVE PUBLIC HEALTH](https://reader031.fdocuments.us/reader031/viewer/2022013012/56816957550346895de1055a/html5/thumbnails/9.jpg)
What is the black box, and why unpack it?
If RCT finds non-significant effect, it is UNKNOWN Whether any components in the box are worth
retaining
Whether one component in the box had a negative effect that offset the positive effect of others
Specifically what went wrong and how to do it better the next time
![Page 10: UNPACKING THE BLACK BOX ENGINEERING MORE POTENT BEHAVIORAL INTERVENTIONS TO IMPROVE PUBLIC HEALTH](https://reader031.fdocuments.us/reader031/viewer/2022013012/56816957550346895de1055a/html5/thumbnails/10.jpg)
What is the black box, and why unpack it?
The treatment package approach Encourages stuffing the black box with as
many components as possible to get a significant effect
Downplays considerations such as efficiency, cost-effectiveness, time-effectiveness
Places focus on attaining statistical significance rather than meeting a criterion
![Page 11: UNPACKING THE BLACK BOX ENGINEERING MORE POTENT BEHAVIORAL INTERVENTIONS TO IMPROVE PUBLIC HEALTH](https://reader031.fdocuments.us/reader031/viewer/2022013012/56816957550346895de1055a/html5/thumbnails/11.jpg)
What is the black box, and why unpack it?
This is NOT how engineers build products. They take an approach that is
► Systematic
► Efficient
► Focused on the clear objective of optimizing the product
Can we borrow ideas from engineering… … and build optimized behavioral interventions?
![Page 12: UNPACKING THE BLACK BOX ENGINEERING MORE POTENT BEHAVIORAL INTERVENTIONS TO IMPROVE PUBLIC HEALTH](https://reader031.fdocuments.us/reader031/viewer/2022013012/56816957550346895de1055a/html5/thumbnails/12.jpg)
Resource management principle How engineers think: This is what I need to find out: ______ These are the resources I have: ______ How can I manage my resources strategically
to find out what I need to know?
![Page 13: UNPACKING THE BLACK BOX ENGINEERING MORE POTENT BEHAVIORAL INTERVENTIONS TO IMPROVE PUBLIC HEALTH](https://reader031.fdocuments.us/reader031/viewer/2022013012/56816957550346895de1055a/html5/thumbnails/13.jpg)
Resource management principle Logic: Objective is to identify ONE OF THE
TWO OR THREE BEST approaches
![Page 14: UNPACKING THE BLACK BOX ENGINEERING MORE POTENT BEHAVIORAL INTERVENTIONS TO IMPROVE PUBLIC HEALTH](https://reader031.fdocuments.us/reader031/viewer/2022013012/56816957550346895de1055a/html5/thumbnails/14.jpg)
0
1
2
3
4
5
6
7
8
9
10
Hypothetical distribution of outcome variable for different combinations of components
Score on outcome variable
Freq
uenc
y
![Page 15: UNPACKING THE BLACK BOX ENGINEERING MORE POTENT BEHAVIORAL INTERVENTIONS TO IMPROVE PUBLIC HEALTH](https://reader031.fdocuments.us/reader031/viewer/2022013012/56816957550346895de1055a/html5/thumbnails/15.jpg)
Resource management principle Logic: Objective is to identify ONE OF THE
TWO OR THREE BEST approaches Manage research resources strategically
► Decide what information most important, and target resources there
Choose efficient experimental designs Take calculated risks
![Page 16: UNPACKING THE BLACK BOX ENGINEERING MORE POTENT BEHAVIORAL INTERVENTIONS TO IMPROVE PUBLIC HEALTH](https://reader031.fdocuments.us/reader031/viewer/2022013012/56816957550346895de1055a/html5/thumbnails/16.jpg)
Resource management principle Note that the starting point is the resources
you have By definition, MOST does not require an
increase in research resources► But may require a realignment of research
resources
![Page 17: UNPACKING THE BLACK BOX ENGINEERING MORE POTENT BEHAVIORAL INTERVENTIONS TO IMPROVE PUBLIC HEALTH](https://reader031.fdocuments.us/reader031/viewer/2022013012/56816957550346895de1055a/html5/thumbnails/17.jpg)
The Multiphase Optimization Strategy (MOST)
component
component
component
Optimizedintervention
component
component
component
RCT
component component
Screening (component
selection) and refinement
experiment(s)
Note: MOST is a framework, not an off-the-shelf procedure.
![Page 18: UNPACKING THE BLACK BOX ENGINEERING MORE POTENT BEHAVIORAL INTERVENTIONS TO IMPROVE PUBLIC HEALTH](https://reader031.fdocuments.us/reader031/viewer/2022013012/56816957550346895de1055a/html5/thumbnails/18.jpg)
Outline What is a behavioral intervention? What is the black box, and why unpack it? What is optimization? A few examples Concluding remarks
![Page 19: UNPACKING THE BLACK BOX ENGINEERING MORE POTENT BEHAVIORAL INTERVENTIONS TO IMPROVE PUBLIC HEALTH](https://reader031.fdocuments.us/reader031/viewer/2022013012/56816957550346895de1055a/html5/thumbnails/19.jpg)
Definition of optimized “The best possible solution… subject to given
constraints” [emphasis added] (The Concise Oxford Dictionary of Mathematics)
► Optimized does not mean best in an absolute or ideal sense
► Instead, realistic because it includes constraints
Optimization always involves a clearly stated optimization criterion
► A working definition of what YOU mean by “better”
![Page 20: UNPACKING THE BLACK BOX ENGINEERING MORE POTENT BEHAVIORAL INTERVENTIONS TO IMPROVE PUBLIC HEALTH](https://reader031.fdocuments.us/reader031/viewer/2022013012/56816957550346895de1055a/html5/thumbnails/20.jpg)
Selecting an optimization criterion: what you mean by “best”
Your definition of “best possible, given constraints”
This is the goal you want to achieve Suppose you are developing a behavioral
intervention to encourage HAART adherence in HIV+ people called “Living with HIV”
![Page 21: UNPACKING THE BLACK BOX ENGINEERING MORE POTENT BEHAVIORAL INTERVENTIONS TO IMPROVE PUBLIC HEALTH](https://reader031.fdocuments.us/reader031/viewer/2022013012/56816957550346895de1055a/html5/thumbnails/21.jpg)
One possible optimization criterion: Intervention with no “dead wood”
► Example: Health care settings are finding it difficult to fit Living with HIV into their busy day, and are watching costs carefully. The investigators want to be confident that every component is necessary so that no time or money is wasted.
► Achieve this by selecting only active intervention components.
![Page 22: UNPACKING THE BLACK BOX ENGINEERING MORE POTENT BEHAVIORAL INTERVENTIONS TO IMPROVE PUBLIC HEALTH](https://reader031.fdocuments.us/reader031/viewer/2022013012/56816957550346895de1055a/html5/thumbnails/22.jpg)
Another possible optimization criterion:
Most effective intervention that can be delivered for ≤ some $$
► Example: To have a realistic chance of being adopted by HMOs, Living with HIV must cost no more than $200/participant to deliver, including materials and staff time.
► Achieve this by selecting set of components that represents the most effective intervention that can be delivered for ≤ $200.
![Page 23: UNPACKING THE BLACK BOX ENGINEERING MORE POTENT BEHAVIORAL INTERVENTIONS TO IMPROVE PUBLIC HEALTH](https://reader031.fdocuments.us/reader031/viewer/2022013012/56816957550346895de1055a/html5/thumbnails/23.jpg)
Another possible optimization criterion:
Most effective intervention that can be delivered in ≤ some amount of time
► Example: Interviews with health care clinic staff suggest that Living with HIV has the best chance of being implemented well if it takes no more than 15 minutes to deliver.
► Achieve this by selecting set of components that represents the most effective intervention that can be delivered in ≤ 15 minutes.
![Page 24: UNPACKING THE BLACK BOX ENGINEERING MORE POTENT BEHAVIORAL INTERVENTIONS TO IMPROVE PUBLIC HEALTH](https://reader031.fdocuments.us/reader031/viewer/2022013012/56816957550346895de1055a/html5/thumbnails/24.jpg)
Evaluation and optimization:Both important;
not the same thing.
Evaluation: Is the intervention’s effect
statistically significant?Optimization: Is the intervention the best possible, given constraints?
No Yes
NoMay wish to
optimize using effect size as
criterion
Intervention can probably be
improved
Yes Different intervention
strategy neededWhat we should
be aiming for
![Page 25: UNPACKING THE BLACK BOX ENGINEERING MORE POTENT BEHAVIORAL INTERVENTIONS TO IMPROVE PUBLIC HEALTH](https://reader031.fdocuments.us/reader031/viewer/2022013012/56816957550346895de1055a/html5/thumbnails/25.jpg)
Outline What is a behavioral intervention? What is the black box, and why unpack it? What is optimization? A few examples Concluding remarks
![Page 26: UNPACKING THE BLACK BOX ENGINEERING MORE POTENT BEHAVIORAL INTERVENTIONS TO IMPROVE PUBLIC HEALTH](https://reader031.fdocuments.us/reader031/viewer/2022013012/56816957550346895de1055a/html5/thumbnails/26.jpg)
Three examples of MOST All currently in the field All three have the same optimization criterion:
No inactive components
![Page 27: UNPACKING THE BLACK BOX ENGINEERING MORE POTENT BEHAVIORAL INTERVENTIONS TO IMPROVE PUBLIC HEALTH](https://reader031.fdocuments.us/reader031/viewer/2022013012/56816957550346895de1055a/html5/thumbnails/27.jpg)
Example 1: Clinic-based smoking cessation study funded by NCI
Tim Baker Mike Fiore University of Wisconsin
Team also includes B.A. Christiansen, L.M. Collins, J.W. Cook, D.E. Jorenby, R.J. Mermelstein, M.E. Piper, T.R. Schlam, S.S. Smith
Project funded by NCI grant P50CA143188
![Page 28: UNPACKING THE BLACK BOX ENGINEERING MORE POTENT BEHAVIORAL INTERVENTIONS TO IMPROVE PUBLIC HEALTH](https://reader031.fdocuments.us/reader031/viewer/2022013012/56816957550346895de1055a/html5/thumbnails/28.jpg)
Example 1: Clinic-based smoking cessation study funded by NCI
Tim Baker Mike Fiore University of Wisconsin
Objective: Develop an effective “lean” clinic-based smoking cessation intervention (no inactive
components)
![Page 29: UNPACKING THE BLACK BOX ENGINEERING MORE POTENT BEHAVIORAL INTERVENTIONS TO IMPROVE PUBLIC HEALTH](https://reader031.fdocuments.us/reader031/viewer/2022013012/56816957550346895de1055a/html5/thumbnails/29.jpg)
Some interesting features of Example 1
Study being implemented in health care settings
Involves both behavioral and pharmaceutical components
![Page 30: UNPACKING THE BLACK BOX ENGINEERING MORE POTENT BEHAVIORAL INTERVENTIONS TO IMPROVE PUBLIC HEALTH](https://reader031.fdocuments.us/reader031/viewer/2022013012/56816957550346895de1055a/html5/thumbnails/30.jpg)
Baker & Fiore’s model of the smoking cessation process
PRECESSATION(3 weeks prior up to
quit day)
CESSATION(quit day to 2 weeks
after quit day)
MAINTENANCE(2 weeks to 6 months
after quit day)MOTIVATION
![Page 31: UNPACKING THE BLACK BOX ENGINEERING MORE POTENT BEHAVIORAL INTERVENTIONS TO IMPROVE PUBLIC HEALTH](https://reader031.fdocuments.us/reader031/viewer/2022013012/56816957550346895de1055a/html5/thumbnails/31.jpg)
Six components being considered for the smoking cessation intervention
Precessation nicotine patch (No, Yes) Precessation ad lib nicotine gum (No, Yes) Precessation in-person counseling (No, Yes) Cessation in-person counseling (Minimal,
Intensive) Cessation phone counseling (Minimal, Intensive) Maintenance medication duration (Short, Long)
![Page 32: UNPACKING THE BLACK BOX ENGINEERING MORE POTENT BEHAVIORAL INTERVENTIONS TO IMPROVE PUBLIC HEALTH](https://reader031.fdocuments.us/reader031/viewer/2022013012/56816957550346895de1055a/html5/thumbnails/32.jpg)
Experiment to examine individual component effects
We decided to conduct a factorial experiment► Special type called a fractional factorial
N=512 subjects TOTAL provides power ≥ .8
![Page 33: UNPACKING THE BLACK BOX ENGINEERING MORE POTENT BEHAVIORAL INTERVENTIONS TO IMPROVE PUBLIC HEALTH](https://reader031.fdocuments.us/reader031/viewer/2022013012/56816957550346895de1055a/html5/thumbnails/33.jpg)
Engineering the intervention Experiment will give us
► Main effect of each individual intervention component on outcomes► e.g. number days abstinent in 2-wk post-quit period
► Selected interactions between intervention components
This information will be used to select components/component levels
Result: optimized clinic-based smoking cessation intervention
Plan to conduct an RCT to establish statistical significance of effect
![Page 34: UNPACKING THE BLACK BOX ENGINEERING MORE POTENT BEHAVIORAL INTERVENTIONS TO IMPROVE PUBLIC HEALTH](https://reader031.fdocuments.us/reader031/viewer/2022013012/56816957550346895de1055a/html5/thumbnails/34.jpg)
Example 2: School-based drug abuse/HIV prevention study funded by NIDA
Linda L. Caldwell Edward A. SmithPenn State
Team also includes D. Coffman, L. Collins, J. Cox, I. Evans, J. Graham, M. Greenberg, J. Jacobs, D. Jones, M. Lai, C. Matthews, R. Spoth, L. Wegner, T. Vergnani, E. Weybright
Project funded by NIDA grant R01DA029084
![Page 35: UNPACKING THE BLACK BOX ENGINEERING MORE POTENT BEHAVIORAL INTERVENTIONS TO IMPROVE PUBLIC HEALTH](https://reader031.fdocuments.us/reader031/viewer/2022013012/56816957550346895de1055a/html5/thumbnails/35.jpg)
Example 2: School-based drug abuse/HIV prevention study funded by NIDA
Linda L. Caldwell Edward A. SmithPenn State
Objective: To develop a strategy for maintaining implementation fidelity in which all components contribute
![Page 36: UNPACKING THE BLACK BOX ENGINEERING MORE POTENT BEHAVIORAL INTERVENTIONS TO IMPROVE PUBLIC HEALTH](https://reader031.fdocuments.us/reader031/viewer/2022013012/56816957550346895de1055a/html5/thumbnails/36.jpg)
Some interesting features of Example 2
Components being examined relate to how the intervention is delivered
► “sealed intervention”
Cluster randomization
![Page 37: UNPACKING THE BLACK BOX ENGINEERING MORE POTENT BEHAVIORAL INTERVENTIONS TO IMPROVE PUBLIC HEALTH](https://reader031.fdocuments.us/reader031/viewer/2022013012/56816957550346895de1055a/html5/thumbnails/37.jpg)
Background HealthWise school-based ATOD/HIV prevention
intervention Has previously been evaluated in South Africa Metropolitan South Education District in South
Africa wants to implement HealthWise in all its schools
Question: how to maintain fidelity? Metro South allowed us to conduct an experiment
![Page 38: UNPACKING THE BLACK BOX ENGINEERING MORE POTENT BEHAVIORAL INTERVENTIONS TO IMPROVE PUBLIC HEALTH](https://reader031.fdocuments.us/reader031/viewer/2022013012/56816957550346895de1055a/html5/thumbnails/38.jpg)
Caldwell & Smith’s model of implementation fidelity
![Page 39: UNPACKING THE BLACK BOX ENGINEERING MORE POTENT BEHAVIORAL INTERVENTIONS TO IMPROVE PUBLIC HEALTH](https://reader031.fdocuments.us/reader031/viewer/2022013012/56816957550346895de1055a/html5/thumbnails/39.jpg)
Components Enhanced teacher training
► Standard training (one and one-half days) vs. enhanced (three days + two additional days four months later)
Structure, support, and supervision► No additional vs. additional (e.g., weekly text messages; monthly
visits from support staff; option to call support staff with questions as needed)
Enhanced school climate► No climate enhancement vs. climate enhancement (e.g., form
committee of parents and teachers to promote HealthWise; develop visuals; issue newsletter)
![Page 40: UNPACKING THE BLACK BOX ENGINEERING MORE POTENT BEHAVIORAL INTERVENTIONS TO IMPROVE PUBLIC HEALTH](https://reader031.fdocuments.us/reader031/viewer/2022013012/56816957550346895de1055a/html5/thumbnails/40.jpg)
How to conduct an experiment to examine individual component
effects We decided to conduct a factorial experiment. Why? Enables examination of individual component effects
AND Statistical power achieved with smaller sample sizes
than alternative designs► Yes, I mean smaller
BUT they also usually require more experimental conditions than we may be accustomed to
Experiment uses all 56 schools in district
![Page 41: UNPACKING THE BLACK BOX ENGINEERING MORE POTENT BEHAVIORAL INTERVENTIONS TO IMPROVE PUBLIC HEALTH](https://reader031.fdocuments.us/reader031/viewer/2022013012/56816957550346895de1055a/html5/thumbnails/41.jpg)
Factorial experiments 101 Example: 2 X 2, or 22, factorial design
Factorial experiments can have► ≥ 2 factors
► ≥ 2 levels per factor
Component A
Component B Off On
Off A,B off A on, B off
On A off, B on A,B on
![Page 42: UNPACKING THE BLACK BOX ENGINEERING MORE POTENT BEHAVIORAL INTERVENTIONS TO IMPROVE PUBLIC HEALTH](https://reader031.fdocuments.us/reader031/viewer/2022013012/56816957550346895de1055a/html5/thumbnails/42.jpg)
Experi-mental
conditionN of
schoolsHealthWise
program Training
Structure, support, & supervision
Enhanced school climate
1 7 2 7 3 7 4 7 5 7 6 7 7 7 8 7
HealthWise experiment in South African school district. 56 schools in all; 7 schools assigned to each experimental condition
![Page 43: UNPACKING THE BLACK BOX ENGINEERING MORE POTENT BEHAVIORAL INTERVENTIONS TO IMPROVE PUBLIC HEALTH](https://reader031.fdocuments.us/reader031/viewer/2022013012/56816957550346895de1055a/html5/thumbnails/43.jpg)
Are 7 schools per experimental condition enough?
We estimated power ≥ .8 for main effects Remember that each main effect estimate is
based on ALL schools In a factorial experiment you DO NOT
compare individual conditions
![Page 44: UNPACKING THE BLACK BOX ENGINEERING MORE POTENT BEHAVIORAL INTERVENTIONS TO IMPROVE PUBLIC HEALTH](https://reader031.fdocuments.us/reader031/viewer/2022013012/56816957550346895de1055a/html5/thumbnails/44.jpg)
Experi-mental
conditionN of
schoolsHealthWise
program Training
Structure, support, & supervision
Enhanced school climate
1 7 2 7 3 7 4 7 5 7 6 7 7 7 8 7
Main effect of Training is mean of (5,6,7,8) vs. mean of (1,2,3,4).
Note that all 56 schools are used in estimating the main effect.
![Page 45: UNPACKING THE BLACK BOX ENGINEERING MORE POTENT BEHAVIORAL INTERVENTIONS TO IMPROVE PUBLIC HEALTH](https://reader031.fdocuments.us/reader031/viewer/2022013012/56816957550346895de1055a/html5/thumbnails/45.jpg)
Main effect of Structure, support, & supervision is mean of (3,4,7,8) vs. mean of (1,2,5,6).
Note that all 56 schools are used in estimating the main effect.
Experi-mental
conditionN of
schoolsHealthWise
program Training
Structure, support, & supervision
Enhanced school climate
1 7 2 7 3 7 4 7 5 7 6 7 7 7 8 7
![Page 46: UNPACKING THE BLACK BOX ENGINEERING MORE POTENT BEHAVIORAL INTERVENTIONS TO IMPROVE PUBLIC HEALTH](https://reader031.fdocuments.us/reader031/viewer/2022013012/56816957550346895de1055a/html5/thumbnails/46.jpg)
Experi-mental
conditionN of
schoolsHealthWise
program Training
Structure, support, & supervision
Enhanced school climate
1 7 2 7 3 7 4 7 5 7 6 7 7 7 8 7
Main effect of Enhanced school climate is mean of (2.4.6.8) vs. mean of (1,3,5,7).
Note that all 56 schools are used in estimating the main effect.
![Page 47: UNPACKING THE BLACK BOX ENGINEERING MORE POTENT BEHAVIORAL INTERVENTIONS TO IMPROVE PUBLIC HEALTH](https://reader031.fdocuments.us/reader031/viewer/2022013012/56816957550346895de1055a/html5/thumbnails/47.jpg)
Engineering the intervention Experiment will give us
► Main effect of each individual intervention component on outcome variables
► Also interactions between intervention components
This information will be used to select the best set of the three components
Result: Intervention engineered to optimize fidelity according to our criterion
Note: optimized ≠ the best possible
![Page 48: UNPACKING THE BLACK BOX ENGINEERING MORE POTENT BEHAVIORAL INTERVENTIONS TO IMPROVE PUBLIC HEALTH](https://reader031.fdocuments.us/reader031/viewer/2022013012/56816957550346895de1055a/html5/thumbnails/48.jpg)
Example 3: Internet-delivered drug abuse prevention program aimed at NCAA
athletes
David Wyrick UNC Greensboro
Prevention Strategies, LLC
Melodie Fearnow-KenneyPrevention Strategies, LLC
Team includes L. M. Collins, J. Milroy, K. RulisonProject funded by NIDA grant R44DA023735
![Page 49: UNPACKING THE BLACK BOX ENGINEERING MORE POTENT BEHAVIORAL INTERVENTIONS TO IMPROVE PUBLIC HEALTH](https://reader031.fdocuments.us/reader031/viewer/2022013012/56816957550346895de1055a/html5/thumbnails/49.jpg)
Example 3: Internet-delivered drug abuse prevention program aimed at NCAA
athletes
OBJECTIVE: No inactive components. Secondary aim: Maximize overall program effect within available research
resources.
David Wyrick UNC Greensboro
Prevention Strategies, LLC
Melodie Fearnow-KenneyPrevention Strategies, LLC
![Page 50: UNPACKING THE BLACK BOX ENGINEERING MORE POTENT BEHAVIORAL INTERVENTIONS TO IMPROVE PUBLIC HEALTH](https://reader031.fdocuments.us/reader031/viewer/2022013012/56816957550346895de1055a/html5/thumbnails/50.jpg)
Some interesting features of Example 3
Intervention is internet-delivered Experimental design is a cluster-randomized
fractional factorial Takes an iterative approach
![Page 51: UNPACKING THE BLACK BOX ENGINEERING MORE POTENT BEHAVIORAL INTERVENTIONS TO IMPROVE PUBLIC HEALTH](https://reader031.fdocuments.us/reader031/viewer/2022013012/56816957550346895de1055a/html5/thumbnails/51.jpg)
Background of MyPlaybook Study
Theoretical Model for MyPlaybook
Social NormsSocial Norms Theory
Positive & Negative Expectancies
Health Belief Model
Intentions to Avoid Use and Prevent Harm
Theory of Planned Behavior
Alcohol and Other Drug Use
Consequences
![Page 52: UNPACKING THE BLACK BOX ENGINEERING MORE POTENT BEHAVIORAL INTERVENTIONS TO IMPROVE PUBLIC HEALTH](https://reader031.fdocuments.us/reader031/viewer/2022013012/56816957550346895de1055a/html5/thumbnails/52.jpg)
Intervention components Five components, each corresponding to a substance:
I. Alcohol
II. Tobacco
III. Marijuana
IV. Performance enhancers
V. Prescription and OTC drugs Each component aimed at
► social norms
► positive and negative expectancies
► intentions to avoid use and prevent harm
![Page 53: UNPACKING THE BLACK BOX ENGINEERING MORE POTENT BEHAVIORAL INTERVENTIONS TO IMPROVE PUBLIC HEALTH](https://reader031.fdocuments.us/reader031/viewer/2022013012/56816957550346895de1055a/html5/thumbnails/53.jpg)
MOST as implemented in the My Playbook study
Optimizedintervention
Alcohol
Performance enhancers
Tobacco
RCT
Marijuana
Prescription/OTC drugs
Experiment 1
Revision of componentsExperiment 2
![Page 54: UNPACKING THE BLACK BOX ENGINEERING MORE POTENT BEHAVIORAL INTERVENTIONS TO IMPROVE PUBLIC HEALTH](https://reader031.fdocuments.us/reader031/viewer/2022013012/56816957550346895de1055a/html5/thumbnails/54.jpg)
Design of MyPlaybook experiments: 25-1 fractional factorial. 64 schools assigned to conditions; overall N approx. 3500
Condition Number
Intro Component
Intervention Components Manipulated
I II III IV V
1 On Off Off Off Off On
2 On Off Off Off On Off
3 On Off Off On Off Off
4 On Off Off On On On
5 On Off On Off Off Off
6 On Off On Off On On
7 On Off On On Off On
8 On Off On On On Off
9 On On Off Off Off Off
10 On On Off Off On On
11 On On Off On Off On
12 On On Off On On Off
13 On On On Off Off On
14 On On On Off On On
15 On On On On Off Off
16 On On On On On On
![Page 55: UNPACKING THE BLACK BOX ENGINEERING MORE POTENT BEHAVIORAL INTERVENTIONS TO IMPROVE PUBLIC HEALTH](https://reader031.fdocuments.us/reader031/viewer/2022013012/56816957550346895de1055a/html5/thumbnails/55.jpg)
Engineering the intervention Each experiment will give us
► Main effect of each individual intervention component on outcome variables
► e.g. substance use at 30-day follow-up
► Also selected interactions between intervention components
We will select the best version of each component for inclusion in the optimized intervention
The optimized intervention will be evaluated via an RCT
![Page 56: UNPACKING THE BLACK BOX ENGINEERING MORE POTENT BEHAVIORAL INTERVENTIONS TO IMPROVE PUBLIC HEALTH](https://reader031.fdocuments.us/reader031/viewer/2022013012/56816957550346895de1055a/html5/thumbnails/56.jpg)
Outline What is a behavioral intervention? What is the black box, and why unpack it? What is optimization? A few examples Concluding remarks
![Page 57: UNPACKING THE BLACK BOX ENGINEERING MORE POTENT BEHAVIORAL INTERVENTIONS TO IMPROVE PUBLIC HEALTH](https://reader031.fdocuments.us/reader031/viewer/2022013012/56816957550346895de1055a/html5/thumbnails/57.jpg)
Frequently asked questions Nice idea, but will it be fundable? Can this approach be carried out with the level of
funding typically available? Will I be able to publish based on the results of the
screening experiments? I don’t see how I can implement all the experimental
conditions required by a factorial experiment. What if the outcome of interest is in the distant
future?
![Page 58: UNPACKING THE BLACK BOX ENGINEERING MORE POTENT BEHAVIORAL INTERVENTIONS TO IMPROVE PUBLIC HEALTH](https://reader031.fdocuments.us/reader031/viewer/2022013012/56816957550346895de1055a/html5/thumbnails/58.jpg)
If you are interested in learning more about MOST
FACULTY► “A Taste of Methodology” brief workshop, Monday,
April 30, 10:30 am to 1:30 pm (co-sponsored by the Methodology Center and SSRI; includes lunch)
GRAD STUDENTS► HDFS 597x Quantitative Methods for Intervention
Science
![Page 59: UNPACKING THE BLACK BOX ENGINEERING MORE POTENT BEHAVIORAL INTERVENTIONS TO IMPROVE PUBLIC HEALTH](https://reader031.fdocuments.us/reader031/viewer/2022013012/56816957550346895de1055a/html5/thumbnails/59.jpg)
The future I would like to see Behavioral interventions systematically
engineered to meet specific criteria Standards of intervention effectiveness,
efficiency, cost-effectiveness Cumulative base of scientific knowledge Behavioral intervention engineering a subfield
in industrial engineering
![Page 60: UNPACKING THE BLACK BOX ENGINEERING MORE POTENT BEHAVIORAL INTERVENTIONS TO IMPROVE PUBLIC HEALTH](https://reader031.fdocuments.us/reader031/viewer/2022013012/56816957550346895de1055a/html5/thumbnails/60.jpg)
THANK YOU!