Selecting An Intervention In A Complex Adaptive Health Care System

&Evaluating Its Impact

Malaz Boustani, MD, MPHIndianapolis Discovery Network for DementiaIndiana University center for Aging Research

Regenstrief Institute, Inc

www.indydiscoverynetwork.com

04/08/23 indydiscoverynetwork.com 2

The US Health Care System Report Card

• Sub-optimal quality– Dementia recognition:< 40%– FDA approved Trx: < 10%– Inappropriate Trx: > 25%

• Compromised safety– 90,000 death / yr– 1,000,000 injury / yr

• Waste – 30% - 40% health expenditure (overuse, underuse, misuse, duplication)

• Discovery to Delivery:– 17 yrs– $800 millions

• Generalizability of Research Findings:– < 1% of real world patients enrolled in AD studies

Boustani, 2007; IOM, 2001, Westfall, 2007; Faison et al, 2003;

04/08/23 indydiscoverynetwork.com 3

Independent Agent E

Other Local CASsStress

Feedback

Independent Agent D

Independent Agent C

Independent Agent B

Independent Agent A

Changing Environment

Connection

The health care CAS

Emergent Performance

04/08/23 indydiscoverynetwork.com 4

Complex Adaptive Health Care System • An open, dynamic, flexible, adaptive, and complex

network• Complex due to

– Numerous interconnected, semi-autonomous, competing, and collaborating members

• Adaptive due to– Its capability of learning from its prior experience– Its flexibility to change its members connecting

patterns to fit better with its surrounding environment

Holden, 2005; Litaker et al. 2006 Feb.

04/08/23 indydiscoverynetwork.com 5

Complex Adaptive Health Care System

• Emergent behaviors – NOT predetermined ones

• Self-organized controls – NOT central controls

Holden, 2005; Litaker et al. 2006 Feb.

04/08/23 indydiscoverynetwork.com 6

The Paradox of CAS

• Each CAS is unique in its – Members diversity– Members interactions– Surrounding environment– Previous history– Evolving and learning process.

• Health care research centers generate scientific data from group-based studies – Then regulators/guideline developers ask each unique CAS to

implement the findings.

Holden, 2005; Litaker et al. 2006 Feb; Stoebel et al, 2005.

04/08/23 indydiscoverynetwork.com 7

The Paradox of CAS

• The performance of a CAS fluctuates over time

• The connecting patterns of its members are dynamic and change over time in respond to the stress or the fitness requirements of its surrounding environment.

Holden, 2005; Litaker et al. 2006 Feb; Stoebel et al, 2005.

04/08/23 indydiscoverynetwork.com 8

How Can I Assess CAUSALITY?Can I Predict?

1. How can I identify or determine that a system is a CAS?2. How can a system with no bosses, no mandates and no

plans produce repeatedly complicated structures and perform difficult tasks?

3. Can I build a complex adaptive system model to study? What can I do with a simulator?

4. What data do I have & how can I analyze data in CAS?5. How can I manage the problems of uncertainty,

variability, and variable interdependency in a CAS?6. Is replication essential in science?

04/08/23 indydiscoverynetwork.com 9

7. How can I select a change in a CAS?8. How can I evaluate the success or the impact of my

proposed intervention or change in a CAS?9. How are experiments performed in a complex adaptive

system?10. Is a randomized controlled trial the best accurate study

design to evaluate an intervention in a CAHS?

How Can I Assess CAUSALITY?Can I Predict?

Selecting an Intervention

04/08/23 indydiscoverynetwork.com 11

The 9 Principles of Selecting a Change in a Complex Health Care System

1. View your system through the lens of complexity.

2. Build a good enough vision with minimum specifications

3. Lead from the edge by balancing data and intuition, planning and acting, safety and risk.

4. Foster the "right" degree of information flow, diversity and difference, connection among agents, power differential, and anxiety.

5. Uncover and work with paradox and tension.

Paul E Plsek. 1997.

04/08/23 indydiscoverynetwork.com 12

The 9 Principles of Selecting a Change in a Complex Health Care System

6. Go for multiple actions at the fringes, let direction arise.

7. Listen to the informal relationships, gossip, rumor, and hallway conversations that contribute significantly to the individuals’ perceptions about their surrounding environment and their subsequent actions.

8. Allow complex systems to emerge out of the links among simple systems that work well and are capable of operating independently.

9. Build a community of members who collaborate, create, learn and compete

Paul E Plsek. 1997.

04/08/23 indydiscoverynetwork.com 13

Reflective Adaptive Process

• Build a local “Think-Tank: the RAP Team” responsible of introducing an acceptable, locally matched, flexible and effective change in its CAS.

• External or internal facilitators who encourage the RAP Team to select, adopt or create local processes to – solve the CAS problem – enhance the CAS performance– guide the CAS respond to its surrounding environment

• RAP is the second generation of CQI

Stroebel et al.2005.

04/08/23 indydiscoverynetwork.com 14

The RAP Principles

• Vision, mission, and shared values are fundamental in guiding ongoing change processes in a CAS

• Creating time and space for learning and reflection is necessary for a CAS to adapt to and plan change

• Tension and discomfort are essential and normal during CAS change

• Improvement teams should include a variety of system’s agents with different perspectives of the system and its environment

• System change requires supportive leadership that is actively involved in the change process, ensuring full participation from all members and protecting time for reflection

McDaniel et al, 2003; Stroebel et al.2005; Boustani et al 2007.

04/08/23 indydiscoverynetwork.com 15

Evaluation of Interventions in CAS

• Selecting the content and the delivery of the intervention

A- The content based on SER of past research or guidelines

B- Develop a RAP team to • Localize the content• Localize / Invent the delivery process• Monitor the delivery process• Monitor the agents interactions• Detect emergent behaviors

Evaluating the Impact of the Selected Interventions

04/08/23 indydiscoverynetwork.com 17

Methodological Quality

“Extent to which a study’s design, conduct, and analysis has minimized selection, measurement, and confounding biases”

West et al, 2002

04/08/23 indydiscoverynetwork.com 18

• Random assignment against – both unconscious and deliberate human influence

on the assignment of subjects to different groups– unknown confounders

• Blind outcome assessment ensures – treatment and analysis of outcomes are not

colored by prejudice

The “Safeguards” Evidence

Guyatt et al, 1993; West et al, 2002

04/08/23 indydiscoverynetwork.com 19

The Hierarchical Strength of

Evidence in Treatment Decisions

• N-of-1 RCT • SER of RCT • Single RCT • SER of observational studies • Single observational study • Physiologic studies • Unsystematic clinical observations

Guyatt et al, 1993; West et al, 2002

04/08/23 indydiscoverynetwork.com 20

• Design:– N-OF-1 RCT

• System outcomes– Overall Cost– Overall Safety– Overall Quality– Overall Patient Satisfaction– Overall Staff Satisfaction

• Intervention specific outcomes

Proposed Evaluation Process

04/08/23 indydiscoverynetwork.com 21

Applying N-OF-1 RCT in CAS

• A CAS receives pairs of evaluation periods– Each pair including one period of active intervention

and another of usual care in random order.

• CAS members are kept masked to allocation • Outcomes are monitored blindly and

continuously

04/08/23 indydiscoverynetwork.com 22

0----------1------------2-----------3-----------4----------5-----------6------------7-----------8-------------9-----------10------------11------------12

Time

Outcome Data

Outcome Data

Outcome Data

Outcome Data

Outcome Data

Outcome Data

Outcome Data

R to Int R to Int R to Int

R to UC R to UCR to UC

N-OF-1 RCT in CAS

04/08/23 indydiscoverynetwork.com 23

Issues of –OF-1 RCT

1. Selecting the Intervention: – Is there enough data to suggest the efficacy of the

intervention? – Did the RAP team modify the intervention delivery

and content to match the local resources? 2. Determining the rapidity of onset of the intervention

effect: – How quickly does the intervention begin to act and

cease acting?3. Optimizing dose:

– What is the best dose? – Does it differ between systems?

Guyatt et al, 1986; Price & Evans, 2002

04/08/23 indydiscoverynetwork.com 24

Issues of –OF-1 RCT

4. Selecting the outcomes:– the overall common system outcomes not related specifically

to the intervention.– the outcomes that the intervention effect directly.– the process that mediates the effect of the interventions such

as the relationship and the new connection between the CAS members.

5. Interpreting the Results of N-of-1 RCTs: – What a priori criteria should be established for classifying a

trial as definitely negative or indefinite?

Guyatt et al, 1986; Price & Evans, 2002

04/08/23 indydiscoverynetwork.com 25

The Requirement for N-of-1 RCT:

1. The problem that requires intervention needs to be chronic.

2. The problem that requires intervention needs to be stable.

3. The effect of the new intervention begins to acts within days to weeks at most.

4. The effect of the new change ceases acting after discontinuing within days to weeks at most.

Guyatt et al, 1986; Price & Evans, 2002

04/08/23 indydiscoverynetwork.com 26

The Science of CAS

• “A general feeling of unease when contemplating complex systems dynamics. Its devotees are practicing fact-free science. A fact for them is, at best, the outcome of a computer simulation; it is rarely a fact about the world.”

» (Smith 1995, p. 30).

04/08/23 indydiscoverynetwork.com 27

The Science of CAS

“Prediction is our best means of distinguishing science from pseudo-science”

» Karl Popper “The history of 20th-century science should also give

complexologists pause. Complexity is simply the latest in a long line of highly mathematical ‘theories of almost everything’ that have gripped the imaginations of scientists in this century”

» (Horgan 1995, p. 104).