1

PCORI Case Study: The Effectiveness of Patient Engagement

M. Suzanne Schrandt, JD Deputy Director of Patient Engagement

Snapshot of Funded Projects

Number of projects: 360 Amount awarded: $671 million Number of states where we are funding research: 39 plus the District of Columbia))

Our Review Criteria

Impact of the condition on health of individuals and populations Potential for improving care and outcomes Technical merit Patient-centeredness Patient and stakeholder engagement

During a rigorous merit review process, proposals are evaluated to assess:

Learning From Our Portfolio

Engagement Officers

We-ENACT

Engagement Rubric

The Engagement Rubric

The rubric is intended to provide guidance to applicants, merit reviewers, awardees, and engagement/program officers (for creating milestones and monitoring projects) regarding patient and stakeholder engagement in the conduct of research. It is divided into four segments:

Planning the Study

Conducting the Study

Disseminating the Study Results

PCOR Engagement Principles

Planning the Study

Real World Examples; •  Mental health study: Patient partners and community

members helped craft the study name and materials to reduce the potential for stigma and to reframe the goal of the study as a movement toward emotional well-being rather than away from a mental health challenge.

Conducting the Study

Real World Examples; •  Asthma study: Clinicians and patients both provided

guidance on who should deliver the intervention, when it should be provided during the process of care, and how it should be delivered.

Disseminating the Study Results

Real World Examples; •  Cardiac study: A Patient Dissemination Board is

helping to craft the dissemination plan and advise the research team on how to best share study findings.

Engagement Principles

Real World Examples; •  Compensation for patient partners is included in the

budget at an appropriate level. •  Training and educational opportunities are provided, for

patient and stakeholder partners such as training in human subjects protection.

•  Training is provided for researchers such as instruction in better communication with patients, led by patient instructors.

Learning From Our Portfolio

Engagement Officers

We-ENACT

Engagement Rubric

Putting the Rubric to Work: Engagement Officers

Greater understanding of engagement

Stronger engagement in

proposals

Strong engagement

during research?

Role and Responsibilities

 Partners with Program Officers and research teams to oversee and cultivate engagement throughout the life of the project §  Ensures robust engagement is integrated into projects

from time of contract formation §  Participates in ongoing project management activities §  Available for assistance and discussion  Gleans promising practices from projects to share and replicate

Learning From Our Portfolio

Engagement Officers

We-ENACT

Engagement Rubric

ENgagement ACTivity (ENACT) Inventory

  Who is engaged   Partnership characteristics §  How formed, length, frequency of engagement, etc.

  Level of engagement   When in the research process they are engaged   Perceived level of influence of partners   Perceived effects of engagement on research questions, study design, study implementation, and dissemination of results   Challenges, facilitators   Lessons learned for engagement   PCOR principles – respect, co-learning, etc.

WE-ENACT Data Collection

  Self-report §  Principal Investigators §  Patient and stakeholder partners

  Completed at baseline and annually   Versions developed for §  PCORI Pilot Projects §  PCORnet projects §  PCORI broad and targeted portfolio

Engagement in Planning the Study Researcher report

45%

49%

18%

44%

52%

52%

0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%

Study Design

Adding more people to the research team

Developing the Budget

Proposal Development

Developing the Research Question

Identifying Research Topics

Percent of Projects

Identifying Research Questions: Perceived Influence

5%

20%

29%

46%

0

26%

52%

22%

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

None A Small Amount A Moderate Amount

A Great Deal

Per

cent

of P

roje

cts

Researcher Stakeholder

Identifying Research Questions: Impact of Research Engagement

“We ended up with different research questions and framing than I would have initially thought, and this was specifically because input from stakeholders

concerning the research question.”  

“Topics were more tailored to parent and family

concerns.”

“Their insight into the problem among patients in

their community helped focus the research project.”  

Study Design: Perceived Influence

0

22%

42% 36%

0

47%

30%

20%

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

None A Small Amount

A Moderate Amount

A Great Deal

Per

cent

of P

roje

cts

Researcher Stakeholder

Study Design: Impact Researcher Feedback

“Patients and stakeholders helped form the content of interventions… to better meet the needs of [patients].”

“Our community discussions… led to several modifications of our study design… This led us to include a 3rd group in our research

design: community-based group exercise. We also decided to use… [a specific] outcome measure, based upon input from… patients who told us that their biggest concern was the ability to

walk and stay active.”  

PCOR Principles

79%

35%

76%

91% 86%

50%

86%

71%

0% 10% 20% 30% 40% 50% 60% 70% 80% 90%

100%

Per

cent

of P

roje

cts

Researcher

Stakeholder

Engagement Challenges

50%

38%

23% 21% 15%

11% 11% 6%

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Lack of Stakeholder

Time

Lack of Stakeholder

Knowledge of Engagement

Lack of Research Team Knowledge of Engagement

Lack of Research Team

Time

Per

cent

of P

roje

cts

Researcher Stakeholder

Overcoming Challenges to Engagement Researchers’ recommendations 1

“One research team member is primarily tasked with maintaining contact with patients and advisors

engaged on the project to ensure that there is a point of contact for engagement at all times.”  

“We have paid stakeholders for their time. We have tried to schedule meetings at their convenience. We have solicited

information from stakeholders individually (as opposed to being in a group) whenever the stakeholder could not make a meeting.”  

Overcoming Challenges to Engagement Researchers’ recommendations 2

“More experience and learning over the course of the research project; developed capacity building materials.

We still believe there is a role of a short research curriculum …that could be completed by stakeholders.”  

“We learn as we go by immersing ourselves in each others' cultures and explicitly valuing what each does.”  

Patient and Stakeholder Feedback

“Was very impressed that this research team is open to discussion and took a lot of time and consideration in how the community wants to see some of the things they're doing. Very different than what has happened in the past. Institutions are

opening up and valuing what the community has to say.”  

“The researchers kept in very good contact with me, always answered my emails and always sent prompt updates on the project, I never wondered what was being worked on or what was needed from me, all data was shared with me, I felt very

included in the team at all times.”  

Lessons Learned (so far…)

 Relationships take time  Training is important  Creativity is critical  Widespread need for “matchmaking” and awareness-raising  Need to engage “unusual suspects”

Tier I Up to $15,000

Up to 9 month term

Tier II Up to $25,000

Up to 12 month term

Tier III Up to $50,000

Up to 12 month term

PCORI Funding Announcement

Pipeline to Proposal Awards

Or submissions to other PCOR/CER Funders

Pipeline to Proposal Awards

 Tier I: building community and capacity around a topic of interest to patients, researchers or other stakeholders.  Tier II: formalizing partnerships, identifying potential CER question to be researched, developing infrastructure or governance processes.  Tier III: creating research proposal, including robust engagement plan for later submission to PCORI or other funders of PCOR.

Learning Laboratory

 Supplementing information gleaned from larger portfolio  Providing opportunity to learn as research teams/proposals are formed from the ground up  Offering options for many historically excluded or disconnected from the research funding paradigm

Thank you!