Institute of Health Services and Policy Research (IHSPR)

Institute of Gender and Health (IGH)Institute of Population and Public Health (IPPH)

CIHR Reform to Open Programs and Peer Review Processes:

A Community Call to Action: Reforms Bootcamp for the Pillars III and IV Communities

2

What this slide deck contains:

• What is programmatic research?• The Challenge – Pillar III and IV Track Record

in the Open• Our Strategy – Pillar III and IV Call to Action:

Open Reforms Bootcamp• Preparing for the Reforms - Important things to know:

Transition timelinesEligibility for the Foundation Scheme Live Pilots

(and contact information to confirm your eligibility)Strategy scenariosWhere to find additional information

3 What is programmatic research?

The Foundation Scheme is intended to provide long-term funding support for programmatic research.

Programmatic research is:• Multiple research projects that are conceptually linked

and implemented over several years, based on an analysis of gaps in current knowledge in the field;

• A series of research projects inside a clearly conceptually linked programmatic theme.

4The Challenge:

Pillar III and IV Track Record in the Open

• An analysis of CIHR data was performed to understand differences in application and funding trends across CIHR’s four Pillars.

• Areas for concern for Pillar 3 and 4 include:• Funding trends• Peer review committee behaviour• Determinants of funding success rates• Funding sustainability

5

What does the data tell us?

Health Services (Pillar III) and Population Health (Pillar IV) research communities are more likely to:

• Apply for new grants rather than renewals• Apply for shorter duration grants• Succeed in getting funding with a re-submission of a grant

compared to a first time submission (usually to address methods issues/ reviewer questions)

• Score systematically lower compared to the biomedical community

• Score a higher proportion of grants in the non-fundable range compared to biomedical reviewers

• Have unsustained funding over a 5 year period (short duration grants+resubmission “requirement”=more grants need to be submitted to maintain sustained funding

6 Funding Trends Open Operating Grants Program 2001-2010: Number of yearly

grant applications by pillar

2001 2002 2003 2004 2005 2006 2007 2008 2009 20100

1000

2000

3000

4000

5000

6000

7000

8000

9000

10000

0

5

10

15

20

25

30

35

40

45

50

Biomedical Clinical Health Services & Policy

Population & Public Health Fundable, Not Funded Funded

Not Fundable

Nu

mb

er o

f g

ran

t ap

pli

cati

on

s

% F

un

din

g S

tatus

7 Proportion of Funding and Nominated Principal Investigators by pillar through Open

Grant Program Over 10 Years

Proportion Funding Open Grant Competition Proportion NPI Funded Through Open Grant Competition

82%

66%

9%

14%

3%

7%

6% 10%

Biomedical ClinicalHealth Services & Policy Population Health

$4,3

00,7

77,7

28

5 07

1 N

PI

8Funding Trends Open Operating Grants Program 2001-

2011: Funding status of grant applications by pillar

Biomedical Clinical Health Services & Policy Population & Public Health0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

46

2820 21

25

20

25 24

29

52 55 55

Fundable, Not Funded Funded Not Fundable

9Open Operating Grants Program 2001-2011:

Funded Grant Duration Requested versus Awarded

Biomedical Clinical Health Services & Policy Population & Public Health0

10

20

30

40

50

60 -7

-6

-5

-4

-3

-2

-1

0

Grant duration requested and awarded for funded Open grants by pillar (2001-2011)

Duration requested

Duration awarded

Difference between duration requested and duration awarded

10 Open Operating Grants Program 2001-2011: New Application versus Renewal

by Self-Identified Pillar

Biomedical Clinical Health Services Population Health

050

0010

000

1500

020

000

2500

0

New Renewal

Num

ber o

f App

licati

on

11 Open Operating Grants Program 2001-2011: Renewal Funding Success by Self-

Identified Pillar

Biomedical Clinical Health Services Population Health0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

2697

166

42

39

1921

101

28

38

694 121 54 82

Funded Fundable Not Fundable

% o

f A

pp

lica

tio

ns

12 Pillar Review Committee Behaviour OOGP 2001-2011. Peer Review Committee Consensus and

Summary Scores by Pillar and Year

2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 20112.8

3

3.2

3.4

3.6

3.8

4

Biomedical Consensus Score Biomedical Summary ScoreClinical Consensus Score Clinical Summary ScoreHSPR Consensus Score HSPR Summary ScorePopulation Health Consensus Score Population Health Summary Score

Cons

enus

and

Sum

mar

y Sc

ores

Triage Score (until 2003)

Triage Score (from 2003)

13 Proportion of Nominated Primary Investigators with Sustained Funding in the First Five Years of

Receiving Grant between 2001 and 2005

Grant Receipt 1 Year Later 2 Years Later 3 Years Later 4 Years Later 5 Years Later0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Biomedical Clinical Health Services & Policy Population Health

14 Sustained Funding Predictors for HSPR

Ref: < 10 10-20 21-35 36-50 51-100 101+ Ref: Mixed

Open Strategic Ref: first time

Resub Duration0

1

2

3

4

5

6

7

8

1

3.81 3.89

5.21*

3.4

6.69**

1

0.28 0.17*

1

3.52**

1.03*

Od

ds

Ra

tio

Mean number active colleagues 2001-2005 Competition Type ResubmissionDuration baseline appl.

*P<0.05**P<0.01

See Appendices A and B for complete list of variables and results

Models were adjusted for:  age, gender, competition type, number of applicants involved in application, Average # of Active Colleagues between 2001-2005, First-time Application or Resubmission (Follow-up), Switched Pillars (Follow-up), Switched Universities (Follow-up), received career award (Follow-up)

15 Sustained Funding Predictors for PPH

Ref: Less 10

10-20 21-35 36-50 51-100 Ref: Mixed

Open Strategic Ref: first time

Resub. Duration0

0.5

1

1.5

2

2.5

3

1

2.2

1.32

1.73

1.91

1

0.24**

0.76

1

1.76*

1.03*Od

ds

Ra

tio

Mean number active colleagues 2001-2005

Competition Type ResubmissionDuration baseline appl.

*P<0.05**P<0.01

Models were adjusted for:  age, gender, competition type, number of applicants involved in application, Average # of Active Colleagues between 2001-2005, First-time Application or Resubmission (Follow-up), Switched Pillars (Follow-up), Switched Universities (Follow-up), received career award (Follow-up)

See Appendices A and B for complete list of variables and results

16 What is the Time and Cost of Applying and Reviewing?

Applicants

Mean time to write application 169 hours

Mean cost/application @64.52/hour $10,904

Peer Reviewers

Mean time to review/ application 8.33 hours

Mean cost/review @64.52/hour* $537

Source: http://www.cihr-irsc.gc.ca/e/45846.html

*Note that the mean cost to review excludes the time spent at the review meeting and the travel costs to attend the meeting. The full peer review cost is estimated to be closer to $1,812.

17Our Strategy:

A Pillar III & IV Community Call to Action

1. Prepare the community to develop competitive programmatic grants for the foundational program.

• Defining expectations of excellence, training materials and opportunities.

2. Prepare the community to provide excellent peer review for the foundational scheme.

• Defining expectations of excellence in foundational program review.

• Training reviewers and identifying relevant qualifications for entry into the College of Reviewers for the foundational scheme (multi-stage review based on a research summary and CV will be new).

3. Develop and implement a monitoring and feedback program

18

Open Reforms Bootcamp

• On October 10 and 11, 2013, the Institute of Health Services and Policy Research, Institute of Gender and Health and the Institute of Population and Public Health hosted an Open Reforms Bootcamp in Montreal.

• The meeting brought together Pillar III and IV thought-leaders from across the country who were nominated from their institutions as well as NAPHRO and CIHR partners to discuss how to best prepare our research communities for success, including defining what constitutes excellence for programmatic grant writing and reviewing.

19

Bootcamp Objectives

• To gain consensus on what constitutes excellence for Programmatic Grant Writing, Peer Review and Monitoring and Evaluation;

• To develop an implementation plan for culture change: beliefs and behaviours in the Pillar III and IV research community.

20

Anticipated Outcomes

• An Action Plan for culture change in Pillar 3 and 4 communities and a supporting toolkit;

• A reporting and feedback system for monitoring success; and

• A group-authored consensus document published in a major Canadian journal, focused on implementing culture change in the Pillar III and IV research communities.

21Preparing for the Reforms - Important things to

know:

• Transition timelines• Eligibility for the Foundation Scheme Live Pilots

(and contact information to confirm your eligibility)

• Examples of programmatic grants• Eligibility strategy scenarios • Where to find additional information

22

• The transition to the new Open Suite of Programs and peer review processes will occur over a number of years. Course corrections and adjustments may be required along the way as we learn from the results of the pilots.

• The transition strategy includes three phases:1. 2012 to 2015: Piloting key peer review design elements2. 2014 to 2016: Gradually phasing-in the new funding schemes3. 2014 to 2016: Gradually phasing-out the existing Open funding programs

22

Timelines and Eligibility: Transitioning to the New Open Suite of

Programs

23Transition Timelines

Note: The timelines for the Open Operating Grant Program (OOGP), Transitional OOGP, Foundation Scheme, and Project Scheme competitions reflect current planning and are subject to change. Check CIHR’s website for the latest timelines: http://www.cihr-irsc.gc.ca/e/47380.html

24 2014 Live Pilot Applicant Eligibility

The following health researchers are eligible to apply to the first Foundation Scheme "live pilot" competition:

- As of July 30, 2013, Nominated Principal Investigators and Co-Principal Investigators of an Open grant with a grant term expiry date no earlier than October 1, 2014 and no later than September 30, 2015. This includes: An OOGP grant with a grant term expiry

date of March 31, 2015; or An OOGP grant with a grant term expiry

date of September 30, 2015.

- New/early career investigators;

- Researchers who have never held Open CIHR funding as a NominatedPrincipal Investigator or Co-Principal Investigator.

Determine your Eligibility!

Ask your VPs of research of your CIHR University Delegate to find out if you are one of eligible.

Or you can e-mail CIHR directly at:Roadmap-Plan.Strategique@cihr-irsc.gc.ca

25 Programmatic Grant Examples

Purpose: The Foundation Scheme will support programmatic research. Programmatic research is: (1) Multiple research projects that are conceptually linked and implemented over several years, based on an analysis of gaps in current knowledge in the field; and (2) A series of research projects inside a clearly conceptually linked programmatic theme.

The following excerpts from three funded grant applications are intended to serve as illustrative examples of different kinds of programmatic grants.

26Examples of Programmatic

Grants in Pillars III and IVSex, gender and equity in prescription drug access, appropriateness, and affordability: Steve Morgan:

The proposed program will generate important evidence from previously uncorrelated data on how biological and social dimensions of sex and gender – interacting with age, ethnicity, income, and context – influence equity In prescription drug access, appropriateness, and affordability.  The proposal is for a set of five interrelated projects focusing on the ways in which sex and gender – acting alone and in conjunction with other characteristics – shape behaviours and outcomes related to the availability and use of prescription drugs in Canada. While conducting this work over a four-year period, the team will be contributing to the development (and refining) of analytic frameworks and methods for conducting quantitative, sex- and gender-based analyses in pharmaceutical policy.  The approach relies on available population-based databases: market research data, linked administrative health care data, and national surveys. Over the course of the program of research, the team will be working to advance methods for operationalizing gender analyses with these commonly available databases. Moreover, by producing quality sex- and gender-based work by using available databases, the team will contribute evidence to support the case for more purposefully capturing gender constructs in surveys that can be linked to those databases.

27Examples of Programmatic

Grants in Pillars III and IVStrengthening Nurses’ Capacity in HIV Policy Development in Sub-Saharan Africa and the Caribbean: Nancy Edwards:

This sequential program of research examines what influences HIV/AIDS nursing care and strategies to strengthen HIV/AIDS nursing care in four Lower and middle income countries.  Program objectives are to strengthen health systems for HIV and AIDS in the participating countries by improving the quality of HIV and AIDS nursing care, to support the scale-up of innovative HIV and AIDS programs and practices, and to foster dynamic and sustained engagement of researchers and research users in the policy development process.  Primary research objectives are: a) to examine the dynamic interplay of multi-level factors that influence nurses’ engagement in strengthening health care systems for HIV and AIDS in lower and middle income countries (LMICs), and b) to determine the impact of leadership hubs and a participatory action research process on nursing care and workplace polices for HIV and AIDS.  Three conceptually linked studies are planned.  A qualitative study of front-line managers and national decision-makers will examine bottom-up and top down innovation in the context of HIV/AIDS.  A mixed methods study will examine nurses’ experience of the HIV/AIDS epidemic including stigma, and the impact of HIV/AIDS on their nursing care and work life.  A quasi-experimental study, which is grounded in a participatory action research approach, will examine the impact of leadership hubs on nursing care and health care organizational policies related to HIV/AIDS.

28Examples of Programmatic

Grants in Pillars III and IV

E-Integration in the Management of Respiratory and Circulatory Disease: Elucidating the Multi-Level Mechanisms that Optimize Population Outcomes: Robyn Tamblyn:

The proposed research program comprises a team of clinical and management scholars who will focus on the study of e-integration for the management of respiratory and circulatory diseases, and provide training for future scientists. The research program will incorporate three of the four pillars of the CIHR: 1) clinical research, 2) health systems and services research, and 3) research in the health of populations. This cross-theme research program will be based upon the unique competencies of health and management to tackle the complex issue of effectively using technology in integrated health systems. The program will involve a series of investigations in four inter-related areas to enhance our understanding of how to design and implement integrated care delivery systems to enhance the effective management of chronic disease, and assess their impact on existing disparities in care delivery at the level of the population. While projects will be led by individual team members, an integrated approach will be taken to research design, data collection and analyses as well as preparation of joint grant applications for these projects. With the cooperation of team members, students and collaborators, we will be able to build a cohesive body of knowledge for determining how technology-enabled integrated disease management can achieve optimal care delivery and patient outcomes through the proposed research program.

29

Eligibility Scenarios

Purpose: The following scenarios are hypothetical and intended to illustrate the diversity of career stages, track records and programs of research that the Foundation Scheme will support. The scenarios are also intended to help individuals strategize about their application and the team that may be involved by illustrating the implications for all those involved in a Foundation Scheme grant.

30

New Investigator Scenarios

Dr. S. is new investigator who has never applied to CIHR. She has held a modest research grant of $50,000 per year for 2 years from a research foundation that has allowed her to establish pilot work for her research program. She has published her pilot work in one of the top journals in her research area. She has identified the series of studies that she needs to conduct in the next steps of her program of research. While she has collaborated with more senior researchers, she is ready to build on her pilot work and pursue an independent program of research.  Dr. S. decides to apply to the Foundation Scheme as a Program Leader on her own.

31

New Investigator Scenarios

Dr. T. is a new investigator who is completing a CIHR-funded project with a budget of $100,000 per year for 2 years.  He thinks that program of research will ramp up considerably over the next year or two and rather than applying to the Foundation Scheme has decided to apply to the Project Scheme in order to have a few projects underway before he applies for the Foundation Scheme.  He does this because he realizes that if he applies to the Foundation Scheme, it is unlikely that he will be able to justify a significantly larger budget than the one he presently has. 

32Mid-career Investigator

Scenarios

Dr. L. is a mid-career researcher who for the past eight years has consistently held CIHR strategic grants of, on average,  $150,000 per year. Because she has never held Open CIHR Funding, she is eligible to apply to the Foundation Scheme pilot. She decides to apply the CIHR Foundation Scheme with a budget of $150,000 per year.  She realizes that if she is successful in the Foundation Scheme, as a Program Leader she will be ineligible to apply to the Project Scheme but recognizes that there will be opportunities to supplement her Foundation grant with strategic funding opportunities at CIHR in the future.

33Mid-career Investigator

Scenarios

Drs. M and R are mid-career researchers who have worked together over the past seven years on a number of multi-year project grants in the same thematic area, building a shared program of research. They have demonstrated shared leadership and together have made significant contributions to their field. Dr. M is eligible for the 2014 Foundation Scheme live pilot but Dr. R is not because her current grant does not expire in the required timeframe. Because Dr. M is eligible, they are eligible to apply together. 

34Mid-career Investigator

Scenarios

Drs. A and C are mid-career researchers who have worked together over the past nine years on a several Partnerships for Health System Improvement grants. Dr. A was also the Nominated Principal Investigator on an Emerging Team Grant. They would like to continue working together, but decide that Dr. A will apply as Program Leader for a Foundation Scheme grant and Dr. C will be a Program Expert so that Dr. C is eligible to apply for grants in the Project Scheme.    

35Senior Investigator

Scenarios

Dr. X is a senior investigator who has held a number of CIHR grants over the past 20 years.  Over the last 10 years she has established a joint program of research with another senior scientist. In partnership they have made significant contributions and have a strong vision for a shared program of research going forward. They plan to continue to do their work together and so they plan to jointly apply for a Foundation grant as Program Leaders. Should their application be successful, neither Dr. X nor her colleague will be able (or required!) to apply for any other Project Scheme grants for the seven-year duration of their research program.   

36Senior Investigator

Scenarios

Dr. Y is a senior investigator who has held a number of CIHR grants. In his work he has primarily partnered with another senior scientist.  They plan to continue to work together but agree that Dr. Y should apply for a Foundation grant on his own, and his research colleague will not be named as a Program Leader on this grant. This decision is made strategically so that Dr. Y’s colleague can be eligible to apply for funding under the Project scheme and support their work through multiple funding sources and vehicles.   

37 Team Researcher/Collaborator Scenario

Dr. N is a researcher who always works in large teams and is having trouble with the notion of the programmatic grant.  He conducts community-based research and wants to find ways to acknowledge his community collaborators when he applies for research grants.  He also has consistently brought New Investigators onto his applications as co-investigators.  He decides that the best way to proceed is to apply for a Foundation grant as a single Program Leader.  In his applications he highlights the collaborative nature of his work and names a number of partners as collaborators as Program Experts. He ensures he develops a budget that will support these collaborations. He then works with a number of new investigators to assist them with developing strong applications to the Project Scheme.

38Integrated Knowledge

Translation ScenarioDr. E is a senior investigator in province A who has spent the last eight years of her career engaged in a program of research that included cross-jurisdictional integrated knowledge translation research in provinces A and X through programs like Partnerships for Health System Improvement (PHSI). She has consistently worked with another senior investigator who leads the projects in province X, as well as lead her own projects involving a variety of decision makers in both provinces. She knows that in the Foundation Scheme, the Program Leaders must remain the same throughout the seven years of funding but the Program Experts can change over time. She considers the Project Scheme, but decides that a Foundation grant would provide valuable continuity of funding to advance her program of research and flexibility to engage decision makers as Program Experts at various points in time. She decides to apply for a Foundation grant and includes her colleague as well as two decision makers as Program Experts.

39

For more information..

• Please visit the CIHR website for further information about the Reforms of Open Programs and peer review: http://www.cihr-irsc.gc.ca/e/44761.html

• For more information about Open Operating Grant Program statistics and funding trends, please see: http://www.cihr-irsc.gc.ca/e/46341.html#fig1

• To check your eligibility for the live pilot Foundation Scheme, email Roadmap-Plan.Strategique@cihr-irsc.gc.ca

40

Special Thanks to…

Our Group Chairs: Our Facilitator:- Carole Estabrooks - Andreas Laupacis- Michael Schull- Adrian Levy Our note-takers:

- Joanne Simala-Grant - Hawa Kombian - Meg McMahon - Kate Wood

…and the 2013 Open Reform Bootcamp participants!