Office of Portfolio Analysis (OPA) Division of Program Coordination, Planning, and Strategic Initiatives (DPCPSI) Office of the Director (OD)

CASE STUDY

Ability of New Innovator Awardees to Obtain Subsequent NIH Support

January 2015

2

Table of Contents Background ................................................................................................................................................... 3

Methods ........................................................................................................................................................ 4

Identification of the cohorts of interest ................................................................................................... 4

STEP 1: DEFINE YOUR DATASET ............................................................................................................ 4

STEP 2: SEARCHING FOR GRANTS USING iSearch ................................................................................. 4

STEP 3: DOWNLOAD DATA (AWARDEE AND APPLICATION DATA) ....................................................... 4

Subsequent awards ................................................................................................................................... 4

STEP 4: SEARCHING FOR SUBSEQUENT AWARD DATA USING iSearch ................................................. 5

STEP 5: DOWNLOAD DATA (AWARDEE AND APPLICATION DATA) ....................................................... 5

Results ........................................................................................................................................................... 6

DP2, R00, and R01-ESI cohorts.................................................................................................................. 6

DP2, R00, and R01-ESI awardees with subsequent NIH awards ............................................................... 7

Success rate of awarded applications ....................................................................................................... 8

Percentage of multi-PI applications .......................................................................................................... 9

NIH-awarded applications by activity code ............................................................................................ 10

Length of time to first subsequent NIH award ....................................................................................... 11

Conclusions ................................................................................................................................................. 12

Background

Between 1980 and 2001, the average age at which principal investigators (PIs) receive their first R01

award increased from approximately 36 to 42, where it has remained stable since. In 2009, NIH

established the Early Stage Investigator (ESI) status for PIs who are within 10 years of receiving their

terminal degree and have not yet received NIH funding as a PI. The purpose of this policy change was to

encourage peer reviewers and ICs to take into account the career stage of investigators when evaluating

their projects. Though ESI status is primarily intended to assist new investigators applying for research

support via traditional funding mechanisms such as R01 awards, several NIH mechanisms are specifically

directed at ESIs, including the NIH Director’s New Innovator Award (DP2).

The NIH DP2 program, which began in FY2007, is intended to support investigators of exceptional

creativity with the potential to produce high impact research. Only ESIs are eligible for DP2 awards. DP2

awardees receive five years of support, and lose their ESI status when submitting subsequent

applications. Peer review of DP2 applications is distinct from the traditional NIH study section process,

and attempts to identify the most creative applicants and projects with transformative potential in the

applicant’s field of biomedical/behavioral research.

Another NIH mechanism directed at ESIs is the Pathway to Independence Award (K99/R00). The goal of

the K99/R00 program, which began in FY2006 and targets researchers with no more than four years of

postdoctoral experience, is to increase and maintain a strong cohort of new and talented independent

investigators. K99/R00 awardees receive up to five years of support, consisting of two phases: an initial

mentored research interval (K99 phase, approximately two years), followed by the scientist's first

independent research support phase (R00 phase, approximately three years). K99/R00 recipients

maintain their ESI status when applying for another NIH award.

The purpose of this analysis is to investigate how successful DP2 awardees are in obtaining subsequent

NIH awards, and compare them with two control groups: R00 awardees and a randomly selected cohort

of R01-ESI awardees.

4

Methods

Identification of the cohorts of interest We extracted the data for all Type 1 DP2, R00, and R01-ESI awardees (FY2009 through FY2012) from

iSearch.

STEP 1: DEFINE YOUR DATASET The cohorts of interest for this study are all DP2, R00, and R01-ES1 awardees from FY2009-2012.

STEP 2: SEARCHING FOR GRANTS USING iSearch If you are not currently an iSearch user, please send an e-mail to isearch@od.nih.gov to request access.

Field DP2 R00 R01

ADVANCED FILTER

Fiscal Year range 2009 - 2012 2009 - 2012 2009 - 2012

Awarded

Yes (+) Yes (+) Yes (+)

Activity Code

DP2 R00 R01

FACET TAB

Agency

NIH NIH NIH

Application Type 1 4 1

Number of awardees

206 692 3009

STEP 3: DOWNLOAD DATA (AWARDEE AND APPLICATION DATA) To download the results for each search hit the Download button in the top right of the screen. Select ‘Export to CSV (20k limit)’, and select Contact PI as your download field. You will have three csv files that you will use in step 4.

In order to test a comparable number of R01-ESI awardees against the DP2 and R00 awardees, 100 R01-

ESI awardees were selected at random for each fiscal year (400 awardees total), and this cohort was

used for identification of subsequent awards.

Subsequent awards Subsequent applications and awards for these cohorts were identified as follows:

All Type 1 (T1) applications

All P, R, and/or U mechanisms

5

The time interval ranged from one year after the initial DP2/R00/R01-ESI award through the end

of FY 2014. For example, if an investigator received a DP2 award in FY2009, the interval in which

we searched for subsequent applications and awards was FY2010 through FY2014.

STEP 4: SEARCHING FOR SUBSEQUENT AWARD DATA USING iSearch

1. For each ESI cohort, prepare a CSV file with a single column containing the Contact PI names

(delete the header and first column from your iSearch download).

2. Making sure that you have cleared your last search, upload your files, one at a time by clicking

on the Upload button and browsing to where you saved your files. Select one of the CSV files

you created, and click Upload. This will automatically run the search and return all applications

for your cohort of interest.

3. Limit the results by using the Filter option with the following search criteria:

Fiscal Year: 2010 - 2014

Agency: National Institutes of Health

4. On the facet view select a. Type 1 applications. b. P, R, and U mechanisms:

i. click on the Menu icon in the top right corner of the Activity Code facet ii. Type P, R, or U into the search bar and select all activity codes of interest by

holding Ctrl while clicking. iii. Drag the selected activity codes into the Selected Facets box and ensure that

the radio button for delimiting the search is set to OR:

Click Apply Changes.

Subsequent awards data was downloaded and used to determine various aspects of awardee success.

STEP 5: DOWNLOAD DATA (AWARDEE AND APPLICATION DATA)

To download the results, select the Results tab then hit the Download button . Select ‘Export to Excel (20k limit)’. You can reduce the number of fields in the download if you wish but include Contact PI, ApplID, Multi PI Indicator, Activity Code, Award Status, and Fiscal Year at minimum.

6

Results

DP2, R00, and R01-ESI cohorts From FY2009 to FY2012 (Table 1) there were:

206 DP2 awardees

692 R00 awardees

3,037 R01-ESI awardees

o 400 R01-ESI awardees (100 per year) were randomly selected as a control group

Table 1. Number of awardees in each cohort.

7

DP2, R00, and R01-ESI awardees with subsequent NIH awards Over half of DP2 and R00 awardees who submitted a subsequent T1 award were successful (61.8% and

54.0%, respectively), compared with 37.7% of the R01-ESI cohort (Figure 1). DP2 awardees were also

more successful than the other two cohorts in obtaining more than one subsequent T1 award by the

end of FY2014 (24.1%, 15.0%, and 14.4% of DP2, R00, and R01-ESI awardees, respectively).

Figure 1. (A) Subsequent T1 applications from awardees in each cohort. (B) Awardees in each cohort

with one or more subsequent T1 awards.

0

20

40

60

80

DP2 R00 R01-ESI

% w

ith

su

bse

qu

ent

awar

d

% w/only one award % w/more than one award

A B

8

Success rate of awarded applications We also determined the success rate for each cohort by calculating the number of awards as a

percentage of the total number of T1 applications (Figure 2). The success rate was similar for the DP2

and R00 cohorts (20.1% and 19.4%, respectively), and lower for the R01-ESI cohort (16.5%). The

maximum number of applications per PI was similar across all three cohorts, as was the maximum

number of awarded applications (Figure 2A).

Figure 2. (A) Fate of each cohort’s subsequent applications. (B) Success rate for each cohort.

0

5

10

15

20

25

DP2 R00 R01-ESISu

cce

ss r

ate

(%)

A B

9

Percentage of multi-PI applications Some new investigators might attempt to increase their chance of getting an award by applying for a

multi-PI award, often partnering with more established investigators. The total number of multi-PI

applications was determined for each cohort. The percentage of all applications that were multi-PI was

higher for DP2 awardees than for R00 or R01-ESI awardees, as was the percentage of those multi-PI

applications that received an award (Figure 3).

Figure 3. (A) Multi-PI applications and awards. (B) Percentage of multi-PI applications and awards.

0

25

50

75

100

DP2 R00 R01-ESIFu

nd

ed a

pp

licat

ion

s (%

)% P awards % R01s awards

% other R awards % U awards

A B

Table 4. Multi-PI applications and awards.

10

NIH-awarded applications by activity code Most subsequent awards in each cohort were R01s, followed by other R awards (e.g., R21s) and a small

number of P and U awards (Figure 4). Note that a similar percentage of subsequent R01 awards to DP2

and R01-ESI investigators were multi-PI (29.9% and 32.5%, respectively); for R00 awardees, a much

lower percentage of subsequent R01 awards were multi-PI (8.2%).

Figure 4. (A) Funded applications by activity code. (B) Percentage of funded applications by activity code.

0

25

50

75

100

DP2 R00 R01-ESI

Fun

ded

ap

plic

atio

ns

(%)

% P awards % R01s awards

% other R awards % U awards

A B

11

Length of time to first subsequent NIH award Among those who submitted at least one application (see Figure 1A), 40.6% of DP2 awardees obtained a

subsequent award within two years, as opposed to 23.5% and 19.9% of R00 and R01-ESI awardees,

respectively (Table 2, Figure 5). The most striking difference was observed within the first year, when

over a quarter of DP2 PIs had obtained a subsequent award

Table 2. Length of time to first subsequent NIH T1 award.

Figure 5. Percentage of awardees in each category receiving a T1 award in subsequent years.

12

Conclusions

Of those submitting subsequent applications, DP2 and R00 awardees were more successful than the

R01-ESI control group (61.8%, 54.0%, and 37.7% of applicants, respectively, received an award; Figure

1). DP2 and R00 awardees were also slightly more successful than R01-ESI awardees on a per application

basis, calculated as the percentage of all T1 applications that received an award (20.1%, 19.4%, and

16.5% respectively; Figure 2). The percentage of awardees applying for subsequent awards, as well as

the average and maximum number of applications per PI, were similar among all three cohorts (Figures

1A and 2A).

A higher percentage of DP2 awardees applied for funding as part of a multi-PI application relative to the

other two cohorts, and DP2 awardees were also more successful at obtaining multi-PI funding (Table 4;

Figure 3). The majority of the awards in each cohort were R01s, followed by other R mechanisms; U and

P awards comprised a small percentage of awards in all three cohorts (Figure 4).

Overall, DP2 awardees seem to be more successful than the other two cohorts in obtaining subsequent

NIH funding. Relative to R00 and R01-ESI PIs, more than twice the percentage of DP2 PIs who applied for

an award were successful within one year (8.9%, 11.3%, and 25.9%, respectively). The method of peer

review used to evaluate DP2 applications, which is distinct from the traditional NIH method of peer

review, therefore appears to succeed in identifying more successful grant-writers.