American Evaluation AssociationEVALUATION 2009November 14, 2009

Building Data Systems to Support Evaluation in a Biomedical Research and Development (R&D) Environment

Multipaper Session 736

Session Participants

James CorriganChair

Linda KupferCareerTrac - An aid to Effective Evaluation

Kevin WrightThe Electronic Scientific Portfolio Assistant (e-SPA): A Tool

for Analysis of NIH Project Portfolios

James OnkenData Infrastructure and Access:  The National Institutes of

Health Research Portfolio Online Reporting Tool(RePORT)

Robin WagnerDiscussant

National Institutes of Health

Office of the Director

National Cancer Institute

National EyeInstitute

National Heart, Lung and Blood

Institute

National HumanGenome Research

Institute

NationalInstitute on Aging

National Instituteon Alcohol Abuseand Alcoholism

National Institute of Allergy and

Infectious Diseases

National Institute of Arthritis &

Musculoskeletal &Skin Diseases

National Institute of BiomedicalImaging and

Bioengineering

National Institute of Child Health and

Human Development

National Institute onDeafness and Other

CommunicationDisorders

National Institute of Dental andCraniofacial

Research

National Institute of Diabetes &

Digestive & KidneyDiseases

National Institute on Drug Abuse

National Institute ofEnvironmental

Health Sciences

National Institute of General Medical

Sciences

National Institute of Mental Health

National Institute of Neurological

Disorders and Stroke

Fogarty International

Center

National Library of Medicine

National Institute

of Nursing

Research

National Center forComplementary &

Alternative Medicine

National Center onMinority Health &Health Disparities

National CenterFor Research

Resources

Clinical CenterCenter for InformationTechnology

Center for Scientific Review

About the National Institutes of Health (NIH)

Primary US agency for supporting and conducting biomedical research

Invests > $30B in biomedical research

> 80% funding awarded through nearly 50,000 competitive grants to researchers at 3,000 research institutions

About 10% funding supports 6,000 intramural researchers

Systems Supporting Internal Decision-Making

Providing timely data and analyses in support of evidence-based decisions

Tools supporting monitoring, analysis, and evaluation of NIH-sponsored research portfolios at the portfolio, project, and researcher levels

Examples presented: CareerTrac; e-SPA

Analyses performed with these systems have the potential to be incorporated into publicly available electronic resources

Enterprise-Wide Public Systems

For large scientific enterprises, the aggregation of data across many and varied sponsored programs and projects is a significant challenge in multiple aspects:

Data collection Aggregation Analysis Presentation/access

An example of standard reports and analyses that integrate information on research activities from across the 27 institutes and centers of NIH will be presented:  NIH RePORT Website

An example that integrates information on research activities from across the 27 institutes and centers of NIH will be presented:  NIH RePORT Expenditures and Results (RePORTER) system

Sample Questions Addressed

What are the career outcomes of individuals trained through Fogarty International Center programs? [CareerTrac]

What is the quality of the research supported through NIH infectious disease programs? [e-SPA]

What are the trends in characteristics of the biomedical research workforce? [RePORT]

What projects did NIH fund in 2008 in the area of…  and what findings have been published? [RePORTER]

8

NIH Reporting Data Systems

Program Information

e-SPA RePORT DatabaseCareerTrac

External Databases

Research Results

Projects and People

FIRST

NIH Guide

USPTO

RCDC

IMPAC II