Charities, excellence & assessment

Dr Mark Walport

Director

The Wellcome Trust

• how can research charities meet their mission?

• how do we pick winners?

• how do we assess outputs?

• the future

Outline

the RAE has:

• driven excellence

• focussed attention on outputs

but has had unintended consequence:

• increased volume of research at expense of physical infrastructure

• lecturers appointed rather than support staff

• pressure for short term research results

• damaged clinical research capacity

• the work of teams devalued

• teaching devalued

What do universities look like?

RAE1996

RAE2001

RAE1992

*Cancer Research UK was formed in 2002, following the merger of ICRF and Cancer Research Campaign (CRC). Figures before 2002 show the combined spending of ICRF and CRC (Source: Cancer Research, Science and Technology Committee)

The AMRC line shows the total spend on UK medical and health research by AMRC members charities. The figure for 2005 is an estimate based on AMRC subscription data.

Trust funding includes international spend.The Gates Foundation figure only includes grants to institutions in the UK.

0

100

200

300

400

500

600

700

800

1999 2000 2001 2002 2003 2004 2005Financial year

Ex

pe

nd

itu

re (

£ m

illio

n)

Wellcome Trust

Cancer Research UK*

British HeartFoundationGates Foundation(UK grants only)AMRC total

MRC

UK Strength

Plurality of funding for biomedical research

• universities ‘fit for purpose’

• a funding model to promote this

we hope to be able to…

• fund the best researchers

• fund the best ideas

• fund in the best research environments

• get value for money

Charities can meet their missions if…

Picking winnersIdentifying excellence

• are you a good scientist? what has been discovered? who has been trained? how have outputs been communicated?

• what is your question?

• why is it important?

• how will you approach the question?

• what resources do you need?

• robust and rigorous

• more thorough than RAE methods

• international

• but can be risk averse

Peer review of inputs

Charity Research Support Fund• a metric based award mechanism in relation to quality

charitable income

• an explicit volume of QR ring-fenced as partnership fund

• included in block grant to VCs

• merits: directly rewards success rigorous peer review responsive to change between RAEs enables universities to support infrastructure for research in

manner that best suits university

• adoption of true/realistic measures/indicators:- recognition of the incremental & unpredictability of research

- challenge of identifying ‘impact’ from basic research

• avoiding perverse incentives- weighing papers

- citation/impact factors

• managerial vs professional targets- quantitative vs qualitative indicators

• focus on national indicators for an increasingly global pursuit

• attribution - multiple funding partners, international collaboration

Assessing outputs

The issues

• leadership of Professor Ian Donald 1950-60s

• remarkable collaboration between engineers and clinicians

• resistance from doctors who had spent years ‘training their hands to see’

• demonstration, refinement and results led to huge change in clinical practice

Witness Seminars – obstetric ultrasound

‘PubMed 1000’ project

Funds awarded vs. papers produced for the top 15 institutions & trend line

0

10

20

30

40

50

60

70

80

90

100

£0 £50,000,000 £100,000,000 £150,000,000 £200,000,000 £250,000,000 £300,000,000 £350,000,000

Funds awarded 1995-2004

No

of

Pa

pe

rs f

rom

'P

UB

ME

D 1

00

0' s

am

ple

Oxford

Cambridge

UCL

Glasgow

Manchester

KCL

Edinburgh

Imperial

Cardiff

Leed

BirminghamNewcastl

Nottingham

Dundee

Bristol

0

10

20

30

40

50

60

For the record 3 Useful stepforward

5 Major additionto knowledge

7 Landmarkpaper

importance rating

% o

f p

aper

s

Clinical

Mixed

Basic

Importance rating by nature of research

base = 573 original research papers

0.1

1

10

100

Review Scores

Jo

urn

al I

mp

ac

t F

ac

tor

For therecord

3 Useful stepForward

5 Major addition to knowledge

7 Landmarkpaper

Importance assessments of original research papers vs JIFs (log scale)

• very strong biomedical research base in UK

• reward excellence

• drive innovation

• build on successful work of UK CRC

• get the Health Research Fund right

• ensure post-RAE metrics reward national strength in biomedical research

Biomedical research in the UK: The future