IRD/Savoirs et Développement 1

Evaluation of Research Systems

Rigas ArvanitisRoland Waast

ESTIME Coordinating team

IRD/Savoirs et Développement 2

The best promotion of research relies on enhancing quality

Morocco has multiplied its scientific production by four in 10 years….

Many South American countries have improved the efficiency of their research….

In all cases, research has been evaluated at the national level and quality has been enhanced.

IRD/Savoirs et Développement 3

Example: MoroccoEvolution de la production scientifique: 5 pays d'Afrique, 1987-

2001

0

100

200

300

400

500

600

700

800

900

1000

1987 1988 1989 1990 1991 1992 1993 1994 1995 1996 1997 1998 1999 2000

Années

Maroc

Nigeria

Tunisie

Kenya

Algérie

IRD/Savoirs et Développement 4

Tools for the evaluation

Bibliometrics : general and detailed Surveys by questionnaires Field work Evaluation by foreign experts

IRD/Savoirs et Développement 5

Evaluation is a process...

Phase 1 :Preparation : bibliometrics / Information gathering / budgets, personnel, higher education

Phase 2: Preliminary analysis : specific topics/fieldwork/questionnaires

Phase 3: Expert selection Phase 4: Expert evaluation Phase 5: Restitution at local and national level

IRD/Savoirs et Développement 6

...that needs independent and professional advice The team needs to be independent from

political interference and coordinated with national and local authorities

The process needs to be transparent Results need to be fully discussed

Undesirable results need also to be discussed

IRD/Savoirs et Développement 7

Evaluation needs some basic conditions to be fulfilled

Political willingness Openness to foreign eyes A clearly defined budget A small operational team Coordination, not imposition

IRD/Savoirs et Développement 8

Expected outputs –1

A quasi exhaustive review of the research system at the national and regional levelBy disciplinesBy institutions/at the level of departments and

laboratoriesBy geographical location

IRD/Savoirs et Développement 9

Expected outputs –2

A description of institutional capabilities InfrastructuresBudgets and fundingSize and organization International collaborationsCapabilities to produce and circulate

knowledge Capabilities for use of research

IRD/Savoirs et Développement 10

Expected outputs –3

A quality assessment by scientific experts Internationally recognized experts National experts as associates Experts with a view on uses of research With no previous linkages with the national teams Accepted by University authorities and research

heads Assessment made by on site visits to labs

IRD/Savoirs et Développement 11

Expected outputs –4

A wide discussion of the quality assessment by scientific experts and other results As open as possible Including as many experts as possible With no previous linkages with the national teams Assessment made by on site visits to labs and

meetings

IRD/Savoirs et Développement 12

Benefits for research

The quality of the evaluation will reveal strong points and shortcomings

The implication of all parts of the research system will permit to reveal ways to enhance research

The large discussions will validate results and suggest actions

IRD/Savoirs et Développement 13

Benefits for further action

The policy process will benefit from first class information, socially acceptedAccepted national programmesA certification process for high achieving labs

and teamsAn accepted identification of priority actions