Research Assessment Exercise 2005 (RAE2005) University of Helsinki Arto Mustajoki Based on the...
-
Upload
paulina-elliott -
Category
Documents
-
view
213 -
download
0
Transcript of Research Assessment Exercise 2005 (RAE2005) University of Helsinki Arto Mustajoki Based on the...
Research Assessment Exercise 2005(RAE2005)
University of Helsinki
Arto Mustajoki Based on the material of
Vice-Rector Marja Makarow
2
Assessment of university research
University of Helsinki (UH) University Senate’s decision RAE of UH every 6th year 1st time in 1999 (first in Finland) 2nd time in 2005
University of Tampere 2004 University of Jyväskylä 2005
Concept different from UK RAE Carried out by UK Council, every 8th year Peer review, no site visits Important financial consequences
3
Why a RAE at the UH?
Current challenges for universities in Finland Profiling Strategic spearheads of research International competitiveness Productivity progam Financial constraints
How to best meet the challenges Knowledge of strengths, weaknesses and potential External evaluation yields
solid objective data on quality of researchrecommendations for the future
4
What was evaluated?
Quality of research of departments Grade 1-7 (7 is best) Verbal arguments
Concepts of institutes, research networks and stations Only verbal arguments No grades
Interaction with society Only verbal arguments No grades
5
What was evaluated?1 Quality of research
Quality of research compared to that of SIMILAR
EUROPEAN INSTITUTIONS Expressed with grades from 1 to 7 Substantiated with verbal comments
NO comparison to results of UH REA 1999 NO comparison between UH departments
6
Target of assessment
All research performed in UH during 1999-2004
Units of assessment Faculty departments: 70 in 11 faculties Research institutes independent of faculties: 5
Individual researchers or research groups were NOT
evaluated
7
What was evaluated?2 Concepts
Verbal evaluation only of Research (field) stations: 5 Research networks: 5 Independent institutes: 5
Verbal evaluation of quality of research: Medical research programmes: 6
8
What was evaluated?3 The third task – interaction with society
Pilot project Legal obligation of universities since Aug 2005 Documentation of accomplishments not systematic
For example Expert tasks, popularization of science, text books,
clinical and commercial application of research data Units of assessment chose what to report Evaluators commented verbally The data serves to
Clarify how new knowledge has been tranferred to
be used in society Develop and document activities
9
How was the assessment carried out?
Peer review Documents on acitivities of 94 units of assessment
covering 1999-2004 via Evaluation Office to evaluators Desk work at home Preliminary drafting of Evaluation Report
Each panel one working week in Helsinki General info on Finnish science policy, university
system, UH Site visits to premises, infrastructure Interviews, researchers, PhD students, post-docs
Writing of Evaluation Report before leaving Finalizing of the Report by Panel Chair Editing the Report in Evaluation Office
10
Who were the evaluators?
148 mostly international scientific experts, in 21 panels From 21 countries 83% from Europe 9% from Finland 12% were there in 1999 18% from LERU universities 30% of panelists and chairs were women
Panelists were chosen from suggestions obtained from Scientific Council of UH Chairs of national Research Councils Rectors of LERU universities Intl top scientists
11
Research active staff (RAS) in 1999-2004
Researchers, post-docs and PhD students 4,000 annual work years Results of altogether 24,000 researchers’ work years was
evaluated
Number of RAS per unit of assessment: 3 - 640 Humanities and social sciences: 3-76 Natural science: 14-154 Medicine 24-640 Agriculture, Forestry, Bioscience, Pharmacy 7-340
12
Publications in 1999-2004
Only publications in the official data base of UH were
encluded in the assessment Altogether 60,000 publications
21,000 peer reviewed publications 22,000 other publications 2,400 monographies 2,000 PhD theses 10,000 popularized publications 600 text books
13
Competitive funding fetched by the researchers in 1999-2004 (kiloEUR)
Research Councils (Academy of Finland) 220.000 Ntl Technology Agency (Tekes) 61.000 Ministries 127.000 Ntl foundations 37.000 Intl foundations 27.000 EU FWPs 49.000
TOTAL 521 M€
14
Results1 Grades of the quality of research
Average 1999 4,66
Average 2005 5,8
15
Results3 Summary
From the 75 units 66 were evaluated also in 1999. From them
29 (44%) improved their grade 31 (47%) got the same grade 6 (9%) got a lower grade
20 units (27%) got the best grade 7 6 units improved from 4 to 6 1 unit improved from 3 to 6
16
Criteria of the grades
7: >50% of submitted works are at high intl level and all
others are at good intl level 6: >33% at high intl level and many others at good intl
level 5: >50% at least at good intl level and others at fair intl
level 4: >33% at good intl level and many others at fair intl level 3: >50% at least at fair intl level 2: >50% at fair intl level 1: none at fair intl level
17
Quality of research increased - why?
Performance of individual researchers and teams
Structural development at university level Concentration of activities to 4 campuses Fusion of departments (115>75)
Evaluation culture adopted
Recommendations of RAE1999 implemented
Background Sufficient national resources for research Intelligent national science policy
18
Principles of financial consequences of results
University Senate’s decision before publication of results
The best units and faculties are rewarded The resources will come from the university’s private
funds, not state budget No unit or faculty looses resources due to poor
performance
19
Rewards to the best units of assessment
Units of assessment 1.600 €/RAS/year will be awardes in 2007-2012
to units which obtained grade 7 1.600 €/RAS/year in 2007-2009
to units which improved to grade 6 from grade 3 or 4 27 units to be rewarded annually with
30.000 - 288.000 €
20
Rewards to the beast faculties
Faculties, rewarded during 2007-2009 6 faculties with the best average grade Amount of reward relative to number of RAS
after deduction of the RAS of units to be awarded
directly
6 faculties to be rewarded annually with 32.000 – 309.000 €
21
Total investment to quality research in 2007-2012
12 M€ to units of assessment 3 M€ to faculties
Total investment 15 M€
22
Evaluators’ comments to leadership
Structures which best support quality research in
universities Independent institutes Research programmes
Collaboration and strategic alliences Infrastructure and its sharing Proactive recruitment of researchers Funding of research Allocation of time for research Researchers’ careers Leadership
23
Governance of RAE2005Director of RAE2005 Vice-Rector for ResearchProf. Marja Makarow
Steering committee (Chair prof. M. Makarow) Prof. A. Mustajoki Director U. Mansikkamäki Mr. H. Kallasvaara
Panels and panelists: Steering committee >Reasearch Council of UH
ToR for evaluators & Guidance for units of assessment: Evaluation Office (K. Haila & R. Holm) > Steering
Committee > Reasearch Council of UH Principles of financial consequences
Research Council of UH Decision by University Senate
24
Publication of results
Duration of RAE procedure from May 2004 to March 2006
Site visits of panels in Helsinki May-June 2005: panels 1-4 September-November 2005: panels 5-21
Publication of results on the web on March 1, 2006 Summary Report (also available in printed fomat) Individual Evaluation Reports
www.helsinki.fi/research2005