Research Management for Selectivity and Concentration - With What Effect?

@ Basil Blackwell Ltd. 1993, 108 Cowley Road, Oxford OX4 IJF, UK and 238 Main Street, Cambridge, MA 02142, USA. Higher Education Quarlerly 095 1-5224 Volume 47 No. 1, Winter 1993

Research Management for Selectivity and Concentration - With What Effect? Bruce Williams, Centre for Higher Education Studies, Institute of Education, University of London

Abstract

There have been three research assessment exercises in British universities in 1986, 1989 and 1992. Average scores have risen over the period which is usually interpreted as showing how research performance is improving. However, it also means that universities with the highest ratings progressively receive a smaller share of the total research funds available. In 1992 over a hundred additional institutions were included in the exercise and this further diluted the research allocations.

The Robbins Committee on Higher Education and the Trend Committee on the Civil Service both reported in 1963 and both viewed the functions of teaching and research as so organically interwoven that it would be disastrous to separate them for purposes of administration. Yet within 20 years the view that they should be separated ceased to be novel, and indeed the UGC pressed each university to separate them.

The five per cent reduction in the staffktudent ratio in the second half of the 1960s created fears for the future of research, and both the rise in the proportion of university research financed by the Research Councils, charities and government departments on a less-than-full cost basis, and their growing tendency to concentrate their grants for work in fields which they judged important, increased the need for specific internal research policies. After the severe and selective reduction in UGC block grants announced in July 1981, the UGC and the ABRC (Advisory Board for the Research Councils) appointed the Merrison committee, which in 1982 recommended that universities should concentrate research in selected areas and create new mechanisms to cope with the problems of selectivity and concentration. In 1983 in response to pressure from the Secretary of

Research Management for Selectivity and Concentration 5

State, the UGC addressed 29 questions to the universities, which in addition to questions on dual funding and earmarked research grants, asked whether there was a need for greater selectivity of research grants to universities and within them. In 1984 the UGC informed the Secretary of State that it planned to develop a more systematic and selective approach to the allocation of funds for research, and that for success Universities would need to adopt complementary research management strategies.

In 1985 the UGC asked universities to submit detailed research plans to 1990 within a framework of seventeen possible subject areas, and objective evidence of research quality. Sub-committees of the UGC then gave a research rating for cost centres in each university. The above average cost centres, or units with them, thought to be outstanding were starred. No university was assessed as better than average in all cost centres and only one had no cost centres below average. The distribution of the ratings are shown in Table 1. Seventy-eight per cent of the starred cost centres or parts of cost centres were in the top third of the universities, 17 per cent in the middle third and 5 per cent in the lowest third.

For the first allocation of funds under the new system, 61 per cent was allocated overall for teaching and 39 per cent for research. The major factors in the allocation from that research fund were student numbers and research ratings - each roughly 37 per cent of the research component. Another 13 per cent was based on the size of grants from research councils and charities, a little less than 2 per cent based on success in attracting research contracts from industry and others who were supposed to meet full cost, and a further 10 per cent was used to provide for special factors - mostly in the London institutions - ‘not necessarily merit-related’.

The 1986 research ratings exercise was rather hurried. Varying scales and standards were used in different subject areas, and in some cases the published ratings had been statistically normalised. For the 1989 ratings, which presumably would be influenced by the UGC reallocation of funds in favour of the universities with the highest research ratings, it was decided to use not a relative but a ‘criterion-referenced’ numerical scale for

Table I 1986 Research Assessments

Universities YO below average YO average YO above average

Top third 14 Middle third 29 Bottom third 54

30 42 32

56 28 14

0 Basil Blackwell Ltd. 1993

6 Higher Education Quarterly

research quality in cost centres of (l), when not up to attainable levels of national excellence (ALNE) in any or virtually any of their sub areas of activities, (2) for up to ALNE in up to half their sub-areas of activity, (3) for up to ALNE in a majority of their activities, (4) for up to ALNE in virtually all their activities, and possibly some evidence of international excellence, and ( 5 ) for attainable levels of international levels in some, and up to ALNE in virtually all, other activities.

The results of the 1989 ratings are shown in Table 2. The universities in Group 1 attained 65 per cent or more of their maximum possible score, i.e. ratings of 5 in each of their cost centres. Those in Group 2 attained between 64 per cent and 55 per cent and those in Group 3 between 54 per cent and 40 per cent.

If, for the sake of a very crude coinparison between 1986 and 1989, ratings 1 and 2 are treated as ‘below average’, and ratings 4 and 5 as ‘above average’, and groups 1, 2 and 3 in Table 2 are assumed - not quite correctly - to be the same as the top third, middle third and lower third in Table 1, the main differences are that in Group 1 the percentages below average fell from 14 to 10, and above average rose from 56 to 63, while in Group 3 the above average percentages fell from 14 to 8 with a corresponding rise in the percentage in the average capacity.

Responses

The UGC made some reallocation of funds in favour of the universities judged to be best at research. Both the (division of the block grant from the UGC between teaching and research and the division of research funds between the centres with their different research ratings were matters for each university. There were however incentives for them to follow the UGC policy of greater selectivity in the allocation of funds and the concentration on strengths. The first incentive was the role of the research ratings and success in gaining research grants from research councils and charities in the UGC’s allocation of funds and the announcement that in

Table 2 1989 Research Ratings

Percentages of cost centres in categories 1-5

Universities in: 1 2 3 4 5

Group 1 1 9 27 32 31 Group 2 7 23 41 21 8 Group 3 19 36 37 7 1

@ Basil Blackwell Ltd. 1993


later years the proportion of the total grant based on research ratings would rise at the expense of the proportion based on student numbers. A second incentive was the requirement to report annually to the UGC on arrangements made for the selective distribution of research resources as proposed by the Merrison Committee. The UGC had also given a clear indication of its view on the relation between size and efficiency when, following Governments financial restrictions, it decided in 198 1 to make selective reductions in grants, to give financial assistance for the closure of low-grade departments, and then to arrange the transfer of staff between universities in fields where it judged there were too many departments. This influenced the preparations of universities for the 1989 and 1992 research reviews.

The UGC also appointed subject review committees. In 1988 that in Physics recommended a concentration of resources in departments of at very least 20 full-time-equivalent academic staff implying ‘the loss of free- standing physics teaching and research in perhaps 15 to 20 or so universities’, and the committee in Chemistry also set 20 academic staff as the minimum size for self-standing departments in Chemistry and recommended that 20 of its proposed 30 departments - there were at that time over 50 - have 30 or more academic staff. The report on earth sciences published in 1987 proposed a concentration of research facilities and opportunities in about 10 large earth sciences departments - there were then about 40 - and that of the other departments some concentrate on teaching on MSc level, and other to first degree level or less. In the same month the ABRC, in A Strategy for the Science Base, proposed three types of higher education institution - R with a substantial range of research activities, X with substantial research activities in particular fields, and T without advanced research facilities. These reports had a considerable influence on the way universities prepared for the 1992 review.

Rational decisions on the allocation of UGC block grants or UFC grants for research did not always lead to the ‘concentration on strengths’. Possible responses include decisions:

(i)

(ii)

to close weak departments in line with Secretary of State and UGC views on rationalization at the time of the first review; to reduce staffktudent ratios and supporting staff/academic ratios in the low-rated departments, and transfer staff to the high-rated departments; (after the 1989 exercise) to let the allocation of resources to each cost centre reflect the UFC calculations;

(iii)

0 Ban” Blackwell Ltd. 1993

Higher Education Quarterly

(influenced by the ABRC report in 1983 on the need for more multi-disciplinary research, and the later subject reviews and the ABRC’s Srrategy for the Science Base) to bring together staff in closely related or complementary fields to create more strong research groups and/or more multi-disciplinary research; (after the 1989 exercise) to aim to raise as far as possible the percentage of their total score at the next review and to reduce the risk of becoming an X university by giving more emphasis to raising the low ratings towards the middle than to raising the middle ratings towards the top; to pay more attention to potential than to past performance by retiring staff in centres with poor records and attracting staff with good research records and research council grants and a capacity for leadership with promises of more supporting staff and equipment; to pay more attention to fields of research currently showing most promise of exciting results by making a substantial part of the UFC research grant available for competitive bidding within the university;

(viii) to require all cost centres to submit research plans involving all members of academic staff, put pressure on staff with a poor research record but research potential to become members of research teams, press staff with seemingly little potential in research but otherwise capable to do more teaching and administration, and press capable but excessively perfectionist staff to reduce their periods of production.

Responses i-iv involve, to a greater or lesser degree, a greater concentration of resources, but responses v-viii do not. In the inquiry that I conducted in 1990 in a sample of universities I found examples of all these responses. There were many signs that the pressures on universities from the Secretary of State and the UGC to make better use of resources through attention to selectivity and rationalization were likely to lift overall research performance. The results of my inquiry were given in University Responses to Research Selectivity, published by the Centre for Higher Education Studies in 1991.

The 1992 research rankings

The description of the performances required in the 1992 review for grades 1 to 5 were the same for the 1989 exercise. At first sight that

@ Basil Blackwell Ltd. 1993


suggested that there would be a better opportunity to judge the effects of the UGCiUFC policies for selectivity and concentration by comparing the results for 1989 and 1992 than by comparing the results of the 1986 and 1989 exercises. Regrettably, as we shall see, that was far from so.

A comparison of the rankings in 1989 and 1992 is shown in Table 3. The basis of groups 1, 2 and 3 was explained earlier. Although there was a change in the relative performance of universities, the universities that were in groups 1, 2 and 3 in 1989 were kept in those groups for comparative purposes.

In the Group 1 universities there was a small increase in ratings, in Group 2 universities a marked increase - a reduction in the percentage of 1 and 2 ratings from 30 to 13 and an increase in the percentage of 4 and 5 ratings from 29 to 48 - and in Group 3 universities there was a still more marked increase - a reduction in the percentage of 1 and 2 ratings from 55 to 25 and an increase in the percentage of 4 and 5 ratings from 8 to 26. In Group 1 universities, the lowest percentage of a maximum score rose from 65 to 70, in Group 2 the lowest percentage rose from 55 to 61, with three quarters scoring more than 65 per cent, and all Group 3 universities scored more than 55 per cent which was the qualifying score for Group 2 in 1989; 18 per cent were just under the 1989 qualifying score for Group 1.

According to a report in the Times of 18 December 1992, the Chairman of the Vice Chancellor’s Committee welcomed the exercise as evidence that the traditional universities had improved by an average of half a point out of five since the last assessment, and the acting General Secretary of the Association of University Teachers hailed the assessment exercise as a celebration of achievement, with ratings improving since 1989 despite a real fall in research funding. Before drawing conclusions from a comparison of 1989 and 1992 results an attempt should be made to allow for the changes in the nature of the research assessment exercise.

Table 3 Research Rankings for the same grouped Universities in 1989 and 1992

1 2 3 4 5 ~~~~

Group 1 1989 1 9 27 32 31 1992 1 6 28 31 34

Group2 1989 7 23 41 21 8 1992 1 12 39 34 14

Group 3 1989 19 36 37 7 1 1992 3 22 49 19 7

@ Basil Blackwell Lrd. 1993


The effect of ending the binary system

Under the binary system, universities were financed for teaching and research, and the other institutions of higher education for teaching. The Education Reform Act 1988 created a Universities Funding Council to replace the University Grants Committee, and, reflecting a major change in the roles of Local Authorities and Central Government, a Polytechnics and Colleges Funding Council (PCFC) replaced the National Advisory Body for Local Authority Higher Education. The PCFC was given power to make grants for research. In 198911990 the UFC research grants to the universities were &860 million and PCFC research grants to its institutions were &20 million. The sum of all research grants and contracts to universities was 21,620 million and to the polytechnics and colleges &70 million. In May 1991 the Government announced that it had decided to abolish the binary line in higher education and that all institutions of higher education would be able to compete for research funds. The UFC in circular 5/92 listed just under 200 higher education institutions. There was therefore the possibility that in 1992 almost 200 institutions would be covered by the research assessment exercise instead of 55 in 1989. In the event 170 institutions were asssessed, and that great increase in the number of institutions without a strong tradition of research inevitably influenced the nature of the exercise.

Circular 5/92 on the research assessment exercise for 1992 listed the following significant changes:

(i) to ensure full recognition of the work directly relevant to the needs of commerce, industry, and the public and voluntary sectors, panels in science and engineering were asked to assign separate ratings for applied as well as for basicktrategic research, and to take into account ‘the extent of participation in other technology transfer activities such as the Teaching Company Scheme’;

(ii) whereas in the 1989 exercise information was sought on all academic staff, for the 1992 exercise the institutions were required to put forward for assessment only those ‘actively engaged in research’ ;

(iii) institutions were advised when considering whether or not to take part in the exercise that they should expect that no research allocations would be given to departments with a rating of only 1.

Sir Peter Swinnerton-Dyer stated in his Rede Lecture for 1991 that there had been ‘a systematic bias against departments that do very applied

@ Basil Bhckwell Ltd. 1993


work’, and it was inevitable that with the inclusion of polytechnics in the 1992 exercise an effort would be made to overcome that bias. It is not known how much the change influenced the ratings. The universities in the 1989 exercise most likely to profit from the change were the former colleges of technology and central institutions. All but two of them were in Group 3 and their ratings increased significantly more than those of the other universities in the Group, as did also the two in Group 2.

Freedom to choose the staff to be assessed had its greatest effect on the institutions not included in the 1989 exercise, but the freedom was exercised by many of the others. Several of the former CATS and central institutions submitted less than 90 per cent of their staff, but so too did several colleges of the University of London and three universities in Scotland. It is not possible to calculate how much that exercise of choice influenced the ratings, but doubtless the institutions expected it to increase their ratings. The three universities in Scotland achieved a substantial improvement in their ratings.

The announcement that institutions should not expect ratings of 1 to qualify departments for research grants, when taken together with the addition of many institutions without any known record of significant research, I thought was certain to produce a reduction in the percentage of rankings of 1 and 2 in the institutions in the 1989 exercise. Following my inquiries in a sample of universities in 1990 into preparations for the 1992 exercise, I expected that the much greater emphasis on measures to raise research ratings would lead to a considerable fall in ratings 1 and 2 in both groups 2 and 3, but - apart from the effects of the changes in the nature of the exercise - a much smaller change occurred.

The view of the Chairman of the Funding Council for England - as reported in The Times of 18 December 1992 - is that a more selective approach to research within universities has raised standards. That view cannot be fairly based on the results of the review, though it accords with the view I formed during my investigations in 1990. It then seemed clear also that the new system had increased research activity by putting pressure on staff to perform better, and on universities to be less willing to wait for the retirement of professors and other senior staff who had ceased to be productive and provide leadership in research activities. I made a list of cost centres in which I expected new leadership would lead to significant increases in ratings. There were such increases, but the 1991 decision to end the binary system made it impossible to be certain of the extent of the increase due to the changes in research management.

@Basil Blackwell Ltd. 1993


The effects on the allocation of research grants

The allocations made following the 1989 research assessments disappointed several institutions with high research ratings. They had expected to gain more than they did. The problem is that a rise in average ratings reduces the share of those whose ratings did not rise - most likely for those with the very high scores in the last exercise - unless the sums available for distribution on the basis of research ratings increase sufficiently. That requires an increase in the total of available research funds, or an increase in the proportion of the research funds allocated on the basis of research ratings. While that part of the allocation based on ratings was rising relative to the part based on student numbers, there was room for an increase in average ratings that would not reduce the funds available for those with very high ratings. The decision to end the binary system brought with it a decision to reduce further the allocation of research funds based on numbers - ‘the floor provision’ - and therefore greater room for a rise in average ratings without reducing funds to the best performers. But it was associated also with the decision to make many more institutions eligible to apply for funds from the Funding Councils. Could that lead to an inefficient deployment of funds for research?

The distribution of gradings in the institutions assessed in 1989 and 1992 and those such as the Open University, Cranfield, the Scottish Central Institutions and the former PCFC Institutions assessed only in 1992 is shown in Table 4.

To get some idea of the impact of the introduction of the new institutions on research grants, multiply the percentage in columns 1-4 by 1, 2, 3 and 4 respectively. The sum of the first row is then 255 and the of the second 135. The assessed academic staff in cost centres with ratings greater than 1 was approximately 6.5 times greater in the pre-1991 universities. Multiplying 255 by 6.5 gives 1657.5, which implies - subject to a further qualification - that the research allocation to the new

Table 4 Percentage of Grades in Categories 2-5

-

Institution 2 3 4 5

Assessed in 1989 & 1992 13 38 30 19 Assessed only in 1992 72 23 4 1

@ Basil Blackwell Lid. 1993


institutions will be 7-8 per cent of the total. The qualification is that of the eligible units of assessment in the new institutions, only 40 per cent of those with ratings of 2, 30 per cent of those with ratings of 3 and 20 per cent of those with ratings of 4 and 5 were in the sciences and technologies. In other words, the main allocations to the new institutions will be in cost centres where the WT ratios are low or relatively low, so bringing the prospective research allocations below 7-8 per cent, apart from ‘encouragement’ grant.

At the time of writing it was known that the element of ‘encouragement’ will be 2.6 per cent of the total, but it was not known whether the volume measure will include in addition to the assessed academic staff the postgraduate research assistants and fellows and the resources secured from Research Councils and charities.

There is another unknown which stands in the way of estimating the likely effects of the new system. That arises from a problem in allocation that would have arisen had the binary system not been abolished. That is the demonstrated capacity of the pre-1991 universities to raise the ratings of cost centres below ratings of 4 and 5, leading to a threat to the grants of the high rated centres unless the grant based on assessments is increasing sufficiently. It will increase on this occasion because of the abolition of the floor provision for research and the likelihood that the grants to the new institutions - including the development grant - will be less than 10 per cent of the total. The decision to consider only ratings 2-5 had added to the problem of catch-up. Doubtless an extension of the scale will be considered before the next review. But how well to reward 5 ratings relative to 4 ratings, 4 relative to 3 and 3 relative to 2 will presumably be treated as an important issue in current decisions on the allocation of grants.

One of the most significant aspects of the 1992 ratings was the performance of the newly assessed institutions despite the generally low levels of research grants and contracts. Of the units submitted for assessment 43 per cent were assessed as 1,41 per cent as 2 , 13 per cent as 3, 3 per cent as 4 or 5 compared to 2 per cent, 13 per cent, 38 per cent and 47 per cent respectively in the pre-1991 universities. Had there not been choice in the units, and in the percentage of staff in them, submitted for assessment, the performance of the newly assessed institutions would have looked much less impressive. For the most part the best performances were in fields that do not require very expensive equipment in addition to ‘teaching equipment’, or high ratios of support staff to academic staff. The totals of assessed staff in units given ratings of 4 or 5 , were just over 100 in art and design, 70 in education (all in the Open University) just under 50

0 Basil Blackwell Ltd. 1993


in mechanical, aeronautical and manufacturing engineering (all in Cranfield) and in town and country planning, and 20 in the built environment (all in Oxford Brookes University). The total of assessed staff in units with ratings of 3 were predominantly in education (about 300), art and design (just over 250), and then in descending order between 75 and 20 were history of art, town and country planning, applied maths, sociology, electrical engineering, political and international studies, business management (Cranfield), English language and literature, computer science, mining (Cambourne), geography, psychology, and social and political studies. Most of these staff were engaged in activities where teaching and research or scholarship if not ‘inextricably interwoven’ are certainly closely related.

Addendum: The Research Allocations

The Higher Education Funding Council for England published its list of research grants after the paper above had been sent for publication. There were widespread expectations of major changes in the distribution of research grants as a consequence of a further transfer of funds from the University Funding Council to the Research Councils, and particularly, following the abolition of the binary line, the opening of research assessments to the whole of the higher education sector, the ending of the floor provision for research previously assured by SR grants, the restriction of research grants to departments or cost centres with research ratings of more than 1, and the introduction of development grants for new universities. There were major changes.

The abolition of DR grants had an equalizing effect on the research grants distributed by the University Funding Council (although because considerably more than the DR grants in 1991/92 have been transferred from the University Funding Councils to the Research Councils the overall effect is that those universities most successful in getting Research Council grants have gained at the expense of the less successful).

The abolition of SR grants has the effect of increasing the share of those universities which in 1991/92 had a high ratio of JR to SR grants. That high ratio may have been due to high research ratings and/or to above average percentages of staff in fields where the ratios of R to T grants are high, as in the technologies and experimental sciences.

Making departments with research ratings of less than 2 ineligible for research grants of course reduces the share of universities with departments or centres with research ratings of 1. However, the potential of that change to bring a greater concentration of research grants was much

@Basil Blackwell Ltd. 1993


reduced by the sharp reduction between 1989 and 1992 in ratings of 1 in pre- 1991 universities.

In both 1989 and 1992 the ratings scale of 1-5 was used. The rise in average ratings from around 3.28 to 3.76 has the effect of reducing differences between universities in the allocation of research grants. Such an effect may of course be offset or more than offset by giving higher ratings a more than proportionate share of grants. If there were four universities of equal size and subject distribution with ratings of 2, 3, 4 and 5, grants based directly on those ratings would give the university with the 2 rating, 14.3 per cent of the grant (2 -+ 14), and the others 21.4 per cent, 28.6 per cent and 35.7 per cent respectively. But suppose for purposes of allocation, 3 were raised by 10 per cent, 4 by 20 per cent and 5 by 30 per cent. The percentage allocations would then be 12, 19.9, 28.9 and 39.2 respectively. The share of university 1 would fall by 16 per cent and that of university 4 would rise by 10 per cent.

In 1991192 British universities received research grants of 5551 million from the UFC. Those same universities have been allocated 5566 million for 1993/94. By comparison with 1991192, the gains and losses in the percentage share of grants for 1993-94 range from +9 per cent for Cambridge to -29 per cent for Brunel. Six universities - Cambridge, Manchester, Exeter (including Cambourne School), Oxford and Warwick - gained between 9 and 5 per cent, and six - Bath, Birmingham, Durham, East Anglia, Nottingham and Lancaster - between 4 and 2 per cent. For eight universities - London (as a whole), UMIST, Essex, Sussex, York, Reading, City and Keele - there was not a significant change in shares. The share of grants for 4 universities - Southampton, Leicester, Newcastle and Liverpool - declined by between 2 and 5 per cent, and the share of the other five - Kent, Sheffield, Bristol, Surrey and Hull - declined by between 6 and 9 per cent. Five - Loughborough, Bradford, Salford, Aston and Brunel - declined in share by between 10 and 29 per cent. All five universities were expected to benefit from the greater recognition of applied research. They did make significant increases in their research rankings but that effect was offset by the general increase in rankings.

There has been a considerable redistribution of grants between the pre- 1991 universities despite a degree of capping and safety netting. What of the effect of the research grants to new institutions? Those grants have reduced the share of the pre-1991 institutions in the total grant of 5618 million, but how far their command over research resources has declined depends on how far the grant of &52 million - 8.4 per cent of the total - to the new institutions, of which S12.5 million is for development grants, was

0 Basil Blackwell Lrd. 1993


covered by new money. When allowance is made for PCFC research grants to its institution and the implied research grants from the DFE to the Open University, Cranfield and the Royal College of Art, up to three- fifths of the &52 million may have been transfers to HECFE. But there is as yet not enough published information to estimate just how far the abolition of the binary line has run counter to the declared policy of ‘selectivity and concentration’.

@Basil BIackurell Lrd. 1993

Research Management for Selectivity and Concentration - With What Effect?

Documents

Transcript of Research Management for Selectivity and Concentration - With What Effect?