Assessing regional engagement and knowledge transfer – ranking or benchmarking?

David Charles, EPRC, University of Strathclyde

KT and engagement

• Qualitatively different to assess than teaching and research

• Not same consensus over idea of quality• Not simply in control of university• Does not indicate institutional excellence• Partly dependent on external demand and

environment• Subjective assessment depending on perspective

Different forms of KT and RE

• Different paths to KT – research exploitation or informal exchange

• KT as codified vs tacit knowledge – who benefits?• Other forms of engagement – cultural, social,

governance relationships etc

• Varied possible forms of excellence, some easier to measure than others

Ranking

• Comparison across diverse activities• No sensible means of weighting activities• Are we assessing university or regional

environment?• Balance of private and community benefit

Simple exploitation measures

• Patents, licences, spin offs, contract income• Discipline-specific opportunities and partly

demand driven• Example of HEBCIS survey in UK, AUTM in US

and Canada• Different rankings of universities for different

indicators

Benchmarking instead of ranking

• Comprehensive set of indicators• Identify areas of strength and weakness• University and partners to decide on prioritisation• Benchmarking with other universities to learn

how to improve those areas seen as important• Differentiation as an objective to better meet

needs of stakeholders

Engagement embedded in university vision and mission

1 2 3 4 5

Vision and mission does

not recognise

engagement as a key role

for the university

Some reference to the

need to engage with

the region is placed in

the vision or mission,

usually in terms of

identifying a regional

community as being of

interest. Vision is

developed from a top-

down position and is

not driving strategy or

seen as an influence

on staff behaviour.

Engagement is a central

element of the vision and

mission and is the result of

a sophisticated debate

within the institution

involving staff from various

levels of the institution.

Engagement is seen as

part of the DNA of the

university and is considered

as important in everything

they do.

Rewarding and valuing engagement

1 2 3 4 5

No staff incentives for

engagement – positive

discrimination against

engagement in

promotions processes

with an emphasis on

research.

Formal recognition of

engagement in

promotions procedures

as one of the areas of

performance that can

be recognised, but

little evidence of it

having major impacts

on behaviour. Little

recognition elsewhere

in the system.

Engagement is

tolerated and possibly

rewarded where

excellence is achieved

but not systematically.

Clear and well

communicated recognition

of engagement in a wide

range of staff policies.

Engagement is supported

through workload and line

management and good

performance is recognised

in promotion and through

salary. Resources are

available to help staff

develop engagement skills

including study leave.

University recognises

scholarship of engagement.

Issues for discussion

• Does it make sense to try and reduce engagement to one or two composite indicators?

• Why do we want to measure engagement, and how does this affect what we try to measure?

• What are the merits of benchmarking approaches that mix output and process indicators?

• Should we focus on mutual learning rather than ranking in this field?