Architecture of the EM

and some key

experience from

abroad

www.metodika.reformy-msmt.cz

Road map

• Making sure we all start our discussion in the same place

• The architecture of the new Evaluation Method

• The funding principles

• Scenarios and their potential effects

• The community’s comments

• Overview of comments

• Our reflections based on the comments

• Discussion

• Breakouts

• A. The Evaluation Methodology, its added value and role

• B. Stabilization and motivation roles of institutional funding

• C. How can – and should – the Evaluation and Funding System foster change in the

Czech RD&I system?

• Reports from breakout groups to the plenary

2

Policy development and implementation are at the heart of the new methodology

3

Assessment process

Funding process

Policy development

Some key principles

• Only research-performing research units should be included

• Implicitly, research units are only included if some part of the state is willing to provide

institutional funding

• Central role of ‘informed’ peer/panel review

• Common assessment ‘spine’ with field-specific variations, allowing

aggregation of assessment results (eg at field or institutional level)

• Common assessment criteria covering performance in 5 different dimensions

– no expectation that everyone does well against all criteria

• Translation of performance into funding is based on the societal function of

the research unit

• Institutional funding contains elements that

• Provide a substantial measure of funding stability

• Reward the quality and impact of the research unit

• Encourage strategic development of the research unit

4

Building up from the fundamental unit of assessment: the Research Unit

5

Links from assessment to policy and institutional feedback

6

The use of peers ties the system to the field level

7

Overview of the funding system

8

RUs

Entry criteria

Assessment

Agreement

Automatic on renewal

PRFS

Performance Agreement

Block Grant

Process Funding

PRFS at a more detailed level

9

• -----------• -----------• -----------

• -----------• -----------• -----------

• -----------• -----------• -----------

• -----------• -----------• -----------

Management & potential

Membership of research

community

Research excellence

Research performance

Information Peer review

• Star rating• Strategic feedback

• Star rating• Strategic feedback

• Star rating• Strategic feedback

• Star rating• Strategic feedback

• -----------• -----------• -----------

Societal relevance

• Star rating• Strategic feedback

Money ‘pot’ per RO type

% % % %

% % % %

% % % %

% % % %

% % % %

Institutional research

funding per evaluated

RU

Where does the money come from?

10

PRFS

Performance Agreement

Block Grant

‘Owning’ ministry

Pot1 Pot2 Pot3 Pot4

‘Owning’ ministry

Groups of owning ministries

Institutional funding is normally grounded in the economics of research• In economic theory, knowledge is ‘non-rival’ in the sense that it can be

used by many people without being consumed

• If I make a cake and you eat it, I cannot consume it

• But if I make some knowledge, we can both use it

• A rare case where you can have your cake and eat it?

• Knowledge is ‘non-excludable’ in the sense that it is difficult to prevent

people who want it from getting hold of it

• Non-rival, non-excludable goods are ‘public goods’. They cannot be

produced by the market so the state must make them

• In economic reality, however, there is imperfect information, path-

dependency and costs to acquire and use knowledge

• It appears even more costly to absorb science than technology

• Knowledge is useless without know-how

11

The rate of subsidy is typically consistent with the degree of spillover

12

Basic Applied Experimental Development

Applications Engineering …

Market FailureSpilloversRisk

Applicability

Subsidy rate in state intervention

These ideas about the economics of knowledge drive funding behaviour• Actual rates of institutional funding do vary among countries – but the

broad principle implied by the previous slide is followed

• Public service research performers generally have high rates of

‘subsidy’

• They produce public goods, for which there are few other producers

• They produce information for policy and regulation, for which there are few other

customers than the state

• All types of RO are increasingly becoming ‘hybrid organisations’, so as

ever reality is more complex than theory

• States generally act as ‘intelligent customers’ when they supply

institutional funding – making needs-based buying decisions

themselves, rather than allowing ROs to qualify for such funding

bottom up

13

14

Conventional governance of the research and innovation system

R&D Institutes

Parliament

Government Policy council

Ministry of Education

Research Councils and Academies

Universities

Other Sectoral Ministries

Producers: Firms, farms, hospitals, etc

Ministry of Industry

Technology & Innovation Agencies

Support Programme Agencies

Programme Contractors

Instructions, resourcesAdviceResultsHorizontal co-ordination and integration

Level 1High-level cross-cutting policy

Level 2Ministry mission-centred co-ordination

Level 3Detailed policy development, co-ordination

Level 4Research and innovation performers

Key

• Governance relies on a ‘sector’ principle – where individual ministries

address the needs of specific ‘sectors’ of society – including ministries’

own research needs

• This system of principals and agents embodies various types of

‘performance agreements’

• Generally these are becoming increasingly explicit

• Based on management by objectives, as opposed to micro-management

• Ministries may choose to share policy implementation (for example, by

using multi-principal agents) but need to allocate their own resources

• Ensuring their own missions are performed

• Requiring the ministries to be ‘intelligent customers’ and to have significant

policymaking capacity

• In general, funding streams and the responsibility for assessment are

clearly separated, especially among different types of RO

15

Děkujeme za pozornost!

www.metodika.reformy-msmt.cz