Conclusions of the project Features of PPPs in FPI

MUTUAL LEARNING EXERCISE

CARLOS MARTÍNEZ

BRUSSELS, 19th OCTOBER 2016

Definition

Definition of PPPs in this MLE exercise

• What they are NOT: • Merely project-based collaboration programmes

• Organisations such as research and technology organisations (RTOs, e.g., the Fraunhofer institutes), • They have a market or quasi-market relationship with their customers.

• PPPs in our sense do not have customers: they involve ‘partners’ or ‘members’.

Definition of PPPs in this MLE exercise

• Complex policy instruments

• Involving funding research and innovation activities in partnerships between • the ‘knowledge infrastructure’

• i.e. universities and research institutes

• and ‘producers’ • i.e. mainly companies, but increasingly also state agencies and organisations such as

hospitals, welfare services and so on.

• The companies involved may be public as well as private, • on the assumption that the fact that they are organised as companies and

operate in markets makes them liable to market- and other failures, which can be addressed through STI policy.

System-level issues

Pre-conditions

• the existence of potential industrial partners with competencies and absorptive capacity in the fields

• a well-developed ecosystem

• the extent of technological development

• characteristics of a country in terms of social and cultural capital

• A systems view regards PPPs as part of the overall policy mix of a country

• risk-taking programme thus a certain degree of failure at project level is to be expected.

• A well-designed evaluation system for the PPPs can provide feedback to policy learning

Design

Design of PPPs

• “Design” refers to all the principles and arrangements of a PPP with the purpose to efficiently and effectively fulfil its mission.

• It has to take into consideration aspects of • subject (what?), • actors (who?), • functioning (how?) and • span (how far, for how long, how big, how much), • questions all needed to best address the mission (what for?) • derived from the original drivers (why?).

• Other aspects : • Environmental, regulatory or historical constraints (why so?).

• Design includes basic aspects of implementation and governance.

Design choices

‘Centre’ PPPs

• Organised around physical or virtual research centres to extend traditional research and innovation collaboration between the knowledge infrastructure and companies

• Generate changes in structure and culture within the knowledge infrastructure (mainly universities).

• Define strategies focused on the work of the centre itself, which is executed by a consortium that works at the centre to varying degrees.

Design choices

‘Network’ PPPs

• Focused on a community of firms and research organisations

• Working in concert on a common, innovation-related agenda devised by a branch of industry, a supply chain or another grouping with common economic, thematic and technological interests.

• Constructed to identify and address research and innovation needs across the whole community, so members of the community, outsiders or a mixture may execute their research work.

• There seems to be a trend for network-based PPPs to be on the rise.

Examples: Swedish Strategic Innovation Programmes and Societal Challenge Consortia

Design choices

Mixed model: ‘Competence Centres’ (CREST 2008)

• 1) Strongly based in academia but having industry on their boards • Swedish Competence Centres

• Austrian K+ Centres;

• 2) Virtual centres such as • Some Dutch Leading Technological Institutes; and

• 3) Dedicated ‘physical’ Centres • IMEC in Belgium,

• Dutch Telematics Institute which “aim to get real depth in expertise and facilities as a core support to the development of industry sectors”

OTHER DESIGN ISSUES • Top-down vs bottom-up selection of research areas

• Top-down: Dutch Top Sectors and Finnish SHOKs

• Bottom-up: • Norwegian Centres for Research Based Innovation and

Calls for proposals

• Swedish Strategic Innovation Programmes (the latter, modulated by Gvmt intervention) Consultation process. Modulation of proposals into final projects.

• Mixed: the Swedish competence centres programme • 28 areas. Subsequent calls to fill gaps.

• Principal-Agent relationship • Potential conflict: Norway’s Centres for Research-based Innovation

• Clearly separated roles: Swedish Strategic Innovation Programme

Governance

GOVERNANCE

In PPPs, Governance needs to address:

• a complex set of institutions and actors

• with a range of differing interests

• which need to establish and implement a common plan that is in their own interests, and in those of their funder.

GOVERNANCE ISSUES IN A GLANCE • Adaptability (i.e. Climate KIC)

• Transparency & Accountability

• Efficient governance and management structures

• Balance between the private and public partners

• Selection: number and/or quality of proposals

• Keep the central partners committed

• Influence of political challenges

• Efficient administrative procedures

• Appropriate life-time of an instrument

PPPs – evaluation and impacts

Erik Arnold

Group chairman Technopolis

Prof University of Twente

Visiting Academic, MIoIR

Brussels 19 October 2016

Road map

• Innovation systems

• Implications for PPP evaluation

• Evaluating centre PPPs

• Evaluating network PPPs

• The long-term context

• Methods over time

17

18

PPPs are structural interventions in National Innovation Systems

Source: Stefan Kuhlmann and Erik Arnold, 2000

Political System Education and Research System

Industrial System

Demand Consumers (final demand)

Producers (intermediate demand)

Framework Conditions Financial environment; taxation and incentives; propensity to

innovation and entrepreneurship; mobility …

Large companies

Mature SMEs

New, technology-based firms

Professional education, training

Higher education and research

Public sector research

Government

Governance

RTD policies

Infrastructure

Banking, venture capital IPR, information services Innovation support Standards and norms

Intermediary Institutions

Research institutes Brokers

19

Some implications of the Innovation Systems approach

• Business enterprises are central actors in the system

• Innovation activities are much more than R&D

• Interconnection and interdependence are at the heart of the innovation system concept

• Demand, not just supply, drives innovation systems

• National systems are internationally open

• Innovation systems develop in specific cultures and have unique histories – their institutions co-evolve, so different systems may employ different structures to perform similar functions

• In policy, balance – or ‘mix’ – is key

It’s a system : things are interdependent

• Human resource production – needs to match societal skill needs

• Problem orientation, knowledge production and exchange in the research and higher education sectors need orientation to external needs

• Innovation propensity is partly related to consistent rule of law, IPR regime, competitive regime (‘framework conditions’)

• The thematic focus of the research and higher education sector needs partly to match industry structure and needs

• The character of consumer demand, including propensity to adopt innovations, affect rates of innovation and FDI

• Infrastructures need to be adjusted to enable some types of innovation

20

Implications for system-level PPP evaluation

• Success depends on context as well as the PPP intervention

• Context-dependence undermines the idea of benchmarking

• A PPP has to make sense within the wider policy mix

• Overall indications of the system performance using indicators and other information

• Take into account the considerations that led to the launch of the evaluated programme or, if it is a long-standing programme, changes in its objectives and expected impacts

• Consider other programmes that fulfil objectives close to or similar to the evaluated programme

• Compare programme impacts with close to or similar programmes, if such information is available or can be achieved.

• Consider potentially hindering factors for performance such as institutional features (a lack of or poorly performing institutions

21

The agenda-setting, system-changing character of PPPs demands a meso-level analysis

22

Analysis of system health

Meso-level ‘bottleneck analysis’ and

evaluation

Evaluating programmes and

portfolios

Policy development

Centre PPPs: Competence centre goals, internationally

• Industrially relevant research of a more fundamental kind than is normal in academic-industrial cooperation

• High-quality scientific outputs, often inter-disciplinary

• Scientifically qualified people with industrially relevant skills

• Integrating PhD training into the centres

• Focusing the skills and experience of academic and industrial R&D workers in the scientific and technological domains of the centres

• Critical mass or de-fragmentation within academia in areas of industrial relevance

• Changing research culture

• Encouraging companies to engage in ‘open’ innovation and jointly exploring more fundamental questions than normal

• Encouraging greater interest in and acceptance of the value of industrial collaboration within academia

• Producing innovations in participating companies and spin-outs

23

Some lessons from the international CC literature

• Big economic impacts, over extended periods of time

• Changing research culture in universities and companies

• Key effects result from integrating and changing education

• Producing more industrially usable PhD-holders

• Importance of “sweat equity” (ERCs)

• Governance, balance of power are key to success in centres

• Integrated programmes and centres work best

• Sort out a fair IPR arrangement then get on with your life – the sooner the lawyers are kicked out, the better the centres work

• Behavioural additionality does not conquer the market failure associated with fundamental research – when the high subsidy runs out, the party’s over

24

Hypothesised impacts of the Swedish competence centres

25

Figure 14

Competence Centres

•Public goods

•Knowledge

•Standards

•Labour supply

Universities

•Changed education

•Changed research behaviour and organisation

•New, relevant directions in education and research

Other Companies

•People

•Knowledge spillovers

Participating Companies

•People

•Intermediate knowledge products

•Behavioural changes

•Networks

•Innovations, IPR, etc

•Income

Consumers

•Consumer surplus

26

Sweden example: Groups of competence centre impact on companies

• Direct impacts on industry, through generating directly usable outputs

• Direct impacts through behavioural additionality, including creation of knowledge networks

• Economic impacts on participants

• Economic development of individual SMEs participating in CCs

• Indirect effects through adding to the firms’ stock of internal resources

• Spillovers

• Indirect effects, via the university system

Not much evaluation of Network PPPs has been done yet

• Overall goals are generally competitiveness and innovation-related, sometimes in the context of a grand challenge

• Agenda-setting, community reinforcement and other network goals are important – reform or restructuring of individual institutions is not (unlike in centre PPPs)

• Evaluative attention is currently on strategy, start-up activities and learning

• Evaluation findings therefore tend to focus on quality-assurance, and validating and improving the organisational models used to run the PPPs

27

PPP evaluation issues need eventually to be considered in a long-term context

Impacts

• Short- and long-term innovation impact

• Economic impact

• Impact on institutions, capacity

• Effects on education

• Knowledge Value Collectives

• Agenda setting, focusing devices

Other issues

• Governance

• Learning curve

• Inherent conservatism of the instrument

• Programming

• Top-down, bottom-up

• Role in instrument portfolio

28

PPPs’ long lives and restructuring tasks need to be reflected in evaluation: from formative to summative

1. Early: is this PPP working?

1. Does it conform to the programme model?

2. Does it have the right governance and processes in place?

3. Is it equipped to produce and maintain quality

4. Does it appear to be sustainable?

2. Growing: is it beginning to produce good work, relevant to the stakeholders and with potential for wider impact? Is it setting new agendas, as intended? Governance?

3. Maturing: is it beginning to have visible impact beyond the stakeholder group while maintaining quality? Governance?

4. Late in life: what has it achieved (outputs, outcomes, impact)? At a good quality level? Succession/continuation? Governance?

29

Diversity of methods is necessary. Methodological focus should shift during the life cycle

• Intervention logic

• Peer review: quality; strategy

• Document analysis

• Surveys, interviews with participants and stakeholders

• Scientometrics

• Economic impact analysis

• Little versus big firms – what is really feasible?

• Human capital analysis, Knowledge Value Collectives

• Social Network Analysis

• Long-term tracing (backwards and forwards)

30

31

Thank you

technopolis |group| has offices in Amsterdam, Brighton, Brussels, Frankfurt/Main, Paris, Stockholm, Tallinn and Vienna