Final Evaluation of the Clean Sky Joint Undertaking …...Clean Sky Final Evaluation Report 30 June...

Clean Sky Final Evaluation Report 30 June 2017

Final Evaluation of the Clean Sky Joint Undertaking (2008-

2016) operating under FP7

Written by: Cheryl Atkinson Helge Pfeiffer Piotr Doerffer Expert group: Cheryl Atkinson, Helge Pfeiffer, Piotr Doerffer, Heather Allen, Michael Dooms June - 2017


2

Final Evaluation of the Clean Sky Joint Undertaking (2008-

2016) operating under FP7

Written by Cheryl Atkinson

Helge Pfeiffer

Piotr Doerffer

Expert group:

Cheryl Atkinson, Helge Pfeiffer, Piotr Doerffer,

Heather Allen, Michael Dooms

EUROPEAN COMMISSION

Directorate-General for Research and Innovation Directorate H — Transport Unit H.3 — Aviation

Contact: Francky Callewaert

E-mail: [email protected]

European Commission B-1049 Brussels


4

LEGAL NOTICE

This document has been prepared for the European Commission however it reflects the views only of the

authors, and the Commission cannot be held responsible for any use which may be made of the information contained therein.

More information on the European Union is available on the Internet (http://www.europa.eu).

Luxembourg: Publications Office of the European Union, 2017

ISBN 978-92-79-69189-8 doi: 10.2777/292781

© European Union, 2017 Reproduction is authorised provided the source is acknowledged.

Europe Direct is a service to help you find answers

to your questions about the European Union.

Freephone number (*):

00 800 6 7 8 9 10 11

(*) The information given is free, as are most calls (though some operators, phone

boxes or hotels may charge you).

http://europa.eu.int/citizensrights/signpost/about/index_en.htm#note1#note1

Table of Contents

ABSTRACT ........................................................................................................................................................ 7

1 EXECUTIVE SUMMARY ............................................................................................................................. 8

2 INTRODUCTION ..................................................................................................................................... 12

2.1 PURPOSE OF THE EVALUATION .................................................................................................................... 12 2.2 SCOPE OF THE EVALUATION ........................................................................................................................ 12

3 BACKGROUND TO THE INITIATIVE ......................................................................................................... 13

3.1 DESCRIPTION OF THE INITIATIVE .................................................................................................................. 13 3.1.1 The Clean Sky Initiative ................................................................................................................... 13 3.1.2 Intervention logic ............................................................................................................................ 16 3.1.3 Consistency of the JU with EU’s general transport objectives ........................................................ 18

3.1.3.1 White Paper on transport ...................................................................................................................... 18 3.1.3.2 Vision 2020............................................................................................................................................. 19 3.1.3.3 ACARE .................................................................................................................................................... 19

3.2 BASELINE ................................................................................................................................................ 19

4 EVALUATION QUESTIONS ...................................................................................................................... 23

5 METHOD/PROCESS FOLLOWED ............................................................................................................. 24

5.1 PROCESS/METHODOLOGY ......................................................................................................................... 24 5.2 LIMITATIONS – ROBUSTNESS OF FINDINGS ..................................................................................................... 25

6 IMPLEMENTATION OF THE CLEAN SKY JOINT TECHNOLOGY INITIATIVE ................................................ 26

6.1 IMPLEMENTATION IN GENERAL .................................................................................................................... 26 6.2 STRUCTURE OF CLEAN SKY JU ..................................................................................................................... 27 6.3 BUDGET ALLOCATION ................................................................................................................................ 28 6.4 LEADERS AND ASSOCIATES ......................................................................................................................... 30 6.5 CALL FOR PARTNERS 2008-2013 ................................................................................................................ 31

7 ANSWERS TO THE EVALUATION QUESTIONS ......................................................................................... 43

7.1 MAIN ACHIEVEMENTS ............................................................................................................................... 43 7.1.1 Direct Achievements ........................................................................................................................ 44

7.1.1.1 ITD 1 Smart fixed wing aircraft (SFWA) .................................................................................................. 44 7.1.1.2 ITD 2 Green Regional Aircraft (GRA): ..................................................................................................... 47 7.1.1.3 ITD 3 Green Rotorcraft (GRC) ................................................................................................................. 49 7.1.1.4 ITD 4 Sustainable and Green Engines (SAGE) ......................................................................................... 50 7.1.1.5 ITD 5 SGO ITD ......................................................................................................................................... 52 7.1.1.6 ITD 6 Eco Design (ED) ............................................................................................................................. 53 7.1.1.7 Technology evaluator ............................................................................................................................. 54 7.1.1.8 Achievement of ACARE goals ................................................................................................................. 55 7.1.1.9 Impact .................................................................................................................................................... 57

7.1.2 Effectiveness of Implementation..................................................................................................... 59 7.2 CLEAN SKY JOINT UNDERTAKING'S PERFORMANCE IN 2008 - 2014 ................................................................... 60

7.2.1 Clean Sky JU mission and governance ............................................................................................ 60 7.2.1.1 Governing board (GB) ............................................................................................................................ 60 7.2.1.2 The JU Programme Office ...................................................................................................................... 62 7.2.1.3 ITD Steering Committees ....................................................................................................................... 63 7.2.1.4 National State Representatives Group (NSRG) ...................................................................................... 64 7.2.1.5 The Scientific and Technical Advisory Board (STAB) .............................................................................. 64 7.2.1.6 The Interface with “SESAR” .................................................................................................................... 65 7.2.1.7 Other Stakeholders ................................................................................................................................ 65 7.2.1.8 Participation ........................................................................................................................................... 66 7.2.1.9 Conclusions on Mission and Governance............................................................................................... 69

7.2.2 Operational effectiveness ............................................................................................................... 70


6

7.2.2.1 CS Programme Management ................................................................................................................. 70 7.2.2.2 Clean Sky Quality –Partners and Research ............................................................................................. 73 7.2.2.3 CS Communication ................................................................................................................................. 74

7.2.3 Operational efficiency ..................................................................................................................... 76 7.3 EU ADDED VALUE .................................................................................................................................... 78 7.4 COHERENCE ............................................................................................................................................ 81 7.5 RELEVANCE ............................................................................................................................................. 83

8 CONCLUSIONS ....................................................................................................................................... 85

9 RECOMMENDATIONS ............................................................................................................................ 88

10 ANNEXES ............................................................................................................................................... 90

10.1 CONSIDERATIONS ON R&D SUBSIDIES FOR INDUSTRY AND THE 5% DILEMMA ....................................................... 90 10.2 BIBLIOGRAPHY ......................................................................................................................................... 91 10.3 INTERVIEWS CONDUCTED ........................................................................................................................... 93 10.4 ABBREVIATIONS ....................................................................................................................................... 93 10.5 TABLE OF FIGURES .................................................................................................................................... 95 10.6 TABLE OF TABLES ...................................................................................................................................... 95 10.7 ANNEX COORDINATOR SURVEY ................................................................................................................... 97 10.8 ANNEX - SURVEY PUBLIC CONSULTATION .................................................................................................. 111

Abstract

This report is the final evaluation of the Clean Sky Joint Undertaking (CSJU) in executing the Clean Sky programme from 2007 to 2016 as required by Article 11 of the Council Regulation (EU) No 71/2008 of 20 December 2007 council regulation [1]. The evaluation was conducted between January 2017 and June 2017 by a team of independent experts and is based on relevant documentation, survey results, stakeholder interviews and data analysis. This evaluation complies with the requirements of the revised guidelines of the Better Regulation Package and covers the five main evaluation criteria: relevance, efficiency, effectiveness, coherence and EU added value. In addition, the criteria: openness, transparency and research quality are considered.


8

1 EXECUTIVE SUMMARY Scope This document presents the results of the Final Evaluation of the Clean Sky Joint Undertaking (CSJU) established during the Seventh Framework Programme (2008-2014) but with an end date of December 2017 to allow for exploitation aspects. The evaluation was mandated by the regulation establishing the CSJU and was conducted by a team of independent experts from January 2017 to June 2017. The evaluation was carried out following the “Terms of Reference” of the Evaluators retention contract, which addressed the requirements of the revised guidelines of the Better Regulation Package as well as the main evaluation criteria: relevance, efficiency, effectiveness, coherence and EU added value. In addition, the criteria: openness, transparency and research quality are considered. The evaluation is intended to inform the European Commission’s views on the effectiveness of the CSJU and shape the implementation of future Public-Private Partnerships (PPP) for the purpose of promoting R&D in the aeronautics domain. The Clean Sky Research Programme The CSJU was responsible for the execution and management of a multidisciplinary research program focusing on opportunities to accelerate the industrial implementation of ‘green’ technologies in air transport vehicles. Its approach is to use demonstrators (TRL 6) to validate technological concepts that can meet the societal objective of mitigating the environmental impact of air transportation but are beyond the research investment capacity of industry to develop. The intention was to pave the way for evolutionary concepts to replace the normal incremental product development strategy. The objectives for Clean Sky were based on the FP7 research priority ‘Greening of Air Transportation’ and the visionary ACARE document “Flightpath 2020” The contribution of the Clean Sky research agenda that was targeted in the JTI proposal was: 50% less CO2, 80% less NOx and 50% less perceived noise at the completion of the project in 2017. This should be realised together with safeguarding competitiveness and ensuring continued economic growth of the aeronautical sector in Europe. Achieving these goals is a critical enabler for achieving sustainable Air Transport growth and hence growth within the wider EU economy in line with the Lisbon agenda. Demonstrators were defined for the most used vehicles in the Air Transport market – the large passenger aircraft, the regional aircraft and the rotorcraft – and were configured for laboratory, ground or flight test validation depending on the technologies incorporated. The technologies were developed by discipline based units for air vehicle (structure, aerodynamics etc.), engines and systems. An ‘Eco Design’ focus group developed life cycle analysis tools and influenced all of the development work. A Technology Evaluator tracked the extent to which the environmental objectives could be considered achievable. What are the main achievements of Clean Sky? The CSJU was established as an “EU body” subject to EU Financial Regulations, with funding from the Seventh Framework budget appropriation of 800 M€ to be matched by the participating aeronautics industrial, research and academic members of the PPP. Grants were allocated to supporting partners on the basis of open calls for topics contributing to each of the core research areas and more than 500 participants were joined to the CSJU.

A high quality of research capability was realised in a geographical distribution that approximated the economic contribution in aeronautics. A good balance of industrial, research and aeronautics institutes of higher education was achieved and much organisational capability in the execution of demonstrators was acquired. For example, the characteristics of demonstrator projects required a very high engagement of SMEs who contributed about 35% of the programme activity. Universities enriched their curriculum with the capacity to build and test the innovative parts that were integrated in the demonstrators. And industrial research staff gained significant product development experience in executing their part of the Clean Sky programme. Many new technologies were investigated and progressed into the demonstrator configurations validated in Clean Sky. The highly ambitious “Counter Rotating Open Rotor” engine and the Laminar Flow wing were progressed to readiness for flight testing on Airbus test aircraft in the successor programme Clean Sky 2. A broad range of opportunities for future development were identified and are being progressed towards validation in Clean Sky 2. What are the main findings of the evaluation? Clean Sky has achieved widespread recognition around the world for the unprecedented level of collaboration of its research participants in a focused and coherent research programme that significantly reduced the fragmentation of other funding instruments. The PPP approach was effective in the governance and the execution of the programme through the CSJU Programme Office. Its dedicated support to the members in administrative matters, the monitoring and interventions of its technically competent project officers and the leadership of a visionary and inspiring Executive Director all contributed to the realisation of the Clean Sky ‘movement’. A well-crafted Communication Strategy mobilised participation and broadcast Clean Sky activities and accomplishments leading to a focusing of national and industrial research priorities in its wake. In terms of the main evaluation criteria: Effectiveness Remarkable achievements were made in Clean Sky, both in technological advances and in the realisation of a well-functioning partnership in the programme execution from a somewhat chaotic starting point. This is largely attributable to the recognition by all stakeholders that “Clean Sky” was the right approach to the coordination of demonstrator oriented aeronautics research and their resulting dedication to making it work. The formation of a JU around the ACARE Strategic Research agenda environmental objectives created the momentum for a focused coordinated response to the need for the ‘greening’ of air transport. The alignment of the Clean Sky multipoint research agenda with the product development strategies of the founding members, both long and short term, gave them the incentive to invest in, and wholeheartedly support, Clean Sky. A shortcoming in the implementation may have been the perceived rigidity of the initial programme over the Clean Sky life cycle, not recognising that Clean Sky would be an evolutionary process that merited a more flexible budgetary framework than was realised. Efficiency The Clean Sky programme office has a broad portfolio of operational tasks, related to the administration of the programme participation and the monitoring of research progress, combined with ‘ad hoc’ responsibilities such as preparing the Clean Sky 2 programme. The rigorous review schedule, the high frequency of meetings inherent in the governance structure, and the geographical reach of the programme participation places a high travel burden and demands almost 24/7 ‘reachability’ for the staff members, who fortunately all think they have the best job in the world. Overall, the Clean Sky Programme Office performs remarkably well and efficiently.


10

Relevance Political developments underwrite the continuing relevance of reducing the environmental impact of air transport. The adoption in 2015 of the historic Paris Agreement and its ratification in November 2016 underscore the global intention to ‘resist’ climate change. The International Civil Aviation Organisation (ICAO) agreement in February 2016 on a CO2 standard for new aircraft, followed by their accord on global market-based measure to control CO2 emissions from international aviation in October, highlight an emerging regulatory framework. The radical, disruptive technology lines that the Clean Sky programme has pursued are the foundations for aircraft industry product steps that skip a generation of ‘evolutionary’ development and make the Clean Sky concepts the “new normal” in the operating fleet and the biggest challenge is to ready them for the ‘next plus 1’ aircraft fleet renewal cycle. The policy and rationale that underlay the Clean Sky programme in 2007 is still in line with the current challenges in the Air Transport sector and the portfolio of tasks entrusted to the Clean Sky Joint Undertaking, and the effective execution of them in Clean Sky 1, continues to underwrite the PPP approach. EU Added Value The European Union is the only place on the planet that could have realised Clean Sky and the JTI approach is the only way that the critical mass for the success of the Clean Sky research agenda could have been obtained. The JTI approach has been able to focus the aeronautical research community on the goal of mitigating the environment impact of aviation, a mission that would not have been undertaken based only on market incentives, on a much larger scale than would otherwise be realised. The European Parliament has noted that “"the concept of European added value must not be limited to advanced cooperation between Members States but should also contain a visionary' aspect". [2] There is no doubt that that Clean Sky research agenda, which targeted to double the rate of progress in fuel consumption reduction and half the time to market for the resulting technologies, was ambitious. Coherence Four aspects of the coherence of the Clean Sky programme have been evaluated.

The internal coherence in the Clean Sky programme itself is very high and, in spite of the

complexity of the programme interfaces, the participants have a common vision

FP7 calls are providing bottom up project proposals that “fill the innovation pipeline” with ideas

that, if shown to be feasible in L1 research, may be developed to a higher TRL in Clean Sky.

There were difficulties in implementing a working relationship that achieves the necessary

‘complementarity and synergy’ with SESAR and this relationship has been further developed in

the transition to Clean Sky 2.

Coordination with national research programs has not been explicitly accomplished although anecdotally such alignment is reported. As Clean Sky has not been granted the ‘monopoly’ that SESAR has in ATM research a ‘persuasive’ approach to coordination with national research programs is the only recourse.

Openness and Transparency The CSJU has been implemented in an open and transparent manner. The Governing Board records are complete and publically available as are the Annual reports which developed over time to give high quality insight into the programme. Clean Sky activities and accomplishments are well promoted. The website possibly did not keep pace with programme, particularly in making technical documentation accessible.

Research Quality The Clean Sky PPP is clearly achieving its objective of grouping the best quality aeronautics research entities in a programme of common European interest with a clear relationship to the societal challenge of climate change and benefitting the competitiveness of European aeronautics stakeholders. Clean Sky has produced ‘world class’ Aeronautical Research, Development Test and Evaluation (RDT&E) that enables promising technologies for the European Aeronautical Industry to be developed and tested in large scale demonstrators. Recommendations As this is a final evaluation, and the successor programme is well underway and taking good account of its ‘lessons learned’, the addition here of specific recommendations seems somewhat redundant. However the recommendations that had been made by the Interim Review Panels and by the CS2 Impact Assessment Panel were surveyed to make a global assessment of the extent to which these recommendations had been given effect in Clean Sky 1.

The JU Programme Office was generally successful in achieving actions that were within their

scope

Recommendations related to the role of the various demonstrators, and the question of

whether TRL6 was always an appropriate goal when TRL5 may have been ‘good enough’ appear

to have only marginally influenced the programme. This is related to the relatively low level of

flexibility in the programme implementation.

Several recommendations spoke quietly to a problem with the programme architecture and its

very high interface intensity between ITDs plus a distributed project management structure that

hampered insight in the coordination effectiveness. It was also felt to be important to future

Clean Sky programmes that the JU Programme Office find easy ways to track ‘in programme’ the

things that are important to the programme credibility; TRL evolution, demonstrator uptake of

in programme innovations, achieved system and manufacturing readiness levels in addition to

technology readiness levels, industrial implementation The evaluators found in this aspect a

focal point for future programmes.

We will trust that our reflections above are of influence in the evolution of Clean Sky 2 and the definition of future aeronautics programmes that hopefully can rely on the huge expertise created by the CSJU.


12

2 INTRODUCTION

2.1 Purpose of the evaluation

Article 11 of the Council Regulation (EU) No 71/2008 of 20 December 2007 [1] establishing the Clean Sky Joint Undertaking provided the base for the external reporting for the Clean Sky programme. In addition to annual reports on the progress achieved to the European Parliament and to the Council, two interim evaluations evaluation [3, 4] and a final evaluation, carried out with the support of external experts, are prescribed. Detailed and thoughtful guidance for the evaluation was provided by the Commission (Directorate-General for Research & Innovation, Transport (H)) in an appendix to the retention contract for the selected experts [5]. As a final evaluation, the progress made in achieving the objectives set for the Clean Sky programme and the extent to which the Clean Sky JU was managed and operated efficiently are the central themes. In addition, an assessment of the openness and transparency with which the Clean Sky programme was accomplished will permit comparison with the future Clean Sky 2 second generation JTI JU. The results of this evaluation will be used by the Commission to inform the European Parliament and the European Council, national authorities, the research community and other stakeholders of the achievements and outcomes realised by the Clean Sky JU operating under the Seventh Framework. It will also identify the foundations for the refinements that were made, based on the experience gained and lessons learned, in the Clean Sky 2 implementation under Horizon 2020.

2.2 Scope of the evaluation

This final evaluation of the Clean Sky Programme has targeted a ‘helicopter view’ of the life cycle of the Clean Sky movement. It presents the incentives for the initiative and discusses the implementation of the Clean Sky JU with attention to the adjustments that were considered appropriate for future aeronautical research initiatives such as Clean Sky 2. A ‘best effort’ has been made to ground the observations and conclusions in this report in the documented history of the Clean Sky JTI, wherein the two previous Interim Assessments were instrumental, but it also includes personal perceptions obtained through stakeholder interviews. In view of the comprehensive technical evaluation carried out in the Second Interim Review, and the Annual Reviews by independent external experts whose recommendations are conscientiously addressed, this evaluation has not included a strong technical focus. Instead an effort was made to review the architecture of the programme, the relationship engendered between the actors in its execution and the principles and practices of its management by the JU Executive and the individual research partners. It has been a great honour to conduct this evaluation and the reviewers are grateful for the time and care that has been taken by the Commission and JU staff to provide the information we requested and thoughtful, well considered responses to our many questions. We are in awe of the openness and honesty of all of the stakeholders we engaged with in the course of this exercise. While we cannot claim in any sense to have fully evaluated the Clean Sky programme and presented in this report all of the nuances that influenced its execution, we sincerely hope that we have highlighted those aspects that have most significantly impacted the aeronautical research community and the

public trustees that have shown their confidence in the cooperativeness and professionalism of this community by investing in the Clean Sky JTI.

3 BACKGROUND TO THE INITIATIVE

3.1 Description of the initiative

Joint Technology Initiatives (JTIs) were a major new feature of the seventh framework programme (FP7), introduced to support key areas of research and technological development that can contribute to Europe’s competitiveness and quality of life by providing for Community contribution to the establishment of long term public private partnerships (PPP). They were established as ‘Community Bodies’ by specific regulation, with generally similar structures, under the Treaty establishing the European Community [6] as Joint Undertakings. Joint Undertakings were referenced in the Council decision [7] implementing the Seventh Framework Programme in which the budget for the ‘Cooperation Programme – Transport (including Aeronautics)’, including JTI’s, was established. The JTIs under FP7 emerged from the European Technology Platforms (ETPs), identified for Aeronautics in the Clean Sky regulation as the Advisory Council for Aeronautics Research in Europe (ACARE) and are described in the FP7 decision as «These initiatives, mainly resulting from the work of European technology platforms and covering one or a small number of selected aspects of research in their field, will combine private sector investment and national and European public funding, including grant funding from the Research Framework Programme and loan finance from the European Investment Bank. » The Commission deemed the Clean Sky initiative to be at an appropriate stage of preparedness to undertake the submission of a “JTI proposal” [8] and the Clean Sky Joint Undertaking was established by Council Regulation (EC) No 71/2008 of December 20 2007 [1] as a public-private partnership between the European Commission and selected stakeholders of the aeronautics research community. Its objectives were a marriage of the ACARE Strategic Research Agenda [9] priority for the reduction of aviation’s environmental impact and the ‘Lisbon Growth and Jobs Agenda’ [10], which highlighted the need to develop favourable conditions for investment in knowledge and innovation in Europe on behalf of competitiveness and growth.

3.1.1 The Clean Sky Initiative

The principal objectives of the Clean Sky JTI, as captured in the founding regulation, were:

To provide integration and demonstration at the level of the system as a whole to decrease the risk for private investment in developing new environmentally friendly aeronautics products.

To accelerate the development of Air Transport technologies to realise the earliest possible deployment of contributions to Europe’s strategic environmental and social priorities.

To ensure the coordinated use and efficient management of the funds assigned to the Clean Sky JTI.

These objectives were also based on the recognition of the economic benefit of the air transport sector to the European Union both directly, in a strong contribution to employment and GDP, and in aviation’s role in the overall competitiveness and growth of the European Union. It was expected in that period that the rate of air transportation growth experienced in the preceding decade would continue at a rate of 5% per annum into the foreseeable future with consequential increases in environmental impact. Technological advances that mitigate the environmental impact of the future global fleet could prevent constraints on the growth of air traffic due to emissions and on airport capacity due to noise limitations. However private investment in these technologies is only partly justified by the economies of the fuel consumption reductions that would accompany reductions in


14

emissions and the competitive advantage to be gained is highly dependent on the prevailing oil price level. With the environmental costs to society thus far external to the air transport operators and manufacturers, public funding of the related research is a necessary additional incentive for the industry to invest in clean technologies. Competitiveness in the aeronautics sector requires exceptional levels of research investment, a higher than normal level of risk, long return on investment and lower return on investment than in other sectors. However it is a leading edge industry with significant spill over benefits in other sectors and the European Union’s long commitment to building and maintaining world class capability in aeronautics manufacturing has had a significant economic impact in the community. It is also widely recognised that the main global competitors, such as the United States, enjoy a significantly higher level of public support for aeronautics research and that other strong competitors in various niche markets (Brazil, China, Russia, South Korea and India) have emerged in recent years. In comparison to this global trend, the Framework programmes, while generating significant contributions in innovations and concepts, have not been able to sufficiently emphasise the validation of complex systems at a high level of integration. The Clean Sky programme was therefore designed to focus on technologies that offer the potential for step changes in performance but that are currently seen as too high a risk to be funded privately. It emphasised the realisation of high technology readiness demonstrators that could position industry for accelerated exploitation at reduced risk and influence the timing of new product development. The scope of the Clean Sky JTI addressed the breadth of three main ‘axis’ – all segments of civil air transport (regional aircraft and rotorcraft in addition to the economically dominant large commercial aircraft); the full depth of the supply chain (including engines, systems and materials supporting technologies) and the product development life cycle (to the limit of what may be realised with public funding), through an integrated approach leading to multiple full scale ground and flight test demonstrators. In the course of the execution of Clean Sky, and one of its important ‘lessons learned’ and achievements, is that this approach enabled many of the participants to retain (or re-train) their internal capability for product development. The Clean Sky JU was established in 2008 to run until 31 December 2016, with a budget of 1.6 B€ equally shared between the European Commission (EC) and the aeronautics research community members. Its members were mainly Industry representatives (and their affiliates) and one research organisation in the role of ITD Leaders as well as 74 Associates chosen on the basis of excellence from an evaluation of solicited “Expressions of Interest”. In the course of the programme more than 500 Partners were selected through “Calls for Proposals”. The Clean Sky JU was charged with the top level management of research activities, including for the two years beyond the end of the Seventh Framework, to encompass the exploitation phase of the product development life cycle, and with monitoring of the technological progress and impact of the research activities during the overall programme. Their role explicitly included the management of the vast amount of knowledge that the programme would generate. The European Commission, as a co-founding member, was responsible for setting up the JU, which became autonomous about two years into the programme when the legal and financial framework, and the capacity to manage the programme budget, had been established. The JU staffing consisted of contract and temporary agents for a maximum of 7 years of engagement under Executive Director Eric DAUTRIAT from September 200910), a highly qualified industrial research manager, who shaped and inspired the Clean Sky programme during his term of office. It may be self-evident then, that this unprecedented level of coordinated participation in support of common objectives with coherent targets provided a focal point around which aeronautics industry leaders and their knowledge networks could coalesce. A greater degree of coordination between

national, EU and industry sponsored research across the supply chain was foreseen and the involvement of new actors, including from other industrial sectors, would be stimulated by the high visibility of such an ambitious programme. The broad lines of the organisation of the Clean Sky JU and the architecture of the research workplan are discussed here and elaborated in other sections of this report, as needed to address specific evaluation criteria. The Clean Sky research plan was based on 6 “Integrated Technology Demonstrators” (ITDs) as shown below (Table 1) together with the respective funding levels at the outset of the programme. The matrix character of the relationships between these ITDs is reflected in the accompanying illustration (Figure 1) and its impact of the coordination between ITDs formed one of the main ‘lessons learned’ in the Clean Sky programme.

ITD scope Co-leaders Budget

VEHICLE ITDs

SMART fixed wing aircraft

active wing technologies that sense the airflow and adapt their shape as required, new aircraft configurations to optimally incorporate these novel wing concepts.

Airbus SAAB

24%

Green Regional Aircraft

low-weight configurations and technologies using smart structures, low-noise configurations and the integration of technology developed in other ITDs, such as engines, energy management and mission and trajectory management

Alenia Aeronautica EADS Casa

11%

Green Rotorcraft innovative rotor blades and engine installation for noise reduction, lower airframe drag, diesel engine and electrical systems for fuel consumption reduction and environmentally friendly flight paths.

Agusta-Westland Eurocopter

10%

TRANSVERSE ITDs

Sustainable and Green Engine

Integrating technologies for low noise and lightweight low pressure systems, high efficiency, low NOx and low weight core, novel configurations such as open rotors

Rolls-Royce Safran

27%

Systems for Green Operations

all-electric aircraft equipment and systems architectures, thermal management, capabilities for “green” trajectories and mission and improved ground operations.

Thales Avionics Liebherr Aerospace

19%

Eco-Design full life cycle of materials and components, focusing on issues such as optimal use of raw materials, decreasing the use of non-renewable materials, natural resources, energy, the emission of noxious effluents and recycling.

Dassault Aviation Fraunhofer Gesellschaft;

7%

Table 1 Integrated demonstrators (ITD) addressed in Clean Sky

A Technology Evaluator (TE) led by Thales Avionics and DLR was the core of Clean Sky with the purpose of assessing the environmental performance of the technologies developed in CS at air vehicle, airport and global fleet level. With a budget of 2% of the EC contribution, it was envisaged as the first available European complete integrated tool delivering direct relationship between advanced technologies, still under development, and high-level local or global environment impact.


16

Figure 1 Matrix character of the different ITDs in Clean Sky

3.1.2 Intervention logic

It is current practice to present an intervention (Figure 2) such as the Clean Sky PPP initiative in the overall context of the European research priorities and the global influences on the industrial sector package [2]. While not exhaustive, this analysis of the Clean Sky context and incentives for the JTI approach does provide key ‘checkpoints’ for this evaluation and the assessment of whether the intervention has been implemented as intended and realised its aspirations. The intervention logic of the Clean Sky initiative is rooted in two major ‘NEEDS’ that follow from the objectives for the Framework 7 research priorities relevant for the aeronautics sector. These are:

Technological breakthroughs for environmental impact mitigation – The ACARE SRA environmental targets are so ambitious that they will not be reached without technological breakthrough, i.e. radical changes in technology requiring a substantial amount of research and validation but represent global leadership in signalling clear environmental objectives. Industrial readiness of green technologies – a demonstrator based programme will ensure supply chain preparedness and product competitiveness of ‘green’ technologies and level the playing field relative to global competitors with more public funding than in Europe.

SPECIFIC OBJECTIVES to be realised by the implementation of the Clean Sky JTI to meet these needs are:

Validated ‘green’ technologies arising from concepts developed within collaborative research projects (Framework programmes) and brought to system level demonstrator (TRL 6) in the Clean Sky programme.

The operational flexibility to cooperatively optimise the ongoing research programme to evolving industry priorities and to fast track promising technologies and thereby obtain long term commitment and investment from the aeronautics research community.

The mobilisation of a critical mass of resources to accelerate the realisation of common objectives through demonstrators with a high degree of functional integration.

Continuity and consistency of research activities over the programme life cycle to reduce the fragmentation of the collaborative research ‘lottery’ and increase the efficiency of access to public funding.

Centralised programme management with efficient systems to reduce the administrative burden/barrier -*for research entities.

The RESOURCES to be applied in the PPP intervention are:

The research capacity of the highest quality eligible actors, deployed to well-defined and rational research objectives.

An adequate level of funding for the programme objectives in a balanced contribution and distribution among the participating entities.

Streamlined and efficient procedures for grant administration and programme monitoring and control.

The ACTIVITIES to be conducted are:

The implementation of the joint technology proposal via development and work plans.

The adaptation of the work plan to external factors and intermediate research results.

The management of the programme.

The monitoring and communication of the outcomes (knowledge management, dissemination, exploitation).

The influential EXTERNAL FACTORS are emerging competition, economic conjuncture (air traffic growth, fuel price evolution), trade agreements, global climate change mitigation measures, change of political support and offshore ownership of European actors. As well as a host of validated demonstrators and green aircraft concepts, the OUTPUTS of the intervention will be research infrastructure, such as new simulation tools, updated test capacity (wind tunnels, test benches, flight test vehicles, test instrumentation etc.) and generated knowledge. The RESULTS of the intervention are expected to be validated Clean Sky technologies with the potential to realise the ACARE environmental targets and at a high level of technology readiness to accelerate market introduction. Spill-over results will be leading edge knowledge that can be applied in other industrial sectors.

The IMPACTS will be:

Strengthened research and innovation capacity in the aeronautics research eco-system

Strengthened industrial competitiveness

Mitigation of the environmental impact of air traffic growth

Avoidance of noise limitations to airport utilisation increase

Increased mobility and economic growth for EU citizens These aspects are summarised in the intervention logic diagram below (Figure 2).


18

Figure 2 Intervention logic diagram

3.1.3 Consistency of the JU with EU’s general transport objectives

A policy designed to stimulate research and development in the aeronautics sector, leading to environmentally efficient aircraft, is one pillar of the global strategy presented in the communication COM(2005)459 of September 2005 to reduce the climate impact of aviation, which was endorsed by the Council (2 December 2005) and the Parliament (4 July 2006). Stimulating aeronautics R&D is complementary to measures such as the Commission proposal to include aviation in the EU Emission Trading Scheme (ETS). In addition it is widely recognised that greener aviation technologies will contribute towards mobility within an enlarged EU, which will be particularly important for accession states where traffic is growing rapidly from a low base.

3.1.3.1 White Paper on transport

The general transport objectives in the period when Clean Sky was designed were mainly based on the White Paper [11] developed in 2001 and its successor documents which were adopted by the EC as a “roadmap” for research policy development. However, the Council Regulation for the establishment of the Clean Sky JU [1] does not refer to this background nor do, generally, CS documents. The White Paper underlines that the “RTD priorities in the aeronautics field will focus, on the one hand, on lessening the environmental impact of engine emissions and noise and improving aircraft safety …”. It further states that “ … As regards the environment, the aim is to compensate for the increase in air traffic by reducing CO2 emissions by 50 % and NOx by 80 % and by reducing aircraft noise by 10 dB in

order to cut the perceived noise level by 50 %. Research will focus on aircraft technology, low-drag aerodynamics and flight operating procedures.” The policy of the White Paper for transport is in many aspects also strongly passenger-centred and also driven by economic aspects in a more competitive integrated, safe and intermodal transport system. Aspects related to tax policy for kerosene etc. are also addressed. All by all, one can state that the objectives of CS clearly support the major objectives of the White Paper related to aeronautics.

3.1.3.2 Vision 2020

In 2001, the report of the Group of Personalities "European Aeronautics: A vision for 2020 " [12] pioneered an integrated vision of the European Air Transport System (ATS) for the next 20 years and became a popular reference for research policy development. It established, as its top-level objectives, the need to respond to society's needs and to secure European leadership in the aeronautics field." Society's needs embrace the whole range of benefits that all citizens of Europe expect of the air transport industry now and in the future. These benefits are direct, as in the quality and price of travel, and indirect, as in the preservation of security and safety in a more global world. They encompass the personal needs of travellers and the collective needs of non-travellers who want to live in quiet, pollution-free neighbourhoods." It is clear that the range of CS research also address these aspects.

3.1.3.3 ACARE

The Advisory Council for Aeronautics Research in Europe (ACARE), the first European Technology Platform (ETP), produced a set of strategic research objectives (SRA1) in 2002 [9] and a second updated edition (SRA-2) in 2004 [13]. The SRA-1 is built around 5 Challenges for technology development SRA [9]. It has been used as a reference guide for a number of national and institutional bodies for establishing their research programmes. The SRA-2 describes six High Level Target Concepts (HLTCs) and their associated technologies with respect to different socio-economic scenario [13]. Each HLTC stresses a particular aspect of the Air Transport System. Clean Sky contributes to meeting of the ACARE HLTC such as Ultra Green Air Transport System, i.e. reducing the impact of air transport on the environment.

The ACARE SRA’s have been the most influential on Clean Sky objectives and the environmental impact mitigation targets in the programme are directly linked to this reference.

3.2 Baseline

Aeronautics research has taken place under the European “Framework” Research programmes which have been operating since the mid 80’s with progressive budget increases and continual changes to the priorities, allocations and instruments within each programme. While industrial competitiveness was generally part of the scope of the successive work programmes, the Fourth Framework was the first implementation detailing a ‘Transport’ agenda in which “for air transport, research will focus on reducing congestion of airspace and of airports, particularly taking into account the results of transport telematics, as well as on further improving human safety and reducing the negative impact on the environment».


20

The evolution of the Framework program budgets and the approximate appropriation for aeronautics (exclusive of air transport) is shown in Table 2 . It can be seen that the growth in aeronautics research has not kept pace with the general trend. In addition, collaborative research funding dropped dramatically in Horizon 2020 with major impact on the research not engaged in Clean Sky and its successor.

ID Framework Programme Period Budget (B€)

appr. Budget for aeronautics (B

€)

Approx. Budget for aeronautics

collaborative research (B€)

FP1 First 1984–1987 3.8 na na

FP2 Second 1987–1991 5.4 na na

FP3 Third 1990–1994 6.6 na

1990–1991 0.35 [14] 0.35 [14]

1992-1994 0.71 [14] 0.71 [14]

FP4 Fourth 1994–1998 13.2 Na na

1995-1998 0.245 [14] 0.245 [14]

FP5 Fifth 1998–2002 15.0 0.700 [14] 0.700 [14]

FP6 Sixth 2002–2006 17.9 0.85 [14] 0.85 [14]

FP7 Seventh 2007–2013 50.5 2.1 [14] 0.950 [14]

FP8 Horizon 2020 (Eighth) 2014–2020 80 2.1 0.185 Table 2 Key financial data for the different FP's

The Fifth Framework programme specifically addressed aeronautics research which, being considered ‘close to market’ had previously been largely conducted in the context of the ‘EUREKA’ instrument, and over 600 projects are identified as related to FP5 calls for ‘aerospace’. Still familiar “Technology Platform” projects such as AWIATOR (Aircraft Wing Advanced Technology Operations), POA (Power Optimised Aircraft), FACE (Friendly Aircraft Cabin Environment), TANGO and ENHANCE were conducted in the FP5 programme and are in many cases the foundations upon which the Clean Sky demonstrator ambitions have been built. In general the Framework Programmes made extensive use of the “Specific Targeted Project” (STP) instrument which in FP6 involved an average of 9 participants who received around EUR 2 million of EC contribution for a period of three years. In comparison, FP6 Integrated Platform (IP) contracts (40% of the funding) engaged 25 participants and received around EUR 9.5 million of EC contribution for a period of four years. The topic “aerospace” was allocated 6.4% of the overall FP6 budget. The success rate of Aerospace proposals in FP6 was 30% averaged over all instruments and 57% for IPs. Private participation (BES, substantially Industry) in FP6 projects is around 55% compared to 30% for the overall programme. The higher success rate for IP’s is related to the participation of stakeholders in the preparation of the Work Program and in particular to the very targeted large project scope and objectives that did not induce submissions from unprepared consortia or allow for their success The FP7 collaborative research structure was similar although STREPS were called ‘Level 1, IPs ‘Level 2’ and the hypothetical ‘Level 3’ was realised in the Clean Sky JTI. In FP7 the overall success rate dropped to 20%. Considering the effort in expertise, time and travel to establish a collaboration and align the capabilities of the prospective partners into a cohesive research program, an unsuccessful proposal has a significant negative economic contribution. However, many institutions depend on research funding for their growth (or stability!) and proposals therefore must continue to be submitted. It is

worth noting here that these same institutions are besieged with fully funded PhD candidates from countries with an emerging aeronautical industry. The aeronautical work plans for the Framework programmes, in which the calls for proposals were defined, were largely discipline oriented ‘thematic’ descriptions of the desired project result. They provide the opportunity for new concepts conceived in academic research to be developed to meet industrial needs. Generally several successful proposals were selected for funding under each call topic. While on the one hand this format provided plenty of scope for bottom-up creativity in project definitions, it did not provide for any continuity of the knowledge acquired from one project to the next (which was often granted to a different consortium of research organisations). Nor could this approach filter out duplication and repetition of research activity or research on matters which were already ‘state of the art’. The Technology Platforms (or IPs in FP6) were an exception where the research objective, and quite probably the related proposals, were crafted with the support of the ETP organisations for aeronautics, ACARE and IMG4, “The European Aeronautics industry network for R&T” and targeted an industrially defined need. A typical IP would aspire to reach TRL 4 (Component and/or breadboard validation in laboratory environment – basic (low fidelity) technological components are integrated to establish that they will work together) or possibly TRL 5 where higher fidelity breadboard technology is integrated in a representative, possibly simulated, environment. This is a significant advance from the conceptual and analytical/experimental work which precedes these levels.

Figure 3 Relative financing for framework programmes over the last frame programs relating the trend for global funding to the share appr. allocated for aeronautics and in that context, for collaborative projects (CP).

The next advance, to a representative model or prototype system, to be tested in a relevant environment, is a huge step forward in industrialisation and manufacturing of the system components and is the domain of the TRL 6 “Demonstrators” that are the essence of Clean Sky. However, in the steps towards realising a demonstrator there are many technology requirements that are unfulfilled at the start of the integration effort and a great many ‘inventions’, from concept to prototype are swept up in the race to demonstrate the original technological concept. It is then apparent that the undifferentiated bottom up calls and funding ‘lottery’ of the traditional framework instruments could not create the unity of purpose and focus of capability that can be realised in a Joint Technology Initiative such as Clean Sky.


22

In view of the key role of the ‘Project Officer’ in the Clean Sky JU it is necessary to consider the historical approach to project management, which was carried out by similarly titled ‘Project Officers’ in the Commission Directorate (RTD Transport) responsible for the workprogram, the calls and the management of the evaluations of the proposals. The multitude of small projects were primarily monitored through the negotiation of a grant agreement consistent with the proposal and the timely appearance of deliverables and a web presence. However the large ‘Integrated Projects’ were more closely monitored, with the ongoing support of external experts, and serious efforts were often needed to resolve scope and partner commitment issues. The strong aeronautics background of the Commission project officers and their broad familiarity with the sector enabled them to effectively meet this challenge. However, running research projects and maintaining contact with beneficiaries has transitioned to the Innovation and Networks Executive Agency (INEA), who has not engaged sector expertise. FP7 projects have only been transferred to INEA as RTD project officers retire. Horizon 2020 calls do not clearly distinguish project size but it is believed that no very large projects have been initiated thus far.

4 EVALUATION QUESTIONS This final evaluation of the Clean Sky Joint Undertaking addresses the need for an in depth assessment of whether the public-private partnership is implemented in an open, transparent and efficient way. A broad range of topics are considered in the factual inquiry (document review) and the solicitation of opinions in surveys and interviews. The evaluation criteria of the Better Regulation Package [2]; relevance; efficiency; effectiveness; coherence and EU added value are addressed as appropriate. The evaluation begins with an examination of the BACKGROUND AND DESIGN OF INITIATIVE AND INTERVENTION LOGIC in which the prevailing funding instruments for aeronautics research are discussed and the incentives for the Clean Sky JU implementation are identified. This insight is presented in section 3 of this report. Section 6 of this report IMPLEMENTATION OF CLEAN SKY JOINT TECHNOLOGY INITIATIVE reviews the formation of the Clean Sky JU, and the manner in which funding was allocated in the course of the Clean Sky programme, to assess the extent to which openness and transparency was achieved. The research work carried out by the Clean Sky JU is reviewed in section 7.1 MAIN ACHIEVEMENTS AND EFFECTIVENESS OF IMPLEMENTATION. The technical accomplishments (and setbacks) are presented for each ITD as is the progress in achieving the ACARE targets set for the programme. Research related aspects such as dissemination, exploitation and the value added of the research results is also addressed. A number of aspects of the overall functioning of the JU are addressed in section 7.2 JU PERFORMANCE IN 2008-2014. The overall operation of the JU is evaluated in section 7.2.1 JOINT UNDERTAKING MISSION AND GOVERNANCE to assess whether the regulatory framework has been coherently implemented and whether the governance structure, and each of its components, are effective in supporting the JU. Section 7.2.2 OPERATIONAL EFFECTIVENESS looks more closely at the functioning of the JU Programme Office to assess whether their methods and procedures have been effective in realising a new paradigm for the aeronautical research community. The performance of the JU Programme Office in their management role is the subject of 7.2.3 OPERATIONAL EFFICIENCY. In section 7.3 EU ADDED VALUE the effectiveness of the large scale demonstrator programme, in comparison to alternative research deployment schemes, is discussed. The relationship between the Clean Sky programme and the related research activities such as the Seventh Framework collaborative research, SESAR and nationally funded aeronautics research is presented in Section 7.4 COHERENCE. The incentives for continuation of the Clean Sky programme into Clean Sky 2 due to the ongoing contribution to societal objectives is discussed in section 7.5 RELEVANCE. Section 8 presents the overall CONCLUSIONS of the evaluation and identifies the aspects that were positive as well as negative. It emphasises that the main key to the success of the Clean Sky programme was the commitment and dedication of both the public and private partners in meeting the expectations of the initial concept for the JTI. A review of the recommendations that have preceded this review and their impact on this Clean Sky programme and on its successor Clean Sky 2, plus some considerations related to longer term strengthening of the JU concept for aeronautics research, are presented in section 9 RECOMMENDATIONS.


24

5 METHOD/PROCESS FOLLOWED

5.1 Process/Methodology

The Clean Sky Final Evaluation and the Clean Sky 2 Interim Evaluation were conducted in parallel by two domain expert reviewers (one shared with SESAR) and a rapporteur, all appointed mid December 2016. A team including the S2R and S2020 (SESAR) reviewers was led by two ‘horizontal’ experts that contributed cross-cutting input. A comprehensive and carefully articulated “Terms of Reference (ToR)” [5] contract annex outlined the daunting task ahead. An initial documentation set addressing the establishment of the JU’s, the annual reports and the two CS Interim Assessments [3, 4] were provided in preparation for the Kick-off meeting in Brussels in mid-January. It was immediately noted that the previous Interim Assessments were conducted by a dedicated team of not less than 5 evaluators over a period of 7 months and set accordingly a much higher “granularity” for technical insight than we are able to achieve. The Kick-off meeting (see also a history of the evaluation in Table 3 provided an opportunity to meet the staff of the various Commission entities involved in the Evaluation, to learn of the interrelationship of our work with the H2020 Interim Review and the related schedule challenges, and to meet the Clean Sky management team and our appointed JU interlocutors (a very wise and necessary arrangement for which we are all grateful). A first Evaluation Team meeting followed and provided an opportunity to exchange views on the approach to the task. Following this ‘Kick-off’ event the team moved into familiarisation and information gathering phase with interviews beginning just 5 weeks later. Two days of interviews with many of the JU staff provided background and key attention points which effectively guided the follow-up planning. These interviews coincided with a Rapporteurs meeting in Brussels which was attended for Clean Sky by teleconference. Unfortunately, despite the best efforts of the Commission meeting organiser, the poor tele-conferencing facilities did not support meaningful participation. However it was clear that the Clean Sky Evaluation team would be challenged to make a contribution to the H2020 Interim Evaluation due to its very short lead time for our input. Period Key Activity

December 2016 Evaluator contracts. Initial documents (establishment, interim assessments and annual reports)

January 19 2017 Commission Kick-off meeting and first evaluation team meeting in Brussels

Urgent input to coordinator survey

Statistical data (Corda) for calls available.

February 2017 Inventory of documentation and interview requirements

Preparatory discussions with JU management

Mid February First set of documents from JU (Dev Plan, CSMM, WP 2016-2017)

Feb 28, March 1 Interviews with JU staff (rapporteur connected via teleconference)

March 2 Rapporteurs meeting (rapporteur connected via teleconference)

March 7 Coordinators survey results posted

March 17 Public Survey Results posted

March 13 to 31 JU colocation of the rapporteur

March 21 Clean Sky Forum, incl. CS Closing Event, GB,SC,SRG meetings, Interviews (GB, ACARE, DLR, Honeywell)

April 26-28 Annual Review meeting Rotorcraft IADP in Turin (Italy)

April 27 ITD Steering committee meetings held

May 3-5 Annual Review meeting Systems ITD in Linkoping (Sweden)

June 22 Final meeting Table 3 Key activities of the evaluation

The Clean Sky Closing Event on March 21 and 22 drew many of the Clean Sky stakeholder representatives to Brussels and the JU had planned coincident meetings of most of their governing entities. The Clean Sky JU generously provided office accommodation and supervised access to the

‘shared drive’ through which all of the programme documentation is available in a very orderly and accessible directory. The last three weeks in March provided thus a very efficient closure of the “Information Gathering and Interview” phase of the evaluation effort. The Annual Reviews of two of the key WORKPACKAGES in Clean Sky 2 (the Large Passenger Aircraft IATP and the Systems ITD) fell within the timeframe of the evaluation process and were attended in part by one of the members of the evaluation team. The JU arranged coincident related site visits where the Clean Sky demonstrator ‘hardware’ could be seen and appreciated. Additional interviews were conducted with MEP Marian Marinescu on SESAR and Clean Sky content, as well as Thales Avionics and Airbus Helicopter representatives. Stakeholder input through the Coordinator and Public Surveys was analysed and absorbed into the evaluation report outline.

5.2 Limitations – robustness of findings

A comprehensive review of the technical accomplishments of the Clean Sky programme has been beyond the scope of this evaluation. The reviewers consider this to have been more than adequately addressed within the communication and dissemination scope of the programme itself, most particularly in the ‘Clean Sky Forum’ closing event held on March 21 2017. It is more important that this evaluation presents and analyses the relationship that the Clean Sky JTI has with the ‘eco-system’ of aeronautics research and the impact that it continues to have on the productivity and effectiveness of this community. The evaluation findings in this report may be compromised by limitations of the utility of the management information available to the JU Programme Office, albeit generously shared with the reviewers. Insight in the composition of the membership and partners, and the contribution of each to the technical objectives of the programme, seem to be spread across a multitude of spreadsheets that each reflect the purpose for which they were created without elaboration of their data sources. Similarly, our inquiries regarding the technical accomplishments and their contributions to the overall objectives of the Clean Sky programme were answered with review presentation material, supplemented by dissemination deliverables if we wanted to make further inquiries. While there is no doubt that the JU’s Project Officers and the Executive Director know the Clean Sky programme so intimately that they can make a presentation at the drop of a hat, or find a presentation in response to a reviewer’s inquiry, we did not see management information at the level of maturity that would be expected. Many important relationships, such as the start and finish TRL of a project; its participants, budget and timeline and its relationship to a demonstrator (or the cause of its failure to contribute) should be more readily accessible. The deliverables, dissemination activities and eventual exploitation success (or not) should form an integral part of the JU Programme Office’s management information system. The reviewers experienced many set-backs to their original aspirations regarding the quality and content of this evaluation due to the severely limited time for its creation combined with the challenge of obtaining an overview of some aspects of the programme that were relevant to the evaluation.


26

6 IMPLEMENTATION of the Clean Sky Joint Technology Initiative

6.1 Implementation in general

The CS initiative was inspired by a small, informal industrial group and in May 2007 a dedicated “Coordinating and Supporting Action (CSA) project was established, shortly after the submission of the Clean Sky JTI proposal [8] (March 2007). This resulted in the 2008 creation of the Clean Sky JTI under its founding regulation (EU) No 71/2008 of 20 December 2007 [1] as a public private partnership between the European Commission and the aeronautics industry, to run until 31 December 2016. Its objectives were adopted from the ACARE Strategic Research Agenda (SRA [9, 13]) and targeted, at aircraft level, to reduce CO2 emissions between 20-40%, NOx by around 60% and noise by up to 10dB compared to the reference aircraft using year 2000 technologies. The Clean Sky Joint Undertaking was established to manage the programme and the European Commission, as a co-founding member, was responsible for setting up the JU Executive. A two year process to build up the legal and financial framework, in cooperation with several other newly minted JTIs, was completed in November 2009 and, with the demonstrated capacity to manage its own budget, the Clean Sky JU was granted autonomy.

Figure 4 Early logo of CS as from the JTI proposal (March 2007) [8]

The Founding Members of the CSJU were thus the European Union, represented by the European Commission (EC), and 12 Integrated Technology Demonstrator (ITD) leaders and 74 Associates. These 86 organisations in 16 countries, among which 54 industries, including 20 SMEs, 15 Research Centres, and 17 Universities, were later joined by over 500 partners selected through calls for proposals. The JU work plan was based on the ‘top down’ vision of the “Joint Technical Programme”. The Clean Sky JTI was ‘operational’ during the formation phase under the leadership of Liam Breslin, DG-RTD Transport and supported by members of the H3 unit as well as by an extensive investment from other Commission units providing legal and financial expertise to the establishment of the FP7 family of JUs.

Early 2000’s Recognition of a lack of large scale projects

2001 “Vision for 2020” published by “personalities” in followed by the Strategic and innovation agenda (SRA) developed by ACARE

app. 2005 Initiative by informal industrial group.

June 2006 Clean Sky Workshop - by the AeroSpace and Defence Industries of Europe (ASD), takes place in Brussels for candidate associates and partners to express their views 200 representatives from

industry, research centres and universities participate in the workshop.

23 June 2006 “Study on the proposed Aeronautics JTI structure and rules of participation” Report on JTI structure and Rules for Participation, Bertolini, Huguet

October 2006 AgustaWestland, Airbus, Alenia Aeronautica, Dassault Aviation, Eurocopter, Liebherr-Aerospace, Rolls-Royce, Safran and Thales sign a Memorandum of Understanding (MoU) for ‘Clean Sky

2006-2007 Definition of the ITD content

February 2007 Saab, EADS CASA and Fraunhofer join the MoU and become leaders.

March 2007 “CLEAN SKY Aeronautics & Air transport JTI Proposal” submitted

May 2007-May 2008 Within FP7 the CSA project was launched “Clean Sky Support

Action” with a budget of 2 M€ for preparing the JU, the future leaders where the partners in this project which was coordinated by Airbus SAS.

13 June 2007 Ex-Ante Evaluation of Clean Sky – Impact Assessment

20 June 2007 Paris Air Show at Le Bourget - European Science and Research

Commissioner, Janez Potocnik, gives a press conference to announce the Clean Sky JTI.

December 2007 Legal establishment of CS as a community body involving a PPP under the EU Financial Regulations.

2008 Accomplishment of the Technical evaluation of Clean Sky

4 February 2008 Clean Sky regulations and statutes are published in the

Official Journal of the European Union.

September 2008 First ITD coordination meeting in that period.

15 June 2009 Launch of Clean Sky’s first Call for Proposals.

20 November 2009 Full autonomy of the Clean Sky JTI

Table 4 Timeline for the establishment of the CSJU

In comparison to the “Single European Sky ATM Research Programme (SESAR) JU, established in 2007, Clean Sky has been a more industry-led response to the opportunity in FP7 to realise a program of high level objectives in an overall JTI programme. The SESAR JU was charged with providing the technical foundations of the SES (Single European Sky) and to execute the results of the “definition phase” and the European ATM master plan and follows another funding scheme (see also Table 5). Clean Sky JU SESAR JU

Basic

objective

Achievement of ACARE targets for

environmental impact mitigation

Technical foundations for the realisation of

the SES and maintenance of the “European ATM master plan”.

Funding principle

Equally shared public-private investment

Equally shared public-private investment

Commission

sponsor

DG Transport (DG-RTD-H3) DG Move.

Membership Proposed consortium of OEMs in the aeronautics industry accepted by EC. Associates by selection through EoI

Open call for members.

Legal status Joint undertaking (JU) organised as

Joint technology initiative (JTI)

Joint undertaking, but not a JTI.

Scope a research agenda focused on demonstrator realisation for a range of aeronautics products

“Monopoly” in ATM research:“The SESAR project aims to integrate and coordinate research and development activities which

were previously undertaken in a dispersed and uncoordinated manner in the Community, including the most remote and outlying regions thereof, referred to in Article 299(2) of the Treaty.”

Table 5 Comparison between key characteristics between CSJU and SESAR JU

6.2 Structure of Clean Sky JU

The Clean Sky JU was implemented precisely as required by its governing regulation [1] including the governing Statutes in Annex 1. The central decision making body is the Governing Board (GB) comprised of the European Community, represented by the Commission, and 18 representatives of the private members, the ITD Leaders (12, as named in the statutes) and one Associate representing each ITD on a rotating basis. Each member has one vote and the EC has a veto in strategic, financial and procedural matters affecting the responsibilities of the Commission as a public trustee. An ITD Steering Committee for each of the 6 ITD’s is responsible for the technical and financial management of the ITD participants and there is a Steering Committee, established by a GB decision, for the Technology Evaluator. The JU Programme Office is led by the Executive Director (ED) and


28

consists of a support staff unit (i.e. administrative, legal, financial, human resources etc.) and a project coordination role headed by the Chief Project Officer (to the good for the programme, a person who has been involved in Clean Sky since its inception). The day to day management of the research agenda is done by the team of Project Officers. Partners who were joined to the programme through selection in calls for project proposals are members of the General Forum (GF) together with the other members of the JU and by two thirds majority may make recommendations and raise issues to the Governing Board. The GF meets annually under the rules of procedure adopted by the GB and, after a poorly attended meeting in June 2010, in the form of a (generally lively) workshop dedicated to informing and discussing the ‘state of play’ of the JU. The ED is further supported by a statutory National States Representatives Group (NSRG) and has formed a Scientific and Technical Advisory Board (STAB) in addition. An overview of the governance structure is given by Figure 5.

Figure 5 Governance structure of the CSJU

6.3 Budget allocation

The overall EC contribution of 800 M€ was, in view of the need to reach out to a very broad aeronautical research community, allocated for 50% to the founding members (as ITD Leaders and their Affiliates) and for 25% to the Associates that allied to the programmes in its formative stages, often in the form of ‘clusters’). Such clustering of synergic capabilities in relation to a Clean Sky objective allowed a diversity of entities, including SME’s, to meet the financial commitments required to achieve ‘Associate’ status. All of these participating entities matched equally the EC contribution to their scope of work with ‘in kind contributions’. In addition the Members also contributed to the running costs of the JU Programme Office according to the establishing regulation [1] The remaining 25% of the EC contribution was ‘ring-fenced’ for disbursement in ‘calls for proposals’ (broadly under FP7 guidelines) that drew shorter term Partners into the programme at funding rates between 50% and 75%, depending on the nature of the activity and the type of organisation. The overall average funding rate was 65%. CS1

EU contribution 800 M€

Share for Leaders (members) 50% 400 M€

Associates 25% 200 M€

Partners 25% 200 M€ Table 6 EU contribution broken down to the different types of participants

The proportion of the budget allocated to each ITD was intended to match approximately with the sectorial share, expressed by the GDP contribution of the sector (large passenger aircraft, engine, helicopters etc.). The result was:

Figure 6 EU funding distribution to the different ITD's following appr. sector share according to the establishing

regulation [1]

While initially the mandated 50/25/25 distribution between Leaders, Affiliates and Partners was flowed down through all levels of the Work Breakdown Structure it was found to be a constraint on the varying nature of the support needed at a lower level. The requirement was then interpreted as a global one and the JU Programme Office monitored its realisation. Similarly, in October 2013 the remaining overall budget for Partners was re-assigned at JU level as a flexible approach for the last call planning exercise. Additional insight in the operational execution of the programme can be found in the level of funding to various types of work. For example wind tunnel testing was an important precursor of flight test demonstrations and was used to eliminate concepts and optimise the final configuration. Similarly, a significant effort would be expected in manufacturing both in hardware and tooling to realise a demonstrator and appropriate ground and air vehicle test facility were also a significant investment. Unfortunately the JU could not provide this breakdown. SFWA GRA GRC SAGE SGO ECO

Concept and configuration studies

Wind tunnel models and testing

Manufacturing -tooling and materials

Demonstrator Test Facility

Innovative Instrumentation Figure 7 Categories of activity per ITD

There was no contingency budget in CS1 to adapt financing for unforeseen costs, etc. The CS1 Interim Evaluations and the CS2 Impact Assessment [15] therefore proposed the introduction of an contingency budget to be allocated by the GB based on proposals by the ED. In 2014 a “de facto”


30

contingency budget was created based on a 10 M€ savings from JU running costs and an Eco-Design ITD contribution. The allocation of these extra funds was agreed by GB after a ranking of the different possible projects by the JU ED, according to a few criteria.

6.4 Leaders and Associates

The selection of the ITD Leaders as members of the JU evolved from the industrials that developed the JTI Proposal with additions influenced by the nature of the work that was foreseen in the Clean Sky Programme. The resulting designation of ITD Leaders and Affiliates, as captured in Annex II A of the Council Regulation [1] is shown below. It is noted that the ownership and trade name of several members changed in the course of the programme but the original naming is used here for continuity.

AgustaWestland

Airbus: Affiliates: Airbus France Sas, Airbus Deutschland Gmbh, Airbus Espaňa Sl, Airbus Uk Limited

Alenia Affiliates: Alenia Aermacchi Spa, Alenia Sia Spa

Dassault Aviation

EADS-CASA

Eurocopter: Affiliates: Eurocopter Deutschland Gmbh

Fraunhofer Gesellschaft

Liebherr: Affiliates: Liebherr-Aerospace Toulouse S.A.S., Liebherr-Elektronik Gmbh

Thales Affiliates: Thales Atm, Thales Systèmes Aéroportés, Thales Avionics Electrical System, Thales Communication, Thales Air Systems Division Uk

Rolls-Royce: Affiliates: Rolls-Royce Deutschland Gmbh

Saab

Safran Affiliates: Snecma, Turbomeca, Hispano Suiza, Aircelle, Techspace Aero, Snecma Propulsion Solide, Microturbo, Technofan, Sofrance, Messier Dowty, Messier Bugatti,

Labinal, Sagem Sécurité Défense, Snecma Services, Sma

Table 7 Leaders and their affiliates

The Commission and the ITD Leaders undertook to engage the programme Associates in advance of the formation of the JU to give certainty of achieving the requisite critical mass of participants before launching the programme. ITD Leaders prepared descriptions of the scope of work that might be undertaken by an Associate and several briefings were held to introduce the Clean Sky JTI Proposal and invite the submission of Expressions of Interest to the ITD Leaders. Diverse channels were used in this period (2006-2007) to reach potentially interested parties. A selection for each ITD was made, based on a set of ‘excellence’ criteria, by the respective ITD Leader and the Commission representative on the basis of interviews by invitation with candidates. Thus, although an effort to achieve openness and transparency was made, it was not a rigorously orderly process. For example, not all interested candidates were given the consideration of an interview, or reasons for the lack of an invitation. The resulting selection and co-option by the founding members are listed under Annex II, B in the corresponding Council Regulation.

6.5 Call for partners 2008-2013

As the ‘bottom up’ work plan for the Clean Sky programme progressed in 2008 and 2009, the subject areas requiring the contribution the additional ‘Partners’ foreseen in the Council Regulation [1] acquired form and the first Call for proposals (CfP) was issued in June 2009. In total there were 16 calls, with topics as approved by the GB, from then until the FP7 closing deadline at the end of 2013. The system for “calls for proposal (CfP)” in Clean Sky introduced a new quality of call topic description structure and content in the landscape of European aeronautics research. Instead of proposing relatively broad topics for a wider range of applicants, as in the traditional framework programs, there was a well-defined scope of work, including pre-set deliverables and milestones, budget and set of qualifications in the topic description and only one “winning” proposal was accepted (Table 8). The call topics to be proposed to the CSJU are developed in the ITDs/IADPs/TAs and the respective steering committees. The topic descriptions were reviewed by the responsible JU project officer to ensure that sensitive details were not being disclosed and that it was indeed a research topic rather than a matter which could have been subcontracted. Furthermore, on the basis of feedback on the early calls, project officers learned to screen carefully the requested qualifications to ensure the broadest possible eligibility of proposers. The call texts are later on also reviewed by the different parties that need to adopt the respective topics in the GB, such as the Commission. Also the STAB obtained selected call texts for expert comment. In addition, a specific derogation from the Seventh Framework Rules for Participation [16] permitted a single beneficiary to submit a proposal rather than requiring a multinational consortium. The detailed procedures related to CfPs were contained in the Clean Sky Management Manual (CSMM) [17] and evolved with the experience gained. A comparison with the key characteristics with respect to collaborative projects is given by Table 8.

Table 8 Key characteristics showing differences between call topics characteristics in Clean Sky and in traditional collaborative projects.

Call description – Clean Sky CfP Call description - collaborative project

Budget Budget is relatively fixed, variations e.g. due to different funding rates

Broad budget indication

Milestones/ Deliverables

Prescribed by the ITD steering committee

To be provided by applicants

Expert panel 2 experts from EU-pool, 1 experts

from topic manager

Experts from EU-pool

Composition of consortium

No restrictions, preference for small consortia

At least 3 partners from three different eligible countries


32

The first Clean Sky Calls for Proposal (CfP) were launched in July 2009, 5 months before the JU was granted autonomy and was thus still under the direction of the Commission staff. This release of this call, and the second 2009 CfP in November, were followed by Information Days held in Brussels to elaborate on the call content and procedures to potential applicants. Some eligibility issues, related to exceedance by applicants of the ‘topic value’ as indicated in the topic description, were resolved for the 2010 and successive calls. All evaluations were conducted in the presence of two independent reviewers, and a representative of the responsible leader and no significant anomalies were reported by them. Scrupulous redress procedures were implemented and all redress requests were resolved without change to the final ranking list for the respective call. The redress rate was at no time excessive. Figure 8 shows the number of published topics in each call, 16 in all from 2009 to 2013, finally 655 topics has been published, but not for all topics, an appropriate appllication could be selected.

Figure 9 Number of topics published for all calls from 1 to 16 [18]

The breakdown of numbers of topic with respect to the different ITDs is shown in Figure 10. The distribution is fairly respecting the funding share, although, while some ITDs were ready with topic descriptions early in the programme, other ITDs took longer to identify their needs. This is shown in Figure 10. The 4th call (Call 2010-02) is anomalous as it consisted of only 4 SFWA topics that were time sensitive and the GB

Final Evaluation of the Clean Sky Joint Undertaking …...Clean Sky Final Evaluation Report 30 June...

Documents

Transcript of Final Evaluation of the Clean Sky Joint Undertaking …...Clean Sky Final Evaluation Report 30 June...