Pre-Commercial Procurement proposal - HNSciCloud 13 May 2015 Bob Jones, CERN.

14
Pre-Commercial Procurement proposal - HNSciCloud 13 May 2015 Bob Jones, CERN

Transcript of Pre-Commercial Procurement proposal - HNSciCloud 13 May 2015 Bob Jones, CERN.

Page 1: Pre-Commercial Procurement proposal - HNSciCloud 13 May 2015 Bob Jones, CERN.

Pre-Commercial Procurement proposal - HNSciCloud

13 May 2015 Bob Jones, CERN

Page 2: Pre-Commercial Procurement proposal - HNSciCloud 13 May 2015 Bob Jones, CERN.

Pre-Commercial Procurement proposal

• A Pre-Commercial Procurement proposal was submitted to the H2020 ICT 8a call in April 2015 for innovative IaaS cloud services

• The proposal is currently under evaluation by the EC• Official feedback expected in September• These slides give an overview of the proposal

Page 3: Pre-Commercial Procurement proposal - HNSciCloud 13 May 2015 Bob Jones, CERN.

The Helix Nebula Initiative

Strategic Plan

Establish multi-tenant, multi-provider cloud infrastructure

Identify and adopt policies for trust, security and privacy

Create governance structure

Define funding schemes

To support the computing capacity needs for the LHC

experiments

Setting up a new service to simplify analysis of large genomes, for a

deeper insight into evolution and biodiversity

To create an Earth Observation

platform, focusing on earthquake and

volcano research

To improve the speed and quality of research for finding

surrogate biomarkers based on brain

images

Adopters

Suppliers

Additional Users:

3 Bob Jones, CERN

Page 4: Pre-Commercial Procurement proposal - HNSciCloud 13 May 2015 Bob Jones, CERN.

• H2020 Coordination and Support Action

• Total budget: 500 K€• Duration: 18 months • Start date: 1 Oct. 2014• Partners:

4

Page 5: Pre-Commercial Procurement proposal - HNSciCloud 13 May 2015 Bob Jones, CERN.

Example HNX procurementCERN price enquiry for a small fraction of the resources necessary to run the ATLAS experiment simulation software

Several valid responses were received and Atos (data centre in Tenerife, Spain) was awarded the contract accessed via the SlipStream broker and provisioning engine (from SixSq SME) and delivered over the GÉANT network1.2 million CPU hours over 6 weeks producing 11.5 million GEANT4 tt̅� events with a high level of efficiency for CPU intensive workload that confirmed

the use of commercial IaaS is technically feasible for simulation workloadsvariations in RESTful APIs can be overcome with reasonable technical effortindependent monitoring of the quality of service and resources consumed is essential

Bob Jones, CERN5

Page 6: Pre-Commercial Procurement proposal - HNSciCloud 13 May 2015 Bob Jones, CERN.

• Hybrid – link public research organisations, e-Infrastructures & commercial cloud services• Use GEANT network to link Research Infrastructures, repositories

(EUDAT, OpenAIRE), EGI, PRACE etc. to commodity commercial cloud services (multiple providers)

• A cornerstone of the Open Science Commons*

• Trust - Researchers keep control of the cloud and their data• Guarantee a copy of all the data is kept on public resources• Ensure long-term preservation of the data• Insulate users from changes of service supplier and technology

• Economy - Must be cheaper than the ‘build our own’ approach• Avoid separate ‘silos’ for each Research Infrastructure/Community• Profit from the economies of scale in commercial data centres

* http://go.egi.eu/osc

Towards the EuropeanOpen Science Cloud

http://dx.doi.org/10.5281/zenodo.16140

Page 7: Pre-Commercial Procurement proposal - HNSciCloud 13 May 2015 Bob Jones, CERN.

PCP PPI

7

Why PCP?Commercial IaaS exists but not certified, integrated with public e-infrastructures, offering std interfaces with suitable SLA and contractual terms & conditions.

PPIPotential follow-on project if this PCP project is successful

Page 8: Pre-Commercial Procurement proposal - HNSciCloud 13 May 2015 Bob Jones, CERN.

Proposed Joint Pre-Commercial ProcurementThe group of buyers have committed• >1.6M€ of funds• Manpower• Applications & Data• In-house IT resources

To procure innovative cloud services

Integrated into a hybrid cloud model:• Commercial cloud services• e-Infrastructures

• GEANT network• eduGAIN Fed. Id mgmt.• EGI Fed Cloud

• Potentially host data services from EUDAT, INDIGO-Datacloud, etc.

• In-house IT resources

Made available to end-users including BBMI, DARIAH, ELIXIR, EISCAT_3D, EPOS, INSTRUCT, LifeWatch, LHC, etc. Bob Jones, CERN 8

Page 9: Pre-Commercial Procurement proposal - HNSciCloud 13 May 2015 Bob Jones, CERN.

Preparation

• Analysis of requirements, current market offers and relevant standards

• Build stakeholder group

• Develop tender material

Implementation Sharing

• Best practises• Recommendations• Training

Oct’16

PCP project phases

Launch of tender Pilots tested & assessed

6 months 18 months 6 months

Jan’16

Jan’18

Jun’18

All procuring organisations must sign letter to re-confirm commitment of funds

DesignPrototy

pe

Pilot

Page 10: Pre-Commercial Procurement proposal - HNSciCloud 13 May 2015 Bob Jones, CERN.

PCP project implementation phase

Design Prototype

Pilot

Bidders provide written design reportincluding architecture and technical design of components together with unit costs

Contractors build prototypes including all components and make it accessible to experts from buyers group that perform functionality and security tests

Contractors deploy expanded prototypes that are made available firstly to experts who perform scalability tests and then to end-users

Bidders have 3 months to respond to bid

select ≥3 designs taking into account tender criteria

select ≥2 prototypes taking into account feedback from experts

Evaluation committee includes representatives from all buyers

Multiple iterations with feedback to suppliers in prototype and pilot phases

Page 11: Pre-Commercial Procurement proposal - HNSciCloud 13 May 2015 Bob Jones, CERN.

PCP project implementation phase3 phases:• Solution design

• ≥3 designs• 3 months • 15% tender budget

• Prototype development• ≥3 prototypes• 6 months• 25% tender budget

• Deployment of limited scale prototypes• ≥2 pilot deployments • 6 months• 60% tender budget

Design15%

Prototype25%Pilot

60%

Total procurement funds available: ~5M€Each phase is competitive. Bids evaluated/tested against published criteria

Page 12: Pre-Commercial Procurement proposal - HNSciCloud 13 May 2015 Bob Jones, CERN.

Project organisation

WP1: Consortium Management

WP2: Preparation of procurement &Tendering

WP3: Design Platform

WP4: Prototype Platform

WP5: Pilot Platform

WP6: Pilots Evaluation & Recommendation

WP7: Stakeholder Engagement, Dissemination, Impact & Exploitation

CERN

CERN CNRS DESY INFN EGI.eu

TRUST-IT

The EC funding covers the procurement costs, project and procurement process administration as well as work package leadershipAll the buyers provide technical effort to evaluate designs/prototypes/pilots

Page 13: Pre-Commercial Procurement proposal - HNSciCloud 13 May 2015 Bob Jones, CERN.

User GroupsThe cloud resources procured will be made available to user groups during the pilot phase

• LHC experiments via WLCG• Procured resources will count against the buyers’ pledges (during pilot phase)• CERN will provide the interface via Tier-0 (OpenStack in tender spec.)

• ELIXIR• Managed by EMBL-EBI via the ELIXIR Compute platform

• Other research communities via EGI Fed Cloud• Request OCCI interface in tender spec.

• Local users at each buyers site• Each buyer is responsible for integration• Request web GUI interface in tender spec.

• Sites can also use OpenStack or OCCI interface as well

Each buyer decides what fraction of their procured resources is made available to each user group but cannot assign only to their local users

Collectively the users will form a user group with a role in the project to define requirements and provide feedback on pilot deployments

BBMRIDARIAHEISCAT_3DEPOSINSTRUCTLifeWatchLong-tail

Page 14: Pre-Commercial Procurement proposal - HNSciCloud 13 May 2015 Bob Jones, CERN.

Next Steps• CERN will perform another limited scale procurement in

preparation for the PCP• More details at the 6th Helix Nebula General Assembly @ CERN

• 24-25 June restricted to NDA signatories https://indico.cern.ch/event/369721/

• 26 June open day http://indico.cern.ch/e/OpenDay

• Assuming the PCP proposal is funded• September-December 2015:

• Approval of changes to CERN procurement rules to support EC co-funded PCP

• Prepare EC contract and consortium agreement• Make more detailed planning of tasks• Start work on tender material• Investigate to support data-intensive workloads

• Project start January 2016