1

Mobile Cloud

David Wallom

University of Oxford

The NIST Cloud Definition Framework

CommunityCloud

Private Cloud

Public Cloud

Hybrid Clouds

DeploymentModels

ServiceModels

EssentialCharacteristics

Common Characteristics

Software as a Service (SaaS)

Platform as a Service (PaaS)

Infrastructure as a Service (IaaS)

Resource Pooling

High Perf Network Access Rapid Elasticity

Measured Service

On Demand Self-Service

Low Cost Software

Virtualization Service Orientation

Advanced Security

Homogeneity

Massive Scale Resilient Computing

Geographic Distribution

Based upon original chart created by Alex Dowbor - http://ornot.wordpress.com

3

What I think mobile cloud is?

• User

– Utilise any cloud [group, institutional, Community, Public] for any useful applications

– Utilise any suitable tool to make usage easier [Slipstream, Starcluster, a.n.other]

• Resource Provider

– Free choice which cloud system to deploy, migrate when necessary

– Support utilisation of public cloud

• Technology Provider

– Open marketplace where product quality decide uptake

4

• Globally distributed;

• different resources/costs;

• different applications;

• non standardised: different AAA and UI.

Private/Public Multiple Clouds

Users

CLIMB cloudAmazon cloud

Embassy cloud

EGI cloud partner

Azure cloud

5

Mediated Private/Public Multiple Clouds

Management Interface

CLIMB cloudAmazon cloud

Embassy cloud

EGI cloud partner

Users

• Automation;

• load balancing;

• costs reduction;

• usability.

6

• Federation of Local and Global resources

• Elasticity managed by local cloud not user

• different resources/cost;

• different applications;

• non standardised: different AAA but single UI through private provider

Hybrid Multiple Clouds

Users

CLIMB cloud

Amazon cloud

Embassy cloudEGI cloud partner

7

Rationale for mobility

Growth of Providers• High Throughput/Performance Computing

–Academic resource providers• Federated Cloud

–Diversity of resource providers

7

Tens of 1000’s MillionsFew related use cases

Few application modelsMany diverse use cases & application models

Growth of Research Communities

804/15/2023

Value proposition

The EGI Federated Cloud, a federation of institutional private Clouds, offering Cloud Services to researchers in Europe and worldwide

A single cloud system able to • Scale to user needs• Integrate multiple different providers to give resilience• Prevent vendor lock-in • Enable resource provision targeted towards the research

community

Standards based federation of IaaS cloud: • Exposes a set of independent cloud services accessible to users

utilising a common standards profile• Allows deployment of services across multiple providers and

capacity bursting

904/15/2023

EGI Cloud Infrastructure

EGI Core Platform

Federated AAIServiceRegistry Monitoring Accounting

EGI Cloud Infrastructure Platform

VM Management

Information Discovery

Storage Management

Help and Support

Security Co-ordination

Training and Outreach

EGI C

olla

bora

tion

Tool

s

EGI A

pplic

ation

D

BIm

age

Repo

sito

ryEG

I Clo

ud S

ervi

ce M

arke

tpla

ce

Sustainable Business Models

User Communities

Monitoring and control of utilisation

Technical Consultancy and Support

Uniform interfaces to Cloud Compute and Storage

Cloud Management Stacks(OpenStack, OpenNebula, Synnefo, …)

Cloud Management Stacks(OpenStack, OpenNebula, Synnefo, …)Resource Provider InfrastructureResource Provider Infrastructure

Secu

re e

ndor

sed

Appl

icati

on a

nd

Serv

ice

Dep

loym

ent

1004/15/2023

On demand compute to run any kind of workloads on virtual machines

• OCCI as standard interface– Contextualization extension– Profiling to ensure interoperability

• Clients– Commandline– ruby and java clients– OCCI connectors in brokers

• Servers– rOCCI-server

• Open Nebula• StratusLab*• Apache Cloudstack• AWS

– OCCI-OS– snf-occi

VM Management

VM VM

Site A

VM VM

Site B

1104/15/2023

Automatic and secure distribution of endorsed VM images for Virtual Organisations

• Web based front-end in AppDB

• OVF for packaging images

• HEPiX lists for distribution– Endorsed by VO managers– Signed metadata– RP subscribe and download

• EGI endorses basic OS images

VM Image Management

Site B

VMImage

Site A

VMImage Site C

VMImage

EGI AppDBVM

Image

1204/15/2023

Block Storage

• Persistent Block Level Storage to attach to VMs

• Manage through VM OCCI

Object Storage

• Data storage accessible fro anywhere at any time

• Sharing/Serve data• CDMI as standard API

Storage Management

VM

1304/15/2023

Integration with EGI core services:

Accounting

• Collects VM usage metrics in a central accounting repository

1404/15/2023

Integration with EGI core services:

Monitoring

• Ensure service availability and reliability

1504/15/2023

FedCloud Infrastructure

• One year of production• Resources

– 21 providers from 14 NGIs• 55% Openstack, 42% Open

Nebula, 3% Syneffo

– 17 interested in joining from 7 new NGIs

• Usage~700K VMs ~9M CPU hours wall time

EGI Conference 2015

1604/15/2023

Usage Models

Supporting multiple diverse usage models:• Service Hosting

– Long-running services (e.g. web, database or application servers)

• Compute and data intensive workloads– Batch and interactive (e.g. IPython, R, matlab) with

scalable and customized environments not limited to the traditional job model

• Datasets repository– Store and manage large datasets for your applications

• Disposable and testing environments– Host training events, test new developments and

applications without overhead

1704/15/2023

High Level Tools (PaaS & SaaS)

• Extend the IaaS capabilities of the EGI cloud• ‘Alternatives’ of the OCCI client and API• More than OCCI

• External contributions ( support many other clouds too)

• SaaS• Identity Federation

Catania Science

Gateway

• PaaS for automating deployments

• Helix NebulaSlipstream

• Programming framework for auto-parallelisationCOMPSs

• Abstraction on top of various HPC/HTC/cloudVMDIRAC

• Workflow development and enactmentWS-PGRADE

• VM lifecycle managerVcycle

1804/15/2023

Federated Cloud user engagement and support

Dedicated technical consultancy for each community (support@egi.eu)

• Step by step guides• Tutorials• Examples

Doc

• Main OS available• Secure endorsed• Contextualisation

Basic VM Images

• Resource providers commitment

• Support to create a production VO

Migration into

production

• Initial requirements• Allocation of technical

experts• Milestones

F2F/Web Meetings

• Technical integration support from EGI/NGI team

• Periodic meetings

Continuous tracking and

support

• Resources for application prototyping

• Enabled in all sites• Up to 6 months (renew)

Incubator VO

1904/15/2023

Use Case Discipline ClassificationUsecases- 12 @ Launch- 76 to date

- 11 production

2004/15/2023

ESA HN Flagship on the EGI Federated Cloud

The European Space Agency deployed part of the ESA Helix Nebula Flagship into the EGI Federated Cloud:

1. Use SlipStream Infrastructure Broker to deploy a custom computing cluster on multiple Federated Cloud sites (CESNET, GRNET)

2. Connect the computing cluster to the ESA G-POD system and run SAR processing for volcano and earthquake monitoring

Automatic Cluster Deployment

EGI Federated CloudComputing Cluster

Cluster Head

Working Nodes

OCCI

Globus

Job submission

BLUE BOX

2104/15/2023

Peachnote

Peachnote is a music score search engine and analysis platform.

Hundreds of thousands of music scores are being digitized by libraries all over the world. In contrast to books, they generally remain inaccessible for content-based retrieval and algorithmic analysis.

There is no analogue to Google Books for music scores, and no large corpora exists that can empower advanced analysis on music scores.

Peachnote want to help change that providing visitors and researchers access to a massive amount of symbolic music data.

EGI Federated Cloud

OMR Worker

PDF Splitter

OMR Feeder

OMR Worker

…

2204/15/2023

OpenModeller on the Biovel Portal

The Ecological Niche Modeling (ENM) Workflow takes as input a file containing species occurrence points to create a model with the openModeller Web Service.

• The EUBrazilOpenBio ENM service is exposed through an extended openModeller Web Service interface• Multi-staging and multi-parametric oM experiments are implemented through COMPSs that dynamically creates the virtual resources to execute the operations.• An OCCI connector is used for the VMs management while data management supports CDMI endpoints.

ENM Service (OMWS2)

VENUS-C Cloud Middleware

COMPSs Workflow Orchestrator

COMPSs Workflow OrchestratorOCCIOCCI CDMICDMI

EGI Federated Cloud

Service available athttps://portal.biovel.eu/

2304/15/2023

OpenRefine on the BioVel Portal

It is used within BioVeL for taxonomic data to ensure data quality and integrity.

The service integrates existing know-how of taxonomic data with existing functionality in Google Refine to develop BioVeL specific data cleaning tools. Service available at

https://portal.biovel.eu/

OpenRefine (originally developed as Google Refine) is a framework for viewing and cleaning large amounts of messy data.

2404/15/2023

BioSTIF - Biodiversity Spatio-temportal interface on the BioVel Portal

BioSTIF provides an easy GIS interface to vizualize and quality control biodiversity data in time and space

The interface contains; • a map widget (displaying

geographical data)• a timeline widget

(displaying data on a time scale)

• a table widget (displaying data in table format).

Service available athttps://portal.biovel.eu/

25EGI Conference 2015 25

26

27

Conclusions

• There are a number of different models of mobile cloud available, each with different pros + cons

• EGI Federated Cloud is one way of providing mobile cloud [that demonstrably works]

• Most EU countries ahead of UK after years of being behind

• If you want production services operations takes time and effort