Cloud computing with Openstack - HPC Advisory Council€¦ · Openstack Cloud computing with...

OpenstackCloud computing with Openstack

Lugano, 23/03/2016

Saverio Proto

[email protected]

© 2016 SWITCH

• SWITCH role in Openstack and Cloud Computing

• What is Virtualization ?

• Why is Cloud computing more than Virtualization ?

• How Openstack implements Cloud Computing ?

Agenda

2

© 2016 SWITCH

• Openstack public cloud with two regions in Switzerland

– https://www.switch.ch/engines/

• SWITCHengines provides compute and storage services

to Swiss academics. Users of participating institutions

have on-demand access to create and manage virtual

machines and storage.

• Two Regions

– Zurich

– Lausanne

SWITCHengines

3

https://www.switch.ch/engines/

© 2016 SWITCH

• SWITCHengines has been in “public beta” and “internal

production” since December 2014

• Productive service since January 2016

• Several SWITCH services run on it (SWITCHdrive,

SWITCHfilesender, SWITCHtube)

• Over 630 individual users and around 40 research projects

online

• Around 700 VMs running on the cloud right now (2200

cores, 4900 GB RAM, 400 TB of storage allocated)

• Started with Openstack Icehouse, we have upgraded the

system to Juno (8/2015) and then to Kilo (3/2016)

Openstack Current status at SWITCH

4

© 2016 SWITCH

• Key idea: Hypervisor and virtual machines– Commodity technology today

– Typically GUI driven

– Scale-UP approach

– Operators manage each Hypervisor independently

• Well known Hypervisors, and date of birth– VMWare 2001

– Xen 2003

– Virtualbox 2007

– KVM 2007

https://en.wikipedia.org/wiki/Timeline_of_virtualization_development

The starting point: Virtualization

5

© 2016 SWITCH

• How to manage a datacenter with many hypervisors ?

– Most of the management is by hand with GUI/WEB based tools

• What happens if I lose a physical host ?

– Hypervisors are built with a lot of redundancy to prevent failures

– All the hypervisors in a legacy datacenter are Pets, you really have

to make sure they are healthy

• How to manage multi-tenant use cases ?

– Not all the VMs belong to the same project

– Separation is possible but configured by hand

Virtualization before Cloud Computing

6

© 2016 SWITCH

Virtualization – networking

7

Network

Hypervisor

ETH0 br0

VM1

eth0

VM2

eth0

© 2016 SWITCH

Virtualization – Reference Network

8

Data

cente

r S

witch

• Network configuration can

become complex and hard

to manage by hand

• In case of multi-tenancy

VLAN coordination needed

by network-admin and

server-admin

© 2016 SWITCH

• Needs redundancy for hypervisors hardware

• Needs redundancy for networking hardware

• A lot of manual configuration

• No standard strategy for multi-tenancy

Legacy Virtualization - Cons

9

© 2016 SWITCH

• AWS: Started in March 2006 (10 years ago)

• Openstack: Started in 2010

Cloud is already 10 years old

10

© 2016 SWITCH

• In a Public cloud the services are offered to anyone over

the Internet.

– Operators and users of the cloud are usually in different domains

• In a Private cloud the access is restricted to some users

for private use

– Operators and users of the cloud are usually from the same company

Public and Private Cloud

11

© 2016 SWITCH

• It is design to fail

– Hardware is divided in two main categories, Pets and Cattle

– You will need to introduce redundancy only for your Pets

– Cattle are allowed to fail

• API based Control Plane

– The configuration of new VMs is done interacting with a Controller

– API interfaces make possible to automate the configuration.

– Requests for new virtual machines are processed by a controller that

will schedule the machine to right hypervisor.

What is new with Cloud Computing?

12

© 2016 SWITCH

• It is a change regarding Uptime responsibility.

• Hypervisors can fail, this means that VMs should be

disposable.

• Developers are required to design the application in a way

that a server can be lost.

• The application should exploit the new abstractions

provided by the cloud.

Pet and Cattle

13

© 2016 SWITCH

• Scale-up mindset

– Make a server redundant and powefull

Web Application Example: Legacy

14

Web Server VM

Database VM

Hypervisor1

RAID5 disks

Hostname www.mysite.com

Standby VM

Database Backup

Hypervisor2 backup

RAID5 disks

© 2016 SWITCH

Web Application Example: Cloud

15

vm01 vm02 vm03

LBaaS

FloatingIP

DBaaS Object Storage

• Scale-out mindset

• VMs are disposable

• Database is provided by

the Cloud Infrastructure

• Object Storage is

persistent

Autoscaling VMs on load

Hostname

www.mysite.com

© 2016 SWITCH

• API can be accessed via CLI, Web based interface, and

libraries are provided to implement own tools

openstack server create --flavor c1.micro \

--image “Ubuntu 14.04” \

--nic net-id=private \

myVMname

API based Control Plane

16

© 2016 SWITCH

• The best hypervisor is selected automatically

– User starting a VM do not know how many hypervisors

• Multi-tenancy is by default

– Each user have visibility only of his own VMs

– VMs from the same user will appear on the same network

API based Control Plane

17

© 2016 SWITCH

• To make scale-out possible, a cloud computing provides the following abstractions:

• Web Services– FloatingIPs and LBaaS

• Databases– DBaaS : the user receives the endpoint and credentials

– Real DB is managed by Cloud Ops in multi master replication

• Storage– Object store

Scale out

18

© 2016 SWITCH

• Scalable

• Access natively via HTTP

• Replication of objects

New abstraction: Object Storage

19

Server

File SystemFile SystemFile System File System File System

Server

Object storage

User

© 2016 SWITCH

• Quickly move an IP address to another instance or to

another service. The backend is independent.

New abstraction: floating IP

20

vm01

FloatingIP FloatingIP

vm01 Bigger VMvm01 vm02 vm03

LBaaS

FloatingIP

© 2016 SWITCH

• Main components

– Keystone: authentication and authorization

– Nova: Virtual Machine orchestrator

– Cinder: Volume manager

– Glance: Image manager

– Neutron: Network controller

• The openstack big tent

– Horizon

– Heat

– Trove

– Ceilometer

Openstack components

21

© 2016 SWITCH

• Keystone provides a token based authorization to the

Openstack API services.

• export [email protected]

• export OS_PASSWORD=<secret>

• export [email protected]

• export OS_AUTH_URL=https://keystone.cloud.switch.ch:5000/v2.0

• export OS_REGION_NAME=LS

• Keystone publishes a catalog of the current API endpoint

for services

Keystone

22

© 2016 SWITCH

• Cloud Federation is possible with keystone

– However requires Keystone V3 API

– Federation enables Cloud Bursting

• Keystone development is today very active

– Token structures changed

Keystone

23

© 2016 SWITCH

• Cinder is the volume manager in Openstack

• Because we said that VMs are disposable, we have two kind of volumes– Ephemeral volumes

– Persistent volumes

• Ephemeral volumes– Controlled directly by Nova, the disk of the VMs is destroyed when

the VM is destroyed. Created when we boot from image.

• Persistent– Is a volume independent from a VM. You can attach the volume to

multiple VMs

Cinder

25

© 2016 SWITCH

• VMs from the same project appear on the same network

regardless of the hypervisor where they are scheduled

• Hypervisors are running OpenVSWITCH

• An Overlay network is built among the Hypervisors

– Different tunnels are possible (VXLAN, GRE)

Neutron

26

© 2016 SWITCH

• Nova is a cloud computing fabric controller

• It is software layer that hides to the user the hypervisor

technology chosen to implement the cloud.

• The cloud can use even different hypervisor technology at

the same time.

• Supports KVM, XEN, VMWare, Hyper-V, Bare-metal,

Containers

Nova

28

© 2016 SWITCH

Nova API

29

KVM hypervisor

VM

NOVA API

VMVMVMVM

XEN hypervisor

User

• User requests a VM to Nova

• Based in the flavor nova schedules the VM to the

appropriate hypervisor

© 2016 SWITCH

• http://docs.openstack.org/liberty/networking-

guide/scenario-classic-ovs.html

Reference scenario - Hardware

30

© 2016 SWITCH

• Openstack is a great Open Source software ecosystem

– It makes possible to anyone to deploy a cloud

• Application should be designed to exploit the new

abstractions

– Cloud Native Applications

• Cloud design with Pets and Cattles reduces infrastructure

costs

Conclusion

33

© 2016 SWITCH

• ATLAS Experiment at CERN, UNIBE is using

SWITCHengines 24/7 with 300 Cores

• ATLAS Instances on SWITCHengines

– 8cores

– 16GB RAM

– 20 GB Disk

Use case on SWITCHengines

35

Cloud computing with Openstack - HPC Advisory Council€¦ · Openstack Cloud computing with...

Documents

Transcript of Cloud computing with Openstack - HPC Advisory Council€¦ · Openstack Cloud computing with...