Network Multitenancy in Xen-Based Clouds-XPUS13 Vittal

Network Mul,tenancy in Xen-‐based Clouds

Chiradeep Vi;al CloudStack Commi;er

Citrix Systems @chiradeep Sep 18 2013

Agenda

•  Introduc,on to CloudStack •  Mul,-‐tenant IAAS •  Network Virtualiza,on / SDN •  L3 isola,on •  CloudStack’s Network Model •  CloudStack’s na,ve SDN approach

!•  Product from Cloud.com /

Citrix (thru acquisition)!•  Open Source since May 2010!•  Donated by Citrix to the ASF

(Apr 2012)!•  Graduated as Top-level

Project in March 2013!•  In production since 2009!•  Tons of deployments,

including large-scale commercial ones!

Apache CloudStack

How did Amazon build its cloud?

Commodity Servers

Commodity Storage Networking

Open Source Xen Hypervisor

Amazon Orchestration Software

AWS API (EC2, S3, …)

Amazon eCommerce Platform

How can YOU build a Xen-‐based cloud?

Servers Storage Networking

Open Source Xen Hypervisor

Amazon Orchestration Software

AWS API (EC2, S3, …)

Amazon eCommerce Platform

Hypervisor (XenServer/XCP)

CloudStack Orchestration Software

Optional Portal

CloudStack or AWS API

Secondary Storage Image

L3/L2 core

DC Edge

End users

Pod Pod Pod Pod

Zone Architecture

Pod

Access Sw

MySQL

CloudStack

Admin/User API

Primary Storage NFS/ISCSI/FC

Hypervisor (Xen /VMWare/KVM)

VM

VM

Snapshot

Snapshot

Image

Disk Disk

VM

L3/L2 core

Mul,-‐tenancy

Hypervisor

A

C

A

B

A

C

A

A

Internet

Mul,-‐,er virtual networking

!Virtual appliance/!Hardware Devices!

Customer!Premises!

IPSec or SSL site-to-site VPN!

Internet!

Network Services!•  IPAM!•  DNS!•  LB [intra]!•  S-2-S VPN!•  Static Routes!•  ACLs!•  NAT, PF!•  FW [ingress & egress]!

Loadbalancer (virtual or HW)!

MPLS VLAN!

Web VM 1!

Web VM 2!

Web VM 3!

Web VM 4!

Web subnet !10.1.1.0/24!

App subnet 10.1.2.0/24!

App VM 1!

App VM 2!

DB Subnet!10.1.3.0/24!

DB VM 1!

Network Isola,on Op,ons

•  L2 Isola,on – Each network / ,er is a separate subnet – Overlapping IP addresses (between networks) allowed

– L2 adjacency between VMs in same network – Mul,cast / broadcast may be allowed.


•  L3 Isola,on – Mul,ple tenants / applica,on ,ers on the same physical subnet

–  Isolated at IP (L3). – No L2 adjacency in the same ,er / tenant – No Mul,cast / Broadcast


•  PVLAN – Mul,ple tenants are placed on the same L2 domain.

– Only allowed to communicate via upstream router – No mul,cast or broadcast (except ARP) – Limited use cases

L2 Isola,on Op,ons •  Network Virtualiza,on

–  The illusion of isolated networks on top of shared physical infrastructure

•  VLAN –  Old, reliable technology, use OVS or bridge –  4k limit (12 bit VLAN id) –  All usable VLANs need to be trunked down to all hypervisors

•  Overlays (“SDN”) –  E.g., GRE, STT, VxLAN –  Currently only GRE available in Xen (with OVS) –  GRE tunnels are established between hypervisors to carry

Ethernet frames between VMs on the same network –  Requires orchestrator / SDN controller to manage overlays

Network Virtualiza,on in IAAS

Tenant 1 VM 1!Tenant 1 VM 2!Tenant 1 VM 3!Tenant 1 VM 4!

Public Network

Tenant 1 Virtual Network 10.1.1.0/24

Gateway address 10.1.1.1

NAT!DHCP!FW

Public IP address 65.37.141.11!65.37.141.36

10.1.1.2

10.1.1.3

10.1.1.4

10.1.1.5

Tenant 1 !Edge

Services Appliance(s)!Internet!



Public Network



NAT!DHCP!FW


10.1.1.2

10.1.1.3

10.1.1.4

10.1.1.5

Tenant 1 !Edge

Services Appliance(s)!

Internet!

Tenant 1 !Edge


Load Balancing!VPN


Internet!


Public Network



NAT!DHCP!FW


10.1.1.2

10.1.1.3

10.1.1.4

10.1.1.5

Tenant 1 !Edge


Tenant 2 VM 2!Tenant 2 VM 3!

Tenant 2 VM 1!



VPN!NAT!DHCP

10.1.1.2

10.1.1.3

10.1.1.4

Tenant 2 !Edge

Services!


Tenant 1 !Edge

Service(s)!

Load Balancing!

L3/L2 core

DC Edge

Pod Pod Pod Pod Pod

Access Sw

Internet!


Public Network



NAT!DHCP!FW


10.1.1.2

10.1.1.3

10.1.1.4

10.1.1.5

Tenant 1 !Edge Services

Appliance(s)!

Tenant 2 VM 2!Tenant 2 VM 3!

Tenant 2 VM 1!



VPN!NAT!DHCP

10.1.1.2

10.1.1.3

10.1.1.4

Tenant 2 !Edge

Services!


Tenant 1 !Edge

Service(s)!

Load Balancing!

CloudStack’s Network Virtualiza,on

VM A1! VM A2! VM B1! VM C1!

vswitch!

Physical !Nics!

Virtual Nics!

vswitch! vswitch!

VLAN 10!VLAN 20!VLAN 30!

untagged (usually)!

VLAN TRUNK!192.168.1.0/24!

192.168.1.0/24!

10.1.1.0/24!

VLAN example!

…

GRE tunnel example!

… …

User 2 User

1

User 1

User 1

User 1

User 2

OVS

OVS

OVS

OVS OVS

GRE Key 2 GRE Key 1

CloudStack + SDN Technologies

•  Nicira NVP •  Midokura MidoNet •  Nuage •  BigSwitch •  Stratosphere •  Coming soon

– Open Daylight –  Juniper

L3 isola,on with distributed firewalls Tenant 1 VM 1 10.1.0.2

Tenant 2 VM 1 10.1.0.3

Tenant 1 VM 2 10.1.0.4

Tenant 2 VM 2 10.1.16.12

Tenant 2 VM 3 10.1.16.21

Tenant 1 VM 3 10.1.16.47

Tenant 1 VM 4 10.1.16.85

Public Internet

10.1.0.1

Public IP address 65.37.141.11 65.37.141.24 65.37.141.36 65.37.141.80

Load Balancer

L3 Core

Pod 1 L2 Switch

Pod 3 L2 Switch

10.1.16.1

…

… 10.1.8.1

Pod 2 L2 Switch

L3 Isola,on in CloudStack + Xen

•  CloudStack orchestrates dom0 firewall (iptables)

•  Requires iptables across bridge and ‘ipset’ package

•  Does not work with OVS •  Scales to tens of thousands of vms and tenants

CloudStack Network Model: Network Services

Network Services

•  L2 connec,vity

•  IPAM •  DNS •  Rou,ng •  ACL •  Firewall •  NAT •  VPN •  LB •  IDS •  IPS

Network Isola?on

•  No isola,on •  VLAN

isola,on •  Overlays •  L3 isola,on

Service Providers!

ü  Virtual appliances!

ü  Hardware firewalls!

ü  LB appliances!

ü  SDN controllers!

ü  IDS /IPS appliances!

ü  VRF!ü  Hypervisor!

Service Catalog

•  Cloud users are not exposed to the nature of the service provider

•  Cloud operator designs a service catalog and offers them to end users. –  Gold = {LB + FW, using virtual appliances} –  Platinum = {LB + FW + VPN, using hardware

appliances} –  Silver = {FW using virtual appliances, 10Mbps}

Service Catalog examples

10.1.1.0/24!VLAN 100

10.1.1.1

DHCP, DNS!NAT!Load Balancing!VPN

10.1.1.2

VM 1!

10.1.1.3

VM 2!

10.1.1.4

VM 3!

10.1.1.5

VM 4!

CS!Virtual Router!

L2 network with software appliances!

65.37.141.111!65.37.141.112

Service Catalog examples

10.1.1.0/24!VLAN 100

10.1.1.1

DHCP, DNS!NAT!Load Balancing!VPN

10.1.1.2

VM 1!

10.1.1.3

VM 2!

10.1.1.4

VM 3!

10.1.1.5

VM 4!

CS!Virtual Router!

L2 network with software appliances!

65.37.141.111!65.37.141.112

10.1.1.0/24!VLAN 100

DHCP, DNS!

CS!Virtual Router!

10.1.1.112

65.37.141.112

10.1.1.2

VM 1!

10.1.1.3

VM 2!

10.1.1.4

VM 3!

10.1.1.5

VM 4!

Netscaler!Load

Balancer!

10.1.1.1

65.37.141.111

Juniper

SRX!Firewall!

L2 network with hardware appliances!

NAT, VPN!

Upgrade

More Info

•  CloudStack Wiki – h;ps://cwiki.apache.org/confluence/x/fwDFAQ

•  CloudStack Docs – h;p://cloudstack.apache.org/docs/en-‐US/index.html

•  Mailing Lists – h;p://cloudstack.apache.org/mailing-‐lists.html

•  IRC – Freenode #cloudstack-‐dev, #cloudstack

Network Multitenancy in Xen-Based Clouds-XPUS13 Vittal

Technology

Transcript of Network Multitenancy in Xen-Based Clouds-XPUS13 Vittal