Understanding Virtual Networking in the Cloud - RightScale Compute 2013

54
april25-26 sanfrancisco cloud success starts here Understanding and Managing MultiCloud Networking Josep M. Blanquer, Chief Architect

Transcript of Understanding Virtual Networking in the Cloud - RightScale Compute 2013

Page 1: Understanding Virtual Networking in the Cloud - RightScale Compute 2013

april25-26 sanfrancisco

cloud success starts here

Understanding and ManagingMultiCloud NetworkingJosep M. Blanquer, Chief Architect

Page 2: Understanding Virtual Networking in the Cloud - RightScale Compute 2013

# 2# 2

#RightscaleCompute

In this talk…• Introduction and Goals• Landscape

• Public: AWS / GCE / Azure / Rackspace…• Private: CloudStack / Eucalyptus / OpenStack…

• MultiCloud Resource Abstractions• Resource Hierarchy, Naming and Semantics• Managing these resources through the UI and API

• Conclusion

Page 3: Understanding Virtual Networking in the Cloud - RightScale Compute 2013

# 3# 3

#RightscaleCompute

Intro• Networking is messy…

Page 4: Understanding Virtual Networking in the Cloud - RightScale Compute 2013

# 4# 4

#RightscaleCompute

Introduction• Networking is messy…even in the Cloud!

• Different Cloud Providers pick different designs• Leads to different exposed API resources, different behavior• Also leads to different naming conventions, and APIs semantics

• Cloud software can also be heavily customized on installation• So even for the same cloud type, two clouds can behave quite

differently

• All of this changes very rapidly• New versions of APIs, expose new resources• Some changes break semantic compatibility or become defaults

Page 5: Understanding Virtual Networking in the Cloud - RightScale Compute 2013

# 5# 5

#RightscaleCompute

Introduction (contd.)• So what does this mean for me? (you must be

wondering…)• Headaches, and possible hair loss

Page 6: Understanding Virtual Networking in the Cloud - RightScale Compute 2013

# 6# 6

#RightscaleCompute

Introduction (contd.)• But… mess and variability is not bad, it is necessary

• In fact, it is great!• Companies need choice and configuration flexibility• One size doesn’t fit all

• You must embrace it• Take advantage of the features and characteristics that make

sense for you• But not at the cost of loosing focus on your business

• So• Instead of grooming an army of experts on cloud networking • Let others do that for you so you don’t have to

“Maintain control, without having to be bogged down with non-business details”

Page 7: Understanding Virtual Networking in the Cloud - RightScale Compute 2013

# 7# 7

#RightscaleCompute

• Don’t look at your cloud networking from this perspective

Introduction (contd.)

Page 8: Understanding Virtual Networking in the Cloud - RightScale Compute 2013

# 8# 8

#RightscaleCompute

• …look at your cloud networking from this perspective

Introduction (contd.)

Page 9: Understanding Virtual Networking in the Cloud - RightScale Compute 2013

# 9

#rightscalecompute

Cloud Networking LandscapeDifferent strokes for different folks

Page 10: Understanding Virtual Networking in the Cloud - RightScale Compute 2013

# 10# 10

#RightscaleCompute

Cloud Networking Landscape

• Embracing the choices• Amazon EC2• Google Compute Engine• CloudStack

• Not covered today: Azure, Rackspace, Eucalyptus, Openstack…

Page 11: Understanding Virtual Networking in the Cloud - RightScale Compute 2013

# 11# 11

#RightscaleCompute

Amazon EC2

• Each region can have multiple VPCs• Each VPC defines a network isolation perimeter• Incoming/Outgoing communication must go through GW

Amazon EC2

EC2 Regio

n

VP

Cs

x N

GW

Page 12: Understanding Virtual Networking in the Cloud - RightScale Compute 2013

# 12# 12

#RightscaleCompute

Amazon EC2

• Subnets further segment VPCs into IP CIDR groups• Instances can be connected to a Subnet through an ENI• A Subnet is scoped to a single Availability Zone

Amazon EC2

Subnets

Elastic NetworkInterfaces

Subnet 1

Elastic NetworkInterfaces

Subnet 2

Elastic NetworkInterfaces

Subnet 3AZ 1

EC2 Regio

n

VP

Cs

x N

GW

AZ 1AZ 2

Page 13: Understanding Virtual Networking in the Cloud - RightScale Compute 2013

# 13# 13

#RightscaleCompute

Amazon EC2

• A VPC also scopes (and therefore contains)• SecurityGroups• Routing Tables• Network ACLs

Security Groups

Amazon EC2

Routing Tables

Network ACLs

Subnets

Elastic NetworkInterfaces

Subnet 1

Elastic NetworkInterfaces

Subnet 2

Elastic NetworkInterfaces

Subnet 3AZ 1

EC2 Regio

n

VP

Cs

x N

GW

AZ 1AZ 2

Page 14: Understanding Virtual Networking in the Cloud - RightScale Compute 2013

# 14# 14

#RightscaleCompute

Amazon EC2

• Instances can be bound to multiple Subnets (of a matching AZ)• The Security Groups are bound to each attached ENI

• And not to the Instance as a whole

Security Groups

Amazon EC2

Routing Tables

Network ACLs

Subnets

Elastic NetworkInterfaces

Subnet 1

Elastic NetworkInterfaces

Subnet 2

Elastic NetworkInterfaces

Subnet 3AZ 1

EC2 Regio

n

VP

Cs

AZ 1AZ 2

x N

GW

AZ 1AZ 2

Page 15: Understanding Virtual Networking in the Cloud - RightScale Compute 2013

# 15# 15

#RightscaleCompute

Amazon EC2 (Classic)

• There is a single (implicit) network for each region• Incoming/Outgoing traffic is fully NATted

Amazon EC2

EC2 Regio

n

Sin

gle

Ne

two

rkx

1

NAT

Page 16: Understanding Virtual Networking in the Cloud - RightScale Compute 2013

# 16# 16

#RightscaleCompute

Amazon EC2 (Classic)

• There aren’t any Subnets, Routing Tables or Network ACLs• Security Groups are scoped to the implicit single Network

Security Groups

Amazon EC2

Routing Tables

Network ACLs

SubnetsEC2 Regio

n

Sin

gle

Ne

two

rkx

1

NAT

Page 17: Understanding Virtual Networking in the Cloud - RightScale Compute 2013

# 17# 17

#RightscaleCompute

Amazon EC2 (Classic)

Security Groups

Amazon EC2

Routing Tables

Network ACLs

SubnetsEC2 Regio

n

Sin

gle

Ne

two

rk

AZ 1AZ 2

x 1

NAT

• There aren’t any subnets, routing tables or Network ACLs• Security Groups are scoped to the implicit single Network

• And their rules apply to the Instance as a whole (only 1 implicit Interface)

Page 18: Understanding Virtual Networking in the Cloud - RightScale Compute 2013

# 18# 18

#RightscaleCompute

Google Compute Engine

• GCE cloud is global: there aren’t different regional endpoints• Networks within the cloud define a network isolation perimeter• Incoming/Outgoing communication must go through the GW

Amazon EC2

Global

Ne

two

rks

x N

GW

Page 19: Understanding Virtual Networking in the Cloud - RightScale Compute 2013

# 19# 19

#RightscaleCompute

Google Compute Engine

• A Network cannot be further segmented• A Network has firewalls (some functionality is close to a SG)• Routing controls are currently not exposed

Firewalls (SG-like)

Amazon EC2

Firewalls

Global

Ne

two

rks

x N

Subnets

GW

Routing Tables

Page 20: Understanding Virtual Networking in the Cloud - RightScale Compute 2013

# 20# 20

#RightscaleCompute

Google Compute Engine

• A Network can span multiple Zones• And Firewall rules can be applied to instances in a global way

Firewalls (SG-like)

Amazon EC2

Firewalls

Global

Ne

two

rks

x N

Subnets

GW

Routing Tables

Zone 1

Zone 2

Page 21: Understanding Virtual Networking in the Cloud - RightScale Compute 2013

# 21# 21

#RightscaleCompute

CloudStack: Basic Mode

• Flat Networking (modeled after EC2 Classic)• One (Shared) Network per Zone

Amazon EC2

No Regions

Ne

two

rkx N

NAT

Page 22: Understanding Virtual Networking in the Cloud - RightScale Compute 2013

# 22# 22

#RightscaleCompute

CloudStack: Basic Mode

• Supports SecurityGroups• But they belong to the “Domain” and apply to all uses of the shared

network

Security Groups

Amazon EC2

Subnets

Routing Tables

Network ACLs

NAT

Ne

two

rk

No Regions

x N

Page 23: Understanding Virtual Networking in the Cloud - RightScale Compute 2013

# 23# 23

#RightscaleCompute

CloudStack: Basic Mode

• Instances within a Network are scoped to a Zone• Each instance can have multiple SecurityGroups attached to it

Security Groups

Amazon EC2

Subnets

Routing Tables

Network ACLsZone 1

Zone 1

NAT

Ne

two

rk

No Regions

x N

Page 24: Understanding Virtual Networking in the Cloud - RightScale Compute 2013

# 24# 24

#RightscaleCompute

CloudStack: Advanced Mode

• A Cloud can have multiple Networks• Each Network is scoped to a Zone

Amazon EC2N

etw

ork

s

GW

No Regions

x N

Page 25: Understanding Virtual Networking in the Cloud - RightScale Compute 2013

# 25# 25

#RightscaleCompute

CloudStack: Advanced Mode

• There is no further segmentation based on Subnets• Supports Firewalls (and SGs if the network is shared)

Amazon EC2

Firewalls

Ne

two

rks

Zone 1

x N

Subnets

GW

Security Groups

Zone 1

Routing Tables

* Except KVM

No Regions

Page 26: Understanding Virtual Networking in the Cloud - RightScale Compute 2013

# 26# 26

#RightscaleCompute

CloudStack: Advanced Mode (VPC)

• A Cloud can have multiple VPCs• A VPC is scoped to a Zone

Amazon EC2V

PC

s

x N

GW

No Regions

Page 27: Understanding Virtual Networking in the Cloud - RightScale Compute 2013

# 27# 27

#RightscaleCompute

CloudStack: Advanced Mode (VPC)

• A VPC is segmented by Tiers (still scoped to a Zone)• No explicit Network interface support in API

Amazon EC2V

PC

s

x N

Tiers

Elastic NetworkInterfaces

Tier 1

Elastic NetworkInterfaces

Tier 2

Elastic NetworkInterfaces

Tier 3

GW

Zone 1

Zone 1

Zone 1

No Regions

Page 28: Understanding Virtual Networking in the Cloud - RightScale Compute 2013

# 28# 28

#RightscaleCompute

CloudStack: Advanced Mode (VPC)

• Support for:• Static Routing• Firewalls

Amazon EC2

Firewalls

VP

Cs

x N

Security Groups Tiers

Elastic NetworkInterfaces

Tier 1

Elastic NetworkInterfaces

Tier 2

Elastic NetworkInterfaces

Tier 3

GW

Zone 1

Zone 1

Zone 1

Routing Tables

No Regions

Page 29: Understanding Virtual Networking in the Cloud - RightScale Compute 2013

# 29# 29

#RightscaleCompute

CloudStack: Advanced Mode (VPC)

• Note: a CloudStack cloud can mix all 3 networking modes:• Basic, Advanced and VPC• The mode is set at the Zone level

Amazon EC2

Firewalls

VP

Cs

Zone 1

x N

Security Groups

Zone 1

Tiers

Elastic NetworkInterfaces

Tier 1

Elastic NetworkInterfaces

Tier 2

Elastic NetworkInterfaces

Tier 3

GW

Zone 1

Zone 1

Zone 1

Routing Tables

No Regions

Page 30: Understanding Virtual Networking in the Cloud - RightScale Compute 2013

# 30

#rightscalecompute

Multicloud Resource AbstractionsRightscale’s Abstractions

Page 31: Understanding Virtual Networking in the Cloud - RightScale Compute 2013

# 31# 31

#RightscaleCompute

MultiCloud Resource Hierarchy

Cloud

Networks

Instances

Subnets NetworkInterfaces

IpAddressBindings

SecurityGroups

Network ACLs

Routing Tables

IpAddresses

Images

Volume Snapshots

Volumes

Datacenters

Page 32: Understanding Virtual Networking in the Cloud - RightScale Compute 2013

# 32# 32

#RightscaleCompute

Multicloud Network Abstractions

• A Cloud has multiple Networks• A Network defines an isolation perimeter (and has a CIDR block)• Incoming/Outgoing communication must go through GWs

Amazon EC2

Cloud

Ne

two

rks

x N

GW

Page 33: Understanding Virtual Networking in the Cloud - RightScale Compute 2013

# 33# 33

#RightscaleCompute

Multicloud Network Abstractions

• Subnets further segment Networks into IP CIDR sub-blocks• Instances can be connected to a Subnet through NetworkInterfaces• A Subnet is scoped to one (or zero) Datacenters

Amazon EC2

Subnets

NetworkInterfaces

Subnet 1

NetworkInterfaces

Subnet 2

NetworkInterfaces

Subnet 3

Cloud

Ne

two

rks

x N

GW

DC 1

No DCDC 2

Page 34: Understanding Virtual Networking in the Cloud - RightScale Compute 2013

# 34# 34

#RightscaleCompute

Multicloud Network Abstractions

• Networks contain:• SecurityGroups• Routing Tables• Network ACLs

Security Groups

Amazon EC2

Routing Tables

Network ACLs

Subnets

NetworkInterfaces

Subnet 1

NetworkInterfaces

Subnet 2

NetworkInterfaces

Subnet 3

Cloud

Ne

two

rks

x N

GW

DC 1

No DCDC 2

Page 35: Understanding Virtual Networking in the Cloud - RightScale Compute 2013

# 35# 35

#RightscaleCompute

Multicloud Network Abstractions

• Instances are launched within a Datacenter (placement)• Instances connected to multiple Subnets via Network Interfaces

(connectivity)• Connectivity restrictions may apply based on the Cloud.

• SecurityGroups are bound to Network Interfaces (i.e, different rules per subnet)

Security Groups

Amazon EC2

Routing Tables

Network ACLs

Subnets

NetworkInterfaces

Subnet 1

NetworkInterfaces

Subnet 2

NetworkInterfaces

Subnet 3

Cloud

Ne

two

rks

x N

GW

DC 1

No DCDC 2

DC 1DC 2

Page 36: Understanding Virtual Networking in the Cloud - RightScale Compute 2013

# 36# 36

#RightscaleCompute

Multicloud Network Abstractions

Security Groups

Amazon EC2

Routing Tables

Network ACLs

Subnets

NetworkInterfaces

Subnet 1

NetworkInterfaces

Subnet 2

NetworkInterfaces

Subnet 3

Cloud

Ne

two

rks

x N

GW

DC 1

No DCDC 2

DC 1DC 2

Page 37: Understanding Virtual Networking in the Cloud - RightScale Compute 2013

# 37# 37

#RightscaleCompute

Multicloud Network Abstractions

Security Groups

Amazon EC2

Routing Tables

Network ACLs

Subnets

NetworkInterfaces

Subnet 1

NetworkInterfaces

Subnet 2

NetworkInterfaces

Subnet 3

Cloud

Ne

two

rks

x N

GW

DC 1

No DCDC 2

VolumesImages +Volume Snapshots

No DC

Datacenters

DC 1

DC 2

DC 2

DC 1

DC 1DC 2

Page 38: Understanding Virtual Networking in the Cloud - RightScale Compute 2013

# 38# 38

#RightscaleCompute

Multicloud Network Abstractions

Security Groups

Amazon EC2

Routing Tables

Network ACLs

Subnets

NetworkInterfaces

Subnet 1

NetworkInterfaces

Subnet 2

NetworkInterfaces

Subnet 3

Cloud

Ne

two

rks

x N

GW

DC 1

No DCDC 2

VolumesImages +Volume Snapshots

No DC

Datacenters

DC 1

DC 2

DC 2

DC 1

IP Addresses(assignable)

IpAddress BindingsInstance+[IP]+[ports]DC 1

DC 2

Page 39: Understanding Virtual Networking in the Cloud - RightScale Compute 2013

# 39# 39

#RightscaleCompute

Managing Multicloud Resources• Accessible both through our new UI and API• It presents a single interface for your cloud Network

infrastructure• Aggregates resources across regions, providers and software

versions.• Network/Security operators design and analyze from a single

pane of glass• Infrastructure operators can manage those abstractions in

deployments

• How will this look in the UI?...

Page 40: Understanding Virtual Networking in the Cloud - RightScale Compute 2013

# 40# 40

#RightscaleCompute

Managing Multicloud Resources: UI

Page 41: Understanding Virtual Networking in the Cloud - RightScale Compute 2013

# 41# 41

#RightscaleCompute

Managing Multicloud Resources: UI: Awesome Game US (East)

Page 42: Understanding Virtual Networking in the Cloud - RightScale Compute 2013

# 42# 42

#RightscaleCompute

Managing Multicloud Resources: UI: Awesome Game US (East)

Page 43: Understanding Virtual Networking in the Cloud - RightScale Compute 2013

# 43# 43

#RightscaleCompute

Managing Multicloud Resources: UI: Awesome Game US (East)

Page 44: Understanding Virtual Networking in the Cloud - RightScale Compute 2013

# 44# 44

#RightscaleCompute

Managing Multicloud Resources: UI: Awesome Game US (East)

Page 45: Understanding Virtual Networking in the Cloud - RightScale Compute 2013

# 45# 45

#RightscaleCompute

Managing Multicloud Resources: UI: Awesome Game US (East)

Page 46: Understanding Virtual Networking in the Cloud - RightScale Compute 2013

# 46# 46

#RightscaleCompute

Managing Multicloud Resources: API• RESTful API : multicloud as of version 1.5

• Creating a Network/Subnet• New resources, very simple attributes (Name, CIDR…)

POST /api/networks{

name : “Foobar App Network”,cidr_block : “10.1.2.0/24”,cloud_href : “/api/clouds/1234”,tenancy : “default”

}

HTTP Code: 201 CreatedLocation: /api/networks/10

Page 47: Understanding Virtual Networking in the Cloud - RightScale Compute 2013

# 47# 47

#RightscaleCompute

Managing Multicloud Resources: API• Creating a Server

• Can specify which Network it belongs to• Can set the list of subnets it needs to be attached to (or default

subnet)• Alternatively, can specify which already existing Network

Interfaces to attachPOST /api/servers{

name: “My Foobar Server”,network_href : “/api/networks/10”,subnet_hrefs : [ “/api/subnets/11”, “/api/subnets/12” ],security_group_href : [ “/api/security_groups/6”,

“/api/security_groups/7” ],datacenter_href : “/api/datacenters/1”,

…cloud_settings, server_template, inputs …}

HTTP Code: 201 CreatedLocation: “/api/servers/50”

Page 48: Understanding Virtual Networking in the Cloud - RightScale Compute 2013

# 48# 48

#RightscaleCompute

Managing Multicloud Resources: API• IpAddressBinding resource also manage ports:

• Attaching an IP without port ranges maps all ports of the IP to the instance

• An IpAddress can be restricted to a port range (for clouds that support it)

POST /api/ip_address_bindings{

instance_href : “/api/instances/1”,public_ip_address_href : “/api/ip_addresses/2”,protocol : “tcp”,public_port : 80, *optionalprivate_port: 8080 *optional

}

HTTP Code: 201 CreatedLocation: /api/ip_address_bindings/9

Page 49: Understanding Virtual Networking in the Cloud - RightScale Compute 2013

# 49# 49

#RightscaleCompute

Managing Multicloud Resources: API• Available soon:

• Networks• Subnets• SecurityGroups (bound to Networks an NetworkInterfaces)• IpAddresses / Bindings (with the port forwarding abstractions)

• Routing tables and Network ACLs• API and UI are being designed• Implementation not started yet• But expect being able to create/delete routes and rules soon

Page 50: Understanding Virtual Networking in the Cloud - RightScale Compute 2013

# 50# 50

#RightscaleCompute

Note on Synthetic Resources• What about resources that are required but non-existent

in cloud?• A server can be connected to subnets (and SecurityGroups

through them)

• We will create (wrap) these resource synthetically for you• So you can have consistency for clients using the API.

• Example: Subnets in Amazon EC2 classic

Page 51: Understanding Virtual Networking in the Cloud - RightScale Compute 2013

# 51# 51

#RightscaleCompute

Synthetic Resources for EC2 Classic

• EC2 classic doesn’t have subnets• But you still want to create your servers using the same abstractions

Security Groups

Amazon EC2

Routing Tables

Network ACLs

SubnetsEC2 Regio

n

Sin

gle

Ne

two

rk

AZ 1AZ 2

x 1

NAT

Page 52: Understanding Virtual Networking in the Cloud - RightScale Compute 2013

# 52# 52

#RightscaleCompute

Synthetic Resources for EC2 Classic

• We will create a Synthetic Network to refer to the implicit classic EC2 Network

• We will create one Synthetic Subnet for each available Datacenter• So you can specify the server configuration in a consistent manner• Regardless of EC2 Classic, Amazon VPC, or any other clouds

Security Groups

Amazon EC2

Routing Tables

Network ACLs

Synthetic SubnetsEC2 Regio

n

Sin

gle

Ne

two

rkx

1

NATSynthetic

Interface 1

Synth Subnet 1

Elastic NetworkInterfaces

Synth Subnet 2

Elastic NetworkInterfaces

Synth Subnet 3DC 1

DC 2DC 3

DC 1DC 3

Page 53: Understanding Virtual Networking in the Cloud - RightScale Compute 2013

# 53# 53

#RightscaleCompute

Summary• Cloud Networking is messy and it varies greatly• But choice and configurability is very important

• RightScale abstractions allow you to• Operate and manage your Cloud networking from a single pane

of glass• Using higher level, easier abstractions• While keeping the power to go down to the guts when needed• Available through a both UI and API• Portable across clouds, cloud providers and cloud versions

• Give it a try• Manage your Networking more consistently, and at a higher level• While still taking advantage of the cloud features that make

sense for you• But not at the cost of loosing focus on your business• You don’t have to be a multicloud user to get the advantages…

Page 54: Understanding Virtual Networking in the Cloud - RightScale Compute 2013

april25-26 sanfrancisco

cloud success starts here

Questions?