CloudStack Networking
-
Upload
cloudstack-open-source-cloud-computing-project -
Category
Technology
-
view
26.435 -
download
2
Transcript of CloudStack Networking
CloudStack Networking
Chiradeep VittalMay 2 2012
Outline
• CloudStack Networking Features• CloudStack Networking Configuration• CloudStack Networking APIs• CloudStack Network Architecture• Virtual Router deep dive
Feature overview• Orchestration of L2 – L7 network services
– IPAM, DNS, Gateway, Firewall, NAT, LB, VPN, etc• Mix-and-match services and providers• Out-of-the-box integration with automated deployment of virtual routers
– Highly available network services using CloudStack HA and VRRP• Orchestrate external providers such as hardware firewalls and load
balancers– Devices can provide multiple services– Admin API to configure external devices– Plugin-based extensions for network behavior and admin API extensions
• Multiple multi-tenancy [network isolation] options• Integrated traffic accounting• Access control • Software Defined Networking too
Basic vs Advanced Networking
• Segmentation based on feature set and ease-of-deployment
• Both are feature-rich• Basic implements true AWS-style L3-isolation
– Tenants do not get contiguous IP addresses or subnets– Network segmentation based on Security Groups– Tremendous scale (tens of thousands)
• Advanced Zone offers full L3 subnets– VLANs are default implementation (4K limit)– More features (source NAT, PF, VPN)
CloudStack Terminology
• Guest network– The tenant network to which instances are attached
• Storage network– The physical network which connects the hypervisor to primary storage
• Management network– Control Plane traffic between CloudStack management server and hypervisor clusters
• Public network– “Outside” the cloud [usually Internet]– Shared public VLANs trunked down to all hypervisors
• All traffic can be multiplexed on to the same underlying physical network using VLANs
– Usually Management network is untagged– Storage network usually on separate nic (or bond)
• Admin informs CloudStack how to map these network types to the underlying physical network
– Configure traffic labels on the hypervisor– Configure traffic labels on Admin UI
Storage 1
Hypervisor 1
Hypervisor N
Hypervisor 8
Access Switch(es)CloudstackServer
VM Traffic
Control Plane Traffic
Storage Traffic
CloudstackServers
Storage k
…
Pod 1
CLUSTER 1
…
CLUSTER 4
Core (L3) Network
…
Pod 2 Pod N
PHYSICAL NETWORK IN A ZONE
Storage 2
Hypervisor N+1
Public Traffic
L2 Features• Choice of network isolation
– Physical, VLAN, L3 (anti-spoof), Overlay[GRE]– Physical isolation through network labels [limited to # of nics or bonds]
• Multi-nic– Deploy instance in multiple networks– Control default route
• Access control– Shared networks, project networks– Dedicated VLANs offer MPLS integration
• Anti-spoofing for L3-isolated networks• QoS [max rate]• Traffic monitoring• Broadcast & multicast suppression in L3-isolated networks• Hot-plug / detach of nics [upcoming]
L3 Features• IPAM [DHCP], Public IP address management
– VR acts as DHCP server– Can request multiple public IPs per tenant
• Gateway (default gateway)– Redundant VR (using VRRP)– Inter-subnet routing [upcoming]– Static routing control [upcoming]
• Remote Access VPN– L2TP over IPSec using PSK– Virtual Router only
• Firewall based on source cidr• Static NAT [1:1]
– Including “Elastic IP” in Basic Zone• Source NAT
– Per-network, or interface NAT• Public Traffic usage
– Monitoring on the Virtual Router / External network device – Integration with sFlow collectors
• Site-to-Site VPN [upcoming]– IPSec VPN based on VR
• L3 ACLs [upcoming]
L4 Features
• Security groups for L3-isolation– “Basic Zone” in docs– Default AWS-style networking– Scales much better than VLANs
• Stateful firewall for TCP, UDP and ICMP• Port forwarding [“Advanced Zone”]
– Conserve public Ips
L7 features• Loadbalancer
– VR has HAProxy built in– External Loadbalancer support
• Netscaler (MPX/SDX/VPX)• F5 BigIP• Can dedicate an LB appliance to an account or share it among tenants
– Loadbalancer supported with L3-isolation as well– Stickiness support– SSL support [future]– Health Checks [future]
• User-data & meta-data– Fetched from virtual router
• Password change server
Router
L3 Core Switch
Access Layer
Switches
………… …
Availability Zone
Servers
CloudStack Mgmt Server Cluster
Secondary Storage
Pod 1 Pod 2 Pod 3 Pod N
MySQL
Load Balancer
Operations Admin and Cloud API
Users
Physical Network
…
DB Security Group
WebSecurity Group
Layer 3 cloud networking
… …
Web VM
Web VM
Web VM
Web VM
DB VM
Web VM
DB VM
Web VM
Guest Networks with L3 isolationGuest
1 VM 1
Guest 2 VM 1
Guest 1 VM 2
Guest 2 VM 2
Public Internet
10.1.0.1
Public IP address 65.37.141.1165.37.141.2465.37.141.3665.37.141.80
Guest address 10.1.0.2Guest address 10.1.0.3Guest address 10.1.0.4
Guest address 10.1.16.12
Load Balancer
Guest 2 VM 3
Guest 1 VM 3
Guest 1 VM 4
Guest address 10.1.16.21Guest address 10.1.16.47Guest address 10.1.16.85
L3 Core Switch
Pod 1 L2 Switch
Pod 3 L2 Switch
10.1.16.1
…
…10.1.8.1Pod 2 L2 Switch
Hypervisor 1
Hypervisor N
Hypervisor 8
Access Switch(es)
VM Traffic
…
Pod K
CLUSTER 1
…
CLUSTER 4
Core (L3) Network
…
Pod M Pod N
Virtual Networks (L2 isolation)
Hypervisor N+1
Public Traffic
Hypervisor
R
R V
VV
V
HypervisorV V
V
RTenant VMTenant Virtual Router
Guest virtual layer-2 network
Guest 1 VM 1
Guest 1 VM 2
Guest 1 VM 3
Guest 1 VM 4
Public Internet
Public Network
Guest Virtual Network 10.1.1.0/24
Gateway address 10.1.1.1
NATDHCPLoad BalancingVPN
Public IP address 65.37.141.1165.37.141.36
Guest address 10.1.1.2Guest address 10.1.1.3Guest address 10.1.1.4Guest address 10.1.1.5
Guest 1 Virtual Router
Guest 2 VM 1
Guest 2 VM 2
Guest 2 VM 3
Guest Virtual Network 10.1.1.0/24
Gateway address 10.1.1.1
NATDHCPLoad BalancingVPN
Guest address 10.1.1.2Guest address 10.1.1.3Guest address 10.1.1.4
Guest 2 Virtual Router
Public IP address 65.37.141.2465.37.141.80
Layer-2 Guest Virtual Network
Public Network/Internet
Guest Virtual Network 10.1.1.1/8VLAN 100
Gateway address 10.1.1.1
DHCP, DNSNATLoad BalancingVPN
Public IP 65.37.141.11
10.1.1.1Guest VM 1
10.1.1.3Guest VM 2
10.1.1.4Guest VM 3
10.1.1.5Guest VM 4
CSVirtual Router
Public Network/Internet
Guest Virtual Network 10.1.1.1/8VLAN 100
Private IP10.1.1.112
DHCP, DNS
Public IP 65.37.141.112
10.1.1.1Guest VM 1
10.1.1.3Guest VM 2
10.1.1.4Guest VM 3
10.1.1.5Guest VM 4
NetScalerLoad
Blancer
Private IP10.1.1.111
Public IP 65.37.141.111
Juniper SRX
Firewall
CS Virtual Router provides Network Services External Devices provide Network Services
CSVirtual Router
Other Topologies
Guest Virtual Network 10.1.1.0/24VLAN 100
Gateway address 10.1.1.1
10.1.1.1Guest VM 1
10.1.1.3Guest VM 2
10.1.1.4Guest VM 3
10.1.1.5Guest VM 4
Guest Virtual Network 10.1.1.0/24VLAN 100
DHCP, DNSUser-data
10.1.1.1Guest VM 1
10.1.1.3Guest VM 2
10.1.1.4Guest VM 3
10.1.1.5Guest VM 4
No services [Static Ips] Dedicated VLAN with DHCP and DNSUser can request specific IP[s] for NIC
CSVirtual Router
Core switch
Gateway address 10.1.1.1
Core switch
Other topologies
Guest Virtual Network 10.1.1.0/24VLAN 100
Gateway address 10.1.1.1
10.1.1.100Guest VM 1
10.1.1.200Guest VM 2
10.1.1.101Guest VM 3
10.1.1.115
Guest VM 4
Guest Virtual Network 10.1.1.0/24VLAN 100
DHCP, DNSUser-data
10.1.1.1Guest VM 1
10.1.1.3Guest VM 2
10.1.1.4Guest VM 3
10.1.1.5Guest VM 4
MPLS Shared VLAN with DHCP and DNS
CSVirtual Router
Core switch
Gateway address 10.1.1.1
Core switch
MPLS VLAN 100
DHCP, DNSUser-data
CSVirtual Router
Multi-tier network
Public Network/Internet
Private IP10.1.1.112
DHCP, DNSUser-data
Public IP 65.37.141.112
10.1.1.1Web VM
1
10.1.1.3Web VM
2
10.1.1.4Web VM
3
10.1.1.5Web VM
4
NetscalerLoad
Balancer
Private IP10.1.1.111
Public IP 65.37.141.111
Juniper SRX
Firewall
Multi-tier network
CSVirtual Router
CSVirtual Router
Virtual Network 10.1.1.0/24VLAN 100
Virtual Network 10.1.2.0/24VLAN 1001
10.1.2.21
10.1.2.18
10.1.2.38
10.1.2.39
10.1.2.31App VM
1 10.1.3.21
Virtual Network 10.1.3.0/24VLAN 141
10.1.2.24App VM
2 10.1.3.45
10.1.3.24 DB VM 1
CSVirtual Router
DHCP, DNS, User-data
DHCP, DNSUser-data,Source-NAT, VPN
Public IP 65.37.141.115
VM VM VM
VR
Your Routing
VM
Public VLAN(s)
Guest VLAN
Monitoring VLAN(shared)
Bring-your-own Service
Customer installs static route to point to his routing vm
VM VM VM
VR
Your Routing
VM
Public VLAN(s)
Guest VLAN
Shared Public VLAN
Bring-your-own Service[site-to-site-vpn]
Customer installs static route (manually/automated config) to point to his routing vm. Routing VM provides Site-to-site VPN (configured directly on routing VM, not by CloudStack)
Multi-tier unified [vision]
10.1.1.1Web VM
1
10.1.1.3Web VM
2
10.1.1.4Web VM
3
10.1.1.5Web VM
4
Virtual Network 10.1.1.0/24VLAN 100
Virtual Network 10.1.2.0/24VLAN 1001
10.1.2.31App VM
1
Virtual Network 10.1.3.0/24VLAN 141
10.1.2.24App VM
2
10.1.3.24 DB VM 1
CSVirtual Router /
OtherCustomerPremises
IPSec or SSL site-to-site VPN
Internet
Monitoring VLAN
Virtual Router Services• IPAM• DNS• LB [intra]• S-2-S VPN• Static Routes• ACLs• NAT, PF• FW [ingress & egress]• BGP
Loadbalancer
Multi-tier unified with SDN[vision]
10.1.1.1Web VM
1
10.1.1.3Web VM
2
10.1.1.4Web VM
3
10.1.1.5Web VM
4
Overlay Network 10.1.1.0/24
Overlay Network 10.1.2.0/24
10.1.2.31App VM
1
Overlay Network 10.1.3.0/24
10.1.2.24App VM
2
10.1.3.24 DB VM 1
CSVirtual Router /
OtherCustomerPremises
IPSec or SSL site-to-site VPN
Internet
Monitoring VLAN
Virtual Router Services• IPAM• DNS• LB [intra]• S-2-S VPN• Static Routes• ACLs• NAT, PF• FW [ingress & egress]• BGP
LoadbalancerVirtual Appliance
• Cloud provider defines the feature set for guest networks
• Toggle features or service levels– Security groups on/off– Load balancer on/off– Load balancer software/hardware– VPN, firewall, port forwarding
• User chooses network offering when creating network
• Enables upgrade between network offerings
• Default offerings built-in– For classic CloudStack networking
Network Offerings
CPU Cores
CPU (MHz)
Memory (MB)
Name
Compute
Specify Resource Levels
Service Offerings
Custom Disk Size
Disk Size (GB)
Storage Tag
Public
Name
Disk
Network Rate
Redundant VR
Public
Name
Network
Firewall
Load balancer
CPU Cap
Host Tag
Enable HA
Configure Properties
Public
Define Scope
CloudStack Network Service Providers
• A Network Service Provider is hardware or virtual appliance that makes a network service possible in CloudStack ; for example, a Citrix NetScaler appliance can be installed in the cloud to provide Load-Balancing services.
• Administrators can have multiple instances of the same service provider in a network; for example, more than one Citrix NetScaler or Juniper SRX device can be added to CloudStack
• CloudStack supports the following Network Providers:
– CloudStack Virtual Router (default)– Citrix NetScaler SDX, VPX and MPX models– Juniper SRX– F5 BigIP
Adding an Additional Network Offerings
Network Offering Status
Network Offering Order control
Network Service Providers Matrix
Feature Virtual Router Citrix NetScaler
Juniper SRX F5 BigIP
Remote Access VPN YES N/A N/A N/AFirewall YES N/A YES N/ASource NAT YES N/A YES N/AStatic NAT YES YES YES N/ALoad Balancing YES YES N/A YESPort Forwarding YES N/A YES N/AElastic IP N/A YES N/A N/AElastic LB N/A YES N/A N/ADHCP/DNS/User Data YES N/A N/A N/A
• Network offerings is basically a definition of what Network Services are available when this offering is used. The available Network Services are: VPN, DHCP, DNS, Firewall, Load Balancer, User Data, Source NAT, Static NAT, Port Forwarding and Security Groups*
CloudStack User APIs [sample]
• Networks (L2)– createNetwork [requires network offering id], – deleteNetwork (A), listNetworks, – restartNetwork (A): restarts all devices (if allowed)
supporting the network and re-applies configuration
– updateNetwork: update network offering and restart network
Adding a Shared Guest Network
• Only Administrators can add a Shared Guest Network for an Advanced zone
Adding a Shared Guest Network
VLAN required!
Editing Guest Networks
When editing a guest network users can change the network offering. They can either upgrade to a “premium” network offering (for example offering that uses hardware Load-balancer) or downgrade to a “cheaper” network.
• Restarting the network will simply resend all the LB, Firewall and Port-Forwarding rules to the network provider
• Restarting the Network with “Clean up”:
• restarting network elements - virtual routers, DHCP servers
• If virtual router is used, it will be destroyed and recreated
• Reapplying all public IPs to the network provider
• Reapplying load-Balancing/Port-Forwarding/Firewall rules
Restarting and Cleaning Up a Guest Network
• An Isolated Guest Network can only be deleted if no VMs are using these network (e.g. Completely destroyed and expunged)
• Deleting a Network will Destroy the Virtual Router (if used) and will release the Public IPs back to the IP Pool
Deleting a Guest Network
Extending CloudStack Networking
Network Manager
Network Element
DnsService
MyDnsElementMyDnsDeviceMa
nager
MyDnsDeviceService
PluggableService
MyDnsDeviceResource
AgentManager Queue
1. prepare (part of start vm)2. prepare (Network, Nic, DeployDestination, VmInfo)
3. addDnsRecord(ip, fqdn)
4.Enqueue AddDnsRecord
5.API call to Dns Device
Device ConfigurationAdmin API (CRUD)
MySQL
Needs to be added as of 5/2/2012
Demonstrates one way to inform an external DNS server when an instance starts.Classes shaded blue form a plugin / service bundle to integrate an external DNS server. Clients of the instance can then use DNS names to access the instance.
CloudStack Virtual Router (Virtual Router)
• The Virtual Router will be deployed once (when the first instance is deployed in a Zone) when a Shared Network is used providing DHCP and DNS services for the Zone’s Instances (IPs will be allocated from the Public IP Range entered in CloudStack)
• When Advanced is used the Router will be deployed Per-Account (and Per Unique Isolated Guest Network)
• Virtual Router can serve and isolate VMs even if deployed on a different Hypervisor
CloudStack Virtual Router
• The Virtual Router will have 3 NICs:– Eth0 will be connected to the Isolated Guest Network (for Advanced VLAN). It will have the first IP in
the CIDR (for example10.1.1.1) and it will be the DNS, DHCP and Gateway for the Instances in the Private Guest Network.
– Eth1 resides on local-link network (only for KVM and XenServer) or the Management Network (on VMware) and is used by CloudStack to configure the virtual router. On VMware it will use an IPs from the Management Network IP Range (e.g. Pod Private Range)
– Eth2 resides on the Public Network and assigned with a Public IP from the range entered in CloudStack (users can ‘Acquire New IPs’ if needed)
• In the default Isolated Mode - Source NAT is automatically configured on the virtual router to forward outbound traffic for all guest VMs and block all incoming traffic (users can manage incoming rules from UI)
Virtual Router Information (applies to all Sys. VMs)
• Debian 6.0 ("Squeeze"), 2.6.32 kernel with the latest security patches from the Debian security APT repository. No extraneous accounts
• 32-bit for enhanced performance on Xen/VMWare• Only essential software packages are installed. Services such as, printing, ftp, telnet, X, kudzu,
dns, sendmail are not installed.• SSHd only listens on the private/link-local interface. SSH port has been changed to a non-
standard port. SSH logins only using keys (keys are generated at install time and are unique for every customer)
• pvops kernel with Xen paravirt drivers + KVM virtio drivers + VMware tools for optimum performance on all hypervisors. Xen tools inclusion allows performance monitoring
• Template is built from scratch and is not polluted with any old logs or history• Latest versions of haproxy, iptables, ipsec, apache from debian repository ensures improved
security and speed• Latest version of jre from Oracle ensures improved security and speed