Triangle K8S Meetup 06092016 - Nuage Network
Transcript of Triangle K8S Meetup 06092016 - Nuage Network
Copyright 2016 Nokia. All rights reserved.
Copyright 2013 Alcatel-Lucent. All rights reserved. CONFIDENTIAL - SOLELY FOR AUTHORIZED PERSONS HAVING A NEED TO KNOW
PROPRIETARY – USE PURSUANT TO COMPANY INSTRUCTION Nuage Networks
Overlay Networking for Openshift and Kubernetes with Nuage VCS Vishal Patil, Aniket Bhat
Copyright 2016 Nokia. All rights reserved.
Agenda
• Kubernetes (K8s) Overview
• Nuage VCS Integration with OpenShift/K8s
• Demo
Copyright 2016 Nokia. All rights reserved.
Kubernetes Architecture
Master
SCHEDULER
API PROXY
AUTH
REPLICATION CONTROLLER
Node
KUBELET KUBE PROXY
POD (SVC 2)
POD (SVC 2)
Node
KUBELET KUBE PROXY
POD (SVC 1)
POD (SVC 1)
CLIENT
C1 C1
C1 C2 C1 C2
ETCD
Copyright 2016 Nokia. All rights reserved.
▪ etcd ▪ key value store that stores the persistent master state
▪ API Server ▪ Authenticated interface between users and Kubernetes
▪ Scheduler
▪ Schedules pods to nodes/minions
▪ Controller Manager
▪ Monitors replication controllers and adds/removes pods to maintain desired number of pods in a service
Master
Kubernetes Components
Master
SCHEDULER
API PROXY
AUTH
REPLICATION CONTROLLER
CLIENT
ETCD
Copyright 2016 Nokia. All rights reserved.
▪ Docker
▪ Engine for containers
▪ Kubelet
▪ Manages Pods on the nodes
▪ Kube-Proxy
▪ Simple network proxy to implement ‘Services’
▪ Two modes supported - iptables based or user-space proxying
▪ TCP and UDP stream forwarding (round robin) across a set of backends
Node
Kubernetes Components
Node
KUBELET KUBE PROXY
POD (SVC 2)
POD (SVC 2)
C1 C2 C1 C2
Copyright 2016 Nokia. All rights reserved.
Kubernetes Networking
▪ Pod gets an IP from a flat shared networking namespace. ▪ Containers within the pod share the same IP. ▪ Container in a pod use localhost networking. ▪ Pods should typically use service IPs and service ports to communicate with
other pods ▪ Kubernetes is non-prescriptive about the network design per se. ▪ Several vendor choices:
● Calico ● Flannel ● Nuage ● Open-Contrail ● Weave
Copyright 2016 Nokia. All rights reserved.
JBoss xPaaS
VCS Integration with Kubernetes/Openshift
User Experience
Containerized Services
Container Orchestration
Container API
Container Host
Integration layer
Nuage VCS
Copyright 2016 Nokia. All rights reserved.
Cloud Service Management Plane
Data Center Control Plane
Data Center Data Plane
Virtual Routing & Switching
Virtualized Services Directory
Virtualized Services Controller
HYPERVISOR
HYPERVISOR
HYPERVISOR
HYPERVISOR
HYPERVISOR
HYPERVISOR
Virtualized Services Directory (VSD) • Network Policy Engine – abstracts complexity • Service templates and analytics
Virtualized Services Controller (VSC) • SDN Controller, programs the network • Rich routing feature set
Virtual Routing & Switching (VRS) • Distributed switch / router – L2-4 rules • Integration of bare metal assets
Nuage Networks Virtualized Cloud Services Platform (VCS)
IP Fabric
Bare Metal Servers
Nuage Networks Virtualized Cloud Services
MP-BGP
NSG
Copyright 2016 Nokia. All rights reserved.
Enterprise
Domain
Zone
Subnet
VPort
Copyright 2016 Nokia. All rights reserved.
VCS OBJECTS AND OPENSHIFT/KUBERNETES CONCEPTS
Project/Namespace
nuage.io/policy-group Labels
Pods
Zone
Policy Groups
VPorts
Cluster Domain
VCS
Copyright 2016 Nokia. All rights reserved.
Overlay-based Virtual Networks OPENSHIFT DEPLOYMENT WITH VCS
OpenShift Master Openshift Node Openshift Node
VSD
K A
PI
XMPP
VRS VRS nuage-openshift-monitor
vsp-openshift plugin
vsp-openshift plugin
OpenShift Cluster
Virtualized Services
Controller
Virtualized Services Directory
Copyright 2016 Nokia. All rights reserved.
• Runs on master node(s)
• Exercises the VSD REST API to ensure that the VSD objects are created
• Create delete vsd zones Projects
• Create/delete network macros Services
• Dynamically scale-up or scale-down subnets
nuage-openshift-monitor
• Runs on each of the nodes
• Implementation of the k8s network exec plugin
• Gets invoked when a node is initialized as well as during pod lifecycle events:
• Create/delete pod
• Status hook that queries pod’s IP information
vsp-openshift plugin
Copyright 2016 Nokia. All rights reserved.
▪ Pod is created with a veth pair – is eventually mapped to a vPort
▪ Pod gets an IP allocated from the subnet pools for that Openshift Project/K8S Namespace (VCS Zone).
▪ Pods in a given zone belong to one or more subnets irrespective of which node they are spawned on
▪ Labels are optionally used to do the Security and QoS Policy resolution with the VSD
Pods with VCS
OPENSHIFT DEPLOYMENT WITH VCS
Copyright 2016 Nokia. All rights reserved.
Dev Workflow OPENSHIFT DEPLOYMENT WITH VCS
OpenShift Master Node
VSD
K A
PI
XMPP
VRS nuage-openshift-monitor
vsp-openshift plugin
OpenShift Cluster
Virtualized Services
Controller
Virtualized Services Directory
1. Project creation triggers creation of VSD Zone and subnet
2. Node Plugin invoked during Pod creation
4. VSC gets network and security policy from VSD
5. VSC sends network and security policy to the VRS
3. VRS contacts VSC with Project name and metadata information
Copyright 2016 Nokia. All rights reserved.
Operations Workflow OPENSHIFT DEPLOYMENT WITH VCS
OpenShift Master Node
VSD
K A
PI
XMPP
VRS nuage-openshift-monitor
vsp-openshift plugin
OpenShift Cluster
Virtualized Services
Controller
Virtualized Services Directory
1. User creates Domain/Zone and defines Network and Security Policies on VSD
2. Labels in a Pod configuration are used to pass metadata to VSD
3. Node Plugin invoked during Pod creation will fetch Labels from Pod configuration
5. VSC gets network and security policy from VSD
6. VSC sends network and security policy to the VRS
4. VRS contacts VSC with Project name and metadata information
Copyright 2016 Nokia. All rights reserved.
Rich policy model
Use of labels as metadata
Dynamic scaling of subnets in
a given project.
Support for both
operations and
developer workflows.
Support for double
overlays
Support for asymmetric
overlays
HIGHLIGHTS
Copyright 2016 Nokia. All rights reserved.
DEMO
▪ Scenario 1 – Developer Workflow ▪ Communication between an app with Front end (Mail service)
and a pod acting as a mail client
▪ Scenario 2 - Policies ▪ Inter-project communication
Copyright 2016 Nokia. All rights reserved.
Copyright 2016 Nokia. All rights reserved.
THANK YOU