Docker 1.12 networking deep dive
-
Upload
madhu-venugopal -
Category
Technology
-
view
1.004 -
download
9
Transcript of Docker 1.12 networking deep dive
![Page 1: Docker 1.12 networking deep dive](https://reader034.fdocuments.us/reader034/viewer/2022052300/586fdcf01a28ab18428b66af/html5/thumbnails/1.jpg)
Docker Networking Deep Dive
@MadhuVenugopal
online meetup 08/24/2016
![Page 2: Docker 1.12 networking deep dive](https://reader034.fdocuments.us/reader034/viewer/2022052300/586fdcf01a28ab18428b66af/html5/thumbnails/2.jpg)
• What is libnetwork • CNM • 1.12 Features
• Multihost networking • Secured Control plane & Data plane • Service Discovery • Native Loadbalacing • Routing Mesh
• Demo
Agenda
![Page 3: Docker 1.12 networking deep dive](https://reader034.fdocuments.us/reader034/viewer/2022052300/586fdcf01a28ab18428b66af/html5/thumbnails/3.jpg)
Overview
![Page 4: Docker 1.12 networking deep dive](https://reader034.fdocuments.us/reader034/viewer/2022052300/586fdcf01a28ab18428b66af/html5/thumbnails/4.jpg)
It is not just a driver interface
• Docker networking fabric • Defines Container Networking Model • Provides builtin IP address management • Provides native multi-host networking • Provides native Service Discovery and Load Balancing • Allows for extensions by the ecosystem via plugins
What is libnetwork?
![Page 5: Docker 1.12 networking deep dive](https://reader034.fdocuments.us/reader034/viewer/2022052300/586fdcf01a28ab18428b66af/html5/thumbnails/5.jpg)
Design Philosophy
• Users First: • Application Developers • IT/Network Ops
• Plugin API Design • Batteries Included but Swappable
![Page 6: Docker 1.12 networking deep dive](https://reader034.fdocuments.us/reader034/viewer/2022052300/586fdcf01a28ab18428b66af/html5/thumbnails/6.jpg)
Docker Networking
1.7 1.8 1.9 1.10 1.11
- Libnetwork - CNM - Migrated Bridge, host,
none drivers to CNM
- Multihost Networking - Network Plugins - IPAM Plugins - Network UX/API
Service Discovery (using /etc/hosts)
Distributed DNS
- Aliases - DNS Round Robin LB
1.12
- Load Balancing - Encrypted Control and
data plane - Routing Mesh - Built-in Swarm-mode
networking
![Page 7: Docker 1.12 networking deep dive](https://reader034.fdocuments.us/reader034/viewer/2022052300/586fdcf01a28ab18428b66af/html5/thumbnails/7.jpg)
Container Networking Model
• Endpoint • Network • Sandbox • Drivers & Plugins
https://github.com/docker/libnetwork/blob/master/docs/design.md
![Page 8: Docker 1.12 networking deep dive](https://reader034.fdocuments.us/reader034/viewer/2022052300/586fdcf01a28ab18428b66af/html5/thumbnails/8.jpg)
Network driver overview
![Page 9: Docker 1.12 networking deep dive](https://reader034.fdocuments.us/reader034/viewer/2022052300/586fdcf01a28ab18428b66af/html5/thumbnails/9.jpg)
Use-case1 Default Bridge Network
(docker0)
![Page 10: Docker 1.12 networking deep dive](https://reader034.fdocuments.us/reader034/viewer/2022052300/586fdcf01a28ab18428b66af/html5/thumbnails/10.jpg)
eth0 eth0 eth0
docker0 docker0 docker0
C1eth0 eth0
C2eth0C3 C1
eth0 eth0C2
eth0C3 C1
eth0 eth0C2
eth0C3
ToR switch / Hypervisor switch / …
iptables : NAT / port-mapping
iptables : NAT / port-mapping
iptables : NAT / port-mapping
![Page 11: Docker 1.12 networking deep dive](https://reader034.fdocuments.us/reader034/viewer/2022052300/586fdcf01a28ab18428b66af/html5/thumbnails/11.jpg)
Use-case2
User-Defined Bridge Network
![Page 12: Docker 1.12 networking deep dive](https://reader034.fdocuments.us/reader034/viewer/2022052300/586fdcf01a28ab18428b66af/html5/thumbnails/12.jpg)
Host1 : $ docker network create -d bridge -o com.docker.network.bridge.name=brnet brnet $ docker run --net=brnet -it busybox ifconfig
eth0
brnet 172.18.0.1
ToR switch / Hypervisor switch / …
eth0C1
Host1
eth0C2
eth0C3
iptables : NAT / port-mapping
eth0
brnet 172.18.0.1
eth0C4
Host2
eth0C5
eth0C6
iptables : NAT / port-mapping
eth0
brnet 172.18.0.1
eth0C7
Host3
eth0C8
eth0C9
iptables : NAT / port-mapping
Host2 : $ docker network create -d bridge -o com.docker.network.bridge.name=brnet brnet $ docker run --net=brnet -it busybox ifconfig
Host3 : $ docker network create -d bridge -o com.docker.network.bridge.name=brnet brnet $ docker run --net=brnet -it busybox ifconfig
![Page 13: Docker 1.12 networking deep dive](https://reader034.fdocuments.us/reader034/viewer/2022052300/586fdcf01a28ab18428b66af/html5/thumbnails/13.jpg)
Use-case3
Bridge Network plumbed to underlay with built-in IPAM(no NAT / Port-mapping)
![Page 14: Docker 1.12 networking deep dive](https://reader034.fdocuments.us/reader034/viewer/2022052300/586fdcf01a28ab18428b66af/html5/thumbnails/14.jpg)
Host1 : $ docker network create -d bridge --subnet=192.168.57.0/24 --ip-range=192.168.57.32/28 --gateway=192.168.57.11 --aux-address DefaultGatewayIPv4=192.168.57.1 -o com.docker.network.bridge.name=brnet brnet $ brctl addif brnet eth2$ docker run --net=brnet -it busybox ifconfig
Host2 : $ docker network create -d bridge --subnet=192.168.57.0/24 --ip-range=192.168.57.64/28 --gateway=192.168.57.12 --aux-address DefaultGatewayIPv4=192.168.57.1 -o com.docker.network.bridge.name=brnet brnet $ brctl addif brnet eth2$ docker run --net=brnet -it busybox ifconfig
Host3 : $ docker network create -d bridge --subnet=192.168.57.0/24 --ip-range=192.168.57.128/28 --gateway=192.168.57.13 --aux-address DefaultGatewayIPv4=192.168.57.1 -o com.docker.network.bridge.name=brnet brnet $ brctl addif brnet eth2$ docker run --net=brnet -it busybox ifconfig
eth2192.168.57.11
brnet 192.168.57.11
ToR switch / Hypervisor switch / Virtual-box host-only / … (Gateway : 192.168.57.1)
eth0C1
Host1
eth0C2
eth0C3
eth2192.168.57.12
brnet 192.168.57.12
eth0C4
eth0C5
eth0C6
eth2192.168.57.13
brnet 192.168.57.13
eth0C7
eth0C8
eth0C9
Host2 Host3
![Page 15: Docker 1.12 networking deep dive](https://reader034.fdocuments.us/reader034/viewer/2022052300/586fdcf01a28ab18428b66af/html5/thumbnails/15.jpg)
Use-case4
Docker Overlay Network
![Page 16: Docker 1.12 networking deep dive](https://reader034.fdocuments.us/reader034/viewer/2022052300/586fdcf01a28ab18428b66af/html5/thumbnails/16.jpg)
eth0
C1eth1 eth1 eth1
ToR switch / Hypervisor switch / …
docker0docker_gwbridge
eth0
eth1 eth1 eth1
docker0docker_gwbridge
eth0
eth1 eth1 eth1
docker0docker_gwbridge
ov-net1 ov-net1 ov-net1VXLAN-VNI 100 VXLAN-VNI 100
eth0 eth0 eth0 eth0 eth0 eth0 eth0 eth0 eth0
VXLAN-VNI 100
iptables : NAT / port-mapping
iptables : NAT / port-mapping
iptables : NAT / port-mapping
Docker overlay networking
C2 C3 C4 C5 C6 C7 C8 C9
![Page 17: Docker 1.12 networking deep dive](https://reader034.fdocuments.us/reader034/viewer/2022052300/586fdcf01a28ab18428b66af/html5/thumbnails/17.jpg)
Use-case5
Plumbed to underlay vlan with built-in IPAM
macvlan driver (& experimental ipvlan) https://github.com/docker/docker/blob/master/experimental/vlan-networks.md
![Page 18: Docker 1.12 networking deep dive](https://reader034.fdocuments.us/reader034/viewer/2022052300/586fdcf01a28ab18428b66af/html5/thumbnails/18.jpg)
# vlan 10 (eth0.10)$ docker network create -d macvlan —subnet=10.1.10.0/24 —gateway=10.1.10.1 -o parent=eth0.10 mcvlan10
$ docker run --net=mcvlan10 -it --rm alpine /bin/sh
# vlan 20 (eth0.20)$ docker network create -d macvlan —subnet=10.1.20.0/24 —gateway=10.1.20.1 -o parent=eth0.20 mcvlan20
$ docker run --net=mcvlan20 -it --rm alpine /bin/sh
# vlan 30 (eth0.30)$ docker network create -d macvlan —subnet=10.1.30.0/24 —gateway=10.1.30.1 -o parent=eth0.30 mcvlan30
$ docker run --net=mcvlan30 -it --rm alpine /bin/sh
![Page 19: Docker 1.12 networking deep dive](https://reader034.fdocuments.us/reader034/viewer/2022052300/586fdcf01a28ab18428b66af/html5/thumbnails/19.jpg)
Docker 1.12 Networking
![Page 20: Docker 1.12 networking deep dive](https://reader034.fdocuments.us/reader034/viewer/2022052300/586fdcf01a28ab18428b66af/html5/thumbnails/20.jpg)
New features in 1.12 swarm mode
CNM
Routing Mesh
Multi-host Networking without external k/v store
Service Discovery
Secure Data-Plane
Secure Control-Plane
Load Balancing
• Cluster aware • De-centralized control
plane • Highly scalable
![Page 21: Docker 1.12 networking deep dive](https://reader034.fdocuments.us/reader034/viewer/2022052300/586fdcf01a28ab18428b66af/html5/thumbnails/21.jpg)
Swarm-mode Multi-host networkingManager
Network CreateOrchestrator
Allocator
Scheduler
Dispatcher
Service Create
Task Create
Task Dispatch
Task Dispatch
Gossip
Worker1 Worker2
Engine
Libnetwork
Engine
Libnetwork
• VXLAN based data path • No external key-value store • Central resource allocation • Improved performance • Highly scalable
![Page 22: Docker 1.12 networking deep dive](https://reader034.fdocuments.us/reader034/viewer/2022052300/586fdcf01a28ab18428b66af/html5/thumbnails/22.jpg)
• Gossip based protocol • Network scoped • Fast convergence • Secure by default
• periodic key rotations • swarm native key-exchange
• Gossips control messages • Routing-states • Service-discovery • Plugin-data
• Highly scalable
Secured network control planeCluster Scope Gossip
W1W2
W3
W1W5
W4
Network Scope Gossip
Network Scope Gossip
![Page 23: Docker 1.12 networking deep dive](https://reader034.fdocuments.us/reader034/viewer/2022052300/586fdcf01a28ab18428b66af/html5/thumbnails/23.jpg)
• Available as an option during overlay network creation
• Uses kernel IPSec modules • On-demand tunnel setup • Swarm native key-exchange • Periodic key rotations
Secure dataplaneWorker1
Worker2
Worker3
secure network
secure network
IPSec Tunnel
IPSec Tunnel
IPSec Tunnel
secure network
secure network
non-secure network
non-secure network
Open UDP traffic
![Page 24: Docker 1.12 networking deep dive](https://reader034.fdocuments.us/reader034/viewer/2022052300/586fdcf01a28ab18428b66af/html5/thumbnails/24.jpg)
• Provided by embedded DNS • Highly available • Uses Network Control Plane to learn state • Can be used to discover both tasks and
services
Service Discovery
engine
DNS Server
DNS Resolver DNS Resolver
DNS requests
![Page 25: Docker 1.12 networking deep dive](https://reader034.fdocuments.us/reader034/viewer/2022052300/586fdcf01a28ab18428b66af/html5/thumbnails/25.jpg)
• Internal & Ingress load-balancing • Supports VIP & DNS-RR • Highly available • Uses Network Control Plane to learn state • Minimal Overhead
Load balancerTask1
ServiceA
Task2 ServiceA
Task3 ServiceA
Client1 Client2
VIP LB VIP LB
![Page 26: Docker 1.12 networking deep dive](https://reader034.fdocuments.us/reader034/viewer/2022052300/586fdcf01a28ab18428b66af/html5/thumbnails/26.jpg)
• Builtin routing mesh for edge routing • Worker nodes themselves participate in
ingress routing mesh • All worker nodes accept connection
requests on PublishedPort • Port translation happens at the worker
node • Same internal load balancing mechanism
used to load balance external requests
Routing mesh External Loadbalancer
(optional)
Task1 ServiceA Task1
ServiceA
Task1 ServiceA
Worker1 Worker2
Ingress Network
8080 8080
VIP LB VIP LB
8080->80 8080->80
![Page 27: Docker 1.12 networking deep dive](https://reader034.fdocuments.us/reader034/viewer/2022052300/586fdcf01a28ab18428b66af/html5/thumbnails/27.jpg)
Routing Mesh• Operator reserves a swarm-
wide ingress port (8080) for myapp
• Every node listens on 8080 • Container-aware routing mesh
can transparently reroute traffic from Worker3 to a node that is running container
• Built in load balancing into the Engine
• DNS-based service discovery
Worker 1:8080
Manager
User accesses myapp.com:8080:8080
Worker 2:8080
Worker 3:8080
frontend frontend
$ docker service create --replicas 3 --name frontend --network mynet
--publish 8080:80/tcp frontend_image:latest
frontend
![Page 28: Docker 1.12 networking deep dive](https://reader034.fdocuments.us/reader034/viewer/2022052300/586fdcf01a28ab18428b66af/html5/thumbnails/28.jpg)
Routing Mesh: Published Ports• Operator reserves a swarm-
wide ingress port (8080) for myapp
• Every node listens on 8080 • Container-aware routing mesh
can transparently reroute traffic from Worker3 to a node that is running container
• Built in load balancing into the Engine
• DNS-based service discovery
Worker 1:8080
Manager
User accesses myapp.com:8080:8080
Worker 2:8080
Worker 3:8080
frontend frontend
$ docker service create --replicas 3 --name frontend --network mynet
--publish 8080:80/tcp frontend_image:latest
frontend
![Page 29: Docker 1.12 networking deep dive](https://reader034.fdocuments.us/reader034/viewer/2022052300/586fdcf01a28ab18428b66af/html5/thumbnails/29.jpg)
Deep Dive
![Page 30: Docker 1.12 networking deep dive](https://reader034.fdocuments.us/reader034/viewer/2022052300/586fdcf01a28ab18428b66af/html5/thumbnails/30.jpg)
Service , Port-Publish & Network
iptables
eth0 Host1
default_gwbridge
ingress-sboxeth1
ingress-overlay-bridge
Ingress- Network
eth0
vxlan tunnel to host2 - vni-100vxlan tunnel to host3 - vni-100
eth0
Container-sboxeth1
eth2
mynet
mynet-br vxlan tunnel to host2 - vni-101
docker service create —name=test —network=mynet -p 8080:80 —replicas=2 xxx
iptables
ipvs
iptables
ipvs
Host1: 8080
DNS Resolver
daemon embedded DNS server
service -> VIP
![Page 31: Docker 1.12 networking deep dive](https://reader034.fdocuments.us/reader034/viewer/2022052300/586fdcf01a28ab18428b66af/html5/thumbnails/31.jpg)
Day in life of a packet - IPTables & IPVS
![Page 32: Docker 1.12 networking deep dive](https://reader034.fdocuments.us/reader034/viewer/2022052300/586fdcf01a28ab18428b66af/html5/thumbnails/32.jpg)
Day in life of a packet - Routing Mesh & Ingress LB
iptables NAT table DOCKER-INGRESS
DNAT : Published-Port -> ingress-sbox
eth0 Host1
default_gwbridge
ingress-sboxeth1
iptables NAT table PREROUTING
Redirect -> service-port
iptables MANGLE table PREROUTING
MARK : Published-Port -> <fw-mark-id>
IPVS Match <fw-mark-id> -> Masq
{RR across container-IPs)
ingress-overlay-bridge
Ingress- Network
eth0
iptables NAT table DOCKER-INGRESS
DNAT : Published-Port -> ingress-sbox
eth0 Host2
default_gwbridge
ingress-sbox
eth1
ingress-overlay-bridge
eth0
vxlan tunnel with vni
Ingress- Network
eth0
Container-sbox (backs a task/
service)
eth1
![Page 33: Docker 1.12 networking deep dive](https://reader034.fdocuments.us/reader034/viewer/2022052300/586fdcf01a28ab18428b66af/html5/thumbnails/33.jpg)
Day in life of a packet - Internal LB
eth0 Host1
container-sbox(service1)
eth1
iptables MANGLE table OUTPUT
MARK : VIP -> <fw-mark-id>
IPVS Match <fw-mark-id> -> Masq
{RR across container-IPs)
mynet-overlay-bridge
mynet
eth2
Host2
mynet-overlay-bridgevxlan tunnel with vnimynet
eth2
Container-sbox (service2)
Application looks up service2 (using embedded-DNS @ 127.0.0.11)
DNS Resolver
daemon embedded DNS server service2 -> VIP2
vxlan tunnel with vni
![Page 34: Docker 1.12 networking deep dive](https://reader034.fdocuments.us/reader034/viewer/2022052300/586fdcf01a28ab18428b66af/html5/thumbnails/34.jpg)
Thank you!