Micro Load Balancing for Low-latency Data Center...

DRILLMicro Load Balancing for

Low-latency Data Center Networks

Soudeh GhorbaniZibin YangBrighten GodfreyYashar GanjaliAmin Firoozshahian

Micro Load Balancing forLow-latency Data Center Networks

Data center topologies provide high capacity

BUT WE ARE STILL NOT USING THE CAPACITY EFFICIENTLY!

Networks experience high congestion drops as utilizationapproaches 25%[1].

Further improving fabric congestion response remains an ongoing effort[1].

A Decade of Clos Topologies and Centralized Control in Google’s Datacenter Network,Google, SIGCOMM 2015.

[1] Jupiter Rising :

THE GAP?

Facebook data center fabric : https://code.facebook.com/posts/360346274145943

High bandwidth provided via massive Multipathing.

Facebook data center fabric : https://code.facebook.com/posts/360346274145943

High bandwidth provided via massive Multipathing.

Congestion happens even when there is spare capacity to mitigate it elsewhere[2].

[2] Network Traffic Characteristics of Data Centers in the Wild, T. Benson et al., IMC 2010.

THE GAP?

OPTIMAL LOAD BALANCER?

A STARTING POINT :

“EQUAL SPLIT FLUID” (ESF)

Equally split all incoming flow to other leaves along all shortest outgoing paths.

A STARTING POINT :


Each of n spines receives1/n of all interleaf traffic.

A STARTING POINT :


Therefore, any two paths between the same source and destination experience the same utilization (and

mix of traffic) at all corresponding hops.

ESF is optimal for all traffic demands.

APPROXIMATING ESF

Packet

Flowcell /Flowlet

Flow

O(µs)

ECMP

PrestoCONGA

Plank

Hedera

Control loop latencyLoad aware

O(10ms) O(100ms) O(1s)O(1ms)

Load Oblivious

ElephantFlow

Granularity

APPROXIMATING ESF

Packet

Flowcell /Flowlet

Flow

O(µs)

ECMP

PrestoCONGA

Plank

Hedera


O(10ms) O(100ms) O(1s)O(1ms)

Load Oblivious

ElephantFlow

Granularity

Mic

ro b

urst

s

Packet

Flowcell /Flowlet

Flow

O(µs)

ECMP

PrestoCONGA

Plank

Hedera


O(10ms) O(100ms) O(1s)O(1ms)

Load Oblivious

Granularity

Mic

ro b

urst

s

MICRO LOAD BALANCINGElephant

Flow

• Per-packet decisions• Microsecond timescales• Microscopic, i.e., local to each

switch

SIMPLIFIED MICRO LOAD BALANCING

(1) Discover topology.(2) Compute multiple shortest paths.(3) Install into FIB.

Switch control plane :

Switch data plane:(1) Look up the candidate ports.(2) Check the queue occupancy of

the candidate ports.(3) Enqueue the packet in the least

loaded candidate port.

~ECMP {

BA

dst port

1.2.3.0/24

… …

1, 2

Forwarding engine

Dst: 1.2.3.4

Efficient micro load balancing implementation inside a switch

Reordering caused by per-packet decisions

Poor decisions in asymmetric topologies

CHALLENGES

MICRO LOAD BALANCING INSIDE A SWITCH

• is hard, especially for high radix switches.

• can cause synchronization effect in switches with distributed architecture.

Checking all queues at line rate for every packet:

fwd engine

fwd engine

fwd engine

fwd engine

MICRO LOAD BALANCING WITH DRILL

• Discover topology.• Compute multiple shortest paths.• Install into FIB.


• Look up the candidate ports.• Sample queue lengths of 2 random

queues plus the least loaded queue from the previous packet.

• Send the packet to the least loaded one among those three.

Switch data planefwd engine

fwd engine

fwd engine

fwd engine

MICRO LOAD BALANCING WITH DRILL

• Discover topology.• Compute multiple shortest paths.• Install into FIB.


• Look up the candidate ports.• Sample queue lengths of 2 random

queues plus the least loaded queue from the previous packet.

• Send the packet to the least loaded one among those three.

Switch data planefwd engine

fwd engine

fwd engine

fwd engine

Verilog switch implementation shows that DRILL imposes less than 1% switch area overhead.

THE POWER OFTWO CHOICES

• n bins and n balls• Each ball choosing a bin

independently and uniformly at random

• Max load:

Y. Azar, A. Z. Broder, A. R. Karlin, and E. Upfal. Balanced allocations.SIAM Journal on Computing, 1994

𝜃(𝑙𝑜𝑔𝑛

𝑙𝑜𝑔𝑙𝑜𝑔𝑛)


• n bins and n balls• Balls placed sequentially, • in the least loaded of d≥2

random bins

• Max load:


𝜃(𝑙𝑜𝑔𝑙𝑜𝑔𝑛

𝑙𝑜𝑔𝑑)


• n bins and n balls• Balls placed sequentially, • in the least loaded of d≥2

random bins

• Max load:


Optimal

d=1

d=2

𝜃(𝑙𝑜𝑔𝑙𝑜𝑔𝑛

𝑙𝑜𝑔𝑑)

WHAT WE WANT

Switch has queues, instead of bins, from which packets leave.

Each forwarding engine chooses a queue independently, no coordination.

WHAT WE WANT

fwdengine

fwdengine

fwdengine

fwdengine

Switch has queues, instead of bins, from which packets leave.

Each forwarding engine chooses a queue independently, no coordination.

THE PITFALLS OF CHOICE

Topology: Clos topology with 8 spines, 10 leafs, each connected to 48 hosts. Links: 10Gbps.Switches have 6 forwarding engines. Network is under 75% load.

Trace: Inside the social network’s (datacenter) network, Facebook, SIGCOMM 2015.

d (number of sampled queues)

Queu

e lengths [STD

V]





Queu

e lengths [STD

V]


Que

ue le

ngth

s [S

TDV]





Queu

e lengths [STD

V]


Que

ue le

ngth

s [S

TDV]

Balls and bins





Queu

e lengths [STD

V]


Que

ue le

ngth

s [S

TDV]

Balls and bins

DRILL is provably stable and delivers 100% throughput.




CHALLENGES

THOU SHALT NOT SPLIT FLOWS!

Splitting flows Reordering Throughput degradation

DRILL SPLITS FLOWS ALONG MANY PATHS

Splitting flows Throughput degradation

Reordering

Splitting flows Throughput degradation

Reordering


YET CAUSES LITTLEREORDERING!

Splitting flows Latency variance Throughput degradation

Reordering


YET CAUSES LITTLEREORDERING!

Splitting flows Latency variance Throughput degradationReordering


BUT HAS LOW QUEUEINGDELAY VARIANCE

AND THEREFORE CAUSES LITTLE REORDERING!

Insight: Queueing delay variance is so small that it doesn’t matter what path the packet takes.

ASYMMETRYWHAT CAN GO WRONG?

?

1. Different queueing results in reordering.

2. TCP flows split along multiple paths will respond to congestion on the worst path, causing bandwidth inefficiency.

KEY PROBLEMS

dst

ASYMMETRYWHAT CAN GO WRONG?

?

Flow splitting across asymmetric paths.

ROOT CAUSE:

KEY IDEA: Decompose the network into symmetric components and micro-LB inside components.

dst

ASYMMETRY: GRAPH DECOMPOSITIONIntuition: Two paths are symmetric if they have equal number of hops and the ith queue carries the same flows on all paths for all i.

dst

MICRO LOAD BALANCING IN ASYMMETRIC TOPOLOGIES

• Discover topology.• Compute multiple shortest paths.• Decompose the network into symmetric

components.

SWITCH CONTROL PLANE :

• Hash flows to component.• Micro load balance inside a component.

Switch data plane:

Forwarding engine

dst

Hdst

EXPERIMENTAL EVALUATION

Environment Topology Workload

• OMNET++ simulator

• Ported Linux 2.6 TCPimplementation

• 2 and 3stage Clos

• Asymmetric topologies

• Varying failures

• Real measurements [1]

• Synthetic

• Incast patterns

[1] Inside the social network’s (datacenter) network, Facebook, SIGCOMM 2015.

DRILL REDUCES FCT ESPECIALLY UNDER LOADClos with 16 spines, 16 leafs, each connected to 20 hosts, links: 10Gbps

ECMPCONGAPresto

DRILL

ECMPCONGAPresto

DRILL

30% load 80% load

DRILL ALLOWS HIGHER UTILIZATION WITH EQUAL LATENCYClos with 16 spines, 16 leafs, each connected to 20 hosts, links: 10Gbps

1.52x 1.44x1.28x

DRILL HANDLES ASYMMETRYClos with 4 spines and 16 leafs, each connected to 20 hosts, edge links: 10Gbps, core links: 40Gbps. 5 randomly selected leaf-spine links fail.

ECMPCONGA

Presto

DRILL

DRILL CUTS THE TAIL LATENCY IN INCASTClos with 4 spines and 16 leafs, each connected to 20 hosts, edge links: 10Gbps, core links: 40Gbps. Network is under 20% load. 10% of hosts send simultaneous requests for 10KB flows to 10% of the other hosts (all randomly selected).

ECMPCONGAPresto

DRILL

UNDER THE HOOD

Leaf Spine: big improvementSpine Leaf: some improvementLeaf Host: no improvement

DRILL

PrestoPer-packet Round

RobinPer-packet Random

Enough duplicate ACKs to trigger TCP retransmission

Fine granularity is helpful but not enough: load-awareness keeps dup ACKs under control.

DRILL’S KEY IDEAS Key results:

Graph decomposition

Micro-LB:Randomized algorithm

Simple switch designLess than 1% area overheadNo host changesNo global coordination

Low FCT2.5x tail FCT reduction in incast

Theoretical analysis: Stable and 100% throughput

Micro Load Balancing for Low-latency Data Center...

Documents

Transcript of Micro Load Balancing for Low-latency Data Center...