VINI: Virtual Network Infrastructure Nick Feamster Georgia Tech Andy Bavier, Mark Huang, Larry...

32
VINI: Virtual Network Infrastructure Nick Feamster Georgia Tech Andy Bavier, Mark Huang, Larry Peterson, Jennifer Rexford Princeton University

Transcript of VINI: Virtual Network Infrastructure Nick Feamster Georgia Tech Andy Bavier, Mark Huang, Larry...

Page 1: VINI: Virtual Network Infrastructure Nick Feamster Georgia Tech Andy Bavier, Mark Huang, Larry Peterson, Jennifer Rexford Princeton University.

VINI: Virtual Network Infrastructure

Nick FeamsterGeorgia Tech

Andy Bavier, Mark Huang, Larry Peterson, Jennifer RexfordPrinceton University

Page 2: VINI: Virtual Network Infrastructure Nick Feamster Georgia Tech Andy Bavier, Mark Huang, Larry Peterson, Jennifer Rexford Princeton University.

VINI Overview

• Runs real routing software• Exposes realistic network conditions• Gives control over network events• Carries traffic on behalf of real users• Is shared among many experiments

Simulation

Emulation

Small-scaleexperiment

Livedeployment

?VINI

Bridge the gap between “lab experiments” and live experiments at scale.

Page 3: VINI: Virtual Network Infrastructure Nick Feamster Georgia Tech Andy Bavier, Mark Huang, Larry Peterson, Jennifer Rexford Princeton University.

Goal: Control and Realism

• Control– Reproduce results– Methodically change or

relax constraints

• Realism– Long-running services

attract real users– Connectivity to real Internet– Forward high traffic

volumes (Gb/s)– Handle unexpected events

TopologyActual network

Arbitrary, emulated

TrafficReal clients, servers

Synthetic or traces

Network EventsObserved in operational network

Inject faults, anomalies

Page 4: VINI: Virtual Network Infrastructure Nick Feamster Georgia Tech Andy Bavier, Mark Huang, Larry Peterson, Jennifer Rexford Princeton University.

Overview

• VINI characteristics– Fixed, shared infrastructure– Flexible network topology– Expose/inject network events– External connectivity and routing adjacencies

• PL-VINI: prototype on PlanetLab• Preliminary Experiments• Ongoing work

Page 5: VINI: Virtual Network Infrastructure Nick Feamster Georgia Tech Andy Bavier, Mark Huang, Larry Peterson, Jennifer Rexford Princeton University.

Fixed Infrastructure

Page 6: VINI: Virtual Network Infrastructure Nick Feamster Georgia Tech Andy Bavier, Mark Huang, Larry Peterson, Jennifer Rexford Princeton University.

Shared Infrastructure

Page 7: VINI: Virtual Network Infrastructure Nick Feamster Georgia Tech Andy Bavier, Mark Huang, Larry Peterson, Jennifer Rexford Princeton University.

Arbitrary Virtual Topologies

Page 8: VINI: Virtual Network Infrastructure Nick Feamster Georgia Tech Andy Bavier, Mark Huang, Larry Peterson, Jennifer Rexford Princeton University.

Exposing and Injecting Failures

Page 9: VINI: Virtual Network Infrastructure Nick Feamster Georgia Tech Andy Bavier, Mark Huang, Larry Peterson, Jennifer Rexford Princeton University.

Carry Traffic for Real End Users

s

c

Page 10: VINI: Virtual Network Infrastructure Nick Feamster Georgia Tech Andy Bavier, Mark Huang, Larry Peterson, Jennifer Rexford Princeton University.

Participate in Internet Routing

s

c

BGP

BGP

BGP

BGP

Page 11: VINI: Virtual Network Infrastructure Nick Feamster Georgia Tech Andy Bavier, Mark Huang, Larry Peterson, Jennifer Rexford Princeton University.

PL-VINI: Prototype on PlanetLab

• First experiment: Internet In A Slice– XORP open-source routing protocol suite (NSDI ’05)– Click modular router (TOCS ’00, SOSP ’99)

• Clarify issues that VINI must address– Unmodified routing software on a virtual topology– Forwarding packets at line speed– Illusion of dedicated hardware– Injection of faults and other events

Page 12: VINI: Virtual Network Infrastructure Nick Feamster Georgia Tech Andy Bavier, Mark Huang, Larry Peterson, Jennifer Rexford Princeton University.

PL-VINI: Prototype on PlanetLab

• PlanetLab: testbed for planetary-scale services• Simultaneous experiments in separate VMs

– Each has “root” in its own VM, can customize

• Can reserve CPU, network capacity per VM

Virtual Machine Monitor (VMM)(Linux++)

NodeMgr

LocalAdmin

VM1 VM2 VMn…PlanetLab node

Page 13: VINI: Virtual Network Infrastructure Nick Feamster Georgia Tech Andy Bavier, Mark Huang, Larry Peterson, Jennifer Rexford Princeton University.

XORP: Control Plane

• BGP, OSPF, RIP, PIM-SM, IGMP/MLD

• Goal: run real routing protocols on virtual network topologies

XORP(routing protocols)

Page 14: VINI: Virtual Network Infrastructure Nick Feamster Georgia Tech Andy Bavier, Mark Huang, Larry Peterson, Jennifer Rexford Princeton University.

User-Mode Linux: Environment

• Interface ≈ network• PlanetLab limitation:

– Slice cannot create new interfaces

• Run routing software in UML environment

• Create virtual network interfaces in UML

XORP(routing protocols)

UML

eth1 eth3eth2eth0

Page 15: VINI: Virtual Network Infrastructure Nick Feamster Georgia Tech Andy Bavier, Mark Huang, Larry Peterson, Jennifer Rexford Princeton University.

Click: Data Plane

• Performance– Avoid UML overhead– Move to kernel, FPGA

• Interfaces tunnels– Click UDP tunnels

correspond to UML network interfaces

• Filters– “Fail a link” by blocking

packets at tunnel

XORP(routing protocols)

UML

eth1 eth3eth2eth0

Click

PacketForwardEngine

Control

DataUmlSwitch

element

Tunnel table

Filters

Page 16: VINI: Virtual Network Infrastructure Nick Feamster Georgia Tech Andy Bavier, Mark Huang, Larry Peterson, Jennifer Rexford Princeton University.

Intra-domain Route Changes

s

c

1176

587 846

260

700

6391295

2095

902

548

233

1893

366

856

Page 17: VINI: Virtual Network Infrastructure Nick Feamster Georgia Tech Andy Bavier, Mark Huang, Larry Peterson, Jennifer Rexford Princeton University.

Ping During Link Failure

70

80

90

100

110

120

0 10 20 30 40 50

Pin

g R

TT

(m

s)

Seconds

Link down

Link up

Routes converging

Page 18: VINI: Virtual Network Infrastructure Nick Feamster Georgia Tech Andy Bavier, Mark Huang, Larry Peterson, Jennifer Rexford Princeton University.

Close-Up of TCP Transfer

2.1

2.15

2.2

2.25

2.3

2.35

2.4

2.45

17.5 18 18.5 19 19.5 20

Meg

abyt

es in

str

eam

Seconds

Packet receiv ed

Slow start

Retransmitlost packet

PL-VINI enables a user-space virtual networkto behave like a real network on PlanetLab

Page 19: VINI: Virtual Network Infrastructure Nick Feamster Georgia Tech Andy Bavier, Mark Huang, Larry Peterson, Jennifer Rexford Princeton University.

Challenge: Attracting Real Users

• Could have run experiments on Emulab

• Goal: Operate our own virtual network– Carrying traffic for actual users– We can tinker with routing protocols

• Attracting real users

Page 20: VINI: Virtual Network Infrastructure Nick Feamster Georgia Tech Andy Bavier, Mark Huang, Larry Peterson, Jennifer Rexford Princeton University.

Conclusion

• VINI: Controlled, Realistic Experimentation

• Installing VINI nodes in NLR, Abilene

• Download and run Internet In A Slice

http://www.vini-veritas.net/

Page 21: VINI: Virtual Network Infrastructure Nick Feamster Georgia Tech Andy Bavier, Mark Huang, Larry Peterson, Jennifer Rexford Princeton University.
Page 22: VINI: Virtual Network Infrastructure Nick Feamster Georgia Tech Andy Bavier, Mark Huang, Larry Peterson, Jennifer Rexford Princeton University.

TCP Throughput

0

2

4

6

8

10

12

0 10 20 30 40 50

Meg

abyt

es t

rans

ferr

ed

Seconds

Packet receiv ed

Link down

Link up

Zoom in

Page 23: VINI: Virtual Network Infrastructure Nick Feamster Georgia Tech Andy Bavier, Mark Huang, Larry Peterson, Jennifer Rexford Princeton University.

Ongoing Work

• Improving realism– Exposing network failures and changes in the

underlying topology– Participating in routing with neighboring networks

• Improving control – Better isolation– Experiment specification

Page 24: VINI: Virtual Network Infrastructure Nick Feamster Georgia Tech Andy Bavier, Mark Huang, Larry Peterson, Jennifer Rexford Princeton University.

Resource Isolation

• Issue: Forwarding packets in user space– PlanetLab sees heavy use– CPU load affects virtual network performance

Property Depends On Solution

Throughput CPU% received PlanetLab provides CPU reservations

Latency CPU scheduling delay

PL-VINI: boost priority of packet forward process

Page 25: VINI: Virtual Network Infrastructure Nick Feamster Georgia Tech Andy Bavier, Mark Huang, Larry Peterson, Jennifer Rexford Princeton University.

Performance is bad

• User-space Click: ~200Mb/s forwarding

Page 26: VINI: Virtual Network Infrastructure Nick Feamster Georgia Tech Andy Bavier, Mark Huang, Larry Peterson, Jennifer Rexford Princeton University.

VINI should use Xen

Page 27: VINI: Virtual Network Infrastructure Nick Feamster Georgia Tech Andy Bavier, Mark Huang, Larry Peterson, Jennifer Rexford Princeton University.

Experimental Results

• Is a VINI feasible?– Click in user-space: 200Mb/s forwarded– Latency and jitter comparable between network and

IIAS on PL-VINI.– Say something about running on just PlanetLab?

Don’t spend much time talking about CPU scheduling…

Page 28: VINI: Virtual Network Infrastructure Nick Feamster Georgia Tech Andy Bavier, Mark Huang, Larry Peterson, Jennifer Rexford Princeton University.

Low latency for everyone?

• PL-VINI provided IIAS with low latency by giving it high CPU scheduling priority

Page 29: VINI: Virtual Network Infrastructure Nick Feamster Georgia Tech Andy Bavier, Mark Huang, Larry Peterson, Jennifer Rexford Princeton University.

Internet In A SliceXORP• Run OSPF• Configure FIB

Click• FIB• Tunnels• Inject faults

OpenVPN & NAT• Connect clients

and servers

S

C

S

C

C

S

Page 30: VINI: Virtual Network Infrastructure Nick Feamster Georgia Tech Andy Bavier, Mark Huang, Larry Peterson, Jennifer Rexford Princeton University.

PL-VINI / IIAS Router

• Blue: topology– Virtual net devices– Tunnels

• Red: routing and forwarding– Data traffic does not enter

UML

• Green: enter & exit IIAS overlay

UML

XORP

eth1 eth3eth2

UmlSwitch

UmlSwitchelementFIB

Encapsulation table

eth0

Control

Data

Click

tap0

Page 31: VINI: Virtual Network Infrastructure Nick Feamster Georgia Tech Andy Bavier, Mark Huang, Larry Peterson, Jennifer Rexford Princeton University.

PL-VINI SummaryFlexible Network Topology

Virtual point-to-point connectivity Tunnels in Click

Unique interfaces per experiment Virtual network devices in UML

Exposure of topology changes Upcalls of layer-3 alarms

Flexible Routing and Forwarding

Per-node forwarding table Separate Click per virtual node

Per-node routing process Separate XORP per virtual node

Connectivity to External Hosts

End-hosts can direct traffic through VINI Connect to OpenVPN server

Return traffic flows through VINI NAT in Click on egress node

Support for Simultaneous Experiments

Isolation between experiments PlanetLab VMs and network isolation

CPU reservations and priorities

Distinct external routing adjacencies BGP multiplexer for external sessions

Page 32: VINI: Virtual Network Infrastructure Nick Feamster Georgia Tech Andy Bavier, Mark Huang, Larry Peterson, Jennifer Rexford Princeton University.

PL-VINI / IIAS Router

• XORP: control plane• UML: environment

– Virtual interfaces

• Click: data plane– Performance

• Avoid UML overhead• Move to kernel, FPGA

– Interfaces tunnels– “Fail a link”

XORP(routing protocols)

UML

eth1 eth3eth2eth0

Click

PacketForwardEngine

Control

DataUmlSwitch

element

Tunnel table