Interative Traffic Engineering in Changing Internet Economics - Tom Daly at LISA 2015
-
Upload
fastly -
Category
Technology
-
view
564 -
download
0
Transcript of Interative Traffic Engineering in Changing Internet Economics - Tom Daly at LISA 2015
Name of Presentation
Iterative Traffic Engineeringin Changing Internet Economics
Tom DalyVP, [email protected]
Name of Presentation
• Provide a framework to make complex bandwidth purchasing decisions:– Business needs– Traffic throughput and performance– Direct cost management and cost optimization– Staff size and cost
• Interest in:– Internet Economics Megatrends– Disruptive Events– Metrics and Tools
• Creation of a long term bandwidth acquisition strategy
Goals
Name of Presentation
• Industry experience: 15+ years of datacenter and connectivity acquisition deals
• I work for a CDN: Fastly– We use a lot of bandwidth– Therefore, we acquire much bandwidth– Partnerships with Sales / Deal Desk, Finance, Marketing, and
Engineering• Internet T/E techniques: DNS, BGP, and IP Anycast
– And related measurement techniques: Performance Monitoring, Real User Monitoring (RUM), TCP Flow Analysis
• Education:– BS ECE – Worcester Polytechnic Institute– MBA – Bentley University
Presenter Qualifications
Name of Presentation
• Pricing:– Don’t ask for pricing at volume; I honor non-disclosure
agreements.– IP Transit costs are normalized to USD$10/Mbps/MRC
for services delivered in carrier neutral datacenter.– IP Peering costs are normalized to USD $5000/10G
Port/MRC.• Net Neutrality:
– I/My employer have a duty to deliver content bits on behalf of our customers in an performance optimal and efficient manner.
• Opinions and views expressed are my own; not the viewpoint of Fastly, Inc.
Assumptions
Name of Presentation
• IP Transit: Paid access to the full Internet routing table.• Peering: Access to a subset of Internet routes via agreed
upon interconnection.– Settlement Free Peering: Free and balanced.– Paid Peering: Dominant party pays
• Internet Exchange Point (IXP): Peering across a shared Layer 2 switch domain
• Cloud Interconnect Service: Peering across an allocated Layer 2 switch domain
• Private Network Interconnect (PNI): Peering across a physical circuit
Nomenclature
Name of Presentation
The Challenge: Growth and Scale
Name of Presentation
The Challenge: Growth and Scale
Name of Presentation
The Challenge: Growth and Scale
$$$
Name of Presentation
• 10Gb Ethernet is mainstream; 100GbE is accessible
• Net Neutrality rulings in US and EU– Highly varying benefits and downside between US and
EU• The “dumb” pipes problem: Transport and IP
services increasingly commoditized• Resultantly; the global IP Transit market
compresses 16-18% per year, depending on region
• Movements to private datacenter/cloud interconnection
• Massive improvements to measurement and monitoring
Industry Shifting Economic Trends
Name of Presentation
Capitalize the Open Market Economy
Name of Presentation
Question: Transit vs. Peering?
Name of Presentation
Phase 1: Transit
Name of Presentation
Transit Pricing at Scale
Name of Presentation
Transit Pricing at Scale
“Yolo…just go peer some traffic”
Name of Presentation
Phase 2: Transit + Some Offload
IP Transit
PeeringCloud Interconnect
Name of Presentation
Peering Offload Modeling (20Gbps)
Name of Presentation
Peering Offload Modeling (20Gbps)
Name of Presentation
Transit Pricing – Compressing 17% YoY
Name of Presentation
Transit Pricing – Compressing 17% YoY
No consideration for peering staff overhead or
peering externalities
Name of Presentation
Transit Pricing – Compressing 17% YoY
No consideration for peering staff overhead or
peering externalities2x Peering Coordinators3x NOC---------------------------------$425,000/yr + Benefits
Name of Presentation
Peering Path Connection Complexity
Name of Presentation
Peering Offload Modeling (40Gbps)
Name of Presentation
Peering Offload Modeling (40Gbps)
Name of Presentation
• IXP outages• Congestion through IXP ports• Unbalanced peers won’t upgrade
PNIs• “de-peering” between
interconnected networks• Changes to routing and geo-location
degrade or improve performance
Disruptive Peering Events and Behaviors
Name of Presentation
Single Month Outage (40Gbps)
Name of Presentation
Question: Capacity Planning + Cost Models
What we know
The Future!
Name of Presentation
Regression Analysis - Excel
Name of Presentation
Regression Analysis – Peering Prediction
Name of Presentation
Scenario Analysis: Expected Peering Productivity
Name of Presentation
Question: 10Gb vs. 100Gb Ethernet12x10Gb Ports- 12x 10GBase-LR Optics:
$150 each = $1,800- 12x SMF Cross Connects:
$350/MRC = $50,400/ARC- Total: $52,200/1 YR
Name of Presentation
100GbE Scenario12x10Gb Ports- 12x 10GBase-LR Optics:
$150 each = $1,800- 12x SMF Cross Connects:
$350/MRC = $50,400/ARC- Total: $52,200/1 YR- Consider line card ports 2x100Gb Ports
- 2x 100GBase-LR4 Optics:$5600 each = $11,200
- 2x SMF Cross Connects:$350/MRC = $8,400/ARC
- Total: $19,600/1 YR- Consider line card ports
Name of Presentation
Eggs? Why are they in one basket?12x10Gb Ports- 12x 10GBase-LR Optics:
$150 each = $1,800- 12x SMF Cross Connects:
$350/MRC = $50,400/ARC- Total: $52,200/1 YR
2x100Gb Ports- 2x 100GBase-LR4 Optics:
$5600 each = $11,200- 2x SMF Cross Connects:
$350/MRC = $8,400/ARC- Total: $19,600/1 YR
Reliability Modeling?- Afford to loose 10G at a time?- Afford to loose 100G at time?
Name of Presentation
An incomplete story…
Performance Matters...
Volumetric cost improvements…
Name of Presentation
Latency and Bandwidth Delay Product
Name of Presentation
ms and BDP: Doubled Packet Loss
Name of Presentation
ms and BDP: Halved Latency
Name of Presentation
“You can't manage what you can't measure.”
– Peter F. DruckerSocial Ecologist;
Management Consultant;Professor
Name of Presentation
Detailed Measurements• Synthetic Agents: Latency, packet loss,
and path validation• RUM: Geolocation, Recursive DNS
divergence and positioning, /latency/• NetFlow / sFlow: Bandwidth by Remote
ASN and Next-Hop Peer• Linux TCPExt: Retransmission and Packet
Loss• ICMP Ping: Position confirmation;
triangulation
Name of Presentation
RIPE Atlas – https://atlas.ripe.net
Name of Presentation
Real User Monitoring (RUM)
Name of Presentation
NetFlow / sFlow Analysis
Name of Presentation
Linux TCP Extensions
Name of Presentation
ICMP Ping – “Fastly Style”
Name of Presentation
Methodology and Take Away(s)
Name of Presentation
• Early Stage: Just buy transit!– Too small and too inefficient to peer–Work on tooling to help you peer later
by analyzing traffic patterns– Automate automate automate your
routine peering and monitoring workloads
Iterative Traffic Engineering (1)
Name of Presentation
• Mid Stage: Identify growth plans and potential– Regression analysis across 18-24
months data– Identify and connect strategic / high
value peers– Use cost or performance scenario
analysis– Note: Not all networks are good peers;
see risks
Iterative Traffic Engineering (2)
Name of Presentation
• Ongoing: Constant evolution– Transit providers apprised of growth plan– Contract duration adjusted to long term risk of
market price compression– Peering load adjusted to team side– Measure, measure, measure
Iterative Traffic Engineering (3)