Material from Computer Networking: A Top Down Approach, 6th edition. J.F. Kurose and K.W. Ross
ISPs, Backbones and Peering
14-740: Fundamentals of Computer NetworksBill Nace
Administrivia
• Norton2010 Paper Review for today
• Lab0 is posted
• I'm in Japan!
2
Last Lecture• Internet TCP/IP architecture• Layering not strictly enforced• “Hourglass” -- IP ueber alles• Allows rapid innovation at layers below• Flexible applications / services above• IETF process• “Rough consensus and running code”
• End-to-end argument• Does it still apply?
3
traceroute
• ISPs and Backbones• Peering and Settlements
• Peering Evolution
• Interconnections
4
A Packet’s Journey• Packets travel across many networks• Particular protocols will be studied later• This lecture motivates why routing
mechanics are necessaryCMU Pgh
PennRen
Some NW
Tier 1 ISP
Tier 1 ISP
Tier 1 ISPSome ISP
Little ISP
Cheap ISP
Huge ISP
CMU Kobe
AB
C
E
F
D
G
H
J
5
CMU’s Infrastructure• Two 10 Gigabit Ethernet links
(singlemode fiber) to Pittsburgh Supercomputer Center
• From Cisco 6500s located in Cyert and Stever House
• “Redundancy” links to PennRen / Level3 over 1Gbps rate-limited 10 Gigabit Ethernet
• Money
• $40K per year to PennRen
• A PA state education network
• $100K to PSC, $100K to Level3
• $300K per year on fiber leases
1Gbps/10Gbps10Gbps
10Gbps 1Gbps
Internal
External
CMUInternal
PennRen
PSCInternet2
Level3Comodity Internet
Backbones• In the beginning ... of Internet time• Single backbone: NSFNet• Everyone on the “Internet” was on NSFNet• A backbone network enables all connected
end-hosts (users and companies) to communicate with each other
• No interconnection problems
7
Backbones (2)• Commercial backbone providers emerged• Technology transfer!• A “Good Thing”• If US Government was managing the
Internet, might not be so successful• We all expect universal Internet
connectivity
8
Interfaces: Transit
• Transit / provider-customer
• ISP sells access to another ISP or company
• e.g. CMU buys transit (or access to Internet) from Level3
9
Interfaces: Peering• Reciprocal access to each other’s customers
• Usually free exchange of traffic
• DO NOT serve as transit for 3rd party data• E.g. Google and MSN peer with each other, so email
messages between Gmail and Hotmail are transferred directly, without going through their transit providers
• These relationships are confidential business secrets
• Roughly hierarchical, though the topology is flattening
10
Tier 1 ISP• Internet backbone providers• Peers with every other Tier-1 in “Internet Region”
• Who is a major ISP in your country?• US?
• Japan?
• India?
• China?
• Careful: “Tier-1” is an overloaded and misleading term
11
Images courtesy of Level-3
Images courtesy of Verizon>4200 networks in 142 nations, many at OC-768 speeds
Image courtesy of tatacommunications.com/network
Tata: "Discover the World's Largest Global Footprint"
“Typical” Infrastructure• High speed links• Level3 operates 40Gbps DWDM networks
• High performance routers
• Over-provisioned bandwidth• 40 ms delay within region• <1% packet loss
• Global presence (or at least multi-continent)
15
“Typical” Tier-1 Relationships
• Directly connected to other Tier-1 ISPs (i.e. peer with)
• Connected to a large number of Tier-2 ISPs
• Vertically integrated: sell services directly to customers
• International in coverage
• Ecosystem: do not buy transit from another provider in order to reach the whole Internet
16
Why do Tier-1 ISPs need to peer with each other?
• No single Tier-1 ISP can reach the whole Internet on its own• Internet is a network of networks
• But Tier-1 ISPs have a restrictive peering policy• Do not peer with other non-Tier-1 ISPs• They are potential revenue generating
customers• No incentive to accept additional peers
17
What is a Tier-2 ISP?• Network infrastructure is usually regional
• Customer of Tier-1 ISP(s) – needs to buy transit
• Provider of customers – also re-sells this transit
• Peers with other Tier-2 ISPs – settlement free
18
Tier 1 ISP
Tier 1 ISP
Tier 1 ISP
IXP
Tier-2 ISP
Tier-2 ISPTier-2 ISP
Tier-2 ISPTier-2 ISP pays Tier-1 for connectivity to rest
of internet
Multi-connection relationships possible
Tier-2 ISPs also peer privately. Interconnection can be direct or via IXP
Thinking time
19
• Why do Tier-2 ISPs need to buy transit from a Tier-1 provider?
Thinking time
20
• Why do Tier-2 ISPs peer with each other?
Open Peering Policy• Tier-2 ISPs have an open peering policy• Peer with anyone possible
• Costs of peering have to be balanced against gains for a Tier-2 ISP• Management cost: usually settlement-free
peering means both parties should send approx equal amounts of traffic to each other
• Maintenance cost: extra equipment, transmission capacity to meeting point, exchange and other fees
21
Content Providers• Do not sell transit
• Category A: Focus on content creation• Do not want to operate a network• Do not have expertise in maintaining peering
relationships — router, policy, negotiations, …• No Peering Policy
• Category B: Sophisticated, large-scale players• Use peering to improve user experience• Open Peering Policy
22
Peering Ecosystem
Tier 1 ISPs
Tier 2 ISPs
Content / Enterprise Companies
Full MeshPeering
Partial MeshPeering
GenerallyNo Peering
Must BuyTransit
Transit Free
Must BuyTransit
Internet Region
Things are never so clear-cut . . .
traceroute
• ISPs and Backbones• Peering and Settlements
• Peering Evolution
• Interconnections
24
US Evolution• 1999/2000 economic collapse • Telecom sector• Need to rethink their business model• Need to cut costs
• General dotcom bust• Lots of cheap equipment on eBay• Transit rates drop• Upstream provider @Home for cable companies went
bankrupt
• Peer-to-peer file sharing grows exponentially in popularity
25
Transit Prices are Falling• Price per Mbs ➙ 63% per year decline
• Luckily, volume is increasing even more
• Somewhat bad for Tier-1 ISPs
26
Content Companies are Peering
• Network savvy, large scale enterprises• MSN, Yahoo, Google, Ebay, Walmart, …• Yahoo has an open peering policy• Reduce transit costs• Improve end-user experience• Good for content companies• Bad for Tier-1 ISPs
27
traceroute
• ISPs and Backbones• Peering and Settlements
• Peering Evolution
• Interconnections
28
Interconnection
• How do two networks interconnect with each other?• We are talking about routers in two
different companies, under separate administrative control
29
Public Peering• U.S. government decided to let commercial companies
take over management of the backbone networks• 1991: Commercial Internet Exchange (CIX)• 1 router in Santa Clara, CA
• 1995: Network Access Points (NAPs) in SF, Chicago, NY, D.C.
• More recent: Internet Exchange Point (IXP)• 30 in US, 6 in Japan
• “More is good” because of congestion, at one point 20% of traffic going through the 1st NAP was dropped!
30
ISP Connections
Tier 1 ISP
Tier 1 ISP
Tier 1 ISP
IXP(Meet-me
room)
Tier 1 ISP
Tier-1 providers interconnect
privately (Peer)
Tier-1 providers also interconnect at Internet Exchange Points (IXP) (30 in US)
Private Peering• Two networks bypass IXPs and directly peer• In response to congestion at some IXPs• Business reasons
• Also more cost-effective• There are limited number of IXPs at major regions• Can privately peer in other locations
• Sometimes take a hybrid approach• Public exchange for some peers• Private meeting point for others
32
Provider-customer Interconnection
• Point-of-presence (PoP)
• Customer typically leases a high-speed link from a 3rd party telecomm provider
• Directly connects a router at a provider’s PoP
33
Various Methods
• How do two networks interconnect with each other?• Many methods: Public Peering, Private
Peering, Provider-customer PoP
• There are really no industry-specific regulations, mostly purely driven by commercial and economic forces
34
How does the world connect to Facebook?
Facebook’s network
IXP in Germany
IXP in Japan
IXP in Sweden
Level3
Amazon
Directly Connected Peer?
Image courtesy of Teun Vink (teun.tv)
Lesson Objectives• Now, you should be able to:• analyze business practices of various enterprises
using the multi-tier network model (Tier-1, 2, etc) and common peering practices
• describe the relationships and associated motivations for enterprises on the internet
• analyze the effect of recent trends in internet usage patterns on the various business enterprises on the internet
• describe interconnection methods between enterprise networks
36
Top Related