Post on 24-Dec-2015
Architectures for Peer-to-Peer Content Distribution
Christos GkantsidisMicrosoft Research, Cambridge, UK
Joint work with: Pablo Rodriguez, John Miller, Dinan Gunawardena, and others
Peer-to-Peer Applications
BBC iMP
SETI@Home .NET (PeerChannel, WCF)JXTA
P2P content distribution is hugely popular: >40-60% of Internet trafficEasy access to an amazing variety of content
Challenges of P2P: P2P traffic difficult to manage
ISP complaints Dependence of free bandwidth
e.g. from universities Massive copyright infringement
2
Microsoft scenarios where P2P can helpLarge scale software distribution:
Windows / Microsoft updates Beta versions of software (e.g. Visual Studio) Free software
MSN Video File sharing (e.g. Foldershare, Groove, Live Messenger)
Windows XP P2P Name Resolution Protocol (PNRP) P2P Graphing and Grouping Peer Identity Manager
Windows VistaPNRP, Graphing and Grouping, Peer Identity Manager, P2P Contacts, People Near MeWindows Communication Foundation Peer Channel
Incidentally, MS has been interested in P2P for some time …
3
Peer-to-Peer Distribution• Avalanche: P2P file distribution
– Reduce download times, use network resources optimally
– Secure distribution:• Publisher and content protection• Only authorized users can join the distribution networks
– Cache-friendly distribution– Enable network locality
• RedCarpet: P2P Video-on-Demand– Watch movies while downloading …– … without loosing delivery efficiency.– New P2P capability!
• RewindTV: Enhanced Live TV– Watch live TV …– … and be able to re-play missed scenes (from other
peers) 6
Peer-to-Peer Distribution• Avalanche: P2P file distribution
– Reduce download times, use network resources optimally
– Secure distribution:• Publisher and content protection• Only authorized users can join the distribution networks
– Cache-friendly distribution– Enable network locality
7
File Swarm = Block Scheduling
8
System’s progress in current File Swarming systems
9
(From Tian et al., Infocom’06)
A lot of time spent at the beginning and finish of download:• Beginning of download: finding good blocks to exchange• End of download: discovering the last missing blocks
Network Coding Simplified
10
File to Transfer
Block 1 Block 2 Block 3
Encoding
With Network Coding
11
System’s progress
12
• Smooth download progress:– No start-up delay– No last-block problem
13
Bandwidth Contribution
• Easily withstands flash crowds• Server contribution is fixed, Client contribution scales• >10 fold savings in content provider’s bandwidth using peer-to-peer.
Results from distributing Visual Studio
14
Data from distribution of beta versions of Visual Studio 2008 Beta (Nov’07)
Median speeds:~1.5Mbps for VS Pro~2.7Mbps for the others
Peer-to-Peer Distribution• Avalanche: P2P file distribution
– Reduce download times, use network resources optimally
– Secure distribution:• Publisher and content protection• Only authorized users can join the distribution
networks– Cache-friendly distribution– Enable network locality
15
Strong security model
• All connections TLS-authenticated
• Data is self-certifying– Block data validated by
signed homomorphic hash equivalent
• Strong identity enables solid DOS defenses– Misbehaving parties
cannot misrepresent themselves
16
Certificate Engine
Registrars
Publishers
Clients
Caches
Peer-to-Peer Distribution• Avalanche: P2P file distribution
– Reduce download times, use network resources optimally
– Secure distribution:• Publisher and content protection• Only authorized users can join the distribution
networks– Cache-friendly distribution– Enable network locality
17
ISP Cache Integration
• Protocol designed for cacheability– Network operators can redirect clients to
Avalanche-compatible caches– Reduce ISP pain– Improve experience for clients, publishers
• Experience can be accelerated• Cache can proxy or relay for poorly connected
nodes
18
Effect of P2P on inter-ISP traffic
19
Transit ISPTransit ISP
Broadband ISP
Broadband ISP
Broadband ISP
$$$$
$$$$
serverInternetIncrease in egress
traffic!!$$ => $$$$$
Does locality work?
20
Transit ISPTransit ISP
Broadband ISP
Broadband ISP
Broadband ISP
$$$$
$$$$
serverInternet
Benefits of locality for current P2P scenarios
Each ISP is required to upload just over a copy of the file (1.9
GB)
May require internal re-engineering 21
Architecture for P2P distribution
`
Consumer
`
Consumer
`
Consumer
CertificateAuthority
Seed Servers
Registrars
`
Publisher
22
Architecture for P2P distribution
`
Consumer
`
Consumer
`
Consumer
CertificateAuthority
Seed Servers
Registrars
`
Publisher
Cache
`
23
Peer matching based on locality
Provides extensibility points for ISPs and content owners
Content caches
Preference to local connections
Peer-to-Peer Distribution• Avalanche: P2P file distribution
– Reduce download times, use network resources optimally
– Secure distribution:• Publisher and content protection• Only authorized users can join the distribution networks
– Cache-friendly distribution– Enable network locality
• RedCarpet: P2P Video-on-Demand– Watch movies while downloading …– … without loosing delivery efficiency.– New P2P capability!
• RewindTV: Enhanced Live TV– Watch live TV …– … and be able to re-play missed scenes (from other
peers) 24
Large Scale Content Delivery
In current Peer-to-Peer systems:
• >60% of traffic is for video files• Users need to wait to download
the entire file before watching
Emerging: P2P Live Streaming:• Very popular: >200K concurrent
users in one swarm• Huge selection of content
(mostly Asian)
25
Large Scale Content Delivery
• Unstructured Peer-to-Peer– Scalability– Easy to construct/maintain
• Typical applications:– File distribution
E.g. eMule, BitTorrent
– Live streamingE.g. CoolStreaming et al.
Q) Can we use unstructured P2P to deliver Q) Can we use unstructured P2P to deliver a video-on-demand experience?a video-on-demand experience?Our approach:Our approach:• Pseudo-sequential downloads Pseudo-sequential downloads • Pre-fetching of dataPre-fetching of data• Network coding Network coding for efficient swarmingfor efficient swarming
26
Challenges of Video on Demand• Small start-up times
– Fast channel switching
• Good sustainable playback rate
• Users join at arbitrary points in time
• Video Seeking
futurefuture
futurefuture27
Network Coding vs. Non-NetCoding
• Flash crowd of 20 clients
• Best non-Network Coding scheme– Global-rarest
• Network Coding– 14% improvement in
playback rates– Low jitter in download
times Better user experienceGlobal rarest vs. Network Coding
(Based on a prototype implementation)
28
Other topic: Topology Management
29
- Issue: Peers at different stages of the download.- Issue: Heterogeneous capacities:
- Fast nodes should not be penalized.
Dat
a (%
)
Dat
a (%
)
30
Mass Scale Content DistributionMulti-purpose P2P platform
Lessons learned• P2P technologies are growing in acceptance
– Being deployed by Movie Studios, Television broadcasters, Music companies, Software distributors
• P2P systems need to respect the current ecosystem– Network operators: locality, caching– Content producers: strong security
• Network coding is feasible and practical today, and leads to performance improvements
• The unstructured P2P paradigm helps even for video distribution 31