Post on 22-Dec-2015
An Introduction to Peer-to-Peer System
Diganta GoswamiIIT Guwahati
2
Outline
Overview of P2P overlay networks Applications of overlay networks Classification of overlay networks
Structured overlay networks Unstructured overlay networks Overlay multicast networks
2
3
Overview of P2P overlay networks
What is P2P systems? P2P refers to applications that take advantage of
resources (storage, cycles, content, …) available at the end systems of the internet.
What is overlay networks? Overlay networks refer to networks that are constructed
on top of another network (e.g. IP). What is P2P overlay network?
Any overlay network that is constructed by the Internet peers in the application layer on top of the IP network.
3
4
What is P2P systems?
Multiple sites (at edge) Distributed resources Sites are autonomous (different owners) Sites are both clients and servers Sites have equal functionality
4
5
Internet P2P Traffic Statistics
Between 50 and 65 percent of all download traffic is P2P related.
Between 75 and 90 percent of all upload traffic is P2P related.
And it seems that more people are using p2p today
So what do people download? 61.4 % video
11.3 % audio27.2 % games/software/etc.
Source: http://torrentfreak.com/peer-to-peer-traffic-statistics/
5
6
P2P overlay networks properties
Efficient use of resources Self-organizing
All peers organize themselves into an application layer network on top of IP.
Scalability Consumers of resources also donate resources Aggregate resources grow naturally with
utilization
6
7
P2P overlay networks properties
Reliability No single point of failure Redundant overlay links between the peers Redundant data source
Ease of deployment and administration The nodes are self-organized No need to deploy servers to satisfy demand. Built-in fault tolerance, replication, and load
balancing No need any change in underlay IP networks
7
8
P2P Applications
P2P File SharingNapster, Gnutella, Kazaa, eDonkey, BitTorrentChord, CAN, Pastry/Tapestry, Kademlia
P2P CommunicationsSkype, Social Networking Apps
P2P Distributed ComputingSeti@home
8
9
Popular file sharing P2P Systems
Napster, Gnutella, Kazaa, Freenet Large scale sharing of files.
User A makes files (music, video, etc.) on their computer available to others
User B connects to the network, searches for files and downloads files directly from user A
Issues of copyright infringement
9
10
P2P/Grid Distributed Processing
seti@home Search for ET intelligence Central site collects radio telescope data Data is divided into work chunks of 300 Kbytes User obtains client, which runs in background Peer sets up TCP connection to central computer,
downloads chunk Peer does FFT on chunk, uploads results, gets
new chunk Not P2P communication, but exploit Peer
computing power
10
11
Key Issues
Management How to maintain the P2P system under high rate of
churn efficiently Application reliability is difficult to guarantee
Lookup How to find out the appropriate content/resource that a
user wants Throughput
Content distribution/dissemination applications How to copy content fast, efficiently, reliably
11
12
Management Issue A P2P network must be self-organizing.
Join and leave operations must be self-managed. The infrastructure is untrusted and the components are
unreliable. The number of faulty nodes grows linearly with system size. Tolerance to failures and churn
Content replication, multiple paths Leverage knowledge of executing application
Load balancing Dealing with free riders
Freerider : rational or selfish users who consume more than their fair share of a public resource, or shoulder less than a fair share of the costs of its production.
12
13
Lookup Issue
How do you locate data/files/objects in a large P2P system built around a dynamic set of nodes in a scalable manner without any centralized server or hierarchy?
Efficient routing even if the structure of the network is unpredictable. Unstructured P2P : Napster, Gnutella, Kazaa Structured P2P : Chord, CAN, Pastry/Tapestry,
Kademlia
13
14
Classification of overlay networks
Structured overlay networks Are based on Distributed Hash Tables (DHT) the overlay network assigns keys to data items and
organizes its peers into a graph that maps each data key to a peer.
Unstructured overlay networks The overlay networks organize peers in a random graph
in flat or hierarchical manners. Overlay multicast networks
The peers organize themselves into an overlay tree for multicasting.
14
15
Structured overlay networks Overlay topology construction is based on NodeID’s that
are generated by using Distributed Hash Tables (DHT).
The overlay network assigns keys to data items and organizes its peers into a graph that maps each data key to a peer.
This structured graph enables efficient discovery of data items using the given keys.
It Guarantees object detection in O(log n) hops.
15
16
Unstructured P2P overlay networks An Unstructured system composed of peers
joining the network with some loose rules, without any prior knowledge of the topology.
Network uses flooding or random walks as the mechanism to send queries across the overlay with a limited scope.
16
17
Unstructured P2P File Sharing Networks
Centralized Directory based P2P systems Pure P2P systems Hybrid P2P systems
17
18
Unstructured P2P File Sharing Networks Centralized Directory based P2P systems
All peers are connected to central entity Peers establish connections between each
other on demand to exchange user data (e.g. mp3 compressed data)
Central entity is necessary to provide the service
Central entity is some kind of index/group database
Central entity is lookup/routing table Examples: Napster, Bittorent
18
19
Napster was used primarily for file sharing NOT a pure P2P network=> hybrid system Ways of action:
Client sends server the query, server ask everyone and responds to client
Client gets list of clients from server All Clients send ID’s of the data they hold to the
server and when client asks for data, server responds with specific addresses
peer downloads directly from other peer(s)
19
20
Centralized Network
Napster model• Nodes register their
contents with server• Centralized server for
searches• File access done on a peer
to peer basis
– Poor scalability
– Single point of failure
Client
Server
Client
Query
Reply
File Transfer
20
21
Napster
Further services: Chat program, instant messaging service, tracking
program,…
Centralized system Single point of failure => limited fault tolerance Limited scalability (server farms with load balancing)
Query is fast and upper bound for duration can be given
21
22
Gnutella
pure peer-to-peer very simple protocol no routing "intelligence" Constrained broadcast
Life-time of packets limited by TTL (typically set to 7)
Packets have unique ids to detect loops
22
23
Query flooding: Gnutella fully distributed
no central server
public domain protocol
many Gnutella clients implementing protocol
overlay network: graph edge between peer X and
Y if there’s a TCP connection
all active peers and edges is overlay net
Edge is not a physical link Given peer will typically
be connected with < 10 overlay neighbors
23
24
Gnutella: protocol
Query
QueryHit
Query
Query
QueryHit
Query
Query
QueryHit
File transfer:HTTP
Query messagesent over existing TCPconnections peers forwardQuery message QueryHit sent over reversepath
Scalability:limited scopeflooding
24
25
Gnutella : ScenarioStep 0: Join the networkStep 1: Determining who is on the network
• "Ping" packet is used to announce your presence on the network. • Other peers respond with a "Pong" packet. • Also forwards your Ping to other connected peers• A Pong packet also contains:
• an IP address • port number • amount of data that peer is sharing • Pong packets come back via same route
Step 2: Searching•Gnutella "Query" ask other peers (usually 7) if they have the file you desire• A Query packet might ask, "Do you have any content that matches the string ‘Hey Jude"? • Peers check to see if they have matches & respond (if they have any matches) & send packet to connected peers if not (usually 7)• Continues for TTL (how many hops a packet can go before it dies, typically 10 )
Step 3: Downloading• Peers respond with a “QueryHit” (contains contact info)• File transfers use direct connection using HTTP protocol’s GET method
25
26
Gnutella: Peer joining
1. Joining peer X must find some other peer in Gnutella network: use list of candidate peers
2. X sequentially attempts to make TCP with peers on list until connection setup with Y
3. X sends Ping message to Y; Y forwards Ping message.
4. All peers receiving Ping message respond with Pong message
5. X receives many Pong messages. It can then setup additional TCP connections
26
27
Gnutella - PING/PONG
1 52
4
3
6
7
8
Ping 1
Ping 1
Ping 1
Ping 1
Ping 1
Ping 1
Ping 1Known Hosts:2
3,4,5
6,7,8
Pong 2
Pong 4
Pong 3
Pong 5Pong 3,4,5
Pong 6,7,8 Pong 6
Pong 7
Pong 8
Pong 6,7,8
Query/Response analogous
27
28
Unstructured Blind - Gnutella
= forward query
= processed query
= source
= found result
= forward response
Breadth-First Search (BFS)
28
29
Unstructured Blind - Gnutella
A node/peer connects to a set of Gnutella neighbors
Forward queries to neighbors
Client which has the Information responds.
Flood network with TTL for termination
+ Results are complete
– Bandwidth wastage
29
30
Gnutella : Reachable Users(analytical estimate)
T : TTL, N : Neighbors for Query
30
31
Gnutella : Search Issue
Flooding based search is extremely wasteful with bandwidth A large (linear) part of the network is covered irrespective of
hits found Enormous number of redundant messages All users do this in parallel: local load grows linearly with
size What can be done?
Controlling topology to allow for better search Random walk, Degree-biased Random Walk
Controlling placement of objects Replication
31
32
Gnutella : Random Walk Basic strategy
In scale-free graph: high degree nodes are easy to find by (biased) random walk
Scale-free graph is a graph whose degree distribution follows a power law
And high degree nodes can store the index about a large portion of the network
Random walk avoiding the visit of last visited node
Degree-biased random walk Select highest degree node, that has
not been visited This first climbs to highest degree node,
then climbs down on the degree sequence Provably optimal coverage
32
33
Gnutella : Replication
Spread copies of objects to peers: more popular objects can be found easier
Replication strategies Owner replication Path replication Random replication
But there is still the difficulty with rare objects.
33
34
Random Walkers
Improved Unstructured Blind•Similar structure to Gnutella•Forward the query (called walker) to random subset of its neighbors+ Reduced bandwidth requirements– Incomplete results
Peer nodes34
35
Unstructured Informed Networks
Zero in on target based on information about the query and the neighbors.
Intelligent routing
+ Reduces number of messages
+ Not complete, but more accurate
– COST: Must thus flood in order to get initial information
35
36
Informed Searches: Local Indices
Node keeps track of information available within a radius of r hops around it.
Queries are made to neighbors just beyond the r radius.
+ Flooding limited to bounded part of network
36
37
Routing Indices For each query, calculate goodness of each
neighbor.
Calculating goodness: Categorize or separate query into themes Rank best neighbors for a given theme based on
number of matching documents
Follows chain of neighbors that are expected to yield the best results
Backtracking possible37
38
Free riding
File sharing networks rely on users sharing data Two types of free riding
Downloading but not sharing any data Not sharing any interesting data
On Gnutella 15% of users contribute 94% of content 63% of users never responded to a query
Didn’t have “interesting” data
38
39
Gnutella:summary Hit rates are high High fault tolerance Adopts well and dynamically to changing peer
populations High network traffic No estimates on duration of queries No probability for successful queries Topology is unknown => algorithm cannot exploit it Free riding is a problem
A significant portion of Gnutella peers are free riders Free riders are distributed evenly across domains Often hosts share files nobody is interested in
39
40
Gnutella discussion Search types:
Any possible string comparison Scalability
Search very poor with respect to number of messages Updates excellent: nothing to do Routing information: low cost
Robustness High, since many paths are explored
Autonomy: Storage: no restriction, peers store the keys of their files Routing: peers are target of all kind of requests
Global knowledge None required
40
41
Exploiting heterogeneity: KaZaA Each peer is either a group
leader or assigned to a group leader. TCP connection between
peer and its group leader. TCP connections between
some pairs of group leaders.
Group leader tracks the content in all its children.
ordinary peer
group-leader peer
neighoring re la tionshipsin overlay network
41
42
iMesh, Kazaa Hybrid of centralized Napster and
decentralized Gnutella Super-peers act as local search
hubs Each super-peer is similar to a
Napster server for a small portion of the network
Super-peers are automatically chosen by the system based on their capacities (storage, bandwidth, etc.) and availability (connection time)
Users upload their list of files to a super-peer
Super-peers periodically exchange file lists
Queries are sent to a super-peer for files of interest
42
43
Overlay Multicasting
IP multicast has not been deployed over the Internet due to some fundamental problems in congestion control, flow control, security, group management and etc.
For the new emerging applications such as multimedia streaming, internet multicast service is required.
Solution: Overlay Multicasting Overlay multicasting (or Application layer multicasting) is
increasingly being used to overcome the problem of non-ubiquitous deployment of IP multicast across heterogeneous networks.
43
44
Overlay Multicasting
Main idea Internet peers organize themselves into an
overlay tree on top of the Internet. Packet replication and forwarding are
performed by peers in the application layer by using IP unicast service.
44
45
Overlay Multicasting
Overlay multicasting benefits Easy deployment
It is self-organized it is based on IP unicast service There is not any protocol support requirement by the Internet
routers. Scalability
It is scalable with multicast groups and the number of members in each group.
Efficient resource usage Uplink resources of the Internet peers is used for multicast data
distribution. It is not necessary to use dedicated infrastructure and bandwidths
for massive data distribution in the Internet.
45
46
Overlay Multicasting
Overlay multicast approachesDHT basedTree based Mesh-tree based
46
47
Overlay Multicasting
DHT based Overlay tree is constructed on top of the DHT based
P2P routing infrastructure such as pastry, CAN, Chord, etc.
Example: Scribe in which the overlay tree is constructed on a Pastry networks by using a multicast routing algorithm
47
48
Structured Overlay Networks / DHTs
Keys of ValuesKeys of Values
Keys of NodesSet of Nodes
…
Chord, Pastry, Tapestry, CAN, Kademlia, P-Grid, Viceroy
Node IdentifierValue Identifier
Hashing
Hashing
Common Identifier Space
ConnectThe nodesSmartly
48
49
The Principle Of Distributed Hash Tables A dynamic distribution of a hash table onto a set of cooperating
nodes
Key Value
1 Algorithms
9 Routing
11 DS
12Peer-to-Peer
21 Networks
22 Grids
• Basic service: lookup operation • Key resolution from any node
• Each node has a routing table • Pointers to some other nodes• Typically, a constant or a logarithmic number of pointers
node A
node D
node B
node C
→Node D : lookup(9)
49
50
DHT Desirable Properties
Keys mapped evenly to all nodes in the network
Each node maintains information about only a few other nodes
Messages can be routed to a node efficiently
Node arrival/departures only affect a few nodes
50
51
Chord [MIT]
Problem adressed: efficient node localization
Distributed lookup protocol Simplicity, provable performance, proven
correctness Support of just one operation: given a key,
Chord maps the key onto a node
51
52
The Chord algorithm –Construction of the Chord ring
the consistent hash function assigns each node and each key an m-bit identifier using SHA 1 (Secure Hash Standard).
m = any number big enough to make collisions improbable
Key identifier = SHA-1(key)
Node identifier = SHA-1(IP address) Both are uniformly distributed Both exist in the same ID space
52
53
Chord consistent hashing (SHA-1) assigns each
node and object an m-bit ID IDs are ordered in an ID circle ranging from
0 – (2m-1). New nodes assume slots in ID circle
according to their ID Key k is assigned to first node whose ID ≥ k successor(k)
53
54
Consistent Hashing - Successor Nodes
6
1
2
6
0
4
26
5
1
3
7
2identifier
circle
identifier
node
X key
successor(1) = 1
successor(2) = 3successor(6) = 0
54
55
Consistent Hashing – Join and Departure
When a node n joins the network, certain keys previously assigned to n’s successor now become assigned to n.
When node n leaves the network, all of its assigned keys are reassigned to n’s successor.
55
56
Consistent Hashing – Node Join
0
4
26
5
1
3
7
keys1
keys2
keys
keys
7
5
56
57
Consistent Hashing – Node Dep.
0
4
26
5
1
3
7
keys1
keys2
keys
keys6
7
57
58
Simple node localization
// ask node n to find the successor of id
n.find_successor(id)
if (id (n; successor])
return successor;
else
// forward the query around the
circle
return successor.find_successor(id);
=> Number of messages linear in the number of nodes !
58
59
Scalable Key Location – Finger Tables To accelerate lookups, Chord maintains additional routing
information. This additional information is not essential for correctness,
which is achieved as long as each node knows its correct successor.
Each node n, maintains a routing table with up to m entries (which is in fact the number of bits in identifiers), called finger table.
The ith entry in the table at node n contains the identity of the first node s that succeeds n by at least 2i-1 on the identifier circle.
s = successor(n+2i-1). s is called the ith finger of node n, denoted by n.finger(i)
59