Outline:

• Background

• Client-Server vs. Peer to Peer Model.

UOZ–FS-CS

• Client-Server vs. Peer to Peer Model.

• BitTorrent Protocol.

• DHT Networks.

• Peer Exchange.

• Magnet Links

• History

• Use of Content Hashes• Use of Content Hashes

• Technical Description

• The Pirate Bay

Karwan Jacksi22/04/2012

Background

• Client Server Model

– The server has to upload the file to all clients that are requesting.

UOZ–FS-CS

– The server has to upload the file to all clients that are requesting.

– The server bandwidth is the bottleneck when many concurrent applicants request.

– Would get congested and overload the server with too many requests.

– Lacks the robustness.

– since it has a single point of failure.

• Peer to Peer (P2P) Model• Peer to Peer (P2P) Model

– Offers more than a single source for files to be downloaded.

– Getting pieces from other peers increase while the number of concurrent peers increase.

– bandwidth is used efficiently.


Background

• BitTorrent Protocol

– One of many P2P file sharing prototypes in existence. e.g. Napster, Kazaa… etc.

UOZ–FS-CS

– One of many P2P file sharing prototypes in existence. e.g. Napster, Kazaa… etc.

– One of few P2P protocols that has managed to attract millions of users.

– One of the most common protocols for transferring large files.

– Its power comes from splitting the file into several smaller pieces.

– once a piece is obtained by a peer, it can be shared with other peers in the swarm.

– To download a file via BitTorrent, you need:

– Torrent file: a small metadata file with .torrent extension.

• Contains: information about files e.g. names, size, etc., and URL of a Tracker.

– Tracker: a navigation centre for the swarm and is responsible for helping clients to

find each other in their swarm.


Background

• Distributed hash table (DHT)

– A class of a decentralized distributed system that provides a lookup

UOZ–FS-CS

– A class of a decentralized distributed system that provides a lookup

service similar to a hash table;

– Usually most file sharing programs use a distributed hash table.

– The DHT Network is used to find IP addresses of peers present in a

swarm, instead of those provided by a tracker.

– DHT allows to search for peers using queries based on info hash and – DHT allows to search for peers using queries based on info hash and

requires no interaction whatsoever with the tracker(s) of that torrent.

– Search engines use DHT networks to look up

– what search terms are the most popular, and

– what different parts of the search engine most people use most frequently.


Background

• Peer Exchange (PEX)

– A communications protocol that augments the BitTorrent protocol.

UOZ–FS-CS

– A communications protocol that augments the BitTorrent protocol.

– It allows a group of peers that are collaborating to share a given file.

– The original design of the BitTorrent protocol, peers in a "swarm“ relied

upon a central computer server “tracker” to find each other.

– PEX greatly reduces the reliance of peers on a tracker by allowing each

peer to directly update others in the swarm as to which peers are

currently in the swarm.

– By reducing dependency on a centralized tracker, PEX increases the

speed, efficiency, and robustness of the BitTorrent protocol.


What is Magnet Link?

– According to the original BitTorrent design, .torrent files are

downloaded from torrent web sites (usually index sites).

UOZ–FS-CS

– Upon downloading the file, the BitTorrent client calculates a 20-byte

SHA-1 hash of the info key from the .torrent file which it uses in the

query made to the tracker to uniquely identify the torrent and find out

IP addresses of other peers sharing that torrent

– to which it will subsequently connect and download the contents referred in the

.torrent file.

– Magnet Links take that a step further, since they contain embedded as

a parameter, not the link to a .torrent file but instead, the info-hash a parameter, not the link to a .torrent file but instead, the info-hash

value already calculated for that specific torrent file.

– Therefore, by clicking on a Magnet Link your client gets the info-hash of

the torrent passed to it, which it further uses to query the DHT Network

and find other peers which share that torrent.


Background

• ‘.torrent ‘ files

– For years, BitTorrent clients, trackers and indexers have relied on .torrent files to

UOZ–FS-CS

– For years, BitTorrent clients, trackers and indexers have relied on .torrent files to

store information on the files shared with the popular p2p protocol.

– These files are stored by indexing sites and are used by BitTorrent clients to

connect to the tracker sites.

– The files hold several types of data, a URL of the tracker site, names for the files

it shared, as well as hash codes of files.

– All of this is used by the client to connect with peers that have the files in the torrent,

or portions of them, and also to ensure that the downloaded data is accurate.

– This system has several disadvantages, some technical, but one of the

biggest is that BitTorrent indexers have to store the .torrent files on

their servers, which leaves them vulnerable to legal threats if the

content shared happens to be infringing despite containing no actual

infringing data by themselves.Karwan Jacksi

22/04/2012

Magnet Links

– Magnet links though are just links, they have no files associated with

them just data.

UOZ–FS-CS

– The links are an evolving URI standard developed primarily to be used by

P2P networks.

– They differ from URLs in that they don't hold information on the

location of a resource but rather on the content of the file or files to

which they link.

– Technically, magnet links are made up of a series of parameters

containing various data in no particular order. containing various data in no particular order.

– In the case of BitTorrent :

– they hold the hash value of the torrent which is then used to locate copies of the files

among the peers.

– they may also hold file name data or links to trackers used by the torrent.


Magnet Links

– With magnet links, BitTorrent indexers don't have to store any file at all,

just a few snippets of data leaving the individual client apps to do all the

UOZ–FS-CS

heavy lifting.

– Magnet links can be copy-pasted as plain text by users and shared via

email, IM or any other medium.

– For the indexer sites, the allure is clear, using magnet links makes it

harder for them to be accused of any wrong-doing in court.

– Theoretically, magnet links should not have any disadvantages for the

users over .torrent files either. users over .torrent files either.

– It would also potentially make downloads faster as it would enable the

clients to download from peers which have identical files but with

different names.


Magnet Links

– In practice though, since the technology is still being actively developed,

some kinks still creep up.

UOZ–FS-CS

– Up until very recently, many of the major BitTorrent clients didn't support

magnet links at all.

– After the Pirate Bay introduced them, this is no longer a problem, but there

are still things to work out.

– Indexer sites haven't agreed on a single link format, so it’s up to the

clients to support the various implementations.


Magnet Links

– And for the users, the experience isn't on par with using plain .torrent

files yet.

UOZ–FS-CS

– For example, magnet links on the Pirate Bay don't have any additional data

on the torrent other than its content so when the link is opened in uTorrent,

for example, the torrent won't have a name or list the files in it.

– This leads to a second problem, without knowing the contents of the

torrent, uTorrent starts downloading it directly in the default location,

preventing users from selecting a custom location or selecting just some files

in a multiple-file torrent.

– These are likely to be just temporary set-backs, the recently-launched – These are likely to be just temporary set-backs, the recently-launched

TorrIndex, the world's first magnet link-only BitTorrent indexer, is listing

links which have additional information like tracker URLs and the torrent's

name.

– And with broader support from BitTorrent clients and indexers, magnet links

will eventually replace .torrent files sooner than you might expect.


Magnet Links

– Magnet links don't require a tracker (since it uses DHT), nor does it

require you to download a separate file before starting the download,

which is convenient.

UOZ–FS-CS

which is convenient.

– The main reason torrent sites are moving toward magnet links—apart

from convenience to the user—is that these links (probably) free torrent

sites like The Pirate Bay from legal trouble. Since The Pirate Bay won't

be hosting files that link to copyrighted content—that is, the torrent

files—it's more difficult to claim the site is directly enabling the

downloading of copyrighted material.


Magnet Links

• History

– The standard was developed in 2002, partly as a "vendor- and project-neutral

UOZ–FS-CS

– The standard was developed in 2002, partly as a "vendor- and project-neutral

generalization" of the ed2k: and freenet: URI schemes used

by eDonkey2000 and Freenet, respectively, and attempts to follow

official (Internet Engineering Tast Force) IETF URI standards as closely as

possible.

– Applications supporting magnet links include

μTorrent, aMule, BitComet, BitSpirit, BitTorrent, DC++, Deluge, FrostWire, gtkg

nutella, I2P, KTorrent, MLDonkey, Morpheus, Qbittorrent, rTorrent,Shareaza, Tra

nsmission and Vuze.nsmission and Vuze.


Magnet Links

• Use of content hashes

– The most common use of magnet links is to link to a particular file based

UOZ–FS-CS

– The most common use of magnet links is to link to a particular file based

on a hash of its contents, producing a unique identifier for the file,

similar to an ISBN or catalog number.

– Unlike traditional identifiers, however, content-based signatures can be

generated by anyone who already has the file, and so do not need a

central authority to issue them.

– This makes them popular for use as "guaranteed" search terms within

the file sharing community where anyone can distribute a magnet link to the file sharing community where anyone can distribute a magnet link to

ensure that the resource retrieved by that link is the one intended,

regardless of how it is retrieved.


Magnet Links

• Use of content hashes

– While it is theoretically possible that two files could have the same hash

UOZ–FS-CS

– While it is theoretically possible that two files could have the same hash

value (known as a "hash collision"), cryptographic hash functions are

designed so that the probability of this event is a practical impossibility

– even if an expert is intentionally looking to find two files with the same hash value.

– Another advantage of magnet links is their open nature and platform

independence:

– the same magnet link can be used to download a resource from one of any number of

applications on almost any operating system. applications on almost any operating system.

– Because magnets are concise and plain-text, it is possible for users to

simply copy-and-paste the links into emails or instant messages, a

property not found in, for example,BitTorrent files.


Magnet Links

• Technical description

– Magnet links consist of a series of one or more parameters, the order of

UOZ–FS-CS

– Magnet links consist of a series of one or more parameters, the order of

which is not significant, formatted in the same way as the query

string on the end of many HTTP URLs.

– The most common parameter is "xt", meaning "exact topic", which is

generally a URN formed from the content hash of a particular file, e.g..

magnet:?xt=urn:sha1:YNCKHTQCWBTRNJIV4WNAE52SJUQCZO5C

referring to the Base32 encoded SHA-1 hash of the file in question.

– Note that although this refers to a particular file, a search must still be

carried out by the client application to determine where, if anywhere, it

can obtain that file.


Magnet Links

• Technical description

– Other parameters defined by the draft standard are:

UOZ–FS-CS

– Other parameters defined by the draft standard are:

– "dn" ("display name"): a filename to display to the user, for convenience

– "kt" ("keyword topic"): a more general search, specifying search terms

rather than a particular file

– "mt" ("manifest topic"): a URI pointing to a "manifest", e.g. a list of

further items

– The standard also suggests that multiple parameters of the same type – The standard also suggests that multiple parameters of the same type

can be used by appending ".1", ".2", etc. to the parameter name, e.g.:

magnet:?xt.1=urn:sha1:YNCKHTQCWBTRNJIV4WNAE52SJUQCZO5C&xt.2=urn:sha1:T

XGCZQTH26NL6OUQAJJPFALHG2LTGBC7


Magnet Links

• The Pirate Bay

– The world's largest BitTorrent tracker is shutting down!.

UOZ–FS-CS

– The world's largest BitTorrent tracker is shutting down!.

– As of January 2012, The Pirate Bay has switched to magnet links as the

default option and may use magnet links exclusively eventually.

– On February 28, 2012, The Pirate Bay started using magnet links

entirely.

– It has decided that there is no need to run a tracker anymore, so it will

remain down! It's the end of an era, but the era is no longer up to date.

– We have put a server in a museum already, and now the tracking can be

put there as well,” the Pirate Bay announced.


Magnet Links

• The Pirate Bay

– Recently though, technologies like Distributed Hash Table (DHT) and

UOZ–FS-CS

– Recently though, technologies like Distributed Hash Table (DHT) and

Peer Exchange (PEX) have rendered trackers useless as they are able to

find peers without the need for a tracker server. This makes the whole

system a lot more stable and resilient to technical problems, but

perhaps even more important, it makes it a lot harder to be attacked by

anti-piracy organizations.

– Instead, the Pirate Bay, now feature a magnet link, allowing users to get

access to a torrent without the need to download any file making the access to a torrent without the need to download any file making the

sites even less susceptible to legal threats.


Magnet Links

• Advantages

– They do not need a central authority to issue.

UOZ–FS-CS

– They do not need a central authority to issue.

– They has open nature and platform independence, the same magnet link can be

used as long as the system has the appropriate application.

– They are more user based, easy to use.

– All the system need is an application that support magnet links.

– Since it is user based, it is so easy to share resources.

– Most Magnet links application has a search function.

Disadvantages:• Disadvantages:– Slower speed.

– Less control on speed and on the contend that is being downloaded


Torrents

• Advantages

• Faster connection.

UOZ–FS-CS

• Faster connection.

• Easier to search through the web.

• Disadvantages:

– Trackers are needed when downloading a contend. If the tracker is

down and there are no existing connections, the download may never be

finished.

– Most torrent client do not have search function. Torrents usually would – Most torrent client do not have search function. Torrents usually would

be find from the internet.

– If a torrent was stored on web for long time, the tracker may be expired

already. It is almost impossible to find existing seed or leechers.


References

• Further Development of BitTorrent Simulator in Erlang (Karwan Jacksi)

• Is P2P dying or just hiding? (Thomas Karagiannis UC Riverside [email protected])

UOZ–FS-CS

• Is P2P dying or just hiding? (Thomas Karagiannis UC Riverside [email protected])

• Distributed algorithms for improving BitTorrent performance (ANIL CAN AKAY)

• Incentives Build Robustness in BitTorrent (Bram Cohen)

• http://lifehacker.com/5411311/bittorrents-future-dht-pex-and-magnet-links-explained

• http://en.wikipedia.org/wiki/Distributed_hash_table

• http://en.wikipedia.org/wiki/Peer_exchange• http://en.wikipedia.org/wiki/Peer_exchange

• http://en.wikipedia.org/wiki/The_Pirate_Bay

• http://en.wikipedia.org/wiki/Secure_Hash_Algorithm


Questions?

• Thanks for coming ☺

UOZ–FS-CS


Magnet links

Technology

Transcript of Magnet links