Applications

CPSC 363 Computer Networks

Ellen Walker

Hiram College

(Includes figures from Computer Networking by Kurose & Ross, © Addison Wesley 2002)

Review of Layers

• Layer 5: Application (messages)– Implemented on hosts

• Layer 4: Transport (segments)– Connection-oriented service (connects hosts)

• Layer 3: Network (datagrams / packets)– Routing service (packets between hosts)

• Layer 2: Link (frames)• Layer 1: Physical (bits - electric or light)

– Layers 2 & 1 in network card. Accurate transmission of bit sequences across physical links (wires, cable, radio, etc.).

Application Layer point of view

• Client requests/establishes a connection with server (via the network layer)

• Client sends and receives messages across the connection

• Client (or server) closes the connection (again via the network layer)

• A host can maintain multiple connections at the same time!– Multiple ports– Multiple processes

Application vs. Protocol

• Network application can consist of many components– Data storage & retrieval– Data formatting & presentation– Encoding / decoding encrypted data– Messages between client & server (or peers)

• Only the last of these uses network application protocol

Not all standards are protocols

• Web services– Browser-server communication (HTTP)– Document formatting standard (HTML or XML or

PDF or JPG or …)

• Email services– Client-server communication (SMTP, IMAP, POP)– Message attachment formatting (MIME)

Protocols define…

• What types of messages are exchanged?• How is the message formatted into fields?

(Syntax)• What do the fields mean? (Semantics)• What are legal messages and responses to

messages? (Rules)

Finding Internet Protocols

• Internet protocols are defined in RFC (Request for Comments) documents

• A searchable database of these documents is available at http://www.rfc-editor.org

• Not all RFC documents are standards; some are informational, experimental, etc.

Client and Server

• Client– Where the “user” sits, usually– Initiates the conversation (in nearly all cases)

• Server– Responds to client’s requests– Provides “services” such as file storage, database

query, etc.

• Peer to Peer– Both hosts play both roles

Process Communication

• A process is a running program with its own program counter, registers and memory

• Most modern operating systems run many processes simultaneously

• Processes communicate with each other through sockets (also called APIs)– Limited interaction with network layer– Establish type of transport protocol and a few

parameters (e.g. segment size) only

Message Addressing

• Which host is this message going to?– IP address (e.g. 143.206.149.21)– We’ll discuss in detail later (network layer)

• Which process on that host will receive the message?– Port number (e.g. 80 for HTTP)– Standard port numbers have been assigned (see

http://www.iana.org/assignments/port-numbers)– We’ll discuss ports in detail later (transport layer)

Applications’ Demands on Transport Layer

• Reliable Data Transfer– How much loss is acceptable?– None (e.g. financial applications) vs. some (e.g multimedia)

• Bandwidth– What transmission rate is necessary?– Minimum requirements (e.g. streaming video) vs. use

whatever is available (e.g. web, file transfer)

• Timing– What end-to-end delay is acceptable?– Some applications (e.g. telephony) have strict constraints

Transport Layer Services

• TCP– Connection-oriented (handshake)– Reliable transmission– Congestion control (throttling)– No guaranteed minimum transmission rate

• UDP– Connectionless (no handshake)– Unreliable (no guarantee of receipt or ordering)– No guaranteed minimum transmission rate

Examples (fig. 2.5)Application App-Layer

ProtocolTransport Protocol

E-mail SMTP TCP

Remote terminal

Telnet TCP

Web HTTP TCP

File Transfer FTP TCP

Streaming MM Proprietary UDP or TCP

Internet Telephony

Proprietary UDP

Telnet

• Perhaps the simplest protocol• Opens a TCP connection between two hosts through

a specified port• Whatever you type is sent through the connection• Typically used for terminal connection (now

superseded by SSH for secure connections)• Can telnet to any port

– telnet cs.hiram.edu 23 (terminal connection; default)– telnet cs.hiram.edu 80 (web server connection)

The WorldWide Web

• Basis in hyperlinks and hypertext (documents)– Proposed by Vannevar Bush (Memex) 1945– “Hypertext” coined by Ted Nelson 1965– Hypertext in education at Brown (FRESS) 1966 - 198x?– Hypercard (Apple) 1987– See http://ei.cs.vt.edu/book/chap1/htx_hist.html

• Historical path…– FTP (file transfer protocol - files aren’t displayed)– Gopher (displays directories & text files)– Web (embedded links can link to any document)

• Search engines• Multimedia indexes• Front ends to databases• ETC!

Web Application Vocabulary

• Web page (document) - collection of objects– Usually base HTML file + several referenced

objects

• Object - any file addressable by a single URL• URL (Uniform Resource Locator) - how to

reach an object– Host address + object’s path name

• Browser - user agent (client) for the web• Web server - houses objects

HyperText Transfer Protocol (HTTP)

• Request / response protocol– Client requests an object (URL)– Server provides the object(s) corresponding to that URL– Non-persistent (1 object only) vs persistent (explicit close)

• Stateless protocol– The server doesn’t store any knowledge (state) of the client– E.g. server doesn’t remember what pages client looked at

HTTP Uses TCP

• Client initiates TCP connection to server, port 80

• Server accepts connection• Client sends HTTP message & Server

responds (one or more times)• TCP connection closed

Non-persistent HTTP conversation

• Client to server: “[address] I’d like to talk to you”• Server to client: “OK, I will talk with you”• Client to server: “Thanks. Please send me [path]”• Server to Client: “Here’s the object you asked for.

[Object] Goodbye.”• Client to server: “Got it. Goodbye”

• If the object contains embedded links, an additional complete conversation is needed for each!

How long does it take?

• Define RTT (Round Trip Time) as time for one message to travel from client to server & server to client (includes all delays)

• Total time is 2*RTT+ file transmission– Beginning of handshake (1 RTT)– End of handshake + transmit request + first packet

of response (1 RTT)– Transmit the rest of the file (depends on file size)

• If file has 10 images, 22*RTT + file times

Persistent Transmission

• TCP connection remains open until explicitly closed.

• Previous example now takes 2 RTT for setup, plus 1 RTT per request, plus file time (12RTT+ file time)

• With pipelining, new requests are sent as files are received, so server is never idle. Only 1RTT for setup, plus 1 RTT for all objects. Example now takes 2RTT + file time.

Message Format (Request)

• Request line (command, addr, protocol)GET /~walkerel/cs363/index.html HTTP/1.1

• Header lines (fields & values)Host: cs.hiram.eduConnection: close nonpersistent

User-agent: Mozilla/4.0 browser idAccept-language: en preferred lang.

• Entity body (for POST) contains contents of forms filled out

• End with 2 CR/LFs

HTTP Commands

• GET path– Get a file

• GET path?var=val…– Get a file, specify values (from form)

• POST path– Run a program that resides at the specified path– Program will generate a web page, which is the

server’s response

Additional Commands

• HEAD– Requests header lines but not the actual file

• PUT (1.1 only)– Uploads file to path specified in URL field

• DELETE (1.1 only)– Deletes file specified by URL field

Message Format (Response)

• Status line (protocol, status code, message)HTTP/1.1 200 OK

• Header linesConnection: close

Date: Sat, 25 Jan 2003 12:15:00 EDT

Server: Apache/1.3.0 (Unix)

Last-Modified: …

Content-Length: …

Content-Type: text/html

• Data

Status Codes

• 200 OK• 301 Moved Permanently (new URL provided

in Location: header)• 400 Bad Request (generic error message)• 404 Not Found• 505 HTTP Version Not Supported (on this

server)

Practice HTTP

• Telnet to your favorite server, e.g. cs.hiram.edu, using port 80telnet cs.hiram.edu 80

• Enter HTTP message, followed by a blank lineGET /~walkerel/cs363/index.html

Host: cs.hiram.edu

Cookies

• Information stored by the client to identify the client to the server

• The “cookie” is a unique identification number for the user. (e.g. to index purchase history)

• It is stored in a “cookie file” on the client machine, and provided to the server as part of a request message

• The server can then create “personalized” responses• Cookies can also authenticate, so you can “save your

password”

Cookies in HTTP

• In the HTTP Response message (from the server)– Set-cookie: 1253261

• In future HTTP Request messages (from the client)– Cookie: 1253261

Web Cache (Proxy Server)

• Stores copies of recently requested items• Browser first requests item from Proxy Server

– If item is stored, it is sent– Otherwise, item is retrieved from external server, stored,

then sent

• Proxy Server acts as both client and server• Proxy server can also refuse requests

– Prevent browsing to “inappropriate” sites

• Risk: page has changed since saved (stale page)

Advantage of Web Cache

• Increases average response time– Response time of “hit” is very fast (item in cache)– Response time across network much slower than

LAN response time– Hit rates 0.2 - 0.7 in practice (20%-70% of

accesses are repeats)

• Average response time = – Hit rate * LAN delay + (1-Hit rate) * net delay

Example: Cache Advantage

• Assumptions (Section 2.2.6)– LAN delay = 0.01 sec– Net delay = 2.01 sec– Hit rate = 40% (0.4)

• No cache– 2.01 seconds delay

• With cache– .4 * 0.01 + .6 * 2.01 = 0.004+ 01.206 = – 1.21 seconds delay

Protocol for Avoiding Stale Pages

• Server requests page only if changed (Conditional GET)GET file HTTP/1.1Host: IpaddressIf-modified-since: date

• Response if not changed:HTTP/1.1 304 Not ModifiedDate: dateServer: server

• Response if changed is the same as beforeHTTP:/1.1 200 OKAdditional headers + data

File Transfer (FTP)

• Send a file from one host to another– User can sit on “donor” or “recipient” host

• User provides authentication information once for all transfers– Username & password, or ‘anonymous’ & email

address

• Connection is persistent until an explicit close

• Example: ftp pub/reid.txt from rtfm.mit.edu

FTP Uses 2 Connections

• Control connection – Sends user id, password, commands– “Out of band” because not interspersed with data– Port 21 (TCP)

• Data connection (TCP)– Sends actual files– A new data connection is created for each file

Unlike HTTP, State is maintained

• Server remembers which user is connected– vs. HTTP Authorization header in every message

• Server remembers current directory– vs. HTTP full path in every message

• Because state is maintained, the number of simultaneous connection is limited, relative to HTTP

FTP Commands

• USER username• PASS password• LIST (list the files in the current directory)• RETR filename (retrieve from remote host)• STOR filename (store onto remote host)

• Client commands aren’t quite identical (eg. GET, PUT) and may allow additional arguments

Electronic Mail

• User Agent– Allows user to send and receive email– Generally allows access to stored email– e.g. MS Outlook, Eudora

• Mail Server– Delivers email, stores it in user’s mailbox (at least)

until read– Sends off-site email; queues and retries if external

host isn’t available

SMTP (Simple Mail Transfer Protocol)

• All messages (not just headers) restricted to 7-bit ASCII (must be encoded/decoded by user agent)

• Transfers mail from origin host to destination host (no intermediate servers)

• Commands include HELO, MAIL FROM, RCPT TO, and DATA

• To try it: telnet serverName 25

SMTP vs. HTTP

• HTTP is “pull protocol”, SMTP is “push protocol”

• SMTP requires 7-bit ASCII, even for data; HTTP allows any format

• SMTP puts all data into one message– MIME encoding (Multipurpose Internet Mail

Extensions)• Content-Type: and Content-Transfer-Encoding:

headers

MIME Types

• Multipart/mixed– Look for part boundaries, content headers

• Text/plain• Text/html• Image/gif or image/jpeg• Application/msword, application/pdf, etc.

Mail Headers

• Some from user (e.g. To, cc)• Some from user agent (e.g. Date, From,

MIME headings)• Some from servers (e.g. Received)

• Most user agents allow “full headers” to be viewed.

Mail Header Example

• From: Thomas Bagley bagley_rep@acm.org• Subject: ACM Member Technical Interest Service January 2010• Date: January 26, 2010 5:46:47 PM EST• To: Ellen Walker walkerel@hiram.edu• Received: from mail.hiram.edu ([206.57.41.42]) by hiramr.hiram.edu with

Microsoft SMTPSVC(6.0.3790.3959); Tue, 26 Jan 2010 17:46:54 -0500• Received: from smtp161.redcondor.net ([206.57.41.40]) by mail.hiram.edu with

Microsoft SMTPSVC(6.0.3790.3959); Tue, 26 Jan 2010 17:46:54 -0500• Received: from acm26-2.acm.org ([199.222.69.107]) by smtp161.redcondor.net

({6c7b74fb-260e-4729-9476-f743470f315e}) via TCP (inbound) with ESMTP id 20100126224651153 for <walkerel@hiram.edu>; Tue, 26 Jan 2010 22:46:51 +0000

• Received: from acm28-8 ([192.168.1.104]) by acm26-2.acm.org (IceWarp 9.4.2) with SMTP id IXU98141 for <walkerel@hiram.edu>;

• Tue, 26 Jan 2010 17:46:41 -0500

Mail Header Example (Cont’d)

• X-Rc-From: bagley_rep@acm.org

• X-Rc-Rcpt: walkerel@hiram.edu

• Message-Id: 10171514.34121264546007418.JavaMail.Administrator@acm28-8

• Mime-Version: 1.0

• Content-Type: text/plain; charset=UTF-8

• Content-Transfer-Encoding: 7bit

• X-Mailer: ColdFusion 8 Application Server

• Return-Path: bagley_rep@acm.org

• X-Originalarrivaltime: 26 Jan 2010 22:46:54.0837 (UTC) FILETIME=[7568BA50:01CA9ED9]

Mail Access Protocols

• Protocols for conversation between user agent and mail server (SMTP is for mail server to mail server communication)

• POP3– Authorization, transaction, update (after client quit)

• IMAP– Allows users to store mail in folders on server– Clients can access message components (e.g. headers

only)

• WebMail (HTTP)– Mail accessed through a web page using a browser; no

application-specific client or protocol

Domain Name System (DNS)

• Translates between mnemonic hostnames and numeric IP addresses– www.hiram.edu = 206.57.41.47– Command “host” can look up an address

• Distributed database implemented in hierarchy of DNS servers

• Application-layer protocol that allows hosts to query this database– Used by other application-layer protocols such as HTTP for

name-address translation– This adds a delay to each HTTP request

Database is Distributed• Root servers (13) point to…• Top-level domain servers (com, org, edu, uk,…) point to…• Authoritative servers (per organization)• Local servers (per ISP)

b USC-ISI Marina del Rey, CAl ICANN Los Angeles, CA

i Autonomica, Stockholm (plus 28 other locations)

k RIPE London (also 16 other locations)

m WIDE Tokyo (also Seoul, Paris, SF)

a Verisign, Dulles, VAc Cogent, Herndon, VA (also LA)d U Maryland College Park, MDg US DoD Vienna, VAh ARL Aberdeen, MDj Verisign, ( 21 locations)

Translating a Domain (iterative)

• Ask local DNS• Local DNS asks root DNS• Root DNS responds with appropriate top-level

DNS• Local DNS asks top-level DNS • TL DNS responds with appropriate

organization’s authoritative DNS• Local DNS asks Authoritative DNS, and

receives address (which it probably caches)

Translating a Domain (recursive)

• Ask local DNS• Local DNS asks root DNS• Root DNS asks top-level DNS• TL DNS asks organization’s authoritative

DNS• Organization’s DNS responds to TL DNS,

which forwads to Root DNS, which forwards to local DNS, which responds to original client

• (Caching as appropriate throughout)

DNS Record Types

• A– Name = hostname, value = IP address

• NS– Name = domain (hiram.edu), value = hostname of

authoritative DNS

• CNAME– Name = canonical hostname (foo.com), value = real

hostname (relay1.bar.foo.com)

• MX– Name = canonical mail name (gmail.com), value = real

hostname (gsmtp183.google.com)

Creating a new DNS Record

• Entity “registers” the site, by paying a registrar and providing authoritative DNS server IP addresses

• Registrar verifies uniqueness of name and enters NS and A type records into database

• You provide A and MX records for your own servers at your authoritative DNS servers

Why Not Central Name Server?

• Single point of failure• Too many requests• Does not scale !!!!!

P2P (Peer to Peer) Applications

• All content transferred directly between peers without passing through third-party servers– Does not rely on always-on (24/7) servers

• When a client requests an object– Find a server, currently connected that has that object– Server transmits object to client

• A given host can be both client and (transient) server• Applications include:

– File distribution– VOIP (e.g. Skype)

File Sharing

• Goal: Distribute a file from a single server to many hosts

• Client-server– Huge burden on source host (must connect to all

recipients)

• P2P– Any host that has received some portion of the file

can redistribute to others (sharing the burden)– Most popular (2009): BitTorrent

How long does it take? (C/S)

• N bits in file, u = upload rate, dmin = min download rate

• 1 server -> N clients• Upload time = N*F/u• Max download time = F / dmin• Overall: max (N*F/u, F/dmin)• Assume N is big enough, result is N*F/u

How Long Does It Take? (P2P)

• To get file out, server must send every bit once (F/u)

• Slowest recipient will take F/dmin time to get its complete file

• Total upload capacity is sum of upload capacity of all uploaders: – uTotal = u + u1 + u2…

• Overall time = max (F/u, F/dmin, N*F/uTotal)• When N gets bigger, so does uTotal!

BitTorrent

• P2P Protocol for file distribution• Torrent = collection of all peers involved in

distribution of a single file• Chunks = equal-sized pieces of file (e.g.

256K)• Tracker = special infrastructure node (one

tracker per torrent)

When a peer joins a torrent…

• Tracker sends peer N (say 50) random addresses of hosts in the torrent

• Try to connect to all of them. (Successful connections called ‘neighbors’)

• Ask each neighbor for lists of chunks they have

• Request (from appropriate neighbor) each chunk you don’t have

Rarest First

• Ask for the chunk which is held by the fewest of my neighbors

• Result: more copies of that chunk, roughly equalizing the availability of chunks

Responding to Requests (Tit for Tat)

• Host receives many requests• Respond to requests from 4 neighbors

sending bits at highest rate (unchoked)– These are fastest– These are ‘most generous’

• Respond to requests from a fifth neighbor at random (optimistically unchoked)– Might become ‘top-4’ of this neighbor!– Allows more neighbors to get in on the action.

Distributed Hash Tables

• Need to maintain searchable index of (key, value) pairs

• Cannot contain it on a single host (point of failure) – Napster did this in the early days of P2P

• Distribute pairs among hosts– How to avoid having all hosts contain all pairs?– How to avoid having all hosts contact all hosts?

Answer: Use the Hash Value!

• Assign an integer to each host (same range as hash values)

• Assign each (key, value) pair to the host whose integer is closest to hash(key)– Equal is closest, then successor (wraps around)

• Each host knows its successor (last -> first) – circular overlay network

To Add or Find

• Peer receives message with hash(key)• If ID is closer to hash(key) than successor’s

ID, then peer responds directly to message’s sender– ID ≥ hash(key)– Successor ID < hash(key)

• Else peer passes on the message to its successor

Evaluating Circular Network

• Advantage:– Every peer needs to keep track of only 2

neighbors (predecessor and successor)

• Disadvantage:– When the circle gets big, messages take a long

time to go around!

• Solution:– Add a few “shortcut” links across the circle– Trades off more neighbors vs. shorter travel time

Peer Churn

• Remember, hosts come & go, not 24/7• By the original plan, if my successor is lost, I

am disconnected!• Instead

– Each node tracks 2 successors– Periodically check both your successors are there– If one is gone, find the other’s successor so you

still have 2 successors

• To join, pass a message around the circle

From Kurose & Ross Slides v. 5 67

P2P Case study: Skype

• inherently P2P: pairs of users communicate.

• proprietary application-layer protocol (inferred via reverse engineering)

• hierarchical overlay with SNs

• Index maps usernames to IP addresses; distributed over SNs

Skype clients (SC)

Supernode (SN)

Skype login server