The Application Layer – HTTP and FTP

Tahir Azim

Application Layer Protocols

• QoS lectures postponed to next week

• This week: Application Layer Protocols– FTP, HTTP, Bittorrent

Materials reused from Philip Levis, David Mazieres (Stanford) and Vern Paxson (Berkeley)

File Transfer Protocol (FTP)

• Allows a user to copy files to/from remote hosts– Client program connects to FTP server– … provides a login id and password– … allows the user to explore the directories– … and download and upload files with the server

• A predecessor of the Web (RFC 959 in 1985)– Requires user to know the name of the server machine– … and have an account on the machine– … and find the directory where the files are stored– … and know whether the file is text or binary– … and know what tool to run to render and edit the file

• That is, no URL, hypertext, and helper applications

How Do You Transfer Files Today?

• HTTP - the usual Web transfer mechanism (http://)

• FTP – ftp:// links in web pages (e.g.: in www.kernel.org)

• sftp and scp– E.g.: to upload your project files to cms.niit.edu.pk

• BitTorrent and other file-sharing software• Any others?

Why Study FTP?

• Helps cement familiarity with text/status-code based protocols

• Illustrates use of multiple concurrent connections– One for control (commands & replies)– Depending on command, can be additional one for

data

• Illustrates reversal of roles– For data connection, FTP user’s process can play the

server role, FTP server can play the client role

Example commands

• Authentication– USER: specify the user name to log in as– PASS: specify the user’s password

• Exploring the files– LIST: list the files for the given file specification– CWD: change to the given directory

• Downloading and uploading files– TYPE: set type to ASCII (A) or binary image (I)– RETR: retrieve the given file– STOR: upload the given file

• Closing the connection– QUIT: close the FTP connection

Server Response Codes

• 1xx: positive preliminary reply– The action is being started, but expect another reply before

sending the next command.

• 2xx: positive completion reply– The action succeeded and a new command can be sent.

• 3xx: positive intermediate reply– The command was accepted but another command is now

required.

• 4xx: transient negative completion reply– The command failed and should be retried later.

• 5xx: permanent negative completion reply– The command failed and should not be retried.

FTP Data Transfer

• Separate data connection– To send lists of files (LIST)– To retrieve a file (RETR)– To upload a file (STOR)

control

data

Creating the Data Connection

• Client acts like a server– Creates a socket

• Assigned an ephemeral port number by the kernel

– Listens on socket– Waits to hear from FTP server

socket

Creating Data Connection (cont.)

• But, the server doesn’t know the port number– So after starting to listen, client tells it to the server

– Using the PORT command on the control connection

– Server can tell the client a port to connect to using PASV or EPSV

PORT <IP address, port #>

Why Out-of-Band Control?

• Avoids need to mark the end of the data transfer– Data transfer ends by closing of data connection– Yet, the control connection stays up

• Aborting a data transfer– Can abort a transfer without killing the control connection– … which avoids requiring the user to log in again– Done with an ABOR on the control connection

• Third-party file transfer between two hosts– Data connection could go to a different host– … by sending a different client IP address to the server– e.g., a user can coordinate a transfer between two servers– But: this is rarely needed, and presents security issues

Example

• Collected using:– tcpdump -s 0 -w ftp.trace host www.niit.edu.pk– ftp www.niit.edu.pk– Ethereal

Example

blade1:~ # ftp www.niit.edu.pkConnected to www.niit.edu.pk.

Server sends back the following:

220---------- Welcome to Pure-FTPd [privsep] [TLS] ----------220-You are user number 1 of 50 allowed.220-Local time is now 18:54. Server port: 21.220-This is a private system - No anonymous login220 You will be disconnected after 15 minutes of inactivity.

Example continued…Name (www.niit.edu.pk:root): tahir

USER tahir331 User tahir OK. Password required

Password:

PASS tahirazim

230-User tahir has group access to: tahir

230 OK. Current restricted directory is /SYST215 UNIX Type: L8FEAT211-Extensions supported: EPRT IDLE MDTM SIZE REST STREAM MLST type*;size*;sizd*;modify*;UNIX.mode*;UNIX.uid*;UNIX.gid*;unique*; MLSD ESTP PASV EPSV SPSV ESTA AUTH TLS PBSZ PROT211 End.

Example continued…

• ftp> pwdPWD257 "/" is your current location

• ftp> cd public_html/tinyosCWD public_html/tinyos250 OK. Current directory is

/public_html/tinyos

Example continued…• ftp> lsEPSV229 Extended Passive mode OK (|||22710|)LIST150 Accepted data connection226-Options: -l 226 5 matches total

Received from port 22710 of the server:-rw-r--r-- 1 tahir tahir 1143243 Feb 1 14:23 BVR-T2-0.1.tar.gz-rw-r--r-- 1 tahir tahir 535 Jan 18 20:03 Makefile-rw-r--r-- 1 tahir tahir 2788 Jan 18 20:03 RadioCountToLedsAppC.nc-rw-r--r-- 1 tahir tahir 5219 Jan 18 20:03 RadioCountToLedsC.nc-rw-r--r-- 1 tahir tahir 429056 Jan 18 20:03 nesC-wksp.ppt

Example continued…• ftp> get MakefileTYPE I200 TYPE is now 8-bit binarySIZE Makefile213 535EPSV229 Extended Passive mode OK (|||9442|)RETR Makefile150 Accepted data connection226-File successfully transferred226 0.000 seconds (measured here), 1.11 Mbytes per secondMDTM Makefile213 20080118150308

From port 9442 of the server:COMPONENT=RadioCountToLedsAppCBUILD_EXTRA_DEPS = RadioCountMsg.py RadioCountMsg.class

RadioCountMsg.py: RadioCountToLeds.h.mig python -target=$(PLATFORM) $(CFLAGS) -python-classname=RadioCountMsg RadioCountToLeds.h radio_count_msg -o $@

RadioCountMsg.class: RadioCountMsg.java.javac RadioCountMsg.java

RadioCountMsg.java: RadioCountToLeds.h.mig java -target=$(PLATFORM) $(CFLAGS) -java-classname=RadioCountMsg RadioCountToLeds.h radio_count_msg -o $@

#CFLAGS += -I/home/Tahir/T2SerialT1.include $(MAKERULES)

Example continued…

• ftp> exit

QUIT

221-Goodbye. You uploaded 0 and downloaded 1 kbytes.

221 Logout.

HTTP

• Server listens on a port (by default, 80)

• On connection, waits for a request

• Protocol (but not data) is in ASCII

• Sends response, maybe closes connection (client can ask it to stay open)

Parsing a URL

http://www.niit.edu.pk/~tahir/tcpip/index.html

Protocol Host File path on host

HTTP Request Format

• Request types: GET, PUT, POST, HEAD, DELETE

• A trivial browser request: http://localhost:8000

A trivial browser request

GET / HTTP/1.1Host: localhost:8000User-Agent: Mozilla/5.0 (Macinto ...Accept: text/xml,application/xm ...Accept-Language: en-us,en;q=0.5Accept-Encoding: gzip,deflateAccept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7Keep-Alive: 300Connection: keep-alive

Other useful header fields

• Range: Request a partial range of data

• Authorization: Present authorization credentials to a server (not HTTPS)

• Proxy-Authorization: Present proxy credentials to a proxy server

• Referer: URL of the web page the user was on, when the HTTP request was made

HTTP Response Format

• 1xx codes: Informational• 2xx codes: Successes• 3xx codes: Redirection• 4xx codes: Client Error, 5xx codes: Server Error

Response from thenews.com.pkHTTP/1.0 200 OKDate: Fri, 14 Mar 2008 10:13:05 GMTServer: Microsoft-IIS/6.0X-Powered-By: ASP.NETContent-Length: 54063Content-Type: text/htmlSet-Cookie: ASPSESSIONIDAADTQQAR=PFKAADCBBDGBDDJLMKLFJNMI; path=/Cache-Control: privateX-Cache: MISS from micronet-proxy.niit.edu.pkX-Cache-Lookup: MISS from micronet-proxy.niit.edu.pk:8080X-Cache: MISS from proxy-ptcl.niit.edu.pkX-Cache-Lookup: MISS from proxy-ptcl.niit.edu.pk:8080Via: 1.0 micronet-proxy.niit.edu.pk:8080 (squid/2.6.STABLE5), 1.0 proxy-

ptcl.niit.edu.pk:8080 (squid/2.6.STABLE5)Connection: keep-alive

<html><head>

<title>The News - International - Friday, March 14, 2008</title> …

HTTP Performance

• What matters most?

• Different kinds of requests– Lots of small requests (loading a web page)– Big request (fetching a download)

• Require different solutions

Small requests

• Latency matters• Governed by RTT between hosts• Two major causes of delay:

– Opening a TCP connection– Data response-request

• Solutions: – Persistent connections– Pre-fetching– Others??

Browser request, revisited

GET / HTTP/1.1Host: localhost:8000User-Agent: Mozilla/5.0 (Macinto ...Accept: text/xml,application/xm ...Accept-Language: en-us,en;q=0.5Accept-Encoding: gzip,deflateAccept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7Keep-Alive: 300Connection: keep-alive

Big requests

• Problem is throughput on bottleneck links (usually edge links)

• Use an HTTP proxy cache or mirror– Can also improve latency!

Stale caches

• Items in the cache can go stale (you don’t want to read yesterday’s paper)

• Cache needs a way to conditionally ask for a document• Cache can issue a conditional GET (with an If-modified-since header)– Server can reply with a 304 Not Modified

GET / HTTP/1.1Host: www.niit.edu.pkIf-modified-since: Wed, 2 April 2008 08:00:00

Client-server vs Peer-to-peer

• Server can be a bottleneck– Download time can scale down O(n) with n clients– Scaling up server bandwidth can be expensive

(CDNs)– Slashdotting/flash crowds

• Peer-to-peer: get a bunch of end-hosts to collaboratively distribute content

• A common peer-to-peer challenge is finding whom to collaborate with

Bittorrent