Lecture07 - UHjohnsson/COSC4377_2013/Lecture07.pdfprogramming with UDP and TCP Slide ... FTP client...
Transcript of Lecture07 - UHjohnsson/COSC4377_2013/Lecture07.pdfprogramming with UDP and TCP Slide ... FTP client...
9/21/2013
1
Lennart Johnsson
2013‐09‐17COSC4377
COSC4377
Lecture 7
Lennart Johnsson
2013‐09‐17COSC4377
2
Chapter 2: outline
2.1 principles of network applications– app architectures
– app requirements
2.2 Web and HTTP
2.3 FTP
2.4 electronic mail– SMTP, POP3, IMAP
2.5 DNS
2.6 P2P applications
2.7 socket programming with UDP and TCP
Slide from Kurose & Ross, 6th Ed
9/21/2013
2
Lennart Johnsson
2013‐09‐17COSC4377
3
FTP: the file transfer protocol
file transferFTP
server
FTPuser
interface
FTPclient
local filesystem
remote filesystem
user at host
transfer file to/from remote host client/server model client: side that initiates transfer (either to/from remote) server: remote host
ftp: RFC 959, October 1985, http://tools.ietf.org/html/rfc959 RFC 2228, June 1997, security ext., http://tools.ietf.org/html/rfc2228 RFC 2428, September 1998, IPv6, http://tools.ietf.org/html/rfc2428
ftp server: control - port 21, data – port 20Slide derived from Kurose & Ross, 6th Ed
Lennart Johnsson
2013‐09‐17COSC4377
4
FTP: separate control, data connections
• FTP client contacts FTP server at port 21, using TCP
• client authorized over control connection
• client browses remote directory, sends commands over control connection
• when server receives file transfer command, serveropens 2nd TCP data connection (for file) to client
• after transferring one file, server closes data connection
FTPclient
FTPserver
TCP control connection,server port 21
TCP data connection,server port 20
server opens another TCP data connection to transfer another file
control connection: “out of band”
FTP server maintains “state”: current directory, earlier authentication
Slide from Kurose & Ross, 6th Ed
9/21/2013
3
Lennart Johnsson
2013‐09‐17COSC4377
5
FTP commands, responses
sample commands:
• sent as ASCII text over control channel
• USER username
• PASS password
• LIST return list of file in
current directory
• RETR filename retrieves
(gets) file
• STOR filename stores
(puts) file onto remote host
sample return codes
• status code and phrase (as in HTTP)
• 331 Username OK, password required
• 125 data connection already open; transfer starting
• 425 Can’t open data connection
• 452 Error writing file
Slide from Kurose & Ross, 6th Ed
Lennart Johnsson
2013‐09‐17COSC4377
6
FTP: Active and Passive modes Active FTP
Client initiates connection to server port 21 using a client port N>1023 for control and port N+1 for data
Server using port 21 responds to client port N
Client sends command PORT N+1 to server
FTP server using port 20 initiates data connection to client's data port N+1
FTP client sends ACKs to server's port 20
Passive FTP (to avoid firewall, NAT, and related issues, e.g. used by Web browsers)
Client initiates connection to server port 21 using a client port N>1023 for control and port N+1 for data
Server using port 21 responds to client port N
Client sends command PASV to the server
Server creates a port P>1023 and informs client to connect to P for data transfer
Client using port N+1 initiates connection to server port P
Server using port P sends ACK to client port N+1
Reference: http://slacksite.com/other/ftp.html6
9/21/2013
4
Lennart Johnsson
2013‐09‐17COSC4377
File Transfer Protocols using UDPExamples: SABUL (Simple Available Bandwidth Utilization Library)
http://pubs.rgrossman.com/dl/journal‐028.pdf
RBUDP (Reliable Blast UDP)http://www.evl.uic.edu/cavern/papers/cluster2002.pdf
FOBS (Fast Object‐Based File Transfer System)http://www.umcs.maine.edu/~dickens/pubs/HPDC.FINAL.pdf
………………..
SABUL, Chicago Amsterdam, RTT 110ms
http://pubs.rgrossman.com/dl/journal‐028.pdf
7
Three concurrent SABUL streams, each with 1 Gbps theoretical peak
Lennart Johnsson
2013‐09‐17COSC4377
GridFTP: The Protocol• A high‐performance, secure, reliable data transfer protocol optimized for high‐bandwidth wide‐area networks– FTP with well‐defined extensions– Uses basic Grid security – Multiple data channels for parallel transfers– Partial file transfers– Third‐party transfers– Reusable data channels– Command pipelining
• GGF recommendation GFD.20http://www.ogf.org/documents/GFD.20.pdf, April 2003http://www.ogf.org/documents/GFD.21.pdf, July 2003
Current version, 5.5.4, http://www.globus.org/toolkit/docs/latest‐stable/gridftp/Tutorial, http://www.mcs.anl.gov/~kettimut/tutorials/SC07GridFTPTutorialSlides.pdf
8
9/21/2013
5
Lennart Johnsson
2013‐09‐17COSC4377
Control and Data Channels
Control
Data
Typical Installation
Control
Data
Separate Processes Striped Server
Control
Data
GridFTP (and FTP) use (at least) two separate socket connections: A control channel for carrying the commands and responses A data Channel for actually moving the data
Control Channel and Data Channel can be (optionally) completely separate processes.
9
Lennart Johnsson
2013‐09‐17COSC4377
Parallel Data Streams
• Multiple TCP streams between sender and receiver
• Sender pushes multiple blocks in parallel streams
• Blocks reassembled at receiving side and put into correct order
• Protection against dropped packets for each stream
Parallel TransferFully utilizes bandwidth of
network interface on single nodes10
9/21/2013
6
Lennart Johnsson
2013‐09‐17COSC4377
Striped GridFTP Service
• Multiple nodes work together as a single logical GridFTP server
• Every node of the cluster is used to transfer data into/out of the cluster– Each node reads/writes only pieces
they’re responsible for
– Head node coordinates transfers
• Multiple levels of parallelism– CPU, bus, NIC, disk etc.
– Maximizes use of Gbit+ WANs
Striped TransferFully utilizes bandwidth of
Gb+ WAN using multiple nodes.
Par
alle
l Fil
esys
tem
Par
alle
l Fil
esys
tem
11
Lennart Johnsson
2013‐09‐17COSC4377
Source: Bill Alcock, GlobusWorld 200612
9/21/2013
7
Lennart Johnsson
2013‐09‐17COSC4377
GridFTP
• 100% Globus code– No licensing issues– Stable, extensible
• IPv6 Support• XIO for different transports• Striping multi‐Gb/sec wide area transport• Pluggable
– Front‐end: e.g., future WS control channel– Back‐end: e.g., HPSS, cluster file systems– Transfer: e.g., UDP, NetBLT transport
p g
0
2000
4000
6000
8000
10000
12000
14000
16000
18000
20000
0 10 20 30 40 50 60 70
Degree of Striping
Ban
dw
idth
(M
bp
s)
# Stream = 1 # Stream = 2 # Stream = 4
# Stream = 8 # Stream = 16 # Stream = 32
Disk-to-disk onTeraGrid
13
WS = Web Services, HPSS = High‐Performance Storage SystemNetBLT = Network Block Transfer, RFC 998, March 1987
Lennart Johnsson
2013‐09‐17COSC4377
GridFTP Features
• TCP buffer size control
– Tune buffers to latency of network
– Regular FTP optimized for low latency networks, not tunable
• Dramatic improvements for high latency WAN transfers
– 90% of network utilization possible
– 27 GB/s achieved with commodity hardware
14
9/21/2013
8
Lennart Johnsson
2013‐09‐17COSC4377
• Integrated instrumentation: Developers can use client API and plug‐in mechanism to leverage different instrumentation
– Performance markers
– Restart markers
– Throughput performance
– Netlogger style performance tracking
GridFTP
15
Lennart Johnsson
2013‐09‐17COSC4377
Extensive Logging in the Server
• Multiple log levels: ERROR, WARN, INFO, DUMP, ALL
• Log to stdio, syslog, file, …
• Log all connections/transfers to single file or unique files
• Netlogger style logging
• Control permissions on log files
16
9/21/2013
9
Lennart Johnsson
2013‐09‐17COSC4377
GridFTP Features
• Data Storage Interface (DSI)– Interfaces to various storage types– Implement simple functions such as send, receive, mkdir,…
– DSI modules available for HPSS and SRB
• Globus FTP client library (API):– Integration of data transport capabilities directly into applications
– Plug‐in architecture for installing fault recovery and performance tuning algorithms
– Asynchronous programming model
17
Lennart Johnsson
2013‐09‐17COSC4377
GridFTP: Client API• Simple client flow comprises:
1. Setup transfer details including number of parallel data channels, TCP buffer size, local buffer number and size
2. Open connection to server URL and provide a “completion callback” function to be called when transfer complete
3. Setup local buffers to hold read/write
4. Register “data callback” function to be called for filling/flushing buffers
5. Set “not done flag”
6. Loop/wait until “completion callback” clears not done flag
• Work is done inside the “data callback” function– Local buffer filled with data (receiver) & ready to be flushed
– Receive the offset into the file and any error code
– fseek() to the correct place and fwrite() to file
– Register another empty buffer/callback combination
18
9/21/2013
10
Lennart Johnsson
2013‐09‐17COSC4377
GridFTP: Tool Mechanics
• Server mechanics– globus-gridftp-server
– Usually runs as root
– Usually run as a daemon; connections fork new process and setuid
– Can run inetd/xinetd if so desired
– Port 2811 is standard but is configurable
– Logging and security highly configurable
• Client mechanics– globus-url-copy
– Options for parallel channels, TCP buffer size, data buffer size, debugging, recursive directory transfers, etc.
19
Lennart Johnsson
2013‐09‐17COSC4377
• Server‐side computation– Extended retrieve (ERET), Extended store (ESTO)
– Simple pre‐processing (partial get, sub‐sampling )
– Can greatly reduce network load
– Client must also support ESTO/ERET functionality
GridFTP:Secure, High Performance Data
Transport
20
9/21/2013
11
Lennart Johnsson
2013‐09‐17COSC4377
21
Chapter 2: outline
2.1 principles of network applications– app architectures
– app requirements
2.2 Web and HTTP
2.3 FTP
2.4 electronic mail– SMTP, POP3, IMAP
2.5 DNS
2.6 P2P applications
2.7 socket programming with UDP and TCP
Slide from Kurose & Ross, 6th Ed
Lennart Johnsson
2013‐09‐17COSC4377
e‐mail and Messaging Statistics
• e‐mail
• 2.2 billion users
• 247 billion emails are sent each day
http://www.radicati.com/wp/wp‐content/uploads/2012/04/Email‐Statistics‐Report‐2012‐2016‐Brochure.pdf
Over-the-Top (OTT) messaging v SMS messaging.
OTT messages SMS messages
MMS messages
2011 3,492 billion 7,844 billion 207 billion2012 5,846 billion 8,600 billion 228 billion2016 20,293 billion 9,554 billion 277 billionSources: Portio ResearchFebuary 2012
via: mobiThinking
http://mobithinking.com/mobile‐marketing‐tools/latest‐mobile‐stats/c#mobilemessaging
Key Global Telecom Indicators for the World (estimates)
Globalmillions
Per 100 people
Mobile cellular subscriptions 5,981 86.7
Fixed telephone lines 1,159 16.6
Active mobile broadband subscriptions 1,186 17.0
Fixed broadband subscriptions 591 8.5
Source: International Telecommunication Union(November 2011)
22
9/21/2013
12
Lennart Johnsson
2013‐09‐17COSC4377
23
Electronic mail
Three major components:• user agents • mail servers • simple mail transfer protocol:
SMTP
User Agent• a.k.a. “mail reader”• composing, editing, reading
mail messages• e.g., Outlook, Thunderbird,
iPhone mail client• outgoing, incoming messages
stored on server
user mailbox
outgoing message queue
mailserver
mailserver
mailserver
SMTP
SMTP
SMTP
useragent
useragent
useragent
useragent
useragent
useragent
Slide from Kurose & Ross, 6th Ed
Lennart Johnsson
2013‐09‐17COSC4377
24
Electronic mail: mail servers
mail servers:• mailbox contains incoming
messages for user
• message queue of outgoing (to be sent) mail messages
• SMTP protocol between mail servers to send email messages
– client: sending mail server
– “server”: receiving mail server
mailserver
mailserver
mailserver
SMTP
SMTP
SMTP
useragent
useragent
useragent
useragent
useragent
useragent
Slide from Kurose & Ross, 6th Ed
9/21/2013
13
Lennart Johnsson
2013‐09‐17COSC4377
25
Electronic Mail: SMTP [RFC 2821]
• uses TCP to reliably transfer email message from client to server, port 25
• direct transfer: sending server to receiving server
• three phases of transfer– handshaking (greeting)
– transfer of messages
– closure
• command/response interaction (like HTTP, FTP)– commands: ASCII text
– response: status code and phrase
• messages must be in 7‐bit ASCI
Slide from Kurose & Ross, 6th Ed
Lennart Johnsson
2013‐09‐17COSC4377
26
useragent
Scenario: Alice sends message to Bob1) Alice uses UA to compose
message “to”[email protected]
2) Alice’s UA sends message to her mail server; message placed in message queue
3) client side of SMTP opens TCP connection with Bob’s mail server
4) SMTP client sends Alice’s message over the TCP connection
5) Bob’s mail server places the message in Bob’s mailbox
6) Bob invokes his user agent to read message
mailserver
mailserver
1
2 3 4
5
6
Alice’s mail server Bob’s mail server
useragent
Slide from Kurose & Ross, 6th Ed
9/21/2013
14
Lennart Johnsson
2013‐09‐17COSC4377
27
Sample SMTP interaction
S: 220 hamburger.edu C: HELO crepes.fr S: 250 Hello crepes.fr, pleased to meet you C: MAIL FROM: <[email protected]> S: 250 [email protected]... Sender ok C: RCPT TO: <[email protected]> S: 250 [email protected] ... Recipient ok C: DATA S: 354 Enter mail, end with "." on a line by itself C: Do you like ketchup? C: How about pickles? C: . S: 250 Message accepted for delivery C: QUIT S: 221 hamburger.edu closing connection
Slide from Kurose & Ross, 6th Ed
Lennart Johnsson
2013‐09‐17COSC4377
28
SMTP: final words
• SMTP uses persistent connections
• SMTP requires message (header & body) to be in 7‐bit ASCII
• SMTP server uses CRLF.CRLF to determine end of message
comparison with HTTP:
• HTTP: pull
• SMTP: push
• both have ASCII command/response interaction, status codes
• HTTP: each object encapsulated in its own response msg
• SMTP: multiple objects sent in multipart msg
Slide from Kurose & Ross, 6th Ed
9/21/2013
15
Lennart Johnsson
2013‐09‐17COSC4377
29
Mail message format
SMTP: protocol for exchanging email msgs
RFC 822: standard for text message format:
• header lines, e.g.,– To:
– From:
– Subject:
different from SMTP MAIL FROM, RCPT TO:commands!
• Body: the “message”– ASCII characters only
header
body
blankline
Slide from Kurose & Ross, 6th Ed
Lennart Johnsson
2013‐09‐17COSC4377
30
Mail access protocols
• SMTP: delivery/storage to receiver’s server
• mail access protocol: retrieval from server– POP: Post Office Protocol [RFC 1939]: authorization, download
– IMAP: Internet Mail Access Protocol [RFC 1730]: more features, including manipulation of stored msgs on server
– HTTP: gmail, Hotmail, Yahoo! Mail, etc.
sender’s mail server
SMTP SMTPmail access
protocol
receiver’s mail server
(e.g., POP, IMAP)
useragent
useragent
Slide from Kurose & Ross, 6th Ed
9/21/2013
16
Lennart Johnsson
2013‐09‐17COSC4377
31
POP3 protocol
authorization phase• client commands:
– user: declare username
– pass: password
• server responses
– +OK– -ERR
transaction phase, client:• list: list message numbers
• retr: retrieve message by number
• dele: delete
• quit
C: list S: 1 498 S: 2 912 S: . C: retr 1 S: <message 1 contents>S: . C: dele 1 C: retr 2 S: <message 1 contents>S: . C: dele 2 C: quit S: +OK POP3 server signing off
S: +OK POP3 server ready C: user bob S: +OK C: pass hungry S: +OK user successfully logged on
Slide from Kurose & Ross, 6th Ed
Lennart Johnsson
2013‐09‐17COSC4377
32
POP3 (more) and IMAP
more about POP3• previous example uses
POP3 “download and delete”mode
– Bob cannot re‐read e‐mail if he changes client
• POP3 “download‐and‐keep”: copies of messages on different clients
• POP3 is stateless across sessions
IMAP
• keeps all messages in one place: at server
• allows user to organize messages in folders
• keeps user state across sessions:
– names of folders and mappings between message IDs and folder name
Slide from Kurose & Ross, 6th Ed
9/21/2013
17
Lennart Johnsson
2013‐09‐17COSC4377
Extra material on SMTP• ESMTP (Extended SMTP;
RFC 1869, 1995) allows 8‐bit data transmission; use “EHLO” (extended Hello) to replace “HELO”
• Authentication process in FTP, POP3, and IMAP is not encrypted. Use FTP/SSL, POP3S, or IMAPS if possible.
Multipart MIME (Multi‐part Internet Mail Extension)From: "Senders Name" <[email protected]>
To: "Recipient Name" <[email protected]>
Message‐ID: <[email protected]>
Date: Sat, 24 Sep 2005 15:06:49 ‐0400
Subject: Sample Multi‐Part
MIME‐Version: 1.0
Content‐Type: multipart/alternative;
boundary="‐‐‐‐=_NextPart_DC7E1BB5_1105_4DB3_BAE3“
‐‐‐‐‐‐=_NextPart_DC7E1BB5_1105_4DB3_BAE3
Content‐type: text/plain; charset=iso‐8859‐1
Content‐Transfer‐Encoding: quoted‐printable
Sample Text Content
‐‐‐‐‐‐=_NextPart_DC7E1BB5_1105_4DB3_BAE3
Content‐type: text/html; charset=iso‐8859‐1
Content‐Transfer‐Encoding: quoted‐printable
<html><head></head>
<body>Sample HTML Content</body></html>
‐‐‐‐‐‐=_NextPart_DC7E1BB5_1105_4DB3_BAE3
Slide from Kurose & Ross, 6th Ed
http://en.wikipedia.org/wiki/Extended_SMTP
33
Lennart Johnsson
2013‐09‐17COSC4377
Another Example – Binary encoded Attachment
From: <[email protected]> CRLFTo: <[email protected]> CRLFSubject: foobar CRLFMIME‐Version: 1.0 CRLFContent‐Type: multipart/mixed; CRLF boundary= "KkK170891tpbkKk__FV_KKKkkkjjwq" CRLFCRLF‐‐KkK170891tpbkKk__FV_KKKkkkjjwq CRLFContent‐Type: text/plain; charset=US‐ASCII CRLFCRLFhere goes the text message CRLFCRLF‐‐KkK170891tpbkKk__FV_KKKkkkjjwq CRLFContent‐Type: application/octet‐stream CRLFContent‐Transfer‐Encoding: base64 CRLFContent‐Disposition: attachment; CRLF filename= "suggested name of the attachment" CRLFCRLFhere goes the Base64 encoded attachment CRLF‐‐KkK170891tpbkKk__FV_KKKkkkjjwq‐‐ CRLFCRLF.CRLF
Blank line
Blank line
Blank line
Blank line
Part 1
Part 2
34
9/21/2013
18
Lennart Johnsson
2013‐09‐17COSC4377
35
Chapter 2: outline
2.1 principles of network applications– app architectures
– app requirements
2.2 Web and HTTP
2.3 FTP
2.4 electronic mail– SMTP, POP3, IMAP
2.5 DNS
2.6 P2P applications
2.7 socket programming with UDP and TCP
Slide from Kurose & Ross, 6th Ed
Lennart Johnsson
2013‐09‐17COSC4377
36
DNS: domain name system
people:many identifiers:
– SSN, name, passport #
Internet hosts, routers:
– IP address (32 bit) ‐used for addressing datagrams
– “name”, e.g., www.yahoo.com ‐used by humans
Q: how to map between IP address and name, and vice versa ?
Domain Name System:• distributed database
implemented in hierarchy of many name servers
• application‐layer protocol:hosts, name servers communicate to resolvenames (address/name translation)– note: core Internet function,
implemented as application‐layer protocol
– complexity at network’s “edge”
Slide from Kurose & Ross, 6th Ed
9/21/2013
19
Lennart Johnsson
2013‐09‐17COSC4377
37
DNS: services, structure
why not centralize DNS?• single point of failure
• traffic volume
• distant centralized database
• maintenance
DNS services
• hostname to IP address translation
• host aliasing– canonical, alias names
• mail server aliasing
• load distribution
– replicated Web servers: many IP addresses correspond to one name
A: doesn’t scale!
Slide from Kurose & Ross, 6th Ed
Lennart Johnsson
2013‐09‐17COSC4377
Domain Names Statistics
TLD Total DomainsCOM 102,764,716NET 14,160,747ORG 9,832,140INFO 5,713.205BIZ 2,327,124
TOTAL 134,797,932
http://www.webhosting.info/registries/
9/16/2013
As of October 2009 there were 21 generic TLDs and 250 country domains (based on ISO‐3166 country abbreviations)http://en.wikipedia.org/wiki/Domain_name
• 324 – Number of TLDs
• 86.9 million – Number of country code
top‐level domains (.CN, .UK, .DE, etc.).
• 220 million – Number of registered
domain names.
Q3 2011
http://royal.pingdom.com/2012/01/17/internet‐2011‐in‐numbers/
38
Domain names managed by IANA, the Internet Assigned Numbers Authority, www.iana.org
9/21/2013
20
Lennart Johnsson
2013‐09‐17COSC4377
39
Root DNS Servers
com DNS servers org DNS servers edu DNS servers
poly.eduDNS servers
umass.eduDNS servers
yahoo.comDNS servers
amazon.comDNS servers
pbs.orgDNS servers
DNS: a distributed, hierarchical database
client wants IP for www.amazon.com; 1st approx:
• client queries root server to find .com DNS server
• client queries .com DNS server to get amazon.com DNS server
• client queries amazon.com DNS server to get IP address for www.amazon.com
… …
Lennart Johnsson
2013‐09‐17COSC4377
40
DNS: root name servers
• contacted by local name server that can not resolve name
• root name server:
– contacts authoritative name server if name mapping not known
– gets mapping
– returns mapping to local name server
13 root name “servers”worldwide
a. Verisign, Los Angeles CA(5 other sites)
b. USC-ISI Marina del Rey, CAl. ICANN Los Angeles, CA
(41 other sites)
e. NASA Mt View, CAf. Internet Software C.Palo Alto, CA (and 48 other sites)
i. Netnod, Stockholm (37 other sites)
k. RIPE London (17 other sites)
m. WIDE Tokyo(5 other sites)
c. Cogent, Herndon, VA (5 other sites)d. U Maryland College Park, MDh. ARL Aberdeen, MDj. Verisign, Dulles VA (69 other sites )
g. US DoD Columbus, OH (5 other sites)
Slide from Kurose & Ross, 6th Ed
9/21/2013
21
Lennart Johnsson
2013‐09‐17COSC4377
DNS Root Servers
http://www.root‐servers.org/
377 servers as of 2013‐09
41
Lennart Johnsson
2013‐09‐17COSC4377
Server Operator Locations
A VeriSign, Inc.Sites: 6 Global: 6 Local: 0Los Angeles, CA, US; New York, NY, US *; Frankfurt, DE *; Hong Kong, HK; Palo Alto, CA, US *; Ashburn, VA, US *
B Information Sciences InstituteSites: 1 Global: 0 Local: 1Earth
C Cogent CommunicationsSites: 6 Global: 6 Local: 0Herndon, VA, US; Los Angeles, CA, US; New York, NY, US; Chicago, IL, US; Frankfurt, DE; Madrid, ES
D University of MarylandSites: 1 Global: 1 Local: 0College Park, MD, US *
E NASA Ames Research CenterSites: 12 Global: 1 Local: 11Mountain View, CA, US; Sydney, AU; Cape Town, ZA; Tokyo, JP; Brussels, BE; London, UK; San Paulo, BR; Atlanta, GA, US; Chicago, IL, US; New York, NY, US; Seattle, WA, US; Los Angeles, CA, US
F Internet Systems Consortium, Inc.
Sites: 49 Global: 2 Local: 47Ottawa, Canada *; Palo Alto, CA, US *; San Jose, CA, US; New York, NY, US *; San Francisco, CA, US *; Madrid, ES; Hong Kong, HK; Los Angeles, CA, US *; Rome, Italy; Auckland, NZ *; Sao Paulo, BR; Beijing, CN; Seoul, KR *; Moscow, RU *; Taipei, TW; Dubai, AE; Paris, FR *; Singapore, SG; Brisbane, AU *; Toronto, CA *; Monterrey, MX; Lisbon, PT *; Johannesburg, ZA; Tel Aviv, IL; Jakarta, ID; Munich, DE *; Osaka, JP *; Prague, CZ *; Amsterdam, NL *; Barcelona, ES *; Nairobi, KE; Chennai, IN; London,UK *; Santiago de Chile, CL; Dhaka, BD; Karachi, PK; Torino, IT; Chicago, IL, US *; Buenos Aires, AR; Caracas, VE; Oslo, NO *; Panama, PA; Quito, EC; Kuala Lumpur, Malaysia *; Suva, Fiji; Cairo, Egypt; Atlanta, GA, US; Podgorica, ME; St. Maarten, AN *
G U.S. DOD Network Information CenterSites: 6 Global: 6 Local: 0Columbus, OH, US; San Antonio, TX, US; Honolulu, HI, US; Fussa, JP; Stuttgart‐Vaihingen, DE; Naples, IT
H U.S. Army Research LabSites: 2 Global: 2 Local: 0Aberdeen Proving Ground, MD, US *; San Diego, CA, US *
I Netnod (formerly Autonomica)
Sites: 43 Stockholm, SE *; Helsinki, FI; Milan, IT *; London, UK *; Geneva, CH *; Amsterdam, NL (2 sites) *; Oslo, NO *; Bangkok, TH *;Hong Kong, HK *; Brussels, BE *; Frankfurt, DE *; Ankara, TR *; Bucharest, RO *; Chicago, IL, US; Washington, DC, US; Tokyo, JP (2 sites) *; Kuala Lumpur, MY *; Palo Alto, CA, US *; Jakarta, ID; Wellington, NZ; Johannesburg, ZA *; Perth, AU *; Singapore, SG; Miami, FL, US; Mumbai, IN; Beijing, CN; Manila, PH; Doha, QA *; Colombo, LK *; Vienna, AT *; Paris, FR *; Taipei, TW *; Porto Alegre, BR; Yerevan, AM *; Thimphu, BT *; Manama, BH; Karachi, PK; Tallinn, EE *; Kiev, UA *; Ulaanbaatar, MN *; Lulea, SE
J VeriSign, Inc.
Sites: 70 Global: 63 Local: 5Dulles, VA, US (2 sites); Dulles, VA, US (1 sites); Ashburn, VA, US *; Miami, FL, US; Atlanta, GA, US; Seattle, WA, US; Chicago,IL, US; New York, NY, US *; Honolulu, HI, US; Mountain View, CA, US (1 sites); Mountain View, CA, US (1 sites); San Francisco, CA, US (2 sites) *; Dallas, TX, US; Amsterdam, NL; London, UK; Stockholm, SE (2 sites); Tokyo, JP; Seoul, KR; Beijing, CN; Singapore, SG; Dublin, IE; Kaunas, LT; Nairobi, KE; Montreal, CA; Perth, AU; Sydney, AU; Cairo, EG; Cairo, EG; Warsaw, PL (2 sites); Brasilia, BR; Sao Paulo, BR; Sofia, BG; Prague, CZ; Johannesburg, ZA; Toronto, CA; Buenos Aires, AR; Madrid, ES; Fribourg, CH; Hong Kong, HK (2 sites); Turin, IT; Mumbai, IN; Oslo, NO; Brussels, BE; Paris, FR (2 sites); Helsinki, FI; Frankfurt, DE *; Riga, LV; Milan, IT; Rome, IT; Lisbon, PT; San Juan, PR; Edinburgh, UK; Tallinn, EE; Taipei, TW; New York, NY, US *; Palo Alto, CA, US *; Anchorage, US; Moscow, RU; Manila, PH; Kuala Lumpur, MY; Luxembourg City, LU; Guam, GU, US; Vancouver, CA; Wellington, NZ
http://www.root‐servers.org/42
9/21/2013
22
Lennart Johnsson
2013‐09‐17COSC4377
Server Operator Locations
K RIPE NCC
Sites: 18 Global: 5 Local: 13London, UK *; Amsterdam, NL *; Frankfurt, DE *; Athens, GR *; Doha, QA; Milan, IT *; Reykjavik, IS *; Helsinki, FI *; Geneva, CH*; Poznan, PL *; Budapest, HU *; Abu Dhabi, AE; Tokyo, JP *; Brisbane, AU *; Miami, FL, US *; Delhi, IN; Novosibirsk, RU; Dar esSalaam, TZ
L ICANN
Sites: 120Ezeiza, Argentina *; Mascot, Australia (2 sites) *; Brisbane, Queensland, Australia *; Perth, Western Australia, Australia (2 sites) *; Vienna, Austria *; Al Muharraq, Bahrain *; Dhaka, Bangladesh *; Brussels, Belgium (2 sites) *; Rio de Janeiro, Brazil *; Salvador, Bahia, Brazil *; Fortaleza, Ceara, Brazil *; Sao Jose dos Campos, Brazil *; Sao Paulo, Brazil *; Brasilia, Distrito Federal, Brazil *; Belo Horizonte, Minas Gerais, Brazil *; Londrina, Parana, Brazil *; Curitiba, Parana, Brazil *; Porto Alegre, Rio Grande do Sul, Brazil *; Florianopolis, Santa Catarina, Brazil *; Campinas, Sao Paulo State, Brazil *; Sofia, Bulgaria *; Mississauga, Ontario, Canada (2 sites) *; Ottawa, Ontario, Canada (2 sites) *; Toronto, Ontario, Canada *; Santiago, Chile *; Concepcion, Biobio Region, Chile *; Bogota, Colombia *; Prague, Czech Republic *; Copenhagen, Denmark *; Quito, Ecuador *; Cairo, Egypt *; Lyon, France *; Marseille, France *; Paris, France *; Paris‐Orly, Paris, France *; St Denis, Reunion, France *; Berlin, Germany *; Hamburg, Germany *; Munich, Bavaria, Germany *; Dortmund, North Rhine‐Westphalia, Germany *; Dusseldorf, North Rhine‐Westphalia, Germany *; Heraklion, Crete, Greece *; Hagatna, Guam *; Jakarta, Java, Indonesia *; Dublin, Ireland *; Turin, Italy *; Narita, Japan *; Tokyo, Honshu, Japan *; Semey, Kazakhstan *; Nairobi, Kenya *; Luxembourg‐Findel, Luxembourg *; Monterrey, Nuevo Leon, Mexico *; Maputo, Mozambique *; Kathmandu, Nepal *; Haarlemmermeer, Netherlands *; Christchurch, New Zealand *; Mangere, New Zealand (2 sites) *; Wellington, New Zealand *; Ullensaker, Norway *; Beijing, Peoples Republic of China (2 sites) *; Callao, Peru *; Metro Manila, Philippines *; Warsaw, Poland *; Moscow, Russia *; Dammam, Saudi Arabia *; Jeddah, Saudi Arabia *; Riyadh, Saudi Arabia *; Dakar, Senegal *; Changi, Singapore *; Cape Town, South Africa (2 sites) *; Johannesburg, Gauteng, South Africa *; Incheon, South Korea *; El Prat de Llobregat, Spain *; Malmo, Sweden *; Stockholm, Sweden *; Geneva, Switzerland *; Zurich, Switzerland *; Istanbul, Turkey (3 sites) *; Kharkiv, Ukraine *; Kiev, Ukraine *; Odessa, Ukraine *; Saginaw Bay, Alaska, United States *; Burbank, California, United States *; Los Angeles, California, United States (2 sites) *; San Jose, California, United States *; San Jose, Costa Rica *; Santa Ana, California, United States *; Denver, Colorado, United States *; Jacksonville, Florida, United States *; Melbourne,Florida, United States *; Miami, Florida, United States (2 sites) *; Atlanta, Georgia, United States (2 sites) *; Honolulu, Hawaii, United States *; Chicago, Illinois, United States *; Boston, Massachusetts, United States (2 sites) *; New York City, New York, United States *; Portland, Oregon, United States *; Philadelphia, Pennsylvania, United States *; Culpeper, Virginia, United States *; SeaTac, Washington, United States *; Montevideo, Uruguay *; Sanaa, Yemen *; Vancouver, Canada *; Leeds, United Kingdom *; Anchorage, Alaska, United States *; Porto, Portugal *
M WIDE ProjectSites: 6 Global: 5 Local: 1Tokyo, JP (3 sites) *; Seoul, KR; Paris, FR *; San Francisco, CA, US *
http://www.root‐servers.org/ 43
Lennart Johnsson
2013‐09‐17COSC4377
DNS Root Server K
44
9/21/2013
23
Lennart Johnsson
2013‐09‐17COSC4377
45
TLD, authoritative servers
top‐level domain (TLD) servers:– responsible for com, org, net, edu, aero, jobs, museums, and all top‐level country domains, e.g.: uk, fr, ca, jp
– Network Solutions maintains servers for .com TLD
– Educause for .edu TLD
authoritative DNS servers:– organization’s own DNS server(s), providing authoritative hostname to IP mappings for organization’s named hosts
– can be maintained by organization or service provider
Slide from Kurose & Ross, 6th Ed
Lennart Johnsson
2013‐09‐17COSC4377
46
Local DNS name server
• does not strictly belong to hierarchy
• each ISP (residential ISP, company, university) has one– also called “default name server”
• when host makes DNS query, query is sent to its local DNS server– has local cache of recent name‐to‐address translation pairs (but may be out of date!)
– acts as proxy, forwards query into hierarchy
Slide from Kurose & Ross, 6th Ed
9/21/2013
24
Lennart Johnsson
2013‐09‐17COSC4377
47
requesting hostcis.poly.edu
gaia.cs.umass.edu
root DNS server
local DNS serverdns.poly.edu
1
23
4
5
6
authoritative DNS serverdns.cs.umass.edu
78
TLD DNS server
DNS name resolution example
• host at cis.poly.edu wants IP address for gaia.cs.umass.edu
iterated query: contacted server
replies with name of server to contact
“I don’t know this name, but ask this server”
Slide from Kurose & Ross, 6th Ed
Lennart Johnsson
2013‐09‐17COSC4377
48
45
6
3recursive query: puts burden of name
resolution on contacted name server
heavy load at upper levels of hierarchy?
requesting hostcis.poly.edu
gaia.cs.umass.edu
root DNS server
local DNS serverdns.poly.edu
1
27
authoritative DNS serverdns.cs.umass.edu
8
DNS name resolution example
TLD DNS server
Slide from Kurose & Ross, 6th Ed
9/21/2013
25
Lennart Johnsson
2013‐09‐17COSC4377
49
DNS: caching, updating records
• once (any) name server learns mapping, it cachesmapping– cache entries timeout (disappear) after some time (TTL)
– TLD servers typically cached in local name servers• thus root name servers not often visited
• cached entries may be out‐of‐date (best effort name‐to‐address translation!)– if name host changes IP address, may not be known Internet‐wide until all TTLs expire
• update/notify mechanisms proposed IETF standard– RFC 2136
Slide from Kurose & Ross, 6th Ed
Lennart Johnsson
2013‐09‐17COSC4377
(18 nodes, 5 global, 13 local)
Global
http://k.root‐servers.org/
Some statistics for the K DNS Root server 9/25/2012
50
9/21/2013
26
Lennart Johnsson
2013‐09‐17COSC4377
DNS Root server I
http://www.netnod.se/
43 sites
51
Lennart Johnsson
2013‐09‐17COSC4377
DNS Root Server I load 9/25/2012
http://www.netnod.se/ix‐stats/sums/All.html
'Daily' graph (5 Minute Average)
52
9/21/2013
27
Lennart Johnsson
2013‐09‐17COSC4377
DNS Root Server I load 9/18 – 9/25 2012'Weekly' graph (30 Minute Average)
http://www.netnod.se/ix‐stats/sums/All.html
53
Lennart Johnsson
2013‐09‐17COSC4377
DNS Root Server I load 2 years ending 9/25/12
'Two year' graph (1 Day Average)
http://www.netnod.se/ix‐stats/sums/All.html54