Lecture 3: Application Layer1 מבוא לרשתות תקשורת : רמת היישום דואר...

Lecture 3: Application Layer 1

מבוא לרשתות תקשורת:רמת היישום דואר אלקטרוני

( DNS)ומערכת השמות (chapter 2 – application)

פרופ' אמיר הרצברג

Computer Networking: A Top Down Approach Featuring the Internet, 2nd edition. Jim Kurose, Keith RossAddison-Wesley, July 2002.

Based on foils by Kurose & Ross ©, see: http://www.aw.com/kurose-ross/

My site: http://AmirHerzberg.com

http://www.aw.com/kurose-ross/

http://amirherzberg.c/

http://amirherzberg.c/


Application Layer (ch. 2): outline

2.1 Principles of app layer protocols

2.2 Web and HTTP HTTP Overview Connections:

Persistent, pipeline Requests/responses Authorization &

cookies Caching and proxies

2.3 FTP 2.4 Electronic Mail

SMTP, POP3, IMAP 2.5 DNS 2.9 Content

distribution Content distribution

networks P2P file sharing


Network Applications & Application-layer protocolsNetwork Application:

application involving processes in different hosts e.g., e-mail, Web, FTP runs in end systems communicates using…

Application-layer protocols Communication protocol

for network applications Use transport services

(TCP, UDP) Define messages and their

processing (actions) Standard, public or

proprietary

application

transportnetworkdata linkphysical

application


application



App-layer protocol defines Rules for when and how

processes communicate Type of communication,

e.g., request & response messages

Syntax of messages: what fields, & how fields are delineated

Semantics of the fields, i.e., meaning of information in fields

Public-domain application layer protocols:

defined in RFCs allows for

interoperability e.g., HTTP, SMTPProprietary protocols: e.g., KaZaA

Based on (proprietary) FastTrack protocol

Allowing them to disable `clone` clients


Electronic MailThree major components: Mail user agents (mail client) Mail servers

Some w/ mailboxes Protocols: SMTP (Simple Mail

Transfer Protocol), POP3,…

User Agent (UA) a.k.a. “mail reader/client” e.g., Outlook, Eudora, web-

mail (e.g. Hotmail) Compose mail, send to server Pick-up mail from mailbox in

server

user mailbox

outgoing message queue

mailserver

useragent

useragent

useragent

mailserver

useragent

useragent

mailserver

useragentPOP3

SMTP

SMTP

SMTP

SMTP

SMTP


Mail Pick-Up protocols

Mail Pick-Up (access) protocol: retrieval from server POP3: Post Office Protocol ver. 3 [RFC 1939]

• Authentication (agent <-->server) and download IMAP: Internet Mail Access Protocol [RFC 1730]

• More features, e.g. manage messages in folders• More complex; less common

HTTP: Hotmail , Yahoo! Mail, etc.• Authentication usually by password, cookies

useragent

sender’s mail server

useragent

SMTP(or ___) SMTP Pick-up

protocol

receiver’s mail server


Electronic Mail: Mail ServersMail Servers Message queue of outgoing (to

be sent) mail messages Mailboxes of incoming mail Use SMTP: standard protocol to

send, receive email messages Focus: interoperability and

connectivity Initiated by (user agent or

mail-server) sending mail• `mail transfer client`

To destination or `relay`• `mail transfer server`

Other (proprietary) protocols exist, e.g. Exchange, Notes

Receive mail from User Agent: often also via SMTP…

mailserver

useragent

useragent

useragent

mailserver

useragent

useragent

mailserver

useragent

SMTP

SMTP

SMTP

Lecture 3: Application Layer

Alice'sMUA

MSA

Bob'sMUA

MDA

Internet

AlDomaina.com

Domainb.com

MTAA

MTAB

Different Types of Mail ServersUser Agent

submits (sends) via Mail Submit Agent (MSA)

User’s mailbox kept on Mail Delivery Agent (MDA)

Relaying by Mail Transfer Agents (MTA)

Relaying is tricky…


Alice'sMUA

MSA

Bob'sMUA

MDA

Internet

AlDomaina.com

Domainb.com

MTAA

MTAB

Mail Relaying Services Basic relaying:

within domain… MUA to MSA MSA to MTA MTA to MDA

Sending Domain’s MTA to Receiving Domain’s MTA

Open Relay: not in sender, receiver domains

Long ago: many open relays (connectivity)

Abused by spammers

So… now `forbidden`

`Legitimate` intermediary services: forwarding, mailing lists,,,

Open Relay MTA


Alice'sMUA

MSA

Bob'sMUA

MDA

Internet

Carl'sMUA

AlDomaina.com

Domainb.com

MTAA

MTAB

MTAX

Standard Anti-Spam Mechanisms Spam: ùnsolicited bulk` Ads/other òften

undesirable` mail Without clear label

A big problem 80% of mail…

No òpen relays` Relay in domain Ok

Block port 25 Prevent spam

directly from computer, exit only via MSA/MTA

Authenticate senders

Esp: from outside Quotas So: is email sender

ID authenticated? Not…

© Gila Kaplan)with permission(


The Simple Mail Transfer Protocol (SMTP) [RFC 2821] Uses TCP to reliably transfer email message Initiated by sending agent, called SMTP-client SMTP-server listens on port 25, receives email

MUA to MSA, MSA/MTA to MTA, MTA to MDA three phases of transfer

handshaking (greeting) transfer of messages closure

command/response interaction commands: ASCII text response: status code and phrase


SMTP Phases Handshaking (greeting)

TCP Connection Setup: SMTP-server rejects if overloaded Server hello: sends 220 B.com Server may delay hello (until ready) Client hello: sends Or EHLO (extended hello, supports options)

Transfer of one or more messages: MAIL FROM: sender email (for error report) RCPT TO: recipient that Receiver-SMTP should deliver to Data: RFC 822 message (with headers), then terminator

Closure SMTP closure: C: QUIT, S: 221 TCP connection closure (initiated by SMTP-client)


SMTP Envelope SMTP delivers each message by the sequence:

MAIL FROM: [email protected] RCPT TO: [email protected]

• Repeat for more recipients DATA

Mail envelope: MAIL FROM:, RCPT TO: addresses SMTP uses these fields to route / reject messages

Accept only mail to known users More on this later

SMTP passes mail envelope with the message To the mail delivery system / protocol (e.g. POP3) MDA may copy to message headers (often not)

Distinct from “similar” RFC822 message headers

RCPT TO: [email protected]

RCPT TO: [email protected]

MAIL FROM: [email protected]

mailto:[email protected]






Mail message format (RFC 822/2822)SMTP: protocol for

exchanging email msgsRFC 822/2822: standards

for text message format: header lines, e.g.,

To: From: Subject:different from SMTP

commands! Blank line body

the “message”, printable ASCII characters only

How to send non-ASCII?

header

body

blankline


MIME: multimedia mail extension How to send non-ASCII data, e.g. picture, program? Encode: map data bytes to (more) printable-ASCII bytes

E.g.: Hex encoding (x’A3’”A3”); usually use base64 (more compact)

Identify in header: MIME version, encoding, data type, subtype Types: Text, Image, Audio, Video, Application, Multipart

From: [email protected] To: [email protected] Subject: Picture of yummy crepe. MIME-Version: 1.0 Content-Transfer-Encoding: base64 Content-Type: image/jpeg

base64 encoded data ..... ......................... ......base64 encoded data

multimedia datatype, subtype,

parameter declaration

method usedto encode data

MIME version

encoded data

Multipart: multiple parts (with separators defined in header)


Envelope vs. Message [from, to] S: 220 B.com C: HELO A.com S: 250 OK C: MAIL FROM: <[email protected]> S: 250 OK C: RCPT TO: [email protected] S: 250 OK C: DATA S: 354 Enter mail, end with "." on a line by itself C: from: [email protected] C: to: [email protected], [email protected] C: C: what a wonderful message C: . S: 250 OK C: QUIT S: 221 b.com closing connection

RFC822/2822Email Message Syntax



Envelope RCPT TO: vs. Message to: S: 220 B.com C: HELO A.com S: 250 OK C: MAIL FROM: <[email protected]> S: 250 OK C: RCPT TO: [email protected] S: 250 OK C: DATA S: 354 Enter mail, end with "." on a line by itself C: from: [email protected] C: to: [email protected] C: C: what a wonderful message C: . S: 250 OK C: QUIT S: 221 b.com closing connection

Forwarding services, Mailing-lists, …




Envelope RCPT TO: for Message bcc: S: 220 B.com C: HELO A.com S: 250 OK C: MAIL FROM: <[email protected]> S: 250 OK C: RCPT TO: [email protected] S: 250 OK C: RCPT TO: [email protected] C: DATA S: 354 Enter mail, end with "." on a line by itself C: from: [email protected] C: to: [email protected] C: C: I _still_ did not receive the goods you promised C: . S: 250 OK C: QUIT S: 221 b.com closing connection

Notice also multiple RCPT TO)one per recipient, all in same domain(




SMTP – Basic Entities S: 220 B.com C: HELO A.com S: 250 OK C: MAIL FROM: <[email protected]> S: 250 OK C: RCPT TO: [email protected] S: 250 OK C: DATA S: 354 Enter mail, end with "." on a line by itself C: from: [email protected] C: to: [email protected] C: C: what a wonderful message C: . S: 250 OK C: QUIT S: 221 b.com closing connection

HELO name HELO name

Message originatorMessage originator

Message RecipientMessage Recipient

Envelope RecipientEnvelope Recipient

Envelope originatorEnvelope originator

Server greeting nameServer greeting name


SMTP – Basic Entities (cont’)

S: 220 B.com C: HELO A.com S: 250 OK C: MAIL FROM: <[email protected]> S: 250 OK C: RCPT TO: [email protected] S: 250 OK C: DATA S: 354 Enter mail, end with "." on a line by itself C: from: [email protected] C: to: [email protected] C: C: what a wonderful message C: . S: 250 OK …

HELO nameHELO name

Message originatorMessage originator

Message RecipientMessage Recipient

Envelope RecipientEnvelope Recipient

Envelope originatorEnvelope originator

Server greeting nameServer greeting name

TCP connection parms: SMTP-Client (33.4.5.6) connect to SMTP-Server-IP (35.6.7.8)


Email Sender Identities SMTP-Client identifies only relaying agent (MTA)

Often unrelated to email sender identity MUA displays only the From: header (usually)

RFC822 From: asked for message to be sent Most MSAs allow use of any From: identity

To allow transparent mobility But, spoofing email becomes trivial… Partial alternate solution: use Reply-To: header

Several other sender-related identifiers: Sender: “entered message into email stream” Resent-from, resent-sender:

• RFC2822: identifies party (time, etc.) resending message• Specifically not intended for use for forwarding (by user or by

service)• But used, e.g. by sendmail, mailing-lists…

Bounce address to report errors (next topic)


Some Email Originator Identities

… C: MAIL FROM: <[email protected]> S: 250 OK C: RCPT TO: [email protected] S: 250 OK C: DATA S: 354 Enter mail, end with "." on a line by itself C: from: [email protected] C: sender: [email protected] C: reply-to: [email protected] C: to: [email protected] C: C: what a wonderful message C: . S: 250 OK …

Bounce-To address


SMTP Error Handling Reliability: Key goal of SMTP If agent cannot deliver before sending OK:

Recipient not known, sender not allowed, queue full, etc.

Signal to Sender-SMTP by error code Permanent (5xx) or transient (4xx) error codes

On error in transfer to next mail server: Transient error (4xx): retransmit after reasonable delay May try alternate route (use secondary MX record)

If agent cannot deliver (after sending OK): Send Delivery Status Notifications (DSN) to bounce

address DSN has empty MAIL FROM field


Bounce Address: MAIL FROM or Return-Path?

Bounce address: where to send DSN Usually (agent is server): MAIL FROM:

address Bounce from recipient’s MUA:

No envelope, MAIL FROM: Bounce address copied to Return-Path: header


SMTP Traceback Delivery Status Notifications (DSN) depend on

bounce address (MAIL FROM / Return-Path:) What if bounce error is corrupted?

Possibly intentionally (to hide traces – e.g. spoofed email) Solution: Traceback message route Origin selects, puts unique Message-ID: Each SMTP-receiver <rec> adds header to

messagereceived: from <src> by <rec> for <dest>;

<date> <src>, <rec> : domain names and IP addresses

SMTP servers maintain logs to confirm / deny received: records Traceback conflict one of the pair lies


Relaying in SMTP Usually: Relaying is Inside sender / receiver

domain Receiving Relay: recipient in the domain

Authorized-sender Relay: from authenticated domain user, or from authenticated MTA / MSA in domain

Or: Direct from sender domain to receiver domain From outgoing MTA in sender domain (e.g. a.com)

To incoming MTA in recipient’s domain (e.g. b.com)

Traceable Relaying: Inside (sender/recipient) domain, or direct (sender to receiver domain)

Allows responding to traceback conflict


Relaying and Traceback Conflict Assume: Authorized-sender or receiving

relaying Direct from sender domain to receiver domain

Or: Inside sender / receiver domain

Consider traceback (e.g. of spoofed email)

No conflict using log files and traceback information, identify “real” source of email

Traceback conflict: If among servers within domain: detect corruption!

If sender domain denies: may refuse future connections

Caution: Can also be corruption of“our” server or a spoofed connection


Email Relaying Services Relaying within

domain is fine MUA to MSA MSA to MTA MTA to MDA

Sender-Receiver relaying is fine From sending MTA To receiving MTA

Open Relay: Not in sender and

receiver domains

AliceMUAAliceMUA

MSAMSA

BobMUABobMUA

MDAMDA

Domaina.com

Domainb.com

MTAA

MTAA

MTAB

MTAB

Open Relay MTA

Open Relay MTA


Open Relay Considered Harmful Open Relay: MTA that relays from and to outside

its domain Years Ago: Open relaying was critical for

connectivity (e.g. before DNS were adopted) Today: Often abused to send bad email (e.g.

spam) Upon traceback, Open Relay claims it got email from xxx Is xxx bad? Or is the Open Relay lying and faulty?

Most MTAs are not Open Relays Relay only inside domain or from sender’s domain

Many MTAs refuse email from Open Relays


Chapter 2 outline

2.1 Principles of app layer protocols clients and servers app requirements –

choosing transport srvc 2.6 Socket

programming 2.2 Web and HTTP

HTTP Overview Connections:




SMTP, POP3, IMAP 2.5 DNS 2.8 Building a Web

server 2.9 Content




DNS: Domain Name System

People: many identifiers: Name, ID#, e-mail

Internet hosts, routers: IP address (32 bit) - used

for addressing datagrams• One per network adapter

(may have two) Domain names: biu.ac.il Fully Qualified Domain

Name (FQDN) or “name”, e.g., gaia.cs.umass.edu

Names : used by humans

Q: map between IP addresses and names ?

Domain Name System: distributed names database

implemented in hierarchy of many name servers

application-layer protocol host, routers, name servers communicate to resolve names (address/name translation) Application-layer

protocol for Basic Internet function

complexity at network’s “edge”

Used to distribute other data as well


DNS

Why not centralize DNS? single point of failure traffic volume Proximity (close to

client) Maintenance and

security

doesn’t scale!

DNS services Hostname to IP

address translation Host aliasing

Canonical and alias names

Mail server aliasing Load distribution

Replicated Web servers: set of IP addresses for one canonical name


TLD and Authoritative Servers Top-level domain (TLD) servers: responsible

for com, org, net, edu, etc, and all top-level country domains uk, fr, ca, jp. Network solutions maintains servers for com TLD Educause for edu TLD

Authoritative DNS servers: organization’s DNS servers, providing authoritative hostname to IP mappings for organization’s servers (e.g., Web and mail). Maintained by organization or service provider


Root DNS Servers

com DNS servers org DNS servers edu DNS servers

poly.eduDNS servers

umass.eduDNS servers

yahoo.comDNS servers

amazon.comDNS servers

pbs.orgDNS servers

Distributed, Hierarchical Database

Client wants IP for www.amazon.com; 1st approx: Query root server to find com DNS server Query com DNS server (TLD) to get

amazon.com DNS server Query amazon.com DNS server to get IP

address for www.amazon.com


Local Name Server

Queries processed by a Local Name Server, not by DNS client

Does not strictly belong to hierarchy Each ISP (residential ISP, company,

university) has (at least) one. Also called “default name server”

When a host makes a DNS query, query is sent to its local DNS server Acts as a proxy, forwards query into

hierarchy.


DNS: Root name servers contacted by local name server that can not resolve name root name servers:

if name mapping not known: get from authoritative name server

return mapping to local name server

b USC-ISI Marina del Rey, CAl ICANN Marina del Rey, CA

e NASA Mt View, CAf Internet Software C. Palo Alto, CA

i NORDUnet Stockholm

k RIPE London

m WIDE Tokyo

a NSI Herndon, VAc PSInet Herndon, VAd U Maryland College Park, MDg DISA Vienna, VAh ARL Aberdeen, MDj NSI )TBD( Herndon, VA

13 root name servers worldwide


requesting hostcis.poly.edu

gaia.cs.umass.edu

root DNS server

local DNS serverdns.poly.edu

1

23

4

5

6

authoritative DNS serverdns.cs.umass.edu

78

TLD DNS server

Example

Host at cis.poly.edu wants IP address for gaia.cs.umass.edu


requesting hostcis.poly.edu

gaia.cs.umass.edu

root DNS server

local DNS serverdns.poly.edu

1

2

45

6

authoritative DNS serverdns.cs.umass.edu

7

8

TLD DNS server

3

Recursive queries

recursive query: puts burden of

name resolution on contacted name server

heavy load?

iterated query: contacted server

replies with name of server to contact

“I don’t know this name, but ask this server”


DNS: caching and updating records DNS mappings are kept and sent in resource records

(RR) DNS servers cache mappings (RRs)

cache entries timeout (disappear) after ttl seconds The ttl value is specified in each DNS record

Several different types of RR Used for domain name resolution And for other purposes… where the distribution

among DNS servers is useful• E.g. distribute black list of spam mail servers• Own domain (e.g. easy.10$.com), run DNS server,

define arbitrary DNS `subtree` under it: policy.easy.10$.com,…


DNS records

DNS: distributed db storing resource records (RR)

Type=NS name is domain (e.g. foo.com) value is IP address of

authoritative name server for this domain

Type=TXT value is some text

RR format: (name, value, type, ttl)

Type=A name is hostname value is IP address

Type=CNAME name is alias name for (real)

“canonical” name value www.ibm.com is really servereast.backup2.ibm.com

Type=MX value is name of mail server associated

with name

Other types… Problem: support by servers


Example: e-mailDNSServer

SenderMTA

RecipientMTA

Sender Recipient

Send mail

MX requestb.com

MX responsemail.b.com

Deliver mail

Helo sender.com 250 OkMAIL FROM: [email protected] 250 OKRCPT TO: [email protected] 250 OKDate: Sun, 25 Jul 2004 17:30Subject: Hello messageHello world

SRC IP=xx.xx.xx.xx

rDNS requestxx.xx.xx.xx

rDNS responsesender.com


DNS protocol, messagesDNS protocol : query and reply messages, both with same message formatUses UDP (except for large replies) [Why?]

Components: Header:

identification: 16 bits (match reply to query)

• Why is this needed?

4 flags: query/reply, recursion desired, recursion available, authoritative reply

Number of questions and reply records


Chapter 2 outline

2.1 Principles of app layer protocols clients and servers app requirements –

choosing transport srvc 2.6 Socket

programming 2.2 Web and HTTP

HTTP Overview Connections:




SMTP, POP3, IMAP 2.5 DNS 2.9 Content




Content Distribution Problem Clients are spread over the Net Using one centralized server is bad…

One server cannot serve all clients Shipping data across Net: overhead, congestion

Solutions: Distribute data as DNS records

• Used e.g. for black lists of spammers Use HTTP: data kept by proxies Load Balancing: many servers with same IP

• Hardware switch selects server randomly• Preserve TCP connections to same server

DNS gives IP address from pool or by location• Both load balancing and by proximity (close to client)


Content distribution networks (CDNs) CDN company provides

content distribution services

The content providers are the CDN customers.

Content replication CDN company installs

hundreds of CDN servers throughout Internet in lower-tier ISPs, close

to users CDN replicates its

customers’ content in CDN servers. When provider updates content, CDN updates servers

origin server in North America

CDN distribution node

CDN serverin S. America CDN server

in Europe

CDN serverin Asia


CDN example

origin server www.foo.com distributes HTML Replaces: http://www.foo.com/sports/ruth.gif

with

http://www.cdn.com/www.foo.com/sports/ruth.gif

HTTP request for

www.foo.com/sports/sports.html

DNS query for www.cdn.com

HTTP request for

www.cdn.com/www.foo.com/sports/ruth.gif

1

2

3

Origin server

CDNs authoritative DNS server

NearbyCDN server CDN company

cdn.com distributes gif files uses its authoritative

DNS server to route redirect requests Also for load balancing


Application architectures

Client-server Peer-to-peer (P2P) Hybrid of client-server and P2P


Client-server architectureserver:

always-on host permanent IP address server farms for scaling Sends responses to

requests

clients: Initiate communication with

server (sends requests) may be intermittently

connected may have static or dynamic

IP addresses do not communicate

directly with each other


Pure P2P architecture

no always-on server arbitrary end systems

directly communicate peers are intermittently

connected and change IP addresses

example: Gnutella

Highly scalable

But difficult to manage


Hybrid P2P: centralized directory

original “Napster” design

1) when peer connects, it informs central server: IP address content

2) Alice queries for “Hey Jude”, receives Bob IP address

3) Alice requests file from Bob

centralizeddirectory server

peers

Alice

Bob

1

1

1

12

3


Hybrid of client-server and P2PClients register, find peer via management

server(s)Napster:

File transfer P2P File search centralized:

• Peers register content at central server• Peers query same central server to locate content

Instant messaging: Chatting between two users is P2P Presence detection/location centralized:

• User registers its IP address with central server when it comes online

• User contacts central server to find IP addresses of buddies

Lecture 3: Application Layer1 מבוא לרשתות תקשורת : רמת היישום דואר...

Documents

Transcript of Lecture 3: Application Layer1 מבוא לרשתות תקשורת : רמת היישום דואר...