hgs/IWQoS 1

Quality of Service - 20 years oldand ready to get a job?

Henning SchulzrinneDept. of Computer Science

Columbia UniversityNew York, New York

(sip:)schulzrinne@cs.columbia.edu

IWQoS (Karlsruhe, Germany)

June 6th, 2001

June 2001

hgs/IWQoS 2

Overview

• why has QoS failed in the Internet?

• do we still need it?

• some thoughts on future traffic

• musings on TCP fairness

• what’s left to do?

June 2001

hgs/IWQoS 3

QoS

• the bad news

– hardly anybody uses QoS techniques

– talking about the same issues year after year

• the good news

– more than 3,000 papers since 1984

– limited TOS support in end systems emerging

– still need it – but rarely

June 2001

hgs/IWQoS 4

Why is QoS unpopular?

• need to admit failure – “bandwidth too cheap to meter”

• undemocratic: some traffic is more equal than other

• reminds you of your mom: no, you can’t have that 10 Mb/s now

• socialist: administer scarcity - we like SUVs (or to drive 100 mph)!

• “risky scheme”: security

• only displacement applications (such as telephony) need QoS

• requires cooperation: edge-ISP, transit ISPs, end systems

• snake oil: add QoS, lose half your bandwidth

June 2001

hgs/IWQoS 5

Why is QoS hard?

• dishonesty: we only talk about the beneficiaries

• network has become harder to evolve:

– network address translation

– firewalls

– high packetization overhead (VPNs, IPv6)

• to be useful, has to be nearly universally supported (“no, you can’t make calls toAS 123”)

• network QoS vs. business class model: “coach is empty, please refund fare”

• currently, the ISP interface is IP and BGP – adding a third one is a big deal

• new Internet service model: TCP client (inside) – server (outside)

– exception: peer-to-peer on college campuses

• network to host: you first, no, you first

• failure of IP QoS−→ MPLS

June 2001

hgs/IWQoS 6

Cost of aλ

$11.28

$2.60

$0.87

$2.78

$400.00

$1,200.00

$300.00

$1,900.00

cable laying

conduit laying

other construction

72 fiber pair cable

DWDM 80x10 Gb/s

SDH 10 Gb/s

optical amplifier

(Data: Telegeography)

June 2001

hgs/IWQoS 7

Retail cost of bandwidth

$ per kb/s per month

0.00

0.10

0.20

0.30

0.40

0.50

0.60

0.70

0.80

0.90

modem ISDN T1 T3/6 T3/21 T3/45

June 2001

hgs/IWQoS 8

Bandwidth is cheap

• T3: approx. $25k/month

• OC-n hardware cost rule: step = 4x bandwidth, 2x cost

• assume 50% average utilization

• cost: roughly, 0.2c/64 kb/s channel (vs. 5c/minute for US long distance)

June 2001

hgs/IWQoS 9

Transatlantic bandwidth

$0.0

$0.5

$1.0

$1.5

$2.0

$2.5

$3.0

$3.5

$4.0

$4.5

TAT-12/13 (Sep.1996)

Atlantic Crossing-1(May 1998)

Yellow/AtlanticCrossing-2 (Sep.

2000)

360atlantic (March2001)

FLAG Atlantic-1 (April2001)

Apollo (August 2002)

mio

.US

$/S

TM

-1

initial STM-1 full STM-1

June 2001

hgs/IWQoS 10

But bottlenecks are plenty. . .

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

Oct-99 Dec-99 Feb-00 Apr-00 Jun-00 Aug-00

Link

Util

izat

ion

Input Peak Rate

Output Peak Rate

Input Average Rate

Output AverageRate

(NORDUNet)

June 2001

hgs/IWQoS 11

Voice-over-IP as the savior of QoS?

• streaming media: looked promising, now mostly TCP

• 90-97% of bytes are TCP

• definitely not TCP: VoIP

• 2001: 6.2 billion minutes of international traffic (out of 140 billion total)

• most IP phones already support TOS bytes

• for reservation: embedded systems➠ simple, self-configuring protocols

June 2001

hgs/IWQoS 12

Example: Pingtel SIP phone

June 2001

hgs/IWQoS 13

Example: Cisco and 3Com SIP phones

Cisco 3Com

June 2001

hgs/IWQoS 14

QoS and VoIP

• users are accustomed to paying

• predictable traffic

• suitable for ABE (low delay or high throughput)

• harder: don’t have simple LEC–LD–LEC model

• primary consideration: reliability (< 1% call failure/blocking)

June 2001

hgs/IWQoS 15

Traffic

0

20,000

40,000

60,000

80,000

100,000

120,000

140,000

US local PSTN US interstate US intrastate Internet other public data private line

TB

/mon

th

A. Odlyzko and FCC. Data estimates for 2000; PSTN for 1998.

June 2001

hgs/IWQoS 16

Voice and data traffic

100

1000

10000

100000

1996 1997 1998 1999 2000 2001 2002

wor

ldw

ide

traf

fic

(Gb/

s)

datavoice

June 2001

hgs/IWQoS 17

The three-minute myth

Local calls are about 2.4 minutes on average, but long distance calls are muchlonger,about 8.9 minutes:

10

20

30

40

50

60

70

80

90

100

0 10 20 30 40 50 60

% le

ss th

an

Call duration (minutes)

dataexponential

June 2001

hgs/IWQoS 18

Calls get longer with distance

0.0

2.0

4.0

6.0

8.0

10.0

12.0

14.0

1-10 mi 22 mi 55 mi 124 mi 292 mi 430 mi 925 mi 1910 mi > 1910 mi

Distance

Avg

.dur

atio

n(m

in)

average 310 mi 7.8 min.

median 60 mi 3.0 min.

June 2001

hgs/IWQoS 19

What made other services successful?

VoIP is always happening next year . . .

email: available within self-contained community (CS, EE)

web: initially used for local information

IM: instantly available for all of AOL

All of these . . .

• work with bare-bones and variable connectivity (≥ 14.4 kb/s)

• had few problems with firewalls and NATs

• don’t require a reliable network

• work well with PCs

June 2001

hgs/IWQoS 20

Future traffic

• voice growing at 10%/year, Internet traffic at 100%

• with provisioning cycle, means that you need to run your network at half capacity

• TCP (and kin) is likely to continue to dominate:

– dominance of dial-up: only 13% DSL and cable modem➠ applications aretailored for 34 kb/s

– data on mobile devices (not just web browsing)

– video P2P

– movement of private-line networks to Internet VPNs

• interest is inasynchronouscommunication, not interactive

• exception: video games?

June 2001

hgs/IWQoS 21

Lessons learned

• queueing delay doesn’t matter if you have OC-3 links

• generality can hurt:

– receiver diversity: complexity, not that useful in practice

– multicast orientation−→ very little multicast

– diversity of signaling protocols can be beneficial

• try to support niche applications, e.g., high-quality video conferencing (ISDNreplacement)

June 2001

hgs/IWQoS 22

QoS is about reliability

• typical utilization: 30% – but not during restoration

• can’t sell premium service that’s unavailable one day a year

• cablewill be cut once per year/200 km

• consistent 5% packet loss is much better than 5% probability that network isunavailable for seconds

• who are you going to ask for a refund?

June 2001

hgs/IWQoS 23

BGP Convergence Times

(From Abha Ahuja’s IETF50 plenary talk and Geoff Huston’s talk)

Failures, Fail-overs and Repairs

June 2001

hgs/IWQoS 24

Reliability Issues

• −: software updates require “scheduled downtime”

• +: but signaling servers can be made redundant much easier than SS7 SCPs

• BGP convergence times of severalminutes: 2 minutes to withdraw routes, 30minutes to advertise routes

• “80% of withdraws take more than a minute”

• no clear IP reliability definition – reachability of any node? some large subset?“local calls”?

June 2001

hgs/IWQoS 25

Have we explored the design space?

flow aggregate

signaling flow IntServ –

aggr. ? DiffServ + RNAP/BGRP/. . . ?

June 2001

hgs/IWQoS 26

Congestion pricing is back

Myth: “people want fixed prices” – but congestion pricing is common:

• Traffic:

Thanks to advances in technology, it’s becoming easier to use tolls totame congestion. Solo drivers on Interstate 15 in San Diego, for example,can purchase the privilege of driving on the road’s high-occupancy-vehiclelanes. The price changes as often as every six minutes and depends on theamount of congestion on the HOV lanes. When traffic is light, the toll,which is collected using vehicle transponders and overhead antennas, goesas low as 50 cents; when it’s heavy, it may cost as much as $8.00. If youcarpool, the ride is free. (USN&WR)

• electricity, airline and hotel pricing

• many T3’s are priced on 95th percentile of five-minute intervals

• not for 10c items – 1 Mb/s video conferences, not phone calls

• More correctly: people want to know what they’re being charged

June 2001

hgs/IWQoS 27

Example: RNAP (Resource Negotiation and Pricing)

������

��������

��������

������������

�����������������

���������������

������������

��������������������

NRN NRNNRN

B1

B2B3

B5

B6

B7B8

S1

Transit NetworkAccess Network Access Network

S2

B4

R1

RNAP messages

Intra-domain Messages

R2

HRN

HRNHRN

HRN

June 2001

hgs/IWQoS 28

RNAP

• many applications have a wide operating range

• per-bit value decreases with bandwidth

• provide incentive to use only resources needed

• compute price by tatonnement orM -bid auction

• optimize user perceived value = utility− price

• take “holding costs” into account for multiple classes

• allow range of commitment times, typically minutes

• allow ISP “buy back” if advance purchase price was low

June 2001

hgs/IWQoS 29

What’s needed?

• forget scalability – most successful applications don’t scale (e.g., AOL IM)

• likely to be only for tiny fraction of traffic (but much larger revenue fraction. . . )

• high-value streams (conferences, VPNs)➠ advance reservation needed

• prime QoS determinant is path reliability – zero bandwidth is not

• useful measurements – not just monthly averages

• reservations for one or two bottlenecks:

– access links

– wireless links

• single sign-on (e.g., Microsoft Passport)

June 2001

hgs/IWQoS 30

The myth of TCP fairness

• = non-TCP flow should not send more than a TCP flow under similar networkconditions

• over what time scale – short vs. long connections

• userspay for network access, not flows – longer-term user benefit equalization?

• should a multicast stream benefiting a million users get the same throughput as asingle TCP stream?

• parallel connections to one or several web servers

• quirks: TCP throughput decreases with RTT

• very little UDP traffic is TCP-friendly

(see also Christophe Diot, J.Y. Le Boudec)

June 2001

hgs/IWQoS 31

Challenges

• optimizing components−→ buildable system

• roughly standardized service classes – allow settlements and advertising

• separation intra-domain vs. inter-domain

• security – new opportunity for massive DOS

• integrated services➠ use second channel (PSTN)

• useful busy-hour measurements, not just monthly averages

• reliability measurements

• simple, manageable signaling with advance reservation

June 2001

hgs/IWQoS 32

Conclusion

• from central issue➠ niche application

• QoS is primarily reliability, then packet loss, then queueing delay

• sell reliability and predictability, not delay jitter

• need BGP fix plus “nailed down” circuits:

– routes must stay in QoS-enabled network

– faster recovery – try backup route on failure

– doesn’t have to scale

– two types of carriers: “classical IP” vs. “voice heritage”?

June 2001

hgs/IWQoS 33

Cost of Bandwidth (per km)

cable laying $65,000

conduit laying $15,000

conduits, manholes $ 5,000

24 fiber pair cable $ 7,000

72 fiber pair cable $16,000

DWDM, 80x10 Gb/s (perλ) $ 550

10 Gb/s OC-192 terminal (perλ) $ 1,200

optical amplifier $ 300

(Source: Telegeography)

June 2001