1

Some QoS Deployment Issues

Shumon HuqueUniversity of PennsylvaniaMAGPI GigaPoP

April 15th 2002 - NSF/ITR Scalable QoS Workshop

2

University of Pennsylvania network

Large research university in Philadelphia, PA 22,000 students, 4,000 faculty, 10,000 staff

48,000 registered IP addresses

200 switched subnets

Central routing between them and out to Internet and Internet2

3

MAGPI GigaPoP

An Internet2 GigaPoP

Value added services Commodity Internet transit Facilitator of regional edu/research initiatives

Subscribers UPENN, Lehigh U, Princeton U, PA county school

units, J&J Pharmaceuticals

4

MAGPI GigaPoP (cont)

External Connectivity Internet2

OC-12c POS to Abilene Commodity Internet

UUNET: OC-3 Cogent: Gigabit Ethernet Yipes: Gigabit Ethernet (rate limited) DCANet: Fast Ethernet

5

Who wants QoS?

University researchers QoS researchers in CS department Research applications needing strict guarantees

on latency, b/w, jitter etc

Networking staff (Univ and gigaPoP) Manage exploding b/w needs Enable new classes of applications

Eg. VoIP, video conferencing, streaming Run non-mission critical traffic at lower priority

Eg. File sharing apps, dorm traffic, bulk transfers

6

What types of QoS?

DiffServ in routing core and gigaPoP

Layer2 priority (802.1p) in the switched portions of the campus network

Mapping L3 QoS to/from L2 QoS

Signalling and admission control? RSVP intra-domain? Aggregate reservations &

map to Diffserv traffic class at edge? Bandwidth Broker signalling?

7

DiffServ

Types of forwarding behavior we are most interested in: EF (Expedited Forwarding) BE (Best Effort - default PHB) LBE/Scavenger (eg. QBSS) ABE - low delay form

8

Interdomain Internet QoS

Not very optimistic

Some ISPs are starting to offer services

Multiplicity of providers Need for them to run interoperable QoS

implementations Mechanisms to ask for QoS reservations across

administrative domains Peering/SLA issues

9

Interdomain Internet2 QoS

More optimistic

Typically one or a few QoS enabled I2 backbone networks (eg. Abilene)

Agreed upon QoS architecture

Common set of operational practices and procedures

Some provisioning procedures in place

Existing demand from researchers

10

Deployment Challenges

I2 backbone is an R&E network, but ..Universities are using it to transport production traffic between themAnd not just traffic associated with meritorious research applications (one of the original ideas)GigaPoP is a production network providing access to I2 and Commodity Internet

11

Deployment Challenges (2)

So, we need to be very careful about changes we introduce to the network to facilitate QoS

Don’t jeopardize existing production traffic

12

Router support for QoS

Not mature or well tested

Often the features are in experimental code trains, unsuitable for deployment in a production network

Marking, re-marking, policing, traffic shaping, appropriate queue scheduling disciplines etc

Insufficient #queues to support large scale service differentiation

Often software implementations of required queueing disciplines instead of hardware

Obviously this situation will improve in the future

13

Router code support (cont)

Example: Juniper routers4.x release: Can police DS BA’s but not much more

5.x release: More queue scheduling disciplines Per queue traffic shaping DSCP marking and re-marking DSCP based prioritization and forwarding

Eg. Assigning EF BA to a high priority queue Mapping of 802.1p to Layer-3 QoS

14

Parallel Network Infrastucture

Deploy parallel network infrastructure

Place QoS enabled routers on this

Researchers are happy, but ..

Cost prohibitive

15

QoS policy issues

Where does marking occur? Endstations First hop routers or switches Edge routers

Who’s allowed to mark? How to validate? Complexity of deploying policies Additional controls and checks to enforce the

policies Policy servers: COPS, bandwidth brokers etc

16

Inter-domain signalling

No suitable mechanisms today for end2end inter-domain signalling of QoS reservations, call admission control

Manual/static provisioning

Bandwidth brokers/SIBBS work ongoing

17

What we do today

To facilitate researchers doing wide-area QoS experimentation:

Conscious effort not to impede Provide research labs with an uncongested path

though campus/gigapop to QoS enabled Abilene network

Make sure intervening routers don’t mark or re-mark DSCP code points

18

Abilene QoS testbed

19

Bandwidth Management Alternatives

University has experienced rapidly increasing bandwidth requirements

Overprovision the campus network

Buy more commodity Internet bandwidth through the gigaPoP

Employ rate limiting where appropriate

Employ lightweight QoS, eg. LBE/Scavenger

20

Endstation problems

Network apps often unable to use available bandwidth because of problems on end-stations: Poorly designed applications, application protocols Insufficiently sized socket buffers Inefficient, insufficiently tuned network stacks Duplex mismatch MTU mismatch

Having QoS in the network does not address this class of performance problems

21

Conclusion

We’re interested in QoS

Too early to deploy end2end reservation based QoS in many production networks

Intra-domain QoS a near-term possibility Both reservation based and lightweight VoIP, degrading non-mission critical traffic

End2End Inter-domain QoS difficult Co-ordination, SLAs, inter-domain signalling