�

Service Disciplines For GuaranteedPerformance Service in Packet�Switching

NetworksHUI ZHANG� MEMBER� IEEE

Abstract� While today�s computer networks supportsonly best�e�ort service� Future packet�switching integrated�

services networks will have to support real�time communi�cation services that allow clients to transport informationwith performance guarantees expressed in terms of delay�delay jitter� throughput and loss rate� An important issuein providing guaranteed performance service is the choice ofthe packet service discipline at switching nodes�

In this paper� we survey several service disciplines thatare proposed in the literature to provide per�connectionend�to�end performance guarantees in packet�switching net�works� We describe their mechanisms� their similarities anddi�erences� and the performance guarantees they can pro�

vide� Various issues and tradeo�s in designing service dis�ciplines for guaranteed performance service are discussed�and a general framework for studying and comparing thesedisciplines are presented�

I� Introduction

Communication systems have been revolutionized bytechnological advances in the last decade� The speed andcapacity of various components in a communication sys�tem� such as transmissionmedia� switches� memory� proces�sors� have all followed technological curves that have growneither linearly or exponentially over the last ten years �����At the periphery of the network� driven by the same un�derlying technology microelectronics� the capability ofcomputers has been drastically increased while the costhas been signicantly reduced� The advent of high speednetworking has introduced opportunities for new applica�tions such as video conferencing� scientic visualization andmedical imaging� These applications have stringent per�formance requirements in terms of throughput� delay� de�lay jitter and loss rate� Current packet�switched networks�such as the Internet� o er only a best�e ort service� wherethe performance of each session can degrade signicantlywhen the network is overloaded� There is an urgent needto provide network services with performance guaranteesand to develop algorithms supporting these services�One of the most important issues in providing guaran�

teed performance services is the choice of the packet servicediscipline at the switch� In a packet�switching network�packets from di erent connections will interact with eachother at each switch� without proper control� these interac�tions may adversely a ect the network performance expe�rienced by clients� The service disciplines at the switchingnodes� which control the order in which packets are ser�viced� determine how packets from di erent connections

Hui Zhang is with the School of Computer Science� CarnegieMellonUniversity� ���� Forbes Avenue� Pittsburgh� PA ��������

interact with each other�

Although service disciplines and associated performanceproblems have been widely studied in the contexts of hardreal�time systems and queueing systems� results from thesestudies are not directly applicable in the context of pro�viding guaranteed performance service in packet�switchingnetworks� Analyses of hard real�time systems usually as�sume a single server environment� periodic jobs� and thejob delay bounded by its period ����� However� the net�work tra�c is bursty� and the delay constraint for eachindividual connection is independent of its bandwidth re�quirement� In addition� bounds on end�to�end performanceneed to be guaranteed in a networking environment� wheretra�c dynamics are far more complex than in a singleserver environment� Queueing analysis is often intractablefor realistic tra�c models� Also� classical queueing analy�ses usually study average performance for aggregate tra�c����� ����� while for guaranteed performance service per�

formance bounds need to be derived on a per�connection

basis ����� ����� In addition to the challenge of provid�ing end�to�end per�connection performance guarantees toheterogeneous and bursty tra�c� service disciplines mustbe simple so that they can be implemented at very highspeeds�

Recently� a number of new service disciplines that areaimed to provide per�connection performance guaranteeshave been proposed in the context of high�speed packet�switching networks ����� ����� ����� ����� ����� ����� ����� �����Also� new analysis techniques have been proposed to ad�dress the performance issues of these disciplines ���� ���� �������� ����� ����� ����� ����� ����� ����� ����� ����� ����� ����� ���������� ����� In this paper� we give an overview of the pro�posed service disciplines� and discuss the issues and trade�o s in designing service disciplines in providing guaranteedperformance service in packet�switching networks�

The rest of the paper is organized as follows� In Sec�tion II� we review general issues associated with providingperformance guarantees in packet�switching networks anddemonstrate the important role of service disciplines in thenetwork control architecture� Sections III and IV discussthe two classes of service disciplines� work�conserving andnon�work�conserving disciplines respectively� In each of thetwo sections� a brief description of each discipline is rstgiven before a general framework is presented to show thesimilarities and di erences among them� The end�to�enddelay characteristics� bu er space requirement� and im�plementation issues of each discipline are then discussed

�

within the framework� In Section V� we summarize the pa�per by providing a taxonomy for classifying and comparingexisting solutions�

II� Background

A� Network Model

We consider a network with arbitrary topology of linksand switches�� Link are assumed to have bounded delay�Switches are assumed to be �non�blocking�� i�e�� when pack�ets arrive at an input link� they can be routed directly tothe appropriate output links without switching con�icts�Packets destined for di erent output links do not interferewith each other� and queueing occurs only at the outputports of the switch ����� With these assumptions� a con�nection in such a network can be modeled as traversing anumber of queueing servers� with each server modeling theoutput link of a switch� The network supports variable�sizepackets�

B� Service Model

We consider the following guaranteed performance ser�vice model� before the communication starts� the clientneeds to specify its tra�c characteristics and desired per�formance requirements� When the network accepts theclient�s request� it guarantees that the specied perfor�mance requirements will be met provided that the clientobeys its tra�c specication�In this model� the guaranteed performance service de�

nes a contractual relationship between the communicationclient and the network ����� ����� ����� the network promisesto fulll its obligation �guaranteeing the performance forthe client�s tra�c� only if the client honors its own partof the contact �not sending more data than declared�� Inaddition� the network may reject the client�s request dueto lack of resources or administrative constraints� In itsbasic form� the contract is signed before data transfer dur�ing a connection establishment process and is kept e ectivethroughout the life time of the connection ����� To increasedynamicity and �exibility� the model can also be extendedto allow contract to be modied in the middle of a connec�tion �����Recently� a new service model called the predicted ser�

vice was proposed ���� There are two important di erencesbetween the predicted service and the guaranteed perfor�mance service discussed in this paper� First� while theadmission control� which decides whether there are enoughresources within the network to accept a new connection� isused to support both types of service� the criteria are quitedi erent� In order to decide whether there are enough re�sources� one has to know the current network load� Forpredicted service� the current network load is based onmeasurement� for guaranteed service� it is based on pre�specied characterization of existing connections� Since themeasured network load may vary� the service commitment

�In the literature� the term �switch� is used in the context of ATMnetworks� while �gateway� or �router� is used more often in an in�ternetworking environment In this research� we will call switchingelements as �switches�

by predicted service is less reliable� Secondly� in the pre�dicted service� the delay bound or playback point for a con�nection is provided by the network and may vary due to thenetwork load �uctuation� It is assumed that applicationsusing the predicted service can adapt to the changing ofthe playback point and tolerate infrequent service disrup�tions� In the guaranteed performance service model� delaybound is specied by the application and does not changeduring the life time of the connection without the explicitrequest by the client�

B�� Performance Parameters in Guaranteed Service

The most important clauses in the service contract arethe specications of performance requirements and tra�ccharacteristics� For the performance parameters� the singlemost important one is the end�to�end delay bound� whichis essential for many applications that have stringent real�time requirements� While throughput guarantee is alsoimportant� it is provided automatically with the amountspecied by the tra�c characterization �Section II�B����Another important parameter is the end�to�end delay jit�ter bound� The delay jitter for a packet stream is dened tobe the maximum di erence between delays experienced byany two packets ����� ����� For continuous media playbackapplications� the ideal case would be that the network in�troduces only constant delay� or zero delay�jitter� Having abounded delay�jitter service from the network makes it pos�sible for the destination to calculate the amount of bu erspace needed to eliminate the jitter� The smaller the jitterbound� the less amount of bu er space is needed� Since itis more important to provide end�to�end delay and delay�jitter bounds than average low delay for guaranteed serviceclass� packets arriving too earlier may not even be desir�able in such a environment� In fact� the earlier a packetarrives before its delay bound or playback point� the longerit needs to occupy the bu er� This is one of the most impor�tant di erences between the performance requirements ofthe guaranteed�performance service and the best�e ort ser�vice provided by the traditional computer networks� per�formance bounds are more important for the guaranteedservice while average performance indices are more impor�tant for the best�e ort service�A third important parameter is the loss probability�

Packet loss can occur due to bu er over�own or delaybound violation� A statistical service ����� ����� ���� al�lows a non�zero loss probability while while a deterministicservice guarantees zero loss� With a deterministic service�all packets will meet their performance requirements evenin the worst case� With a statistical service� stochasticor probabilistic bounds are provided instead of worst casebounds� Statistical service allows the network to overbookresources beyond the worst�case requirements� thus mayincrease the overall network utilization by exploiting sta�tistical multiplexing gain�

B�� Tra�c Models in Guaranteed Service

Although there is a general consensus within the researchcommunity on the �super� set of parameters to character�

�

ize performance requirements� there is no agreement onwhich tra�c model or which set of tra�c parameters shouldbe adopted� In the traditional queueing theory literature�most models are based on stochastic processes� Amongthe more popular ones are the Poisson model for data �����on�o model for voice sources ��� and more sophisticatedMarkovian models for video sources ����� A good surveyfor the probabilistic models for voice and video sourcesis presented in ����� In general� these models are eithertoo simple to characterize the important properties of thesource or too complex for tractable analysis�

Recently� several new models are proposed to bound

the tra�c rather than characterize the process exactly�Among them are� �Xmin� Xave� I� Smax� ����� ��� �� �����r� T � ����� ����� and the D�BIND model ����� A tra�cstream satises the �Xmin�Xave� I� Smax� model if theinter�arrival time between any two packets in the streamis more than Xmin� the average packet inter�arrival timeduring any interval of length I is more than Xave� and themaximum packet size is less than Smax� Alternatively� atra�c stream satises the ��� �� model if during any inter�val of length u� the number of bits in that interval is lessthan �� �u� In the ��� �� model� � and � can be viewed asthe maximumburst size and the long term bounding rate ofthe source respectively� Similarly� a tra�c stream is said tosatisfy �r� T � model if no more than r�T bits are transmittedon any interval of length T � Rather than using one bound�ing rate� the Deterministic Bounding Interval�Dependent�D�BIND� model uses a family of rate�interval pairs wherethe rate is a bounding rate over the corresponding inter�val length� The model captures the intuitive property thatover longer interval lengths� a source may be bounded bya rate lower than its peak rate and closer to its long�termaverage rate�

In each of the above models� the exact tra�c pattern fora connection is unknown� the only requirement is that thevolume of the tra�c be bounded in certain ways� Suchbounding characterizations are both general and practi�cal� They can characterize a wide variety of bursty sources�In addition� it is su�cient for resource management algo�rithms to allocate resources by knowing just the bounds onthe tra�c volume�

A bounding characterization can either be deterministicor stochastic� A bounding deterministic tra�c characteri�zation denes a deterministic tra�c constraint function� Amonotonic increasing function bj��� is called a determinis�tic tra�c constraint function of connection j if during anyinterval of length u� the number of bits arriving on j dur�ing the interval is no greater than bj�u�� More formally� letAj�t�� t�� be the total number of bits arrived on connec�tion j in the interval of �t�� t��� bj��� is a tra�c constraintfunction of connection j if Aj�t� t � u� � bj�u�� �t� u � ��Notice that bj��� is a time invariant deterministic boundsince it constrains the tra�c stream over every intervalof length u� For a given tra�c stream� there are an in�nite number of valid tra�c constraint functions� out ofwhich� a deterministic tra�c model denes a parameter�ized family� All of the above tra�c models have corre�

sponding tra�c constraint functions� For example� thetra�c constraint function of ��� �� model is � � �u� Thetra�c constraint can also be stochastic� In ����� a fam�ily of stochastic random variables are used to characterizethe source� Connection j is said to satisfy a characteri�zation of f�Rt��j � t��� �Rt��j� t��� �Rt��j� t�����g� where Rti�j

are random variables and t� � t� � � � � are time intervals� ifRti�j is stochastically larger than the number of bits gener�ated over any interval of length ti by source j� This modelis extended in ���� by explicitly considering the interval�dependent property of the source� over longer intervallengths� a source may be bounded by a rate lower than itspeak�rate and closer to its long�term average� The resultedmodel is called Stochastic Bounding Interval Dependent orS�BIND model� Another related tra�c model is the Expo�nentially Bounded Burstiness �E�B�B�� process proposed in����� ����� A source is said to be E�B�B� with parameters���A� �� if PrfA�s� s � t� � �t � �g � Ae��� �� � � ands� t � � where random variable A�t�� t�� denotes the totalnumber of bits generated by a source in the interval �t�� t���In this paper� we assume that a communication client

uses a deterministic bounding tra�c model to specify itstra�c if it requests a deterministic service and use astochastic bounding tra�c model to specify its tra�c ifit requests a statistical service�

C� Tra�c Management Algorithms

In packet�switching networks� there is the possibility thatthe aggregate rate of the input tra�c to the network �ora portion of the network� temporarily exceeds the capac�ity of the network� in which cases packets may experi�ence long delays or get dropped by the network� This iscalled congestion� Although networks are expected to be�come even faster� the problem of congestion is not likelyto go away ����� Various congestion control or tra�c man�agement algorithms have been proposed in the literature�These solutions can be classied into two classes� reactive�or feedback control schemes ����� ����� and proactive� orresource reservation algorithms ����� ����� �����Reactive approaches detect and react dynamically to

congestion inside the network by relying on the feedbackinformation from the network� while proactive approacheseliminate the possibility of congestion by reserving networkresources for each connection� From the point of view ofcontrol time scale� reactive approaches operate in a timescale of several round�trip times since the length of the in�terval between the time when the congestion is detectedand the time when the congestion signal is passed back tothe source is on the order of one round�trip time� Proac�tive approaches� on the other hand� operate on at least twotimescales� connection level and packet level� At the con�nection level� when a new connection request comes in� aset of connection admission control conditions are testedat each switch� The new connection is accepted only ifthere are enough resources to satisfy the requirements ofboth the new connection and existing connections� At thepacket level� the packet service discipline at each switchselects which packet to transmit next by discriminating

�

Input Link 1

Input Link 2

Switch

Non-Guaranteed Traffic

Guaranteed Traffic

Scheduler 1

Scheduler 2 Output Link

Fig � Servicing both Guaranteed�Service and Non�Guaranteed Tra�c

packets based on their performance requirements� Usually�di erent service disciplines need di erent admission controlalgorithms� A complete solution needs to specify both theservice discipline and the associated connection admissioncontrol conditions�

Connection Lifetime

min.ms.

Roundtrip TimePacket

us.

Service

Fig � Control Timescales For Tra�c Management Algorithms

The three timescales used by tra�c management al�gorithms are illustrated in Figure �� While a reactiveapproach is suitable for supporting best�e ort service� aproactive tra�c management architecture is better forguaranteed performance service� The two approaches canco�exist in an integrated services network�

D� Service Disciplines

As can be seen from Figure �� packet service disciplinesoperate at the smallest time scale� or with the highest fre�quency� Together with connection admission control algo�rithms� they provide the two most important componentsin a proactive tra�c management architecture� While con�nection admission control algorithms reserve resources dur�ing connection establishment time� packet service disci�plines allocate resources according to the reservation dur�ing data transfer� Three types of resources are being allo�cated by service disciplines ����� bandwidth �which pack�ets get transmitted�� promptness �when do those packetsget transmitted� and bu er space �which packets are dis�

carded�� which� in turn� a ects three performance parame�ters� throughput� delay� and loss rate�Even in reactive or feedback�based tra�c management

architecture� appropriate scheduling at packet switches willmake end�to�end control more e ective ����� ����� In therest of the paper� we consider architecture shown in Fig�ure � for service disciplines in integrated services networks�There are separate queues and service policies for guar�anteed service and other packets� Best�e ort packets aretransmitted only when no packets from the guaranteed ser�vice queue are available for transmission� It should be no�ticed that the two queues in Figure � are logical ones� De�pending on the service discipline� each logical queue cancorresponds to multiple physical queues� For example� if

a Static Priority discipline with n priority levels is usedfor guaranteed tra�c and a round robin discipline with m

classes is used for non�guaranteed tra�c� the number ofphysical queues is n �m at each output port� In this pa�per� we will focus on the service disciplines for guaranteedtra�c�

Although it is possible to build a guaranteed performanceservice on top of a vast class of service disciplines ����� wewould like a service discipline to be e�cient� protective��exible� and simple�

E�ciency� To guarantee certain performance require�ments� we need a connection admission control policy tolimit the guaranteed service tra�c load in the network� orlimit the number of guaranteed service connections thatcan be accepted� A service discipline is more e�cient thananother one if it can meet the same end�to�end performanceguarantees under a heavier load of guaranteed service traf�c� An e�cient service discipline will result in a higherutilization of the network�

Protection� Guaranteed service requires that the net�work protects well�behaving guaranteed service clients fromthree sources of variability� ill�behaving users� network load�uctuation and best�e ort tra�c� It has been observedin operational networks that ill�behaving users and mal�functioning equipments may send packets to a switch ata higher rate than declared� Also� network load �uctua�tions may cause a higher instantaneous arrival rate from aconnection at some switch� even though the connection sat�ises the tra�c constraint at the entrance to the network�Another variability is the best�e ort tra�c� Although theguaranteed service tra�c load is limited by connection ad�mission control� best�e ort packets are not constrained� Itis essential that the service discipline should meet the per�formance requirements of packets from well�behaving guar�anteed service clients even in the presence of ill�behavingusers� network load �uctuation and unconstrained best�e ort tra�c�

Flexibility� The guaranteed performance service needsto support applications with diverse tra�c characteris�tics and performance requirements� Scientic visualizationand medical imaging will have very di erent characteris�tics from video� Even for video� conferencing applications�movie applications and HDTV require di erent qualities ofservice� Other factors� such as di erent coding algorithmsand di erent resolutions� also contribute to the diversity ofvideo requirements� Because of the vast diversity of traf�c characteristics and performance requirements of existingapplications� as well as the uncertainty about future appli�

�

cations� the service discipline should be �exible to allocatedi erent delay� bandwidth and loss rate quantities to dif�ferent guaranteed service connections�

Simplicity� The service discipline should be both concep�tually simple to allow tractable analysis and mechanicallysimple to allow high speed implementation�

III� Work�Conserving Service Disciplines

A service discipline can be classied as either work�conserving or non�work�conserving� With a work�conserving discipline� a server is never idle when there isa packet to send� With a non�work�conserving discipline�each packet is assigned� either explicitly or implicitly� an el�igibility time� Even when the server is idle� if no packets areeligible� none will be transmitted� As will be shown later inthis paper� whether a service discipline is work�conservinga ects the end�to�end delay analysis� bu er space require�ments� and delay�jitter characteristics�

In this section� we will study the work�conserving dis�ciplines� Delay Earliest�Due�Date �Delay�EDD� ����� ���������� Virtual Clock ����� Fair Queueing �FQ� ���� and itsweighted version �WFQ� also called Packetized GeneralizedProcessor Sharing �PGPS� ����� Self�Clocked Fair Queueing�SCFQ� ����� and Worst�case Fair Weighted Fair Queueing�WF�Q� ���� We rst describe each of these disciplines�then present a framework to show the similarities and dif�ferences among them� Finally� we examine the end�to�enddelay characteristics and bu er space requirements for eachof them� Non�work�conserving disciplines will be discussedin Section IV�

A� Virtual Clock

The Virtual Clock ���� discipline aims to emulate theTime Division Multiplexing �TDM� system� Each packetis allocated a virtual transmission time� which is the timeat which the packet would have been transmitted were theserver actually doing TDM� Packets are transmitted in theincreasing order of virtual transmission times�

Figure � gives a simple example to illustrate how VirtualClock works� In the example� there are three connectionssharing the same output link� All three connections specifytheir tra�c characteristics and reserve resources accord�ingly� Connections � has an average packet inter�arrivaltime of � time units� connection � and � have an averagepacket inter�arrival time of � time units� For simplicity� as�sume packets from all the connections have the same size�and the transmission of one packet takes one time unit�Hence� each of connections � and � reserve ��� of the linkbandwidth� while connection � reserves ��� of the linkbandwidth� The arrival pattern of the three connectionsare shown in the rst three timelines� As can be seen� con�nections � and � send packets at higher rates than reserved�while connection � sends packet according to the speciedtra�c pattern� The fourth timelines show the service or�der of packets when the service discipline is FCFS� In thiscase� even though connection � reserves more resources� themisbehaviors of connections � and � a ect its performance�

connection 3

connection 1:average inter-arrival: 2 units

Virtual Clock service order

service orderFirst-Come-First-Served

average inter-arrival: 5 unitsconnection 2:

virtual times

average inter-arrival: 5 units

Fig � Comparison of Virtual Clock and FCFS

The Virtual Clock algorithm assigns each packet a vir�tual transmission time based on the arrival pattern and thereservation of the connection to which the packet belongs�The fth timeline shows the virtual transmission time as�signment� The transmissions of packets are then orderedby the virtual transmission times� The service order ofpackets under the Virtual Clock discipline is shown in thesixth timeline� Notice that although connections � and �are sending packets at higher rates� the Virtual Clock algo�rithm ensures that each well�behaving connection� in thiscase connection �� gets good performance�

B� WFQ and WF�Q

WFQ and WF�Q are two packet policies that try to ap�proximate the same Fluid Fair Queueing �FFQ� or Gener�alized Processor Sharing �GPS� policy� FFQ is a generalform of the head�of�line processor sharing service discipline�HOL�PS� ����� With HOL�PS� there is a separate FIFOqueue for each connection sharing the same link� Dur�ing any time interval when there are exactly N non�emptyqueues� the server serves the N packets at the head of thequeues simultaneously� each at a rate of one N th of thelink speed� While a HOL�PS server serves all non�emptyqueues at the same rate� FFQ allows di erent connectionsto have di erent service shares� A FFQ is characterized byN positive real numbers� �� �� � � � � N � each correspond�ing to one queue� At any time � the service rate for anon�empty queue i is exactly �iP

j�B���

C where B� � the

set of non�empty queues and C is the link speed� There�fore� FFQ serves the non�empty queues in proportion totheir service shares� FFQ is impractical as it assumes thatthe server can serve all connections with non�empty queuessimultaneously and that the tra�c is innitely divisible� Ina more realistic packet system� only one connection can re�ceive service at a time and an entire packet must be served

�

before another packet can be served�

There are di�erent ways of approximating FFQ servicein a packet system� Among them� the most well�knownone is the Weighted Fair Queueing discipline �WFQ� ���also known as Packet Generalized Processor Sharing orPGPS �� �� In WFQ� when the server is ready to transmitthe next packet at time � � it picks� among all the pack�ets queued in the system at � � the �rst packet that wouldcomplete service in the corresponding FFQ system if noadditional packets were to arrive after time � �

While WFQ uses only �nish times of packets in the FFQsystem� WF�Q uses both start times and �nish times ofpackets in the FFQ system to achieve a more accurate em�ulation� In WF�Q� when the next packet is chosen for ser�vice at time � � rather than selecting it from among all thepackets at the server as in WFQ� the server only considersthe set of packets that have started �and possibly �nished�receiving service in the corresponding FFQ system at time� � and selects the packet among them that would completeservice �rst in the corresponding FFQ system�

WFQ service order

connection 2

connection 1

connection 11

WF Q service order2

... ...

Fig� �� WFQ and WF�Q

The following example� shown in Figure �� illustratesthe di�erence between WFQ and WF�Q� For simplicity�assume all packets have the same size of and the linkspeed is � Also� let the guaranteed rate for connection be ���� and the guaranteed rate for each of the other �connections be ����� In the example� connection sends back�to�back packets starting at time � while each ofall the other � connections sends only one packet at time�� If the server is FFQ� it will take time units to serviceeach of the �rst � packets on connection � one time unit toservice the th packet� and � time units to service the �rstpacket from another connection� Denote the kth packet onconnection j to be pkj � then in the FFQ system� the startingand �nishing times are �k � � and k respectively forpk�� k � � � � �� � and respectively for p��� � and � and� respectively for p�j � j � � � ��

For the same arrival pattern� the service orders underthe packet WFQ and WF�Q systems are di�erent� UnderWFQ� since the �rst � packets on connection �pk�� k � � � � �� all have FFQ �nish times smaller than packets on

other connections �� the server will service � packets onconnection back to back before service packets from otherconnections�

Under WF�Q� At time �� all packets at the head of eachconnection�s queue� p�i � i � � � � � � � have started servicein the FFQ system� Among them� p�� has the smallest �nishtime in FFQ� so it will be served �rst in WF�Q� At time� there are still packets at the head of the queues� p��and p�i � i � � � � � � � Although p�� has the smallest �nishtime� it will not start service in FFQ until time � therefore�it won�t be eligible for transmission at time � The other� packets have all started service at time � at the FFQsystem� thus are eligible� Since they all �nish at the sametime in the FFQ system� the tie�breaking rule of givinghighest priority to the connection with the smallest numberwill yield p�� as the next packet for service� At time �� p��becomes eligible and has the smallest �nish time amongall packets� thus it will start service next� The rest of thesample path for the WF�Q system is shown in Figure ��

There are two noteworthy points� First� at any giventime � � the accumulated service provided for each connec�tion �the total amount of bits transmitted� by either packetsystem never falls behind the �uid FFQ system by morethan one packet size� Such a relationship with FFQ and thefact that end�to�end delay bounds can be provided in FFQare the basis for establishing end�to�end delay bounds inWFQ and WF�Q� Also� since in the worst case both WFQand WF�Q can fall behind FFQ by the same amount ofservice� they provide the same end�to�end delay bounds�However� as shown in the example� the service order un�der WFQ and WF�Q can be quite di�erent� Even thoughWFQ cannot fall much behind FFQ in terms of service� itcan be quite far ahead of the FFQ system� In the example�by the time �� � packets on connection have been servedin the WFQ system� while only � packets have been servedin the FFQ system� The discrepancy between the serviceprovided by WFQ and FFQ can be even larger when thereare more connections in the system� In contrast� WF�Qdoes not have such a problem� In the above example� bythe time �� WF�Q will have served � packets� exactly thesame as FFQ� In fact� it can be shown that the di�erencebetween the services provided by WF�Q and FFQ is alwaysless than one packet size� Therefore� WF�Q is the most ac�curate packet discipline that approximates the �uid FFQdiscipline�

Even though the di�erence between WFQ and WF�Qdoes not a�ect the end�to�end delay bounds they provide�it is shown in �� that such a di�erence may have importantimplications if they are used to provide best�e�ort services�

C� Self�Clocked Fair Queueing

Both WFQ and WF�Q need to emulate a reference FFQserver� However� maintaining the reference FFQ server iscomputationally expensive� One simplier packet approx�

�The FFQ �nish time of packet p���

is ��� the same as that of packets

p�j� j � � � � � � If we adopt the following tiebreaking policy in which

the packet from the connection with the smallest connection numberhas a higher priority� packet p��

�will be served before p�

j� j � � � � ��

�

imation algorithm of FFQ is Self�Clocked Fair Queueing�SCFQ� �� also known informally as �Chuck�s Approxi�mation� ��� The exact algorithm of SCFQ and the ex�amples illustrating the di�erence between WFQ and SCFQwill be given in Section III�E�

D� Delay�Earliest�Due�Date

Delay�Earliest�Due�Date algorithm or Delay�EDD ��� isan extension to the classic Earliest�Due�Date�First �EDDor EDF� scheduling ���� In the original EDD� each packetfrom a periodic tra�c stream is assigned a deadline and thepackets are sent in order of increasing deadlines� The dead�line of a packet is the sum of its arrival time and the periodof tra�c stream� In Delay�EDD� the server negotiates a ser�vice contract with each source� The contract states that ifa source obeys its promised tra�c speci�cation� such as apeak and average sending rate� then the server will providea delay bound� The key lies in the assignment of deadlinesto packets� The server sets a packet�s deadline to the timeat which it should be sent had it been received according tothe contract� This is just the expected arrival time addedto the delay bound at the server� For example� if a clientassures that it will send packets every �� seconds� and thedelay bound at a server is second� then the kth packetfrom the client will get a deadline of ��k� �

E� Framework for Work�Conserving Disciplines

Virtual Clock� Delay�EDD� WFQ� WF�Q� and SCFQ alluse a similar sorted priority queue mechanism� In such amechanism� there is a state variable associated with eachconnection to monitor and enforce its tra�c� Upon ar�rival of each packet from that connection� the variable isupdated according to �a� the reservation made for the con�nection during the connection establishment time and� �b�the tra�c arrival history of this connection and�or otherconnections during the data transfer� The packet is thenstamped with the value of the state variable for the con�nection to which it belongs� The stamped value is usedas a priority index� Packets are served in the order of in�creasing priority index values� This is shown in Figure ��WF�Q also needs additional mechanism to mark whetherpackets are eligible for transmission� As will be discussedin Section IV� this can be implemented with a calendarqueue�

In Virtual Clock� the state variable is called auxiliaryVirtual Clock �auxV C�� in WFQ� WF�Q� and SCFQ� itis called the Virtual Finish Time �F �� in Delay�EDD� itis called Expected Deadline �ExD�� In all three cases�auxV C� F and ExD are used as priority indices of packets�The computations of auxV C� F and ExD are based on theformula shown in Table I� In the table� the subscripts i�j� and k denotes server number� connection number� andpacket number respectively� In Delay�EDD� Xminj is theminimum packet inter�arrival time for connection j� di�j isthe local delay bound assigned to connection j at server iat connection establishment time� In Virtual Clock� V tickjis the average packet inter�arrival time for connection j� InWFQ and WF�Q� V �t� is the system virtual time at time

t� where the virtual time� de�ned below� is a measure ofthe system progress� Lkj is the packet length measured in

number of bits� and aki�j is arrival time of the kth packet onconnection j at switch i�

Virtual Clock auxV Cki�j

� maxfaki�j� auxV Ck

i�jg� V ticki�j

WFQ � WF�Q F ki�j � maxfVi�a

ki�j � F

k��i�j g�

Lkj

�i�j

SCFQ F ki�j

� maxf�Vi�aki�j � Fk��i�j

g�Lkj

�i�j

Delay�EDD ExDki�j

� maxfaki�j

� di�j � ExDki�j

�Xminjg

TABLE I

Comparison of Work�Conserving Disciplines

As shown in Table I� the priority index updating algo�rithms are very similar� However� there are also two im�portant di�erences� The �rst is whether the calculation isbased on just the rate parameter or both the delay and rateparameters� The second is whether the updating is basedon system�load independent parameters or system�load de�pendent parameters�

Notice that in Delay�EDD� two parameters are used toupdate the priority index� Xminj � which is the minimumpacket inter�arrival time for connection j� and di�j� which isthe local delay bound for connection j at switch switch i� Inother disciplines� only one rate parameter is used �V tickj or�i�j�� Although delay bounds can be provided for all thesedisciplines� having only one rate parameter introduces theproblem of coupling between the allocation of delay boundand bandwidth� For example� in Rate�Proportional Pro�cessor Sharing �RPPS�� which is a special case of WFQ orPGPS and the ��s are allocated proportional to the band�width required by connections� if the tra�c is constrainedby ��� �� � characterization� the end�to�end delay bound of

the connection will be ���n���Lmax

��Pn

i�Lmax

Ci� where n

is the number of hops traversed by the connection� and Ci

is the link speed of the ith server� Notice that the delaybound is inversely proportional to the allocated long termaverage rate� Thus� in order for a connection to get a lowdelay bound� a high bandwidth channel needs to be allo�cated� This will result in a waste of resources when the lowdelay connection also has low throughput� Delay�EDD� onthe other hand� does not have such a restriction ���� �����

The second di�erence between these disciplines iswhether the updating of the state variables depends onsystem load� In Virtual Clock and Delay�EDD� the updat�ing depends only on per connection parameters �V tick forVirtual Clock� Xmin and d for Delay�EDD� but not onsystem load� In WFQ� WF�Q� and SCFQ� the updatingis based on a notion of virtual time� The evolution of vir�tual time measures the progress of the system and dependson system load� For WFQ and WF�Q� the virtual timefunction V ��� during any busy period �t�� t�� is de�ned asfollows�

V �t�� � � ��

�As mentioned in Section � � is the maximum burst size� � is thelong term average rate�

Output link 2

Input link2

Input link 1Switch

Output link 1

Output link 3TransmissionOrdering and

PriorityAssignment

Fig� �� Sorted priority mechanism

�V �� �

���

P

j�BFFQ�� ��j

�t� � � � t� ��

where BFFQ�� � is the set of backlogged connections� attime � under the reference FFQ system� In FFQ� if con�nection j is backlogged at time � � it receives a service

rate of �V �� ���

�jC� where C is the link speed� Therefore�V can be interpreted as increasing proportionally to themarginal rate at which backlogged connections receive ser�vice in FFQ�

Connection 1

Virtual Clock Service Order

Connection 2

WFQ Service Order898 900 902 904

Fig� �� WFQ and Virtual Clock

The following example� given in �� � and illustrated inFigure �� shows the di�erence between WFQ and VirtualClock� Suppose that there are two connections� both witha speci�ed average rate of one packet every two seconds�All packets are �xed size and require exactly one secondto service� Starting at time zero� ��� packets from con�nection arrive at a rate of packet�second� No packetsfrom connection arrive at the interval �������� Startingat time ���� ��� connection packets arrive at a rate of packet�second� Since the �rst ��� packets from connec�tion are served in the interval �������� there are no pack�ets in queue from either connection at time ����� If Vir�tual Clock algorithm is used� at time ���� the connection auxV C reads ��� and the connection clock reads ����as can be seen in Table I� the auxV C value is at least thereal�time value�� When connection packets arrive� theywill be stamped ���� ��� � � �� ��� while the connection packets that arrive after time ��� will be stamped ������� � � �� ���� Thus� all of the connection packets will

�A connection is said to be backlogged at time � if it has packetsqueued at time � �

be served before any of the connection packets are served�On the other hand� if WFQ discipline is used� the numberof active connections is before time ��� and after time���� Since both connection and connection reserve halfof the link bandwidth� after time ���� the WFQ server willservice packets from both connections interleavingly�

The di�erent behaviors of Virtual Clock and PGPS aredue to the fact that Virtual Clock is de�ned with referenceto the static TDM system and the calculation of the virtualtransmission time is independent of the behaviors of otherconnections� The delay of a packet depends on the entirearrival history of the connection� which is summarized inthe state variable auxV C� Once a connection is mishaved�sending more packets than speci�ed�� it may be punishedby Virtual Clock� regardless whether such misbehavior af�fects the performance of other connections� WFQ� on theother hand� is de�ned with reference to another dynamicqueueing system FFQ� The virtual time of the system de�pends on how many other connections are active in thesystem�

......

SCFQ service order

connection 2

connection 1

WFQ service order

connection 11

Fig� �� SCFQ and WFQ

The dependency on virtual time also introduces extracomplexities for WFQ and WF�Q since the system needsto emulate FFQ and keep track of the number of active con�nections at any moment in FFQ� To reduce the complexityof computing virtual times� SCFQ introduces an approxi�mation algorithm� The algorithm is based on the observa�tion that the system�s virtual time at any moment tmay beestimated from the virtual service time of the packet cur�rently being serviced� Formally� the approximate virtual

�

time function �V �t� is de�ned to be F p where sp � t � fp�sp and fp denote the times packet p starts and �nishesservice in the SCFQ system�

While the calculation of virtual time is simplier in SCFQ�the inaccuracy incurred can make SCFQ perform muchworse than WFQ� Consider the example illustrated in Fig�ure � Again� assume all packets have the same size of �the link speed is � the guaranteed rate for connection be���� and the guarantee rate for each of the other � connec�tions be ����� Under FFQ� the �nish times will be k forpackets pk� � k � � � � �� � for packets p�j � j � � � �� and

for p��� � Transmitting packets in order of �nish timesunder FFQ� WFQ will produce the service order as shownon the fourth timeline in Figure � If SCFQ is used� attime �� same as in WFQ� it is p�� that has the smallest vir�tual �nish time� therefore� it receives service �rst� At time� all packets p�i � i � � � � � �� have virtual �nish time ofF �i � �� With the tie�breaking rule� the �rst packet on

connection � p��� is served� Since SCFQ uses the �nishtime of the packet in service as the the current virtual timevalue� we have �V �� � �V �� � F �

� � �� As a result whenp�� arrives at time � its virtual �nish time is set to be�

F �� � max�F�

� ��V ��� � L

r�� max�� �� � � � Among

all the packets ready to be served� p�� has the largest �nishnumber� Therefore� p�� won�t start service until all other� p�i � i � � � � � �� packets �nish services� i�e� it won�tdepart until time � In this example� even though connec�tion sends packet according to the speci�ed average rate�its packets still get signi�cantly delayed�

F� Tra�c Characterization Inside the Network

As discussed in Section I� we are interested in providingend�to�end delay bounds on a per connection basis in a net�working environment� One solution is to obtain worst�caselocal delay bounds at each switch independently and usethe sum of these local delay bounds as the end�to�end delaybound ���� Alternatively� smaller end�to�end delay boundscan be obtained by taking into account the dependencies inthe sucessive switches that a connection traverses �� �� ��������� ���� � �� ���� For the �rst type of solution� in order toderive local delay bound� tra�c needs to be characterizedon a per connection basis at each switch inside the network�For the second type of solution� while the end�to�end de�lay bound may be derived for Virtual Clock� WFQ� SCFQbased only on the source tra�c characterization �� �� ����� �� ���� as will be shown in Section III�G� the delay boundcouples with bandwidth allocation� In �� �� such a sourceallocation strategy is called rate�proportional allocation� Ithas been shown more general resource allocation strategiesthat decouples throughtput and delay bounds can result inhigher utilization of the network� In general� for both typesof solutions� the tra�c needs to be characterized on a perconnection basis at each switch inside the network�

The di�culty arises in a networking environment� whereeven if a connection�s tra�c can be characterized at theentrance to the network� tra�c pattern may be distortedinside the network� thus make the source characterizationnot applicable at the servers traversed by the connection�

After Switch 1

Entrance to Network

After Switch 2

After Switch 3

Fig� �� Tra�c pattern distortions due to load �uctuations

This is shown in the following example illustrated by Fig�ure �� In the example� four packets from one connectionare sent with a certain inter�packet spacing from the sourceinto a network where links have constant delay� At the �rstserver� the �rst packet is delayed by a certain amount oftime �less than the local delay bound� due to instanta�neous cross tra�c load� but the other three packets passthrough instantly� Because the �rst packet was delayedlonger than the second packet� the spacing between �rstand second packets becomes smaller when they arrive atthe second server� At the second server� the �rst and thesecond packet are delayed some more time� but packets �and � pass through instantly� At the third server� the �rstthree packets are delayed but packet � passes through withno delay� The �gure shows tra�c patterns at the entranceto each of the servers� Two things can be observed� �a�the tra�c pattern of a connection can be distorted dueto network load �uctuations� �b� the distortion may makethe tra�c burstier and cause instantaneously higher rates�In the worst case� the distortion can be accumulated� anddownstream servers potentially face burstier tra�c thanupstream servers� Therefore� the source tra�c characteri�zation may not be applicable inside the network�

There are three solutions to address this problem of traf��c pattern distortion�

� controlling the tra�c distortion within the network�� accounting for the distortion during scheduling��� characterizing the tra�c distortion throughout the

network�To control tra�c distortions within the network� some

packets need to be held even when a server has the extracapacity� This requires non�work�conserving disciplines�which we will discuss in more detail in Section IV�

The second solution accounts for tra�c distortions dur�ing scheduling� Instead of scheduling packets according totheir actual arrival times� the server assigns each packet alogical arrival time based on its tra�c characterization andprevious arrival history� and schedules packets based on

�

Server 3

Server 2

Server 1

Server 0

Connection 3Connection 2

Connection 1Connection 0

Link 1Link 2

Link 0Link 3

Fig� �� Example of a Feedback Network

their logical arrival times� Delay�EDD and Virtual Clockuse such an approach� For example� in Delay�EDD� thedeadline of a packet is the sum of the local delay bound �d�and the expected arrival time �ExA� of the packet� Theservice discipline and the admission control policy ensurethat the packet is guaranteed to leave before the deadline�or at most d time units after the expected arrival time ofthe packet� It is possible that a packet is delayed longerin a server than its local delay bound� However� this canonly happen if the packet�s expected arrival time is largerthan its actual arrival time� which means that the packetis ahead of schedule in previous servers� It can be shownthat the amount of the time the packet is queued at theserver more than its delay bound is always less than theamount of time the packet is ahead of schedule at previousservers�

Accounting for tra�c distorting during scheduling worksonly if the server has a concept of expected arrival time� Amore general solution is to characterize the tra�c inside thenetwork� The problem can be formulated as the following�given the tra�c characterization of all the connections atthe entrance to the network and all the service disciplinesat the switches� can the tra�c be characterized on a perconnection basis on all the links inside the network Sev�eral solutions have been proposed to address this problemwith di�erent tra�c models and service disciplines ��� ������ �� �� �� They all employ a similar technique that con�sists of two steps� In the �rst step� a single node analysistechnique is developed to characterize the output tra�c ofa server given the characterizations of all its input tra�c�In the second step� starting from the characterizations ofall the source tra�c� an iterative process push the tra�ccharacterizations from the links at the edge of the networkto those inside the network� There are several limitationsassociated with such an approach�

First� characterizing the tra�c inside the network is dif��cult and may not always be possible� In ���� �� �� ����� itis shown that this is equivalent to solving a set of multi�variable equations� In a feedback network� where tra�c

from di�erent connections forms tra�c loops� the result�ing set of equations may be unsolvable� To illustrate this�consider the following example discussed in ���� �� ��

In the ��nodes network shown in Figure �� there are four��hop connections and the aggregate tra�c of the four con�nections forms a loop� In order to characterize the tra�con link � the characterization of the input tra�c to server has to be known� Assuming links only introduce �xeddelay� the input tra�c to server is identical to the out�put tra�c of server �� or the tra�c on link �� There arethree tra�c streams on link �� which are from connections�� � and �� While connection � tra�c on link � is thesame as its source tra�c� connection and connection �tra�c on link � needs to be characterized� The charac�terizations of connection and � tra�c depend on theircharacterizations on link �� which in turn depend on theircharacterizations on link � This dependency �nally comesback to tra�c characterizations on link �� Because of thisinter�dependency of tra�c� characterizing all the tra�c in�side the network is equivalent to solving a set of multi�variable equations� where each variable corresponds to oneparameter in the tra�c characterization of one connectionon one link� The equations are solvable only under cer�tain conditions� In this particular example� it is shown in��� that if each server has a policy such that the tra�coriginating from the server has a lower priority than thethrough tra�c� the condition for solving the equations isthat the aggregate throughput of all connections must beless than �! of the link bandwidth on each of the fourlinks� This condition is not merely a technical restriction�the network can actually be unstable� i�e�� have unboundedqueue lengths� when the condition is violated �� �� How toderive the stability condition in a general networking envi�ronment is still a open problem� A distributed algorithmis even more di�cult� One notable exception to such a re�striction is the case when the service discipline used is aspecial class of PGPS called Rate Proportional ProcessorSharing �RPPS� �� �� With RPPS� the stability conditioncan be derived� We will discuss the exact formula of thedelay bound in Section III�G�

The second limitation of characterizing tra�c inside thenetwork is that it only applies to networks with constantdelay links� Constant delay links have the desirable prop�erty that the tra�c pattern at the receiving end of the linkis the same as that at the transmitting end� This prop�erty is important for these solutions because central to theanalysis is the technique of characterizing the output traf��c from a server and using it as the characterization ofthe input tra�c to the next�hop server� However� in aninternetworking environment� where the link between twoswitches may be a subnetwork such as an ATM network ora FDDI network ���� load �uctuations within subnetworksmay also introduce tra�c pattern distortions� Though it ispossible to bound delay over these subnetworks� the delaysfor di�erent packets will be variable� Thus� these solutionsneed to be extended in order to be applicable in an inter�networking environment�

��

tra�c end�to�end end�to�end bu�er spaceconstraint delay bound delay�jitter bound at hth switch

D�EDD bj���Pn

i� di�jPn

i� di�j bj�Ph

i� di�j�FFQ ��j � �j�

�jrj

�jrj

�j

VC ��j � �j��j�nLmax

rj�Pn

i�Lmax

Ci

�j�nLmax

rj�j � hLmax

WFQ " WF�Q ��j � �j��j�nLmax

rj�Pn

i�Lmax

Ci

�j�nLmax

rj�j � hLmax

SCFQ ��j � �j��j�nLmax

rj�Pn

i�KiLmax

Ci

�j�nLmax

rj�Pn

i��Ki � �Lmax

Ci�j � hLmax

TABLE II

End�to�end delay bound� delay�jitter bound� and buffer space requirements

Finally� in networks with work�conserving service disci�plines� even in situations when tra�c inside the networkcan be characterized� the characterization usually repre�sents a burstier tra�c inside the network than that at theentrance� This is independent of the tra�c model beingused� In ���� it is shown that if the tra�c of connectionj is characterized by ��j� �j� at the entrance to the net�

work� its characterization will be ��j �Pi��

h� �jdh�j� �j� atthe entrance to the ith server along the path� where dh�j isthe local delay bound for the connection at the hth server�Compared to the characterization of the source tra�c� themaximum burst size increases by

Pi��h� �jdh�j� This in�

crease of burst size grows monotonically along the path�In �� �� a family of stochastic random variables are used

to characterize the source� Connection j is said to satisfya characterization of f�Rt��j� t��� �Rt��j � t��� �Rt��j� t�����g�where Rti�j are random variables and t� � t� �

� � � are time intervals� if Rti�j is stochastically largerthan the number of packets generated over any inter�val of length ti by source j� If the tra�c connec�tion j is characterized by f�Rt��j � t��� �Rt��j� t��� ���g atthe entrance to the network� its characterization willbe f�R

t��P

i��

h��bh�j

� t��� �Rt��P

i��

h��bh�j

� t��� ���g at the hth

switch� where bh is the length of the maximum busy periodat switch h� The same random variableR

tm�P

i��

h��bh�j

that

bounds the maximum number of packets over an intervalof length tm �

Pi��h� bh at the entrance to the network�

now bounds the maximum number of packets over a muchsmaller interval of length tm at server i� In other words�the tra�c characterization is burstier at server i than atthe entrance�

G� End�to�End Delay Characteristics and Bu�er Space Re�quirement

While the problem of deriving end�to�end delay boundsfor a network of work�conserving servers has yet to besolved under general resource assignments� results havebeen obtained for Virtual Clock� WFQ� WF�Q� SCFQ un�der the rate�proportional allocation strategy� and for Delay�EDD by considering each server in isolation� In both cases�the source tra�c speci�cations are su�cient and tra�ccharacterizations inside the network are not needed� Inthe former case� the end�to�end delay bound for a connec�tion is a function of the guaranteed rate� which needs tobe no less than the connection�s average rate� In the latter

case� the end�to�end delay bound is calculated as the sumof worst�case local delays at each switch� Since Delay�EDDscheduling is based on logical rather actual packet arrivaltimes� local delay bounds at all switches can be calculatedusing the same source tra�c characterization� To preventpacket loss� we assume bu�er space is allocated on a perconnection basis at each server during connection estab�lishment time�

Table II presents the end�to�end characteristics andbu�er space requirement of a connection when di�erentwork�conserving service disciplines are used� The table canbe interpreted as the following� If a connection satis�es thetra�c constraint as de�ned in the second column� and isallocated the amount of bu�er space as listed in the �fthcolumn� it can be guaranteed an end�to�end delay boundand delay�jitter bound as listed in the third and fourthcolumn respectively� provided each server along the pathuses the discipline in �rst column and appropriate admis�sion control conditions are satis�ed� In the table� Ci is linkspeed of the ith switch on the path traversed by the con�nection� Ki is the number of connections sharing the linkwith the connection at the ith switch� rj is the guaranteedrate for the connection� and Lmax is the largest packet size�Link delays are omitted from the expressions of end�to�enddelays for simplicity�

Notice that the ��� �� tra�c model is used to character�ize the tra�c in all disciplines except Delay�EDD wherea general tra�c constraint function is used� The orig�inal Delay�EDD uses the �Xmin�Xave� I� Smax� tra�cmodel ���� ����� However� the algorithm can easily beextended to accommodate connections using arbitrary de�terministic tra�c models that have associated tra�c con�straint functions� The corresponding admission control al�gorithm is described in ����� A more general tra�c modelcan characterize sources more accurately� thus resulting ina higher network utilization� A more detailed discussionon the relationship between achievable network utilizationand accuracy of tra�c characterization can be found in ����������

There are several noteworthy points about the table�First� even though Virtual Clock� WFQ� and WF�Q havea number of di�erences� they provide identical end�to�enddelay bounds for connections that have leaky bucket con�strained sources� In fact� if we compare the delay boundprovided by them and that provided by the ideal �uid FFQ

��

discipline� we can see that they share the same main term�jrj� which can be interpreted as the time to send a burst of

size �j in a �uid system with the guaranteed rate of rj� Forthe three packet policies� there are additional terms to ac�count for the fact that tra�c is not in�nitely divisible andthe server needs to serve one packet at a time� Secondly�with the same guaranteed rate� the delay bound providedby SCFQ is larger than that provided by Virtual Clock�WFQ� and WF�Q� This is due to the inaccuracy intro�duced by the approximation algorithm in calculating thevirtual time� For all the four disciplines� since the serverallocates service shares to connections proportional to theiraverage rates� there is a coupling between the end�to�enddelay bound and bandwidth provided to each connection�In particular� the end�to�end delay bound is inversely pro�portional to the allocated long term average rate� Thus�in order for a connection to get a low delay bound� a highbandwidth channel need to be allocated� This will result ina waste of resources when the low delay connection also haslow throughput� WFQ and WF�Q with general resourceassignments do not have such a restriction �� �� However�due to the di�culties of characterizing tra�c inside the net�work� the problem of deriving end�to�end delay bound forWFQ and WF�Q under general resource assignments hasyet to be solved� Delay�EDD does not have the problem ofcoupling between the allocations of delay bound and band�width either� However� the end�to�end delay bound listedin the table was derived without taking into account thedelay dependency among successive switches� and is ratherloose� As a �nal point to be noted� the end�to�end delay�jitter bounds for all disciplines are loose� In fact� the end�to�end delay�jitter bound is equal to the maximum end�to�end queueing delay� This can be easily understood by thefollowing observation� Recall that delay�jitter bound is themaximum di�erence between delays of any two packets�In a network with work�conserving disciplines� a packetcan experience little queueing delay when the network islightly loaded while another packet can experience a muchlonger queueing delay when the network is heavily loaded�Thus the maximum di�erence between delays experiencedby these two packets is the maximum end�to�end queueingdelay�

H� Implementation Issues

As described in Section III�E� all the proposed work�conserving disciplines use the mechanism of a sorted pri�ority queue� The insertion operation for a sorted priorityqueue has an intrinsic complexity of O�logN � ����� whereN is the number of packets in the queue� In a networkthat is designed to support many connections with burstytra�c� the switch usually has bu�er space for a large num�ber of packets� For example� the queue module of each linkof the Xunet switch contains memory to store �K ATMcells ���� Potentially� the queue length can be long� Itmay not be feasible to implement an operation that hasan O�logN� complexity at very high speed� Since all dis�ciplines ensure that packets on the same connection areserviced in the order of their arrivals� a clever implementa�

tion can arrange packets on a per�connection basis and sortonly the �rst packet of each connection� Recently� it wasreported that a sequencer chip clocked at �� Mhz has beenimplemented to support sorting of up to �� packets ����Thus� up to �� connections or classes of connections canbe supported with such an implementation� It is unclearwhether such an implementation can scale to higher speedor more connections�

A sorted priority queue mechanism also requires compu�tation of the priority index on a per packet basis� For ser�vice disciplines that use real time to compute the priorityindex� such as Virtual Clock and Delay�EDD� the computa�tion is simple and straightforward� For service disciplinesthat use virtual times in another reference queueing sys�tem� such as WFQ and WF�Q� the computation is morecomplex� In particular� both WFQ and WF�Q need tokeep track the set of connections that have packets queuedat any time instant� This is very di�cult to implement athigh speed� SCFQ simpli�es the computation by using anan approximate algorithm that does not need to keep trackof the set of active connections�

IV� Non�Work�Conserving Disciplines

In Section III�F� we showed that in order to derive end�to�end delay bounds and bu�er space requirements in anetworking environment� tra�c needs to be characterizedinside the network on a per connection basis� With work�conserving disciplines� the tra�c pattern is distorted insidethe network due to network load �uctuation� and thereare a number of di�culties and limitations in deriving thetra�c characterization after the distortion�

Another approach to deal with the problem of tra�c pat�tern distortions is to control the distortions at each switchusing non�work�conserving disciplines� With a non�work�conserving discipline� the server may be idle even whenthere are packets waiting to be sent� Non�work�conservingdisciplines were seldom studied in the past� This is mainlydue to two reasons� First� in most of previous performanceanalyses� the major performance indices are the average de�lay of all packets and the average throughput of the server�With a non�work�conserving discipline� a packet may beheld in the server even when the server is idle� This may in�crease the average delay of packets and decrease the averagethroughput of the server� Secondly� most previous queue�ing analyses assumed a single server environment� Thepotential advantages of non�work�conserving disciplines ina networking environment were therefore not realized� Inguaranteed performance service� the more important per�formance index is the end�to�end delay bound rather thanthe average delay� In addition� delay needs to be boundedin a networking environment rather than just in a singlenode� Therefore� the above reasons for not using non�work�conserving disciplines do not hold any more�

Several non�work�conserving disciplines have been pro�posed in the context of high speed integrated servicesnetworks� Among them are� Jitter Earliest�Due�Date�Jitter�EDD� ����� Stop�and�Go ��� Hierarchical RoundRobin �HRR� ���� and Rate�Controlled Static Priority

��

Switch iDeadline

Delay Bound

Switch i-1

Delay bound

PreAhead

Arrival Departure Deadline

EligibleArrival

holding

Fig� �� Packet Service in JitterEDD

�RCSP� ���� In this section� we �rst describe each of thealgorithms in turn� then present a uni�ed framework calledrate�controlled service disciplines and show that all of themcan be represented in such a framework� Finally� we dis�cuss the end�to�end delay characteristics and bu�er spacerequirements for these disciplines within the framework ofrate�controlled service disciplines�

A� Jitter�Earliest�Due�Date

The Jitter�EDD discipline ���� extends Delay�EDD toprovide delay�jitter bounds �that is� a bound on the maxi�mum delay di�erence between two packets�� After a packethas been served at each server� a �eld in its header isstamped with the di�erence between its deadline and theactual �nishing time� A regulator at the entrance of thenext server holds the packet for this period before it is madeeligible to be scheduled�

Jitter�EDD is illustrated in Figure �� which shows theprogress of a packet through two adjacent servers� In the�rst server� the packet got served PreAhead seconds beforeits deadline� So� in the next server� it is made eligibleto be sent only after PreAhead seconds� Since there is aconstant delay between the eligibility times of the packetat two adjacent servers� the packet stream can be provideda delay jitter bound� Assuming there is no regulator at thedestination host� the end�to�end delay jitter bound is thesame as the local delay bound at the last server�

B� Stop�and�Go

As shown in Figure � Stop�and�Go uses a framing strat�egy ���� In such a strategy� the time axis is divided intoframes� which are periods of some constant length T � Stop�and�Go de�nes departing and arriving frames for eachlink� At each switch� the arriving frame of each incom�ing link is mapped to the departing frame of the outputlink by introducing a constant delay � where � � � T �According to the Stop�and�Go discipline� the transmissionof a packet that has arrived on any link l during a framef should always be postponed until the beginning of thenext frame� Since packets arriving during a frame f of theoutput link are not eligible for transmission until the nextframe� the output link may be left idle even when there arepackets in the switch to be transmitted� thus� Stop�and�Gois a non�work�conserving policy�

Stop�and�go ensures that packets on the same frame atthe source stay in the same frame throughout the network�If the tra�c is characterized at the source by �r� T �� i�e�� no

Output Link

switch

Input Link 1

Input Link 2

1θ

θ2

Input Link 1

Input Link 2

Output Link

Fig� � Synchronization between input and output links in StopandGo

more than r � T bits are transmitted during any frame ofsize T � it satis�es the same characterization throughout thenetwork� By maintaining tra�c characteristics throughoutthe network� end�to�end delay bounds can be guaranteed ina network of arbitrary topology as long as each local servercan ensure local delay bounds for tra�c characterized by�r� T � speci�cation�

T1 frame

T2 frame

Fig� �� Two levels of framing with T� � �T�

The framing strategy introduces the problem of couplingbetween delay bound and bandwidth allocation granular�ity� The delay of any packet at a single switch is boundedby two frame times� To reduce the delay� a smaller T isdesired� However� since T is also used to specify tra�c�it is tied to bandwidth allocation granularity� Assuminga �xed packet size P � the minimum granularity of band�width allocation is P

T� To have more �exibility in allocating

bandwidth� or a smaller bandwidth allocation granularity�a larger T is preferred� It is clear that low delay bound and�ne granularity of bandwidth allocation cannot be achievedsimultaneously in a framing strategy like Stop�and�Go�

To get around this coupling problem� a generalized ver�sion of Stop�and�Go with multiple frame sizes is proposed�In the generalized Stop�and�Go� the time axis is divided

��

Entrance to Network

After Switch 1

After Switch 2

After Switch 3

Stop�and�Go

Entrance to Network

After Switch 1

After Switch 2

After Switch 3

Hierarchical Round Robin

Fig� �� Di�erence between StopandGo and HRR

into a hierarchical framing structure as shown in Figure� For a n level framing with frame sizes T�� � � � � Tn� andTm�� � KmTm for m � � � � � � n � � packets on a levelp connection need to observe the Stop�and�Go rule withframe size Tp� i�e�� level p packets which arrived at an out�put link during a Tp frame will not become eligible fortransmission until the start of next Tp frame� Also� for twopackets with di�erent frame sizes� the packet with a smallerframe size has a non�preemptive priority over the packetwith a larger frame size� With multi�frame Stop�and�Go�it is possible to provide low delay bounds to some channelsby putting them in frames with a smaller frame time� andto allocate bandwidth with �ne granularity to other chan�nels by putting them in levels with a larger frame time�However� the coupling between delay and bandwidth allo�cation granularity still exists within each frame� In ����a scheme is proposed to add a separate shaping mecha�nism at the network entry point for networks with framingbased disciplines� With tra�c shaping at the entrance tothe network� it is possible to multiplex several connectionson a single slot of a frame� therefore avoid the problemof coupling between frame size and bandwidth allocationgranularity�

C� Hierarchical Round Robin

HRR is similar to Stop�and�Go in that it also uses amult�level framing strategy� A slot in one level can ei�ther be allocated to a connection or to a lower level frame�The server cycles through the frame and services packetsaccording to the assignment of slots� If the server cyclesthrough a slot assigned to a connection� one packet fromthat connection is transmitted� if it cycles through a slotassigned to a lower level frame� it will service one slot fromthe lower level frame in the same fashion� HRR is non�work�conserving in the sense that if it cycles through aslot with no packets waiting� it will leave the server idlefor that slot time rather than sending packets assigned toother slots�

Similar to Stop�and�Go� HRR also maintains traf�

�c smoothness inside the network due to its non�work�conserving nature� However� there are also important dif�ferences between HRR and Stop�and�Go� The exampleshown in Figure � illustrates their di�erence� In the ex�ample� it is assumed that � packet transmission times areallocated to a connection in each frame� In Stop�and�Go�packets that are transmitted in the same frame at the en�trance to the network will be transmitted in the same frameon all the links traversed by the connection� The di�erencebetween delays experienced by any two packets from thesource to any server is bounded by T � where T is the framesize� In HRR� packets that are transmitted in the sameframe at the entrance to the network do not necessarilystay in the same frame inside the network� however� theproperty that no more than three packets from the connec�tion are transmitted during one frame time holds through�out the network�

Since HRR uses the framing strategy� it also has theproblem of coupling between delay and bandwidth alloca�tion granularity�

D� Rate�Controlled Static Priority

While the Earliest�Due�Date algorithm can provide �ex�ible delay bounds and bandwidth allocation� it is basedon a sorted priority mechanism� which is di�cult to im�plement� Stop�and�Go and HRR use a framing strategyinstead of the sorted priority to achieve simplicity� how�ever� such a strategy introduces coupling between delaybound and bandwidth allocation granularity� The goal ofRCSP is to achieve �exibility in the allocation of delay andbandwidth as well as simplicity of implementation�

As shown in Figure �� a RCSP server has two com�ponents� a rate�controller and a static priority scheduler�Conceptually� a rate controller consists of a set of regu�lators corresponding to each of the connections traversingthe server� each regulator is responsible for shaping the in�put tra�c of the corresponding connection into the desiredtra�c pattern� Upon arrival of each packet� an eligibilitytime is calculated and assigned to the packet by the regu�

��

FIFO

FIFO

Priority Level

n

1

...

Regulator h

...

Regulator 2

Regulated TrafficRegulator 1

Output

Rate Controller Scheduler

of the h connections

One regulator for each

Input Traffic

Fig� �� RateControlled Static Priority

lator� The packet is held in the regulator till its eligibilitytime before being handed to the scheduler for transmis�sion� Di�erent ways of calculating the eligibility time ofa packet result in di�erent types of regulators� As will bediscussed in ��� and Section IV�F� many regulators can beused for RCSP� We will consider two examples in this sec�tion� The �Xmin�Xave� I� RJ regulator ensures that theoutput of the regulator satisfy the �Xmin�Xave� I� tra�cmodel� while the DJr regulator ensures that the outputtra�c of the regulator is exactly the same as the the out�put tra�c of the regulator at the previous server� Thus�if the tra�c satis�es the �Xmin�Xave� I� characterizationat network entrance� both types of regulators will ensurethat the output of the regulator� which is the input to thescheduler� will satisfy the same tra�c characterization�

For a �Xmin�Xave� I� RJ regulator� where Xmin �Xave � I holds� the eligibility time of the kth packet onconnection j at the ith server along its path� eki � is de�nedwith reference to the eligibility times of packets arrivingearlier at the server on the same connection�

eki�j � �I� k � � ���

e�i�j � a�i�j ���

eki�j � max�ek��i�j �Xmin�

ek�b I

Xavec��

i�j � I� aki�j�� k ���

where aki�j is the time the kth packet on connection j arrived

at the ith server� ��� is de�ned for convenience so that ���holds for any k �

From this de�nition� we can see that eki�j � aki�j alwaysholds� i�e�� a packet is never eligible before its arrival� Also�if we consider the sequence of packet eligibility times at ith

server � feki�jgk�������� it always satis�es the �Xmin� Xave� I�tra�c characterization�

The eligibility time of a packet for a DJr regulator isde�ned with reference to the eligibility time of the samepacket at the immediately upstream server� The de�ni�tion assumes that the queueing delays of packets on theconnection� and the link delay from the upstream serverto the current server� are bounded� Let di���j be the lo�cal delay bound for the connection in the scheduler at the�i � �th server� and �i be the maximum link delay fromthe �i� �th server to the ith server� The DJr regulator is

de�ned as� is de�ned as�

ek �j � ak �j ���

eki�j � eki���j � di���j � �i� i � � �

It is easy to show that the following holds�

ek��i�j � eki�j � ak�� �j � ak �j �k� i � � ���

i�e�� the tra�c pattern on a connection at the output ofthe regulator of every server traversed by the connection isexactly the same as the tra�c pattern of the connection atthe entrance to the network�

The scheduler in a server RCSP uses a non�preemptiveStatic Priority policy� it always selects the packet at thehead of highest priority queue that is not empty� The SPscheduler has a number of priority levels with each prioritylevel corresponding to a delay bound� Each connection isassigned to a priority level during connection establishmenttime� Multiple connections can be assigned to the samepriority level� and all packets on the connections associatedwith a priority level are appended to the end of the queuefor that priority level�

E� A Framework For Non�Work�Conserving Disciplines

Output

Regulator 1Regulated Traffic

Regulator 2

...

Regulator h

Rate Controller Scheduler

of the h connections

One regulator for each

Input Traffic

Fig� �� RateControlled Service Disciplines

In previous sections� we described four non�work�conserving disciplines� In this section� we show that allof them can be expressed by a general class of disciplinescalled rate�controlled service disciplines ����� As shown inFigure �� a rate�controlled server can be considered asa generalization of RCSP� it also has two components� arate�controller and a scheduler� The rate controller� whichconsists of a number of regulators� is responsible for shap�ing tra�c� The scheduler is responsible for multiplexingeligible packets coming from di�erent regulators� WhileRCSP uses two types of regulators and the Static Prior�ity scheduler� many other regulators and schedulers can beused� By having di�erent combinations of regulators andschedulers� we have a general class of disciplines� Amongthe four disciplines discussed in this section� RCSP andJitter�EDD are rate�controlled servers� Stop�and�Go andHRR can be implemented by rate�controlled servers by se�lecting appropriate regulators and schedulers�

��

Jitter�EDD can be viewed as a combination of a Earliest�Due�Date scheduler and DJe regulators� which are de�nedas follows�

eki�j � aki�j �Aheadki���j �

where Aheadki���j is the amount of time the packet is ahead

of schedule at the �i� �th server along the path�A Stop�and�Go server with n frame sizes �T� � T� � ��� �

Tn can be implemented by a rate�controlled server with ann�level static priority scheduler and DJs regulators�

eki�j � aki�j � Aheadki���j � �i�j ��

where Aheadki���j is the amount of time the packet is aheadof schedule in switch i � �� and �i�j is the synchronizationtime between the framing structures on the input and out�put links� Each pair of input and output links in a switchmay have a di�erent value of �� Figure �� illustrates thissynchronization time� In the static priority scheduler� thedelay bound associated with level m is Tm� � � m � n�

One regulator for each

priority level

FIFO

FIFO

Regulator 1

Regulator n

Priority Level

n

1

......Input Output

Fig� ��� Implement Stop�and�Go Using a Rate�Controlled Server

Although the above implementation of Stop�and�Go isvery similar to RCSP� there are also important di�erences�as can be seen by comparing Figure �� and Figure ��� In anRCSP server� there is a regulator for each connection� andthe regulated tra�c on each connection can be assigned toany priority level in the scheduler� In a Stop�and�Go server�regulators are associated with priority levels in the sched�uler� In fact� there is a one�to�one correspondence betweenthe regulator and the priority level� The tra�c on a con�nection has to be speci�ed with respect to the frame size�which is the same as the connection�s local delay bound�This not only introduces the coupling between the alloca�tions of bandwidth and delay bounds� but also implies thatadmission control algorithm has to be based on a busy pe�riod argument� which tends to produce looser bounds whencompared to more elaborate analysis ���� �����Because of the framing� there are dependencies among

the local delay bounds at each priority level in a Stop�and�Go server� In particular� Tm�� � KmTm must hold�with � � m � n� and Km being an integer� Furthermore�the delay bound allocations for each connection in di�erentservers are coupled with one another� In ����� a connectionhas to have the same frame size in all the servers� In ����� alooser requirement is presented� the frame times of a con�nection along the path should be non�decreasing� None of

Discipline eki�j de�ned in regulator Scheduler

RCSP�DJr ak� � Aheadki���j � ��i � �ki SPeki���j � di���j � �ki

Jitter�EDD aki�j �Aheadki���j EDDeki���j � di���j � �ki

Stop�and�Go aki�j �Aheadki���j � � SPeki���j � Tm � �ki

RCSP�RJr max�ek��i�j �Xminj � SP

ek�b I

Xavejc��

i�j � I� aki�j

HRR max�aki�j� ek�ai�ji�j � Tm

j SP

TABLE III

Non�Work�Conserving Disciplines

these restrictions apply to RCSP� The impact of �exibilityof allocating delay bounds inside the network on networkutilization was studied in �����A Hierarchical Round Robin server with n frame sizes

�T� � T� � ��� � Tn can be implemented by a rate�controlled server with an n�level static priority schedulerand RJh regulators de�ned by�

eki�j � max�aki�j � �� ek�qmi�ji�j � Tm ���

where aki�j � � is the beginning time of the next frame andqmi�j is the maximum number of packets that can be servedon the connection within each frame of size Tm� In thestatic priority scheduler� the delay bound associated withlevel m is Tm� � � m � n� If a connection traverses a level�m RJh regulator� it has to be assigned to the priority levelm in the scheduler� This introduces the coupling betweendelay and bandwidth allocation� In contrast� in an RCSPserver� a connection can be assigned to any priority levelregardless of its rate parameters�Table III summarizes the regulators and schedulers for

the four disciplines� Notice that there are two equivalentde�nitions of eligibility times for each of the DJr � DJe andDJs regulators�

F� Delay�jitter�control and Rate�jitter�control Regulators

As shown in Table III� the regulators for RCSP�DJr�Jitter�EDD� and Stop�and�Go are very similar� For eachof the three regulators� the eligibility time of a packet at aswitch is de�ned with respect to the eligibility time of thesame packet at the previous switch� Also� the regulatorsfor RCSP�RJr and HRR are similar in that the eligibil�ity time of a packet at a switch is de�ned with respect toearlier arriving packets at the same switch� In ����� twogeneral classes of regulators called delay�jitter controllingregulators and rate�jitter controlling regulators are de�ned�Regulators for RCSP�DJr � Jitter�EDD� and Stop�and�Gofall into the former class� whereas regulators for RCSP�RJrand HRR are in the later class�For a delay�jitter controlling regulator� the eligibility

time of a packet is de�ned with reference to the eligibil�ity time of the same packet at the immediately upstream

��

tra�c end�to�end end�to�end bu�er space at hth switchconstraint delay bound delay�jitter bound

Stop�and�Go �rj � Tj nTj Pn

i���i Tj rj��Tj �i

HRR �rj � Tj �nTj �nTj �rjTjRate�controlled Servers with bj�� D�bj� b�

Pn

i��di�j D�bj� b�

Pn

i��di�j �j b��d��j for �st switch

b��� RJ regulators b��di���j di�j for jth switch j � �

Rate�controlled Servers with bj�� D�bj� b� Pn

i��di�j D�bj� b� dn�j �j b��d��j for �st switch

b��� RJ regulator for �st switch b��di���j di�j for jth switch j � �

and DJ regulators for other switchesTABLE IV

End�to�end delay� delay jitter and buffer space requirement for non�work�conserving disciplines

server� The following de�nition assumes that the queueingdelays of packets on the connection at the immediately up�stream server and the link delay from the upstream serverto the current server are bounded�

ek��j � ak��j ���

eki�j � eki���j � di���j � �i�j � �i�j� i � � ���

where ak��j is the arrival time of the kth packet at the en�trance to the network� and �i�j is a constant delay�While delay�jitter �DJ regulators maintain all the tra�c

characteristics by completely reconstructing tra�c patternat output of each regulator� rate�jitter �RJ regulators onlymaintain certain characteristics of the tra�c� Dependingon which tra�c models are used by the resource allocationalgorithm� di�erent RJ regulators can be de�ned� As dis�cussed in Section II�B�� and in ����� each deterministic traf��c model� such as �Xmin�Xave� I� Smax ����� �r� T ������� ���� and D�BIND ����� de�nes a deterministic tra�cconstraint function b��� A monotonic increasing functionbj�� is called a deterministic tra�c constraint function ofconnection j if during any interval of length u� the num�ber of bits arriving on j during the interval is no greaterthan bj�u� For each tra�c model with a correspondingdeterministic tra�c constraint function b��� we can con�struct a rate�jitter controlling regulator with the followingde�nition of eki�j�

eki�j � minfv � v � max�ek��i�j � aki�j�

Ei�j�u� v � bj�v � u �u � vg ���

where Ei�j��� �� de�ned below� is the number of bits onconnection j that become eligible in interval �u� v at theith server�

Ei�j�u� v �Xk

�Lkj ju � eki�j � v ���

and Lkj is the length of the kth packet on connection j�Equation ��� is very general and de�nes a class of rate�

jitter controlling policies� Any deterministic tra�c modelthat can be de�ned with a tra�c constraint function hasa corresponding rate�jitter controlling regulator� The reg�ulator for HRR is a rate�jitter controlling regulator usingthe �r� T tra�c model� and the regulator for RCSP�RJr isthe one using the �Xmin�Xave� I model� In addition� theimplementation of rate�jitter controlling regulators can be

very simple� For example� the regulator for the �� traf��c model can be implemented by the popular leaky bucketmechanism �����

G� End�to�End Delay Characteristics and Bu�er Space Re�

quirements

The end�to�end delay characteristics and bu�er space re�quirement for non�work�conserving disciplines are shown inTable III� In the table� D�bj � b

� is the worst�case delay in�troduced by a RJ regulator with the constraint functionb��� for a tra�c stream characterized by the constraintfunction bj���As shown in the table� the two frame�based disci�

plines Stop�and�Go and HRR have similar end�to�end de�lay bounds and bu�er space requirements� The only ma�jor di�erence between them is that Stop�and�Go providesa tighter jitter bound than HRR� This is because Stop�and�Go uses delay�jitter control while HRR uses rate�jittercontrol�While the end�to�end delay bounds for Stop�and�Go and

HRR are derived by considering each server in isolation�tighter end�to�end delay bounds can be derived for rate�controlled service disciplines by taking into considerationthe delay dependencies in successive switches traversed by aconnection ���� The key observation is that� b���� the traf��c constraint function used in the regulators� does not haveto be the same as bj��� the tra�c constraint function usedto specify the source� By appropriately setting parametersfor regulators and local delay bounds at schedulers� rate�controlled service disciplines can provide end�to�end delaybounds at least as tight at those provided by FFQ�basedwork�conserving service disciplines� To compare with FFQ�based disciplines� assume that the tra�c on connection j

is characterized by the �j� j model� That is�

bj�u � j � ju ���

We consider two cases� In the �rst case� only RJ regulatorsare used� The tra�c constraint function for the regulatorsand the local delay bound for each scheduler are de�ned asfollows�

b��u � Lmax � ju ���

di�j �Lmax

j�Lmax

Ci

���

In the second case� the �rst switch still uses the RJ reg�ulator de�ned above� but all subsequent switches use DJ

��

regulators with �i�j � � Same local delay bounds are as�signed to each switch�It can be shown that the following holds�

D�bj� b� �

j

j��

According to Table IV� an end�to�end delay bound of�j�nLmax

�j�Pn

i��Lmax

Cican be provided to the connection in

both cases� Compared to Table II� the above delay boundis identical to that provided by WFQ� WF�Q� and VirtualClock servers� The about assignments are just examplesto illustrate the �exibility of rate�controlled service disci�plines� More elaborate assignments of regulators and localdelay bounds can achieve higher network utilization ����With rate�controlled service disciplines� since the tra�c canbe characterized throughout the network� end�to�end delaybounds can be derived for general resource assignments�WFQ� WF�Q� and Virtual Clock do not have such a prop�erty� In fact� it has been shown in ��� that by properlysetting parameters for regulators and local delay boundsfor schedulers� rate�controlled service disciplines can alwaysout�perform FFQ�based disciplines in terms of the numberof connections that can be accepted�Compared to FFQ�based disciplines� rate�controlled ser�

vice disciplines have the additional advantage of requiringless bu�er space inside the network to prevent packet loss�Based on ���� ���� ��� and Table IV� it can be easilyshown that the total amount of bu�er space required forconnection j in a network of rate�controlled servers is�

j � ��n� �Lmax � ��n��Xi��

Lmax

Ci

�Lmax

Cn

j ��

which is less than

j � ��n� �Lmax ���

Alternatively� based on Table II� the total amount of bu�erspace required for connection j in a network of WFQservers is

nj �n�n� �

�Lmax ���

Since j� which is the maximum burst size� is usually muchlarger than a packet size� the terms with j dominate ���and ���� While the amount of the bu�er space requiredfor a connection increases linearly with the number of hopswhen WFQ is used� the amount of bu�er space is almostindependent of the number of hops when rate�controlledservice disciplines are used�

H� Bounding Delay in a Single Scheduler

In the previous section� we showed that end�to�end de�lay bounds can be provided in a network of non�work�conserving servers only when the local delay bound can beprovided at the scheduler in each server� Many schedulerssuch as FCFS� SP� and EDD can be used� Various analysistechniques have been developed to bound the delay in asingle scheduler when the input tra�c to the scheduler is

total bits

cumulative arrivals

link speed

cumulative services

time

backlog

busy period

Fig� ��� Concepts Delay� Backlog and Busy Period

constrained� In a rate�controlled server� the input tra�c tothe scheduler is always constrained due to the use of regu�lators� Therefore� these analysis techniques can be directlyapplied�

Figure �� illustrates the basic concept used in the anal�ysis developed by Cruz ���� The horizontal axis is timeand the vertical axis is bits� The upper curve representsthe total number of bits that have arrived in the schedulerby time t and the lower curve represents the total numberof bits transmitted by time t� The di�erence between thetwo curves is the number of bits currently in the queue� orthe backlog function� When the backlog function returns tozero �the two curves meet there are no bits in the queueand thus a busy period has ended� The key to this analysisis that if the upper curve is a deterministic bounding curve�then the maximum delay can be expressed as a function ofthe two curves� For example� the following two observa�tions hold� the maximum busy period provides an upperbound on delay for any work�conserving server� the maxi�mum backlog divided by the link speed provides an upperbound on delay for a FCFS server� Delay bounds for otherpolicies can also be expressed ���� ���� �� �� �����

Table V shows delay bound tests for FCFS� SP� andEDD schedulers as derived in �� �� Notice that while aFCFS scheduler only provides one delay bound and anSP scheduler provides a �xed number of delay bounds�an EDD scheduler can provide a continuous spectrum ofdelay bounds� In an integrated services networks whereapplications have diverse tra�c characteristics and per�formance requirements� the �exibility of allocating delaybounds a�ects the utilization that can be achieved by guar�anteed service tra�c� In ����� it is shown that SP and EDDschedulers can out�perform FCFS scheduler signi�cantly interms of link utilization when connections have di�erent de�lay bounds� However� there is little di�erence in achievablelink utilization between SP and EDD schedulers� Since anSP scheduler has only a �xed number of FCFS queues� itis much easier to implement than an EDD scheduler whichrequires a sorted queue mechanism� Thus� an SP schedulerstrikes a good balance between simplicity of implementa�tion and �exibility in allocating delay bounds �����

�

Delay Bound Test Condition

FCFS d �Xj�N

bj�t� tmaxk�N

sk for all t � ��

SP ��� � dp t � �Xj�Cp

bj�t

p��Xq��

Xj�Cq

bj�t � maxr�p

sr for all p� t � ��

EDF

�������

t �Xj�N

bj�t� dj for all t � ��

t �Xj�N

bj�t� dj maxdk�t

sk for all d� � t � djN j �

TABLE V

Delay bound tests for FCFS� SP� and EDF packet schedulers�

I� Implementation Issues

Among the four non�work�conserving disciplines dis�cussed in this paper� HRR� Stop�and�Go� and RCSP all usea non�preemptive Static Priority scheduler� Only Delay�EDD use an EDD scheduler which requires a sorted pri�ority queue mechanism� The complexity of implementingsorted priority queue has been discussed in III�H� AmongHRR� Stop�and�Go� and RCSP� the former two disciplinesimplement the rate�controller and the scheduler using oneframing mechanism while RCSP needs to implement bothusing separate mechanisms�

Priority Level

...

nFIFO Queue

FIFO Queue

O O

Tnn

T

1T

1T

1OO

FIFO Queue

FIFO Queue

Fig� ��� Implementation of Stop�and�Go

To implement Stop�and�Go� mechanisms are needed atboth the link level and at the queue management level� Atthe link level� a framing structure is needed� and there is asynchronization requirement such that the framing struc�ture is the same at both the sending and the receiving endsof the link� At the queue management level� two FIFOqueues are needed for each priority level� one storing theeligible packets ready to be transmitted� the other storingthe packets that won�t be eligible until the end of the cur�rent frame time� Mechanisms are needed to swap the twoFIFO queues at the start of each frame time� Also� the setof FIFO queues with eligible packets need to be servicedaccording to a non�preemptive static priority policy� Thisis shown Figure ���

HRR does not need the framing structure at the linklayer� However� it requires bu�ering on a per connectionbasis and a set of timers to perform rate�control� An im�plementation of a prototype HRR server with �� prioritylevels has been reported �����

SchedulerRegulator

Current Time Pointer

Array of Linked Lists

Calendar

Priority Levels

...

...

...

...

...

...

...

Clock

1

2...

n

...output

...

...

...

...

Fig� ��� Implementation of RCSP

RCSP seems to be more complex than Stop�and�Go andHRR since it requires tra�c regulation on a per connectionbasis� However� the conceptual decomposition of the ratecontroller into a set of regulators in RCSP does not implythat there must be multiple physical regulators in an im�plementation� a common mechanism can be shared by alllogical regulators� Figure � shows an example implemen�tation of RCSP based on a modi�ed version of a calendarqueue ���� A calendar queue consists of a clock and a calen�dar� which is a pointer array indexed by time� Each entryin the calendar points to an array of linked lists indexedby priority levels� The clock ticks at �xed time intervals�Upon every tick of the clock� the linked lists in the arrayindexed by the current time are appended at the end ofthe scheduler�s linked lists� Packets from the linked list ofone priority level in the rate�controller are appended to thelinked list of the same priority level in the scheduler� The

��

scheduler just selects the �rst packet at the highest priorityqueue that is non�empty� As can be seen� the data struc�tures used in the proposed implementation are simple� ar�rays and linked lists� The operations are all constant�timeones� array indexing� insertion at the tail of a linked list�deletion from the head of a linked list� Another implemen�tation of RCSP that is based on a two�dimensional shiftersis proposed in �����We would like to point out that a calendar queue is a sim�

pler mechanism that a sorted priority queue� In a calendarqueue� only packets pointed by the current time pointer aredequeued at every clock tick� In a sorted priority queue�the next packet needs to be dequeued each time the server�nishes service of the current packet� If the sorted queueis implemented by a calendar queue� the dequeueing oper�ation potentially needs to go through all the entries in thecalendar�

J� Work�Conserving Rate�Controlled Service Disciplines

In previous sections� we showed that non�work�conserving rate�controlled service disciplines exhibit severalinteresting properties that make them desirable for sup�porting guaranteed performance service� These propertiesinclude��� End�to�end delay analysis can be decomposed into lo�cal delay analysis at each switch� and tight end�to�enddelay bounds can be derived with such simple analysisfor general resource assignments�

�� Heterogeneous servers with di�erent schedulers andregulators can be used at di�erent switches�

�� By separating the rate�control mechanism and thescheduler� the allocation of delay bounds and band�width can be decoupled without using the sorted pri�ority queue mechanism�

�� Due to the tra�c regulation inside the network� lessbu�er space is needed at each switch to prevent packetlos�

�� The tra�c at the exit of the network satis�es certaindesirable properties� for example� bounded rate or de�lay jitter�

However� non�work�conserving disciplines also have sev�eral disadvantages� First� with non�work�conserving dis�ciplines� a client is always punished when it sends morethan speci�ed� Even though this is acceptable under theguaranteed service model� it puts an extra burden on theclient to always characterize its tra�c correctly� For appli�cations that use live sources such as video conferencing�it is di�cult to come up with an accurate tra�c char�acterization before the data transmission� If connectionsare always punished whenever it sends more than speci�edregardless whether there are spare resources available atthat time� they may have to specify the characterizationbased on an over�estimation of the tra�c� which results ina waste of resources� Secondly� while non�work�conservingdisciplines optimize for guaranteed performance service�they may negatively a�ect the performance of other pack�ets� For example� with a non�work�conserving discipline�the server will be idle if there are only guaranteed service

packets queued at the server and none of them are eligiblefor transmission� If some best�e�ort service packets arriveat the server right after these guaranteed service packetsbecome eligible� the best�e�ort packets will have to waitbefore the guaranteed service packets �nish service� How�ever� if a work�conserving policy were used� the guaranteedservice packets would have been served before the arrivalof the best�e�ort service packets� therefore� the best�e�ortservice packets would not have to wait after they arrive�A non�work�conserving rate�controlled server can be eas�

ily modi�ed to be work�conserving ����� �� �� ���� In awork�conserving rate�controlled server� there is one morequeue in the scheduler� called the stand�by queue ����� Itworks as follows�

� All the packets in the rate�controller are also queuedin the stand�by queue� Packets are inserted or deletedfrom the rate�controller and the stand�by queue simul�taneously�

� The scheduler will service the next packet in the stand�by queue only if there are no non�guaranteed packetsand eligible guaranteed packets in the scheduler�

The stand�by queue allows the non�eligible packets to standby at the scheduler� so that they can be transmitted whenthere is spare capacity at the output link�In ���� it has been shown that the resulted work�

conserving rate�controlled server can provide the same end�to�end delay bound as its non�work�conserving counter�part� Among the �ve properties listed at the beginningof the section� the �rst three� and perhaps the more impor�tant ones among all� still hold for rate�controlled serverswith stand�by queues�As a last note in the section� we would like to point out

that even without the stand�by queue� a rate�controlleddiscipline does not necessarily have to be non�work�conserving� In ���� it has been shown that the Worst�caseFair Weighted Fair Queueing �WF�Q is equivalent to arate�controlled server with a WFQ scheduler and regula�tors de�ned by�

eki � bki�FFQ ���

where bki�FFQ is the time the packet starts service in thecorresponding FFQ system�In addition� it has been shown that WF�Q is work�

conserving� Notice that the regulator de�ned above is nei�ther a rate�jitter controlling regulator� which is de�ned bya tra�c constraint function� nor a delay�jitter controllingregulator� which is de�ned by the local delay bound at theprevious server� Instead� it is de�ned with reference to aFFQ system� therefore� the eligibility times of packets aredependent on the system load�

V� Summary

In this paper� we have examined a number of packet ser�vice disciplines that have been proposed to support guar�anteed performance service connections in packet�switchingintegrated services networks� As shown in Figure � � thesedisciplines can be classi�ed along two dimensions� �� howthe service discipline allocates� explicitly or implicitly� dif�

��

With Stand-by Queues

Rate-controlled servers

WF 2Q

Rate-jitter Control

Delay-jitter Control Jitter-EDD RCSPStop-and-Go

Per Connection Parameter

Index Update Based-on

conserving )work

WFQ

SCFQ

Index Update Based-on

HRR

Delay-EDD

Virtual Clock

Reference Queueing Model

AccommodateDistortion( work

conserving )

( nonDistortionControl

Static Priority

Scheduler using

Sorted Queue

Scheduler usingMulti-level

FramingQueue

Sorted Priority

Multiple

Interaction

of

Servers

Delay/Bandwidth Allocation

Server has one mechanism Server has two mechanisms:Rate-controller and Scheduler

Fig� ��� Taxonomy of Service Disciplines

ferent delay bounds and bandwidths to di�erent connec�tions in a single server� �� how the service discipline han�dles tra�c distortions in a networking environment�

The �rst issue relates to the design of a single server� Theobjective of the allocation of delay bound and bandwidthis that� with a certain discipline� a connection can be guar�anteed to receive a certain throughput� and each packet onthat connection can be guaranteed to have a bounded de�lay� In addition to the scheduler� which is responsible formultiplexing packets from di�erent connections and choos�ing the next packet to transmit� a server can also have arate�controller� To provide di�erent quality of services todi�erent connections� a server needs to discriminate pack�ets based on their performance requirements� Either a dy�namic sorted priority queue or a static priority queue canbe used for this purpose� In the case when the server con�sists of a static priority scheduler and no rate�controller�additional mechanisms are needed to ensure that packetsat higher priority levels do not starve packets at lower pri�ority levels� Toward this end� Stop�and�Go and HRR adoptnon�work�conserving multi�level framing strategies� Whencompared to the more general rate�controlled service disci�plines� mult�level framing su�ers from a number of disad�vantages�

The second issue concerns the interaction between dif�ferent servers along the path traversed by the connec�tion� Since the tra�c pattern of each connection canbe distorted inside the network due to load �uctuations�the server either needs to accommodate the distortion bybu�ering or control the distortion by regulating the tra�cinside the network� Controlling tra�c pattern distortionrequires non�work�conserving disciplines� which can be im�plemented by either using a multi�level framing strategy ordecoupling the server into a rate�controller and a scheduler�

There are two classes of algorithms to control tra�c patterndistortion� delay�jitter control� which maintains the sametra�c characteristics at each switch as that at the previ�ous switch� and rate�jitter control� which shapes the tra�caccording to a pre�speci�ed tra�c constraint function� Allwork�conserving disciplines use the sorted priority queuemechanism� This is not coincidental� Only a sorted prior�ity queue has the �exibility to perform both functions ofdelay bound�bandwidth allocation and adjusting for tra�cpattern distortions�

To provide guaranteed performance service� end�to�enddelay bounds need to be provided in a networking envi�ronment on a per connection basis� Various analysis tech�niques have been developed� One solution is to analyze theworst�case local delay at each switch independently andbound the end�to�end delay of a connection by using thesum of the local delay bounds at all switches traversed bythe connection� Alternatively� it has been observed thatsmaller end�to�end delay bounds can be obtained by tak�ing into account the delay dependencies among successiveswitches traversed by the connection� In general� for bothtypes of solutions� the tra�c needs to be characterized ona per connection basis at each switch inside the network�For most of the proposed work�conserving disciplines� dueto the di�culty of characterizing tra�c inside the network�tight end�to�end delay bounds can be derived only for a re�stricted class of resource assignment strategies called rate�proportional assignments� With rate�proportional assign�ment� the allocation of delay bounds and bandwidth arecoupled� For rate�controlled disciplines� since tra�c is reg�ulated inside the network� tight end�to�end delay boundscan be derived for general resource assignments� It hasbeen shown in ��� that by properly setting parametersfor regulators and local delay bounds for schedulers� rate�

��

controlled disciplines can always out�perform WFQ type ofdisciplines in terms of the number of connections that canbe accepted�Among the proposed algorithms� rate�controlled service

disciplines ����� ���� which separate the server into a ratecontroller and a scheduler� exhibit the following distinctadvantages� � simpli�ed stability analysis� which allowstight end�to�end delay bounds to be derived for general re�source assignments� � decoupling delay bound and band�width allocation without using the sorted priority queue�and � allowing heterogeneous servers with di�erent sched�ulers and regulators to be used at di�erent switches� Whilerate�controlled service disciplines are in general non�work�conserving� which has the additional advantage of requiringless bu�er space within the network to prevent packet loss�they can be easily modi�ed to be work�conserving by in�troducing a stand�by queue�Although we have provided important insights into the

issues and tradeo�s of designing service disciplines for inte�grated services networks� there are several important prob�lems that remain unresolved and need to be addressedin future research� For example� it has been shown thattight end�to�end delay bounds can be derived under gen�eral resource assignments for rate�controlled service dis�ciplines but can only be derived under rate�proportionalresource assignments for most work�conserving disciplinesother than those modi�ed from rate�controlled servers� Fu�ture work should develop more advanced techniques tobound end�to�end delay under general resource assignmentsfor FFQ�based work�conserving disciplines Also� how im�portant is it to have general resource assignments� Howmuch higher network utilization can be achieved with gen�eral resource assignments compared with rate�proportionalresource assignments� and under what tra�c mix condi�tions and network environments� We leave these questionsfor future research�As a �nal note� we would like to point out that the focus

the paper is on service disciplines for guaranteed perfor�

mance service� Other services such as the predicted ser�vice and various types of best�e�ort services have di�erentrequirements� and there will be di�erent tradeo�s in de�signing service disciplines for these services� For example�for the same resource assignment� WFQ and WF�Q alwaysprovide identical end�to�end delay bounds for all connec�tions� However� as discussed in ��� and Section III�B� theservices that they provide or best�e�ort tra�c can be quitedi�erent� Issues in designing service disciplines for networkservices other than the guaranteed performance service arebeyond the scope of the paper�

VI� Acknowledgement

The author would like to thank Ed Knightly� Jon C�R�Bennett and anonymous referees for providing insightfulcomments that greatly improved the presentation of thepaper�

References

��� A� Banerjea and S� Keshav� Queueing delays in rate controlled

networks� In Proceedings of IEEE INFOCOM���� pages �������� San Francisco� CA� April �����

��� J�C�R� Bennett and H� Zhang� WF�Q Worst�case fair weightedfair queueing� In Proceedings of IEEE INFOCOM���� pages�������� San Francisco� CA� March �����

��� P� Brady� A techniques for investigating on�o� patterns inspeech� Bell System Technical Journal� �� ����� January �����

��� R� Brown� Calendar queues A fast O�� priority queue imple�mentation for the simulation event set problem� Communica�tions of the ACM� ����� ���������� October �����

��� C� Chang� Stability� queue length� and delay of deterministic andstochastic queueing networks� IEEE Transactions on AutomaticControl� ���� �������� May �����

��� H� Chao� Architecturedesign for regulatingand schedulinguser�stra�c in ATM networks� In Proceedings of ACM SIGCOMM����pages ������ Baltimore� Maryland� August �����

��� D� Clark� S� Shenker� and L� Zhang� Supporting real�time appli�cations in an integrated services packet network Architectureand mechanism� In Proceedings of ACM SIGCOMM���� pages������ Baltimore� Maryland� August �����

��� R� Cruz� A calculus for network delay� part I Network ele�ments in isolation� IEEE Transaction of Information Theory����� �������� �����

��� R� Cruz� A calculus for network delay� part II Network analysis�IEEE Transaction of Information Theory� ���� �������� �����

���� R� Cruz� Service burstiness and dynamic burstiness measures A framework� Journal of High Speed Networks� ��� �������������

���� J� Davin and A� Heybey� A simulation study of fair queue�ing and policy enforcement� Computer Communication Review����� ������ October �����

���� A� Demers� S� Keshav� and S� Shenker� Analysis and simula�tion of a fair queueing algorithm� In Journal of InternetworkingResearch and Experience� pages ����� October ����� Also inProceedings of ACM SIGCOMM���� pp �����

���� D� Ferrari� Client requirements for real�time communication ser�vices� IEEE Communications Magazine� ����� ������ Novem�ber �����

���� D� Ferrari� Real�time communication in an internetwork� Jour�nal of High Speed Networks� ��� ������� �����

���� D� Ferrari� A� Banerjea� and H� Zhang� Network support formultimedia a discussion of the Tenet approach� Computer Net�works and ISDN Systems� ����� ���������� July �����

���� D� Ferrari and D� Verma� A scheme for real�time channel es�tablishment in wide�area networks� IEEE Journal on SelectedAreas in Communications� ��� �������� April �����

���� N� Figueira and J� Pasquale� An upper bound on delay forthe virtualclock service discipline� IEEE�ACM Transactions onNetworking� ���� April �����

���� A� Fraser� Designing a public data network� IEEE Communica�tions Magazine� ����� ������ October �����

���� L� Georgiadis� R� Gu�erin� and V� Peris� E�cient network QoSprovisioning based on per node tra�c shaping� In IEEE INFO�COM���� San Francisco� CA� March �����

���� S� Golestani� Congestion�free transmission of real�time tra�c inpacket networks� In Proceedings of IEEE INFOCOM��� pages�������� San Francisco� California� June ����� IEEE Computerand Communication Societies�

���� S� Golestani� A stop�and�go queueing framework for congestionmanagement� In Proceedings of ACM SIGCOMM��� pages ����� Philadelphia� PA� September �����

���� S� Golestani� A self�clocked fair queueing scheme for broadbandapplications� In Proceedings of IEEE INFOCOM��� pages �������� Toronto� CA� June �����

���� P� Goyal� S� Lam� and H� Vin� Determining end�to�end delaybounds in heterogeneous networks� In Proceedings of the �th In�ternational Workshop on Network and Operating System Sup�port For Digital Audio and Video� pages �������� Durham� NewHampshire� April �����

���� V� Jacobson� Congestion avoidance and control� In Proceedingsof ACM SIGCOMM���� pages �������� August �����

���� R� Jain� Congestion control in computer networks Issues andtrends� IEEE Network Magazine� pages ������ May �����

���� C� Kalmanek� H� Kanakia� and S� Keshav� Rate controlledservers for very high�speed networks� In IEEE Global Telecom�munications Conference� pages ������� � �������� San Diego�California� December �����

�

���� C� Kalmanek� S� Morgan� and R� C� Restrick� A high per�formance queueing engine for ATM networks� In Proceedingsof th International Switching Symposium� Yokahama� Japan�October �����

���� C� Kalmannek and R� Restrick� Xunet � queue module� October����� AT�T Bell Laboratories internal technical report�

���� D� Kandlur� K� Shin� and D� Ferrari� Real�time communica�tion in multi�hop networks� In Proceedings of th InternationalConference on Distributed Computer Systems� May �����

���� M� Karol� M� Hluchyj� and S� Mogan� Input versus output queue�ing on a space�division packet switch� IEEE Transactions onCommunications� ����� ���������� December �����

���� S� Keshav� A control�theoretic approach to �ow control� In Pro�ceedings of ACM SIGCOMM�� � pages ����� Zurich� Switzer�land� September �����

���� L� Kleinrock� Queueing Systems� John Wiley and Sons� ��������� L� Kleinrock� Queueing Systems� Volume �� Computer Appli�

cations� Wiley� ��������� E� Knightly� D� Wrege� J� Liebeherr� and H� Zhang� Fundamental

limits and tradeo�s for providing deterministic guarantees toVBR video tra�c� In Proceedings of ACM Sigmetrics���� pages�������� Ottawa� CA� May �����

���� E� Knightly and H� Zhang� Tra�c characterization and switchutilization using deterministic bounding interval dependent traf��c models� In Proceedings of IEEE INFOCOM���� Boston� MA�April �����

���� D� Knuth� The Art of Computer Programming� Volume �� Sort�ing and searching� Addison�Wesley� �����

���� J� Kurose� On computing per�session performance bounds inhigh�speed multi�hop computer networks� In ACM Sigmet�rics���� �����

���� J� Kurose� Open issues and challenges in providing quality of ser�vice guarantees in high�speed networks� ACM Computer Com�munication Review� ���� ����� January �����

���� A� Lazar and C� Paci�ci� Control of resources in broadband net�works with quality of service guarantees� IEEE CommunicationMagazine� pages ������ October �����

���� J� Liebeherr� D� Wrege� and D�Ferrari� Exact admission controlfor networks with bounded delay services� Technical Report CS������� University of Virginia� Department of Computer Science�July �����

���� C� Liu and J� Layland� Scheduling algorithms for multipro�gramming in a hard real�time environment� Journal of ACM����� ������ January �����

���� S� Low� Tra�c Control in ATM Networks� PhD dissertation�University of California at Berkeley� May �����

���� B� Maglaris� D� Anastassiou� P� Sen� G� Karlsson� and J� Rob�bins� Performance models of statistical multiplexing in packetvideo communications� IEEE Transaction on Communication����� �������� July �����

���� M� Maresca� June ����� Personal communication����� R� Nagarajan� J� Kurose� and D� Towsley� Local allocation of

end�to�end quality�of�service in high�speed networks� In IFIPTC� Task Group�WG�� International Workshop on Perfor�mance of Communication Systems� pages ������� Martinique�January �����

���� I� Nikolaidis and I� Akyildiz� Source characterization and statis�tical multiplexing in atm networks� Technical Report GIT�CC������� College of Computing� Georgia Institute of Technology�Atlanta� GA ����������� �����

���� A� Parekh� A Generalized Processor Sharing Approach to FlowControl in Integrated Services Networks� PhD dissertation�Mas�sachusetts Institute of Technology� February �����

���� A� Parekh and R� Gallager� A generalized processor sharingapproach to �ow control � the single node case� In Proceedingsof the INFOCOM���� �����

���� A� Parekh and R� Gallager� A generalized processor sharingapproach to �ow control in integrated services networks Themultiple node case� In Proceedings of the INFOCOM���� pages�������� San Francisco� CA� March �����

���� C� Parris� H� Zhang� and D� Ferrari� Dynamic management ofguaranteed performance multimedia connections� MultimediaSystems Journal� � �������� �����

���� K� Ramakrishnan� D� Chiu� and R� Jain� Congestion avoidancein computer networks with a connectionless network layer� InProceedings of ACM SIGCOMM���� pages �������� Stanford�CA� August �����

���� D� Saha� S� Mukherjee� and S�Tripathi� Multi�rate tra�c shap�ing and end�to�endperformanceguarantees in ATM networks� InProceedings of �� International Conference on Network Pro�tocols �ICNP���� Boston� MA� October ���

���� J� Stankovic and K� Ramamritham� Hard Real�Time Systems�IEEE Computer Society Press� �����

���� J� Turner� New directions in communications�or which way tothe information age�� IEEE Communication Magazine� ������October �����

���� D� Verma� Guaranteed Performance Communication in HighSpeed Networks� PhD dissertation� University of California atBerkeley� November �����

���� D� Verma� H� Zhang� and D� Ferrari� Guaranteeing delay jitterbounds in packet switching networks� In Proceedings of Tri�comm�� � pages ������ Chapel Hill� North Carolina� April �����

���� R� Wol�� Stochastic Modeling and the Theory of Queues� Pren�tice Hall� �����

���� G� Xie and S� Lam� Delay guarantee of virtual clock server�IEEE�ACM Transactions on Networking� ��� ��������Decem�ber �����

���� O� Yaron and M� Sidi� Calculating performance bounds in com�munication networks� In Proceedings of IEEE INFOCOM����pages �������� San Francisco� CA� April �����

���� O� Yaron and M� Sidi� Performance and stability of commu�nication networks via robust exponential bounds� IEEE�ACMTransactions on Networking� ��� �������� June �����

���� H� Zhang� Providing end�to�end performance guarantees usingnon�work�conserving disciplines� To appear in Computer Com�munications� Special Issue on System Support for MultimediaComputing�

���� H� Zhang and D� Ferrari� Rate�controlled static priority queue�ing� In Proceedings of IEEE INFOCOM���� pages �������� SanFrancisco� California� April �����

���� H� Zhang and D� Ferrari� Improving utilization for deterministicservice in multimedia communication� In �� InternationalConference on Multimedia Computing and Systems� pages �������� Boston� MA� May �����

���� H� Zhang and D� Ferrari� Rate�controlled service disciplines�Journal of High Speed Networks� ��� �������� �����

���� H� Zhang and S� Keshav� Comparison of rate�based service dis�ciplines� In Proceedings of ACM SIGCOMM�� � pages ��������Zurich� Switzerland� September �����

���� H� Zhang and E� Knightly� Providing end�to�end statistical per�formance guarantees with interval dependent stochastic models�InACM Sigmetrics��� pages �������� Nashville� TN� May �����

���� L� Zhang� Virtual clock A new tra�c control algorithmfor packet switching networks� In Proceedings of ACM SIG�COMM��� pages ������ Philadelphia� PA� September �����

���� Z� Zhang� D� Towsley� and J� Kurose� Statistical analysis of gen�eralized processor sharing scheduling discipline� In Proceedingsof ACM SIGCOMM��� London� UK� August �����

���� Q� Zheng and K� Shin� On the ability of establishing real�timechannels in point�to�point packet�switching networks� IEEETransactions on Communications� pages ���������� March�����