THROUGHPUT ANALYSIS OF STOCHASTIC NETWORKS USING FLUID...

Submitted to the Annals of Applied Probability

THROUGHPUT ANALYSIS OF STOCHASTIC NETWORKSUSING FLUID LIMITS.

By John Musacchio∗

University of California, Santa Cruzand

By Jean Walrand∗

University of California, Berkeley

We study the throughput of a stochastic network with inputpolicing. Such policing can be used to improve the fairness of packetswitches and of communication networks as well as of service sys-tems. Our approach consists in showing that the throughput of thenetwork approaches that of a corresponding fluid network. The typeof queueing network we study consists of multiple stations, carriesmultiple flows of customers, each flow is queued separately at eachstation, and there is a single server at each station serving the queuesin a round-robin fashion. The particular control scheme we study dis-cards packets that arrive when any one of the queues holds more thana specified number of packets of the same flow, and does so until allof that flow’s queues fall below their thresholds. To illustrate the ap-plication of our results, we analyze a network resembling a 2-input,2-output communications network switch.

We consider a queuing network of d single-server stations carrying g flows.The stations are equipped with per-flow queues, and the network uses aningress policing mechanism in which a flow’s packets are discarded at thenetwork ingress whenever any of that flow’s queues exceed a threshold. Eachflow f has a weight wf , and each station serves a flow in proportion to itsweight using a weighted round robin or a similar queueing discipline likeweighted fair queueing or generalized head of line processor sharing. Eachflow’s packets arrive according to an independent renewal process of somemean rate, and the service times at each station are also independent. With-out discarding at the ingress, the utilization of some stations may exceed1, and therefore the policing mechanism is necessary for the stability of thenetwork.

∗Research supported in part by NSF Grant ANI-0331659.AMS 2000 subject classifications: Primary 60K25, 68M20, 68M10; secondary 68K20.Keywords and phrases: Fluid Limit, Stochastic Network

1imsart-aap ver. 2006/03/07 file: fluidpaper_post0804.tex date: September 13, 2006

2 J. MUSACCHIO & J. WALRAND

Our main result is to show that there exist large enough policing thresh-olds so that each flow’s long term average throughput can be made arbitrar-ily close to the weighted max-min fair allocation. The other key contributionof this work is to show how the analysis of such a network’s flow rates canbe reduced to analyzing the rates of a corresponding fluid model.

1.1. Motivation. Achieving fairness between flows is one of the principaldesign challenges of any packet switched communication network. In today’sInternet architecture, the Transmission Control Protocol (TCP) bears mostof the burden in achieving fairness [16–18]. TCP is an end-to-end conges-tion protocol that infers congestion when packets are lost and responds tothat congestion with a mechanism known as additive increase multiplicativedecrease. That mechanism halves the rate of the user’s session when con-gestion is perceived (multiplicative decrease), and slowly increases the ratewhen it is not (additive increase). TCP has been tremendously successful incoping with congestion and in particular in avoiding a congestion collapseas the Internet has grown in size and speed by orders of magnitude overthe years. However, as powerful as TCP is, TCP cannot achieve fairness ifthe underlying network is grossly unfair, or if a large proportion of the traf-fic does not use a transport protocol that responds to congestion as TCPdoes. For example, if the underlying network constrains two groups of Nsessions to very different rates, where perhaps each group of sessions orig-inate in a different geographic area, then TCP might succeed in achievingfairness between sessions of a group, but not in achieving fairness betweenthe groups.

Indeed designing a network that is not unfair to particular groups ofsessions is becoming increasingly difficult. The speed of the optical linkswhich make up much of the Internet has grown tremendously over the past10 years, faster even then the pace of progress in the IC technology usedto build the switches and routers that interconnect the links [27]. This haspressured designers of switches, routers, and aggregation points to move todecentralized architectures that switch and process many packets in parallel.The architects of these new designs have been primarily concerned withmaximizing throughput, but unfortunately throughput can come at the priceof fairness.

We consider an example of a two-input and two-output switch to demon-strate the tradeoff between throughput and fairness. Suppose all input andoutput links of the switch have capacity 1, and label the input and outputinterfaces In-1, In-2, and Out-1, Out-2 respectively. Suppose that groups of

imsart-aap ver. 2006/03/07 file: fluidpaper_post0804.tex date: September 13, 2006

THROUGHPUT ANALYSIS USING FLUID LIMITS. 3

Input 1

Input 2

Output 1

Output 2

1-2 Traffic

1-1 Traffic

2-2 Traffic

1-1 VOQ

1-2 VOQ

2-2 VOQ

Fig 1. A two input, two output switch with virtual output queues.

N TCP sessions are sending traffic between each of the following pairs: In-1to Out-1 (1-1 for short); In-2 to Out-2; and In-1 to Out-2. This situation isdepicted in Figure 1. In this situation, a switch that favors the 1-1 and 2-2groups can achieve a higher throughput. In the extreme case the switch canoffer the “unfair” allocation of 1 to groups 1-1 and 2-2, and 0 to the group1-2 to achieve a throughput of 2. In contrast, a more fair allocation wouldbe to offer 0.5 to all three groups, yielding a throughput of only 1.5. Whilethe unfair allocation scenario is extreme, it is possible that a switch with areasonable design might tend toward this extreme case.

Consider the common switch architecture in which each input interfacesplits its traffic into fixed length cells, and then sorts the traffic into “virtual”output queues corresponding to the traffic’s final destination [22]. Then inevery time slot corresponding to the transmission time of one cell, a schedulermatches pairs of virtual queues to corresponding output interfaces. In ourscenario, a scheduler optimized for throughput would favor matching 1-1and 2-2 in each slot so as to move two cells, and not just one as wouldhappen for a 1-2 match.

The end-to-end TCP protocol would not correct the problem. The 1-2sessions would see congestion because their virtual queue is not being servedfast and they would adapt their rates by slowing down. Consequently, the1-1 and 2-2 sessions would find less competition on the interfaces they sharewith 1-2 traffic, adapt their rates higher allowing them to get even more oftheir traffic to their virtual queues, where that traffic will again be favoredby the scheduler over the 1-2 traffic in the 1-2 virtual output queue.

Examples like this show that it is important to design switches, routersand aggregation points that treat traffic from different input output interfacepairs fairly, even with the presence of rate adaptive transport protocols likeTCP.



At the same time, the pressure to design faster and faster switch archi-tectures is making it less feasible to design switches with central schedulersthat coordinate packet movements at the sum of all the incoming line rates.As switch designers are pressured into eliminating the central scheduler en-tirely, the design of a switch is becoming more and more like the designof a network, where the “nodes” of the network are the interfaces and theinterconnections between interfaces – the switch fabric – are the “links.”

This trend toward decentralized architectures, combined with the impor-tance of fairness in switches, are transforming the switch fabric design prob-lem into a problem of designing a queueing network that achieves fair rateswith an extremely simple control policy. The control policy must be simpleso that it can be implemented in silicon at the extremely fast line rates oftoday’s networks.

Surprisingly, while switch design is becoming more and more like networkdesign, there are pressures to make the network look more and more likea switch. In particular, Mckeown and Zhang-Shen [30] propose using theClos topology, popular in switch fabric design, as the basis of a next genera-tion backbone for the proposed “100× 100 network” – a network envisionedto interconnect 100 million users with an access rate of 100 Mbps [2]. TheClos topology consists of some number of input nodes which are fully in-terconnected with some number of middle nodes, which in turn are fullyinterconnected with some number of output nodes [29]. The Clos topologyis popular in switch design because when combined with a simple load bal-ancing algorithm, it can achieve full throughput between inputs and outputsfor any feasible demand rate matrix – meaning any demand rates which donot exceed the capacity of any output. This feature would make it useful inthe 100×100 network backbone, because it would automatically achieve fullthroughput without a complicated routing algorithm. However, the sameissues that apply for a switch also apply for a backbone designed to looklike a switch – fairness is important, and is not automatic in a throughoutmaximizing architecture. Therefore a methodology for designing a queuingnetwork that achieves fairness with a simple control law would be extremelyuseful in the context of networks like the 100 × 100.

The policing scheme that we analyze in this work is extremely simple,achieves fair rates as we show, and therefore would be useful in a networklike 100×100 or in a switch design. Yet the analysis of such a simple schemecould become extremely complex and involve detailed case by case analysisof sequences of discrete events in order to prove that the scheme achievesfair rates.

Therefore we feel that the principal contribution of this work is not the



scheme itself, but rather the methodology we use to analyze it. We show thatthe analysis of the discrete stochastic system can be reduced to analysis of afluid model obeying deterministic differential equations. Though we presentthe methodology in the context of our policing scheme, the same techniquescould be used to analyze other simple rate control schemes for queueingnetworks.

While the primary application area that motivates this work is commu-nication networks, there are other notable application areas. For example itmight be necessary to control the flows of jobs in a manufacturing system,where perhaps the flows correspond to different products or different cus-tomers, and the stations correspond to the capacity constrained pieces ofequipment – a lithography stepper at a semiconductor manufacturing plantfor example. In such a context, it would be very desirable to have a con-trol scheme that ensures that the different flows of jobs get fair rates at thebottleneck stations and at the same time ensure that the queues at each sta-tion remain bounded. In our scheme, the queues would be kept bounded byblocking new jobs from entering the manufacturing plant when a bottleneckstation is congested, but other types of blocking schemes could be analyzedwith the methodology presented in this work.

Another natural application area is for call centers. In a call center, cus-tomers are queued until a server can take the customer’s call. Oftentimesthe first server may not be able to fulfill all of the customer’s needs andtherefore the server may need to transfer the call to a second server, wherethe customer may have to enter yet another queue before seeing the secondserver. In turn, the second server might need to transfer the call again to athird server and so on. Also, the same call center maybe answering calls ofdifferent types. One example would be a contracted call center that answerscalls for several different corporate clients simultaneously. Here the set ofincoming calls for a particular client would be the “flow,” and a call centerwould want to treat the different flows fairly in times of congestion. Fur-thermore, it might be better for the call center to block calls at the ingress,rather than having a customer put on hold, transferred, and put on holdagain, only eventually to be dropped. Therefore a flow-control scheme likethe one we study in this work is a logical choice for such an application.

1.2. Related Work. As we have stated, our objective is to show thatour flow control scheme with large enough thresholds achieves rates closeto the max-min fair allocation. A max-min fair flow allocation is a feasibleallocation such that one flow’s rate cannot be increased without decreasingthe rate of another flow that already has a smaller allocation [1]. There have



been a number of schemes proposed and analyzed that aim to achieve max-min fairness, and most such investigations have involved detailed case bycase analysis of the possible state trajectories of the network. For example,Hahne shows that a network with per-flow queued stations using round-robin service, and with hop by hop window flow control achieves max-minfairness [11]. Round-robin service is the now familiar queuing discipline inwhich the server polls the queues in cyclic order, offering service to queuesthat are nonempty [12, 20]. We adopt a qualitatively different approach toour analysis. We consider a fluid limit of the network obtained by scalingtime and space (including the thresholds) by a factor n that increases –an approach similar to that found in [3, 5, 21] and others. This allows usto exploit the Strong Law of Large Numbers to show that the arrival andservice process converge to smooth, linear growths and also that the otherprocesses describing the system converge to a trajectory of a fluid modeldescribed by simple differential equations.

In addition to the difference in methodology from previous investigations,our network and flow control model is also different than those studied inprevious work. The model that most closely resembles ours is the networkanalyzed in [11]. However, that network differs from our network in that itsflow control is hop by hop rather than done at the ingress, and the arrivaland service times assumptions are more restrictive than being renewal aswe assume. In particular [11] assumes arrivals are either deterministic orBernoulli, and service times are deterministic.

Another related work is by Hayden, who proposes an iterative mechanismfor finding max-min fair rates. In the mechanism, each station measures therates of the flows passing through it, computes a fair rate allocation for theseflows, and then explicitly tells sources what rates to use [15]. Gafni alsoproposes an improvement to Hayden’s original algorithm [10]. The analysisof our fluid model resembles Hayden’s proof in that, as in [15], we stepthrough the stations of the network, at each step finding the most congestedstation, assign rates to the flows passing through it, subtract those flowsfrom the network, and proceed to find the next most congested station inthe reduced network. However, our problem requires extra analysis to showthat within each of these steps the fluid model, described by a set of ODEs,converges. Our proof that the stochastic network converges to a fluid modelis not related to [15].

Our fluid limit proof technique is closely related to known results in theliterature. In particular, we borrow heavily from work by Dai [3]. Dai showsthat for networks without policing, stability of the fluid model implies stabil-ity of the stochastic network – stability meaning that the system is positive



Harris recurrent. Our work differs in that we use the fluid model not onlyto analyze stability, but also to find the flow rates of the stochastic network.Also, the fact that the policing modifies the arrival processes also requiresspecialized treatment to show that our network converges to a fluid limittrajectory.

There is also a difference in how the fluid limit is taken. Dai’s proof con-siders a sequence of initial conditions x, with |x| → ∞, and then obtains afluid limit by scaling time and space by |x|. Dai uses this result to constructa Lyapunov function that shows that the expected state of the system con-tracts, for initial states far enough from the origin. In contrast our proofrequires consideration of two different types of fluid limits. In the first weconsider a sequence indexed by n, and then obtain a fluid limit by scalingthe policing thresholds, as well as space and time by n → ∞. Analysis ofthis fluid limit leads us to conclude that the rates of the stochastic systemare close to that of the fluid model, over a finite time interval, if two con-ditions are met: the stochastic system’s thresholds n are large enough, andits relative initial condition y = x/n − he is small, where he is the equi-librium of the fluid model. In the second fluid limit we consider a sequenceof pairs {(n, y)}, scale the policing thresholds by n, while scaling space andtime by n|y| → ∞. As in [3], we can construct a Lyapunov function thatshows that the expected state of the system contracts for n|y| large enough.Consequently the contraction property holds for smaller and smaller y asn is increased. The combination of both fluid limit results shows that forlarge enough n the invariant distribution of the system is concentrated in aregion of the state space where the expected rates are close to those of thefluid model, and in turn the fluid model rates can be shown to be equal toa max-min allocation.

Another closely related work is by Mandelbaum, Massey, and Reiman[21]. In [21], the authors study the fluid limit of a queueing network withstate dependent routing, where the function describing the arrivals to eachqueue can scale with n and or

√n, in a manner similar to the scaling of

our thresholds. The authors prove a functional strong law of large numbersand a functional central limited theorem in the context of their model. How-ever, the authors assume that the network is driven by poisson processes,rather than just the renewal assumption that we make. An earlier work byKonstantopoulos, Papadakis, and Walrand derives a functional strong lawof large numbers and a functional central limit theorem for networks withstate dependent service rates [28].

There are also several works that use reflected Brownian motion modelsto study queueing networks with blocking [4, 13, 14]. Typically the objective



Flow 2

7

8

3

4

h

5

6

h

1

2

Queue 7, h = 100

0 2000 4000 6000 8000 100000

50

100

150

0 2000 4000 6000 8000 100000

50

100

150Queue 2, h = 100

0 200 400 600 800 10000

5

10

15Queue 2, h = 10

0 200 400 600 800 10000

5

10

15Queue 7, h = 10

Flow 1

Flow 3

Station 1

Station 2

Station 3

Station 4

Fig 2. An example of a queueing network lacking unique bottlenecks.

of most such investigations is to approximate the distribution of the queueoccupancy with a diffusion approximation, whereas in this work our objec-tive is to show almost sure convergence using a strong law of large numbersscaling.

1.3. A Simple Example. In this subsection, we study an example thatmotivates the theory developed in this paper. The example will illustratetwo important phenomena – that the rates of the stochastic system getcloser to those of a corresponding fluid model when policing thresholds areraised, and that when there are not unique bottlenecks, the vector of queuedepths is not attracted to a unique equilibrium point.

Our example is illustrated in Figure 2. The example is analogous to thetwo-input and two-output switch that we discussed in Section 1.1. Two flowsenter the network at station 1, the first input of the switch, and a third flowenters the network at station 2. We concentrate on flow 2, which sharesstations 1 and 4 with flows 1 and 3 respectively. All stations are served atrate 1, have round-robin service with equal weighting to all queues, and haveservice times that are exponentially distributed. The arrival rate of each flowis 0.6, with Pareto inter-arrival distributions given by

(1) P(ξf (j) > s) =1

(0.6s+ 1)2.



for each flow f and for s ≥ 0. We chose the Pareto distribution for thisexample to emphasize that we are in interested in networks whose inter-arrival and service times are not necessarily memoryless. Also the Paretodistribution makes it so that very long inter-arrival times are not unlikely,and thus intuition suggests that it is possible the queues could starve duringlong inter-arrival times causing the flow rates to be less than those of thecorresponding fluid model.

We study the example with two choices of policing thresholds: h = 10and h = 100. Recall that in Section 1.1 we observed that the max-min fairshare for this configuration of switch and flows is a rate of 0.5 per flow.Because flow 2 shares station 1 with flow 1 and station 4 with flow 3, flow2 is bottlenecked at both stations 1 and 4 – meaning that flow 2’s rate islimited by the available capacity at both stations. Intuition suggests thatin a fluid model of this example, flow 2’s queue at one of these bottleneckstations should reach the threshold and “chatter” there, but it is not clearwhich bottleneck will do that. Whichever bottleneck queue fills to threshold,it will share a station with a queue that is a bottleneck to another flow (1 or3), and consequently this station will share its service equally between thetwo flows. By this intuition, flow 2 should achieve a rate of 0.5. In Section 6we will more carefully verify that flow 2 has a rate of 0.5 in this example’sfluid model.

Figure 2 shows the simulated trajectories of flow 2’s queues at both bot-tleneck stations. In the h = 10 case, the simulation shows that queues 2 andqueue 7, which both serve flow 2, are empty for over 100 time units aroundtime 800. This empty period is significant because when flow 2’s queues areempty, flow 2 misses opportunities to have its packets served by the bot-tleneck stations. Indeed the table included in Figure 2 shows the averagerate, averaged over the last 80% of the simulation time to reduce some ofthe initial transient effect, is 0.391. This is substantially below the rate of0.5 predicted by the fluid model. Most likely, a string of long inter-arrivaltimes of flow 2, caused the queues at the bottleneck stations to starve.

Raising the thresholds should reduce starvation, because larger thresholdswould provide the bottleneck queues a larger backlog to smooth over fluctu-ations in the arrival process and the service processes. To test that intuition,we simulate the network with policing thresholds of h = 100. Figure 2 showsthe trajectories of Flow 2’s queues for the increased threshold. We note theneither queue spends all of the time filled to it’s threshold, but instead atmost points in time at least one or the queues is near its threshold. Forinstance, at the beginning of the simulation, the queue 7 (the second bottle-neck) is chattering near the threshold while queue 2 (the first bottleneck) is



below threshold. At some time before the 2000 second mark, the two queuesswitch these rolls. At around the 6000 second mark, the queues switch theseroles again. We also note that flow 2 achieves an average rate of 0.489, whichis much closer to the max-min rate of 0.5.

This example illustrates that by increasing the thresholds, the system’saverage throughput rates approach those of the corresponding fluid model.The example also shows that systems like this that do not have a uniquebottleneck are not attracted to a unique equilibrium point. Instead they areattracted to a set of points where at least one of the bottleneck queues isat threshold, while the other queues are below their thresholds. Similarly influid models of systems like this, one finds that there is not a unique equilib-rium point, but instead there is a set of possible equilibria. Furthermore theequilibrium the fluid model settles at is determined by its initial conditions.Our analysis will show that the stochastic system is attracted to the setof equilibria of the fluid model, and then use this result to show that theflow rates of the stochastic system approach those of the corresponding fluidmodel.

2. Model Overview. In this section, we overview the model and no-tation that we use to develop our results. In order to allow the reader todevelop a conceptual understanding of the model’s most important features,we postpone the discussion of certain details until Section 3.

As we have already said, we consider a queuing network of d single-serverstations carrying g flows. The stations are equipped with per-flow queues,and the network uses an ingress policing mechanism in which a flow’s packetsare discarded at the network ingress whenever any of that flow’s queuesexceed a threshold. Each flow f has a weight wf , and each station servesa flow in proportion to its weight using a weighted round robin or similarqueueing discipline like weighted fair queueing or generalized head of lineprocessor sharing. Each flow’s packets arrive according to an independentrenewal process, and the service times at each station are also independent.

In our analysis, the “baseline” system we consider has policing thresholdsof h in each queue. In other words when any of that flow’s queues exceed h,that flow is policed. We then consider a series of scaled systems for whichthe policing thresholds are multiplied by a scaling factor n that is increasing.The evolution of each system depends on the following three variables: thethreshold scaling factor n, an initial condition vector x which contains statedata that we detail in the next paragraph, and because this is a stochasticmodel, the outcome ω which determines the sample values of the arrival andservice times. We use the notation x = [x;n] to refer to a pairing of initial



condition and threshold scale factor.We will argue that the Markovian state of the system at time t is Xx(t) =[

Qx(t);Ux(t);V x(t);Hx(t)]

where Q is the vector of queue depths, U is the

vector of residual arrival times for each flow, V is a vector of residual servicetimes for each queue, and H is a vector of 1s and 0s that recalls if thepolicing feedback from a queue is in hysteresis. The x superscript on allthese variables emphasizes each processes dependence on initial conditionand threshold scale factor.

At times in our analysis, we will find it convenient to consider what we calla system trajectory Xx(t) = [Xx(t);Tx(t);nh] where X is the state vector,Tx(t) is a vector that tracks how long each server has been in service, andnh is the policing threshold.

Our analysis will show that the state of the stochastic system, with suffi-ciently large threshold scaling threshold n, will tend to stay near the set ofequilibria of the system’s fluid model (we will detail the equations that de-scribe the fluid model later). To reach this conclusion, we suppose that onehas found the set of equilibria for the system’s fluid model. In equilibrium,a queue of the fluid model that is not a bottleneck should be empty, and aqueue that is the unique bottleneck of a flow should be filled to its thresh-old. A queue that is a bottleneck, but not a unique bottleneck, of a flowshould have a queue that can either at the threshold, or below its thresholdprovided that one of the other bottleneck queues is at threshold. By thisobservation, it is clear that the set of equilibrium of the fluid model shouldscale in proportion to the fluid model’s threshold h. To be precise, supposethat E is the set of equilibria of the fluid model with unit thresholds. Thenthe set hE , {he : e ∈ E} should be the set of equilibria of a fluid modelwith thresholds equal to h.

We will often need to refer to the distance of the stochastic system fromthe equilibria set. To do so, we define the notation ‖a‖S , arg mins∈S ‖a−s‖.For instance the distance between the equilibria set of the fluid model withthresholds h and the stochastic system scaled in inverse-proportion to its

threshold scale factor is written∥∥∥ 1

nXx(t)

∥∥∥hE

.

3. Model. We state precisely our network model in this section, in-troducing the necessary notation along the way. Much of our notation isborrowed from [3].

3.1. Flows and Classes. As mentioned, we consider a queueing networkof d single server stations carrying g flows. In addition to the notion offlow, each packet also has a class k ∈ {1, ...,K} that is indicative of both



the packet’s flow and location in the network. Thus the class of a flow fpacket changes as the packet progresses from station to station, but withthe restriction that a flow f packet must always have a class in the set K(f).Conversely, each class k is associated with one and only one flow, and wedenote the class to flow mapping by f(k) = {f : k ∈ K(f)}. We also adoptthe numbering convention that flow f packets enter the network as classk = f , and thus f ∈ K(f).

Each class k is served by a unique station denoted s(k). When a packetof class k finishes service at station s(k), it becomes a packet of class l withprobability Pkl or exits the network with probability 1−∑l Pkl. To preservea packet’s flow, Pkl = 0, if f(k) 6= f(l).

More precisely, the jth packet of class k to complete service has associatedwith it a K dimensional “Bernoulli” random column vector φk(j) whichconsists of all zeros with probability 1−∑l Pkl, or has a single 1 at locationl with probability Pkl. The random vectors φk(j) are independent.

We will consider networks for which each flow’s packets follow loop-freepaths without “splitting.” The matrix P is binary and nilpotent for suchnetworks.

Because each class k is served at a unique station s(k), we may define theClass Constituency Matrix C with Cik = 1 if s(k) = i and Cik = 0 otherwise.At times it is more convenient to represent the same information in setrather than matrix notation, and therefore we define the Class-ConstituencyC[i] = {k : s(k) = i} for each station i. Finally, because each class belongsto a unique flow f(k), we may define the Flow-Constituency F[i] = {f : ∃k ∈C[i] : f = f(k)} for each station i.

3.2. Queues. The state of the network’s queues is the K dimensional col-umn vector Qx where Qx

k is the number of packets of class k packets queuedin the system. Recall that the x superscript emphasizes the processes’ de-pendence on initial condition.

The exogenous arrivals to the network for flow f are described by a re-newal process Ex

f (t) where the inter-arrival times {ξf (j), j ≥ 1} are i.i.d. and

have mean α−1f where αf is the mean arrival rate. We denote the remaining

waiting time for the next flow f customer arrival at time t as Uf (t). Thusthe process Ex

f (t) is described by

Ex

f (t) , max{j : Uf (0) + ξf (1) + ...+ ξf (j − 1) ≤ t}.

The service times of each class {ηk(j), j ≥ 1} are also i.i.d. and have meanmk = µ−1

k , where µk is the mean service rate. We also define the K × Kdiagonal matrix M where the kth entry on the diagonal is mk. Vk(t) denotes



the remaining service time of the class k customer in service, if there is oneat time t, otherwise Vk(t) = 0. We define a service process Sx

k (t) as

Sx

k (t) , max{j : Vk(0) + ηk(1) + ...+ ηk(j − 1) ≤ t}

where Vk(0) = Vk(0) if Vk(0) > 0, otherwise Vk(0) = ηk(0) is a fresh servicetime with the same distribution as ηk(1) and independent of all other servicetimes.

In principle, our assumption that the service times are independent doesnot allow for service times that depend on a packet’s size. Dependence onpacket size would make the service times of stations dependent on each other.To model this explicitly would require a much more complicated model. How-ever we feel that our results in this work would still hold if this assumptionwere relaxed.

3.3. Policing Points. Arriving packets of each flow f first pass througha per-flow policing point at the ingress of the network. Whenever any queuein the set K(f) exceeds a high threshold nh, the policing point drops thepackets of flow f as they arrive. Conversely, when all of the queues in K(f)are below a lower threshold nh−o(n), flow f packets are permitted to enterthe network. Here o(n) can be any function satisfying o(n)/n = 0. Whenthe queues in K(f) are between the two thresholds, the policing follows ahysteresis law that we will make precise below.

To describe the policing hysteresis, we define a binary-policing state Hk(t)for each class k as

Hk(t) =

1 if Hk(t−) = 0 and Qk(t) ≥ nh

0 if Hk(t−) = 1 and Qk(t) ≤ nh− o(n)

Hk(t−) otherwise.

We define the process Λf (t), which we call the thinned arrival process, tocount the flow f packets that are allowed beyond the policing point at theingress. Therefore,

(2) Λx

f (t) =

Ex

f(t)∑

j=1

∏

K(f)

(1 −Hk(τj−))

where τj = Uf (0)+∑j−1

m=1 ξf (m) is the time of the jth arrival to the policingpoint.



3.4. Arrivals, Departures, and Routing. We begin by defining a processΦk(j) to track how often a class k packet is routed to class l for each l. ThusΦk(j) satisfies Φk(j) =

∑ji=1 φ

k(i). We define Ax

k (t) to be the cumulativearrivals, both exogenous and internal, up to time t for class k. Dx

l (t) denotesthe cumulative departures from class l from time 0 to t and therefore,

(3) Ax

k (t) =K∑

l=1

Φl(Dx

l (t)) + Λx

k (t).

For k ∈ {1, ..., g}, the thinned exogenous arrival process Λx

k (t) is definedby (2). Otherwise Λx

k (·) ≡ 0, reflecting that these classes do not receiveexogenous arrivals.

The population of Qx

k for k = 1, ...,K evolves according to

Qx

k (t) = Qx

k (0) +Ax

k (t) −Dx

k (t),(4)

Qx

k (t) ≥ 0.(5)

We define the K-dimensional column vector T x(t) such that the kth elementTx

k (t) is the cumulative time spent serving class k up to time t. Thus wehave

(6) Tx

k (t) is nondecreasing and T x

k (0) = 0.

Using the notation Ci to denote the ith row of the constituency matrix C,we define the cumulative idle time Ixi (t) of station i as

Ixi (t) = t− CiTx(t) is nondecreasing and Ixi (0) = 0.(7)

The service disciplines that we consider are work-conserving therefore,

(8)

∫ ∞

0CiQ

x(t)dIxi (t) = 0.

3.4.1. Queueing Discipline. For a station i, the departures of class k ∈C(i) are determined by the composition of the service time counting processSx(t) and the process T x(t) as described by the equation

(9) Dx

k (t) = Sx

k (Tx

k (t)).

The queueing discipline of a station i serves each flow in proportion to theflow weights over long time intervals. More precisely for some constant c,

Dx

k (t, t+ τ)

wf(k)

≥ Dx

l (t, t+ τ)

wf(l)

− c whenever Qk(s) > 0 ∀s ∈ [t, t+ τ ](10)



for all k, l ∈ C(i), where the notation Dx

k (t, t+ τ) , Dx

k (t+ τ)−Dx

k (t). Wealso assume that the instantaneous service rate of any queue is a functionof the current state.

Tx

k (t) = f(Xx(t)) For some function f(·).If the instantaneous rates depend on other information, like the positionin the polling cycle of a round robin discipline, that information may beappended to the H portion of the state vector X in some way. This alsoensures that X is a true state in that it contains sufficient information tocompute statistics of the future evolution of the network. We assume that|H(t)x | ≤ c for some constant c for all n, y, t. (Even this assumption can berelaxed to allow this encoding to scale with n, at the price of more complexityin the proof of Theorem 1.)

3.4.2. Markovian State. The Markovian state of the system is

Xx(t) ,[Qx(t);Ux(t);V x(t);Hx(t)

].

We claim that the process Xx is Markov by the following argument whichfollows that given by Dai [3] whose argument in turn followed from Kaspi andMandelbaum [19]. Consider the evolution of Xx(t) starting from a particu-lar time t∗. Each component of the residual time vector Ux(t∗ + s) declinesdeterministically at rate 1, while the components of V x(t∗ + s) decline ac-cording to Tx(·), which is a deterministic function of Xx . This continuesuntil one of these components hits 0, say at time t∗ + s′. Because this evo-lution is deterministic, s′ was predictable from the value of Xx(t∗). At timet∗ + s′, some components of Q, and perhaps H are incremented or decre-mented in a fashion that was also predictable with knowledge ofXx(t∗). Alsoat time t∗ + s′, the component of Ux or V x that hit zero takes a jump by afresh inter-arrival or service time that is independent of the past. Hence, theconditional distribution of the future state after the jump, given the stateXx(t∗) is the same as given knowledge of the entire past. As the processcontinues to evolve in this manner, the probability distribution of the stateat any future time t conditioned on the entire past before t∗ is the same as itconditioned on Xx(t∗). This shows that the process is Markov. Furthermore,because the process behaves deterministically between jump times, Xx(t∗)is a type of process termed a piecewise deterministic Markov (PDM) processby Davis [7]. Davis shows that a PDM process whose expected number ofjumps on [0, t] is finite for each t is strong Markov [7]. As we assume that theinter-arrival and service times have a positive and finite mean, the expectednumber of jumps of Xx(t∗) in any closed time interval is finite. ThereforeXx(t∗) has the strong Markov property.



4. Proof Strategy. To obtain our desired result we need to use twodifferent types of fluid limits. The first type is used to study the stochasticsystem when it starts in an intitial condition near the fluid equilibria set.Because it studies the system in the neighborhood of the equilibria set,we refer to it as a “Near” Fluid Limit (NFL). It is used to show that thestochastic system has flow rates that are close to those of the fluid modelfor a finite time after starting.

When looking at a NFL, we consider a sequence of systems indexed by nin which:

• System-n has policing thresholds nh.• Time and space are scaled by the factor n.• The scaled initial condition x of each system-n is within a neighbor-

hood of the fluid equilibrium set (‖x/n‖hE ≤ ζ).• n→ ∞

We show that such a limit converges along some subsequence, uniformly oncompact sets, to a trajectory of a fluid model satisfying a set of equations.We then show that if all trajectories of the fluid model converge, then thefluid limit converges (strongly) along the original sequence. Using this weconclude that there exist large enough thresholds such that whenever theinitial condition is within a neighborhood of the candidate equilibrium, theexpected rates of the stochastic network are close to the rates predicted bythe fluid model over a compact time interval of some length t0 seconds ofn-scaled-time. This alone is not enough to obtain our desired result, becausewe want to show that the network achieves the desired rate over the longterm, not just a compact time interval. This motivates the second type offluid limit.

The second type of fluid limit is used to analyze the behavior of the systemwhen it is distant from a neighborhood of the fluid equilibria set. Hence wecall it a “Far” Fluid Limit (FFL). In looking at a FFL, we consider the limitalong a sequence of system scale and relative-initial condition pairs {(x, n)}such that:

• System-n has thresholds nh.• Time and space are scaled by the factor n ‖x/n‖hE .• The scaled initial condition x of each system-n is outside of a neigh-

borhood of the fluid equilibria set (‖x/n‖hE > ζ).• The sequence of scaled initial conditions grows farther from the fluid

equilibria set (‖x/n‖hE → ∞).

As with the other fluid limit, we will show that such a limit converges alongsome subsequence to a fluid model trajectory and that we can upgrade that



convergence to convergence of the original sequence if the fluid model con-verges. In a similar fashion as [3], we exploit this convergence to constructa Lyapunov function which shows that the expected state of system-n con-tracts whenever n ‖x/n‖hE > L for some constant L. Thus by choosing nlarger than L/ζ, we can make the Lyapunov function “active” for all x/noutside of the ζ-neighborhood (which we call B) of the equilibrium set. Thisallows us to show that the invariant distribution of the system is concen-trated in B. In particular, we show that the expected first return to B thathappens at least t0 seconds (in n-scaled-time) after starting in B, can bemade arbitrarily close to t0 for large n.

To complete the proof of the main result, we combine the results: thatrates are close to desired for t0 seconds after starting in B and that theexpected first return to B after t0 seconds is close to t0 to conclude that thelong term rates are close to the fluid model rates. A stopping time argumentusing the strong Markov property formalizes this reasoning.

Much of the analyses of the Near and Far Fluid Limits are identical. Toavoid repetition, we define a scaling function cxb that allows us to considerboth fluid limits at once. The function cxb is defined by

cxbζ,

{n ‖x/n‖hE ≤ ζ

n ‖x/n‖hE ‖x/n‖hE > ζ

where the ζ parameter is an arbitrary positive constant. To make the nota-tion less cumbersome, we will omit the ζ subscript when either the choiceof ζ does not matter, or when its value is clear from the context.

4.1. Result Summary. The upcoming Section 5 consists of a series oflemmas and theorems that culminate in Theorem 4, which shows that thelong-term rates of the stochastic network are close to the fluid model ratesfor large n. We briefly survey the steps that lead to that result:

Theorem 1: For both Near and Far Fluid Limit sequences {xm}, thereexists a subsequence {xj} for which

Xxj (cxjbt)cxjbt

→ X(t)

for some trajectory X(t) that is dependent both on sample path ω and thechoice of subsequence, but must satisfy a set of fluid model equations (whichare (15)-(29)).

Lemma 5: Suppose we have a functional F that operates on system tra-jectories. (In the proof of the subsequent theorem we will pick F to extract



the difference between the actual service rates vs. a vector of desired ratesover a compact time interval, and later we will pick F to extract the distancebetween the current state and the candidate equilibrium.) Also suppose thatany solution X(t) to the fluid model equations (15)- (29) (we call such a X(t)a fluid trajectory) is such that F ◦ [X(·)](t) goes to zero in a time propor-tional to the distance of the initial condition’s distance to the fluid modelequilibrium he. Note that h is the threshold of the fluid model, which wewill see later may not be h. Under these suppositions,

∣∣∣∣F ◦[X(cxmbt)cxmb

](t)

∣∣∣∣→ 0

a.s. for each t greater than some critical time ζt0. (The functional F mustbe continuous on the topology of uniform convergence on compact sets forthis to hold.)

Theorem 2: Suppose that any fluid trajectory X(t) that solves equations(15)- (29) has service rates equal to some vector R (which we later showto be the max-min fair rates), after some time that is proportional to thestarting distance from the fluid model’s equilibrium set hE . Then for anypositive numbers ζ < 1 and any γ < 1, there exists a critical scale L1(ζ, γ)such that for any system with n > L1, and initial condition ‖x/n‖hE ≤ ζ

E[M−1Tx(nt0)] ≥ R(1 − ζ)(1 − γ)(nt0)

where t0 is the time the fluid model takes to reach the ideal rates afterstarting one unit distance away from the equilibrium.

Theorem 3: Suppose that any fluid trajectory is such that the state goesto the equilibrium in a time proportional to the starting distance from thecandidate equilibrium, then for any ζ < 1, and δ < 1 their exists a criti-cal scale L2(ζ, γ) such that for any system n > L2/ζ and initial condition‖x/n‖hE > ζ,

E ‖Y n,y(t0 ‖y‖hE)‖hE ≤ δ ‖y‖hE

where Y n,y is a “relative” state given by

Y n,y(t) ,1

nXx(nt)

and y , x/n. Roughly, the states tends to get closer to the equilibrium setwhen the system starts outside the ζ neighborhood of the equilibrium set.Also shown in Theorem 3, is that for any b < 1, there exists n large enoughso that

E ‖Y n,y(t0)‖hE ≤ b for all y : |y| ≤ ζ.



Lemma 6: If the conclusions of Theorem 3 hold, then the expected firstreturn of Y to a ζ neighborhood of the origin that happens at least t0 secondsafter having started in that neighborhood, can be made close to t0, if theconstants ζ, b, and δ of Theorem 3 are chosen to be small.

Theorem 4: If the necessary conditions for Theorems 2 and 3 hold, thenthe long term average rates can be made arbitrarily close to the fluid modelrates, by making n large enough.

5. Fluid Limit Analysis.

5.1. Preliminary Lemmas. In this section, we will state and prove The-orem 1 that shows that given any sequence of initial conditions and systemscales {xm} with cxmb→ ∞ there exists a subsequence {xj} ⊂ {xm} forwhich the sequence of system trajectories with initial condition yj , thresh-olds njh, and time and space scaled by cxjb converges to a fluid modeltrajectory satisfying a set of differential equations.

The proof of Theorem 1 depends on the lemmas presented in this sec-tion. The reader may either read this section first, or skip to the proof ofTheorem 1 in Section 5.1 and turn back to this section as needed.

Lemmas 1, 2, and 3 all assume that we start with some sequence {xj} thatsatisfies the following property concerning the residual arrival and servicetime processes in the fluid limit:Property 1: {xj} is a sequence of initial condition yj and scale nj pairs

with cxjb→ ∞ and Uxj (0) → U(0), V xj (0) → V (0) for some U(0),and V (0).

This property ensures that there are well defined residual arrival andservice times in the fluid limit. (When we eventually prove Theorem 1, whena sequence does not satisfy Property 1, we will find a subsequence for whichit does.)

Lemma 1 is a form of the Functional Strong Law of Large numbers forrenewal processes, and is taken from [3].

Lemma 2 is a new result showing that the thinned arrivals (the packetsthat make it beyond the policing point) converge to a fluid limit along asubsequence.

Lemma 3 is a result taken from [3] showing that the residual initial arrivaland service times decline to zero at rate 1 in the fluid limit. The lemmaalso shows that the sequence of functions we use to take the fluid limit areuniformly integrable, which will later be used to show a sequence of expectedvalues evaluated at a time t converges to 0 when the sequence of functionsevaluated at time t converge to 0 almost surely.



We say that fj(t) → f(t) uniformly on compact sets (u.o.c.) if for eacht ≥ 0

limj→∞

sup0≤s≤t

|fj(s) − f(s)| = 0.

We also use the notation f(t) = ddtf(t) where such a derivative exists. If

a function f(·) is differentiable at t, we say that t is a regular point.

Lemma 1 (Dai, Lemma 4.2 of [3]). Suppose that {xj} is a sequencesatisfying Property 1. Then for almost all ω,

Φk(ccxjbtb)cxjb

→ P ′kt u.o.c.,

Exj

k (cxjbt)cxjb

→ αk(t− Uk(0))+ u.o.c.,

Sxj

k (cxjbt)cxjb

→ µk(t− Vk(0))+ u.o.c.

Proof. See Lemma 4.2 of Dai [3]. The result is an instance of the StrongLaw of Large Numbers for Renewal Processes [8].

Lemma 2 (Thinned Arrival Convergence). Suppose that {xv} is a se-quence satisfying Property 1. Then for almost all ω, there exists a subse-quence {xj} ⊂ {xv} such that

Λxj (cxjbt)/cxjb → Λ(t) u.o.c.

where Λ(t) is some Lipschitz continuous process with, for all regular t ≥ 0,

(11) ˙Λf (t) ≤ αf for each flow f .

Proof. By Lemma 1,

(12) Exv

k (cxvbt)/cxvb → (αkt− ¯V (0)k)+ u.o.c.

for each class k. For notational convenience in the development that follows,we define:

Ek(t) , (αkt− V (0)k)+, ∆v(t) , Exv(cxvbt)/cxvb − E(t).(13)

Pick a compact time interval [s0, s1]. Because the number of packets allowedpast the policing point cannot exceed the number of packets arriving to thepolicing point in any time interval (2), we have

(14)1

cxvb[Λxv(cxvb(t+ ε)) − Λxv(cxvbt)] ≤

1

cxvb[Exv(cxvb(t+ ε)) − Exv(cxvbt)]



for any positive ε ≤ s1 − s0 and t : s0 ≤ t ≤ s1 − ε. Adding −∆v(t+ ε) and∆v(t) to both sides and substituting (13) and (12), we have

Λxv(cxvb(t+ ε))

cxvb− ∆v(t+ ε) −

[Λxv(cxvbt)

cxvb− ∆v(t)

]

≤ E(t+ ε) − E(t) ≤ εα.

Define the family of functions:

Lv(s0, t) := sups∈[s0,t]

[Λxv(cxvbs)

cxvb− ∆v(s)

]

Because the arguments of the sup functions are vectors, sup is taken component-wise. Note that for any (t, ε) with t ∈ [s0, s1 − ε],

Lv(s0, t+ ε) = Lv(s0, t) ∨ Lv(t, t+ ε)

and Lv(t, t+ ε) ≤ εα+ Lv(t, t) ≤ εα+ Lv(s0, t).

Thus Lv(s0, t + ε) − Lv(s0, t) ≤ εα and clearly Lv(s0, t + ε) − Lv(s0, t) ≥ 0because Lv(s0, ·) is monotone. Hence the functions Lv(s0, ·) are equicontinu-ous and individually Lipschitz continuous. Thus, by Arzela’s theorem, thereexists a further subsequence {xj} ⊆ {xv} such that

Lj(s0, t) → Λ(t)

uniformly on the compact set t ∈ [s0, s1] for some monotone-nondecreasing,Lipschitz-continuous process Λ(t). But by (12), ∆j(t) → 0 uniformly oncompact sets. Because of this and the fact that cxjb−1Λxj (cxjbs) is mono-tone, it follows that the maximizing values of each sup term approachcxvb−1Λxv(cxvbt). Thus

sups∈[s0,t]

[Λxv(cxvbs)

cxvb− ∆v(s)

]→ Λxv(cxvbt)

cxvb→ Λ(t).

Because the choice of [s0, s1] was arbitrary, we have cxjb−1Λxj (cxjbt) →Λ(t) u.o.c. Furthermore, (12) and (14) imply that Λ(t) satisfies (11).

Lemma 3 (Lemmas 4.3 & 4.5 of Dai [3]). Suppose that {xj} is a sequencesatisfying Property 1. Then almost surely:

limj→∞

Uxj

f (cxjbt)cxjb

= (Uf (0) − t)+ u.o.c., limj→∞

Vxj

k (cxjbt)cxjb

= (Vk(0) − t)+ u.o.c.



Also, for each fixed t ≥ 0, the sets of functions:

{Uxj (cxjbt)/cxjb:cxjb≥ 1} , {V xj (cxjbt)/cxjb:cxjb≥ 1} ,{Qxj (cxjbt)/cxjb:cxjb≥ 1}

are uniformly integrable.

Proof. See Lemmas 4.3 and 4.5 of Dai [3]

The following lemma will be used later to show that because all of thesystems we consider are work conserving, which we stated with the integralexpression (8), the fluid limit must also be work conserving. In the lemmabelow, the notation DR[0,∞) denotes the space of right continuous functionson R+ having left limits on (0,∞), and endowed with the Skorohod topology[9]. CR[0,∞) ⊂ DR[0,∞) is the subset of continuous paths.

Lemma 4 (Lemma 2.4 of Dai and Williams [6]). Let {(zj , χj)} be a se-quence in DR[0,∞)×CR[0,∞). Assume that χj is nondecreasing and (zj , χj)converges to (z, χ) ∈ CR[0,∞)×CR[0,∞) u.o.c. Then for any bounded con-tinuous function f ,

∫ t

0f(zj(s))dχj(s) →

∫ t

0f(z(s))dχ(s) u.o.c.

Proof. See Lemma 2.4 of Dai and Williams [6].

5.2. Convergence to a Fluid Limit along a Subsequence. The followingproof parallels the proof of Theorem 4.1 of Dai [3]. The proof here differsin that we deal with a sequence of systems and two different types of fluidlimits, as discussed in Section 4.

Theorem 1. Suppose one of the following cases hold for some constantζ:Near Fluid Limit Case: {xm} = {(xm;nm)} is a sequence of initial con-dition, system scale pairs satisfying:

∥∥∥xm

nm

∥∥∥hE

=‖xm‖nmhE

nm≤ ζ and nm → ∞.

Far Fluid Limit Case: {xm} = {(xm;nm)} is a sequence of initial condi-tion and system scale pairs satisfying:

∥∥∥xm

nm

∥∥∥hE

=‖xm‖nmhE

nm> ζ and

∥∥∥xm

nm

∥∥∥hE

→ ∞.



Then for almost all ω there exists a subsequence {xj} ⊆ {xm} for which

Xxj (cxjb)t)cxjb

→ X(t) u.o.c.

for some fluid trajectory X(t) with components

X(t) , [X(t); T (t); Λ(t); h]

where, in turn, the process X(t)has components

X(t) , [Q(t); U(t); V (t); B(t)]

where B(t) ≡ 0. The process X(t) may depend upon ω and the choice ofsubsequence {xj} but must satisfy the following properties for all t ≥ 0:

Uf (t) = (t− Uf (0))+, Vk(t) = (t− Vk(0))+,(15)

Tk(t) is nondecreasing and starts from zero,(16)

Ii(t) := t− CiT (t) is nondecreasing ,(17)

Dk(t) := µs(k)(Tk(t) − Vk(0))+,(18)

A(t) := R>D(t) + Λ(t),(19)

Q(t) := Q(0) + A(t) − D(t),(20)

Q(t) ≥ 0,(21)∫ ∞

0(CQ(t))dI(t) = 0,(22)

where (15), (16), and (18) hold for each flow f and class k, while (17) holdsfor each station i. Assignments (17), (18), (19), and (20) define I(t), D(t),A(t), and Q(t) respectively. Also, the following holds for each flow f forregular t ≥ 0:

˙Λf (t) = 0 whenever Qk(t) > h for some k ∈ C(f),(23)

˙Λf (t) = αf1(t ≥ Uf (0)) whenever Qk(t) < h for all k ∈ C(f),(24)

˙Λf (t) ≤ αf .(25)

Also, for station i and for any k, l such that {k, l} ∈ C(i) the followingproperties are satisfied for all regular t ≥ 0:

w−1k

˙Dk(t) ≥ w−1l

˙Dl(t) whenever Qk(t) > 0 ,(26)

w−1k

˙Dk(t) = w−1l

˙Dl(t) whenever Qk(t) > 0 and Ql(t) > 0.(27)



In addition the following case-specific conditions on h, and X(0) hold:

Near FL Case: h = h,∥∥X(0)

∥∥hE ≤ ζ;(28)

Far FL Case: 0 ≤ h ≤ h,∥∥X(0)

∥∥hE = 1.(29)

Proof. We first consider the parts of the proof that require case-specificanalysis.Near FL Case: We have nmh/cxmb= nmh/nm = h, and thus njh/cxjb→ halong any subsequence {xj} ⊂ {xm}. This implies the the first part of (28).Also, ‖xj‖njhE ≤ ζ, and thus by the Bolzano-Weierstrass Theorem, there

exists a subsequence {xr} ⊆ {xj} for which Xxr(0)/ ‖xj‖njhE → X(0) for

some X(0) satisfying the second part of (28).Far FL Case: We have

nmh

cxmb =nmh

‖xm‖nmhE

< h/ζ;

thus, there exists a further subsequence {xi} ⊆ {xm} for which

nih

cxib→ h(30)

for some h satisfying the first part of (29). Also,

∣∣∣∣Xxi(0)

cxib− hE

∣∣∣∣ =∣∣∣∣∣xi − nihei + nihei

‖xi‖nihE

− hE∣∣∣∣∣ =

∣∣∣∣∣vi +nihei

‖xi‖nihE

− hE∣∣∣∣∣

where ei ∈ arg mine∈nihE ||xi − e|| and vi is a unit vector for each i. By (30),the righthand side approaches 1 as i increases. Thus there is a subsequence{xr} ⊆ {xi} for which cxrb−1Xxr(0) → X(0) for some X(0) satisfying thesecond part of (29).Both Near and Far FL Cases: From this point on, the arguments applyfor both the Near and Far Fluid Limit cases. The hysteresis variables satisfycxrb−1Hxr(cxrbt) → 0 u.o.c. because Hxr(cxrbt) is bounded by a constantby its definition. This fact along with the convergence of cxrb−1Xxr(0) →X(0) allows us to use Lemma 3 to conclude

cxrb−1

[Uxr(cxrbt)V xr(cxrbt)

]→[U(t)V (t)

]u.o.c.

where U(t) and V (t) satisfy (15).



The cumulative service time process T xi satisfies

cxrb−1[Txi(cxrbt) − Txr(cxrbs)] ≤ (t− s).(31)

Thus by Arzela’s theorem [25], there exists a further subsequence {xv} ⊆{xr} for which T xv(cxvbt)/cxvb → T (t). Property (16) follows from (6).Property (7) implies Ixv(cxvbt)/cxvb → I(t) u.o.c. where I(t) satisfies (17).

By Lemma 1, Sxv

k (cxvbt)/cxvb → (µkt − V (0)k)+ u.o.c. for each class k.

This fact combined with (9) and (31) gives (18).We have already shown that Xxv(0) → X(0), therefore the Uxv(0) and

V xv(0) components of Xxv(0) converge to a limiting value. This fact allowsus to invoke Lemma 2 to conclude that there is a subsequence {xj} ⊂ {xv}for which Λxj (cxjbt)/cxjb → Λ(t) u.o.c. for some Lipschitz continuousprocess Λ(t) satisfying (25).

Lemma 1 combined with (3) gives us Axj

k (cxjbt)/cxjb → Ak(t) u.o.c.where Ak(t) is defined by (19). Furthermore, Ak(t) is Lipschitz continuousbecause it is equal to a linear combination of functions we have alreadyshown to be Lipschitz continuous. Thus using (4) we have that

Qxj (cxjbt)/cxjb → Q(t) u.o.c.(32)

where Q(t) is a Lipschitz continuous function given by (20). Property (21)follows easily from (5).

The next few arguments are similar to the proof of Proposition 4.2 in [5].Suppose that Qk(t) > h for some k ∈ C(f). By Lipschitz continuity of

Qk(t), there exists some small τ > 0 such that mint≤s≤t+τ Qk(s) > h. By theuniformity of the queue convergence in (32) and the threshold convergencein (30), there exists j∗ such that for all j > j∗, Q

xj

k (cxjbs) > njh for alls ∈ [t, t+ τ ]. Thus, by (2) one finds that Λ

xj

f (cxjbs)−Λxj

f (cxjbt) = 0 ∀s ∈[t, t + τ ]. Therefore, it follows that Λf (s) − Λf (t) = 0 ∀s ∈ [t, t + τ ] and

consequently, ˙Λf (t) = 0, which is (23).Suppose that Qk(t) < h for all k ∈ C(f). First note that in this case h > 0

and therefore (30) implies that nj → ∞. By the Lipschitz continuity of Qk(t)for each k, there exists some small τ > 0 such that maxk∈C(f) maxs∈[t,t+τ ] Qk(s) <h. Because nj → ∞, the uniformity of the convergence in (32), and the con-vergence in (30), there exists j ′ such that for all j > j ′, Q

xj

k (cxjbs) < njh.Furthermore there exists a j∗ ≥ j′ such that for all j > j∗ and k ∈ C(f),Q

xj

k (cxjbs) < njh− o(nj)hς. Thus, by (2),

Λxj

f (cxjbs) − Λxj

f (cxjbt) = Exj

f (cxjbs) − Exj

f (cxjbt) ∀s ∈ [t, t+ τ ]

and consequently we have (24).



Suppose that for some class k, Qk(t) > 0. By the Lipschitz continuityof Qk(t) there exists some small τ > 0 such that mint≤s≤t+τ Qk(s) > 0.Because of the uniformity of convergence in (32) there exists j∗ such thatfor all j > j∗, Q

xj

k (cxjbs) > 0 ∀s ∈ [t, t+ τ ]. By (10), for almost all ω, andall classes l we have

w−1k [Dk(cxjbs) −Dk(cxjbt)] ≥ w−1

l [Dl(cxjbs) −Dl(cxjbt)] ∀s ∈ [t, t+ τ ]

and thus we have (26).If Ql(t) > 0 and Qk(t) > 0, then (26) is true as written or with the k and

l and indices swapped. This implies (27).We observe that (8) is equivalent to

∫∞0 f(χj)dzj = 0 where

χj :=CiQ

xj (cxjbt)cxjb

, zj :=Ixj

i (cxjbt)cxjb

, f(·) := (·) ∧ 1.

Noting that χj and zj meet the required conditions for Lemma 4 we have,∫∞0 [CiQ(t)] ∧ 1dIi(t) = 0 which is equivalent to (22).

5.3. Upgrading Convergence along Subsequences to Convergence on Se-quences. In the previous section, we showed that for both Near and FarFluid Limit sequences, we can extract a sample path dependent subsequencethat converges to a fluid trajectory. The objective of this section is to up-grade this result to convergence along the original sequence, and not justthe sample path dependent subsequence. In particular, we show in Lemma 5which follows that if a functional F of the fluid model trajectory goes tozero in a time proportional to the initial condition’s distance from the fluidmodel equilibrium, then we get our desired convergence property. In latersections, we will invoke Lemma 5 choosing F to extract the service ratesfrom the fluid model, and later choosing F to extract the distance from thefluid model equilibrium. Lemma 5 is a generalization of an argument usedby Dai in the proof of Theorem 4.2 of [3].

Lemma 5. Suppose that F is a functional that maps Rr×R

+ into Rs×R

+

where r is the dimension of Xx(t) and s is arbitrary. Also suppose that F iscontinuous on the topology of uniform convergence on compact sets.

Recall that h is a component of X(t), and suppose that the following state-ment is true:

• The fluid model equations (15) - (27) are such that for any trajectoryX(t) with h > 0 that satisfies them, X(t) must also satisfy

(33) F ◦ [X( · )](t) ≡ 0 ∀t ≥ t0∥∥X(0)

∥∥hE .



Then,(Near FL Case:) For any sequence {xm} = {(xm;nm)} that satisfies1

nm‖xm‖nmhE ≤ ζ and cxmb= nm → ∞ for some ζ > 0,

(34)

∣∣∣∣F ◦[

1

cxmbXxm(cxmb · )](t)

∣∣∣∣→ 0 a.s.

for each t ≥ ζt0, and any initial condition y.(Far FL Case: ) Suppose Condition (33) can be strengthened to hold forany trajectory X with h ≥ 0. Then for any sequence {xm} = {(xm;nm)}that satisfies 1

nm‖xm‖nmhE > ζ and cxmb= ‖xm‖nmhE → ∞ for some ζ > 0,

conclusion (34) holds for each t ≥ t0.

Proof. By Theorem 1, for almost all sample paths ω, and for any subse-quence {xj} ⊂ {xm} there is a sample-path-dependent further-subsequence{xl(ω)} ⊂ {xj} for which

(35)X

xji(ω)(cxji(ω)bt, ω)

cxji(ω)b→ X(t, ω) u.o.c.

where X(t, ω) satisfies (16) - (27). The notation l(ω) and X(t, ω) emphasizethat the further-subsequence and fluid trajectory depend on ω. Now fix an ωfor which subsequences have convergent further subsequences as described.For the next few steps we suppress the ω arguments to simplify notation,but the reader should remember we are working with a particular ω. Wealso note that the h component of X satisfies h > 0 in the Near FL Caseand h ≥ 0 in Far FL case by conclusions (28) and (29) of Theorem 1 re-spectively. Consequently, condition (33) holds for the Near FL Case, andthe strengthened version of condition (33) holds in the Far FL Case. Thusin both cases, we have

F ◦ [X( · )] (t) = 0 for each t ≥∥∥X(0)

∥∥hE · t0.(36)

Because F is assumed to be continuous on the topology of uniform conver-gence on compact sets, (35) implies

F ◦[

Xxl (cxlb · )cxlb

](t) → F ◦ [X(·)](t) u.o.c.

This fact combined with (36), and that ‖X(0)‖hE ≤ ζ in the Near FL caseand ‖X(0)‖hE = 1 in the JFL case by (28) and (29) respectivley, yields

(37)∣∣∣F ◦

[X

xl (cxlb · )cxlb

](t)∣∣∣→ 0



for each t ≥ ζt0 in the Near FL case and each t ≥ t0 in the Far FL case. Sofor this fixed ω, any subsequence {xj} ⊆ {xm} has a further subsequence{xl(ω)} ⊆ {xj} for which (37) holds. Therefore the original sequence {xm}converges for this fixed ω:

∣∣∣F ◦[

X(cxmb · ,ω)cxmb

](t)∣∣∣→ 0

for each t ≥ ζt0 in the Near FL case and each t ≥ t0 in the Far FL case.But the same argument can be used to conclude that this holds for almostall ω. Thus, we have (34).

5.4. Convergence to Fluid Model Rates on a Compact Time Interval.The objective of this section is to use Lemma 5 to conclude that the rates ofthe stochastic system are close to those of the fluid model. As a consequenceof the convergence to the fluid limit in Theorem 1 being uniform on compactsets, and not uniform on R

+ we will only be able to show that the rates areclose over a compact time interval.

Theorem 2. Suppose there exists t0 such that

(38) M−1 ˙T (t) ≡ R ∀t ≥ t0∥∥X(0)

∥∥he

for any fluid model trajectory X(t) with limiting threshold h > 0 satisfying(15) - (27). Then, for any positive γ < 1 and ζ < 1, there exists L1(ζ, γ)such that for all n ≥ L1,

(39) inf‖ x

n‖hE≤ζ

E[M−1Tx(nt0)

]≥ R(1 − ζ)(1 − γ)nt0.

Proof. Let {xl} = {(xl, bl)} be a sequence of system scale and relativeinitial condition pairs satisfying nl → ∞ and ‖xl/nl‖hE ≤ ζ. We invoke theNear FL case of Lemma 5 by picking F so that

F ◦ [X(·)](t) := T (ζ−1t) − T (t) −MR(ζ−1 − 1)t.

F is easily seen to be continuous on the topology of uniform convergence oncompact sets. Also note that

F ◦ [X(·)](t) = 0 ∀t ≥ t0∥∥X(0)

∥∥he

by (38). By Lemma 5,

liml→∞

∣∣∣∣Txl(nlt0) − Txl(ζnlt0)

nl(1 − ζ)t0−MR

∣∣∣∣ = 0 a.s.,(40)



where we have selected t = ζt0 and used the fact that cxlb= nl. The lefthand side of (40) is bounded from above by a constant for all l, and thus bythe dominated convergence theorem [8]:

liml→∞

E

∣∣∣∣Txl(nlt0) − Txl(ζnlt0)

nl(1 − ζ)t0−MR

∣∣∣∣ = 0.(41)

Also note (41) holds for any sequence {xl} with cxlb→ ∞ and ‖xl/nl‖hE ≤ ζ,because these were the only restrictions for our initial choice of sequence.

Now pick a positive constant γ < 1. Observe that there exists a constantL1(γ, ζ) such that whenever n > L1,

inf‖x/n‖hE

≤ζ

E [Tx(nt0) − Tx(nζt0)]

n(1 − ζ)t0≥MR(1 − γ)

for if otherwise we could construct a sequence {xl} that violates (41). Bythe monotonicity of T xl(·), we have (39).

5.5. Stochastic System Attracted to Fluid Equilibrium. The objective ofthis section is to show that the stochastic system is attracted to the fluidmodel equilibrium. In particular we show that the expected norm of thestate declines geometrically for starting states outside a neighborhood ofthe equilibrium. We also show that the expected norm of the state is small,some fixed time after starting inside the neighborhood. The proof techniqueis similar that of Theorem 3.1 of Dai [3].

Before stating the theorem of this section, we define the following nota-tion:

y := x/n,

Y n,y(t) :=1

nXx(nt).(42)

Theorem 3. Suppose that there exists t0 such that

(43)∥∥X(t)

∥∥hE ≡ 0 ∀t ≥ t0

∥∥X(0)∥∥

hE

for any fluid model trajectory X(t) with limiting threshold h ≥ 0 satisfyingequations (16) - (27). Then the following conclusions are true:

i) For any ζ > 0, and any positive δ < 1 there exists L2(ζ, δ) such thatfor all n ≥ ζ−1L2 and ‖y‖hE > ζ,

E ‖Y n,y(t0 ‖y‖hE)‖hE ≤ δ ‖y‖hE



ii) For any ζ > 0, and any b > 0 there exits L3(ζ, b) such that for alln ≥ L3 and all ‖y‖hE ≤ ζ,

E ‖Y n,y(t0)‖hE ≤ b

Proof. We first prove conclusion (i). Pick any sequence of pairs {xl} ={(xl, nl)} satisfying ‖xl/nl‖hE → ∞ and ‖xl/nl‖hE > ζ for some ζ > 0 (AFar FL sequence). We invoke Lemma 5, picking F such that

F (t) , F ◦[X(·); T (·); Λ(·); h

](t) :=

∥∥X(t)∥∥

hE .

Note that F (∥∥X(0)

∥∥hE t) = 0 ∀t ≥ t0 by Assumption (43), and F is easily

seen to be continuous on the topology of uniform convergence on compactsets. Lemma 5 yields

∥∥∥∥∥Xxl(‖xl‖nlhE

t)

‖xl‖nlhE

∥∥∥∥∥ nl

‖xl‖nlhE

hE

→ 0 a.s.

for each t ≥ t0. Taking t = t0 we have that

1

‖xl‖nlhE

∥∥∥Xxl(‖xl‖nlhEt0)∥∥∥

nlhE→ 0 a.s.

By Lemma 3, 1‖xl‖nlhE

Xxl(‖xl‖nlhEt0) is uniformly integrable. Therefore

liml→∞

1

‖xl‖nlhE

E∥∥∥Xxl(‖xl‖nlhE

t0)∥∥∥

nlhE= 0.

Applying definition (42), and noting that our initial choice of sequence {xl}was arbitrary, up to some constraints, we have that the following statementis true:

a) For any ζ > 0, and any sequence {xl} with ‖xl/nl‖hE → ∞, and ‖xl/nl‖hE >ζ,

liml→∞

1

‖yl‖hE

E ‖Y nl,yl(t0|yl|)‖hE = 0.

We claim that this fact implies the following statement is true:

b) For any ζ > 0, and any positive δ < 1 there exists L2(ζ, δ) such that forall ‖y‖hE ≥ L2 and ‖y‖hE > ζ,

1

‖y‖hE

E ‖Y n,y(t0 ‖y‖hE)‖hE ≤ δ.



Suppose statement (b) were not true. Then for some ζ > 0 and some pos-itive δ, we would have that for any L2 there would exist a pair (n, y) withn ‖y‖hE > L2 and ‖y‖hE > ζ with 1

‖y‖hE

E ‖Y n,y(t0 ‖y‖hE)‖hE > δ. We there-

fore could construct a sequence that violates statement (a), which is a con-tradiction. A special case of (b) is when n > L2ζ

−1 and ‖y‖hE > ζ. Hencewe have conclusion (i) of the lemma.

We now turn to showing conclusion (ii). Pick an arbitrary sequence ofpairs {xl} = {(xl, nl)} satisfying nl → ∞ and ‖xl/nl‖hE = ‖yl‖hE ≤ ζ forsome constant ζ (a Near FL sequence). We again invoke Lemma 5 by takingF to be the same functional as before, i.e.,

F (t) , F ◦[X(·); T (·); Λ(·); h

](t) :=

∥∥X(t)∥∥

hE .

Using Lemma 5, and the fact that cxlbζ= nl when ‖xl/nl‖hE ≤ ζ we have

∥∥∥∥Xxl(nlt)

nl

∥∥∥∥hE

→ 0 a.s.

for each t ≥ ζt0. Now take t = t0,

1

nl‖Xxl(nlt0)‖nlhE

→ 0 a.s.

By Lemma 3, (nl)−1Xxl(nlt0) is uniformly integrable. Therefore

liml→∞

1

nl‖Xxl(nlt0)‖nlhE

= 0.

Applying definition (42), and noting that our initial choice of sequence {xl}was arbitrary, up to some constraints, we have that the following statementis true:

c) For any ζ > 0, and any sequence {xl} with nl → ∞, and ‖yl‖hE ≤ ζ,

liml→∞

E ‖Y nl,yl(t0)‖hE = 0.

We claim that fact (c) implies conclusion (ii). Suppose (ii) were not true.Then for some choice ζ and b, we would have that for every constant L3,there would exist an n ≥ L3 and ‖y‖hE ≤ ζ satisfying E ‖Y n,y(t0)‖hE > b.This would allow us to construct a sequence that violates statement (c),which is a contradiction.



5.6. Hitting Times of a Neighborhood of the Fluid Equilibrium. The ob-jective of this section is to show that the results of Theorem 3 imply thatthe expected return time to the ζ ball around the fluid equilibrium is small.Later on, we will combine this with the results of Theorem 2 that show thatthe expected rates, when starting from within the ball are close to the fluidrates on a compact time interval, to show the long term rates are close tothe fluid rates.

The proof of Lemma 6 is adapted from the proof of Theorem 2.1(ii) of[24], which was for a discrete time Markov chain.

In the proof of Lemma 6, and in subsequent proofs, when we want toexpress Y n,y(t) without specifying an initial condition we will write Y n(t)where the choice of initial condition is implicit by the choice of probabilitymeasure. We define Py to be a probability measure for which Py{Y n(0) =y} = 1, and thus, Y n(t) = Y n,y(t) Py-a.s. and Ey[Y

n(t)] = E[Y n,y(t)].

Lemma 6. If, for n fixed, we have the following inequalities:

Ey ‖Y n(t0|y|)‖hE ≤ δ ‖y‖hE for all ‖y‖hE > ζ,(44)

Ey ‖Y n(t0)‖hE ≤ b for all ‖y‖hE ≤ ζ,(45)

then Y n is positive Harris recurrent and, furthermore,

supy∈B

Ey[τnB(t0)] ≤ t0

[1 +

ζ + b

1 − δ

](46)

where B , {y : ‖y‖hE ≤ ζ} and τnB(t0) is defined by

(47) τnB(t0) , inf{t ≥ t0 : Y n(t) ∈ B}.

Proof. That Y n is positive Harris recurrent follows directly from Theo-rem 3.1 of [3].

The rest of the argument that follows is adapted from the proof of Theo-rem 2.1(ii) of Meyn and Tweedie [24]. We will use the following Fact takenfrom Theorem 14.2.2 of [23]:

Fact 1: (Meyn and Tweedie [23]) Suppose a discrete time Markovchain Φ = {Φk, k ∈ Z

+} is defined on a general state space X withtransition kernel P(x,A) = P(Φx ∈ A), where A ∈ B(X), the Borelsubsets of X. If V and f are nonnegative measurable functions satis-fying

∫P(x, dy)V (y) ≤ V (x) − f(x) + b1B(x), x ∈ X(48)



then

Ex

τB−1∑

k=0

f(Φk)

≤ V (x) + b

where τB = inf{k ≥ 1 : Φk ∈ B}.

We now turn to setting up our problem to make use of Fact 1. We definethe following two mappings, the first mapping each y to a time n(y), andthe second mapping each y to a integer valued Lyapunov function V (y):

n(y) ,

{‖y‖hE t0 if ‖y‖hE > ζ

t0 if ‖y‖hE ≤ ζ(49)

V (y) ,t0

1 − δ‖y‖hE .(50)

Substituting our assignment of n(y) into relation (44), and adding a termto that relation’s right hand side so that the relation holds for y both insideand outside B, we have

Ey ‖Y n(t0 ‖y‖hE)‖hE

≤ δ ‖y‖hE +

(supy∈B

Ey ‖Y n(t0)‖hE

)1B(y)

≤ ‖y‖hE − 1 − δ

t0n(y) + (1 − δ + b)1B(y)

where the last step makes use of (45. We multiply both sides by t0/(1 − δ)to get

Ey[V (Y n(t0 ‖y‖hE))] ≤ V (y) − n(y) + b1B(y)(51)

where b = t0 +t0

1 − δb.(52)

The transition kernel Pt for the Markov process Y n is defined by

Pt(y,A) = Py(Yn(t) ∈ A)

where A is any set in B(Y), the Borel subsets of the state space Y. Wedefine the discrete time “embedded” Markov chain Φ = {Φk, k ∈ Z+} withtransition kernel P given by

P(y,A) = Pn(y)(y,A).



Note that∫

P(y, dz)V (z) =

∫Pn(y)(y, dz)V (z) = E|V (Y (n(y))) |.(53)

Combining (53) with (51) we have

∫P(y, dz)V (z) ≤ V (y) − n(y) + b1B(y).

Recognizing this is the form of expression (48), we may use Fact 1 to conclude

Ey

τB−1∑

k=0

n(Φk)

≤ V (y) + b(54)

where τB = inf{k ≥ 1 : Φk ∈ B} is the first return time of the embeddeddiscrete time chain Φ to the set B. Fix a particular ω and initial conditiony. If the embedded chain hits B in τB discrete steps, then the original chainmust also hit B in a time equal to the sum of the embedded times that thosediscrete steps correspond to. It is also possible that the original chain hitsB earlier, in addition to hitting at a time equal to the sum of the embeddedtimes. Thus the first hitting time of the original chain satisfies

inf{t ≥ 0 : Y n(t) ∈ B} ≤τB−1∑

k=0

n(Φk) Py-a.s.

for each y ∈ Y. Furthermore, whenever the initial condition y ∈ B, the firstembedded time is t0 seconds by (49). Consequently, the time of the firsthitting of B after t0 seconds expire satisfies

inf{t ≥ t0 : Y n(t) ∈ B} ≤τB−1∑

k=0

n(Φk) Py-a.s.

for each y ∈ B. Substituting (47), taking the expectation, and using (54),we have

Ey[τnB(t0)] ≤ V (y) + b for all y ∈ B.

Taking the supy∈B of both sides, substituting (52) and (50) we have

supy∈B

Ey[τnB(t0)] ≤ t0 +

t01 − δ

[ζ + b] ,

which is (46).



5.7. Convergence of Long Term Rates. The objective of this section isto tie together all of the preceding results to conclude that the long-termrates of the stochastic system are close to the fluid rates for large enough n.We pick n large enough so that the stochastic system’s rates are close to thefluid rates for the first t0 seconds after having started in a ζ neighborhoodand that the expected time of first return to a ζ neighborhood, t0 secondsafter having started there is close to t0. At this point, intuition suggests thatthe long term rates must be close to the fluid rates over the long term. Weformalize that intuition with an argument based on stopping times and thestrong Markov property.

Theorem 4. Suppose both of the following are true:

• For any fluid model trajectory X(t) with limiting threshold h ≥ 0 thatsatisfies (16) - (26),

(55)∥∥X(t)

∥∥hE ≡ 0 ∀t ≥ t0

∥∥X(0)∥∥

hE .

• For any fluid model trajectory X(t) with limiting threshold h > 0 sat-isfying (16) - (26):

(56) M−1 ˙T (t) ≡ R ∀t ≥ t0∥∥X(0)

∥∥hE

where R is a constant K dimensional vector of flow rates.

Then for any ε > 0, there exists a system-scale nc such that for all system-scales n ≥ nc

limt→∞

Dn,y(t)

t≥ (1 − ε)R a.s.

Proof. We observe that equations (56) and (55) are the necessary condi-tions to apply Theorems 2 and 3 respectively. Therefore, we may arbitrarilypick the constants ζ, δ, and b of Theorem 3 and the constants ζ and γ ofTheorem 2 (using the same ζ value in Theorems 2 as we use when we applyTheorem 3), and then fix an n satisfying

(57) n > max[L1(ζ, γ), ζ−1L2(ζ, δ), L3(ζ, b)]

so that the conclusions of both Theorems 3 and 2 hold.In addition, conclusions (i) and (ii) of Theorem 3 allow us to invoke

Lemma 6 to conclude

supy∈B

Ey[τnB(t0)] ≤ t0(1 +

ζ + b

1 − δ)(58)



where τnB(t0) is defined by (47). Because the constants ζ, b, δ could have

been selected to be arbitrarily small, equations (58) and (57) imply thatthe expected first hitting time of B, t0 seconds after having started in B,can be made to be arbitrarily close to t0 by choosing n large enough. Forconvenience we collect some of the constants in (58) in the term t′0 definedby

(59) t′0 = t0

[1 +

ζ + b

1 − δ

].

We have also chosen n large enough so that the following conclusion fromTheorem 2 holds,

(60) inf|y|≤ζ

E [Tn,y(nt0)] ≥MR(1 − ζ)(1 − γ)nt0.

Define the stopping times

σ0 = 0,

σi+1 = inf{t ≥ t0 + σi : Y (t) ∈ B}, ∀i ≥ 0.(61)

Figure 3 illustrates how these stopping times are defined. Note that for anyinitial condition y ∈ Y (the state space of Y n) and index i ≥ 1,

Ey[σi+1 − σi] = EY n,y(σi)[τnB(t0)] ≤ sup

y∈BEy[τ

nB(t0)] ≤ t′0.(62)

This follows from the fact that Y n,y(σi) ∈ B, the strong Markov property,the stopping time definitions (47) & (61), and expressions (58) & (59). Also,Y n is positive Harris recurrent by Lemma 6 and therefore, Ey[σ1] < ∞ forany y ∈ Y. We define a counting process N(t) for the stopping times σi as

N(t) = inf{i : σi ≤ t}.

Because Y n is positive Harris recurrent, σi <∞ almost surely, and therefore

(63) N(t) → ∞ a.s.

We now turn to bounding the expected “arrival” rate of the stopping timesσi. By (62) for each i,

Ey[σi]

i=

∑i−1j=1 Ey[σj+1 − σj ] + Eyσ1

i≤ t′0(1 + 1/i) +

Eyσ1

i(64)



1 i i+1

Min. Time Elapse = t0

Avg. Time Elapse · t0'

Avg. Througphut

Rt0(1- )(1- )

t0

B

Start in B at time i.

B

Wait t0

seconds. Record time of return

to B as i+1 .

B

Y Y Y

Radius

Fig 3. The top half of the figure illustrates the definition of the stopping times σi, σi+1, ...

The bottom half illustrates the intuition behind the proof of Theorem 4 by plotting the

stopping times on a time line, and showing the bound on expected throughput between such

stopping times. That the expected time elapse between σi and σi+1 is upper-bounded by t′0is a consequence of Lemma 6. The lower-bound on expected throughput between stopping

times comes from Theorem 2.

Additionally, along any sample path

t

N(t)≤ σN(t)+1

N(t) + 1

N(t) + 1

N(t).

Thus by taking lim inft→∞Ey(·) of both sides, and using (63), and (64) wehave

lim inft→∞

Ey

[t

N(t)

]≤ t′0.

Also, by Fatou’s Lemma

Ey

[lim inft→∞

t

N(t)

]≤ lim inf

t→∞Ey

[t

N(t)

]≤ t′0.(65)

Recall that the process T x(t) = T n,y(t) is defined in terms of the time scaleof Xn, and not that of Y n, which we defined in expression (42) by scaling



time by a factor of n. Therefore we define a re-scaled service process Tn,y

with time re-scaled to match the time scale of the Process Y (t) accordingto the definition

(66) Tn,y(a, b) = T n,y(nb) − T n,y(na).

We also define the random vectors ρi to track the throughput between stop-ping times σi as made precise by the definition

ρi = M−1Tn(σi, σi + σi+1),(67)

where the y superscript in Tn,y is omitted because the initial condition willbe specified implicitly by the choice of probability measure. Note that fori ≥ 1 and each y ∈ Y,

Ey[ρi] ≥ EY n,y(σi)[M−1Tn(t0)]

≥ infy∈B

Ey[M−1Tn(t0)]

≥ Rt0(1 − ζ)(1 − γ).(68)

This follows from the fact that Y n,y(σi) ∈ B, the strong Markov property,the definition of σi (61), the definition of ρi (67), and relation (60). Figure 3illustrates the fact that the throughput between stopping times σi and σi+1

is lower-bounded according to relation (68).Because, we have shown that Y n is positive Harris recurrent, by [5] the

following ergodic property holds for every measurable f on Y with π(|f |) <∞,

limt→∞

1

t

∫ t

0f(Y n(s))ds = π(f) Py-a.s. for each y ∈ Y

where π is the unique invariant distribution of Y n. Assigning the function

f(y) , M−1 ˙T

n,y(0) to be the instantaneous service rates when the process

is in state y, (Recall that we assumed the service rates are a function of thestate in Section 3.4.1.) we have,

(69) limt→∞

1

t

∫ t

0f(Y n,y(s))ds = lim

t→∞

1

tM−1Tn,y(t) = R a.s.

for some constant vector R.Consider the random variable N , lim inf t→∞

tN(t) . The random variable

N is a Pπ invariant random variable, and therefore is a constant. Henceby (65), N is a constant and N ≤ t′0. A more detailed explanation of thisargument is provided in [26].



We observe that for any sample path the following inequalities hold,

t

N(t)

M−1Tn,y(t)

t≤∑N(t)

j=0 ρj

N(t)≤ t

N(t)

σN(t)+1

t

M−1Tn,y(σN(t)+1)

σN(t)+1.(70)

Taking the lim inft→∞ of both sides, and using (69), and (63) we have that

(71) lim inft→∞

∑N(t)j=0 ρj

N(t)= NR a.s.

Now we need to apply the dominated convergence theorem. Before doing so,we note that

M−1 Tn,y(σN(t)+1)

σN(t)+1≤M−11

where 1 is a column vector of 1’s of appropriate dimension. This fact com-bined with (70) yields that for each i > 0,

infk≥i

∑kj=1 ρj

i≤ lim inf

t→∞

t

N(t)M−11 ≤ t0(1 + φ)M−11.

and thus the random variables{

infk≥i

∑kj=1 ρj

i: i > 0

}

are dominated by a constant. Consequently, the dominated convergence the-orem applied to (71) yields,

lim infi→∞

E

[∑ij=1 ρj

i

]= NR.

Also for each i > 0 by (68),

E

[∑ij=1 ρj

i

]≥ Rt0(1 − γ)(1 − ζ).

Thus,NR ≥ Rt0(1 − γ)(1 − ζ).

Substituting (65) we have that

R ≥ (1 − γ)(1 − ζ)t0t′0

R.



Which by (59), (66), and (69) implies

limt→∞

1

tM−1Tn,y(t) ≥ (1 − γ)(1 − ζ)

1 + ζ+b1−δ

R a.s.

Recall that the parameters γ, ζ, b, and δ may be chosen arbitrarily close to0, and still have all of the preceding development hold by choosing a largeenough n according to (57). Thus for a large enough n,

limt→∞

1

tM−1Tn,y(t) ≥ (1 − ε)R a.s.

for any relative initial condition y. By the strong law of large numbers forrenewal processes [8],

limt→∞

1

tSn,y

k (t) → mk a.s.

Thus by (9),

limt→∞

1

tDn,y(t) ≥ (1 − ε)R a.s.

6. Analysis of Switch Example. In this section we apply the resultsof the preceding section to the second example introduced in Section 1.3.Recall that this example resembles a 2-input 2-output switch and has 3flows. It is useful to refer back to Figure 2 to see the configuration of flowsand stations in the example. Recall that flows 1 and 2 share station 1 whichhas a service rate of 1.0, while flows 2 and 3 share station 4 which also hasa service rate of 1.0. To apply the results of the preceding section, we mustshow that the fluid model of the system, described by equations (15)-(29),admits trajectories that go to a set of equilibria from any initial condition.Furthermore, the trajectories should reach an equilibrium from the set in atime proportional to the initial condition’s distance from the set. Intuitionsuggests that one of the queues that flow 2 passes through should reach itsthreshold and chatter there. Similarly, flow 1’s rate should ultimately belimited by queue 1, and thus queue 1 should fill to its threshold and chatterthere. The other queue that flow 1 passes through should be nearly emptybecause that station has more capacity than the arrival rate of the one flowit handles. By the same reasoning, flow 3 should be limited by queue 7, andthus queue 7 should fill to its threshold and chatter there, while queue 3



should be nearly empty. This intuition suggests that the equilibrium set,which we will call E for the moment, should be given by

E ,

X :

Q1 = Q8 = h,Q3 = Q4 = Q5 = Q6 = 0,(Q2, Q7) ∈

{[0, h] × h

} ∪ {h× [0, h]}

.

In this definition for E , one of the queues that flow 2 passes through mustbe at its threshold, while the other queue must be at or below its threshold.As it will turn out, the most critical part of the analysis of this example’sfluid model is to show that the queues flow 2 passes through, queues 2 and7, go to equilibrium values in a time proportional to the initial condition.Intuition suggests, and we will verify this carefully later, that after a “settlingdown” period flow 1’s rate through queue 1, as well as flow 3’s rate throughqueue 8, settles to 0.5. Consequently the dynamics of (Q2(t), Q7(t)) followthe relations outlined by Table 1 and illustrated by Figure 4. The entries ofTable 1 are easily derived by using the observations that:

• The arrival rate to queue 2 is 0.6 when queue 2 and queue 7 are belowthreshold while the arrival rate to queue 2 is 0 when one of thesequeues is above threshold.

• The departure rate from either queue 2 or queue 7 is 0.5 whenever thequeue is nonempty or has sufficient arrivals to maintain this departurerate. (This relies on our assumption that the flow rates through queues1 and 8 have “settled down” to 0.5).

Figure 4 is “phase-portrait” diagram, showing the dependence of ( ˙Q2(t),˙Q7(t))

on (Q2(t), Q7(t)). It is evident from the diagram that the time to reach theset {

[0, h] × h} ∪ {h× [0, h]

},

which is the projection of E on to this space, is not always in proportion tothe initial condition’s distance from this set. Consider an initial conditionof ( h

2 , h + ε). This initial condition is only a distance of ε from E , but the

time it takes to reach the set E is h + 12ε. (Note that we will use the L1

norm throughout this section.) We can fix the problem by slightly enlargingthe set E to a new set E so that the set is reached in a time proportionalto the initial condition’s starting distance from the set. However, the worstcase flow rates for states in E will ultimately be the rates that we show theoriginal stochastic system reaches. If we make E too large so as to includestates whose flow rates are low, the lower bound on the asymptotic flow ratesof the stochastic system will not be as strong. With these considerations in



1 2

4

5

6

3

Q 2

Q 7

h

h a

Fig 4. The evolution of (Q2(t), Q7(t)). The shaded area indicates the set E.

mind, we define E according to

E ,

X :

Q1 = Q8 = h,Q3 = Q4 = Q5 = Q6 = 0,(Q2, Q7) ∈{

(χ, ψ) :χ ∈ [0, h]ψ ∈ [h− a

hχ, h+ a

h(h− χ)]

}

∪ {h× [0, h]

}

.

Here a is an arbitrary positive constant that should be less than h. With thisdefinition for E , one can show that the set E is reached in a time proportionalto the initial distance from E . The time to reach E , along with the maximumratio of the time to reach E divided by initial distance to E are shown inTable 1. Note that we are slightly abusing the term “equilibrium” slightlyby calling E a set of equilibria because the fluid model’s state can continueto change when starting from a state in E .

We are now ready to formalize the intuition we have outlined in thepreceding paragraphs. We begin by stating a lemma that the system settlesdown so that the behavior flow 2’s queues are as described by Table 1 aftera time τsd (mnemonic for “settle down”) that is in proportion to the initialcondition.

Lemma 7. There exists a time τsd proportional to the initial condition



Q2 Q7˙Q2

˙Q7 Time to E Time to E

‖X‖hE

1 [0, h) [0, h) 0.1 0

if |Q7 − h| < a then

10 ha|Q7 − h|

if |Q7 − h| ≥ a then10|Q2 − h|

10 ha

2 (h,∞) [0, h) -0.5 0 2|Q2 − h| 2

3 (0,∞) (h,∞) -0.5 0

if |Q7 − h| < a then2h

a|Q7 − h|

if |Q7 − h| ≥ a then2|Q2 − h| + 2|Q7 − h − a|

2 ha

4 0 (h,∞) 0 -0.5 2|Q7 − h − a| 2

5 h [0, h] [−0.5,0.1] [−0.5,0] 0 N/A

6 [0, h] h [−0.5,0.1] [−0.5,0] 0 N/ATable 1

Dynamics of (Q2(t), Q7(t)).

as described by the relation

τsd = t01∥∥X(0)

∥∥hE

for some positive t01 such that for all regular points t ≥ τsd:

• The value of ( ˙Q2(t),˙Q7(t)) is determined by the value of (Q2(t), Q7(t))

as specified by Table 1.• Q3(t) = Q5(t) = Q6(t) = 0, and Q8(t) = h.• The remaining time to reach the set E, as well as the maximum ratio

between this time and the distance of (Q2(τsd), Q7(τsd)) from E|Q2,Q7

in any of the Regions 1 through 4 is as specified in Table 1.

Lemma 7’s result is intuitive but tedious to prove. We therefore give theproof in an appendix. We now state and prove the principal result of thissection.

Theorem 5. There exists a system-scale nc, so that when the stochasticsystem describing Example 2 has policing thresholds nch, the asymptotic flowrates approach 0.5. More precisely, for any ε > 0,

limt→∞

D(t)

t≥ (1 − ε)

0.50.50.5

a.s.



Proof. By Lemma 7 the dynamics of the state variables (Q2(t), Q7(t))of the fluid model trajectory evolve according to Table 1 after a time τsd =t01∥∥X(0)

∥∥hE . From Table 1, the Q2 and Q7 components of the fluid model

trajectory reach values in the equilibrium set’s projection in at most an ad-ditional 10 h

a

∥∥X(τsd)∥∥

hE time units. (We say in the “equilibrium set’s projec-tion” to emphasize that some components of the state vector have reachedvalues consistent with E , but that not all components have yet done so.)Because the total arrival rate into the system is less than or equal to 1.8,

∥∥X(τsd)∥∥

hE ≤ (2t01 + 1)∥∥X(0)

∥∥hE .

Thus after a time length of t02∥∥X(0)

∥∥hE where

t02 = 10h

a(2t01 + 1) + 1

all queues but queue 1 have been shown to reach values in the projectionof the equilibrium set. By Lemma 7 queue 5 is empty, so either: queue 1 isabove threshold, in which case policing is on and it will reach threshold in2(Q1(t02∥∥X(0)

∥∥hE

)− h)

time units, or queue 1 is below threshold in whichcase it will reach threshold in 10

(Q1(t02∥∥X(0)

∥∥hE

)− h)

time units. OnceQ1(t) reaches threshold h, it remains there by the following reasoning. Ifqueue 1 were to move some positive amount ε above h, the policing wouldhave turned on before the queue grew to ε and prevented it from gettingthere. Similarly, if queue 1 were to move some positive amount ε below h, thepolicing would have turned off before the queue receded by ε, and preventedthe queue from receding that much. Very loosely, we can bound the rate ofgrowth of queue 1 before time t02

∥∥X(0)∥∥

hE by

Q1(t02∥∥X(0)

∥∥hE

) ≤ 1.6∥∥X(0)

∥∥h .

Thus after a time of length t0∥∥X(0)

∥∥hE where t0 is given by

t0 = 10h

a(2t01 + 1) + 17

all fluid model trajectories will have reached the set E . The departure ratesfor all three flows are easily seen to be 0.5 when the fluid model’s state is inE and threshold h > 0. Thus, by Theorem 4 we have that the asymptoticflow rates approach 0.5.



7. Conclusion. In this work we have shown how the analysis of theflow rates of a stochastic network with a particular flow control schememay be reduced to an analysis of a fluid model. While we have focused ona particular flow control scheme, we feel that the same analysis could becarried out for many other control schemes. The key feature that enabledour approach was that our control scheme has a free parameter, n, whichwhen increased makes the system look more and more like a deterministicfluid system. We have demonstrated how to use the theory developed in thispaper to analyze an example network resembling a 2-input, 2-output switch.

8. Appendix. In order to prove Lemma 7, we will need to state andprove a number of facts.

Fact 1. For regular points t ≥ τv, where τv , maxk∈1..8 Vk(0) , ˙Dk(t) ≥12 whenever Qk(t) > 0.

Proof. ˙Dk(t) = ˙Tk(t) by (18). For k ∈ {1, 3, 5, 7} (the first class in

each station), equations (16), (17), and (20) - (22) imply that that ˙Tk(t) +˙Tk+1(t) = 1 whenever Qk(t) > 0 or Qk+1(t) > 0. Equation (26), combined

with the last two observations, implies that ˙Dk(t) ≥ 12 whenever Qk(t) > 0.

Fact 2. Consider regular points t ≥ τv. Whenever ˙Ak(t) ≥ 12 , then

˙Dk(t) ≥ 12 .

Proof. Suppose not. Then ˙Dk(t) <12 and there would exist ε such that

for any δ ≤ ε where t+δ is regular, ˙Dk(t+δ)− ˙Dk(t) <12δ and

∫ t+εt

˙D2(t)dt <12 . By (20) Qk(t+ δ) > 0 for any δ ≤ ε, and hence by Fact 1 ˙Dk(t+ δ) ≥ 1

2 .

Thus∫ t+εt

˙Dk(t)dt ≥ 12 . This is a contradiction.

Fact 3. ˙D7(t) = ˙D8(t) = 12 for all regular t ≥ τv.

Proof. We first show that ˙D7(t) ≥ 12 for all t ≥ τv. There are 3 cases:

• Case 1: Suppose Q7(t) > 0. Then by Fact 1 ˙D7(t) ≥ 12 .

• Case 2: Suppose Q2(t) > 0 and Q7(t) > 0. Then ˙D2(T ) = ˙A7(t) ≥ 12 .

By Fact 2, ˙D7(t) ≥ 12 .



• Case 3: Suppose that Q2(t) = 0 and Q7(t) = 0. By equation (24),

Λ2(t) = 0.6. By Fact 2, ˙D2(t) ≥ 12 . Since ˙A7(t) = ˙D2(t), we can invoke

Fact 2 again to conclude that ˙D7(t) ≥ 12 .

By the same reasoning, but instead looking at queues 4 and 8 rather than

2 and 7, ˙D8(t) ≥ 12 . Equations (16) through (18) imply that ˙D7 + ˙D8 ≤ 1.

Thus it must be that ˙D7(t) = ˙D8(t) = 12 .

Fact 4. Q5(t) ≡ 0 for all t ≥ τ50 , 2(Q5(0) + Q7(0) + Q6(0)) + τv.

Proof. We first observe that for regular t > s > τv,

(72) D2(t) − D2(s) ≥1

2(t− s) − Q7(s).

This is because Fact 3 fixes the rate of flow out of queue 7, that all flow intoqueue 7 comes from queue 2 departures, and that flow is conserved.

Now we consider the path of flow 1. Note that Fact 1 ensures that queue6 drain at rate at least 1/2 and thus Q6(t) = 0 for all regular t ≥ s :=τv + 2Q6(0). Consequently the fluid equations (16), (17), and (26) imply

that ˙D5(t) = 1 whenever Q5(t) > 0 for all regular t ≥ s := τv +2Q6(0). This

observation combined with the fact that ˙A5(t) ≤ 1, mean Q5(t) is monotonenonincreasing for all t > s. Now pick t = s + 2(Q5(0) + Q7(0)). Over theperiod [t, s], arrivals are at most

A5(t) − A5(s) ≤1

2(t− s) + Q7(s)

= Q5(0) + 2Q7(0).

If the queue 5 never empties during the period then,

D5(t) − D5(s) = 2(Q5(0) + Q7(0)).

Note that the first relation is because of (72), and that the maximum rateleaving the station 1, the home of queues 1 and 2, is 1. (This intuition can beconfirmed by using fluid equations (16) and (17).) Because the departuresexceed the arrivals by Q5(0) during the period [t, s] queue 5 is empty atthe end of the period. If queue 5 empties before the end of the period,than monotonicity ensures that it remain empty. Thus by time t ≥ τ50 ,

s+ 2(Q5(0) + Q7(0)), Q5(t) ≡ 0.

Fact 5. ˙D1(t) ≥ 12 for all regular points t ≥ τ50.



Proof. By Fact 2, Q5(t) = 0. If Q(t) > 0, then ˙D1(t) ≥ 12 by Fact 1.

Otherwise if Q(t) = 0, flow 1’s policing is off, and thus A1(t) = ˙Λ1(t) = 0.6.

By Fact 2, ˙D1(t) ≥ 12 .

Fact 6. For all t ≥ τsd, where τsd ≤ t01∥∥X(0)

∥∥hE for some constant

t01 that does not depend on the initial condition, Q3(t) ≡ 0, Q4(t) ≡ 0,Q5(t) ≡ 0, Q6(t) ≡ 0, and Q8(t) ≡ h.

Proof. Consider t ≥ τv. By Fact 1, ˙D3(t) ≥ 12 whenever Q3(t) ≥ 0.

Also ˙A3(t) = 0 because there are no arrivals to queue 6. Thus Q3(t) ≡ 0for all t ≥ τ30 , 2Q3(0) + τv. By the same reasoning, Q6(t) ≡ 0 for allt ≥ τ60 , 2Q6(0) + τv.

Consider t ≥ τ30. Whenever Q4(t) ≥ 0, ˙D4(t) = 1 because the stationis work conserving and the other queue the station serves is empty. Also˙A4(t) ≤ 0.6. Finally, note that by taking into account the fastest possible

growth of queue 4 over the time interval [0, τ30], we have Q4(τ30) ≤ Q4(0) +0.6τ30. We conclude that for all t ≥ τ40 , 1

0.4(Q4(0) + 0.6τ30), Q4(t) ≡ 0.Consider queue 8 at time τ40. We know that Q8(τ40) ∈ [0, Q8(0) + τ40]

by considering the maximum growth rate of queue 8 over the period [0, τ40].Three cases are possible:

• If Q8(τ40) < h, then flow 3 policing is off and ˙A8(τ40) = 0.6 while˙D8(τ40) = 0.5 by Fact 3. These rates persist until queue 8 reaches h,

which occurs at time τ8h , τ40 + 10(h− Q8(τ40)).• If Q8(τ40) > h, the flow 3 policing is on and the upstream queue,

queue 4, is empty. Thus ˙A8(τ40) = 0 while ˙D8(τ40) = 0.5 by Fact 3.These rates persist until queue 8 reaches h, which occurs at time τ8h ,

τ40 + 2(Q8(τ40) − h).• If Q8(τ40) = h, then take τ8h as equal to time τ40.

Now consider time t ≥ τ8h, and note that we have shown that Q8(τ8h) = h

and Q4(τ8h) = 0. We argue that Q8(t) ≡ h and ˙A8(t) = 0.5 by the followingreasoning. If queue 8 were to move some positive amount ε above h, thepolicing would have turned on before the queue grew to ε and prevented itfrom getting there. If queue 8 were to move some positive amount ε belowh, the policing would have turned off before the queue receded by ε, andprevented the queue from receding that much. The effects of the policing onthe growth rate of queue 8 are “instantaneous” in both cases because theupstream queue, queue 4, is empty. These observations can be verified byusing the fluid model equations (15)-(25).



Finally we have that τ30, τ60, and τ40 are each proportional to a linearcombination of the components of X(0). Also by Fact 4, τ50 is also pro-portional to a linear combination of the components of X(0) that shouldall have values of zero for any point in E . Thus there exists a t1 such thatτ30, τ60, τ40, and τ50 are all not more than t1

∥∥X(0)∥∥

hE . Time τ8h is greaterthan the maximum of three linear combinations of τ40 and |Q8(τ40)− h|, thedistance of the Q8 component of X(τ40) from E . Because the state X(·) canchange by no more over the time [0, τ40] than by an amount proportional toτ40, it follows that there exists a t2 such that τ8h ≤ t2

∥∥X(0)∥∥

hE . Thus thereexists a t0 such that

max(τ30, τ60, τ40, τ50, τ8h) ≤ t0∥∥X(0)

∥∥hE .

We are now ready to prove Lemma 7.

Proof of Lemma 7. By Fact 6 we have that there exists t01 such thatfor all t > t01

∥∥X(0)∥∥

hE the following are true: Q3(t) = Q5(t) = Q6(t) andQ8(t) = h. To prove the remainder of the Lemma, we consider each regionlisted in Table 1.Region 1: (Q2(t), Q7(t)) ∈ [0, h) × [0, h) → ( ˙Q2(t),

˙Q7(t)) = (0.1, 0).

Because policing on flow 2 is off, Fact 2 implies that ˙D2(t) ≥ 0.5, while˙D1(t) ≥ 0.5 by Fact 5. Because the station service capacity is only 1, it must

be that ˙D1(t) = ˙D2(t) = 0.5. Also ˙A2(t) = 0.6 and thus ˙Q2(t) = 0.1. The

rate ˙D7(t) = 0.5 by Fact 3. Consequently ˙Q7(t) = 0. The time to reach Eand the maximum ratio of this time to the starting distance from E is easilycomputed as in Table 1.

Region 2: (Q2(t), Q7(t)) ∈ (h,∞) × [0, h) → ( ˙Q2(t),˙Q7(t)) = (−0.5, 0).

Policing for flow 2 is on, and thus ˙A2 = 0. Fact 1 implies ˙D2(t) ≥ 0.5, while˙D1(t) ≥ 0.5 by Fact 5. Consequently ˙D1(t) = ˙D2(t) = 0.5 and ˙Q2(t) = −0.5.

The rate ˙D7(t) = 0.5 by Fact 3. Consequently ˙Q7(t) = 0. The time to reachE and the maximum ratio of this time to the starting distance from E iseasily computed as in Table 1.

Region 3: (Q2(t), Q7(t)) ∈ (0,∞) × (h,∞) → ( ˙Q2(t),˙Q7(t)) = (−0.5, 0).

The same reasoning we used to study the queue growth rates in Region2 applies for Region 3. The time to reach E and the maximum ratio of thistime to the starting distance from E is easily computed as in Table 1.



Region 4: (Q2(t), Q7(t)) ∈ 0 × (h,∞) → ( ˙Q2(t),˙Q7(t)) = (0,−0.5).

Policing for flow 2 is on, and thus ˙A2 = 0. Because queue 2 is empty

and has no arrivals, ˙D2(t) = ˙A7(t) = 0, while ˙D7(t) = 0.5 by Fact 3. Thus

( ˙Q2(t),˙Q7(t)) = (0,−0.5). The time to reach E and the maximum ratio of

this time to the starting distance from E is easily computed as in Table 1.

REFERENCES

[1] Bertsekas, D. and Gallager, R. (1992). Data Networks. Prentice Hall, EnglewoodCliffs, NJ.

[2] Chuang, J. C.-I., Fraser, A., McKeown, N., Reiter, M. K., and Reddy, R.(2003). Collaborative research: ITR/ANIR: 100 Mb/sec for 100 million households.Proposal to National Science Foundation. http://100x100network.org/index.html.

[3] Dai, J. G. (1995). On positive harris recurrence of multiclass queueing networks: aunified approach via fluid limit models. Ann. Appl. Probab. 5, 1, 49–77.

[4] Dai, J. G. and Harrison, J. M. (1991). Steady-state analysis of rbm in a rectangle:numerical methods and a queueing application. Ann. Appl. Probab. 1, 1, 16–35.

[5] Dai, J. G. and Meyn, S. (1995). Stability and convergence of moments for multiclassqueueing networks via fluid limit models. IEEE Trans. Automat. Control 40, 11, 1889–1904.

[6] Dai, J. G. and Williams, R. J. Existence and uniqueness of semimartingale reflectingbrownian motions in convex polyhedrons. Theory Probab. Appl. 40, 1.

[7] Davis, M. H. A. (1984). Piecewise deterministic markov processes: A general class ofnon-diffusion stochastic models. J. Roy. Statist. Soc. 46, 3, 353–388.

[8] Durrett, R. (2004). Probability: Theory and Examples, Third ed. ThomsonBrooks/Cole, Belmont, CA.

[9] Ethier, S. N. and Kurtz, T. G. (1986). Markov Processes: Characterization and

Convergence. Wiley, New York.[10] Gafni, E. M. and Bertsekas, D. P. Dynamic control of session input rates in

communication networks. IEEE Trans. Automat. Control 29, 11, 1009–1016.[11] Hahne, E. L. (1991). Round-robin scheduling for max-min fairness in data networks.

IEEE J. Sel. Areas Comm. 9, 7, 1024–1039.[12] Hahne, E. L. and Gallager, R. G. (1986). Round robin scheduling for fair flow

control in data communication networks. In Proceedings of IEEE International Con-

ference on Communications. 103–107.[13] Harrison, J. M. (1985). Brownian Motion and Stochastic Flow Systems. Wiley,

New York.[14] Harrison, J. M. and Reiman, M. I. (1981). Reflected brownian motion on an

orthant. Ann. Probab. 9, 2, 302–308.[15] Hayden, H. P. (1981). Voice flow control in integrated packet networks. M.S. thesis,

Electrical Engineering, Massachussets Institute of Technology, Cambridge, MA.[16] Jacobson, V. (1988). Congestion avoidance and control. Proceedings of ACM SIG-

COMM , 314–329.[17] Jain, R. (1986). A timeout-based congestion control scheme for window flow-

controlled networks. IEEE J. Selected Areas Comm. 4, 7, 1162–1167.[18] Jain, R. (1989). A delay-based approach for congestion avoidance in interconnected

heterogeneous computer networks. Comput. Comm. Rev. 19, 5, 56–71.



[19] Kaspi, H. and Mandelbaum, A. (1992). Regenerative closed queueing networks.Stochastics Stochastics Rep. 39, 4, 239–258.

[20] Katevenis, M. G. H. (1987). Fast switching and fair control of congested congestedflow in broadband networks. IEEE J. Sel. Areas Comm. 5, 8, 1315–1326.

[21] Mandelbaum, A., Massey, W. A., and Reiman, M. I. (1998). Strong approxima-tions for markovian service networks. Queueing Systems Theory and Appl. 30, 1-2,149–201.

[22] McKeown, N., Anantharam, V., and Walrand, J. (1999). Achieving 100‘%throughput in an input-queued switch. IEEE Trans. Comm. 47, 8.

[23] Meyn, S. P. and Tweedie, R. L. (1993). Markov Chains and Stochastic Stability.Sringer-Verlag, London.

[24] Meyn, S. P. and Tweedie, R. L. (1994). State-dependent convergence criteria ofmarkov chains. Ann. Appl. Probab. 4, 1, 149–168.

[25] Munkres, J. (2000). Topology , Second ed. Prentice Hall, Upper Saddle River, NJ.[26] Musacchio, J. (2005). Pricing and flow control in communications networks. Ph.D.

thesis, Electrical Engineering and Computer Sciences, University of California, Berkeley,Berkeley, CA.

[27] Stix, G. (2001). The triumph of light. Scientific American.[28] T. Konstantopoulos, S. P. and Walrand, J. (1994). Functional approximation

theorems for controlled renewal processes. J. Appl. Probab. 31, 3, 765–776.[29] Walrand, J. (1991). Communication Networks: A First Course. Aksen Associates

Incorporated, Homewood, IL.[30] Zhang-Shen, R. and McKeown, N. (2004). Designing a predictable internet back-

bone network. In Proceedings of HotNets III. San Diego, CA.

Technology and Information ManagementUniversity of California, Santa Cruz1156 High StreetMS: SOE 3Santa Cruz, CA 95064e-mail: [email protected]

Department of Electrical Engineeringand Computer Sciences

University of California, BerkeleyCory HallBerkeley ,CA 94720e-mail: [email protected]


THROUGHPUT ANALYSIS OF STOCHASTIC NETWORKS USING FLUID...

Documents

Transcript of THROUGHPUT ANALYSIS OF STOCHASTIC NETWORKS USING FLUID...