Queue-and-Idleness-Ratio Controls in Many-Server …ww2040/FQR_SSC_MOR.pdfkey step in providing...

Queue-and-Idleness-Ratio Controls in Many-Server Service Systems

Itay GurvichKellogg School of Management, Northwestern University, 2001 Sheridan Road, Evanston, IL 60208

email: [email protected]

Ward WhittIEOR Department, Columbia University, 304 S. W. Mudd Building, 500 West 120th Street, New York, NY 10027-6699

email: [email protected]

Motivated by call centers, we study large-scale service systems with multiple customer classes and multiple agent pools, eachwith many agents. We propose a family of routing rules called Queue-and-Idleness-Ratio (QIR) rules. A newly availableagent next serves the customer from the head of the queue of the class (from among those he is eligible to serve) whose queuelength most exceeds a specified state-dependent proportion of the total queue length. An arriving customer is routed to theagent pool whose idleness most exceeds a specified state-dependent proportion of the total idleness. We identify regularityconditions on the network structure and system parameters under which QIR produces an important state-space collapse (SSC)result in the Quality-and-Efficiency-Driven (QED) many-server heavy-traffic limiting regime. The SSC result is applied here toprove stochastic-process limits and in subsequent papers to solve important staffing and control problems for large-scale servicesystems.

Key words: heavy-traffic; Halfin-Whitt regime; QED regime; state-space collapse; diffusion limits; control of queueing systems

MSC2000 Subject Classification: Primary: 60K25; Secondary: 60K30, 60F17, 90B15, 90B22

OR/MS subject classification: Primary: Queues; Secondary: Diffusion models, Limit theorems

1. Introduction

1.1 Parallel-Server Systems In this paper we focus on a family of multi-class queueing networks calledParallel-Server Systems (PSS’s). In a PSS, there are multiple classes of customers (or jobs) and multiple pools ofagents (or servers), as depicted in Figure 1. Unlike many queueing networks, in a PSS customers receive at mostone service; they depart from the system after a single service completion.

N3N1 N2

2 31 4

1 2 3 4

µ11 µ12 µ21 µ22 µ23

µ33 µ43

Figure 1: A PSS and its corresponding routing graph

Customers from a common customer class are homogeneous, as are agents within the same service pool; theyhave common parameters. The agents from each service pool are allowed to serve customers from some designatedsubset of the customer classes. The allowed routing is depicted by a routing graph, as shown in Figure 1. The nodesof this graph are the queues and the service pools. There is a set I = 1, . . . , I of customer classes and a setJ = 1, . . . , J of service pools. An arc connecting customer-class i to service-pool j indicates that agents frompool j are permitted to serve customers from class i.

We will be considering Markovian PSS’s. Customers arrive exogenously according to independent Poissonprocesses, with one for each class. The class-i arrival rate is λi. The aggregate arrival rate is λ :=

∑i∈I λi.

1

2 :Mathematics of Operations Research xx(x), pp. xxx–xxx, c©200x INFORMS

Customers from each class enter service in order of arrival. Pool j contains Nj agents. The service times aremutually independent exponential random variables. When a class-i customer is served by a server from pool j,the service rate is µi,j . Whenever type-j agents do not have the required skill to serve class-i customers, µi,j = 0.For some of the results here, we will assume in addition that µi,j depends only upon i, or only upon j, or uponneither. When customers cannot enter service immediately upon arrival, they go to the end of a queue. Waitingcustomers from each queue may elect to abandon if they have not yet started service. The times different customersare willing to wait before abandoning are also mutually independent exponential random variables, having mean1/θi for class i. Unlimited patience is obtained by setting θi = 0.

PSS’s are used to model various manufacturing and service systems, especially call centers; see Gans et al.[15]. Accordingly, we use the terminology customers and agents instead of jobs and servers. In the call-centerliterature, a PSS is often called a call center model with skill-based routing. Motivated by that application, weconsider many-server PSS’s, with a large number of agents in each pool.

1.2 The Queue-and-Idleness-Ratio (QIR) Rule It remains to specify how the customers are assigned toagents. Specifically, we need to specify the routing rule, indicating what to do upon customer arrival, and thescheduling rule, indicating what to do upon service completion. Our proposed Queue-and-Idleness-Ratio (QIR)rule does both, but in a flexible way that depends on additional parameters that remain to be specified.

We now explain the QIR control. It uses two vector functions p(·) := (p1(·), . . . , pI(·)) and v(·) :=(v1(·), . . . , vJ (·)), which we call ratio functions, and which are assumed to satisfy pi(x) ≥ 0, i ∈ I,vj(x) ≥ 0, j ∈ J ,

∑i∈I pi(x) = 1 for all x ∈ R+ and

∑j∈J vj(x) = 1 for all x ∈ R+.

Let Qλi (t) and Iλ

j (t) be, respectively, the properly scaled class-i queue length and type-j number of idle agents attime t in the λth system (see Definition 2.1, and the rest of §2.1 for a fuller explanation). Let Qλ

Σ(t) :=∑

i∈I Qλi (t)

and IλΣ(t) :=

∑j∈J Iλ

j (t) – be the corresponding aggregate quantities. The QIR rule then aims to achieve

Qλi (t) ≈ Qλ

Σ(t)pi(QλΣ(t)), and Iλ

j (t) ≈ IλΣ(t)pi(Iλ

Σ(t)), i ∈ I, j ∈ J .

That is, it aims to set the scaled queue length of each class and the scaled idleness at each pool so that, when wedivide by the corresponding aggregate queue length and aggregate idleness, respectively, they are specified state-dependent ratios. Roughly, QIR will achieve this goal by letting an available agent at time t serve the customerclass with the greatest queue imbalance Qλ

i (t) − QλΣ(t)pi(Qλ

Σ(t)) and by routing an arriving customer at timet to the agent pool with the greatest idleness imbalance Iλ

j (t) − IλΣ(t)pi(Iλ

Σ(t)). The aim of QIR is to drivethese imbalances toward 0. We will show that that goal is achieved asymptotically; i.e., we achieve asymptoticproportionality. In fact, the precise definition of the queue and idleness imbalances will be slightly more intricate;see Definition 2.3.

However, even with this added complexity, QIR has important simplicity, because at each decision epoch –customer arrival or service completion – the decision rule uses only local and aggregate idleness and queue-lengthinformation. An available agent will need to know only the queue length of the customer classes that he can serveand the overall number of customers in the system in order to choose which customer to serve next. In particular,he will neither need to know the queue length of all other customer classes nor will he need to know the detailedoccupancy information that specifies the number of type-j agents giving service to class-i customers. Moreover,the QIR rule does not depend on the model parameters.

1.3 The QED Many-Server Heavy-Traffic Limiting Regime To establish asymptotic-proportionality re-sults for many-server PSS’s, we work in the QED many-server heavy-traffic limiting regime, first formalized byHalfin and Whitt [21] for the M/M/N queue. For the M/M/N model, the QED regime is obtained by letting theaggregate arrival rate λ and the number N of agents grow indefinitely, while holding the service rate fixed, so thatthe utilization, ρ := λ/Nµ, approaches its critical value 1 in an appropriate manner. Specifically, considering asequence of systems indexed by the aggregate arrival rate λ, it is assumed that

√λ(1− ρλ) → β as λ →∞, (1)

where −∞ < β < ∞. This QED regime is now widely accepted as the most useful heavy-traffic limiting regimefor many-server systems. It is to be contrasted with the conventional heavy-traffic regime, in which the number ofagents (servers) is held fixed while letting ρ approach one. Halfin and Whitt showed that the limit in (1) holds forthe M/M/N model if and only if the steady-state probability that a new arrival must wait before beginning serviceapproaches a limit strictly between 0 and 1. Given the appropriate heavy-traffic condition, as in (1), we then seek

:Mathematics of Operations Research xx(x), pp. xxx–xxx, c©200x INFORMS 3

to obtain limits for the properly scaled and normalized processes associated with queue-length, waiting time, etc.However, we point out that the situation is not nearly as straightforward for the more general PSS’s consideredhere. In particular, we will use mathematical programs to specify the QED regime in the present context; see §2.

1.4 State-Space Collapse (SSC) Like many other multi-class queueing networks, PSS’s are quite complexand challenging to analyze. To address this complexity for queueing networks, recent research has establishedspecial asymptotic techniques in order to obtain more elementary descriptions of these complex models. Theseminal papers of Bramson [6] and Williams [32] provide a magnificent example of such a simplification byshowing how complex multi-class queueing networks can be approximated by more elementary diffusion models,known as semi-martingale reflected Brownian motions (SRBMs). A key step in this complexity-reduction is astate-space collapse (SSC) result, based on hydrodynamic limits, that connects a high-dimensional queue-lengthprocess with a lower-dimensional workload process, asymptotically, in the heavy-traffic limit.

These initial papers by Bramson and Williams focus on open queueing networks in the conventional heavy-traffic limiting regime, instead of PSS’s in the QED limiting regime, but it is now clear that SSC can serve as akey step in providing simple solutions for many complex stochastic systems. Indeed, SSC results have previouslybeen established in the QED regime. Paralleling the history for the conventional-heavy-traffic regime, the firstSSC results for PSS’s were based on ad-hoc proofs for specific models and controls. Examples include Armony[1], Gurvich et. al. [17], Armony and Maglaras [3, 2]. Recently, Dai and Tezcan [12] made the important step ofextending Bramson’s framework to the case of the many-server heavy-traffic regime. They applied their frameworkto several settings; see Tezcan [30] and Dai and Tezcan [10, 11].

We contribute to this SSC literature by establishing that our proposed QIR controls produce an important SSCfor a large class of PSS’s in the QED limiting regime. The SSC occurring here is due to the asymptotic proportion-ality mentioned above. It would be natural to apply Dai and Tezcan [12] for this purpose, but their results cannotbe directly applied to the QIR control because of the general structure of the ratio functions. Nevertheless, the keyideas in Dai and Tezcan [12], which in turn build on Bramson [6], lie at the heart of the SSC proofs here.

As a consequence of our SSC results here, asymptotically, the multi-dimensional queue-length and idlenessprocesses will be determined by a process of a lower dimension that contains, in the worst case, only the numberof customers in system of each class. This notion of SSC is somewhat weaker than often seen in the conventionalheavy-traffic regime, where the multi-dimensional queue-length process actually collapses into a one-dimensionalprocess; e.g., see [24]. However, we too obtain a strong one-dimensional form of SSC in the many-server settingwhen the service rates are pool-dependent; see Theorem 4.1.

In closing this discussion of SSC, we mention the important paper by Atar [4], which focuses on establishingasymptotic optimality in heavy-traffic for PSS’s in the QED regime. In §5 we will show that, for special (tree)networks, QIR is equivalent to Atar’s control. In these cases the SSC result follows from the analysis in [4].

1.5 The Value of QIR and SSC We illustrate the value of SSC by applying it to establish stochastic-processlimits for the basic stochastic process of interest with an arbitrary QIR control, under regularity conditions, in§4. As usual, the limits are diffusion processes. However, the SSC results are important even without subsequentstochastic-process limits. We use SSC and the associated asymptotic proportionality to establish desired propertiesof QIR controls, even for cases in which a stochastic-process limit remains to be established. In particular, wedemonstrate the power of QIR for controlling PSS’s in subsequent papers: In [19, 18], we examine a centralquestion in call-center operations, namely how to jointly determine the design, staffing and routing (real-timecontrol) of call-centers with multiple customer classes and agent pools. These decisions need to be made so as tominimize labor-related costs while maintaining pre-determined Quality-of-Service (QoS) constraints. The controlcomponent of our proposed solution is a special case of QIR called Fixed-Queue-Ratio (FQR) routing.

In [20] we show that, as long as the service-rates are pool dependent, i.e, when µi,j = µj for all i and j, QIRwith appropriately chosen parameters is asymptotically optimal with respect to convex holding costs in the QEDregime. Moreover, we show that in certain cases QIR is partially equivalent to the well-known Generalized-cµ(Gcµ) rule. By doing this, we are able to partially extend the important results of Mandelbaum and Stolyar [24] inthe conventional heavy-traffic regime to the many-server QED regime.

1.6 Organization of the Paper We elaborate on the model and define the QIR controls in §2. We state ourmain SSC result (Theorem 3.1) in §3. We apply SSC to establish stochastic-process limits in §4. In §5 we relateQIR to Atar’s control in [4] for a class of tree networks, and use that connection to provide an alternative proof of


SSC. We provide proofs for the results in §§3, and 4, 5, respectively, in §§6, 7 and 8.

2. The Model We now elaborate on the model description given in §1. The possible routing for a PSS has anatural representation as a bipartite graph with vertices V = J ⋃ I. The only edges in the graph connect customerclasses to agent pools: E := (i, j) ∈ I × J : µi,j > 0. An edge (i, j) is present in the routing graph if class-icustomers can be served by type-j agents. Actually, we may choose to use only a subset of the edges of E. Theeventual sub-graph will depend on the solution of a linear program (LP). Figure 1 depicts a PSS routing graph.

Let Qi(t) be the queue length of class-i customers, and Ij(t) the number of idle agent in pool j, at time t.The corresponding aggregate quantities are QΣ(t) :=

∑i∈I Qi(t) and IΣ(t) :=

∑j∈J Ij(t). Let Zi,j(t) be the

number of type-j agents busy giving service to class-i customers, and let Zj(t) :=∑

i∈I Zi,j(t) be the numberof busy agents at pool j at time t. Consequently, Ij(t) = Nj − Zj(t). The overall number of class-i customerspresent in the system at time t is then given by Xλ

i (t) := Qi(t) +∑

j∈J Zi,j(t). Finally, let XλΣ(t) be the overall

number of customers in the system (in service and in queue), i.e.,

XΣ(t) :=I∑

i=1

Xi(t) =I∑

i=1

Qi(t) +

J∑

j=1

Zi,j(t)

.

To construct the heavy-traffic framework, we consider a sequence of systems indexed by λ. We add the super-script λ to express the dependence on the index. Thus, Qλ

i (t) stands for the class-i queue length at time t in theλth system. The routing graph and service rates µi,j , i ∈ I, j ∈ J do not change with λ.

Notational conventions. For an integer d > 0, let Dd := Dd[0,∞) be the space of all RCLL (Right Continuouswith Left Limits) functions with values in d-dimensional Euclidean space Rd, equipped with the Skorohod J1

metric; e.g., see [31]. We will often use convergence in probability (commonly denoted by P→) for a sequence ofrandom elements of Dd to the zero function (the function in Dd that is identically 0), here denoted by 0. However,convergence in probability to a deterministic limit is equivalent to convergence in distribution to that limit, denotedby ⇒. As a consequence, for a family of stochastic processes, Y λ : λ > 0 with sample paths in Dd, we willbe showing that Y λ ⇒ 0 in Dd as λ →∞; we will write Y λ(t) ⇒ 0 in Dd to emphasize that we are consideringprocesses in Dd instead of stationary distributions on R.

Since the limit processes we consider are either the deterministic zero function or diffusion processes, the limitprocess has continuous sample paths, so the notion of convergence on the underlying function space Dd coincideswith uniform convergence on closed bounded intervals. To express that, for a vector-valued process B(t) in Dd, let‖B‖∗s,T = sups≤t≤T ‖B(t)‖, where ‖B(t)‖ =

∑Jk=1 |Bk(t)|. These are defined similarly for a process B(t) in

Dd×m[0,∞), where now ‖B(t)‖ =∑d

k=1

∑ml=1 |Bk,l(t)|. We omit the subscripts whenever we refer to vectors or

to vector processes. For example, Nλ and Xλ(t), will stand, respectively, for the vector in ZJ whose componentsare Nλ

j and the vector in RI whose components are Xλi (t). Using this notation, we will say that a sequence

xλ : λ > 0 of processes with sample paths in Dd satisfies that xλ = oP (1) if ‖xλ‖∗T ⇒ 0 as λ →∞.

We will also consider a weaker notion of convergence, using the space Dd− := Dd(0,∞), where the domain

is treated as open at the left instead of closed. We again let convergence (to continuous limits) be characterizedby uniform convergence over bounded intervals. The restriction to the domain (0,∞) means that we excludeuniform convergence for intervals of the form [0, b]. We have Y λ(t) ⇒ 0 in Dd(0,∞) if and only if, for each0 < s < T < ∞, ‖Y λ‖∗s,T ⇒ 0.

Finally, we mention conventions for vector products: For x, y ∈ Rd, xy is the component-wise product ((xy)i =xiyi). Whenever x ∈ Rd but y ∈ R, xy should be interpreted so that (xy)i = xiy. Finally, x · y is the scalarproduct and ej is the unit vector with 1 in the jth place and 0 elsewhere.

2.1 Heavy-Traffic Conditions and the Fundamental Mathematical Program We need to make two as-sumptions to put our PSS into the QED many-server heavy-traffic limiting regime. The first is a natural general-ization of the condition (1), but that is not enough.

Assumption 2.1 (heavy-traffic conditions) There are constant ai > 0, i ∈ I, and νj > 0, j ∈ J , such that, asλ → ∞, λi/λ → ai > 0, i ∈ I , and Nλ

j /λ → νj > 0, j ∈ J . Also, there exist constants ξi ∈ (−∞,∞), i ∈ I


and γj ∈ (−∞,∞), j ∈ J , such that, as λ →∞,

λi − aiλ√λ

→ ξi andNλ

j − νjλ√λ

→ γj .

The second QED assumption concerns a mathematical program (in the “fluid scale”). We assume that the constantsof Assumption 2.1 are specified. The fundamental mathematical program is:

Minimize ρSubject to:

∑j∈J µi,jνjxi,j = ai, i ∈ I,∑i∈I xi,j ≤ ρ, j ∈ J ,

ρ ≥ 0, xi,j ≥ 0, i ∈ I, j ∈ J .

(2)

Since the constants of Assumption 2.1 are specified, the mathematical program in (2) is an LP. An optimalsolution is a vector (x, ρ) ∈ RI×J

+ × R+ that is feasible for (2) such that no other feasible solution has a lower ρvalue. We next impose a critical-loading assumption.

Assumption 2.2 (critical loading) For any optimal solution (x, ρ) of (2),∑

i∈I xi,j = 1 for all j ∈ J , so thatρ = 1.

The fact that Assumption 2.2 applies to any optimal solution is important. From a practical perspective, this anatural restriction. If there exists a solution in which some of the agent pools are underloaded, that is

∑i∈I xi,j < 1

for some j ∈ J , then it makes sense to decrease the staffing levels. From a mathematical perspective, Assumption2.2 is imposed to prevent the fluid from drifting into an underloaded state, away from the QED regime.

With Assumption 2.2, it suffices to denote the selected optimal solution by its x coordinate, since the value ofρ will necessarily be 1. Hence, fix an optimal solution x for (2) satisfying Assumption 2.2. We now indicate howthe chosen optimal solution x is used. Since we intend to use QIR for the routing, we do not use x for the routing,but x plays a critical role in the design. Specifically, we omit all edges with xi,j = 0 from the network routinggraph; i.e., we do not allow any class i customers to be routed to pool j if xi,j = 0. The routing graph includesall edges with xi,j > 0; we stipulate that the routing graph is (i, j) ∈ I × J : xi,j > 0. If, a priori, pool j isunable to serve class i or if we do not want pool j to serve class i, then we enforce that by imposing the constraintxi,j ≤ 0 in the mathematical program.

The routing graph determined by x is closely linked to the dynamic control. Indeed, our SSC results forQIR will depend on characteristics of the routing graph. With that in mind, we note that the LP may well havemultiple optimal solutions, and different optimal solutions may thus lead to different routing graphs, according tothe construction above.

To facilitate the following discussion, we need to be clear about the network graphs under consideration: Ournetwork graphs are simple undirected bipartite graphs, i.e., with at most one edge connecting any two nodes, andwith edges only between a customer class and an agent pool. Beyond these basic features, we first require that thegraph is connected; i.e., there exists a path between every two nodes in the graph:

Assumption 2.3 (connected routing graph) The selected optimal solution (x) for (2) produces a routing graphdetermined by the edges E(x) := (i, j) ∈ I × J : xi,j > 0 that is connected.

Assumptions 2.1-2.3 are assumed to hold throughout the rest of the paper. The connected-graph assumptionis crucial for the ability to instantaneously balance the system asymptotically; see §2.7 of Atar [4] for elaboration.We will actually need additional structure beyond connectedness. Connected graphs can be cyclic or acyclic (agraph is acyclic if there is a unique path between each pair of nodes). This distinction is important, because QIRworks well with cyclic networks only when certain parametric conditions hold; see Theorem 3.1 and Remark 3.1.

Given an optimal solution (x) to (2), J(i)(x) for i ∈ I is defined to be the set of agent pools connected tocustomer class i, i.e., J(i)(x) = j ∈ J : xi,j > 0. Analogously, we let I(j)(x) for j ∈ J be the set ofcustomer classes connected to agent pool j, i.e., I(j)(x) = i ∈ I : xi,j > 0. We will often omit the argument xwhen it is clear from the context.

Definition 2.1 (scaled and normalized processes) Fix an optimal solution x for (2) for which the edges in E(x)induce a connected routing graph. Then, we define the following scaled processes:


XλΣ(t) :=

XλΣ(t)−Nλ

Σ√λ

; IλΣ(t) :=

IλΣ(t)√

λ; Xλ

i (t) :=Xλ

i (t)−∑j∈J xi,jN

λj√

λ, i ∈ I;

Qλi (t) :=

Qλi (t)√λ

, i ∈ I; Iλj (t) :=

Iλj (t)√

λ, j ∈ J ; Zλ

i,j(t) :=Zλ

i,j(t)− xi,jNλj√

λ, (i, j) ∈ I × J .

Once we fix a solution x, which in turn fixes the corresponding routing graph E(x), we assume that Zλi,j(t) ≡ 0

for all (i, j) /∈ E(x). This can be relaxed to allow for initial conditions, Zλi,j(0), that are not consistent with the

routing graph but here we prohibit this option.

2.2 Definition of QIR We will impose a smoothness condition on our ratio functions. For that purpose,following convention, we say that an Rm-valued function f on a subset S of Rk is locally Holder continuous withexponent α > 0 if, for every compact subset K ⊂ S, there exists a constant CK such that ‖f(x) − f(y)‖ ≤CK‖x− y‖α for all x, y ∈ K, where ‖ · ‖ is a chosen norm inducing the usual Euclidean topology, which we taketo be the L1 norm: ‖x‖ :=

∑i |xi|.

Definition 2.2 (an admissible state-dependent ratio function) For an integer d > 0, a vector valued functionr ≡ r(·) : R+ 7→ Rd

+, is an admissible state-dependent ratio function if∑d

k=1 rk(x) = 1 for all x ∈ R+ andif every component rk : R+ 7→ R+ is locally Holder continuous on the open interval (0,∞) for some exponentαk > 0.

For the following definition, we assume that an optimal solution x for (2) is fixed and the routing graph E(x) isused. We omit the argument x from the notation.

Definition 2.3 (QIR for admissible state-dependent ratio functions) Given two admissible state-dependent ra-tio functions v and p, QIR is defined as follows:

• Upon arrival of a class-i customer at time t, the customer will be routed to an available agent in poolj∗, where

j∗ := j∗(t) ∈ argmaxj∈J(i),Iλ

j (t)>0

Iλj (t)− [Xλ

Σ(t)]−vj

([Xλ

Σ(t)]−)

;

i.e., the customer will be routed to an agent pool with the greatest idleness imbalance. If there are no suchagents, the customer waits in queue i, to be served in order of arrival.

• Upon service completion by a type-j agent at time t, the agent will admit to service the customer fromthe head of queue i∗, where

i∗ := i∗(t) ∈ argmaxi∈I(j),Qλ

i (t)>0

Qλ

i (t)− [XλΣ(t)]+pi

([Xλ

Σ(t)]+)

;

i.e., the agent will admit a customer from the queue with the greatest queue imbalance. If there are nosuch customers, the agent will remain idle.

Ties are broken in an arbitrary but consistent manner, so that the vector-valued stochastic process (Qλ, Zλ) is aCTMC with stationary transition probabilities.

Remark 2.1 (degrees of freedom in routing and scheduling) A careful reading of the results and proofs thatfollow will reveal that the actual way in which j∗ or i∗ are defined is immaterial as long as they are chosen so that

j∗ ∈ V+ :=

j ∈ J(i) : Iλj (t) > 0 and Iλ

j (t)− [XλΣ(t)]−vj

([Xλ

Σ(t)]−)

> 0

i∗ ∈ U+ :=

i ∈ I(j) : Qλi (t) > 0 and Qλ


([Xλ

Σ(t)]+)

> 0

.

Remark 2.2 (Joint work conservation and the definition of QIR) We note that QIR routing rule does not workdirectly with the queue imbalances Qλ

i (t) − pi(QλΣ(t))Qλ

Σ(t). Rather, the queue imbalances are modified so that[Xλ

Σ(t)]+ appears instead of QλΣ(t). This modification is important for technical reasons; indeed, the proofs rely

on this heavily. We now explain why this is not only a technical assumption. We first observe that, for some simple


systems, such are the V and inverted-V models depicted in Figure 3, we have QλΣ(t) = [Xλ

Σ(t)]+ under any workconserving policy. However, in general, that is not the case. Indeed, for the M model in Figure 3, it is possibleto have a positive queue for one of the customer classes while there are idle servers in one of the server pools. Areasonable policy will try to prevent such situations, at least approximately, by ensuring that [Xλ

Σ(t)]+ ≈ QλΣ(t)

and [XλΣ(t)]− ≈ Iλ

Σ(t). Indeed, QIR does exactly that. It guarantees that the queues and idleness have the rightproportions, but at the same time the system satisfies

[XλΣ(t)]+ ≈ Qλ

Σ(t) = oP (1) and [XλΣ(t)]− ≈ Iλ

Σ(t) = oP (1).

Following Atar [4], we call this joint work conservation. We now show that joint work conservation cannot beguaranteed if QIR uses the original queue imbalances, Qλ

i (t) − pi(QλΣ(t))Qλ

Σ(t), rather than the modified ones.To this end, consider the system with 4 customer classes and 3 agent pools as depicted in Figure 2. Assume that

1 2 3 4

Pool 1 Pool 2 Pool 3

Figure 2: An example without joint work conservation

the ratios are p1(·) = p2(·) = p3(·) ≡ 0 and p4(·) ≡ 1. In Figure 2, queues 2 and 4 have some customers waitingwhile the other queues are empty. Also, server pools 2 and 3 are completely busy, while there are some serversavailable at pool 1. Moreover, assume that at this time t, Iλ

1 (t) > QλΣ(t) and, in particular, [Xλ

Σ(t)]+ = 0. Then,because p2 = 0 and p4 = 1, a newly available agent in pool 3 will serve a customer from queue 2, which is thequeue with the highest value of Qλ

i (t)− pi(QλΣ(t))Qλ

Σ(t). However, in order to move quickly towards joint workconservation, we would prefer this agent from pool 3 to serve class 4. We would prefer to have pool 2 serve class2, letting pool 1 handle all the class 1 arrivals. That alternative policy will “move” the work towards the non-busypool 1. Under QIR, which uses [Xλ

Σ(t)]+, this desired action is exactly what the system would do. In other words,the modified definition of the imbalances is important to force the system to move rapidly towards joint workconservation.

In some cases, QIR does not depend on which form of imbalances are used. Specifically, assume that, forall i ∈ I, the function fi(x) := xpi(x) is strictly increasing. Then, if we change [Xλ

Σ(t)]+ to Qλi (t) in the set

i ∈ I(j) : Qλi (t) > 0 and Qλ


([Xλ

Σ(t)]+)

> 0

, then the set is unchanged. Consequently, wecan use the original queue imbalances to define QIR. A special case in which this holds is highlighted in Remark2.3 below. Similar reasoning applies to the idleness ratios. In particular, if the functions gi(x) = xvj(x) are strictlyincreasing, then we can use Iλ

j (t)− vj(IλΣ(t))Iλ

Σ(t) in defining the actions upon customer arrival.

Remark 2.3 (simplification under fixed queue ratios) If p(·) ≡ (p1, . . . , pI) with pi > 0 for all i ∈ I , then QIRis achieved by a newly available agent serving the customer from the head of queue i∗ where

i∗ ≡ i∗(t) ∈ argmaxi∈I(j),Qλ

i (t)>0

Qλ

i (t)pi

.

3. State-Space Collapse Under QIR


Theorem 3.1 (SSC under QIR) Fix an optimal solution x for (2) for which the edges in E(x) induce a con-nected routing graph. Fix the two admissible state-dependent ratio functions p and v. Let QIR be used, followingDefinitions 2.3. Suppose that at least one of the following conditions holds with respect to x:

• C- 1 The service rates depend only on the agent type: For all (i, j) ∈ E(x), µi,j = µj .

• C- 2 The service rates depend only on the customer class: For all (i, j) ∈ E(x), µi,j = µi.

• C- 3 Acyclic routing graph: The routing graph determined by E(x) is acyclic; i.e., it is a tree.

If, in addition, (Xλ(0), Zλ(0)) ⇒ (X(0), Z(0)) in RI+I·J , then we have SSC:

Qλi (t)− Qλ

Σ(t)pi

(Qλ

Σ(t))⇒ 0 and Iλ

j (t)− IλΣ(t)vj

(IλΣ(t)

)⇒ 0 in D− (3)

as λ → ∞ for each i ∈ I and j ∈ J . The convergence in (3) is strengthened to convergence in D if we assumethat

Qλi (0)− Qλ

Σ(0)pi

(Qλ

Σ(0))⇒ 0, i ∈ I, and Iλ

j (0)− IλΣ(0)vj

(IλΣ(0)

)⇒ 0, j ∈ J . (4)

Finally, if condition C-3 holds, then,

1√λ

Zλij(t) ⇒ 0 in D as λ →∞, i ∈ I, j ∈ J . (5)

Remark 3.1 (network-graph characterization) Note that arbitrary connected graphs, including cyclic graphs,are allowed under Conditions C-1 and C-2. However, Condition C-3 rules out cyclic structures. Some simple, yetimportant, tree structures, are the V, N, M and

∧(inverted- V ) models shown in Figure 3.

V-Model N-Model M-Model -Model

Figure 3: The V, N, M and∧

Models

At first glance it might seem that the cyclic networks are not required at all, and that one might obtain a cyclicstructure (thus satisfying C-3) by removing some of the edges. However, that is mathematically incorrect. Thereare examples in which the mathematical program (2) does not have any acyclic solutions that are connected butdoes have some cyclic solutions. A simple example is constructed as follows: consider a model with two customerclasses and two agent groups. Namely, a model with I = J = 1, 2, with λ1 = λ2 (so that a1 = a2 = 0.5),µi,j = 1 for all i, j ∈ 1, 2 and finally with ν1 = ν2 = 0.5. Then, there exists no connected acyclic solutionto (2). However, there exist infinitely many cyclic solutions; e.g., xi,j = 0.25 for all i, j ∈ 1, 2 is an optimalsolution that induces a cyclic connected graph.

Remark 3.2 (class-pool occupancy processes) We now explain why the limit for the class-pool occupancy pro-cesses in (5) holds under condition C-3 but not under any of the other conditions. The reason is the tree structureimposed by condition C-3. In a tree model, controlling the queues and idleness processes uniquely determinesthe class-pool occupancy processes Zλ

i,j(t) through a linear mapping (equation (11)). That is analogous to whathappens in network flows: When the network is a tree, there is a unique way of satisfying all the demand in thenetwork. In the presence of cycles, however, there are many (possibly infinitely many) flows that can satisfy all


the demands. In cyclic structures, then, QIR self-selects these agent ratios, not necessarily consistently with thesolution x, and the outcome might depend on the actual ratios p and v. In general, this self-selection might haveundesired effects on the system capacity, but not when one of the conditions C-1 or C-2 holds. Under one ofthese conditions, the aggregate ’fluid’ capacity of the system is invariant with respect to the self-selection of theclass-pool occupancy processes and is given by

∑j∈J µj νj under condition C-1, or

∑i∈I µiai under condition

C-2.

EXAMPLE 3.1 (SSC in a two-class model) Even though SSC is an asymptotic property, systems of medium sizealready exhibit this phenomenon. To illustrate, consider a system with two customer classes, I = 1, 2, and twoagents types, J = 1, 2. Let the arrival rates be λ1 = λ2 = 200. Assume there is no abandonment. Agents oftype 1 can serve both class-1 and class-2 customers. They serve class-1 customers at rate µ1,1 = 1 and class-2customers at rate µ2,1 = 3. Agents of type 2 can also give service to both classes, and they do so with ratesµ1,2 = 2 and µ2,2 = 3. Assume that N1 = 100 and N2 = 117 which corresponds roughly to ν1 = 1/4, ν2 = 7/24and γ1 = γ2 = 0, so that an optimal solution for (2) is given by x1,1 = 1, x1,2 = 3/7, x2,2 = 4/7, and xi,j = 0otherwise. This solution translates to an N model (see Figure 3).

We simulate this resulting N model to see if there is approximate SSC. Suppose that we use QIR with ratiovector p(x) ≡ (1/3, 2/3) and v(x) ≡ (0, 1). That is, we use a fixed (FQR) ratio function rather than a state-dependent one. With the given ratio vector p = (1/3, 2/3), we should have that Q2(t) ≈ 2 · Q1(t). Indeed, thesimulation results in Figure 4 show that Q2(t) and 2 ·Q1(t) are hardly distinguishable.

0 5 10 15 20 25 30 35 40 45 500

10

20

30

40

50

60

70

80

90

100

Number of customers in queues 1 and 2

Time

Nu

mb

er in

qu

eue

2*Queue 1

Queue 2

Figure 4: The N Model for the Numerical Experiment

4. Stochastic-Process Limits We now apply SSC to establish stochastic-process limits under each of theconditions C-1, C-2, and C-3 of Theorem 3.1. The limits relate the complicated SBR model to much more elemen-tary models, namely, the single-class multi-type inverted-V model and the multi-class single-type V model; seeFigure 3. Toward this end, we define Wλ

i (t) :=√

λWλi (t) to be the scaled virtual waiting time process of class-i

customers in the λth system. Also, as before, we set ai := λi/λ. Throughout this section we fix the admissibleratio functions p(·) and v(·). To simplify the notation, we define for all x ≥ 0:

pi(x) := xpi(x), i ∈ I, and vi(x) := xvi(x), j ∈ J .

The first stochastic-process limit shows the consequence of SSC; the multidimensional limit process is a func-tion of the one-dimensional limiting process XΣ(t). The joint limits for the queue-length and virtual-waiting-timeprocesses imply the heavy-traffic Little’s law applied in the heuristic explanation of service-level differentiation in(4) of [19]. For applications, it is important to realize that we are treating the overall virtual waiting time, includingboth customers who will be served and customers who will abandon. When the abandonment rate is suitably small,there will be little difference between the overall waiting time and the waiting time conditional on being served.

Theorem 4.1 (diffusion limit under condition C-1) Under the assumptions of Theorem 3.1 with condition C-1,


we have the joint convergence(Xλ

Σ(t), Qλ1 (t), . . . , Qλ

I (t), Wλ1 (t), . . . , Wλ

I (t), Iλ1 (t), . . . , Iλ

J (t))⇒

(XΣ(t), p1([XΣ(t)]+), . . . , pI([XΣ(t)]+),

1a1

p1([XΣ(t)]+), . . . ,1aI

pI([XΣ(t)]+),

v1([XΣ(t)]−), . . . , vJ([XΣ(t)]−))

in D2I+J+1− as λ →∞, where XΣ is the unique (possibly weak) solution of the following one-dimensional SDE:

XΣ(t) = XΣ(0)−∑

j∈Jµjγjt +

∑

j∈Jµj

∫ t

0

vj([XΣ(s)]−)ds−∑

i∈Iθi

∫ t

0

pi([XΣ(s)]+)ds +√

2B(t) , (6)

with B := B(t), t ≥ 0 being a standard Brownian motion. Moreover, the convergence of XλΣ(t) can be

strengthened to convergence in D, i.e, XλΣ(t) ⇒ XΣ(t) in D as λ →∞.

Remark 4.1 (when C-1 fails to hold) The result of Theorem 4.1 is quite strong. While the state of the PSSis characterized by the multi-dimensional process (Qλ

i (t), Zλi,j(t); i ∈ I, j ∈ J ) for each λ, asymptotically as

λ →∞ we can characterize the per-class queue-length processes Qλi (t) and the per-pool idleness processes Iλ

j (t)in terms of the overall number of customers in the system through the one-dimensional process XΣ(t). We claimthat condition C-1 is actually necessary to obtain this result. Under condition C-1, it suffices to know the number ofbusy agents Zλ

j (t) (or the corresponding number of idle agents Iλj (t)) in each pool, in order to know the departure

rate from the system. That makes Theorem 4.1 follow directly from SSC, because it allows us to control theproportions of idle agents. In the absence of condition C-1, we need more detailed information in order to knowthe departure rate from the system. In particular, we need to know the actual values of

(Zλ

i,j(t); i ∈ I, j ∈ J ),

over which, in general, we have no control through SSC.

Remark 4.2 (equivalence with the single-class model) Whenever µj ≡ µ and θj ≡ θ for all j, the limit is thesame as the one obtained for a sequence of M/M/N +M queues in the Halfin-Whitt regime. With the replacement

of the space scaling,√

λ, by the scaling√

NλΣ, this limit is given in Theorem 2 of [16]. If, in addition θ = 0, then

the same replacement of scaling leads to the limit process given in Theorem 2 of [21] for a sequence of M/M/Nqueues with R+(

√µ

∑j γj)

√R+o(

√R) agents, where R = λ/µ. Specifically, consider a sequence of M/M/N

queues with arrival rate λ, service rate µ and Nλ = R + (√

µ∑

j γj)√

R + o(√

R). Let Xλ(t) be the overallnumber of customers in the λth M/M/N queue and let Y λ(t) := [Xλ(t)−Nλ]/

√λ. Then, Theorem 2 of [21] is

equivalently stated as follows: Provided that Y λ(0) ⇒ Y λ(0), we have Y λ(t) ⇒ Y (t) in D[0,∞), where Y is adiffusion process satisfying the SDE

Y (t) = Y (0)− µ∑

j∈Jγjt + µ

∫ t

0

[Y (s)]−ds +√

2B(t) , (7)

with B := B(t), t ≥ 0 being a standard Brownian motion. As a consequence, then, whenever θi ≡ 0 andµj ≡ µ, the PSS and the associated M/M/N queue have asymptotically the same probability law.

Remark 4.3 (equivalence with the inverted-V model) Whenever θi = θ for all i ∈ I, the limit we obtain isequal to the limit that we would obtain in the associated inverted-V model, namely, in a model with the same set Jof agent pools, same service rates µj , j ∈ J and same staffing levels Nλ

j , j ∈ J , but with a single customerclass having arrival rate λ. This asymptotic equivalence of the SBR system and the inverted-V model under theassumption of pool-dependent service rates is used extensively in our two subsequent papers [19] and [20].

The diffusion limits given in the Theorem 4.1 characterize the asymptotic behavior on bounded time periods.The next natural step is to try and understand the asymptotic behavior of the steady-state queue length and waitingtime. In some settings one can actually identify the limits of the steady-state variables with the steady-state ofthe diffusion limit. This, however, requires a limit-interchange argument whose main component is to establishtightness of the scaled steady-state variables. Such an argument was used in simple settings like [21], [16], [17],and also in the more complicated case of Tezcan [30]. Establishing such an interchange is, however, extremely hardin general. Gamarnik and Zeevi [14] and Budhiraja and Lee [8] have developed suitable arguments for generalized


Jackson networks in the conventional, single-server, heavy-traffic regime. Their techniques may be adapted, on acase-by-case basis, to certain many-server systems, as was done in [30]. A general framework is, however, stillmissing.

We do not prove the interchange argument for the general PSS case. Assuming that tightness of the scaledsteady-state variables holds, however, we can easily link this with the steady-state of the limit diffusion. This linkis established in Corollary 4.2 below.

Corollary 4.2 (steady-state limits) Assume that there exists a λ0 < ∞ such that steady state exists for all λ > λ0.Assume also that the sequence (Qλ

Σ(∞), IλΣ(∞)), λ > λ0 is tight. Then, as λ →∞,

XλΣ(∞) ⇒ XΣ(∞), Iλ

j (∞) ⇒ vj([XΣ(∞)]−), j ∈ J ,

Qλi (∞) ⇒ pi([XΣ(∞)]+), and Wλ

i (∞) ⇒ 1ai

pi([XΣ(∞)]+), i ∈ I. (8)

If, in addition, the sequence QλΣ(∞) is uniformly integrable then the convergence in (8) holds also in expectation.

The following is a diffusion-limit result under Condition 2. See §6.3 for the definition of βi and the constructionof the processes Xλ

i (t).

Theorem 4.3 (diffusion limit under condition C-2) Under the assumptions of Theorem 3.1 with condition C-2,we have the joint convergence

(Xλ

Σ(t), Xλ1 (t), . . . , Xλ

I (t), Qλ1 (t), . . . , Qλ

I (t), Wλ1 (t), . . . , Wλ

I (t), Iλ1 (t), . . . , Iλ

J (t))⇒

(XΣ(t), X1(t), . . . , XI(t), p1([XΣ(t)]+), . . . , pI([XΣ(t)]+),

1a1

p1([XΣ(t)]+), . . . ,1aI

pI([XΣ(t)]+),

v1([XΣ(t)]−), . . . , vJ([XΣ(t)]−))

in D3I+J+1− as λ → ∞, where (X1(t), . . . , XI(t)) is the unique (possibly weak) solution of the following I-

dimensional SDE:

Xi(t) = Xi(0)− βit− µi

∫ t

0

Xi(s)ds− (θi − µi)∫ t

0


2aiBi(t), (9)

where the processes Bi(t), t ≥ 0, i ∈ I are independent standard Brownian motions.

Remark 4.4 (equivalence with the V model) The limit we obtain in Theorem 4.3 is equal to the limit that wewould obtain in the associated V model, namely, in a model with the same set I of customer classes and respectivearrival and service rates λi, i ∈ I and µi, i ∈ I, but with a single agent pool having

∑j∈J Nλ

j agents. Thisstands in contrast to the case of pool-dependent service rates in which the reduction is to a system with multipleagents pools but a single customer class; see Remark 4.3.

The following is a diffusion-limit result under Condition 3. To abbreviate notation, given x, we let p(x) :=(p1(x), . . . , pI(x)) and, similarly, v(x) := (v1(x), . . . , vJ(x)). The mapping G(·, ·), that is used in the statementof the Theorem, is defined in §5 below.

Theorem 4.4 (diffusion limit under condition C-3) Under the assumptions of Theorem 3.1 with condition C-3,we have the joint convergence

(Xλ

Σ(t), Xλ1 (t), . . . , Xλ

I (t), Qλ1 (t), . . . , Qλ

I (t), Wλ1 (t), . . . , Wλ

I (t), Iλ1 (t), . . . , Iλ

J (t))⇒

(XΣ(t), X1(t), . . . , XI(t), p1([XΣ(t)]+), . . . , pI([XΣ(t)]+),

1a1

p1([XΣ(t)]+), . . . ,1aI

pI([XΣ(t)]+),

v1([XΣ(t)]−), . . . , vJ([XΣ(t)]−))


in D3I+J+1− as λ → ∞, where (X1(t), . . . , XI(t)) is the unique (possibly weak) solution of the following I-

dimensional SDE:

Xi(t) = Xi(0)− βit−∑

j∈J(i)

µi,j

∫ t

0

Gij

(X(s)− p([XΣ(s)]+),−v([XΣ(s)]−)

)ds

−θi

∫ t

0


2aiBi(t), i ∈ I, (10)

where the processes Bi(t), t ≥ 0, i ∈ I are independent standard Brownian motions.

5. QIR and Atar’s Control It turns out that for certain tree networks, under an additional restriction on theratio vector v(·), QIR is equivalent to a special case of the control rule introduced in Atar [4]. Consequently,SSC for these networks follows from the analysis in [4]. To state the condition, fix an optimal solution x for (2)for which the edges in E(x) induce a connected routing graph. Then, we will consider networks that satisfy, inaddition, the following condition:

C- 4 only one pool with cross-trained agents: There exists at most one j ∈ J with skill set I(j)(x) containingmore than one element; denote this pool by j∗. Also, we require that v = ej∗ .

Networks that satisfy this structural condition are depicted in Figure 3. Clearly, these networks are trees, sothat they also satisfy Condition 3. We accomplish two objectives in this subsection: First, we show that underCondition 4, QIR and an instance of the control constructed in [4] are equivalent; that is proved in Lemma 5.1.Second, we show that, although an SSC result does not appear explicitly in [4], the controls constructed in [4] leadto SSC for tree networks (under Condition 3). That appears in Theorem 5.1, whose proof follows from the analysisin [4]; see §8.

Together, these two results provide an alternative proof of SSC for QIR with the networks satisfying C-4. Aspecial case of this equivalence–for the N-model (see Figure 3) with constant ratio functions–was argued and usedin [10] in the context of linear-holding-cost minimization.

We start, then, by defining the control used in [4] and stating and proving the SSC result based on his analysis.We refer to the control in [4] as Generalized QIR (GQIR), even though QIR is not a special case of GQIR. Indeed,GQIR applies only to settings in which the chosen optimal solution x is such that the induced graph E(x) is a tree.It is not applicable when E(x) contains cycles.

Towards the construction of GQIR, define a function G := G(α, β) : RI+J 7→ RIJ , which is defined as theunique solution to the following set of linear equations (see equation (43) in [4]):∑

j∈Jzi,j = αi,

∑

i∈Izi,j = βj , (11)

with no additional constraints. Atar [4] shows that the tree structure guarantees a unique solution. This uniquenesscan be then used to show that the mapping G(·, ·) defines a linear mapping on the domain

DG := (α, β) ∈ RI+J :∑

i

αi =∑

j

βj.

Moreover, the mapping G is such that

Zλij(t) := Gij(Xλ(t)− Qλ(t),−Iλ(t)), (12)

that is, given the vector-valued processes Xλ(t), Qλ(t) and Iλ(t) the values of Zλ(t) can be calculated using thismapping. We define a new stochastic processes Zλ(t) in terms of the triple (Xλ(t), Qλ(t), Iλ(t)) through

Zλij(t) := Gij

(Xλ(t)− [Xλ

Σ(t)]+p([Xλ

Σ(t)]+)

,−[XλΣ(t)]−v

([Xλ

Σ(t)]−))

, i ∈ I, j ∈ J . (13)

We are now ready to introduce the GQIR control, as it was constructed in [4]; see §§2.5 and 2.6 of [4].

Definition 5.1 (Generalized QIR: GQIR) Suppose that the routing graph is a tree.

• Upon an arrival of a class-i customer at time t, if there are any idle agents, route the customer to anyagent pool

j := j(t) ∈ argmaxk∈J(i),Iλ

k >0

(Zλik(t)− Zλ

ik(t))+ ;

if all agents in J(i) are busy, then the customer waits in queue, to be served in order of arrival.


• Upon a service completion by an agent from agent pool j at time t, if a customer in I(j) is available,then admit to service the customer from the head of any non-empty queue

i := i(t) ∈ argmaxk∈I(j),Qλ

k(t)>0

(Zλkj(t)− Zλ

kj(t))+ ;

if all queues in I(j) are empty, then the agent remains idle.

Ties are broken in an arbitrary but consistent manner, so that the vector-valued stochastic process (Qλ, Zλ) is aCTMC with stationary transition probabilities.

The following lemma holds for all λ; hence we fix λ and omit it from the notation. Also, we add a superscriptQIR (or GQIR) to all relevant processes to indicate explicitly the dependence on the control.

Lemma 5.1 (equivalence of QIR and GQIR under Condition C-4) Fix an optimal solution x for (2) forwhich the edges in E(x) induce a connected routing graph and assume that Condition C-4 holds. Assume that(XGQIR(0), ZGQIR(0)) = (XQIR(0), ZQIR(0)) and that the same rule is used to break ties under QIR and

GQIR. Then, (XGQIR, ZGQIR), t ≥ 0 d= (XQIR(t), ZQIR(t)), t ≥ 0.

The following is the SSC result for GQIR for general tree networks (going beyond the domain of equivalence).It is a consequence of parts (i) and (iv) in Proposition 1 of Atar [4].

Theorem 5.1 (SSC under GQIR) Fix an optimal solution x for (2) for which the edges in E(x) induce a con-nected routing graph. Fix two admissible state-dependent ratio functions p and v and an optimal solution xfor (2). Suppose that GQIR is used as defined in Definition 5.1, and that condition C-3 holds. If, in addition(Xλ(0), Zλ(0)) ⇒ (X(0), Z(0)) in RI+I·J , then all the conclusions of Theorem 3.1 hold, including (5).

6. Proof of Theorem 3.1 The rest of this paper is devoted to proving the previous results. In this sectionwe prove Theorem 3.1. The proof decomposes into three separate proofs, one for each of the three conditions.Subsections 6.2, 6.3 and 6.4 are dedicated to conditions C-1, C-2 and C-3, respectively. However, we start withpreliminaries.

6.1 Preliminaries Our approach to the construction of the underlying stochastic processes follows a martin-gale approach that is, by now, quite common; see Pang et al. [25] for an overview.

Sample-path construction. We begin with a sample-path construction that is based on independent unit-ratePoisson processes Ai, Si,j and Ri on R+ for i ∈ I and j ∈ J . Given these Poisson processes, let

Xλi (t) := Xλ

i (0) + Ai(λit)−∑

j∈JSi,j

(µi,j

∫ t

0

Zλi,j(s)ds

)−Ri

(θi

∫ t

0

Qλi (s)ds

), t ≥ 0 . (14)

By direct construction, the stochastic process (Xλ1 (t), . . . , Xλ

I (t)) : t ≥ 0 has the correct distribution and isa process in DI . (A formal argument would follow the proof of Lemma 2.1 in [25].) Equation (14) does notyet fully define the system dynamics because the dynamics of the queue-length processes Qλ

i (t) and the busy-agent processes Zλ

i,j(t) are not yet specified. To complete the definition, let Aλij(t), (i, j) ∈ I × J , be the

cumulative number of class-i customers routed to an idle type-j agent immediately upon arrival by time t; letΦλ

i,j(t), (i, j) ∈ I × J , be the cumulative number of class-i customers assigned to a type-j agent after waiting inqueue, by time t. Then, we write

Zλi,j(t) = Zλ

i,j(0) + Aλi,j(t) + Φλ

i,j(t)− Si,j

(µi,j

∫ t

0

Zλi,j(s)ds

),

Qλi (t) = Qλ

i (0) + Aλi (t)−

∑

j∈JAλ

i,j(t)−∑

j∈JΦλ

i,j(t)−Ri

(θi

∫ t

0

Qλi (s)ds

),

Iλj (t) = Nλ

j −∑

i∈IZλ

i,j(t). (15)


We let Dλj (t) :=

∑k∈I Skj

(µk,j

∫ t

0Zλ

k,j(s)ds)

be the cumulative number of service completions by type-jagents. The general description of the queueing network above does not yet reflect the specifics of QIR (seeDefinition 2.3). To include them, let the scaled queue and idleness imbalances be

Uλi (t) := Qλ

i (t)− [XλΣ(t)]+pi([Xλ

Σ(t)]+), i ∈ I,

V λj (t) := Iλ

j (t)− [XλΣ(t)]−vj([Xλ

Σ(t)]−), j ∈ J . (16)

Establishing SSC means showing that all of these processes converge weakly to 0 in the appropriate sense. Nowwe can write

Aλi,j(t) =

∫ t

0

1

Iλj (s−) > 0, j ∈ argmax

j∈J(i)

V λj (s−)

dAλ

i (s),

Φλi,j(t) =

∫ t

0

1

Qλ

i (s−) > 0, i ∈ argmaxi∈I(j)

Uλi (s−)

dDλ

j (s). (17)

These processes are uniquely defined since we have assumed that ties are broken in a consistent manner.

Martingales. We now develop a martingale representation. We start by defining the σ-algebras

Fλ(t) := σAi(λis), Xλ

i (s), Qλi (s), Zλ

i,j(s), Ri

(θi

∫ s

0Qλ

i (u)du),

Si,j

(µi,j

∫ s

0Zλ

i,j(u)du)

: i ∈ I, j ∈ J , s ≤ t

, t ≥ 0 ,(18)

and make them complete by including all the null sets. The collection of all these σ-algebras Fλ := Fλ(t), t ≥ 0is then the filtration.

We now exploit a martingale decomposition, as in [25]. First, we assume that E[QλΣ(0)] < ∞ for all λ,

but that is without loss of generality, because it can be later relaxed, as in §6.3 of [25]. A straightforward adap-tation of Lemmas 3.2 and 3.4 in [25] to our setting establishes that the processes Ai(λit), Ri

(θi

∫ t

0Qλ

i (s)ds)

and Si,j

(µi,j

∫ t

0Zλ

i,j(s)ds)

admit martingale decompositions with respect to the filtration Fλ. Specifically, thestochastic processes

Mλi,j(t) := Si,j

(µi,j

∫ t

0

Zλi,j(s)ds

)− µi,j

∫ t

0

Zλi,j(s)ds, i ∈ I, j ∈ J ,

MλAi

(t) := Ai(λit)− λit, i ∈ I,

MλRi

(t) := Ri

(θi

∫ t

0

Qλi (s)ds

)− θi

∫ t

0

Qλi (s)ds, i ∈ I, (19)

are square-integrable martingales with predictable quadratic variations defined by

〈Mλi,j〉(t) := µi,j

∫ t

0

Zλi,j(s)ds, i ∈ I, j ∈ J ,

〈MλAi〉(t) := λit, i ∈ I,

〈MλRi〉(t) := θi

∫ t

0

Qλi (s)ds, i ∈ I. (20)

Note that MλRi

(t) = 0, t ≥ 0 for all i with θi = 0.

As integrals of predictable processes with respect to counting processes, the processes Aλi,j(t) and Φλ

i,j(t) alsohave corresponding martingale decompositions for all i and j, e.g.,,

MλAi,j

(t) :=∫ t

0

1


j∈J(i)

V λj (s−)

d

(Aλ

i (s)− λis)

(21)

is a square-integrable martingale with predictable quadratic variation

〈MλAi,j

〉(t) =∫ t

0

1


j∈J(i)

V λj (s−)

λids. (22)


This is a consequence of the preservation of the martingale property under stochastic integration. Specifically,fixing any T > 0, the martingale Mλ

Ai(t ∧ T ) = Ai(λi(t ∧ T ))− λi(t ∧ T ) is square integrable and in particular

it is in the space H2 defined in p 124 of [26]. Since the integrand in (21) is left continuous (because it is definedthrough the left limits), it is predictable. Since the integrand is also bounded, we can apply Theorem IV.2.11 (p.129) of [26] to ensure that the stochastic integral in (21) is itself a square integrable martingale. Finally, item (ii) ofLemma 5.77 (p. 85) in [13] guarantees that the predictable quadratic variation process of (21) is as given in (22).A similar argument is used for the process Φλ

i,j(t).

Instead of giving detailed expressions for the system dynamics in terms of these martingales here, we will givethe required expressions where needed.

Stochastic boundedness and tightness. Our proofs will make extensive use of the concepts of tightness andstochastic boundedness; again see [25] for an overview. In particular, we will be using these notions for stochasticprocesses in Dd := D[0,∞)d. We say that a sequence of stochastic processes Y λ : Y λ(t) : t ≥ 0 in Dd isstochastically bounded if, for all T > 0,

limA→∞

lim supλ→∞

P‖Y λ‖∗T > A = 0,

where the norm ‖ · ‖∗T is as defined in §2. That is a minor modification of the definition for random variables.

We make especial use of C-tightness. A sequence of stochastic processes Y λ := Y λ(t) : t ≥ 0 in Dd

is said to be C-tight if, in addition to being tight (as random elements of Dd), every convergent subsequenceconverges to a limit that is a.s. continuous. Theorem 15.5 from [5] is very useful in establishing C-tightness. Werestate it here:

Theorem 6.1 (C-tightness) Consider a sequence of stochastic processes Y λ in Dd. Suppose that Y λ(0) isstochastically bounded in R and, for each ε > 0 and T > 0,

limδ→0

lim supλ→∞

PwY λ(δ, T ) ≥ ε = 0 where wx(δ, T ) := sup0≤s<t≤T :|t−s|≤δ

|x(t)− x(s)| .

Then Y λ is C-tight.

We will want to establish stochastic boundedness and C-tightness for various martingale processes. We use thegeneral notation Mλ(t), or Mλ(t) when referring to the scaled version Mλ(t)/

√λ using the scaling in Definition

2.1, for martingale components and refer to specific attributes of the martingale in consideration only where this isneeded.

Here are some important general properties:

Lemma 6.1 (properties of the martingales) Let Mλ(t) be any of the scaled martingales introduced above (or

a finite sum of such), using the scaling in Definition 2.1, and assume that Qλ(0)/λP→ 0. Then, the sequence of

processes 〈Mλ〉(t) is C-tight. Also, the sequence of processes Mλ(t) is stochastically bounded. Finally, forany ε > 0,

lim supλ→∞

P

sup

0≤t≤T/√

λ

|Mλ(t)| > ε

= 0. (23)

Proof: We begin with the C-Tightness. Let 〈Mλ〉(t) be the scaled version of any of the predictable-quadratic-variation processes defined in (20). Then,

|〈Mλ〉(t)− 〈Mλ〉(s)| ≤ c(t− s)1λ

max

λ,

∑

j∈JNλ

j ,

∫ t

s

QλΣ(u)du

, (24)

for some positive constant c. By Assumption 2.1,∑

j∈J Nλj ≤ c1λ for all λ large enough and for some constant

c1. Using the trivial inequality QλΣ(u) ≤ Qλ

Σ(0) +∑

i∈I Aλi (u), the assumed convergence Qλ(0)/λ

P→ 0, and therenewal strong law of large numbers we also have that

lim supλ→∞

P

∫ t

sQλ

Σ(u)du

λ> c2(t− s)

= 0,


for some constant c2 large enough. Plugging this back into (24), we conclude that

limδ→0

lim supλ→∞

P

sup

0≤s<t≤T :|t−s|≤δ

|〈Mλ〉(t)− 〈Mλ〉(s)| > ε

= 0,

and, consequently, that 〈Mλ〉(t) is C-Tight.

The last two conclusions follow from Lemma 5.8 of [25], which is based on the Lenglart-Rebolledo inequality,stated as Lemma 5.7 there. The extension to finite sums follows from basic properties of the quadratic variationprocesses (see Problem 1.5.7 in Karatzas and Shreve [22] and using the inequality 2〈M1,M2〉 ≤ (〈M1〉+ 〈M1〉)(see Problem 1.8.9 in Lipster and Shirayev [23]).

6.2 SSC Under C-1 For this case, it suffices to consider a less detailed construction of the system dynamicsthan the one in §6.1. Specifically, we keep Q and I in (15), but instead of Z, we write

Zλj (t) = Zλ

j (0) +∑

i∈IAλ

i,j(t) +∑

i∈IΦλ

i,j(t)− Sj

(µj

∫ t

0

Zλj (s)ds

), (25)

where Zλj (t) :=

∑i∈I Zλ

i,j(t) is the number of busy agents in pool j and Sj(·) is a unit-rate Poisson process, sothat the number of service completions in agent pool j is now given by

Dλj (t) = Sj

(µj

∫ t

0

Zλj (s)ds

)(26)

Also, instead of the martingales Mλi,j(t) in (19), we use the martingales

Mλj (t) := Sj

(µj

∫ t

0

Zλj (s)ds

)− µj

∫ t

0

Zλj (s)ds, j ∈ J .

Finally, instead of Xi in (14), we use

XλΣ(t) = Xλ

Σ(0) +∑

i∈IAi(λit)−

∑

j∈JSj

(µj

∫ t

0

Zλj (s)ds

)−

∑

i∈IRi

(θi

∫ t

0

Qλi (s)ds

). (27)

Following §6.1 with respect to the martingale decomposition and applying some algebraic manipulations we write

XλΣ(t) = Xλ

Σ(0) + (λ−∑

j∈JµjN

λj )t +

∑

j∈Jµj

∫ t

0

Iλj (s)ds−

∑

i∈Iθi

∫ t

0

Qλi (s)ds + Mλ

Σ(t),

MλΣ(t) :=

∑

i∈IMλ

Ai(t)−

∑

j∈JMλ

j (t)−∑

i∈IMλ

Ri(t).

By Assumption 2.1,∑

j∈Jµjγj = lim

λ→∞

∑j∈J µjN

λj − λ√

λ,

and we define β :=∑

j∈J µjγj . Hence, we may write

XλΣ(t) = Xλ

Σ(0)− βt +∑

j∈Jµj

∫ t

0

Iλj (s)ds−

∑

i∈Iθi

∫ t

0

Qλi (s)ds + Mλ

Σ(t) + oP (1). (28)

The oP (1) term will play no role in our subsequent analysis and we will omit it.

Having specified the system dynamics, it would be natural to apply Dai and Tezcan [12] to prove the SSC result.Unfortunately, we can not apply [12] directly. Indeed, a key assumption in [12] is the homogeneity conditionin equation (5.9) there or, in its weaker version, in equation (7.1) there. When interpreted to our setting, thehomogeneity condition requires that there exist c1, c2 > 0 and 0 ≤ α ≤ 1 such that

αc1fi(x) ≤ fi(αx) ≤ αc2fi(x) ,

for fi(x) : R+ → R given by fi(x) = x − xpi(x) where pi(·) is a ratio function. A similar requirement appliesto the idleness imbalances. Since our ratio functions need not satisfy those assumptions, we cannot apply [12].In §7 of [12], it is shown that the homogeneity requirement can be removed if one can, in advance, establish the


stochastic boundeness of the scaled dynamics processes. That stochastic boundeness can then be used to proveSSC even when the homogeneity does not hold. In our setting, however, the stochastic boundeness and SSC areinterwined and it seems impossible to establish the stochastic boundeness in advance, before proving the SSC.

To overcome this difficulty, we devise a stopping argument. Specifically, let

Bλ(t) :=∑

i∈IUλ

i (t), (29)

with Uλi (t) as defined in (16). We first establish SSC assuming that all the processes are stopped at the bounded

stopping timeTλ := ςλ ∧ T where ςλ := inft ≥ 0|Bλ(t) ≥ 2Bλ(0) ∨ 1. (30)

Since all the processes involved are assumed to be right continuous, the stopped processes are well defined (Propo-sitions 1.1.13 and 1.2.18 of [22]). Note that the value of Bλ(Tλ) can be greater than 2Bλ(0) ∨ 1, because therecan be a jump at time Tλ. Still, since all arrival and service-completion processes have jumps of size 1, there existsa constant K such that

Bλ(Tλ) ≤ 2Bλ(0) ∨ 1 + K/√

λ , w.p. 1. (31)

The idea of the stopping argument is that, while it is hard to characterize the limits on the interval [0, T ] directly,it is easier to establish these limits for the stopped processes. Once these limits are established, showing thatςλ P→∞will imply that the same limiting behavior holds on [0, T ]. For the stopped processes, our proof consists oftwo main modules. First, Lemma 6.2 establishes that the sequence of stopped processes is stochastically boundedand that the sequence of processes Xλ

Σ(t ∧ Tλ) is C-tight. Then, Theorem 6.2 establishes SSC for the stoppedprocesses using the results of Proposition 6.2 and the Bramson [6] and Dai and Tezcan [12] SSC frameworks.Finally, the SSC result is extended to the whole interval [0, T ] in Lemma 6.6. Throughout it is assumed that theconditions of Theorem 3.1 hold in addition to condition C-1.

Lemma 6.2 (stochastic boundedness of scaled queueing processes) The sequences QλΣ(t∧Tλ), Iλ

Σ(t∧Tλ) andXλ

Σ(t ∧ Tλ) are stochastically bounded. In addition, the sequence XλΣ(t ∧ Tλ) : λ > 0 is C-tight.

Proof: By the definition of Uλi (t) in (16) and Definition 2.2 for state-dependent ratio functions,

Bλ(t) =∑

i∈IUλ

i (t) = QλΣ(t)− [Xλ

Σ(t)]+, (32)

where the actual state-dependent ratio functions drop out when we sum over i ∈ I . In particular,

QλΣ(t) = [Xλ

Σ(t)]+ + Bλ(t) ≤ |XλΣ(t)|+ Bλ(t). (33)

By Definition 2.1, XλΣ(t) = (Xλ

Σ(t) − NλΣ)/

√λ = (Qλ

Σ +∑

j∈J Zλj (t) − Nλ

Σ)/√

λ, so that IλΣ = Qλ

Σ − XλΣ.

Consequently,IλΣ(t) ≤ |Xλ

Σ(t)|+ QλΣ(t) ≤ 2|Xλ

Σ(t)|+ Bλ(t). (34)

Plugging these into equation (28) and applying Gronwall’s inequality (see Theorem 4.1 and Lemmas 4.1 and 5.6in [25] or Problem 5.2.7 in [22]), we have

‖XλΣ(t ∧ Tλ)‖∗T ≤ c1

(|Xλ

Σ(0)|+ |β|T + ‖Bλ(· ∧ Tλ)‖∗T + ‖MλΣ(· ∧ Tλ)‖∗T

)ec2T ,

for some positive constants c1 and c2. By Lemma 6.1, MλΣ(t) is stochastically bounded and, by the definition

of Tλ and equation (31), Bλ(t ∧ Tλ) is stochastically bounded. Here we also use the fact that Bλ(0) is itselfstochastically bounded by the assumed convergence of Xλ(0).

Finally, the sequence Xλ(0) is stochastically bounded because it converges; see Corollary 5.2 in [25]. Thestochastic boundedness of Xλ

Σ(t ∧ Tλ) now follows from it being bounded by a sum of stochastically boundedsequences; see Lemma 5.5 in [25]. Finally, Iλ

Σ(t∧Tλ) and QλΣ(t∧Tλ) are now stochastically bounded by applying

(33) and (34). The result of the proposition now follows as the sum of stochastically bounded sequences is itselfstochastically bounded.

It remains to establish the claimed C-tightness of Xλ(t ∧ Tλ). Using (28) once more, we have

|XλΣ(t)− Xλ

Σ(s)| ≤ |β|(t− s) +∑

j∈Jµj

∫ t

s

Iλj (s)ds +

∑

i∈Iθi

∫ t

s

Qλi (s)ds + |Mλ

Σ(t)− MλΣ(s)|,


and, as a consequence,

|XλΣ(t)− Xλ

Σ(s)| ≤ c3

(|β|(t− s) + (t− s)‖Iλ

j (· ∧ Tλ)‖∗T + (t− s)‖QλΣ(· ∧ Tλ)‖∗T + |Mλ

Σ(t)− MλΣ(s)|

),

For all 0 ≤ s < t ≤ Tλ. Using (33), (34) and Gronwall’s inequality, we obtain

|XλΣ(t)− Xλ

Σ(s)| ≤ c3

(|β|(t− s) + |Mλ

Σ(t)− MλΣ(s)|+ (t− s)‖Bλ(· ∧ Tλ)‖∗T

)ec4T ,

for 0 ≤ s < t ≤ Tλ and for some positive constants c3 and c4. With the definitions in Theorem 6.1,

wXλΣ(·∧T λ)(δ, T ) ≤ c5δ + c6δ‖Bλ(· ∧ Tλ)‖∗T + c7wMλ

Σ(·∧T λ)(δ, T ), (35)

for positive constants c5, c6 and c7. Since Bλ(t ∧ Tλ) is stochastically bounded by the definition of Tλ, the C-tightness of Xλ(t∧Tλ) is established if we prove the tightness of Mλ

Σ(t∧Tλ). This, however, follows immediatelyfrom Theorem 5.6 in [25] which allows us to deduce the C-tightness of the sequence of martingales Mλ

Σ(t ∧ Tλ)from the C-tightness of the sequence of predictable-quadratic-variation processes, 〈Mλ

Σ〉(· ∧ Tλ) which, in turn,follows from Lemma 6.1. Finally, Xλ

Σ(t ∧ Tλ) is C-tight by (35). ¥

We are now ready to prove SSC for the stopped processes. With the definition of the processes Uλi (t) and V λ

j (t)in (16), SSC for the stopped processes is equivalent to the following theorem, as we show in the corollary below:

Theorem 6.2 (SSC for the stopped-processes) For any ε > 0 and s, 0 < s < T ,

lim supλ→∞

P

sup

s≤t≤T λ

∑

i∈I|Uλ

i (t)| > ε

= 0, and lim sup

λ→∞P

sup

s≤t≤T λ

∑

j∈J|V λ

j (t)| > ε

= 0. (36)

If, in addition, equation (4) holds then

lim supλ→∞

P

sup

0≤t≤T λ

∑

i∈I|Uλ

i (t)| > ε

= 0, and lim sup

λ→∞P

sup

0≤t≤T λ

∑

j∈J|V λ

j (t)| > ε

= 0 (37)

The following simple Corollary shows that Theorem 6.2 implies the desired form of SSC.

Corollary 6.3 Theorem 6.2 implies that

Qλi (t ∧ Tλ)− Qλ

Σ(t ∧ Tλ)pi

(Qλ

Σ(t ∧ Tλ))

⇒ 0 as λ →∞ for all i ∈ I,

Iλj (t ∧ Tλ)− Iλ

Σ(t ∧ Tλ)vj

(IλΣ(t ∧ Tλ)

)⇒ 0 as λ →∞ for all j ∈ J , (38)

where the convergence is in D or D− depending, as before, on whether or not equation (4) holds.

Proof: We only prove the result for the queue processes; because the proof for the idleness processes is similar.Given (16), Theorem 6.2 implies that Qλ

Σ(t∧Tλ)− [XλΣ(t∧Tλ)]+ ⇒ 0, where the convergence is in D if equation

(4) holds, and its D− otherwise. In turn, using the continuity of the state-dependent ratio functions and applyingthe continuous mapping theorem (see §3.4 of Whitt [31]), we have that

pi

(Qλ

Σ(t ∧ Tλ))− pi

([Xλ

Σ(t ∧ Tλ)]+)⇒ 0, as λ →∞.

Consequently,Uλ

i (t ∧ Tλ)−(Qλ

i (t ∧ Tλ)− QλΣ(t ∧ Tλ)pi

(Qλ

Σ(t ∧ Tλ)))

⇒ 0,

implying finally thatQλ

i (t ∧ Tλ)− QλΣ(t ∧ Tλ)pi

(Qλ

Σ(t ∧ Tλ))⇒ 0,

where the convergence is in D or D− depending, as before, on whether or not equation (4) holds. ¥In general, then, it suffices to consider the processes Uλ

i (t), i ∈ I, and V λj (t), j ∈ J , in order to establish

the result of Theorem 3.1. We will prove all the results only for the processes Uλi (t), i ∈ I, as the results for

V λj (t), j ∈ J , follow similarly.


In preparation for the proof of Theorem 6.2, we first introduce some of the required framework. Our SSC proofwill be based on the general SSC framework of Bramson [6], which was recently extended to the many-serversetting by Dai and Tezcan [12]. Since some of the assumptions in [12] do not apply in our setting, we give anindependent proof.

The framework of [6] is based on one key idea: By using an appropriate new scaling of time and space, called thehydrodynamic scaling, we can examine the system dynamics over short time intervals. These hydrodynamically-scaled (HS) processes are shown to be uniformly approximated by a family of deterministic functions, calledthe hydrodynamic-limit (HL) functions, that satisfy certain equations, known as the hydrodynamic model (HM)equations. This uniform approximation guarantees that, whenever the HL function exhibits the desired SSC, sowill the HS processes, as well as the original scaled process, through the appropriate mapping between the originalprocess and its HS version. See Bramson [6] for a more detailed introduction to these concepts.

We begin by defining the HS processes and the HM equations corresponding to our system. To simplify thenotation, let

Rλi (t) := Ri

(θi

∫ t

0

Qλi (s)ds

)and Θλ

i (t) :=√

λ[XλΣ(t)]+pi

([Xλ

Σ(t)]+)

, i ∈ I.

We will also use Dλj (t), which was defined in (26). We start with the basic processes indexed by λ:

Xλ(t) :=(Aλ

i (t), Aλi,j(t), Φ

λi,j(t), D

λj (t), Rλ

i (t), Θλi (t), Qλ

i (t), Uλi (t), Zλ

j (t); i ∈ I, j ∈ J ). (39)

Then, for any fixed L > 0, and every non-negative integer m with m <√

λT , we construct the hydrodynamically-scaled (HS) processes

Xλ,m :=(Aλ,m

i (t), Aλ,mi,j (t), Φλ,m

i,j (t), Dλ,mj (t), Rλ,m

i (t), Θλ,mi (t), Qλ,m

i (t), Uλ,mi (t), Zλ,m

j (t); i ∈ I, j ∈ J)

,

for t ∈ [0, L] as follows:

Aλ,mi (t) :=

1√λ

(Aλ

i

(t√λ

+m√λ

)−Aλ

i

(m√λ

)),

Aλ,mi,j (t) :=

1√λ

(Aλ

i,j

(t√λ

+m√λ

)−Aλ

i,j

(m√λ

)),

Φλ,mi,j (t) :=

1√λ

(Φλ

i<j

(t√λ

+m√λ

)− Φλ

i,j

(m√λ

)),

Dλ,mj (t) :=

1√λ

(Dλ

j

(t√λ

+m√λ

)−Dλ

j

(m√λ

)),

Rλ,mi (t) :=

1√λ

(Rλ

i

(t√λ

+m√λ

)−Rλ

i

(m√λ

)),

Θλ,mi (t) :=

1√λ

(Θλ

i

(t√λ

+m√λ

)−Θλ

i

(m√λ

)),

Qλ,mi (t) :=

1√λ

(Qλ

i

(t√λ

+m√λ

)),

Uλ,mi (t) :=

(Uλ

i

(t√λ

+m√λ

)), and

Zλ,mj (t) :=

1√λ

(Zλ

(t√λ

+m√λ

)−Nλ

j

). (40)

Remark 6.1 (the time and space scaling) The HS processes in equation (40) have a more elementary time-and-space scaling than in [6] and [12] - see §6.1 of [12] and, in particular, equation (6.1) there. The more complexscaling is required if we can not prove a priori that the queue and idleness processes are stochastically bounded. Inthe absence of stochastic boundedness, the resulting notion of SSC is the multiplicative SSC, defined in Theorem5.1 and Remark 5.3 of [12]. However, when stochastic boundedness is available, SSC and multiplicative SSC areequivalent. Then both can be proved using the more elementary scaling, as illustrated in the proof of Theorem7.3 in [12]. Since our SSC argument will focus initially on the stopped processes, the stochastic boundednessestablished in Lemma 6.2 allows us to use the more elementary time-and-space scaling.


We will show that the HS processes are uniformly approximated by Lipschitz, and thus absolutely continuous,deterministic functions called the hydrodynamic-limit (HL) functions, denoted by

X :=(Ai(t), Aij(t), Φij(t), Dj(t), Ri(t), Θi(t), Qi(t), Ui(t), Zj(t); i ∈ I, j ∈ J

),

satisfying the following hydrodynamic model (HM) equations. However, the following HM equations, in general,do not determine the HL functions uniquely. Thus we will be showing that there is some HL function satisfy-ing the HM equations that is suitably close to the HS process above. Since we are not aiming for uniqueness,the hydrodynamic model might contain additional equations, but we specify only those that are relevant for ourpurposes.

The hydrodynamic model (HM) equations are:

Ai(t) = ait, i ∈ I,

Θi(t) = Ri(t) = 0, for all t ≥ 0,

Qi(t) = Qi(0) + Ai(t)−∑

j∈JAij(t)−

∑

j∈JΦij(t), i ∈ I,

Qi(t) ≥ 0, i ∈ I,

Aij(t), Φij(t), i ∈ I, j ∈ J are non-decreasing,

Zj(t) = Zj(0) +∑

i∈IAij(t) +

∑

i∈IΦij(t)− µj νjt, j ∈ J . (41)

The HM equations in (41) appear in the hydrodynamic model of an arbitrary policy. Letting I+(t) := i ∈ I :Ui(t) > 0, and J(I+(t)) := j ∈ J : i ∈ J(i), for some i ∈ I+(t), we also define the following HM equationsthat are specific to QIR

∑

i∈IUi(t) ≥ 0, and Ui(t) = Ui(0) + Ai(t)−

∑

j∈JAij(t)−

∑

j∈JΦij(t), i ∈ I, (42)

∑

i∈I+(t)

∑

j∈J(i)

dΦij(t) =∑

j∈J(I+(t))

µj νj , and in particular, (43)

∑

i∈I+(t)

dUi(t) ≤ −c, whenever I+(t) 6= ∅ for some constant c > 0. (44)

We have stipulated that these HL functions are Lipschitz. The relevant set of HL functions will depend onpositive real parameters k, Lk and N . Specifically, for appropriate parameters, the set of HL functions will bea subset of the family E′ of functions in Dd that satisfy |x(0)| ≤ k and |x(t2) − x(t1)| ≤ N |t2 − t1| for allt1, t2 ∈ [0, Lk]. By the Arzela-Ascoli theorem, p. 221 of [5], the set E′ is a compact subset of Cd and thus of Dd

for appropriate d. Since the HL functions are Lipschitz, they are absolutely continuous; e.g., see §5.4 of [28].

In addition, the HL functions must satisfy the HM equations above. The existence of HL functions satisfyingthose HM equations will be a consequence of our analysis and, in particular, of Lemmas 6.4 and 6.5 below. Notethat the final HM equation, equation (44), implies that there is a constant upper bound c on the rate at which∑

i∈I [Ui(t)]+ decreases whenever it is positive. Consequently, it reaches 0 within a finite time, after which itstays at 0, by the first inequality in (42); see Lemma 5.2 of Dai [9] for technical support. We will use this fact inthe proof of Theorem 6.2.

First, however, we identify the relations between the HS processes Xλ,m and the HL functions X . This is donein the following theorem, which shows that, for λ large enough, the HS process Xλ,m is close enough to some HLfunction X , satisfying the HM equations (41)-(44).

Theorem 6.4 (uniform approximation by HL functions) For any T > 0, δ > 0 and ε > 0, there exist k :=k(δ, ε, T ), Lk, λ0 and subsets Kλ,k of the underlying probability space Ω, such that for all λ ≥ λ0:

(i) ‖Uλ(· ∧ Tλ)‖∗T + ‖Zλ(· ∧ Tλ)‖∗T + ‖Qλ(· ∧ Tλ)‖∗T ≤ k on Kλ,k,

(ii) for each ω ∈ Kλ,k and m with m <√

λTλ, there exists an HL function (a Lipschitz function satisfyingthe HM equations) X , depending on λ and m, such that ‖Xλ,m − X‖∗Lk

≤ ε.

(iii) Finally, PKλ,k ≥ 1− δ.


Theorem 6.4 captures the essence of the hydrodynamic-limit approach: The idea is to show that, on a suitablylarge subset of the sample space, and for all λ sufficiently large, the HS process is close enough to some HLfunction satisfying the HM equations. We postpone the proof of Theorem 6.4 and apply it now to prove Theorem6.2.

Proof of Theorem 6.2: The proof of the first conclusion consists of two steps. First, we focus on an HL function,fix ε and k and show that there exists some finite time s∗ := s∗(k, ε) such that Ui(t) = 0 for t ≥ s∗ and for all iprovided that the HL function X satisfies the HM equations (41)-(44) and

∑i∈I |Ui(0)| ≤ k + ε.

Property (44) implies that the HL function∑

i∈I [Ui(t)]+ decreases at a rate of at least c until it reaches 0,after which it stays there; see Lemma 5.2 of Dai [9] for technical support. Property (44) directly controls only thepositive part, but condition (42) implies that the negative part is dominated by the positive part in absolute value.Hence, when

∑i∈I [Ui(t)]+ = 0, we also have Ui(t) = 0 for all i by virtue of condition (42). And these functions

must remain 0 thereafter, because the positive part cannot increase.

Thus we have the existence of s∗ := s∗(k, ε) such that Ui(t) = 0 for t ≥ s∗ for all i for any HL function Xsatisfying the HM equations (41)-(44) with

∑i∈I |Ui(0)| ≤ k + ε. We now come to the second step of the proof

of the first conclusion, in which we find an HL function appropriately related to the HS process: Fix δ > 0, k > 0and λ0 so that for all λ ≥ λ0, PKλ,k ≥ 1− δ. Also, choose Lk ≥ 2ds∗e, and finally, fix ω ∈ Kλ,k. Consider atime t ≤ Tλ with t ≥ s∗/

√λ (if such time exits), and let

mλ(t) := maxm : m <√

λTλ, (m + s∗)/√

λ ≤ t.Then t ∈

[mλ(t)/

√λ, (mλ(t) + Lk)/

√λ], and, by definition of the HS process, Uλ

i (t) = Uλ,mλ(t)i (

√λt −

mλ(t)). Now, by Theorem 6.4, for λ large enough, there exists and HL function X that satisfies the HM equations

(41)-(44) with |Uλ,mλ(t)i − Ui|∗Lk

≤ ε. In particular, we have∑

i∈I |Ui(0)| ≤ k + ε, which by our previousargument implies that

∑i∈I |Ui(t)| = 0, t ≥ s∗. Since Lk ≥

√λt−mλ(t) ≥ s∗,

∑

i∈I

∣∣∣Ui

(√λt−mλ(t)

)∣∣∣ = 0 on Kλ,k.

Combining these relations, we then have∑

i∈I |Uλi (t)| ≤ ε. Hence,

sups∗/

√λ≤t≤T λ

∑

i∈I|Uλ

i (t)| ≤ ε on Kλ,k, (45)

where we naturally set the value to be 0 whenever Tλ < s∗/√

λ. Since the same holds for any ω ∈ Kλ,k, for allλ ≥ λ0,

P

sup

s∗/√

λ≤t≤T λ

∑

i∈I|Uλ

i (t)| > ε

≤ δ,

from which (36) readily follows.

For the second conclusion, assume that (4) holds and let

Ωλ =

w ∈ Ω :

∑

i∈I|Uλ

i (0)| ≤ ε

.

Then, (4) implies that PΩλ → 1 as λ →∞. Consider the process Uλ,0i (t) and its corresponding approximation

Ui from Theorem 6.4. Then, on Kλ,k⋂

Ωλ, we must have∑

i∈I |Ui(0)| ≤ ε, and repeating the same argumentwe used above, we will have

∑i∈I |Ui(t)| ≤ ε for all t ≥ 0. In particular, Uλ

i (t) ≤ 2ε for all t ≤ Tλ ∧ s∗/√

λ.Adding this to (45), we have on Kλ,k

⋂Ωλ,

sup0≤t≤T

∑

i∈I|Uλ

i (t)| ≤ 2ε.

Since PKλ,k

⋂Ωλ

≥ 1− 2δ for all λ large enough, we have established that

lim supλ→∞

P

sup

s≤t≤T λ

∑

i∈I|Uλ

i (t)| > 2ε

≤ 2δ,

implying (37).


Proof of Theorem 6.4 The proof is similar to corresponding proofs in Bramson [6] and in Dai and Tezcan[12] and, specifically. it parallels the proofs in §C.2-C.3 of [12]. However, there are some minor differences.Hence, we write out the proofs, but abbreviate whenever the proof closely follows either [6] or [12]. The proof isdivided into three lemmas. Lemma 6.3 shows that, on a large enough subspace of the sample space, the processXλ,m is almost Lipschitz. This is used in Lemma 6.4 to establish the uniform approximation by cluster points.Together with Lemmas 4.1 and 4.2 and Proposition 4.1 of Bramson [6], Lemmas 6.3 and 6.4 here imply theuniform approximation by a Lipschitz function. Bramson [6] elaborates on the compactness and cluster-pointstructure. Finally, Lemma 6.5 below establishes that each cluster point satisfies equations (41)-(44). That primarilymeans the last two equations: (43) and (44).

We start by defining some important sets. Fix k, λ and Lk and define the following sets:

Ωλ,k1 :=

ω ∈ Ω : ‖Uλ(· ∧ Tλ)‖∗T + ‖Zλ(· ∧ Tλ)‖∗T + ‖Qλ(· ∧ Tλ)‖∗T ≤ k

, (46)

Ωλ,k2 := Ωλ,k

2 (ε) :=

ω ∈ Ω : maxm<

√λT λ

‖Aλ,m(t)− at‖∗Lk≤ ε

,

Ωλ,k3 := Ωλ,k

3 (ε) :=

ω ∈ Ω : maxm<

√λT λ

‖Dλ,m(t)− µνt‖∗Lk≤ ε

,

Ωλ,k4 := Ωλ,k

4 (ε,N) :=

ω ∈ Ω : maxm<

√λT λ

supt1,t2≤Lk

‖Xλ,m(t2)− Xλ,m(t1)‖ ≤ N |t2 − t1|+ ε

,

where Aλ,m(t) :=(Aλ,m

1 (t), . . . , Aλ,mI (t)

), a := (a1, . . . , aI), Dλ,m(t) :=

(Dλ,m

1 (t), . . . , Dλ,mJ (t)

), µν :=

(µ1ν1, . . . , µJ νJ) and N is some fixed constant that depends only on the vectors a, ν as well as I and J (andwhose specific value will be made explicit within the proof of the following lemma). Set Kλ,k :=

⋂4i=1 Ωλ,k

i . Thefollowing lemma is the analogue of Propositions 6.2 and 6.3 in [12].

Lemma 6.3 For any ε, δ > 0, there exist k and Lk so that lim infλ→∞ PKλ,k ≥ 1− δ for Kλ,k defined above.

Proof: By Lemma 6.2, XλΣ(t∧Tλ), Qλ

Σ(t∧Tλ) and IλΣ(t∧Tλ) are stochastically bounded. Since Zj(t) = −Iλ

j (t)and, by definition,

∑i∈I |Uλ

i (t)| ≤ QλΣ(t) + |Xλ

Σ(t)|, we have

limk→∞

lim supλ→∞

P

Ω− Ωλ,k1

= 0. (47)

We now turn to the set Ωλ,k3 (The argument for the set Ωλ,k

2 is omitted, because it follows similarly and is eveneasier). By construction, for t ≤ Lk,

Dλ,mj (t) =

1√λ

(Sj

(µj

∫ (m+t)/√

λ

0

Zλj (s)ds

)− Sj

(µj

∫ m/√

λ

0

Zλj (s)ds

))

d=1√λ

(Sj

(µj

∫ (m+t)/√

λ

m/√

λ

Zλj (s)ds

)),

where the equivalence in distribution (as processes) follows from the properties of the Poisson process. As aconsequence,

∥∥∥∥∥Dλ,mj (t)− 1√

λµj

∫ (m+t)/√

λ

m/√

λ

Zλj (s)ds

∥∥∥∥∥

∗

Lk

d=1√λ

∥∥Sj(µjNλj t)− µjN

λj t

∥∥∗ψλ ,

where

ψλ =

∫ (m+Lk)/√

λ

m/√

λZλ

j (s)ds

Nλj

.

Using the fact that Zλj (t) ≤ Nλ

j and carefully applying Proposition 4.3 in Bramson [6], we have

P

∥∥∥∥∥Dλ,mj (t)− 1√

λµj

∫ (m+t)/√

λ

m/√

λ

Zλj (s)ds

∥∥∥∥∥

∗

Lk

> εLk

≤ ε

√λ

µjNλj Lk

.


Bounding the distribution of the maximum by summation over the individual probability distributions, we thenhave

P

max

m<√

λT λ

∥∥∥∥∥Dλ,mj (t)− 1√

λµj

∫ (m+t)/√

λ

m/√

λ

Zλj (s)ds

∥∥∥∥∥

∗

Lk

> εLk

≤ ελT

µjNλj Lk

≤ 2εT

µj νjLk, (48)

for all λ large enough, since Nλj /λ → νj as λ →∞. We can then replace ε with ε/Lk and choose Lk large enough

so that the probability in (48) is bounded by δ/8J . Finally, since IλΣ(t∧ Tλ) is stochastically bounded (by Lemma

6.2), and since Nλj = νjλ + γj

√λ + o(

√λ), we have

P

max

m<√

λT λ

∥∥∥∥∥1√λ

µj

∫ (m+t)/√

λ

m/√

λ

Zλj (s)ds− µj νjt

∥∥∥∥∥

∗

Lk

> ε

≤ δ/8J,

for all λ large enough. Repeating the same argument for all j ∈ J and fixing Lk sufficiently large, we then have,for all λ large enough,

P

max

m<√

λT λ

∥∥Dλ,m(t)− µνt∥∥∗

Lk> ε

≤ δ/4,

so that PΩλ,k3 ≥ 1 − δ/4. A similar, but easier, argument for Ωλ,k

2 shows that PΩλ,k2 ≥ 1 − δ/4 for all λ

large enough. The details are omitted.

We turn now to the set Ωλ,k4 . We first prove probability bounds for each process separately and finally combine

all the bounds to obtain the corresponding bound for the multidimensional processXλ,m. We start from the processDλ,m

j (t), j ∈ J . Since Zλj (s) ≤ Nλ

j for all s ≥ 0, we have that

supt1,t2≤Lk

|Dλ,mj (t2)−Dλ,m

j (t1)| ≤ 1√λ

supt1,t2≤Lk

|Sj(µjNλj t2/

√λ)− Sj(µjN

λj t1/

√λ)|.

For t1, t2 ≤ Lk,

|Sj(µjNλj t2/

√λ)− Sj(µjN

λj t1/

√λ)| ≤ µj

Nλj√λ|t2 − t1|+ 2 sup

t≤ Lk√λ

|Sj(µjNλj t)− µjN

λj t|.

Fix ε′ > 0. We then have

P

max

m<√

λT λ

supt1,t2≤Lk


j (t1)| > µj

Nλj

λ|t2 − t1|+ ε′

≤√

λT · P sup

t≤ Lk√λ

|Sj(µjNλj t)− µjN

λj t| > ε′

2

√λ

. (49)

Fixing δ′ > 0, applying Proposition 4.3 of [6] to the right-hand side of (49), and using the fact that Nλj /λ → νj as

λ →∞, we have, for fixed δ′ > 0,

P

max

m<√

λT λ

supt1,t2≤Lk


j (t1)| > µj νj |t2 − t1|+ ε′≤ δ′, (50)

for all λ large enough. A similar argument is repeated for Aλ,mi (t), i ∈ I to show that

P

max

m<√

λT λ

supt1,t2≤Lk

|Aλ,mi (t2)−Aλ,m

i (t1)| > ai|t2 − t1|+ ε′≤ δ′, (51)

We now treat the more complicated routing processes Φλ,mi,j (t) and Aλ,m

i,j (t). We do so by relating their incrementsto those of the previously-treated processes Dλ,m

j (t) and Aλ,mi (t). For Φλ,m

i,j (t) and Dλ,mj (t), it is important not to

try to match the routed customers to the service of those same customers; instead we think of departures allowingnew customers to be assigned to agents. In particular, we apply (17) to get, for all i ∈ I, j ∈ J ,

|Aλ,mi,j (t2)−Aλ,m

i,j (t1)| ≤ |Aλ,mi (t2)−Aλ,m

i (t1)|,|Φλ,m

i,j (t2)− Φλ,mi,j (t1)| ≤ |Dλ,m

j (t2)−Dλ,mj (t1)| . (52)


Combining (50)–(52), we obtain

P

max

m<√

λT λ

supt1,t2≤Lk

|Aλ,mi,j (t2)−Aλ,m

i,j (t1)| > N ′|t2 − t1|+ ε′≤ δ′, i ∈ I, j ∈ J ,

P

max

m<√

λT λ

supt1,t2≤Lk

|Φλ,mi,j (t2)− Φλ,m

i,j (t1)| > N ′|t2 − t1|+ ε′≤ δ′, i ∈ I, j ∈ J , (53)

for all λ large enough, where N ′ := maxiai∨

maxjµj νj.

Now consider the processes Rλ,mi (t), i ∈ I . By construction,

Rλ,mi (t) = Ri

(θi

∫ (t+m)/√

λ

0

Qλi (s)ds

)−Ri

(θi

∫ m/√

λ

0

Qλi (s)ds

).

Hence,

|Rλ,mi (t2)−Rλ,m

i (t1)| ≤ θi1√λ

∫ (m+t2)/√

λ

(m+t1)/√

λ

Qλi (s)ds + sup

t≤Lk

∣∣∣∣∣Rλ,mi (t)− 1√

λ

∫ (m+t2)/√

λ

(m+t1)/√

λ

Qλi (s)ds

∣∣∣∣∣ .

On Ωλ,k1 , however, ‖Qλ‖∗T λ ≤ k and

supt≤Lk

∣∣∣∣∣Rλ,mi (t)− 1√

λ

∫ (m+t2)/√

λ

(m+t1)/√

λ

Qλi (s)ds

∣∣∣∣∣ ≤st1√λ

supt≤Lk/

√λ

∣∣∣Ri(θik√

λt)− θik√

λt∣∣∣ .

Applying Proposition 4.3 of [6] once again, we have, for all λ large enough,

P

sup

t1,t2≤Lk


i (t1)| > θi1√λ|t2 − t1|k + ε′/2

≤ δ′/2 + P(Ωλ,k

1 )c.

Since θiLk/√

λ ≤ ε′/2 and P(Ωλ,k)c ≤ δ′/2 for all λ large enough,

P

sup

t1,t2≤Lk


i (t1)| > ε′≤ δ′, (54)

for all λ large enough. Now note that

Qλ,mi (t) = Qλ,m

i (0) + Aλ,mi (t)−

∑

j∈JAλ,m

i,j (t)−∑

j∈JΦλ,m

i,j (t)−Rλ,mi (t).

Let N ′′ = (2I + 2J)N ′. Fixing δ′′ > 0 and ε′′ > 0, we can that choose new values of δ′ and ε′ and combine(51)–(54) to obtain

P

max

m<√

λT λ

supt1,t2≤Lk

|Qλ,mi (t2)−Qλ,m

i (t1)| > N ′′|t2 − t1|+ ε′′≤ δ′′, i ∈ I, (55)

for all λ large enough. A similar argument is used for Zλ,mj (t) to show that

P

max

m<√

λT λ

supt1,t2≤Lk

|Zλ,mj (t2)− Zλ,m

j (t1)| > N ′′|t2 − t1|+ ε′′≤ δ′′, j ∈ J ,

for all λ large enough. We omit this argument.

We now turn to the processes Θλi (t), i ∈ I. Consider ω ∈ Ωλ,k

1 . Then, ‖XλΣ‖∗T λ ≤ k, and the Holder condition

on the ratio function (see Definition 2.2) implies that there exists constants ck and αk, depending on k and ε′ butnot on λ, so that for all 0 < t1 ≤ t2 ≤ Tλ,

[XλΣ(t2)]+pi

([Xλ

Σ(t2)]+)− [Xλ

Σ(t1)]+pi

([Xλ

Σ(t1)]+)≤ ck|Xλ

Σ(t2)− XλΣ(t1)|αk + ε′/2;

see for example equation (118) in [4]. Hence,

P

max

m<√

λT λ

supt1,t2≤Lk

|Θλ,mi (t2)−Θλ,m

i (t1)| > ε′≤ P(Ωλ,k

1 )c

+P

sup

0≤t1<t2≤T λ:|t2−t1|≤Lk/√

λ

ck|XλΣ(t2)− Xλ

Σ(t1)|αk > ε′/2


Since we can choose k large enough so that for any λ large enough PΩλ,k1 ≥ 1 − δ′/2 for any λ large enough,

since XλΣ(t ∧ Tλ) is C-tight by Lemma 6.2, and since we can move the exponent αk, first outside the supremum

and then to the other side by raising to the reciprocal power, we can conclude that

P

max

m<√

λT λ

supt1,t2≤Lk

|Θλ,mi (t2)−Θλ,m

i (t1)| > ε′≤ δ′. (56)

Combining (55) and (56) and using the fact that

Uλ,mi (t) = Qλ,m

i (t)− 1√λ

Θλi ((m + t)/

√λ)

= Uλ,mi (0) + Aλ,m

i (t)−∑

j∈JAλ,m

i,j (t)−∑

j∈JΦλ,m

i,j (t)−Rλ,mi (t)−Θλ,m

i (t),

we can choose new values of ε′ and δ′ so that

P

max

m<√

λT λ

supt1,t2≤Lk

|Uλ,mi (t2)− Uλ,m

i (t1)| > N ′′|t2 − t1|+ ε′′≤ δ′′, (57)

for all λ large enough. Now set N = 8(I + J + IJ). Then, we can combine (50), (51) and (53)–(57) and choosenew values δ′, δ′′, ε′ and ε′′ appropriately to conclude that PΩλ,k

4 ≥ 1− δ/4 for all λ large enough. Finally, wecan combine the four inequalities for PΩλ,k

i above to conclude that PKλ,k ≥ 1 − δ for some k, Lk, λ0 andfor all λ ≥ λ0. ¥

Having proved that the family of HS processes can be approximated by Lipschitz functions, we now want toestablish the uniform approximation by HL functions satisfying the HM equations. For this important step wehave the following lemma, which is the analog of Proposition 6.1 in [6]. The proof is exactly as in [6] and is henceomitted.

Lemma 6.4 For all ε > 0, there exists k > 0, Lk > 0 and λ0, so that, for all λ > λ0, ω ∈ Kλ,k and m <√

λTλ,there exists an HL function X (a Lipschitz function satisfying the HM equations) such that ‖Xλ,m − X‖∗Lk

≤ ε.

Lemmas 6.3 and 6.4 combined show that, given ε > 0 and δ > 0, we can choose a set Kλ,k (which also dependson ε) with PKλ,k > 1 − δ, on which all the process are stochastically bounded and any HS process can beapproximated by an HL function. Lemmas 4.1 and 4.2 and Proposition 4.1 of [6] imply that the approximating HLfunction is Lipschitz.

To establish Theorem 6.4, it remains only to show that all the hydrodynamic limits satisfy equations (41)-(44).That is done in the following lemma. The lemma below is mostly an analogue of Proposition 6.6 in [12]. Themajor difference is to show that the QIR-specific equations (42)-(44) hold for any hydrodynamic limit X .

Lemma 6.5 Fix k > 0, Lk > 0 and let X be a hydrodynamic limit of the family of HS processes Xλ,m over[0, Lk]. Then X satisfies equations (41)-(44).

Proof: Using the definitions of the HL function and the set Kλ,k, it is immediate that any HL function satisfiesequations (41)-(42); see the proof of Proposition 6.6 in [12]). We turn, then, to prove that any hydrodynamic limitX satisfies (43).

Toward that end, recall the definition I+(t) = i ∈ I : Ui(t) > 0 and consider t ≥ 0 with I+(t) 6= ∅. Letε(t) := mini∈I+(t) Ui(t). Since every HL function x is absolutely continuous (and thus continuous), we must havean interval [t−τ, t+τ ] such that for all u ∈ [t−τ, t+τ ], mini∈I+(u) Ui(u) ≥ ε(t)/2. Moreover, by the continuityof Ui(t), the interval can be chosen so that Ui(u) ≤ ε(t)/8 for all i /∈ I+(t) and u ∈ [t − τ, t + τ ]. Next, sinceX is an HL function, we can fix k and argue that there exists λ large enough, ω ∈ Kλ,k and m <

√λTλ, so that

‖Uλ,mi − Ui‖∗Lk

≤ ε for ε ≤ ε(t)/8. In particular, we can choose Lk (and appropriately choose anew value of λ)so that there exists a neighborhood [t− τ ′, t + τ ′] of t such that for all u ∈ [t− τ ′, t + τ ′], Uλ,m

i (u) ≥ (3/8)ε(t),i ∈ I+(t) and Uλ,m

i (u) ≤ (2/8)ε(t), i /∈ I+(t). Then, for any j ∈ J(I+(t)) = j ∈ J : i ∈ I(j) for some i ∈I+(t),

argmaxi∈I(j)

Uλ,mi (u) ∈ I+(t) (58)


for all u ∈ [t− τ ′, t + τ ′]. In turn, by the definition of Φλi,j(t) (see equation (17)) we have that

∑

i∈I+(t)

∑

j∈J(i)

(Φλ,m

i,j (t + τ ′)− Φλ,mi,j (t− τ ′)

)=

∑

j∈J(I+(t))

(Dλ,m

j (t + τ ′)−Dλ,mj (t− τ ′)

), (59)

which corresponds to the fact that every service completion is followed by an admission of a customer from a classi ∈ I+(t). By definition, ‖Dλ,m

j (t)− µj νjt‖∗Lk≤ ε on Kλ,k. Hence,

∑

i∈I+(t)

∑

j∈J(i)

(Φλ,m

i,j (t + τ ′)− Φλ,mi,j (t− τ ′)

)≥

∑

j∈J(I+(t))

µj νj2τ ′ − ε,

and, since ‖Xλ,m − X‖ ≤ ε,∑

i∈I+(t)

∑

j∈J(i)

(Φij(t + τ ′)− Φij(t− τ ′)

)≥

∑

j∈J(I+(t))

µj νj2τ ′ − 2Jε.

Since ε was chosen arbitrarily and the bound holds for any τ ≤ τ ′,∑

i∈I+(t)

∑

j∈J(i)

dΦij(t) ≥∑

j∈J(I+(t))

µj νj .

Equation (43) now follows. Finally, for equation (44), note that

d∑

i∈I+(t)

Uλi (t) =

∑

i∈I+(t)

dAi(t)−∑

i∈I+(t)

∑

j∈J(i)

dAij(t)−∑

i∈I+(t)

∑

j∈J(i)

dΦij(t) ≤∑

i∈I+(t)

ai −∑

j∈J(I+(t))

µj νj ,

where the last inequality follows from (43). We now observe that, due to Assumptions 2.2 and 2.3, there existsc > 0 such that for any strict subset B ⊂ I,

∑

i∈B

ai −∑

i∈B

∑

j∈J(i)

µj νj ≤ −c.

Equation (44) now follows since I+(t) is necessarily a strict subset of I. We have thus established that any HLfunction X satisfies (41)-(44) and the proof of the lemma is complete. ¥

With Lemma 6.5 we have completed the proofs of Theorems 6.4 and 6.2 for the stopped processes. Theorem6.2, under condition C-1, is then established by showing in the following lemma that ςλ approaches ∞ weakly.That implies the SSC extends to the whole interval [0, T ].

Lemma 6.6 for each T > 0, Pςλ ≤ T → 0 as λ →∞.

Proof: By definition, Bλ(t) =∑

i∈I Uλi (t) ≥ 0. Using (30), we can write

Pςλ ≤ T = P|Bλ|∗ςλ∧T ≥ 2Bλ(0) ∨ 1

= P

sup

0≤t≤T λ

∑

i∈IUλ

i (t) > 2Bλ(0) ∨ 1

, (60)

where we use the equivalence of the events ςλ ≤ T and Bλ(Tλ) ≥ 2Bλ(0)∨ 1 as follows from the definitionof ςλ in equation (30). In order to establish that Pςλ ≤ T → 0 as λ → ∞, it suffices to prove that the righthand side of (60) converges to 0. Toward that end, fix δ > 0. Then, there exists k such that PKλ,k ≥ 1− δ forall λ large enough. Fix 0 < ε < 1/2. By Theorem 6.4, for all λ large enough and ω ∈ Kλ,k, there exists Ui(t) thatsatisfies (42)-(44) such that ‖Uλ,0(t)− U(t)‖∗Lk

≤ ε.

By equation (44),d

∑

i∈I+(t)

Ui(t) ≤ −c

for some positive constant c. Consequently, for all t ≤ Lk,∑

i∈IUλ,0

i (t) ≤ Bλ(0) ∨(

12

+ ε

).

Since Ui(t/√

λ) = Uλ,0i (t) for all t ≤ Lk, we have

∑i∈I Uλ

i (t) ≤ 2Bλ(0) ∨ 1 for all t ≤ Lk/√

λ and

P

sup

0≤t≤Lk/√

λ

∑

i∈IUλ

i (t) > 2Bλ(0) ∨ 1

≤ δ.


Choosing Lk ≥ 2ds∗e with s∗ as defined in the proof of Theorem 6.2 and repeating the arguments in that proof,we have

P

sup

Lk/√

λ≤t≤T λ

∑

i∈I|Uλ

i (t)| > ε

≤ δ,

so that

P

sup

Lk/√

λ≤t≤T λ

∑

i∈I|Uλ

i (t)| > 2Bλ(0) ∨ 1

≤ δ.

We conclude by noting that

P

sup

0≤t≤T λ

∑

i∈IUλ

i (t) > 2Bλ(0) ∨ 1

≤ P

sup

0≤t≤Lk/√

λ

∑

i∈IUλ

i (t) > 2Bλ(0) ∨ 1

+ P

sup

Lk/√

λ≤t≤T

∑

i∈IUi(t) > 2Bλ(0) ∨ 1

≤ 2δ.

Since δ was arbitrary, the proof is complete. ¥With Lemma 6.6, we have completed the proof of Theorem 6.2 and in turn the proof of Theorem 3.1 under

condition C-1. Specifically, we have shown that, for all 0 < s < T ,

lim supλ→∞

P

sup

s≤t≤T

∑

i∈I|Uλ

i (t)| > ε

= 0. (61)

If, in addition, equation (4) holds then

lim supλ→∞

P

sup

0≤t≤T

∑

i∈I|Uλ

i (t)| > ε

= 0. (62)

The proof of SSC for the processes V λj (t) follows similarly. Before turning to the proof of Theorem 3.1 under

condition C-2, we state the following corollary that will be of use in §4.

Corollary 6.5 The sequences QλΣ(t), Iλ

Σ(t) and XλΣ(t) are stochastically bounded. i.e.,

limA→∞

lim supλ→∞

P‖Qλ

Σ‖∗T + ‖IλΣ‖∗T + ‖Xλ

Σ‖∗T > A

= 0. (63)

Also, the process XλΣ(t) is C-tight.

Proof: These results have already been proved for the stopped processes in Lemma 6.2. This additional result thenfollows from Lemma 6.6.

6.3 SSC Under C-2 In §6.2, due to the pool-dependent service rates, we could use a somewhat less detaileddescription of the system dynamics than the general description given (14)-(17). We now return to that generaldescription. First we define

Bλ(t) :=∑

i∈I|Uλ

i (t)| ∨∑

i∈I|V λ

i (t)|, (64)

with Uλi (t) and V λ

i (t) as defined in (16), respectively. Then we let Zλi (t) :=

∑j∈J Zλ

i,j(t) be the number of class-i customers in service at time t and define Zλ

Σ(t) :=∑

i∈I Zλi (t). The arguments leading to (28) are immediately

adapted to show that

Xλi (t) = Xλ

i (0)− βit− µi

∫ t

0

Zλi (s)ds− θi

∫ t

0

Qλi (s)ds + Mλ

i (t) + oP (1) , (65)

where βi = µi

∑j∈J xi,jγj and Mλ

i (t) is a square integrable martingale defined through

Mλi (t) := Mλ

Ai(t)−

∑

j∈JMλ

i,j(t)− MλRi

(t).


Redefining MλΣ(t) :=

∑i∈I Mλ

i (t), we have

XλΣ(t) = Xλ

Σ(0)−∑

i∈Iβit−

∑

i∈Iµi

∫ t

0

Zλi (s)ds−

∑

i∈Iθi

∫ t

0

Qλi (s)ds + Mλ

Σ(t). (66)

The proof proceeds through the same stopping argument used in §6.2, where Tλ := ςλ ∧ T and ςλ is defined as in(30). For the stopped processes, our proof is similar to the proof in §6.2. The main difference between the proofsfor the different conditions, C-1 and C-2, is in the choice of the HS processes and the HM equations. Once theseare redefined, all the statements of the theorems, lemmas and corollaries in §6.2 are the same in this setting withthe exception, of course, of replacing condition C-1 with condition C-2. Moreover, once the HS processes and HMequations are redefined, all the proofs are adapted from §6.2 with only minor changes, that should be clear once therequired definitions are in place. Hence, we omit the detailed proofs and only make the required new definitions.

Toward that end, we extend (39) by adding, for each i ∈ I and j ∈ J , the process of class-i departures frompool j, given by

Dλi,j(t) := Si,j

(µi

∫ t

0

Zλi,j(s)ds

),

as well as the process of cumulative class-i departures given by Dλi (t) :=

∑j∈J Dλ

i,j(t). The hydrodynamically-scaled processes are then defined as in (40) with the addition of the following: For m <

√λT , we define

Dλ,mi (t) :=

1√λ

(Dλ

i

(m√λ

+t√λ

)−Dλ

i

(m√λ

)), i ∈ I,

Dλ,mi,j (t) :=

1√λ

(Dλ

i,j

(m√λ

+t√λ

)−Dλ

i,j

(m√λ

)), i ∈ I, j ∈ J ,

D′λ,mi,j (t) := Dλ,m

i,j (t)− 1√λ

µi

∫ (m+t)/√

λ

m/√

λ

Zλi,j(s)ds, i ∈ I, j ∈ J .

The hydrodynamic model equations for

X :=(Ai(t), Aij(t), Φij(t), Dj(t), Di(t), D′

ij(t), Ri(t), Θi(t), Qi(t), Ui(t), Zj(t); i ∈ I, j ∈ J)

,

are given by the first four equations in equations (41) and (42) with the addition of the following equations:

Zj(t) = Zj(0) +∑

i∈J(i)

Aij(t) +∑

i∈J(i)

Φij(t)− Dj(t), j ∈ J , (67)

Di(t) = ait, i ∈ I, (68)

D′ij(t) = 0, i ∈ I, j ∈ J , (69)∑

i∈I+(t)

∑

j∈J(i)

dΦij(t) ≥∑

i∈I+(t)

ai + c1, (70)

∑

i∈I+(t)

dUi(t) ≤ −c2, whenever I+(t) 6= ∅, (71)

Where c1 and c2 are strictly positive constants and, as before, I+(t) = i ∈ I : Ui(t) > 0, and J(I+(t)) = j ∈J : i ∈ J(i), for some i ∈ I+(t).

We redefine

Ωλ,k3 := Ωλ,k

3 (ε) :=

ω ∈ Ω : maxm<

√λT λ

‖Dλ,m(t)− at‖∗Lk≤ ε

, (72)

with Dλ,m(t) =(Dλ,m

1 (t), . . . , Dλ,mI (t)

)and add the set

Ωλ,k5 := Ωλ,k

5 (ε) :=

ω ∈ Ω : max

(i,j)∈I×Jmax

m<√

λT λ

∥∥∥∥∥Dλ,mi,j (t)−

∫ (m+t)/√

λ

m/√

λ

Zλi,j(s)ds

∥∥∥∥∥

∗

Lk

≤ ε

. (73)

The other subsets Ωλ,k1 , Ωλ,k

2 and Ωλ,k4 remain as defined before. Finally, we redefine Kλ,k =

⋂5i=1 Ωλ,k

i . Withthese definitions, all the statements of §6.2 hold in this section without any change, and the adaptation of the proofsare straightforward, leading to the proof of SSC under condition C-2.


6.4 SSC Under C-3 In §5 we observed that, for some special tree networks and choices of the ratio vectorv(·), QIR control is equivalent to the GQIR control in [4]. Consequently, the SSC result for these networks followsfrom [4]. Here we treat more general tree networks. We use the function G in (13).

Using the sample path construction in (14) and applying the martingale decomposition, we write

Xλi (t) = Xλ

i (0)+λit−∑

j∈J(i)

µi,j xi,jNλj −

∑

j∈J(i)

µi,j

∫ t

0

(Zλi,j(s)− xi,jN

λj )ds−θi

∫ t

0

Qλi (s)ds+Mλ

i (t), (74)

whereMλ

i (t) := MλAi

(t)−∑

j∈J(i)

Mλi,j(t)−Mλ

Ri(t), (75)

with Mλi,j(t), Mλ

Ai(t) and Mλ

Ri(t) as defined in (20). Scaling by

√λ we have

Xλi (t) = Xλ

i (0) + βit−∑

j∈J(i)

µi,j

∫ t

0

Zλi,j(s)ds− θi

∫ t

0

Qλi (s)ds + Mλ

i (t) + oP (1), (76)

where

βi := limλ→∞

λit−∑

j∈J xi,jNλj√

λ

and Mλi (t) := Mλ

i (t)/√

λ. We now apply (12) to get

Xλi (t) = Xλ

i (0) + βit−∑

j∈J(i)

µi,j

∫ t

0

Gij

(Xλ(s)− Qλ(s),−Iλ(s)

)ds− θi

∫ t

0

Qλi (s)ds + Mλ

i (t) + oP (1).

(77)

We now proceed using the stopping argument we used in the previous sections. We use Bλ(t) defined in(29). As in the previous section we first establish SSC assuming that all the processes are stopped at the boundedstopping time Tλ := ςλ ∧ T in (30). Having defined the stopping time, the proof of SSC follows the proof of §6.2very closely. Hence, we will abbreviate all statements and proofs and provide details only when the differencesare significant. The most significant modification is in the proof of stochastic boundedness and C-Tightness asstated in Lemma 6.7 below. Hence, we first state and prove this lemma. We then proceed to outline the requiredmodifications to the arguments in §6.2.

Towards this end, recall that Xλ(t) := (Xλ1 (t), . . . , Xλ

I (t) and Zλ(t) := (Zλi,j(t); i ∈ I, j ∈ J ).

Lemma 6.7 (stochastic boundedness of scaled queueing processes) The sequences QλΣ(t∧Tλ), Iλ

Σ(t∧Tλ),Xλ(t∧Tλ) and Zλ(t∧Tλ) are stochastically bounded. Also, the sequence of processes Xλ

i (t∧Tλ) : λ > 0is C-tight.

Proof: First note that, by the linearity of G(·, ·) in (13), we have

G(Xλ(t)− Qλ(t),−Iλ(t)

)= G

(Xλ(t)− [Xλ

Σ(t)]+p([Xλ

Σ(t)]+)

,−[XλΣ(t)]−v

([Xλ

Σ(t)]−))

−G(Qλ(t)− [Xλ

Σ(t)]+p([Xλ

Σ(t)]+)

, Iλ(t)− [XλΣ(t)]−v

([Xλ

Σ(t)]−))

, (78)

where[Xλ

Σ(t)]+p([Xλ

Σ(t)]+)

:=([Xλ

Σ(t)]+p1

([Xλ

Σ(t)]+)

, . . . , [XλΣ(t)]+pI

([Xλ

Σ(t)]+))

,

and [XλΣ(t)]−v

([Xλ

Σ(t)]−)

is defined similarly from the individual coordinates. Using equation (77) as well asthe linearity of G(·, ·) we then have

‖Xλ‖∗t ≤ ‖Xλ(0)‖+ c1T + c2

∫ t

0

(‖Xλ‖∗s + ‖Bλ‖∗s

)ds + ‖Mλ

i ‖∗t ,

for some positive constants c1 and c2. Applying Gronwall’s inequality (Lemma 4.1 in [25]), we have

‖Xλ(t ∧ Tλ)‖∗T ≤ c1

(‖Xλ(0)‖+ T + ‖Bλ(· ∧ Tλ)‖∗T + ‖Mλ

i (· ∧ Tλ)‖∗T)

ec2T ,


for some re-defined positive constants c1 and c2. By Lemma 6.1, Mλi (t) is stochastically bounded and, by the

definition of Tλ and equation (31), Bλ(t ∧ Tλ) is stochastically bounded. Here we also use the fact that Bλ(0) isitself stochastically bounded by the assumed convergence of Xλ(0).

The stochastic boundedness of Xλ(t ∧ Tλ) now follows from it being bounded by a sum of stochasticallybounded sequences. Next, Iλ

Σ(t∧Tλ) and QλΣ(t∧Tλ) are stochastically bounded by applying (33) and (34), which

still hold with the new definition of Bλ. Then Zλ(· ∧Tλ) is stochastically bounded because (under condition C-3)it involves a linear mapping of Q, I and X (see equation 12). The result of the proposition now follows as thesum of stochastically bounded sequences is itself stochastically bounded. Finally, the proof of the C-tightness ofXλ(t ∧ Tλ) follows very similarly to the proof of Lemma 6.2 by applying Gronwall’s inequality and making theobvious required modifications. ¥

Having Lemma 6.7, the proof of SSC follows very closely the proof in §6.2 for Condition C-1. However, somemodifications to the hydrodynamically-scaled processes and hydrodynamic-limit functions are required. We nowoutline these modifications:

(i) Augment the state descriptor Xλ(t) by adding the coordinates Zλi,j(t) for all i ∈ I and j ∈ J . Also,

replace the processes Dλj (t) with the processes Dλ

i,j(t) defined to be the number of service completionsof class-i customers with pool-j agents. Namely,

Dλi,j(t) := Si,j

(µi,j

∫ t

0

Zλi,j(s)ds

).

(ii) Augment the hydrodynamically scaled process Xλ,m(t) by adding the processes Zλ,mi,j (t) defined by

Zλ,mi,j (t) :=

1√λ

(Zλ

i,j

(t√λ

+m√λ

)− xi,jN

λj

).

Also, replace the process Dλ,mj (t) with the processes Dλ,m

i,j (t) defined in the obvious way from Dλi,j(t).

(iii) Augment the hydrodynamic limit function X by adding Zi,j(t) for each i ∈ I and j ∈ J and replacingDj(t) with Di,j(t).

(iv) Replace the last equation in (41) with

Zi,j(t) = Zi,j(0) +∑

i∈IAij(t) +

∑

i∈IΦij(t)− µi,j xi,j νjt, i ∈ I, j ∈ J .

(v) Replace equation (43) with ∑

i∈I+(t)

∑

j∈J(i)

dΦij(t) =∑

j∈J(I+(t))

µi,j xi,j νj .

Equation (44) would then follow by noting that the tree assumption implies that given a set B ⊂ I,∑i∈I ai −

∑i∈B,j∈J (i) µi,j xi,j νj < −c, for some constant c > 0.

(vi) Re-define

Ωλ,k3 := Ωλ,k

3 (ε) :=

ω ∈ Ω : maxm<

√λT λ

‖Dλ,m(t)− µxνt‖∗Lk≤ ε

, (79)

where now Dλ,m(t) := (Dλi,j(t); i ∈ I, j ∈ J ) and µxν = (µi,j xi,j νj ; i ∈ I, j ∈ J ).

(vii) In the proof of Lemma 6.3 one replaces the argument for Dλ,mj (t) with the corresponding argument for

Dλ,mi,j (t). The arguments follow very similarly with the exception of the following additional step:

supt1,t2≤Lk

|Dλ,mi,j (t2)−Dλ,m

i,j (t1)| ≤ 1√λ

supt1,t2≤Lk

∣∣∣∣∣Si,j

(µi,j

∫ t2/√

λ

0

Zλi,j(s)ds

)− Si,j

(µi,j

∫ t1/√

λ

0

Zλi,j(s)ds

)∣∣∣∣∣ .

In the proof of Lemma 6.3 we use the stochastic boundedness of the process Zλj (t). Similarly, one should

now use Lemma 6.7 and, in particular, the stochastic boundedness of the sequence Zλ(· ∧ Tλ).(viii) Equation (59) should be replaced with

∑

i∈I+(t)

∑

j∈J(i)

(Φλ,m

i,j (t + τ ′)− Φλ,mi,j (t− τ ′)

)=

∑

j∈J(I+(t))

∑

k∈I

(Dλ,m

k,j (t + τ ′)−Dλ,mk,j (t− τ ′)

).

With the above modifications, one can easily follow §6.2 to construct the complete proof of SSC under Condition3.


7. Proofs of the Results in §4.

Proof of Theorem 4.1: The limit for XλΣ is obtained through an application of the continuous mapping theorem,

e.g., Theorem 3.4.1 of [31]. In particular, by SSC, we have ‖XλΣ − Y λ

Σ ‖ ⇒ 0 in D− as λ →∞, where

Y λΣ (t) = Xλ

Σ(0)−∑

j∈Jµjγjt +

∑

j∈Jµj

∫ t

0

vj([XλΣ(s)]−)ds−

∑

i∈I

θi

∫ t

0

pi([XλΣ(s)]+)ds + Mλ

Σ(t),

and XλΣ has the representation given in (28). Applying Corollary 6.5, we have that both Xλ

Σ(t) and Y λΣ (t) are

C-Tight. Consequently, the asymptotic equivalence above in D− extends to D, and we can write

XλΣ(t) = Xλ

Σ(0)−∑

j∈Jµjγjt+

∑

j∈Jµj

∫ t

0

vj([XλΣ(s)]−)ds−

∑

i∈I

θi

∫ t

0

pi([XλΣ(s)]+)ds+Mλ

Σ(t)+oP (1). (80)

By Theorem 4.1 in [25], this integral representation for XλΣ(t) is a measurable continuous mapping from D to

itself. In particular, we can get the convergence of XλΣ(t) from the convergence of the basic martingale Mλ

Σ(t) thatis established in the following lemma.

Lemma 7.1 Under the conditions of Theorem 4.1, MλΣ(t) ⇒ √

2B(t) in D as λ → ∞, where B(t), t ≥ 0 is astandard Brownian motion.

The proof of Lemma 7.1 is postponed to the end of this section. We can now apply the continuous-mappingtheorem to (80) and use Lemma 7.1 to get the convergence of Xλ

Σ(t). By applying the continuous-mapping theoremagain, we can then extend the limit to the vector process

(Xλ

Σ(t), p1([XλΣ(t)]+), . . . , pI([Xλ

Σ(t)]+),1a1

p1([XλΣ(t)]+), . . . ,

1aI

pI([XλΣ(t)]+), v1([Xλ

Σ(t)]−), . . . , vJ [XλΣ(t)]−)

)in D2I+J+1 .

Then we can apply the convergence-together theorem again to show that this process has the same limit as theprocess

(Xλ

Σ(t), Qλ1 (t), . . . , Qλ

I (t), Wλ1 (t), . . . , Wλ

I (t), Iλ1 (t), . . . , Iλ

J (t))

on D2I+J+1− . First, the SSC establishes

the connection for the queue-length processes. Then we can apply Puhalskii’s [27] first-passage-time argument toextend the result from the queue lengths to treat the waiting times as well; see the Corollary in [27], §13.7 of [31](especially Theorem 13.7.4) and Corollary B.3 in the appendix of [17]. Some care is needed in the applying thefirst-passage-time argument in the model with abandonments. The complete detailed argument follows the proofof Theorem 4.1 in Talreja and Whitt [29] involving carefully constructed lower and upper bounds. We omit thecomplete argument and refer the reader to [29]. We conclude this proof by noting that equation (6) has a unique(possibly weak) solution by virtue of established results for one-dimensional SDE’s; see e.g. Remark 5.5.19 andExercise 5.5.38 in [22].

Proof of Corollary 4.2. The proof follows an interchange-of-limits argument, following [21]. It is easy to check(using for example Browne and Whitt [7]) that, with the conditions of the corollary, the diffusion process XΣ(t),as given in Theorem 4.1, has a unique stationary distribution coinciding with the distribution of XΣ(∞). Now, byProhorov’s Theorem the tight sequence Xλ

Σ(∞) has a convergent subsequence Xλk

Σ (∞). Let Xλk

Σ (0) be distributedas Xλk

Σ (∞). Then Xλk

Σ (t) is a strictly stationary process and since we already proved that Xλk

Σ (t) ⇒ XΣ(t),XΣ(t) will be a process with XΣ(0) having the distribution of the limit of Xλk

Σ (∞). However, since Xλk

Σ (t) isstationary for each k, so is XΣ(t). Hence, the limit of Xλk

Σ (∞) must be the unique stationary distribution ofXΣ(t). The same argument applies to any convergent subsequence and hence the sequence Xλ

Σ(∞) itself mustconverge to this limit (see Theorem 2.3 in [5]). The convergence of the moments now follows from uniformintegrability. Since Qλ

i (∞) ≤ QλΣ(∞) almost surely, the sequence Qλ

i (∞) is also uniformly integrable for alli ∈ I. Hence, E[Qλ

i (∞)] → E[pi([XΣ(∞)]+)]. Recalling that λi = aiλ, and using Little’s law, we obtainE[Wλ

i (∞)] ⇒ (1/ai)E[pi([XΣ(∞)]+)].

Proof of Theorem 4.3. The proof is very similar to that of Theorem 4.1 and we only outline the differences.First, similarly to Lemma 7.1, one proves that

(Mλ1 (t), . . . , Mλ

I (t)) ⇒√

2aB(t), in DI , as λ →∞, (81)


where√

2aB(t) ≡ (√

2a1B1(t), . . . ,√

2aIBI(t)) and Bi(t), i ∈ I are independent Brownian motions. Us-ing equation (65) and replacing Zλ

i (t) = Xλi (t) − Qλ

i (t), one then applies the SSC from Theorem 3.1 and theContinuous Mapping Theorem, as in the proof of Theorem 4.1, to get the required convergence.

Proof of Theorem 4.4. Again, the proof is very similar to that of Theorem 4.1 and we only outline the differences.First, as in the proof of Theorem 4.3, one proves that (81) holds, but here with Mλ

i (t), i ∈ I as defined in(75). Using the representation in (77), one then applies the SSC from Theorem 3.1 and the Continuous MappingTheorem, as in the proof of Theorem 4.1, to get the required convergence. In applying the continuous mappingtheorem we use the linearity of the mapping G(·, ·) on the domain DG; see §5.

Proof of Lemma 7.1. Let MλA(t) =

(Mλ

A1(t), . . . , Mλ

AI(t)

), Mλ

S (t) =(Mλ

1 (t), . . . , MλJ (t)

), and Mλ

R(t) =(Mλ

R1(t), . . . , Mλ

RI(t)

). Then, we prove that

(Mλ

A(t), MλS (t), Mλ

R(t))⇒ (√

aBA(t),√

µνBλS(t), 0

), in D2I+J , as λ →∞. (82)

Here BA(t), BS(t) are, respectively, I and J independent Brownian motions. Also a := (a1, . . . , aI), µν :=(µ1ν1, . . . , µJ νJ) and the square-root and the vector product are interpreted componentwise. Also, 0 in (82) is the0 vector in RI . Recalling that

MλΣ =

∑

i∈IMλ

Ai−

∑

j∈JMλ

j −∑

i∈IMλ

Ri,

the result of the lemma then follows from the continuity of the addition operator under continuous limits (seeTheorem 12.7.1 in [31]).

We turn, then, to the proof of (82). The proof is based on a functional central limit theorem for Poisson processesand on a random-time change argument. Specifically, recall that

MλAi

(t) =Ai(λit)− λit√

λ, Mλ

j (t) =Sj

(µj

∫ t

0Zλ

j (s)ds)− µj

∫ t

0Zλ

j (s)ds√

λ, i ∈ I, j ∈ J ,

and

MλRi

(t) =Ri

(θi

∫ t

0Qλ

i (s)ds)− θi

∫ t

0Qλ

i (s)ds√

λ, i ∈ I.

Define

MλAi

(t) =Ai(λt)− λt√

λ, Mλ

j (t) =Sj (λt)− λt√

λ, i ∈ I, j ∈ J ,

and

MλRi

(t) =Ri (λt)− λt√

λ, i ∈ I,

and let MλA(t), Mλ

S (t) and MλR(t) be the corresponding vector valued processes. Then, as Ai(·), Ri(·) and Si(·)

are independent unit-rate Poisson processes, we have that

(MλA(t), Mλ

S (t), MλR(t)) ⇒ (BA(t), BS(t), BR(t)), in D2I+J , as λ →∞,

where BA and BR are independent I-dimensional standard Brownian motions and BS is a J-dimensional standardBrownian motion (see for example Theorem 5.1 in [25]).

The last step of the proof is to apply a random-time-change argument. Let

ΨλSj

(t) :=µj

∫ t

0Zλ

j (s)ds

λ, j ∈ J , and Ψλ

Ri(t) :=

θi

∫ t

0Qλ

i (s)ds

λ, i ∈ I,

and ΨλAi

(t) := λi/λt = ait, i ∈ I. Let ΨλA, Ψλ

S and ΨλR be the corresponding vector valued processes. By

Corollary 6.5, we have that Iλj (t) and Qλ

i (t) are stochastically bounded process for all i ∈ I and j ∈ J . Thisimplies that (

Iλ1 (t)λ

, . . . ,IλJ (t)λ

,Qλ

1 (t)λ

, . . .Qλ

I (t)λ

))⇒ η, in DI+J , as λ →∞,


where η(t) ≡ (0, 0, . . . , 0) (see for example Lemma 5.10 in [25]). In particular, since Zλj (t) = Nλ

j − Iλj (t), we

have that (Zλ

1 (t)λ

, . . . ,Zλ

J (t)λ

,Qλ

1 (t)λ

, . . .Qλ

I (t)λ

))⇒ η′, in DI+J , as λ →∞,

where η′(t) ≡ (µ1ν1, . . . , µj νj , 0, . . . , 0). We can then apply the continuous mapping theorem with the integralmapping

(x1, . . . , xj , y1, . . . , yI) 7→(

µ1

∫ t

0

x1(s)ds, . . . , µJ

∫ t

0

xJ(s)ds, θ1

∫ t

0

y1(s)ds, . . . , θI

∫ t

0

yI(s)ds

),

to show that (Ψλ

A(t), ΨλS(t), Ψλ

R(t)) ⇒ (a, µνt, 0) , in DI+J , as λ →∞.

Finally, applying the random-time-change theorem (see Theorem 13.2.1 in [31]), we conclude that(Mλ

A

(Φλ

A(t)), Mλ

S

(Φλ

S(t)), Mλ

R

(Φλ

A(t))) ⇒ (√

aBA(t),√

µνBλS(t), 0

), in D2I+J , as λ →∞.

Since, by definition,(Mλ

A(t), MλS (t), Mλ

R(t))

=(Mλ

A

(Φλ

A(t)), Mλ

S

(Φλ

S(t)), Mλ

R

(Φλ

A(t)))

,

the proof is complete.

8. Proofs of the Results in §5

Proof of Lemma 5.1. Fix the sample paths of Ai(·), i ∈ I, Si,j(·), i ∈ I, j ∈ J and Ri(·), i ∈ I. Wewill show that (XGQIR(t), ZGQIR(t)) and (XQIR(t), ZQIR(t)) then have the same sample paths. This, in turn,establishes equivalence in distribution.

Recall that j∗ is the only agent type with |I(j∗)| > 1. Fix the vector v = ej∗ and note that, by definition,∑i∈I Zλ

i,k(t) = 0, for every k 6= j∗ (see equations (11) and (13)). Moreover, since by condition C-3, the set I(k)consists of a single class for all k 6= j∗, we have that Zλ

i,k(t) = 0, for all k 6= j∗ and i = I(k). This also impliesthat −Zλ

i,k(t) = Iλk ≥ 0, for k 6= j∗ and i = I(k). In particular, for all t ≥ 0 and all k 6= j∗ and i = I(k) we have

that(Zλ

i,k(t)− Zλi,k(t))+ = Iλ

k (t). (83)

Now, fix a class i, with Qλi (t) = 0. Then, as Zλ

i,k(t) = 0, for all k 6= j∗, we have that

Zλi,j∗(t) = Xλ

i (t) ≤ Xλi (t)−

∑

k 6=j∗Zλ

i,k(t) = Zλi,j∗(t),

and in particular that(Zλ

i,j∗(t)− Zλi,j∗(t))

+ = 0. (84)

Combining (83) and (84), we see that, upon arrival of a class-i customer, if there are any idle agents in pool k 6= j∗

(and in particular Qλi (t−) = 0), then the decision rule of GQIR is equivalent to routing the customer to agent pool

k with k ∈ argmaxk∈J(i),k 6=j∗ Iλk (t). If the only idle agents are in pool j∗, then route the customer to pool j∗.

On the other hand, under condition C-3, QIR is such that, whenever Ij∗(t) > 0, we must have Qλi (t) = 0, i ∈ I,

and IλΣ(t) = [Xλ

Σ(t)]−, by the definition of XλΣ(t). Hence, whenever Iλ

j∗(t) > 0, we also have Iλj∗(t) ≤ Iλ

Σ(t) =[Xλ

Σ(t)]−. In particular,

j∗ = argmaxj∈J(i),Iλ

j (t)>0

Iλj (t)− [Xλ

Σ(t)]−vj

([Xλ

Σ(t)]−)

,

only if Iλk (t) = 0 for all k 6= j∗. Evidently then, the decision rule under QIR reduces to the one given above for

GQIR; i.e., route the customer to agent pool k with k ∈ argmaxk∈J(i),k 6=j∗ Iλk (t). Route the customer to pool j∗

only if the only idle agents are in pool j∗.

We now show that the decision rule in a service completion epoch is the same under both controls. Trivially, thedecision rules are the same under QIR and GQIR when the service completion is in agent pool k 6= j∗ because, bycondition C-3, these pools serve a single queue each. Now consider a service completion epoch in pool j∗. Sincev = e∗j , for all i such that Qλ

i (t) > 0, we must have that Zλi,k(t) = 0 (otherwise Qλ

i (t) = 0) for all k 6= j∗ and,


in particular, Zi,j∗(t) = Xλi (t) − Qλ

i (t) −∑k 6=j∗ Zλ

i,k(t) = Xλi (t) − Qλ

i (t). Using equations (11) and (13) asbefore, we also have Zλ

i,k(t) = 0 for all k ∈ J(i), k 6= j∗, and Zi,j∗(t) = Xλi (t)− [Xλ

Σ(t)]+pi([XλΣ(t)]+). Hence,

upon service completion in pool j∗,

i ∈ argmaxk:Qλ

k(t)>0

(Qk(t)− [XλΣ(t)]+pk([Xλ

Σ(t)]+)

if and only ifi ∈ argmax

k:Qλk(t)>0

(Zλk,j∗(t)− Zλ

k,j∗(t))+.

Since we use the same decision rule for breaking ties, both controls will make the same decision in a servicecompletion epoch. We have shown equivalence of the decision rules of QIR and GQIR under condition C-3. Byinduction, This in turn implies that we will have the same sample paths under both controls.

Proof of Theorem 5.1. We will explain why SSC under these conditions is a consequence of Proposition 1 inAtar [4]. First note that the results in [4] are given for a Markov control policy (see Definition 4 there) given by afunction h := (h1, h2), where hi : RI 7→ U, i = 1, 2,

U := (u, v) ∈ RI+J : ui, vj ≥ 0, i ∈ I, j ∈ J ,∑

i∈Iui =

∑

j∈Jvj = 1 ,

and the functions hi are assumed to be locally Holder continuous away from 0 (with 0 being here the origin ofRI ); see part (iii) of Theorem 2 in [4]. Clearly, these conditions apply to a pair of admissible state-dependent ratiofunctions p and v, as defined in Definition 2.2. Indeed, with two such functions p and v, we can define h for x ∈ Rby

h(x) :=

(p

([∑

i∈Ixi]+

), v

([∑

i∈Ixi]−

)),

and position ourselves in the framework of [4]. To be able to apply the result [4] directly, one additional observationis required. The control proposed in [4] is not precisely GQIR as defined in 5.1. Instead, it is a modification ofthis control in which the function h(·) is replaced by a different function for all times that are greater than a certainstopping time; see equation (56) in [4]. However, a careful reading of [4] reveals that, while this modificationis required for the large-time-estimates in Proposition 2 of [4], it is not used in the proof of Proposition 1 in [4].Consequently, Proposition 1 in [4] is valid for GQIR as defined in 5.1 and it implies the desired SSC result. Indeed,by translation of notation, the first statement in part (iv) of Proposition 1 in [4] corresponds to the statement

sups≤u≤t

∑

i∈I

∣∣∣Qλi (u)− [Xλ

Σ(u)]+pi

([Xλ

Σ(u)]+)∣∣∣ +

∑

j∈J

∣∣∣Iλj (u)− [Xλ

Σ(u)]−vj

([Xλ

Σ(u)]−)∣∣∣

⇒ 0; (85)

see the definition of Jn(t) in equations (74), (52) and (53) of [4]. The second part of (iv) corresponds to thestatement

sups≤u≤t

Qλ

Σ(u) ∧ IλΣ(t)

⇒ 0; (86)

see the definition of Mn(t) in equation (61) of [4]. But, by the definition of XλΣ(t) here, equations (85) and (86)

combined imply the state-space-collapse conclusion in Theorem 5.1. Finally, equation (5) follows directly frompart (i) of Proposition 1 in [4].

Acknowledgments. This research is based on the first author’s doctoral dissertation at Columbia University.The second author was supported by NSF Grant DMI-0457095.

References

[1] M. Armony, Dynamic routing in large-scale service systems with heterogenous servers, Queueing Systems51 (2005), no. 3-4, 287–329.

[2] M. Armony and C. Maglaras, Contact centers with a call-back option and real-time delay information, Oper.Res. 52 (2004), no. 4, 527–545.

[3] , On customer contact centers with a call-back option: customer decisions, routing rules and systemdesign, Oper. Res. 52 (2004), no. 2, 271–292.


[4] R. Atar, Scheduling control for queueing systems with many servers: Asymptotic optimality in heavy traffic,Ann. Appl. Prob. 15 (2005), no. 4, 2606–2650.

[5] P. Billingsley, Convergence of probability measures, J. Wiley & Sons, New York, 1968.[6] M. Bramson, State space collapse with applications to heavy-traffic limits for multiclass queueing networks,

Queueing Systems 30 (1998), 89–148.[7] S. Browne and W. Whitt, Piecewise-linear diffusion processes, Advances in Queueing: Theory, Methods, and

Open Problems (J. H. Dshalalow, ed.), CRC Press, Inc., 1995, pp. 463–480.[8] A. Budhiraja and C. Lee, Stationary distribution convergence for generalized jackson networks in heavy

traffic, Math. Oper. Res. (2008), Forthcoming.[9] J. G. Dai, On positive Harris recurrence of multiclass queueing networks: A unified approach via fluid limit

models, Ann. Appl. Prob. 5 (1995), 49–77.[10] J.G. Dai and T. Tezcan, Dynamic control of N-systems with many servers: asymptotic optimality of a static

priority policy in heavy traffic, Oper. Res. (2008), Forthcoming.[11] , Optimal control of parallel server systems with many servers in heavy traffic, Queueing Systems 59

(2008), no. 2, 95–134.[12] , State space collapse in many server diffusion limits of parallel server systems, Math. Oper. Res.

(2008), Forthcoming.[13] A.W. Van der Vaart, Martingales, diffusions and financial mathematics - lecture notes, Available at:

http://www.math.vu.nl/sto/onderwijs/mdfm/, 2006.[14] D. Gamarnik and A. Zeevi, Validity of heavy traffic steady-state approximations in generalized Jackson net-

works, Ann. Appl. Prob. 16 (2006), 56–90.[15] N. Gans, G. Koole, and A. Mandelbaum, Telephone call centers: Tutorial, review, and research prospects,

Manufacturing & Service Operations Management 5 (2003), no. 2, 79–141.[16] O. Garnett, A. Mandelbaum, and M. Reiman, Designing a call center with impatient customers, Manufactur-

ing & Service Operations Management 4 (2002), no. 3, 208–227.[17] I. Gurvich, M. Armony, and A. Mandelbaum, Service-level differentiation in call centers with fully flexible

servers, Management Science 54 (2008), no. 2, 279–294.[18] I. Gurvich and W. Whitt, Asymptotic optimality of queue-ratio routing for many-server service systems, Work-

ing Paper. Columbia University, New York, NY. (2007).[19] , Service-level differentiation in many-server service systems via queue-ratio routing, Working Paper.

Columbia University, New York, NY. (2007).[20] , Scheduling flexible servers with convex delay costs in many-server service systems, Manufacturing

& Service Operations Management (2008), Forhcoming.[21] S. Halfin and W. Whitt, Heavy-traffic limits for queues with many exponential servers, Oper. Res. 29 (1981),

no. 3, 567–588.[22] I. Karatzas and S.E. Shreve, Brownian motion and stochastic calculus, 2nd ed., Springer-Verlag, New York,

1991.[23] R.Sh. Lipster and A.N. Shirayev, Theory of martingales, Kluwer Acad. Publ., Boston, 1989.[24] A. Mandelbaum and S. Stolyar, Scheduling flexible servers with convex delay costs: Heavy-traffic optimality

of the generalized cµ-rule, Oper. Res. 52 (2004), 836–855.[25] G. Pang, R. Talrejea, and W. Whitt., Martingale proofs of many-server heavy-traffic limits for markovian

queues, Probability Surveys 4 (2007), 193–267.[26] P. Protter, Stochastic integration and differential equations - a new approach, Springer-Verlag, New York

(1992).[27] A. Puhalskii, On the invariance principle for the first passage time, Math. Oper. Res. 19 (1994), no. 4, 946–

954.[28] H.L. Royden, Real analysis, 2nd ed., Macmillan, London, 1968.[29] R. Talreja and W. Whitt, Heavy-traffic limits for waiting times in many-server queues with abandonments,

Working paper, Columbia University, New York, NY, 2008.[30] T. Tezcan, Optimal control of distributed parallel server systems under the halfin and whitt regime, Math.

Oper. Res. (2008), Forthcoming.


[31] W. Whitt, Stochastic-process limits: An introduction to stochastic-process limits and their application toqueues, Springer-Verlag, New York. (2002).

[32] R.J. Williams, Diffusion approximations for open multiclass queueuing networks: Sufficient conditions in-volving state space collapse, Queueing Systems 30 (1998), no. 1-2, 27–88.

Queue-and-Idleness-Ratio Controls in Many-Server …ww2040/FQR_SSC_MOR.pdfkey step in providing...

Documents

Transcript of Queue-and-Idleness-Ratio Controls in Many-Server …ww2040/FQR_SSC_MOR.pdfkey step in providing...