Stochastic optimization of large-scale complex systems › ws › files › 13620787 ›...

Stochastic optimization of large-scale complex systems

Citation for published version (APA):Sanders, J. (2016). Stochastic optimization of large-scale complex systems. Eindhoven: Technische UniversiteitEindhoven.

Document status and date:Published: 28/01/2016

Document Version:Publisher’s PDF, also known as Version of Record (includes final page, issue and volume numbers)

Please check the document version of this publication:

• A submitted manuscript is the version of the article upon submission and before peer-review. There can beimportant differences between the submitted version and the official published version of record. Peopleinterested in the research are advised to contact the author for the final version of the publication, or visit theDOI to the publisher's website.• The final author version and the galley proof are versions of the publication after peer review.• The final published version features the final layout of the paper including the volume, issue and pagenumbers.Link to publication

General rightsCopyright and moral rights for the publications made accessible in the public portal are retained by the authors and/or other copyright ownersand it is a condition of accessing publications that users recognise and abide by the legal requirements associated with these rights.

• Users may download and print one copy of any publication from the public portal for the purpose of private study or research. • You may not further distribute the material or use it for any profit-making activity or commercial gain • You may freely distribute the URL identifying the publication in the public portal.

If the publication is distributed under the terms of Article 25fa of the Dutch Copyright Act, indicated by the “Taverne” license above, pleasefollow below link for the End User Agreement:www.tue.nl/taverne

Take down policyIf you believe that this document breaches copyright please contact us at:[email protected] details and we will investigate your claim.

Download date: 03. Jul. 2020

https://research.tue.nl/en/publications/stochastic-optimization-of-largescale-complex-systems(c6a4fd0e-4cae-44dc-9447-afecdc2cb36f).html

Stochastic Optimization ofLarge-Scale Complex Systems

This work was financially supported by the European Research Council (ERC)in the form of the ERC starting grant CRITIQUEUE.

© Jaron Sanders, 2016Stochastic Optimization of Large-Scale Complex Systems, by Jaron Sanders

A catalogue record is available from the Eindhoven University of TechnologyLibraryISBN: 978-90-386-4005-1

Printed by Gildeprint Drukkerijen, Enschede

Stochastic Optimization of

Large-Scale Complex Systems

P R O E F S C H R I F T

ter verkrijging van de graad van doctor aan deTechnische Universiteit Eindhoven, op gezag van derector magnificus, prof.dr.ir. F.P.T. Baaijens, voor een

commissie aangewezen door het College voorPromoties in het openbaar te verdedigen

op donderdag 28 januari 2016 om 16.00 uur

door

Jaron Sanders

geboren te Eindhoven

Dit proefschrift is goedgekeurd door de promotoren en de samenstelling van depromotiecommissie is als volgt:

voorzitter: prof.dr.ir. O.J. Boxma1e promotor: prof.dr. J.S.H. van Leeuwaarden2e promotor: prof.dr.ir. S.C. Borstleden: prof.dr.ir. P.F.A. Van Mieghem (TUD)

prof.dr. R. Fernandez (UU)prof.dr.ir. P.P.A.M. van der Schootprof.dr. I.S. Popprof.dr. A.P. Zwart (CWI)

Het onderzoek dat in dit proefschrift wordt beschreven is uitgevoerd in overeen-stemming met de TU/e Gedragscode Wetenschapsbeoefening.

Dankwoord

Het is ongeveer vijf jaar geleden dat ik op mijn werkplek zat tijdens mijn be-drijfsstage van mijn studie natuurkunde, toen er onverwachts een e-mail binnen-kwam van een wiskundedocent. Hij herinnerde mij nog van het vak “simulatie”en vroeg of ik op zoek was naar een afstudeerproject. Ik vermoed nu eigenlijkdat het vooral mijn scherp verwoorde evaluatie was die in Johan’s geheugengegrift stond. Ik stapte al snel met Johan, Sem, en Servaas in zee voor een afstu-deerproject en na de succesvolle afronding van mijn masterscriptie stroomde ikdoor naar het vierjarig promotieavontuur onder begeleiding van Johan en Sem.

Sem, Johan, jullie begeleiding is fantastisch: wij promovendi worden opacademisch vlak klaargestoomd en jullie dragen daarnaast sterk bij aan onzepersoonlijke ontwikkeling. De onuitputtelijke energie, onovertroffen ambitiesen enerverend enthousiasme zijn maar enkele aspecten uit een lange lijst dieik nooit zal vergeten. Bedankt voor alle unieke kansen en lessen die onderdeeluitmaakten van onze samenwerking in de afgelopen vijf jaar.

Het was verder een waar genot om met mijn andere coauteurs samen tewerken. Guido, jij hebt mijn wiskundige limiet als geen ander geëxpandeerd.Servaas, Edgar, Rick, jullie passie om als team nieuwe, bruikbare methodes bin-nen de natuurkunde te ontwerpen is aanstekelijk. Matthieu, Paola, the way weachieved a collaboration across international borders and a vast ocean, afteronly one visit by only one of you, was truly astonishing.

Niet alles draaide tijdens mijn promotie om de wetenschap. Ook vriendenen collega’s speelden een belangrijke rol.

Stijn, jij bent die vriend waarmee je over alles kunt praten en je bent iemanddie altijd openstaat voor nieuwe en leuke activiteiten. Dennis, Marc, ik kan meniemand beter bedenken om inhoudelijk te discussiëren over maatschappelijkeonderwerpen en om cultuur te snuiven bij de film. Bas, Chiel, Daniël, Jeroen,Karel, Robbert, ik heb genoten van onze talloze spannende en vaak komische

v

sessies D&D aan de keukentafel.The wonderfully energetic atmosphere in our research group in Eindhoven

is something I will not forget. All staff members are truly supportive people andeach of them has so many interesting stories to learn from, and all of the PhDsare remarkable persons with whom I gladly shared the four-year experience.But because of the great successes of our research group, the list of people thatdeserve a mention has grown close to infinity – so please, forgive me for notwriting your name explicitly. I wish all of you the very best in life.

Ook vele anderen verrijkten mijn leven. Mijn saxofoonspel is op fantasti-sche wijze gegroeid door Xavier’s hulp en ook andere dirigenten en muzikantenhebben hier een steentje aan bijgedragen. De avonden en festivals die ik hebbezocht met dansers hebben mij ook een andere vorm van kunst laten ervarendan dat ik de wiskunde vind. Het onderdeel uitmaken van enkele besturen encommissies, zoals in het bijzonder de PhD Council van Wiskunde & Informatica,waren waardevolle ervaringen.

Mijn ouders en zusje, Hans, Elly en Natasja, waren altijd zeer behulpzaamen boden daarmee een basis om op voort te bouwen. Ongelooflijk onzelfzuchtighebben zij het beste voor met elkaar, net zoals de rest van mijn familie overi-gens: halverwege mijn promotietraject wenste Martien mij bijvoorbeeld met eenlaatste beetje energie, en een laatste woord, “Succes.”

Ik ben jullie allen erg dankbaar.

Contents

1 Introduction 11.1 Interacting particle systems.. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21.2 Stochastic optimization .. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 101.3 Quality-and-Efficiency-Driven systems .. . . . . . . . . . . . . . . . . . . . . . . . . . . 161.4 Scalable admission control and revenue structure .. . . . . . . . . . . . . . 181.5 Overview of this thesis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21

I Complex systems 25

2 Online optimization using Markov processes 272.1 Introduction .. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 292.2 Algorithm ... . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 342.3 Applications .. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 392.4 Proofs .. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 502.5 Conclusions .. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 65

3 Wireless network control of interacting Rydberg atoms 753.1 Introduction .. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 773.2 Model comparison .. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 783.3 Rydberg crystals. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 813.4 Control algorithm ... . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 833.5 Conclusion .. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 87

4 Sub-Poissonian statistics in ultracold Rydberg gases 914.1 Introduction .. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 934.2 Fitting the Erdös–Rényi (ER) graph ... . . . . . . . . . . . . . . . . . . . . . . . . . . . . 944.3 Spatial dynamics and jamming limits . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 954.4 Exploration process and fluid limits . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 964.5 Comparison to random sequential adsorption.. . . . . . . . . . . . . . . . . . . 98

vii

4.6 Mandel Q parameter .. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 994.7 Comparison to measurements .. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1004.8 Conclusion .. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 102

5 Scaling limits for exploration algorithms 1075.1 Introduction .. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1095.2 Random adsorption under a homogeneous relation .. . . . . . . . . . . . 1105.3 Functional law of large numbers .. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1115.4 Diffusion approximations with errors bounds .. . . . . . . . . . . . . . . . . . . 1155.5 Law of large numbers (LLN) and central limit theorem (CLT)

for the hitting time ... . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1175.6 Conclusions .. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 126

II Large-scale systems 129

6 Scaled control in the QED regime 1316.1 Introduction .. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1336.2 Many-server systems with admission control . . . . . . . . . . . . . . . . . . . . . 1356.3 Quality-and-Efficiency-Driven (QED) scaled control. . . . . . . . . . . . . 1386.4 QED approximations for global control . . . . . . . . . . . . . . . . . . . . . . . . . . . 1436.5 QED approximations for local control . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1496.6 Conclusions and outlook .. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 154

7 Optimal Admission Control with QED-Driven Revenues 1657.1 Introduction .. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1677.2 Revenue maximization framework .. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1727.3 Properties of the optimal threshold .. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1827.4 Optimality of threshold policies .. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1957.5 Conclusions and future perspectives.. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 204

8 Optimality gaps in asymptotic dimensioning 2158.1 Introduction .. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2178.2 Model description.. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2198.3 General revenue maximization.. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2228.4 Approaches to asymptotic dimensioning .. . . . . . . . . . . . . . . . . . . . . . . . . 227

Bibliography 235

Summary 245

About the author 247

CHAPTER 1Introduction

Large-scale complex systems ranging from communication networks and datacenter applications to content dissemination systems and physical interactionprocesses, are all influenced by randomness. In spite of their uncertain behaviorand potentially erratic performance, modern society has grown increasingly de-pendent on such systems. Their inherently complex designs, the driving stochas-tic processes, and the limited availability of resources all pose great societal andscientific challenges.

Our goal is therefore to develop analysis techniques and optimization pro-cedures that are broadly applicable to large-scale complex systems. This is alsowhy this thesis focuses on models of interacting particle systems, stochasticnetworks, and service systems, which are all large systems that demonstrateintriguingly complex behavior. At the same time their models are quite general,and their steady-state behaviors can all be described with explicit formulae.

The models, problems, and mathematics discussed throughout this thesishave been grouped naturally into two parts. The first part deals with problemsof analysis and optimization in predominantly complex systems, as opposed tosystems where size is the primary concern. The second part focuses on asymp-totic analysis and revenue maximization in very large systems.

The introduction will now provide an overview of the thesis’s content: it de-scribes many intricacies of different large-scale complex systems, and illustratesour approaches of analysis and optimization.

1

1. IN T R O D U C T I O N

Part I: Complex systems

Part I of this thesis deals with problems of analyzing and optimizing reversibleMarkov processes. The focus is on interacting particle systems and stochastic net-works, two classes of systems that share the feature of simple local interactionsthat together yield complex global system behavior.

1.1 Interacting particle systems

First, we will present a model for interacting particle systems [11, 12]. We discussthe model’s main features in terms of structure, dynamics, and interactions, andthen explain the resulting anticipated complex behavior of these models in thecontext of wireless networks [13–15] and Rydberg gases [3, 4, Chapters 3, 4].

Consider a system with n particles labeled i = 1, . . . , n. Each particle i has aposition p i , and can be in either state or . These particles can represent any-thing with at least two states, including technological devices (e.g. transmitters)and physical objects (e.g. atoms). We will also assume that there exists a binaryrelation between the particles, to which we associate a relation graph in whichvertices are particles such that two particles are connected by an edge if they arerelated. If we denote by X i ∈ , the state of particle i, the configuration ofthe whole system can be described using a vector X = (X1, . . . , Xn)T. Figure 1.1shows an example relation graph.

The relation graph can encode different physical quantities. For example,suppose that we would like to model a gas that is made up of molecules. Thenumber of molecules per unit volume in the gas is called the density, and thedensity can be modeled using the number of edges within the relation graph.Specifically, the density is low in a rarefied gas (so the molecules are far apartand have few interactions), which we can represent using only a few edges. Onthe other hand in a dense gas there are many molecules per unit volume, whichmeans that molecules are bunched close together. This can be described usingrelatively many edges.

We assume that the states of the particles are dynamic and change as timeprogresses, either by their own choosing or through external influences. Wemodel these dynamics by assuming that every particle i changes from state to

after an exponentially distributed amount of time with mean 1/λi . Similarly,we will assume that particle i changes back again from state to after an

2

1.1. Interacting particle systems

Figure 1.1: Interacting particle system with relation graph. The vertices repre-sent the particles that are either in state or . The edges capture the relationsbetween particles.

exponentially distributed amount of time with mean 1/µi . Interactions betweenparticles can then be incorporated by restricting the dynamics to a set of feasibleconfigurations Ω ⊆ , n. The type of interactions that we will consider areblockade effects: once a particle is in state , all other particles in state withinEuclidean distance r are prevented from also entering state . The set of allfeasible configurations Ω then consists only of those configurations in which notwo particles within a distance r of each other are both in state ,

Ω= x ∈ , n|¬(x i , x j = )∀i 6= j:‖p i−p j‖<r. (1.1)

Under these assumptions, the system’s configuration changes dynamically overtime, and we will use X(t) to denote the configuration at time t. Moreover,X(t)t≥0 constitutes a reversible Markov process [16, 17].

The simplest particle system with nontrivial blockade effects is a three-particle system in which particles are positioned on a line and block only theirdirect neighbors. The set of feasible configurations for this particle system isgiven by

Ω= , , , , . (1.2)

This particle system already exhibits an intriguing feature, even though thestate space in (1.2) consists of only a few feasible configurations: the outer

3


particles can both block the inner particle, and therefore also influence eachother by providing more opportunities to enter state . Local interactions canthus reverberate through the particle system, and even though particles areseparated by a significant distance, the particles might still influence each other.Furthermore, because many such smaller particle systems can be the componentsof one large particle system, think of Figure 1.1, we can start to imagine howthe simple local interactions quickly lead to complex global behavior.

We will now illustrate more concretely how the dynamics result in complexbehavior of the interacting particle system as a whole. Specifically, we will ex-plain that the interacting particle system can be used to model wireless networksas well as Rydberg gases, and discuss different complex global phenomena thatoccur in these systems and are captured by the model.

1.1.1 Wireless networks

Random-access algorithms are widely used to implement distributed schedulingin wireless networks, and these algorithms allow transmitters to decide for them-selves when to transmit based on locally available information only. The firstrandom-access algorithm was implemented in a computer networking systemdeveloped at the University of Hawaii called Additive Links On-line Hawaii Area(ALOHAnet) [18]. Transmitters in ALOHAnet would, after finishing a transmis-sion, remain silent for some random time before activating again. This backoffmechanism reduces simultaneous activity of nearby transmitters, lessening theprobability that interference occurs.

Random-access algorithms have since the creation of ALOHAnet been un-der constant development, and we will focus on the more recent improvementcarrier-sense multiple-access (CSMA) [19] that has a carrier-sensing mechanismbesides the backoff mechanism. Every transmitter using CSMA sets a back-offtimer randomly every time it finishes a transmission and remains inactive untilthe timer rings, and inactive transmitters continuously monitor their surround-ings and freeze the back-off timer until the measured interference drops below acertain level. At the moment that the measured interference has dropped belowa certain level, the transmitter resumes the back-off count down.

When wireless networks are saturated, meaning that transmitters alwayshave data to transmit, the dynamics generated by the CSMA algorithm areidentical to those of the interacting particle system [20]. The terminology in

4


this case is that 1/µi denotes the mean transmission time of transmitter i, and1/λi the mean back-off timer of transmitter i. The interference constraints canbe modeled using a relation graph [21], when given the interpretation thattransmitters correspond to vertices, and with the understanding that neighboringvertices correspond to transmitters that cannot simultaneously transmit.

The interacting particle system can be used to analyze the long-term averagebehavior of the activity process of the transmitters. Recall that our assumptionson the dynamics and interactions imply that the process X(t)t≥0 is a reversibleMarkov process, which implies in turn that the steady-state probability of observ-ing the system in configuration x ∈ Ω is given by the product-form equilibriumdistribution

πx = limt→∞P[X(t) = x ] =

1Z

n∏i=1

λi

µi

1[x i= ], (1.3)

where the normalization constant is given by

Z =∑y∈Ω

n∏j=1

λ j

µ j

1[y j= ]. (1.4)

Here, x i denotes the i-th element of the vector x = (x1, . . . , xn)T.We can derive various steady-state performance measures from the equilib-

rium distribution in (1.3). In particular, we can calculate the throughput θ ofindividual transmitters – the rate of successful message delivery over a commu-nication channel. Mathematically, the throughput that transmitter i attains canbe calculated using the linear combination [14]

θi = µi limt→∞P[X i(t) = ] = µi

∑x∈Ω

1[x i = ]πx , (1.5)

which provides with (1.3) a useful analytical expression to evaluate the perfor-mance of wireless networks.

By comparing the throughput of different transmitters, we can determinewhether long-term unfairness occurs in a wireless network. That is, if we seethat θ j θi for some pair j 6= i, transmitter j is able to transmit on averagemuch more than transmitter i per time unit, which is unfair. When consideringa system in which three identical transmitters are positioned on a line and onlyblock their nearest neighbors, i.e. and λi = λ and µi = µ for i = 1, 2, 3, wefor example find after introducing the notation ρ = λ/µ that

θ1 = θ3 = µρ +ρ2

1+ 3ρ +ρ2 µ

ρ

1+ 3ρ +ρ2= θ2 (1.6)

5


whenever ρ 1. We see that the middle transmitter, numbered 2, then has amuch lower throughput than transmitters 1 and 3. This is known in the wirelessnetwork literature as the node-in-the-middle problem. Long-term unfairness, aswell as the node-in-the-middle problem, can be resolved using online optimiza-tion techniques [22], which we will discuss in Section 1.2.

Short-term unfairness is also an issue in wireless networks. To understandshort-term unfairness, consider a network that consists of four identical trans-mitters on a circle, i.e. and λi = λ and µi = µ for i = 1, 2, 3, 4. This networkhas two dominant configurations: one in which all odd transmitters are active

, and one in which all even transmitters active . Suppose without loss ofgenerality that both odd transmitters are active. As long as either odd trans-mitter is active, it is impossible for either even transmitter to activate. Beforean even transmitter can activate, both odd transmitters need to first deactivate,that is, the system has to reach the state first. We will now argue that inheavily-loaded systems that have λ µ, the time until the even transmittersget an opportunity to activate and subsequently transmit is large.

Let us consider the time between the first moment that the system enterseither dominant configuration, that is or , until the first moment the emptystate is reached. Suppose without loss of generality that the system is in theodd dominant configuration. What happens is that the system will enter eitherstate or and immediately jump back to a geometric number of times,before jumping successfully to . The probability of jumping from either ofor to the empty state is p = µ/(λ+µ), so that the hitting time of the emptystate τ is distributed as

(1.7) τd=

Geo(p)−1∑i=0

T 2µ

i + Tλi+ T 2µ

Geo(p) + TµGeo(p),

where each T xi denotes an exponentially distributed random variables with

mean 1/x . The expectation of the hitting time tends to infinity

(1.8) E[τ] =λ

µ

12µ+

1λ

+

12µ+

1µ→∞,

as λ/µ→∞, implying that in heavily-loaded systems it will take a long timebefore even transmitters get an opportunity to activate. In real-life networks,short-term unfairness depends intricately on the network structure, and oneexpects that the dominant configurations crucially govern the extent of theshort-term unfairness.

6


Long- and short-term unfairness are complicated phenomena that negativelyimpact the performance of wireless networks. In Section 1.2, we develop a classof algorithms that resolves the issue of long-term unfairness. The short-termunfairness complicates the operation of these algorithms, but can be overcomeby properly designing the algorithm. A detailed analysis of short-term unfairnesshowever is beyond the scope of this thesis, and we refer interested readers foran in-depth study to [23].

1.1.2 Rydberg gases

Rydberg gases are specially prepared gases that consist of ultracold atoms thatcan be either in a ground state , or in an excited state that has a high principalquantum number. Such an excited state is also called a Rydberg state, and when-ever an atom is in a Rydberg state we call the atom a Rydberg atom [24]. Thetransition between the ground state and Rydberg state is typically facilitated bythe absorption or emission of a photon [25]. An essential feature of these parti-cles is that once an atom is in the Rydberg state, the atom prevents neighboringatoms from reaching their Rydberg state [26, 27]. This blockade effect occursbecause excited atoms shift the energy levels of neighboring atoms, making itunlikely for neighboring atoms to absorb a photon and hence transition fromthe ground state to the Rydberg state.

Simulations and measurements between atoms in the Rydberg state at dif-ferent distances [28, 29] show that the blockade effect can be appropriatelymodeled using a hard blockade radius r. This observation motivated us to initi-ate a study on the relation between the Rydberg gas and the interacting particlesystem described in Section 1.1, see [1], and we found that if the transitionprocess is driven by a stochastic process, the dynamics of the Rydberg gas canbe modeled using the dynamics of the interacting particle system [3, Chapter 3].This correspondence opened up possibilities to understand complex phenom-ena that occur in Rydberg gases. For example, the dominant configurations thatcause unfairness in wireless networks also occur in Rydberg gases [29].

We will now discuss another complex phenomenon that physicists have ex-perimentally observed. Specifically, measurements reveal that the variance ofthe number of excited particles is smaller than the mean of the number of ex-cited particles [30–35]. When plotting a histogram of the measured number ofexcited particles, e.g. Figure 1.2, we see that the histogram has a characteristic

7


shape that we call a sub-Poissonian distribution. This differs substantially fromthe typical Poisson distribution that is generally observed in gases without stronginteractions. This phenomenon is caused by the blockade effect, and we wereable to study the occurrence of this sub-Poissonian distribution quantitativelyby studying jamming limits [4, Chapter 4].

0 10 20

number of Rydberg atoms

prob

abili

tyth

ata

cert

ain

num

ber

ofR

ydbe

rgat

oms

ism

easu

red

Figure 1.2: Histogram of the measured eventual number of Rydberg atoms [32,Fig. 4(a)]. The dashed line indicates a Poisson distribution that has the samemean as the measurement data. The continuous line is a plot of our analyticalformula that describes the sub-Poissonian distribution [4, Chapter 4].

Jamming limits [36] occur in the interacting particle system as follows. Sup-pose for simplicity that particles in state never transition back to , i.e. µi = 0for i = 1, . . . , n. This implies that any particle in state of which a neighbor hasalready entered state , will never transition to . We therefore say that theseparticles are blocked, and we will mark blocked particles as being in a third state. Then if the system initially starts in state X(0) = ( , . . . , )T, the transient

behavior of the interacting particle system is as follows.At a time T1 > 0, a first particle 1 enters state . Due to the blockade effect,

particle 1 will subsequently prevent all other particles within a radius r fromalso entering , and these particles are considered blocked . Later, at a time

8


T2 > T1, a second particle 2 enters state , which cannot be within distance rof particle 1. Particle 2 from that point onward also blocks particles within adistance r of itself. This process continues until some finite random time T∗ <∞when all particles are either blocked or in state . The stochastic process willthus eventually be absorbed in such a configuration, which we call a jamminglimit. The transient behavior and occurrence of a jamming limit are illustratedin Figure 1.3.

Figure 1.3: (left) A random first particle excites. (middle) Subsequently, randomsecond and third particles excite. (right) The process continues until all particlesare either blocked or excited, and the resulting state is a jamming limit.

This in fact describes a continuum random sequential process [37]. Unfortu-nately, apart from the one-dimensional variant, such processes are notoriouslydifficult to analyze due to the spatial correlations between particles. Our con-tribution is that we have instead considered large Erdös–Rényi (ER) randomgraphs [38] that retain the essential features of the blockade effect, which sim-plifies the problem and allows us to analyze the statistics of the jamming limit.An ER random graph can be generated by considering every pair of vertices oneby one, and drawing an edge between the pair of vertices with equal probability,independent of other edges. This solution allowed us to relate the mean andvariance of the number of excitations to the blockade effect using closed-formformulae [5, Chapter 5]. Even though our approach ignores spatial features ofthe problem such as the positions of the particles, our theoretical results providea good fit for the experimental results on Rydberg gases [4, Chapter 4], as canalso be seen in Figure 1.2.

9


1.2 Stochastic optimization

Now that we have seen how simple local interactions can lead to complex globalsystem behavior, we turn to the problem of optimizing complex systems. Thefocus will be shifted away from interacting particle systems, and we will startto place more emphasis on stochastic systems.

The optimization scheme we have developed is namely applicable to thebroad class of reversible Markov processes. Reversible Markov processes providea versatile framework for modeling a wide variety of stochastic systems, andcover a rich family of stochastic models such as loss networks [39, 40], openand closed queueing networks [17, 41], wireless random-access networks [14,42, 43] and various types of interacting particle systems [11, 44], including theones described in Section 1.1.

1.2.1 Online optimization of performance

For reversible Markov processes, key performance measures such as buffer oc-cupancies and loss probabilities can typically be expressed in terms of the sta-tionary distribution π of the Markov process. Reversible Markov processes haveproduct-form stationary distributions [16, 17], such as the distribution in (1.3),which can always be written in the exponential form [2, 9, 10, Chapter 2]

(1.9) π(r ) =1

Z(r )exp (Ar + b),

where A denotes a matrix, b a vector and Z(r ) the normalization constant.Here, r denotes the system parameters that can be controlled, typically serviceor arrival rates.

Our focus is not so much on evaluating the performance of the system forgiven parameter values, but rather on finding parameter settings r opt that opti-mize the performance or achieve an optimal trade-off between service level andcosts. The mathematical problem of interest can be formulated as finding

(1.10) r opt ∈ argminr

u(r ),

where u(r ) denotes an objective function that is expressed in terms of the sta-tionary distribution π(r ), and the parameter vector r .

Optimization problem (1.10) could in principle be solved using mathemati-cal programming approaches such as gradient-based schemes, i.e. by iteratively

10

1.2. Stochastic optimization

settingr [n+1] = r [n] − a[n+1]∇r u(r [n]), n ∈ N0, (1.11)

with the a[1], a[2], . . . denoting consecutive step sizes, and n enumerating theiterations. However, in addition to the usual convexity issues [45], difficultiesarise from the computational burden of calculating the stationary probabilities.For instance, the normalization constant Z(r ) can be NP-hard to evaluate [46].This severely complicates the evaluation of the objective function u(r ), as wellas the calculation of its gradient ∇r u(r ).

We therefore adopt a gradient approach which relies on measuring the em-pirical frequencies of the various states. For each (n+1)-st iteration we observethe stochastic process from a time t[n] until a time t[n+1], and we then calculateestimates for the gradient ∇r u(r ) based on the measured time fractions of thevarious states. Denoting such an approximation by G[n+1], we iteratively updatethe parameters according to

R[n+1] = R[n] − a[n+1]G[n+1]

, n ∈ N0, (1.12)

which can be interpreted as a stochastic gradient descent method [47, 48]. Notethat we have capitalized the parameter vector to indicate that we are now dealingwith random variables instead of deterministic variables.

This online optimization scheme can trace its roots to a method developed forwireless networks. The interacting particle system in Section 1.1.1 was studiedin order to understand, evaluate and optimize the performance of wireless net-works [14, 42, 43]. Subsequently, a distributed algorithm was developed thatwould slowly adjust the activation rate over increasing time intervals, so as tostabilize the throughput in a wireless network [22]. Utilizing techniques fromstochastic approximation, convergence and rate stability of the algorithm wasthen proven [49]. Our contributions [2, 9, 10, Chapter 2] essentially generalizethese methods, now allowing for the distributed algorithm to be applied to anyreversible Markov process.

To understand the distributed implementation of the algorithm, let us con-sider the algorithm in its original context of wireless networks. Provided with anachievable target throughput vector γ [9, 22, Chapter 2], we can write down anobjective function that is convex, and of which its minimizer r opt guarantees thatthe actual normalized throughput ϑ = θ/µ is equal to the target normalizedthroughput γ. The stochastic gradient algorithm in (1.12) can then be applied

11


to this objective function to find r opt, which after a mathematical transformationboils down to updating the deactivation rate according to

(1.13) µ[n+1] = µ[n] expa[n+1](ϑ[n+1] − γ)

, n ∈ N0.

Here, ϑ denotes an empirically obtained estimate of the normalized throughputfor the (n+1)-st iteration of the algorithm through an observation of the systemduring the time period [t[n], t[n+1]).

The algorithm in (1.13) is intuitively clear: if transmitter i observes a higherthroughput than its target throughput, transmitter i must increase its transmis-sion rate, and vice versa. Note in particular that the amount by which transmitteri should increase its transmission rate depends only on its current transmissionrate µ[n]i , a measurement of its actual normalized throughput ϑ[n]i , and its targetthroughput γi . The information that transmitter i needs is all locally available totransmitter i, which is why this algorithm can be implemented in a distributedfashion.

While the measurements bypass the computational effort of calculating thestationary probabilities, they result in inherently noisy and biased estimates forthe gradient, and convergence of the algorithm is therefore not guaranteed.More concretely, if the observation period t[n+1] − t[n] is much smaller than thetypical time-scale on which short-term unfairness is an issue, the estimate forthe gradient will be biased. Generally also, if the observation period is too short,the estimates will be noisy. If the algorithm then also responds aggressively, i.e.a[n] is too large, an erroneous decision made because of a biased and noisygradient will be amplified. Yet if the algorithm responds too carefully, i.e. a[n] ischosen too small, the algorithm becomes unresponsive and can get stuck neara suboptimal point r ∗ 6= r opt.

To overcome these difficulties, we have derived sufficient conditions ont[n+1]− t[n] and a[n] that guarantee convergence to the optimal solution r opt [2,10, Chapter 2]. In order to do so, we focused on reversible Markov processes,for which powerful results are known for the mixing times [50, 51]. Intuitivelyspeaking, the mixing time is a measure for the time-scale beyond which short-term unfairness is of no more concern. The resulting conditions require that onecarefully updates the arrival rate vector and patiently observes the system. Forexample, setting t[n+1] − t[n] = n+ 1 and a[n] = 1/n already suffices for certainproblems [2, 10, Chapter 2], which lengthens the observations periods and at

12


the same time reduces the step sizes taken with each iteration, though not toofast.

Figure 1.4 shows a simulation of the algorithm in the context of the fourtransmitter wireless network . The goal was to ensure that all transmittersachieve the same normalized throughput, so γ1 = . . . = γ4. Because of thesymmetry of the network, the optimal transmission rates must be component-wise identical, i.e. µopt

1 = . . . = µopt4 , and we can see in Figure 1.4 (left) that

the transmissions rate indeed approach the same optimal value. Also note thatthe algorithm initially makes relatively poor decisions, and that the number oferroneous decisions decreases as time progresses. This happens because theobservation periods lengthen. Remark similarly that the algorithm initially re-sponds quite aggressively, but that since the step sizes decrease, this tendencyreduces over time. Finally, note from Figure 1.4 (right) that the network quicklyoperates at least approximately according to the target normalized throughputγ, and that the algorithm spends most of its time to more and more accuratelyapproach the target throughput.

time

rateµ[n]

γ

time

thro

ughp

utϑ[n]

Figure 1.4: Schematic depiction of the evolution of the transmission rate µ[n]

and the throughputϑ[n] over time when applying the control algorithm in (1.13)to the four transmitter network .

1.2.2 Loss networks

We will now illustrate the technique of online optimization in the context of lossnetworks [16]. This demonstrates the algorithm’s versatility and applicabilitybeyond the scope of wireless networks. Note also that a loss network is an

13


extension of the wireless network model in the sense that it consists of severalinteracting components that each can be in multiple states.

Loss networks consist of l links, which have capacities c = (c1, . . . , cl)T thatare shared by d customer classes. Class-i customers arrive according to a Pois-son process with rate λi , and the network processes a class-i customer in anexponentially distributed amount of time with mean 1/µi , referred to as holdingtime. Each class-i customer requires capacity Ci, j on link j for the duration ofits holding time. When an arriving class-i customer finds insufficient capacityavailable on any of the links that it requires, it is blocked and lost.

If we denote the number of class-i customers in the network at time t by X i(t),and define the vector X(t) = (X1(t), . . . , Xd(t))T, then the process X(t)t≥0

constitutes a reversible Markov process with configuration space Ω = x ∈Nd+|C x ≤ c and steady-state probability vector [16]

(1.14) πx (λ) =1

Z(λ)

d∏i=1

ρx ii

x i!

for x ∈ Ω, and with the normalization constant given by

(1.15) Z(λ) =∑y∈Ω

d∏i=1

ρyii

yi!.

Here, we have emphasized that the steady-state probability vector is a functionof the arrival rates λ, which we shall now consider as the control parameters.

To understand what we mean by saying that we will control the arrival rates,consider the loss network depicted in Figure 1.5. If we for example decrease thearrival rate λ1 in this network, fewer class-1 customers will arrive per unit timeon average. Links 1, 3 and 4 would consequently be occupied less. This wouldthen make link 3 available more frequently for class-2 and 4 customers, and thisin turn potentially allows more class-2, 3 and 4 customers into the system.

Similar to the derivation of the distributed algorithm in (1.13), we can cre-ate a convex objective function [2, 9, Chapter 2] of which the minimizer λopt

guarantees that the average number of customers of each class in the systemequals a specified target value, that is,

(1.16) E[X] =∑x∈Ω

xπx (λopt) = γ.

14


2

1

3

6

4

5

λ2

λ1 λ4

λ3

C =

1 0 1 1 0 00 1 1 0 1 00 0 0 1 0 10 1 1 0 0 1

c = (4, 4, 10, 10, 4, 4)T.

Figure 1.5: Example loss network, together with an example capacity matrixC and an example capacity vector c. The edges numbered 1–6 in the graphrepresent the links.

This means that we can prioritize certain customer classes and ensure that onaverage a specified number of customers of every class is in the system.

The accompanying algorithm is to repeatedly observe the system duringtime periods [t[n], t[n+1]), subsequently calculate estimates for E[X], and thenwith such approximations X [n+1] iteratively update the arrival rates accordingto

λ[n+1] = λ[n] exp−a[n+1](X [n+1] − γ)

, n ∈ N0. (1.17)

The resulting sequence λ[0],λ[1], . . . converges to the solution λopt as the num-ber of iterations increases if the step sizes and observation periods are chosenappropriately.

Part II: Large-scale systems

Part II of this thesis focuses on large-scale systems, which often have substantialcosts due to their sheer size. When appropriately designed though, large-scalesystems provide unique opportunities to achieve attractive economies-of-scale.The focus is placed on one such system: a many-server system that can efficientlyoperate even if large numbers of customers arrive per unit time, and we willillustrate how revenue generation in this system can be maximized using controlpolicies.

15


1.3 Quality-and-Efficiency-Driven systems

We start by introducing the many-server system, and explaining how beneficialeconomies-of-scale can be achieved by operating the system in the so-calledQuality-and-Efficiency-Driven (QED) regime [52].

Assume that customers arrive to the many-server system according to aPoisson process [16, 53] of rate λ, and that an arriving customer enters serviceimmediately if any of the s servers is available. If the arriving customer finds allservers occupied, the customer forms or joins a queue until the customer is atthe front of the line and a server becomes available. At this point the customeris taken into service by the available server. We suppose further that customersrequire independent exponentially distributed service times with mean 1/µ, sothat the number of customers in the system at time t, Qs(t), constitutes a birth–death process [17, 53] with a state-transition diagram as depicted in Figure 1.6.This system is also known as the Erlang C system [16].

0 1λ

µ· · ·

λ

2µs

λ

sµs+ 1

λ

sµ· · ·

λ

sµ

Figure 1.6: Transition diagram of the process Qs(t)t≥0.

The birth–death process in Figure 1.6 is positive recurrent if the load ρ =λ/(sµ) < 1. This means that every state is visited infinitely often. The steady-state probability of observing k = 0, 1, . . . customers in the system is then givenby

(1.18) πk = limt→∞P[Qs(t) = k] =

(1Z(sρ)k

k! , k = 0,1, . . . , s,1Z

ssρk

s! , k = s+ 1, s+ 2, . . . ,

and the normalization constant is

(1.19) Z =s−1∑l=0

(sρ)l

l!+∞∑l=s

ssρl

s!=

s−1∑l=0

(sρ)l

l!+(sρ)s

(1−ρ)s!.

From the equilibrium distribution (1.18) we readily obtain the Erlang Cformula, which gives the probability that a customer experiences delay,

(1.20) Cs = limt→∞P[Q(t)≥ s] =

∞∑k=s

πk =(sρ)s

(1−ρ)s!∑s−1l=0

(sρ)ll! +

(sρ)s(1−ρ)s!

.

16

1.3. Quality-and-Efficiency-Driven systems

The delay probability in (1.20) is an important performance measure for ErlangC systems. Customers specifically would generally like to have a low probabilityof experiencing delay.

Ideally in an Erlang C system the servers should always be busy and newlyarriving customers should never have to wait. The former goal is met when theload is close to one because then there are almost always customers availableto serve next, but this clashes with the latter goal. While these dual goals seemincompatible at first glance, Halfin and Whitt [52] found that by coupling thearrival rate to the size of the system via

λ= µ(s− γp

s), (1.21)

where γ > 0 is an arbitrary constant, the load converges toρ = 1−γ/p

s→ 1 andsimultaneously the expected steady-state waiting time converges to E[W ]→ 0as s → ∞. Note in particular that the second term in (1.21), γ

ps, will be

orders of magnitude smaller than the system size s for large systems. This meansthat significant economies-of-scale are achieved while providing high quality-of-service. For this reason the Halfin–Whitt operating regime has also becomeknown as the QED regime for service systems.

Halfin and Whitt [52] showed furthermore that under the QED scaling in(1.21), the scaled stochastic process

Xs(t) =Qs(t)− sp

s(1.22)

converges weakly to a diffusion process X (t) that behaves as a Brownian motionin the upper half plane, and as an Ornstein-Uhlenbeck process in the lower halfplane. The interpretation is such that whenever X (t)> 0, customers are waiting,and when X (t) < 0, servers are idling. The diffusion process is schematicallydepicted in Figure 1.7.

The diffusion limit provides a powerful tool to analyze the many-serversystem. For instance, the Erlang C formula in (1.20) converges in the QED limitto

Cs →1

1+ γ Φ(γ)φ(γ)

as s→∞, (1.23)

which can relatively easily be proven via the diffusion limit. Here, Φ and φdenote the cumulative distribution function and probability density function ofthe standard normal distribution, respectively. The diffusion limit also providesintuition as to the behavior of large many-server systems.

17


0 t

X (t)

Brownian motion ↓

↑ Ornstein–Uhlenbeck

Figure 1.7: The diffusion process X (t) in the QED limit.

1.4 Scalable admission control and revenue structure

We now introduce the scalable admission control policies and revenue structuresthat we have developed, which are suitable for the QED regime [6, 7, Chap-ters 6, 7]. We also illustrate how our extensions can be used to maximize theaverage rate at which revenue is generated by the system.

We assume that whenever there are servers available to help new customers,newly arriving customers are always allowed in. When there are k ≥ s customershowever, we will actively decide whether we allow newly arriving customers in ornot. This decision will be based on the outcome of a Bernoulli random variable:with probability ps(k − s) we allow the customer into the system, and withprobability 1− ps(k− s) we deny the customer access. We suppose furthermorethat the system generates revenue at rate rs(k) when there are k customersin the system. With these extensions in place, the number of customers in thesystem at time t, Qs(t), still constitutes a birth–death process Qs(t)t≥0, butthe process now has a slightly modified state-transition diagram as depicted inFigure 1.8.

The addition of the probabilistic admission control changes the equilibriumdistribution substantially. Specifically, the steady-state probability of observingk = 0, 1, . . . customers is given by

(1.24) πk =

(1Z(sρ)k

k! , k = 1,2, . . . , s,1Z

ssρk

s!

∏k−s−1i=0 ps(i), k = s+ 1, s+ 2, . . . ,

18

1.4. Scalable admission control and revenue structure

0 1λ

µ· · ·

λ

2µs

λ

sµs+ 1

ps(0)λ

sµ· · ·

ps(1)λ

sµrs(0) rs(1) rs(s) rs(s+ 1)

Figure 1.8: The many-server system with an admission control policy ps(k −s)k≥s, and a revenue structure rs(k)k∈N0

.

where ρ = λ/(sµ), and the normalization constant is

Z =s−1∑l=0

(sρ)l

l!+(sρ)s

s!

1+

∞∑n=1

ρnn−1∏j=0

ps( j). (1.25)

To ensure positive recurrence, we now have to assume that Z <∞ [54].The equilibrium distribution in (1.24) is a direct generalization to the equi-

librium distribution in (1.18) because we recover the Erlang C system whensetting ps(k− s) = 1 for k = s, s+ 1, . . .. The equilibrium distribution in (1.24)therefore describes not just the Erlang C system, but also other canonical sys-tems. For example, if we set ps(k − s) = 0, the system behaves as an Erlang Bsystem, while for ps(k− s) = 1/(1+ (k− s+ 1)ϑ/(sµ)), we recover the Erlang Asystem in which waiting customers abandon after an exponential time with rateϑ.

Furthermore, the equilibrium distribution in (1.24) allows us to calculatethe average rate at which the system generates revenue, Rs(ps(k − s)k≥s) .Note that we emphasize the dependency on the admission control. Our goalwill namely be to maximize the revenue generated by the system, and we cando so by optimally choosing the admission control. In formulae, this problem isequivalent with

maximizing Rs(p(k− s)k≥s) =∞∑k=0

rs(k)πs(k) over p(k− s)k≥s. (1.26)

We can expect the optimization problem in (1.26) to be difficult to analyzebecause of the complicated dependency of the equilibrium distribution on theadmission control, see (1.24). To overcome these difficulties we will leverage

19


the fact that mathematical expressions of performance measures in the QEDregime facilitate analysis, as exemplified by the limiting expression in (1.23). Inorder to maintain the QED regime however, we must scale the admission controland revenue structure appropriately. We have shown that the simplest methodto maintain the QED regime is to set rs(k) = r((k− s)/

ps) and ps(0) · · · ps(k−

s) = f ((k − s)/p

s) for appropriately chosen continuous functions r, f [6, 7,Chapters 6, 7]. It is noteworthy that we see the diffusion scaling in (1.22) appearin the arguments of r and f .

If the system is appropriately scaled to reach the QED regime, the averagesystem-governed revenue rate converges to a functional, i.e. [7, Chapter 7]

(1.27) lims→∞

Rs(ps(k− s)k≥s) = R[0]( f )

with

(1.28) R[0]( f ) =

∫ 0

−∞ r(x)e−x2

2 −γx dx +∫∞

0 r(x) f (x)e−γx dxΦ(γ)φ(γ) +

∫∞0 f (x)e−γx dx

.

Furthermore, we can characterize the rate at which the limit in (1.27) is attained.We have namely devised a method [6, Chapter 6] that utilizes the technique ofEuler–Maclaurin (EM)-summation [55] to give for any revenue profile r(x) allcoefficients R[0](γ), R[1](γ), . . . in the asymptotic expansion

(1.29) Rs(ps(k− s)k≥s) = R[0]( f ) +R[1]( f )p

s+

R[2]( f )s

+R[3]( f )

s3/2+ . . . .

Let us now return to solving the optimization problem in (1.26). By usingthe first coefficient of the asymptotic expansion in (1.29) to approximate theoptimization problem, we end up with the limiting optimization problem to

(1.30) maximize R[0]( f ) over f .

Using techniques of variational calculus [56] and Hilbert-space theory [57], wehave shown [7, Chapter 7] that if the revenue profile r(x) peaks around theideal operating point of the system x = 0, the admission control profile thatmaximizes the revenue in (1.30) is of a threshold type:

(1.31) f opt(x) = 1[x ≤ ηopt].

20

1.5. Overview of this thesis

This asymptotic control translated into a prelimit admission control is of theform popt(k− s) = 1[k ≤ s+ bηoptpsc] for k = s, s+ 1, . . ., etc. The ideal bufferthus operates on the order of

ps, which is again in line with the diffusion scaling.

We have furthermore shown that the optimal threshold levelηopt must satisfythe so-called threshold equation [7, Chapter 7]

r(ηopt) = R[0](1[x ≤ ηopt]), (1.32)

and that this asymptotic characterization of the optimal threshold uniquely iden-tifies ηopt. Explaining (1.32) in words, we see that the optimal threshold levelηopt occurs precisely at the point where the marginal revenue of a customerr(ηopt) equals the average system revenue when operating under a thresholdpolicy with threshold level ηopt. From an economical point of view this is clear:if the admittance of a customer earns more than what the system is presentlyearning, there is a profit incentive to admit the customer.

The threshold equation is a fundamental equation that lends itself for anal-ysis. For example, the threshold equation implies that the optimal thresholdlevel is monotone and nondecreasing in γ, and if r(x) is linearly decreasingfor x > 0, the threshold equation can be used to obtain an explicit expressionfor the optimal threshold in terms of the Lambert W function [58]. We haveobtained these results, and more, from the threshold equation, all for generalrevenue structures [7, Chapter 7].

1.5 Overview of this thesis

The remaining chapters of this thesis are all based on different papers. Apartfrom some minor modifications, the contents of the chapters are identical to thecontents of the papers as they have appeared in the open literature. The chaptershave been grouped naturally into the two parts described in this introduction.

Part I: Complex systems

Chapter 2 presents

Online network optimization using product-form Markov processes,[2]by Jaron Sanders, S.C. Borst, and J.S.H. van Leeuwaarden,

which has been accepted for publication in the journal IEEE Transactions on Au-tomatic Control. This paper formally develops the online optimization method

21


described in Section 1.2, establishes sufficient conditions for convergence toa globally optimal solution, and explains how to deal with issues such as theunfairness described in Section 1.1.1. The paper also describes the algorithmin the context of various stochastic systems including queueing networks, lossnetworks and wireless networks, and illustrates the method’s diverse applica-tions. It also discusses the potential of the algorithm to operate in a distributedfashion. Earlier results were published in the conference papers:

Achievable performance in product-form networks, Allerton 2012,[9]Online optimization of product-form networks, Valuetools 2012.[10]

Chapter 3 presents

Wireless network control of interacting Rydberg atoms,[3]by Jaron Sanders, Rick van Bijnen, Edgar Vredenbregt, and Servaas Kokkelmans,

which has been published in Physical Review Letters. This paper identifies arelation between ultra-cold Rydberg gases and the stochastic process that modelswireless networks, as discussed in Section 1.1. It explains how the Rydberg gascan be driven into dominant configurations through our understanding of thebehavior of wireless networks, which is desirable for certain applications ofultra-cold Rydberg gases. The paper also brings the online optimization methodintroduced in Section 1.2 to the realm of Rydberg gases, and discusses how itcan be used experimentally such that the particles in the Rydberg gas are excitedwith specified target probabilities.

Chapter 4 presents

Sub-Poissonian statistics of jamming limits in ultracold Rydberg gases,[4]by Jaron Sanders, Matthieu Jonckheere, and Servaas Kokkelmans,

which has appeared in Physical Review Letters. This paper heuristically analyzesappropriately constructed random-graph models that capture the blockade ef-fect, and derives formulae for the mean and variance of the number of Rydbergexcitations in jamming limits introduced in Section 1.1.2. The paper gives anexplicit relationship between the sub-Poissonian behavior observed in Figure 1.2and the blockade effect, and shows through a comparison with measurementdata that theory and experiment agree strongly.

22

1.5. Overview of this thesis

Chapter 5 presents

Scaling limits for exploration algorithms,[5]by Paola Bermolen, Matthieu Jonckheere, and Jaron Sanders,

a preprint of which has been available on arXiv. This paper rigorously establishesthe claims and heuristics of [4, Chapter 4] for a class of exploration algorithmson random graphs. Similar to the excitation process explained in Figure 1.3,these exploration algorithms activate at each step a random number of verticesand explore their (random) neighboring vertices. This paper formally studies thestatistical properties of the proportion of active vertices using fluid and diffusionscaling limits of Markovian processes.

Part II: Large-scale systems

Chapter 6 presents

Scaled control in the QED regime,[6]by A.J.E.M. Janssen, J.S.H. van Leeuwaarden, and Jaron Sanders,

which was published in Performance Evaluation. This paper develops the asymp-totics in the QED regime for the many-server system when a scalable admissioncontrol is included as discussed in Section 1.4. The paper provides a constructivemethod to obtain all higher-order terms in the asymptotic expansion (1.29), anddue to the generality of the admission control, this generalizes earlier asymptoticresults for the Erlang B and C systems.

Chapter 7 presents

Optimal admission control with QED-driven revenues,[7]by Jaron Sanders, S.C. Borst, A.J.E.M. Janssen, and J.S.H. van Leeuwaarden,

which has been available on arXiv. This paper takes the scalable admissioncontrol of [6, Chapter 6], and builds on top of it the revenue maximizationframework introduced in Section 1.4. The paper shows how the nondegenerateoptimization problem (1.30) arises in the limit, and identifies the fundamentalthreshold equation in (1.32) that characterizes the optimal admission control.The optimal admission control turns out to be a threshold policy, see (1.31), andwe show extensively how the threshold equation can be leveraged to obtain awide range of analytical insights into the optimal threshold value.

23


Chapter 8 presents

Optimality gaps in asymptotic dimensioning of many-server systems,[8]by Jaron Sanders, S.C. Borst, A.J.E.M. Janssen, and J.S.H. van Leeuwaarden,

which is available as a preprint on arXiv. This paper studies the accuracy ofasymptotic dimensioning, which is used to solve problems that trade off revenue,costs and service quality in large many-server systems. It derives bounds foroptimality gaps that capture the difference between true optima of optimizationproblems and asymptotic approximations to those optima that are derived fromthe limiting QED approximations, by leveraging the refined QED approximationsdeveloped in [6, 7, Chapters 6, 7]. The paper ends by discussing a few newapproaches to asymptotic dimensioning.

24

Part I

Complex systems

25

CHAPTER 2Online network optimization using

product-form Markov processes

by Jaron Sanders, Sem Borst, and Johan van LeeuwaardenIEEE Trans. Autom. Control, preprint volume, issue 99

27

2. ON L I N E O P T I M I Z AT I O N U S I N G MA R K O V P R O C E S S E S

Abstract

We develop a gradient algorithm for optimizing the performanceof product-form networks through online adjustment of control pa-rameters. The use of standard algorithms for finding optimal param-eter settings is hampered by the prohibitive computational burden ofcalculating the gradient in terms of the stationary probabilities. Theproposed approach instead relies on measuring empirical frequen-cies of the various states through simulation or online operation soas to obtain estimates for the gradient. Besides the reduction in com-putational effort, a further benefit of the online operation lies in thenatural adaptation to slow variations in ambient parameters as com-monly occurring in dynamic environments. On the downside, themeasurements result in inherently noisy and biased estimates. Weexploit mixing time results in order to overcome the impact of thebias and establish sufficient conditions for convergence to a globallyoptimal solution.

We discuss our algorithm in the context of different systems, in-cluding queueing networks, loss networks, and wireless networks.We also illustrate how the algorithm can be used in such systemsto optimize a service/cost trade-off, to map parameter regions thatlead to systems meeting specified constraints, and to achieve targetperformance measures. For the latter application, we first identifywhich performance measures can be controlled depending on theset of configurable parameters. We then characterize the achievableregion of performance measures in product-form networks, and fi-nally we describe how our algorithm can be used to achieve thetarget performance in an online, distributed fashion, depending onthe application context.

28

2.1. Introduction

2.1 Introduction

Markov processes provide a versatile framework for modelling a wide varietyof stochastic systems, ranging from communication networks and data centerapplications to content dissemination systems and physical or social interactionprocesses [11, 13, 44]. Key performance measures such as buffer occupancies,response times, and loss probabilities can typically be expressed in terms of thestationary distribution π of the Markov process.

The stationary distribution π typically depends on system parameters rthat can be controlled, e.g. admission thresholds, service rates, link weights orresource capacities. In those cases, the interest is often not so much in evaluatingthe performance of the system for given parameter values, but rather in findingparameter settings r opt that optimize the performance or achieve an optimaltrade-off between service level and costs. The mathematical problem of interestcan be formulated as finding

r opt ∈ argminr

u(r ), (2.1)

where u(r ) denotes an objective function. We assume that u(r ) can be expressedin terms of the stationary distribution π(r ) and r . As such, u(r ) can representperformance measures, and describe costs such as capital expense, power con-sumption, etc.

Optimization problem (2.1) could in principle be solved using mathematicalprogramming approaches such as gradient-based schemes, i.e. by iterativelysetting

r [n+1] = r [n] − a[n+1]∇r u(r [n]), n ∈ N0, (2.2)

with the a[n+1] denoting step sizes. However, in addition to the usual convexityissues, difficulties arise from the fact that the stationary distribution π(r ) isoften only implicitly determined as a function of r by the balance equations,and is rarely available in explicit form. This severely complicates the evaluationof the objective function u(r ), as well as the calculation of its gradient ∇r u(r ).

2.1.1 Product-form networks

We focus on the class of reversible product-form networks, which includes multi-dimensional birth–death processes. For such processes, transitions occur fromstate x to states x +e i and x −e i at rates λi,x i

φi(x ), µi,x iψi(x ), respectively. We

29


assume that φi(x )/ψi(x + e i) = Ξ(x + e i)/Ξ(x ) for some function Ξ, in orderto guarantee reversibility. These processes possess the product-form stationarydistribution

(2.3) πx = Z−1Ξ(x )n∏

i=1

x i∏j=1

λi, j−1

µi, j,

and cover a rich family of stochastic models such as loss networks [39, 40], openand closed queueing networks [17, 41], wireless random-access networks [14,42, 43] and various types of interacting-particle systems [11, 44].

Now assume that we can control all the system parameters λi, j . It is thenconvenient to view the stationary distribution as function of the controllableparameters and express the stationary probabilities in an exponential form bydefining ri, j = lnλi, j−1, so that

(2.4) πx = Z−1 exp∑

i, j

Ax ,i, j ri, j + bx

,

where for x ∈ Ω, Ax ,i, j = 1[x i ≥ j], and bx = lnΞ(x )−∑n

i=1

∑x ij=1 lnµi, j . In fact,

more generally, the optimization approach we develop can be applied to anyreversible process for which the stationary distribution can be expressed usingvector notation as

(2.5) π(r ) = Z−1(r )exp (Ar + b),

with A a matrix that will play a central role in our results, b a vector, Z(r )the normalization constant, and r an invertible function of the controllableparameters. This form can for example deal with situations in which controllablerates influence multiple transition rates between different states, or when onlya subset of all system parameters can be controlled.

The partial derivatives ∇rπ(r ) for this class of processes can be written asa linear combination of products of stationary probabilities π(r ), thus reducingthe computation of the gradient to the evaluation of the stationary distribution.The problem that remains, however, is that the stationary probabilities involvea normalization constant whose calculation is computationally intensive andpotentially NP-hard [46]. This issue is particularly pertinent in the context ofiterative optimization algorithms such as the gradient-based scheme in (2.2),where partial derivatives need to be calculated repeatedly.

30

2.1. Introduction

In order to circumvent the computational burden of calculating the station-ary probabilities, we adopt a gradient approach that relies on measuring theempirical frequencies of the various states so as to estimate the partial deriva-tives. In each iteration we observe the stochastic process for some time periodthrough simulation or online operation, and we then calculate estimates for thegradient∇r u(r ) based on the measured time fractions of the various states. De-noting such an approximation by G[n+1], we iteratively update the parametersaccording to

R[n+1] = R[n] − a[n+1]G[n+1]

, n ∈ N0, (2.6)

which can be interpreted as a stochastic gradient descent method.

Although the number of states may be extremely large, it turns out that inmany situations one only needs to track the time fractions of aggregate statesrather than all individual states, and that these aggregate states can be observedin an online, distributed fashion, depending on the application context. Besidesthe reduction in computational effort, a further benefit of the online operationis that the algorithm will automatically adapt to slow variations in ambientparameters which are fairly common in dynamic environments.

While the measurements bypass the computational effort of calculating thestationary probabilities, they result in inherently noisy and biased estimates forthe gradient. The issue of noisy estimates is paramount in the field of stochasticapproximation, in which many robust stochastic approximation schemes havebeen developed that can cope with various forms of random noise [47, 48]. Incontrast, biased estimates present a much trickier issue, which is usually notaccounted for in stochastic approximation schemes. In order to neutralize theimpact of the bias, we focus our attention on the family of reversible processeswithin the above-mentioned class of Markov processes. For such reversible pro-cesses, powerful results are known for mixing times [50, 51], which allows us toderive sufficient conditions guaranteeing convergence to the optimal solution of(2.1), as will be summarized in Theorem 2.11. Intuitively, the mixing times pro-vide an indication for the period of time that we need to observe the stochasticprocess in order to overcome the impact of the bias.

Loosely speaking, the general property of the mixing time needed to guar-antee convergence, is that the average behavior of the process approaches astationary probability measure as time progresses [51]. While we expect such aproperty to hold for a broader class of stochastic processes, reversible Markov

31


processes allow for explicit probabilistic bounds to quantify how close the pro-cess is to stationarity after a certain time [50, 51]. By exploiting these strongcharacterizations of the mixing time, and by designing the algorithm so thatreversibility is maintained, we are able to identify explicitly for what settings(e.g. observation periods and step sizes) the algorithm converges.

2.1.2 Achievable performance

The performance measures that can be manipulated depend on which ratescan be set, and the interpretation of the performance measure depends on thecontext of the system that is being modelled. For example, when setting servicerates of queues in Jackson networks, one can manipulate the steady-state aver-age number of waiting customers in the system. By setting the frequencies oflasers shot at atomic gases, excitation probabilities of particles can be changed[3, Chapter 3]. In Section 2.3.2, we describe which performance measures canbe influenced in product-form networks, depending on the set of configurableparameters. Our results make explicit that the more configurable parametersone has, the more control one can exert on a system.

Any performance measure γ for which there exist finite parameters r opt suchthat the performance of the system equals γ, will be called an achievable target.The collection of achievable targets is called the achievable region. Theorem 2.4will identify the achievable region explicitly, under the assumption that the ad-justable parameters are unbounded, and will show that the achievable regionis a convex hull of a set of vectors. What precise vectors make up this set, de-pends on the set of adjustable parameters, the state space of the Markov process,and more specifically, how these determine the structure in the product-formdistribution.

2.1.3 Related work

The online optimization scheme we will present can trace its roots to a methoddeveloped for wireless networks, specifically those networks that utilize a pro-tocol called CSMA. Models for continuous-time CSMA protocols were initiallybeing studied to understand, evaluate and optimize the performance of wirelessnetworks [14, 42, 43]. Subsequently, a distributed algorithm was developed thatwould slowly adjust the arrival rate of packets over increasing time intervals, so

32

2.1. Introduction

as to stabilize the throughput in a wireless network [22]. Utilizing techniquesfrom stochastic approximation, convergence and rate stability of the algorithmwas proven [49]. This approach towards throughput stabilization in wirelessnetworks provided the inspiration for the present paper.

General algorithms have been designed for solving optimization problemsthrough manipulation of rates, without having specific systems in mind. Forexample in [59, 60], an algorithm is proposed that tackles a parameter opti-mization problem by relying on measurement-based evaluation of a gradient.The parameters change the transition probabilities of the underlying Markovprocess, and the algorithm updates the parameters whenever the process visitsone or more recurrent states.

While [59, 60] consider more general parameter optimization problems andMarkov processes than discussed in this paper, the algorithm does not take ad-vantage of the specific structure of a product-form distribution. Indeed, knowingwhether the entire system is in a recurrent state (and thus when to update) re-quires global state information, as opposed to the distributed operation that isobtained with our updating procedure and the product-form structure of thesteady-state distribution. The algorithm in [22] on the other hand was not con-sidered outside the context of wireless networks. Our contribution is the formal-ization of an algorithm that lies in-between both approaches, offering versatilityof use, yet retaining simplifications that arise from product-form stationary dis-tributions. It is also important to note that our problem formulation differs fromthe typical Markov decision processes [61, 62], which focus on selecting optimalactions in various states rather than identifying optimal parameter values.

Parts of the present paper have appeared in conference proceedings [9, 10].In particular, note that Theorem 2.11 extends [10, Theorem 1] and [9, Theo-rem 2] by generalizing the set of adjustable parameters, and that Section 2.3provides an extensive numerical study of the algorithm, as well as discussion ofthe novel application of mapping (intractable) achievable regions.

2.1.4 Applications

To illustrate the versatility and applicability of our algorithm, Section 2.3 dis-cusses examples outside the context of wireless networks, including M/M/s/squeues, Jackson networks and loss networks. In each case, we discuss how theachievable region depends on the set of adjustable parameters, the state space,

33


and the product-form distribution. We also discuss whether in these applicationsthe algorithm can be implemented in a distributed manner. Because wireless net-works remain an important application of the algorithm and sparked the initialdesign, we conduct an in-depth numerical study of the algorithm in the contextof the continuous-time CSMA model. We illustrate how the convergence ratedepends on the network size, but does not drastically increase with the networksize. We discuss how the target performance measure and mixing times influ-ence the convergence rate, providing insight in the application of our algorithmalso in its broader scope.

As a novel application, we will also use the algorithm to map parameterregions that lead to a system meeting specified constraints. Our approach isparticularly useful in case no analytic characterization of such region is known,and the algorithm hence presents a viable alternative. We will illustrate theapproach for a loss network operator who wants to determine a set of achievableblocking probabilities. Such problems are notoriously difficult to solve, becausethe stationary distribution can be NP-hard to evaluate [46].

2.1.5 Notation

Throughout this paper, we denote the i-th component of a d-dimensional vectorb by bi , where i ∈ 1, . . . , d and d ∈ N+. More generally, if a set Ω is countable,we may instead write bx for x ∈ Ω, to avoid explicit enumeration of all membersof Ω. When taking a scalar function of a vector b, we do this componentwise, forexample exp (b) = (exp (b1), . . . , exp (bd))T, ωr = (ω1r1, . . . ,ωd rd)T, etc. Thenotation of matrix elements is similar. We denote the d-dimensional vector forwhich all components equal 1 by 1d . The d-dimensional unit vector of whichthe i-th component equals 1 will be denoted by ed,i . The inverse of a functionf (x) will be denoted by f ←(x), and its reciprocal 1/ f (x) by ( f (x))−1.

2.2 Algorithm

2.2.1 Model description

Consider a Markov process X(t)t≥0 that is irreducible, reversible and has afinite state space Ω. Assume that through some mechanism, a system operatorcan set v positive, real rates µ1, . . . ,µv and by doing so, can set d ≤ v parameters

34

2.2. Algorithm

r1, . . . , rd . Specifically, we assume that the parameters r are invertible functionsof the rates µ, which can be expressed as r = m(µ) with some appropriatefunction m : (0,∞)v → R ⊆ Rd . We furthermore assume that the stationaryprobability vector π is a function of the parameters r , and has a product-formdistribution which can always be written in the form of (2.5). We discuss severalexamples in depth in Section 2.3.

Let u(r ) be a differentiable, convex function that for example describes theperformance of the system or the trade-off between service level and costs. Ourgoal is to minimize u(r ) over r ∈ R . Here, the simply connected set R ⊆ Rd

represents the range for the parameters r . Specifically, we are interested inoperating the system at the optimal service rates µopt =m←(r opt), with r opt ∈argminr∈R u(r ).

To find r opt, we consider a gradient descent method. We assume that thegradient of u(r ) can be written as a function of π(r ) and r , that is ∇r u(r ) =(∂ u(r )/∂ r1, . . . ,∂ u(r )/∂ rd)T = g (π(r ), r ). The gradient algorithm we con-sider can be written as

r [n+1] = [r [n] − a[n+1]g [n+1]]R , n ∈ N0, (2.7)

where we have introduced the short-hand notation g [n+1] = g (π(r [n]), r [n]) forthe gradient, and n indexes the iteration. The a[n] ∈ (0,∞) denote the stepsizes of the algorithm, and the projection operator [·]R projects any finite pointr outside R to the boundary of R . We assume that the projection operator is anonexpansive mapping, in the sense that

‖[x ]R − [y]R‖2 ≤ ‖x − y‖2, x , y ∈ Rd . (2.8)

Such operators exist, which can be seen by considering the operator definedcomponentwise by

[r ]Ri =maxRmini ,minRmax

i , ri, (2.9)

for r ∈ Rd , i = 1, . . . , d, when R ⊆ Rd is of the form R = [Rmin1 ,Rmax

1 ]× . . .×[Rmin

d ,Rmaxd ], as proven in Appendix 2.A.

2.2.2 Stochastic gradient algorithm

It is well known that under suitable assumptions on the objective function andstep sizes, the gradient algorithm in (2.7) generates a sequence r [n] that con-verges to the optimal solution r opt [63]. Calculating the gradient, however, may

35


be difficult in practice, because it depends on π(r ). Moreover, we are interestedin situations in which certain information is not available to the operator (suchas an arrival rate of customers), so that evaluating the stationary distributionanalytically is not possible. This limits the applicability of the gradient algorithm.Instead of calculating the stationary distribution, we propose to estimate π(r )through online observation of the system and using the estimate instead of theactual stationary distribution.

Observation of the system will take place during time intervals [t[n], t[n+1]],where 0= t[0] < t[1] < . . . denote points in time. At the end of say the (n+1)-thinterval, t[n+1], our algorithm proposes new parameters R[n+1] and the operatoris forced to set the rates to m←(R[n+1]). The stochastic process X(t)t≥0 that de-scribes the system is then no longer time-homogeneous, but can be described as asequence of time-homogeneous Markov processes. Each next time-homogeneousMarkov process starts from the state where the previous time-homogeneousMarkov process ended, and will evolve according to new rates. Thus, duringeach time interval, the system can be described as a time-homogeneous Markovprocess X [n](t)t[n]≤t≤t[n+1] with specified rates m←(R[n]) and initial conditionX [n](t[n]) = X [n−1](t[n]).

Let us now make precise how our algorithm observes the system and makesdecisions. At time t[n+1], marking the end of observation period n+ 1, we cal-culate for x ∈ Ω

(2.10) Π[n+1]x =

1t[n+1] − t[n]

∫ t[n+1]

t[n]1[X [n](t) = x ]d t.

During each interval, one thus keeps track of the fractions of time that thesystem spends in each of the states. The observed vector Π[n+1] then serves asan empirical estimate of π(R[n]). We then estimate the actual gradient G[n+1] =g (π(R[n]),R[n]) by G[n+1] = g (Π[n+1],R[n]). If we then apply (2.7) using G[n+1]

instead of G[n+1], we arrive at the stochastic gradient algorithm

(2.11) R[n+1] = [R[n] − a[n+1]G[n+1]]R , n ∈ N0.

Algorithm (2.7) is deterministic, whereas (2.11) is stochastic. Also note thatbecause we are estimating the gradient instead of explicitly calculating it, the al-gorithm in (2.11) is no longer guaranteed to converge to r opt. In Proposition 2.1below we prove that (2.15) converges under suitable assumptions on a[n+1] and

36

2.2. Algorithm

t[n+1] − t[n]. Furthermore, in practice one may be able to estimate G[n] directly,instead of keeping track of the fraction of time that the process spends in everystate (which is introduced here to prove convergence results). This approachwill become clear in Section 2.3.

2.2.3 Convergence

We need to ensure that when we approximate the gradient of u(r ) using empir-ical distributions that come increasingly closer to the actual distribution π(r ),our approximation of the gradient also comes increasingly closer to the actualgradient. To this end, we assume g to be Lipschitz continuous, in the sense thata constant cl ∈ [0,∞) exists such that

|gi(µ, r )− gi(ν, r )| ≤cl

2

∑x∈Ω|µx − νx |=: cl||µ− ν||var, (2.12)

for i = 1, . . . , d, r ∈ Rd , and all distributions µ,ν ∈ p ∈ (0,1)|Ω||1|Ω|Tp =1. Also, we must exclude cases in which the gradient explodes, because anexploding gradient could cause the algorithm to make a large error. We willassume that there exists a constant cg ∈ [0,∞) so that for r ∈ R ,

‖g (µ, r )‖2 ≤ cg, µ ∈ p ∈ (0, 1)|Ω||1|Ω|Tp = 1. (2.13)

Lastly, as is often the case in stochastic approximation [47], we assume that

∞∑n=1

a[n] =∞,∞∑n=1

(a[n])2 <∞, (2.14)

which ensures that step sizes become smaller as n increases, while remaininglarge enough so that the algorithm does not get stuck in a suboptimal solution.

For clarity we now summarize our primary, technical assumptions, whichwill be in force throughout the paper. We then present our main result for thespecial case m←(r ) = exp (w r ) in Proposition 2.1, which follows from the moregeneral Theorem 2.11 later presented in Section 2.4.2.

Assumptions. (i) X(t)t≥0 is a reversible, finite state Markov process with sta-tionary distribution π(r ), (ii) u(r ) is differentiable, convex, and there exists abounded Lipschitz continuous function g such that ∇r u(r ) = g (π(r ), r ), see(2.12), (2.13), (iii) after specified observation periods, π(R[n+1]) is estimated

37


by Π[n+1] via (2.10), and the gradient by G[n+1] = g (Π[n+1],R[n]), (iv) [·]R is anonexpansive mapping (2.8), and (v) the step sizes satisfy the usual properties(2.14).

Proposition 2.1. When R ⊆ Rd is simply connected and m←(r ) = exp (w r ),w ∈ Rd , wi 6= 0 for i = 1, . . . , d, the sequence

(2.15) R[n+1] = [R[n] − a[n+1]G[n+1]

]R , n ∈ N0,

converges to an optimal solution r opt ∈ arg minr∈R u(r ) with probability one, ifa[n], t[n] and e[n] are such that

(2.16)∞∑n=1

a[n]

n−1∑m=1

a[m]2<∞,

∞∑n=1

a[n]e[n] n−1∑

m=1

a[m]<∞,

and for all finite constants c2, c3, c4 ∈ (0,∞),

(2.17)

∞∑n=1

a[n] n−1∑

m=1

a[m]

expc2

n∑m=1

a[m]

− c3(e[n])2(t[n] − t[n−1])exp

−c4

n∑m=1

a[m]<∞.

In Proposition 2.2 we give a set of sequences a[n], e[n] and t[n], such thatconditions (2.14), (2.16) and (2.17) hold. The proof that this set indeed satisfiesthe conditions is deferred to Appendix 2.E. The choices made for the step sizesand observation periods are based on ideas in [22, 49].

Proposition 2.2. Conditions (2.14), (2.16) and (2.17) of Proposition 2.1 aresatisfied for a[n] = (n ln(n+ 1))−1, e[n] = n−α/2 and t[n] − t[n−1] = nδ with α > 0and δ > 1+α.

With minor modifications to the proof of Proposition 2.1, one can obtaina convergence result that has the added practical benefit that less stringentconditions are required on the step sizes and observation periods, but requiresthe additional assumption that R is compact in Rd . This assumption namelyallows us to simplify bounds on the mixing times, because the rates the algorithmcan set are then finite.

38

2.3. Applications

Proposition 2.3. When R is simply connected and a compact subset of Rd , thesequence generated by (2.15) converges to the optimal solution r opt with probabilityone, if the sequences a[n], t[n] and e[n] are such that

∞∑n=1

a[n]e[n] + exp

−c3(e

[n])2(t[n] − t[n−1])<∞, (2.18)

for all finite constants c3 ∈ (0,∞).

Note that condition (2.18) is less stringent than (2.17). For example, therelatively simple settings a[n] = n−1, t[n] − t[n−1] = n2α+β and e[n] = n−α withα,β > 0 already suffice. In particular, note that for α = β = 1/3, we havea[n] = n−1 and t[n+1] − t[n] = n+ 1. Generally speaking, the algorithm shouldtake smaller steps as time increases, while simultaneously lengthening the ob-servation periods.

2.3 Applications

2.3.1 Optimizing service/cost trade–off

Our first application concerns obtaining an optimal trade-off between perfor-mance and costs in an Erlang loss system, which primarily serves to illuminatethe core features of our algorithm in a relatively simple setting. This examplefalls under our general convergence result in Theorem 2.11.

Consider the M/M/s/s queue. Customers arrive according to a Poisson pro-cess with rate λ and each customer has an exponentially distributed servicerequirement with unit mean. Each of the s parallel servers works at rate µ. Weassume that the system operator is able to set the rate at which servers workanywhere between some minimum value µmin > 0 and some finite, maximumvalue µmax > µmin. Thus r = µ is the parameter that the operator can set. Therange of parameters is then R = [µmin,µmax]. The steady-state probability ofthere being x ∈ Ω = 0,1, . . . , s customers in the system, as a function of theparameter r, is then given by

πx(r) =

λ/r

x/x!

∑sy=0

λ/r

y/y!

, x ∈ Ω. (2.19)

Notably, the probability that an arriving customer finds all servers occupied andis blocked is given by the Erlang loss formula B(s, r) = πs(r).

39


We want to obtain an optimal service/cost trade-off between the blockingprobability B(s, r), and the costs of operating at service rate r, say c(r). We tacklethis problem by minimizing u(r) = vbB(s, r) + vcc(r), with vb, vc > 0 constants.We furthermore assume that c(r) is convex in r, and that its derivative c′(r) isbounded for all r ∈ [µmin,µmax]. The objective function u(r) is then convex inr, because B(s, r) is convex in r [64]. Calculating the gradient, we find that

(2.20) g(π(r), r) = vbB(s, r)(L(s, r)− s)

r+ vcc

′(r).

Here, L(s, r) =∑s

x=1 xπx(r) denotes the mean stationary queue length. InAppendix 2.B, we prove that there exist constants cg, cl ∈ [0,∞) such thatconditions (2.12), (2.13) hold for all probability vectors µ,ν, and all r ∈ R .

All conditions of Theorem 2.11 are met, and the gradient algorithm withn ∈ N0,

(2.21) R[n+1] =R[n] − a[n+1]

vb

B[n+1](L[n+1] − s)R[n]

+ vcc′(R[n])

R,

converges to the optimal solution with probability one when choosing appro-priate a[n], t[n] and e[n]. Here, B[n+1] = Π[n+1]

s denotes an estimate of the lossprobability and L[n+1] =

∑sx=1 xΠ[n+1]

x denotes an estimate of the mean queuelength. The algorithm can be implemented without knowing the arrival rate,which is needed to evaluate (2.19).

Figure 2.1 shows the average behavior of R[n] when R[0] = 1, c(r) = r2, vb =1, vc = 1, λ= 1 and s = 10. It is obtained by simulating 104 independent runsand averaging the result for R[n] at each iteration n. The dotted lines indicate a95%-confidence region. The algorithm was implemented using t[n] − t[n−1] = nand a[n] = 1/n. Calculating the minimum, we find that ropt ≈ 0.20506, whichis well approximated by the algorithm for large n.

2.3.2 Achieving target performance

Our second application concerns the entire class of product-form networks, forwhich we design a method to determine parameters that give a specified targetperformance. We prove that this method works for any achievable target, andwe furthermore identify all achievable targets.

40

2.3. Applications

0 100 200 300 4000

0.2

t

R[n]

Figure 2.1: Average sample path and 95%-confidence region for R[n] when R[0] =1, c(r) = r2, λ = 1 and s = 10. Here, 104 independent runs were conducted.The dashed line indicates the optimal solution ropt ≈ 0.20506.

Log-likelihood objective function

We consider

u(r ) = −αT lnπ(r ) = −∑x∈Ωαx lnπx (r ), r ∈ Rd , (2.22)

with α ∈ p ∈ (0,1)|Ω||1|Ω|Tp = 1. The vector α can be interpreted as a prob-ability distribution on the states of Ω. We prove in Appendix 2.C that u(r ) isconvex in r if π(r ) satisfies the product form in (2.5).

From (2.22), we calculate the gradient ∇r u(r ) and find that

∇r u(r ) = ATπ(r )− ATα= g (π(r )), r ∈ Rd . (2.23)

Assume for a moment that (2.22) has a unique, finite minimizer r opt in Rd .Optimality then requires that g (π(r opt)) = 0 and thus ATπ(r opt) = ATα. We callγ := ATα the target performance, a name inspired by the fact that our algorithmseeks r opt such that ATπ(r opt) = γ. If the operator of the system sets the ratesas µopt = m←(r opt), then the system’s performance measure ATπ(r opt) equalsthe target performance γ. In fact, our goal will be to (i) identify all targets forwhich there exists finite r opt, and to (ii) utilize our online gradient algorithm tofind r opt, and subsequently achieve the target performance γ by setting µopt =m←(r opt).

41


Achievable region

Given some γ, it is not a priori clear whether a finite r opt exists such thatATπ(r opt) = γ. If a finite r opt exists, we call the target achievable. We call thecollection of achievable targets the achievable region, and denote it byA . Theachievable region is characterized in Theorem 2.4, the proof of which we rele-gate to Section 2.4.1.

Theorem 2.4. Any γ ∈A := ATα|α ∈ (0, 1)|Ω|,αT1|Ω| = 1 is achievable.

Theorem 2.4 states that the achievable regionA is the interior of the convexhull of all transposed row vectors of A. This can be seen by writing ATα =∑d

i=1(ATα)ied,i =

∑x∈Ωαx Ax ,·

T, where Ax ,· denotes the row vector in matrixA corresponding to state x . Also, Theorem 2.4 can be leveraged to get a slightgeneralization by applying an affine transformation.

Corollary 2.5. For any B ∈ Rn×d , n ≤ d, that is an affine transformation, thereexists finite r opt such that BATπ(r opt) = γ for all γ ∈ B := BATα|α ∈ (0,1)|Ω|,αT1|Ω| = 1.

Proof. Let δ ∈A , set γ= Bδ. From Theorem 2.4, it follows that there exists afinite r opt such that BATπ(r opt) = Bδ = γ, which gives the result.

Because u(r ) is convex in r ∈ R = Rd and the target γ is achieved by thesolution r opt of (2.1), we want to use our online gradient algorithm (2.15) to findr opt. From (2.23), it follows that |gi(µ)− gi(ν)| ≤ 2 maxx ,i|Ax ,i |||µ−ν||var fori = 1, . . . , d, and that ‖g (µ)‖2 ≤ |Ω|

pd maxx ,i|Ax ,i |, so that (2.12) and (2.13)

are satisfied. Using Proposition 2.1, we then know that given any achievabletarget γ, the online gradient algorithm

(2.24) R[n+1] = R[n] − a[n+1]ATΠ

[n+1] − γ, n ∈ N0,

converges to the optimal solution with probability one, assuming appropriatesequences are chosen for a[n] and t[n].

2.3.3 Distributed implementation

For several product-form networks, the algorithm (2.15) can be implementedin a distributed fashion.

42

2.3. Applications

Wireless networks

We consider the following idealized model of the CSMA protocol. Assume thatthere are d transmitters, labelled i = 1, . . . , d, each of which attempts to initi-ate transmissions according to a Poisson process with rate νi . At every attempt,the transmitter will start a transmission and become active if and only if theresulting state is feasible. A state is called feasible if it corresponds to an inde-pendent set of a so-called conflict graph G(V, E), in which each vertex v ∈ Vrepresents a transmitter, and transmitters connected by an edge cannot transmitsimultaneously. Figure 2.2(a) and Figure 2.2(b) show two examples of conflictgraphs.

Figure 2.2: (a) Conflict graphs G(V, E) are used to model the interference con-straints of transmitters. The vertices represent transmitters, and the edges cap-ture the interference constraints. Active transmitters (black) prevent neighbor-ing transmitters (red) from transmitting. Inactive, non-blocked nodes (white)can still activate. (b) An independent set on a toric graph of 20×20 transmitters.

We assume that the network is saturated, which means that transmittersalways have packets available for transmission. If transmitter i starts a trans-mission, it transmits for an exponentially distributed time with mean 1/µi . Fornotational simplicity, we summarize the rates in the vectors ν = (ν1, . . . ,νd)T

and µ= (µ1, . . . ,µd)T. We define X(t) = (X1(t), . . . , Xd(t))T, where X i(t) = 1 iftransmitter i is transmitting at time t, and X i(t) = 0 if transmitter i is inactive.The stochastic process X(t)t∈[0,∞) now constitutes a continuous-time Markovprocess where X(t) ∈ Ω gives the state at time t.

The process X(t)t∈[0,∞) has a unique steady-state probability vector, given

43


by πx = Z−1∏d

i=1(νi/µi)x i for x ∈ Ω, with Z =∑

y∈Ω∏d

i=1(νi/µi)yi . We nowassume that every transmitter can set its rate νi and we define the parameterri = lnνi , so that for x ∈ Ω,

(2.25) πx (r ) = Z−1 exp d∑

i=1

x i ri −d∑

i=1

x i lnµi

.

Comparing (2.25) to (2.5), we identify Ax ,i = x i and bx = −∑d

i=1 x i lnµi . Wesee that the performance measure θ we can control is componentwise given byθi = (ATπ(r ))i =

∑x∈Ω x iπx , i = 1, . . . , d. This performance measure is called

the normalized throughput, or the fraction of time that transmitter i is active.It was proven in [22] that any throughput γ ∈ intconvhullΩ is achievable,which also follows from Theorem 2.4, implying that there exists a finite νopt

such that θ (νopt) = γ. The independent sets of the conflict graph are on theboundary of the achievable region.

We can find νopt by implementing the online algorithm in Proposition 2.1,which for wireless networks was originally developed in [22], by setting com-ponentwise, i = 1, . . . , d,

(2.26) ν[n+1]i = ν[n]i exp

−a[n+1](θ [n+1]

i − γi), n ∈ N0,

Transmitter i thus increases (decreases) its activity whenever it measures thatits throughput θ [n+1]

i is lower (higher) than its target throughput γi . The updaterule for transmitter i is independent of the throughput of any other transmitterj 6= i, so the update rule can be implemented in a distributed fashion.

We now present a numerical study of the performance of the distributedalgorithm in (2.26). For conflict graphs that are toric grids of sizes l × l, wefocus on the number of iterations N ∗ =minn ∈ N+|maxi=1,...,d |θi − γ|/γ ≤ εrequired until every transmitter measures a throughput within ε ∈ (0, 1) relativedistance of its target throughput γi = γ, i = 1, . . . , d, for the first time, as afunction of the target throughput γ.

Our simulation procedure is as follows. (i) Uniformly at random, we choosea target throughput γ between γmin = 1/16 and γmax = 7/16, because γ= 1/2is on the boundary of the achievable region. (ii) We set the vector of initialactivation rates to ν[0] = γ/(1 − γ)1d , which is to be interpreted as every in-dividual transmitter setting its activation rate equal to the activation rate thatgives the target throughput, if it had no neighbors. (iii) If at the end of a period

44

2.3. Applications

the stopping criterion is met, the simulation stops and subsequently outputs thesample pair (N ∗,γ). If the stopping criterion is not met within a simulation timeof tmax, the simulation restarts. Using trial and error, we chose to set tmax = 106,high enough so that approximately 92% – 94% of simulations met the stoppingcriterion. (iv) This procedure is repeated, and ultimately we generated 5000 –5096 samples per plot. After some trial and error, we chose to set the algorithmparameters to a[n] = 10/n and t[n] − t[n−1] = 10(n− 1)2 + 100.

0.1 0.2 0.3 0.40

20

40

60

80

100

γ

n

Toric grid 4 × 4

0.1 0.2 0.3 0.4γ

Toric grid 6 × 6

0.1 0.2 0.3 0.4γ

Toric grid 8 × 8

0

0.05

0.1

0.15

0.2

0.25

Figure 2.3: Probability of the number of iterations N ∗ required to meet thestopping criterion to be equal to n, P[N ∗ = n], as a function of target throughputγ. We have binned all samples along the x-axis into 20 bins, and normalizedthe number of counts along every column.

From Figure 2.3 we observe that when the target throughput is close to oneof the two theoretical limits 0 or 1/2, the number of required iterations increases,as compared to a target throughput away from 0 and 1/2. The throughput limits0 and 1/2 correspond to solutions νopt ↓ 0 or νopt →∞, respectively. Whenγ ↓ 0 and thus νopt ↓ 0, few transitions occur per unit time. When γ ↑ 1/2and thus νopt→∞, the mixing time increases, i.e. the process resides a largefraction of time in maximum independent sets of the toric grid.

We have also studied the number of iterations N ∗ required to meet the stop-

45


ping criterion, as a function of the size l of l × l toric grids. The simulationprocedure is similar as before, but we now fix γ and vary l, and we generatesample pairs of (N ∗, l). The results are shown in Figure 2.4.

4 6 8 10 12 20

30

40

50

0

0.1

l

n

Target throughput 1/8

0

0.05

0.1

0.15

0.2

0.25

0.3

4 6 8 10 12 10

20

30

40

0

0.1

l

n

Target throughput 1/4

Figure 2.4: Probability of the number of iterations N ∗ required to meet thestopping criterion to be equal to n, P[N ∗ = n], as a function of grid size l. Wenormalized the number of counts along every column.

From Figure 2.4, but also from Figure 2.3, we can observe that as the gridsize increases, the required number of iterations to meet the stopping criterionincreases. Perhaps surprisingly, N ∗ scales in a benign manner with the size ofthe grid.

Closed Jackson networks

The distributed nature of the algorithm is not restricted to wireless networkmodels. Consider a closed Jackson network with d ∈ N+ queues and c ∈ N+

46

2.3. Applications

permanent customers. Customers leaving queue i ∈ 1, . . . , d join queue j ∈1, . . . , d with probability Pi, j , and customers at queue i are served with rate µi .Denote the number of waiting customers in queue i at time t by X i(t). Undersuitable assumptions on P, these closed Jackson networks can be described by areversible Markov process X(t)t≥0, where X(t) = (X1(t), . . . , Xd(t))T. Its statespace is given by Ω= y ∈ Nd

+|∑d

i=1 yi = c. The stationary distribution is given

by πx = Z−1∏d

i=1(λi/µi)x i for x ∈ Ω, where λ ∈ Rd is a non-zero solution ofλ= λP.

We now suppose that every queue i = 1, . . . , d can set its service rate µi .Correspondingly, we define the parameters ri = mi(µ) = − lnµi , so that forx ∈ Ω,

πx (r ) = Z−1 exp d∑

i=1

x i ri + x i lnλi

, (2.27)

from which we identify Ax ,i = x i and bx =∑d

i=1 x i lnλi . Using Theorem 2.4, weconclude that there exists a finite µopt = m←(r opt) ∈ (0,∞)d for γ ∈ A =∑

x∈Ωαx x |α ∈ (0,1)|Ω|,αT1|Ω| = 1, such that the mean stationary queuelengths are equal to γ. Proposition 2.6 simplifies our expression for the achiev-able region, into the intuitive statement that any mean stationary queue lengthvector with non-zero components and in total n customers, is in fact achievable.

Proposition 2.6. A = γ ∈ (0,∞)d |∑d

i=1 γi = n.

Proof. Let γ ∈∑

x∈Ωαx xα ∈ (0,1)|Ω|,αT1|Ω| = 1

. Then,

γ j = ed, jT∑x∈Ωαx x = ed, j

T∑x∈Ω

d∑i=1

αx x ied,i

= ed, jT

d∑i=1

ed,i

∑x∈Ωαx x i =

∑x∈Ωαx x j ,

(2.28)

for j = 1, . . . , d, so that

d∑j=1

γ j(2.28)=

d∑j=1

∑x∈Ωαx x j =

∑x∈Ωαx

d∑j=1

x j(i)=∑x∈Ωαx n

(ii)= n, (2.29)

since (i) x ∈ Ω, and (ii) αT1|Ω| = 1. This proves that∑

x∈Ωαx xα ∈ (0,1)|Ω|,

αT1|Ω| = 1⊆ γ ∈ (0,∞)d |

∑di=1 γi = n.

47


Next consider γ ∈ β ∈ (0,∞)d |∑d

i=1 βi = n. Define δi = βi/n ∈ (0, 1), sothat γ =

∑di=1 δined,i . We see that γ ∈ intconvhullned,1, . . . , ned,d. Finally,

note that intconvhullned,1, . . . , ned,d = ∑

x∈Ωαx x |α ∈ (0,1)|Ω|,αT1|Ω| =1, which concludes the proof.

An algorithm to achieve the target queue length vector can be readily for-mulated through Proposition 2.1, and hence updating

(2.30) µ[n+1] = µ[n] expa[n+1](Q− γ)

, n ∈ N+,

using empirically obtained estimates of the average queue lengths, denoted by Q.From the viewpoint of individual queues, we see that each queue i only requiresinformation on its own queue length, i.e. Q i , and not on that of queues j 6= i.Every queue can autonomously set its service rate, so that this algorithm maybe implemented in a distributed fashion. Figure 2.5 illustrates the algorithm in(2.30).

0 250 500 7500

3

6

t

µ[n]

µ1

µ2

µ3

P1,2 = P1,3 = 1/2P2,1 = P3,1 = 1

Figure 2.5: (left) Simulation of algorithm (2.30) applied to (right) a closedJackson network with 10 customers. After 100 iterations, we find that the aver-age queue lengths E[Q(µ[100])] = (8.00, 0.95, 1.05)T are close to the set targetγ= (8,1, 1)T.

2.3.4 Determining blocking regions

Our algorithm can also be used to map parameter regions that lead to a systemmeeting one or more specified constraints.

48

2.3. Applications

Consider a loss network consisting of l links with capacities c = (c1, . . . , cl)T

shared by d customer classes. Class i customers arrive according to a Poissonprocess with rate λi , and require exponentially distributed holding times withmean 1/µi . Define ρi = λi/µi . Each class i customer requires capacity Ci, j

on link j for the duration of its holding time. We define Li, j = Ci, j/bi , wherebi is the nominal capacity requirement of a class i customer, and Li, j ∈ 0,1indicates whether the route of class i customers contains link j or not. Whenan arriving class j customer finds insufficient capacity available, it is blockedand lost. Let us denote the number of class i customers in the network at timet by X i(t), and define X(t) = (X1(t), . . . , Xd(t))T. Under these assumptions,X(t)t≥0 is a reversible Markov process with state space Ω= x ∈ Nd

+|C x ≤ c,and steady-state probability vector πx (ρ) = Z−1

∏di=1ρ

x ii /x i! for x ∈ Ω, where

Z =∑

y∈Ω∏d

i=1ρyii /yi!.

To manage the system load, we add a control scheme to this system byallowing arriving class i customers into the system with a certain probabilitypi ∈ (0, 1), provided that there is sufficient capacity available to accommodatethe additional customer. This means that we effectively replace the arrival rateλ by pλ. The vector p contains our adjustable parameters, and we write thestationary distribution in the exponential form for x ∈ Ω,

πx (p) = Z−1 exp d∑

i=1

x i ln pi + x i lnρi − ln(x i!)

, (2.31)

When ri = ln pi , (2.31) matches (2.5) with Ax ,i = x i and bx =∑d

i=1 x i lnρi −∑di=1 ln(x i!). We apply our algorithm in (2.15) by setting

p[n+1] = [p[n] exp−a[n+1]

ATΠ

[n+1] − γ][ε,1], (2.32)

for n ∈ N0, so as to achieve target carried traffic levels γ. Here, ε > 0 is thesmallest non-zero probability we allow the operator to set, preventing us fromblocking all future customers based on an earlier measurement of the numberof customers.

We now focus on determining the region of achievable blocking probabilities.To avoid confusion with the achievable region in Theorem 2.4, we call it theblocking region. The fraction of class i customers that upon arrival are admittedand find sufficient capacity available is pi(1− Bi(p)). The quantity 1− pi(1−Bi(p)) is the blocking probability of interest, and we wish to map the region

49


of blocking probabilities b for which there exists popt ∈ (ε, 1) so that bi =1−popt

i (1−Bi(popt)) for i = 1, . . . , d. If such popt exists, we can use Little’s law todeduce that the mean queue length is then given by E[Q i] = popt

i ρi(1−Bi(popt))for i = 1, . . . , d.

The difference between the achievable region in Theorem 2.4 and the block-ing region is important to understand. Theorem 2.4 identifies the achievableregion: For every γ ∈

∑x∈Ω xαx |α ∈ (0,1)|Ω|,αT1|Ω| = 1

, there exists a fi-

nite p ∈ Rd such that∑

x∈Ω xπx (p) = γ. For the blocking region, however, theprobability p is restricted to a compact subset p ∈ [ε, 1]d .

We can investigate the existence of such popt through the use of our onlinealgorithm. We (i) choose a blocking probability b ∈ (0,1)d , and then (ii) set atarget mean queue length of γi = ρi(1− bi) for i = 1, . . . , d. If the algorithmfinds a solution popt ∈ (0,1 − δ)d , with 0 < δ 1 sufficiently small, we canconclude that the blocking probability b is achievable.

Through uniform sampling of δ ∈ (0,1)2, we investigated the blocking re-gion of a link shared by two customer classes, given by C = (2, 3), and c = (11).Customers of each class arrive according to arrival rates of λ= (3, 2)T, and holdlinks for exponential unit mean times. The blocking region as identified throughour simulation is shown in Figure 2.6. The dashed line indicates the concatena-tion of the two parametric curves C1 : δ(p1, 1) =

1− p1(1−B1(p1, 1)), 1− (1−

B2(p1, 1)), for p1 ∈ [0,1], and C2 : δ(1, p2) =

1− (1− B1(1, p2)), 1− p2(1−

B2(1, p2)), for p2 ∈ [0, 1], where Bi(p) =

∑x∈Ω 1[x + ed,i 6∈ Ω]πx (p). Because

this network only consists of one link and two customers, we are still able toevaluate these expressions. In general, however, we expect the boundary to bea complicated expression of the network constraints.

2.4 Proofs

2.4.1 Achievable region

We now prove Theorem 2.4. Our method is inspired by [22, 49], where theachievable region of the throughputs of nodes (transmitter–receiver pairs) in aCSMA network has been determined. We extend the approach to the broaderclass of product-form networks.

The main idea behind the proof is as follows. First, we formulate a convexminimization problem in r for every vector α ∈ (0,1)|Ω| such that 1Tα = 1.

50

2.4. Proofs

0 0.5 10

0.5

1

δ1

δ2

0

0.5

1

Figure 2.6: Blocking region of a two-dimensional loss network.

This minimization problem is constructed such that the minimum is attained inr opt and ATπ(r opt) = ATα. Next, we show that strong duality holds and that thedual problem of our minimization problem attains its optimal value. This thenimplies that the target γ := ATα is achievable.

Define the log-likelihood function u(r ) = −αT lnπ(r ), with α ∈ (0, 1)|Ω| and1Tα= 1. After substituting (2.5), we find that

u(r ) = ln Z(r )−αT(Ar + b), r ∈ Rd . (2.33)

The log-likelihood function in (2.33) has the following desired properties. Forbackground on convexity properties and exponential families, see [65].

Proposition 2.7. The log-likelihood function (2.33) has the properties that (i)infr∈Rd u(r ) ≥ 0, (ii) u(r ) is continuous in r , (iii) u(r ) is convex in r and (iv)ATπ(r opt) = ATα at the critical point r opt for which g(r opt) = 0.

Proof. (i) By irreducibility, we have that πx > 0 for all x ∈ Ω. Because αx > 0for all x ∈ Ω, infr∈Rd u(r ) = infr∈Rd −αT lnπ(r ) ≥ 0. (ii) Being a compositionof continuous functions in r , u(r ) is continuous in r . (iii) Being a compositionof a convex log–sum–exp function v(s) = ln

∑x∈Ω exp sx

−αTs and an affine

transformation Ar + b, u(r ) = v(Ar + b) is convex in r [45]. (iv) Let g (r ) =∇r u(r ) with ∇r = (∂ /∂ r1, . . . ,∂ /∂ rd)T, so that

g (r ) = ATπ(r )− ATα, r ∈ Rd . (2.34)

51


Since g (r opt) = 0, it follows that ATπ(r opt) = ATα.

Recall that for the target ATα to be achievable, we need to prove that thereexists a finite r opt for which ATπ(r opt) = ATα. Using Proposition 2.7, we concludethat such a vector exists, if a finite r opt exists that minimizes (2.33).

Consider the following optimization problem, which we call the primal prob-lem:

(2.35)maximize −βT lnβ + (β −α)Tb,

over β ∈ [0, 1]|Ω|,subject to ATβ = ATα,1Tβ = 1.

It has been constructed in such a way that its dual is the minimization of u(r )over r . This is by design, and we will prove and use this throughout the re-mainder of this section. We note first that (2.35) differs from the minimizationproblem considered in [22, 49], in that there is a second term (β −α)Tb neces-sary to capture the broader class of product-form networks.

Before we can prove that the minimization of u(r ) over r is indeed the dualto (2.35), we need to verify that strong duality holds.

Lemma 2.8. Strong duality holds.

Proof. Slater’s condition [45] tells us that strong duality holds if there exists aβ such that all constraints hold. Considering β = α completes the proof.

Strong duality implies that the optimality gap is zero, specifically implyingthat if (2.35) has a finite optimum, its dual also attains a finite optimum.

Proposition 2.9. The optimal value of the dual problem is attained.

Proof. Assume that the optimal solution χ of (2.35) is such that χx = 0 forall x in some non-empty set I ⊆ Ω. Being the optimal solution, χ must befeasible. Recall that α is feasible by assumption. Any distribution on the linethat connects α and χ is therefore also feasible. When moving from χ towardsα, thus along the direction α−χ , the change of the objective function in (2.35)is proportional to

(2.36) (α−χ)T∇β(β −α)Tb−βT lnβ

β=χ = (α−χ)

T(b− lnχ − 1).

52

2.4. Proofs

For x 6∈ I , χx > 0, so that − lnχx − 1+ bx is finite. For x ∈ I , χx = 0, so that(2.36) equals∞ (recall that αx > 0 for all x ∈ Ω). This means that the objectivefunction increases when moving β away from χ towards α. It is therefore notpossible that χ is the optimal solution, contradicting our assumption.

The optimal solution must thus be such that βx > 0 for all x ∈ Ω. Thisimplies that −βT lnβ + (β −α)Tb <∞ and using Slater’s condition [45], wefind that the optimal value of the dual problem is attained.

Strong duality also implies that there exist finite dual variables so that the La-grangian is maximized. These dual variables turn out to be precisely r opt. Whatremains is to show that the dual problem to (2.35) is indeed the minimizationof u(r ) over r and that the finite dual variables for which the Lagrangian ismaximized are indeed r opt.

Proposition 2.10. The dual problem to (2.35) is

minimize ln Z(r )−αT(Ar + b),over r ∈ Rd .

(2.37)

Proof. By strong duality, we know that there exist finite dual variables r opt ∈ Rd ,w opt ∈ [0,∞)|Ω| and zopt ∈ R such that the Lagrangian

L(β ; r opt, w opt, zopt) = −βT lnβ + (β −α)Tb

+ (ATβ − ATα)Tr opt +βTw opt + (1Tβ − 1)zopt(2.38)

is maximized by the optimal solution βopt. By complementary slackness, w opt =0. Because βopt maximizes the Lagrangian, ∇β L(β; r opt, w opt, zopt)

β=βopt =

zopt1− lnβopt − 1+ b+ Ar opt = 0. This equation can be solved for βopt, result-ing in βopt = exp

(zopt − 1)1+ Ar opt + b

. The constant zopt follows from the

normalizing condition 1Tβ = 1, implying that βopt = Z−1(r opt)expAr opt + b

,

with Z(r opt) = 1T expAr opt + b

.

Because βopt is the optimal solution, we have that

maxβ∈(0,1)|Ω|

L(β ; r opt, w opt, zopt) = L(βopt; r opt, w opt, zopt)

= −βoptT lnβopt + (βopt −α)Tb+ (ATβopt − ATα)Tr opt

= −βoptT(lnβopt − Ar opt − b)−αT(Ar opt + b)

= ln Z(r opt)−αT(Ar opt + b) = u(r opt).

(2.39)

53


Because βopt and r opt solve the primal problem (2.35), r opt is the solution ofminr∈Rd u(r ). This proves that the dual problem to (2.35) is indeed (2.37).

2.4.2 Convergence

We now prove Theorem 2.11, a general convergence result that holds for arbi-trary functions m. We avoided this level of generality earlier in our expositionbecause of the added notational complexity. However, considering general m isrequired to cover problems such as the example in Section 2.3.1.

For notational convenience, we define

(2.40) ϕ(n) = maxi=1,...,d, r∈H [n]

− ln (m←i (r )), n ∈ N+,

where the d-dimensional hyperrectanglesH [n] are given by

(2.41)

H [n] =−|R[0]1 | − cg

n∑m=1

a[m], |R[0]1 |+ cg

n∑m=1

a[m]× · · ·

×−|R[0]d | − cg

n∑m=1

a[m], |R[0]d |+ cg

n∑m=1

a[m], n ∈ N+.

Theorem 2.11. When R ⊆ Rd is simply connected, the sequence generated by(2.15) converges to the optimal solution r opt with probability one, if the sequencesa[n], t[n] and e[n] are such that condition (2.16) holds, and

(2.42)

∞∑n=1

a[n]n−1∑m=1

a[m] expc2

n∑m=1

a[m] − c3(e[n])2(t[n] − t[n−1])

× exp−c4

n∑m=1

a[m] −1[ϕ(n)> 0]ϕ(n)<∞

where the constants c1, c2, c3, c4, and cQ ∈ (0,∞), are given by

c1 = (|Ω|exp (maxx∈Ωbx −min

x∈Ωbx + 2max

i|R[0]i |max

y∈Ω‖Ay ,·‖1))

12 ,(2.43)

c2 = cg maxy∈Ω‖Ay ,·‖1,(2.44)

c3 =4|Ω|2

c21 |G|2|E|2 max1/cQ, 1

,(2.45)

c4 = 2c2, and

(2.46) cQ = minz,v∈Ω|Qz,v>0, 6∃i∈1,...,d:m

←i (R[n])=Qz,v

Qz,v.

54

2.4. Proofs

Proposition 2.1 can be recovered by applying Theorem 2.11 to the casem←(r ) = exp (w r ), assuming w to be any vector with non-zero elements.Specifically for n ∈ N+, ϕ(n) = maxi=1,...,d

|wi |(|R

[0]i | + cg

∑nm=1 a[m])

> 0,

so that 1[ϕ(n)> 0]ϕ(n)≤maxi=1,...,d

|wiR

[0]i |+maxi=1,...,d

|wi |

cg

∑nm=1 a[m]

for n ∈ N+. We see that condition (2.42) holds when replacing c4 by

c4 =min

c4, maxi=1,...,d

|wiR[0]i |+ max

i=1,...,d|wi |cg

, (2.47)

which recovers (2.17) and proves Proposition 2.1.While verifying condition (2.42) can be hard for general ϕ(n), we note that

this difficulty arises specifically when R is not compact. This is because if R isnot compact, the mixing times are harder to control. This issue does not occurfor compact R , in which case we can replace condition (2.42) by condition(2.17) and an appropriate modification of constants c1, c2, c3, c4 and cQ.

We now proceed with the proof of Theorem 2.11. We explain our notion ofconvergence in Section 2.4.2 and then derive conditions on the error bias andzero–mean noise so that convergence is guaranteed. In Section 2.4.2, we showthat under the assumptions of Theorem 2.11, the error bias and zero–mean noiseindeed satisfy the conditions derived in Section 2.4.2.

Conditions for convergence

We will establish that the following two properties hold for arbitrary δ,ε > 0.As a first property, we want that R[n] comes close to r opt infinitely often. Wemake this precise by requiring that for any δ > 0, the setHδ = r ∈ Rd |u(r )≤u(r opt) + δ/2 is recurrent for R[n]n∈N+ . As a second property, we want thatonce R[n] comes close to r opt, it stays close to r opt for all future iterations. Math-ematically, we will require that for any ε > 0, there exists an m ∈ N+ largeenough so that ‖R[n] − r opt‖2

2 ≤ ‖R[m] − r opt‖2

2 + ε for all n≥ m. We henceforthrefer to this property as capture of R[n].

We shall relate both recurrence and capture to the error bias

B[n] = E[G[n]|F [n−1]]−G[n], n ∈ N+, (2.48)

and zero–mean noise

E[n] = G[n] −E[G[n]|F [n−1]], n ∈ N+. (2.49)

55


Consideration of the error bias and zero–mean noise separately makes theanalysis more tractable. Here, F [n−1] denotes the σ-field generated by therandom vectors Z[0], Z[1], . . ., Z[n−1], where Z[0] = (R[0], X (0))T and Z[n] =(G[n]

,R[n], X (t[n]))T for n≥ 1.

Recurrence We begin with deriving conditions under which the set Hδ isrecurrent for R[n]n∈N+ , using the following result.

Lemma 2.12 ([47], p. 115). Let R[n]n be an Rd -valued stochastic process, notnecessarily a Markov process. Let F [n] be a sequence of nondecreasingσ-algebras,with F [n] measuring at least R[i]|i ≤ n. Assume that the a[n+1] are positiveF [n]-measurable random variables tending to zero with probability one and that∑

n a[n] =∞ with probability one. Let V (r )≥ 0 and suppose that there are δ > 0and compactHδ ⊂ Rd such that for all large n,

(2.50) E[V (R[n+1])|F [n]]− V (R[n])≤ −a[n+1]δ < 0, r 6∈ Hδ.

Then the setHδ is recurrent for R[n]n≥0 in the sense that R[n] ∈Hδ for infinitelymany n with probability one.

Consider (2.50) for U(R[n+1]) = ‖R[n+1] − r opt‖22. Substituting (2.15), fol-

lowed by (i) an application of the nonexpansiveness property (2.8), and (ii) anexpansion of the quadratic expression, we conclude that(2.51)

U(R[n+1]) = ‖[R[n] − a[n+1]G[n+1]

]R − r opt‖22

(i)≤ ‖R[n] − a[n+1]G

[n+1] − r opt‖22

(ii)= ‖R[n] − r opt‖2

2 + 2a[n+1]G[n+1]T(r opt −R[n]) + (a[n+1])2‖G[n+1]‖2

2.

After identifying U(R[n]) = ‖R[n]−r opt‖22 on the right-hand side, and substituting

G[n]= G[n] + B[n] + E[n], we conclude that

(2.52)U(R[n+1])− U(R[n])≤ (a[n+1])2‖G[n+1]‖2

2

+ 2a[n+1](G[n+1] + B[n+1] + E[n+1])T(r opt −R[n]).

By (i) convexity of u(r ) (see [45], p. 69, for properties of convex functions), wehave (ii) for R[n] 6∈ Hδ that

(2.53) G[n+1]T(r opt −R[n])(i)≤ u(r opt)− u(R[n])

(ii)< −

δ

2.

56

2.4. Proofs

Using (2.53) to bound (2.52) gives a term −δa[n+1], which we need in order toapply Lemma 2.12. Taking the conditional expectation of (2.52) with respectto F [n], and noting that E[E[n+1]T(R[n] − r opt)|F [n]] = 0, it follows that forR[n] 6∈ Hδ,

E[U(R[n+1])|F [n]]− U(R[n])< −δa[n+1] + Y [n+1], (2.54)

where for n ∈ N0,

Y [n+1] := (a[n+1])2E[‖G[n+1]‖22|F

[n]]

+ 2a[n+1]|E[B[n+1]T(R[n] − r opt)|F [n]]|.(2.55)

Next define ∆[n] = E[∑∞

i=n+1 Y [i]|F [n]] and consider V (R[n+1]) = U(R[n+1])+∆[n+1]. Utilizing (v) the law of total expectation, our construction is such thatthe difference

E[∆[n+1]|F [n]]−∆[n] = EE ∞∑

i=n+2

Y [i]|F [n+1]−

∞∑i=n+1

Y [i]|F [n]

(v)= −Y [n+1],

(2.56)

will exactly cancel the term Y [n+1] in (2.54). This difference is well defined if∑∞i=1 Y [i] <∞ with probability one, so that for R[n] 6∈ Hδ,

E[V (R[n+1])|F [n]]− V (R[n])≤ −δa[n+1]. (2.57)

Note that the bound in (2.57) is of the form in (2.50). Before we can applyLemma 2.12 however, we need to check if

∞∑n=1

Y [n] =∞∑n=1

(a[n])2E[‖G[n]‖22|F

[n−1]]

+ 2∞∑n=1

a[n]|E[B[n]T(R[n−1] − r opt)|F [n−1]]|(2.58)

is finite with probability one. Since∑∞

n=1(a[n])2 <∞ and ‖G[n]‖2 ≤ cg by as-

sumption, the first term is finite. Verifying that the second term is finite with prob-ability one is much harder because it involves regularity conditions on g (π(r ), r )and finiteness of mixing times. This can in fact be shown as stated in the nextlemma, the proof of which we postpone until Section 2.4.2.

Lemma 2.13.∑∞

n=1 a[n]|E[B[n]T(R[n−1] − r opt)|F [n−1]]| <∞ with probabilityone.

57


Capture Having derived conditions for recurrence to occur, we turn our atten-tion to deriving conditions for capture to occur. We will give conditions whichimply the existence of an m ∈ N+ large enough so that U(R[n]) ≤ U(R[m]) + εfor all n≥ m with probability one.

Iterating the bound in (2.52), and utilizing the convexity of u(r ) once moreto derive the bound G[n]T(r opt −R[n−1])≤ 0, we see that for n≥ m,

(2.59)

U(R[n])− U(R[m])≤n∑

j=m

(a[ j+1])2‖G[ j+1]‖22

+ 2n∑

j=m

a[ j+1](B[ j+1] + E[ j+1])T(r opt −R[ j]).

For capture to occur, we need all summations in the right-hand side of (2.59) tobe small when m is sufficiently large. Because

∑∞n=1(a

[n])2 <∞ and ‖G[n]‖2 ≤cg , limm→∞

∑∞j=m(a

[n])2‖G[n]‖2 = 0. This implies that for any ε, there exists

an m0 ∈ N+ so that∑n

j=m(a[n])2‖G[n]‖2 ≤ ε for all n ≥ m ≥ m0. Verifying that

the other two sums are small for sufficiently large m is more difficult. We willestablish Lemma 2.14 using martingale arguments in Section 2.4.2.

Lemma 2.14. For any ε > 0, there exists m0 ∈ N+ so that for any n ≥ m ≥ m0,(a)

∑nj=m a[ j]B[ j]T(r opt − R[ j−1]) ≤ ε, and (b)

∑nj=m a[ j]E[ j]T(r opt − R[ j−1]) ≤ ε

with probability one.

Evaluating the conditions

We now provide the proofs of Lemmas 2.13 and 2.14, which together proveTheorem 2.11. We make heavy use of the following bound.

Lemma 2.15. |R[n]i | ≤ |R[0]i |+ cg

∑nm=1 a[m] for n ∈ N+ and i = 1, . . . , d.

Proof. Fix n ∈ N+. Use the triangle inequality and iterate to obtain |R[n]i | ≤|R[n−1]

i |+ a[n]|G[n]i | ≤ |R[0]i |+

∑nm=1 a[m]|G[m]i |. Lastly, the gradient is bounded,

(2.13).

Error bias We start with showing that the error bias satisfies the propertyclaimed in Lemma 2.13. After (i) using the triangle inequality and (ii) noting

58

2.4. Proofs

that B[n] = E[G[n]|F [n−1]] − g (π(R[n−1]),R[n−1]) is F [n−1]–measurable, onefinds that

∞∑n=1

a[n]|E[B[n]T(R[n−1] − r opt)|F [n−1]]|

(i)≤∞∑n=1

a[n]d∑

i=1

|E[B[n]i |F[n−1]]|(|R[n−1]

i |+ |ropti |)

(ii)=∞∑n=1

a[n]d∑

i=1

|B[n]i |(|R[n−1]i |+ |ropt

i |).

(2.60)

We then (iii) utilize Lemma 2.15 and (iv) upper bound the sequences by takingout the maximum coefficients in both, to come to

∞∑n=1

a[n]|E[B[n]T(R[n−1] − r opt)|F [n−1]]|

(iii)≤

∞∑n=1

a[n]d∑

i=1

|B[n]i ||R[0]i |+ |r

opti |+ cg

n−1∑m=1

a[m]

(iv)≤ max

i=1,...,d|R[0]i |+ |r

opti |

∞∑n=1

a[n]d∑

i=1

|B[n]i |+ cg

∞∑n=1

n−1∑m=1

a[n]a[m]d∑

i=1

|B[n]i |.

(2.61)Next, we bound |B[n]i | from above. After (i) recalling the definitions of error

bias, the gradient and its estimate, (ii) noting that R[n−1] is F [n−1]–measurable,and (iii) using Jensen’s inequality, we find that for i = 1, . . . , d,

|B[n]i |(i)= |E[gi(Π

[n],R[n−1])|F [n−1]]− gi(π(R

[n−1]),R[n−1])|(ii)= |E[gi(Π

[n],R[n−1])− gi(π(R

[n−1]),R[n−1])|F [n−1]]|(iii)≤ E[|gi(Π

[n],R[n−1])− gi(π(R

[n−1]),R[n−1])||F [n−1]].

(2.62)

We then utilize the Lipschitz continuity as asserted in (2.12), to conclude fori = 1, . . . , d,

|B[n]i | ≤ clE[||Π[n] −π(R[n−1])||var|F [n−1]]. (2.63)

Conditions to guarantee∑∞

n=1 a[n]|E[B[n]T(R[n−1]−r opt)|F [n−1]]|<∞withprobability one can therefore be found by upper bounding (2.63), for exampleusing Lemma 2.16, proved in Appendix 2.D.

59


Lemma 2.16. For e[n] ∈ [0, 1], x ∈ Ω, and with c1, c2, c3, c4 defined as in Theo-rem 2.11,

(2.64)

P[|Π[n]x −πx (R[n−1])| ≥ e[n]]≤ c1 exp

c2

n∑m=1

a[m]

− c3(e[n])2(t[n] − t[n−1])× exp

−c4

n∑m=1

a[m] −1[φ(n)> 0]φ(n)

.

Define Φ[n]x = |Π[n]x −πx (R[n−1])| for notational simplicity and let e[n] ∈ [0, 1].

We (i) substitute Φ[n]x in (2.63) to arrive at

(2.65) |B[n]i |(i)≤

cl

2

∑x∈ΩE[Φ[n]x |F

[n−1]],

and then (ii) use the law of total expectation to come to

(2.66)

|B[n]i |(ii)≤

cl

2

∑x∈Ω

P[Φ[n]x < e[n]]E[Φ[n]x |F

[n−1],Φ[n]x < e[n]]

+ P[Φ[n]x ≥ e[n]]E[Φ[n]x |F[n−1],Φ[n]x ≥ e[n]]

.

Next, we (iii) upper bound E[Φ[n]x |F[n−1],Φ[n]x ≤ e[n]]≤ e[n], as well as E[Φ[n]x |

F [n−1],Φ[n]x ≥ e[n]]≤ 1,

(2.67) |B[n]i |(iii)≤

cl

2

∑x∈Ω

e[n] + (1− e[n])P[Φ[n]x ≥ e[n]]

,

and then (iv) upper bound 1− e[n] ≤ 1, and use Lemma 2.16, allowing us toconclude that(2.68)

|B[n]i |(iv)≤

cl

2|Ω|e[n] + c1 exp

c2

n∑m=1

a[m] − c3(e[n])2(t[n] − t[n−1])e−c4

∑nm=1 a[m]

.

After (i) bounding (2.61) from above using (2.68), (ii) realizing that for alllarge enough n,

∑n−1m=1 a[m] ≥ 1 by (2.14), and (iii) recalling assumptions (2.16),

60

2.4. Proofs

(2.42), it follows that∞∑n=1

a[n]|E[B[n]T(R[n−1] − r opt)|F [n−1]]|

(i)≤ max

i=1,...,d|R[0]i |+ |r

opti |, cg

cl|Ω|d2

∞∑n=1

a[n]1+

n−1∑m=1

a[m]

×e[n] + c1 exp

c2

n∑m=1

a[m] − c3(e[n])2(t[n] − t[n−1])

× exp−c4

n∑m=1

a[m] −1[φ(n)> 0]φ(n)

(ii)≤ max

i=1,...,d|R[0]i |+ |r

opti |, cgcl|Ω|d max1, c1

∞∑n=1

a[n]n−1∑m=1

a[m]

×e[n] + exp

c2

n∑m=1

a[m] − c3(e[n])2(t[n] − t[n−1])

× exp−c4

n∑m=1

a[m] −1[φ(n)> 0]φ(n) (iii)

< ∞

(2.69)

with probability one. This completes the proof of Lemma 2.13.We now show that the error bias satisfies assertion (a) in Lemma 2.14. Sim-

ilar to the derivation of (2.60) and (2.61), it follows that∞∑n=1

a[n]|B[n]T(R[n−1] − r opt)| ≤∞∑n=1

a[n]d∑

i=1

|B[n]i |(|R[0]i |+ |r

opti |+ cg

n−1∑m=1

a[m]).

(2.70)Similar to the derivation of (2.69), we conclude that with probability one

∞∑n=1

a[n]|B[n]T(R[n−1] − r opt)|<∞, (2.71)

so that limm→∞∑∞

j=m a[n]B[n]T(r opt − R[n−1]) = 0 with probability one. Thisimplies that there exists an m0 ∈ N+ so that

n∑j=m

a[ j]B[ j]T(r opt −R[ j−1])≤ ε (2.72)

for all n ≥ m ≥ m0 with probability one. The error bias thus satisfies assertion(a) in Lemma 2.14. All that remains is to show that the zero–mean noise satisfiesLemma 2.14(b).

61


Zero–mean noise We use a martingale argument to show that assertion (b)in Lemma 2.14 holds. Let M [n] =

∑nj=1 a[ j]E[ j]T(R[ j−1] − r opt) for n ∈ N+.

Lemma 2.17. M [n] is a martingale.

Proof. We note that M [n] ∈ F [n]. Its expectation is bounded, which can beconcluded after using (i) the triangle inequality, and (ii) Lemma 2.15, for n ∈ N+,

(2.73)

E[|M [n]|](i)≤

n∑j=1

a[ j]E[d∑

i=1

|E[ j]i ||R[ j−1]i − ropt

i |]

(ii)≤

n∑j=1

a[ j] maxi=1,...,d

|R[0]i |+ |ropti |+ cg

j−1∑m=1

a[m]E[‖E[ j]‖1].

Applying (i) the triangle and Jensen’s inequality, as well as (ii) Hölder’s inequal-ity, and recalling (iii) boundedness of the gradient as in (2.13), we bound

(2.74)‖E[ j]‖1

(i)≤ E[‖G[ j]‖1|F [ j−1]] + ‖G[ j]‖1

(ii)≤ d

12E[‖G[ j]‖2|F [ j−1]] + ‖G[ j]‖2

(iii)≤ 2cgd

12 .

Since n ∈ N+ is fixed, it follows that

(2.75)

E[|M [n]|]≤ 2cgd12

n∑j=1

a[ j]maxi|R[0]i |+ |r

opti |+ cg

j−1∑m=1

a[m]

= 2cgd12 max

i|R[0]i |+ |r

opti |, cg

n∑j=1

a[ j] +n∑

j=1

a[ j]j−1∑

m=1

a[m]<∞.

Lastly, using (i) the zero–mean noise property of E[n], we verify for n ∈ N+,

(2.76)E[M [n]|F [n−1]] = E[

n∑j=1

a[ j]E[ j]T(R[ j−1] − r opt)|F [n−1]]

= M [n−1] + a[n]E[E[n]T(R[n−1] − r opt)|F [n−1]](i)= M [n−1],

which concludes the proof that M [n] is a martingale.

Having established the martingale property, we aim at using a martingaleconvergence theorem [66].

62

2.4. Proofs

Theorem 2.18 ([66]). If M [n] is a martingale for which there exists a constantcm < ∞ so that E[(M [n])2] ≤ cm for all n ≥ 0, then there exists a randomvariable Mopt with E[(Mopt)2]≤ cm such that M [n]→ Mopt with probability oneas n→∞. Moreover, E[|M [n] −Mopt|2]

12 → 0 as n→∞.

Before we can apply Theorem 2.18, we need to show the existence of aconstant cm ∈ R such that E[(M [n])2]≤ cm for all n ∈ N+. We therefore expand

supn∈N+E[(M [n])2] = sup

n∈N+

n∑j=1

(a[ j])2E[(E[ j]T(R[ j−1] − r opt))2]

+∑j 6=k

a[ j]a[k]E[E[ j]T(R[ j−1] − r opt)E[k]T(R[k−1] − r opt)],

(2.77)and then consider any one of the cross terms with k < j. By the tower property,

E[E[ j]T(R[ j−1] − r opt)E[k]T(R[k−1] − r opt)]

= E[E[k]T(R[k−1] − r opt)d∑

i=1

E[E[ j]i |F[ j−1]](R[ j−1]

i − ropti )],

(2.78)

and because E[E[ j]i |F[ j−1]] = 0, all cross terms are equal to 0. Because the

summands are positive, we can give an upper bound by summing over all terms,so that

supn∈N+E[(M [n])2]≤

∞∑j=1

(a[ j])2E[(E[ j]T(R[ j−1] − r opt))2]. (2.79)

Using (i) the triangle inequality, and (ii) Lemma 2.15, it follows from (2.74)that

supn∈N+E[(M [n])2]

(i)≤∞∑j=1

(a[ j])2E[ d∑

i=1

|E[ j]i ||R[ j−1]i − ropt

i |2]

(ii)≤∞∑j=1

(a[ j])2E[maxi|R[0]i |+ |r

opti |+ cg

j−1∑m=1

a[m]2‖E[ j]‖21]

(2.74)≤ 4c2

g d maxi|R[0]i |+ |r

opti |, cg2

∞∑j=1

(a[ j])21+

j−1∑m=1

a[m]2

.

(2.80)

63


The right-hand side is finite by conditions (2.14) and (2.16). This can be seenby expanding

(2.81)

∞∑j=1

(a[ j])21+

j−1∑m=1

a[m]2

=∞∑j=1

(a[ j])2 + 2∞∑j=1

(a[ j])2j−1∑

m=1

a[m] +∞∑j=1

(a[ j])2 j−1∑

m=1

a[m]2

and concluding that the first term is bounded by (2.14), and

(2.82)∞∑n=1

(a[n])2 n−1∑

m=1

a[m]2=∞∑n=1

a[n]

n−1∑m=1

a[m]2 (2.16)

< ∞,

and that there exists a finite k ∈ N+, for which holds that∑k−1

m=1 a[m] ≥ 1 for alln≥ k. This is a consequence of (2.14), after all,

∑ j−1m=1 a[m]→∞ as j→∞. It

follows that

(2.83)∞∑n=1

(a[n])2n−1∑m=1

a[m] ≤k−1∑n=1

(a[n])2n−1∑m=1

a[m] +∞∑

n=k

a[n]

n−1∑m=1

a[m]2 (2.82)

< ∞.

We conclude that there exists a coefficient cm so that E[(M [n])2]≤ cm for alln ∈ N+. We now apply Theorem 2.18 and conclude that as n≥ m→∞,

(2.84) E[|M [n]−M [m−1]|2]12 ≤ E[|M [n]−Mopt|2]

12 +E[|M [m−1]−Mopt|2]

12 → 0.

This result enables us to use Doob’s maximal inequality [66], reproduced inthe lemma below, in order to conclude that Lemma 2.14(b) holds.

Lemma 2.19 ([66]). If M [n]n≥0 is a nonnegative submartingale and λ > 0,then λP[supm≤n M [m] ≥ λ]≤ E[M [n]1[supm≤n M [m] ≥ λ]]≤ E[M [n]].

Fix m ∈ N+ and define W [n] = M [n+m−1] − M [m−1] for n ∈ N+. |W [n]| isa submartingale by Jensen’s inequality with respect to the sequence F [m−1],F [m], F [m+1], . . ., since E[|W [n+1]||F [n+m−1]] ≥ E[W [n+1]|F [n+m−1]] = W [n].Applying Lemma 2.19 to |W [n]|, we find that

(2.85)

P[ sup0≤t≤n

|M [t+m−1] −M [m−1]| ≥ λ]≤E[|M [n+m−1] −M [m−1]|]

λ

≤E[|M [n+m−1] −Mopt|] +E[|Mopt −M [m−1]|]

λ

≤E[|M [n+m−1] −Mopt|2]

12 +E[|Mopt −M [m−1]|2]

12

λ

64

2.5. Conclusions

for any λ ∈ (0,∞). This upper bound converges to 0 as n, m→∞, implyingthat there exists an m0 ∈ N+, such that for all n≥ m≥ m0,

M [n] −M [m−1] =n∑

j=m

a[ j]E[ j](R[ j−1] − r opt)≤ ε (2.86)

with probability one.Having established Lemmas 2.13 and 2.14, the proof of Theorem 2.11 is

now completed.

2.5 Conclusions

We have developed an online gradient algorithm for finding parameter valuesthat optimize the performance and cost measures associated with Markov pro-cesses with product-form distributions. As a key feature, the approach avoidsthe computational complexity of calculating the gradient in terms of the station-ary probabilities and instead relies on measuring empirical time fractions of thevarious states so as to obtain estimates for the gradient. While the impact of theinduced measurement noise can be handled without too much trouble, the biasin the estimates presents a trickier issue. In order to exploit mixing time resultsto deal with the bias, we focused on reversible processes. This allowed us toestablish Theorem 2.11, giving sufficient conditions for convergence of the algo-rithm. Our paper contributes to the framework of stochastic approximation theadaptation of concentration inequalities for mixing Markov chains to guaranteeconvergence.

Through Theorem 2.4, we have explicitly identified the achievable perfor-mance for Markov processes with product-form distributions when an operatorcan control one or more parameters (transition rates) of the system. The shapeand size of the achievable region turns out to be intimately related with the statespace and the parameters that can be controlled. Loosely speaking, the largerthe state space is and the more configurable parameters there are, the more canbe achieved.

Furthermore, we have extensively illustrated different uses of the algorithm.Our applications included queueing networks, loss networks, and wireless net-works, and we have illustrated how the algorithm can be used to optimize aservice/cost trade-off, to map parameter regions that lead to systems meeting

65


specified constraints, and to achieve target performance measures. The perfor-mance of the algorithm was studied in the context of wireless networks, and wehave illustrated numerically that the speed of convergence scales in a benignmanner with the size of a network.

A challenging issue for further research is to gain a more detailed under-standing of the effect of step sizes and the role of mixing times in relation tothe convergence speed. A related direction is to explore the trade-off betweenaccuracy in static scenarios and responsiveness in dynamic environments, whichrelates to convergence in distribution for non-vanishing step sizes as opposed tothe almost-sure convergence for decreasing step sizes as considered here. Futurework may also be aimed at relating the achievable region to compact parameterdomains.

Remaining proofs

2.A Nonexpansiveness property of (2.9)

Let i ∈ 1, . . . , d. Define l =Rmini and r =Rmax

i . If x , y ∈ R = [l, r], equalityholds. Consider the case x 6∈ R , y ∈ R . If x > r, |[x]R−[y]R |= |r−y|= r−y ≤x− y = |x− y|. If x < l, |[x]R−[y]R |= |l− y|= y− l ≤ y− x = |x− y|. Finally,consider the case x , y 6∈ R . If x , y > r or x , y < l, |[x]R − [y]R |= 0≤ |x − y|.If x > r, y < l, |[x]R − [y]R |= |r− l|= r− l ≤ x − y = |x − y|. The case x < l,y > r follows from a similar argument.

2.B Lipschitz continuity of (2.20)

Define Bµ = µs and Lµ =∑s

x=1 xµx for all µ ∈ [0, 1]|Ω| for which 1|Ω|Tµ= 1. By

definition of g(µ, r), ‖g(µ, r)‖2 = |g(µ, r)| ≤ vb|Bµ(Lµ − s)|/r + vc|c′(r)| <∞.The first term is finite because r ≥ µmin > 0, Bµ ≤ 1 and Lµ ≤ s <∞. Thesecond term is finite by our assumption that c′(r) is bounded for all r ∈ R . Thisproves that condition (2.13) is met.

We now turn to condition (2.12). Write |g(µ, r)− g(ν, r)|= vb|Bµ(Lµ− s)−Bν(Lν − s)|/r ≤ vb|BµLµ − sBµ − BνLν + sBν|/µmin ≤ vb(|BµLµ − BνLν|+ s|Bµ −Bν|)/µmin. We then conclude that |BµLµ−BνLν|= |BµLµ−BµLν+BµLν−BνLν| ≤Bµ|Lµ − Lν|+ Lν|Bµ − Bν| ≤ |Lµ − Lν|+ s|Bµ − Bν|, so that |g(µ, r)− g(ν, r)| ≤vb(|Lµ−Lν|+2s|Bµ−Bν|)/µmin. Finally, by definition of Bµ, |Bµ−Bν|= |µs−νs| ≤

66

2.C. Convexity of the log–likelihood function

2||µ−ν||var. Similarly for Lµ, |Lµ− Lν| ≤∑s

x=1 x |µx −νx | ≤ 2s||µ−ν||var. Thus|g(µ, r)−g(ν, r)| ≤ 6vbs||µ−ν||var/µ

min, which concludes the proof after settingcl = 6vbs/µmin.

2.C Convexity of the log–likelihood function

Substituting (2.5) into (2.22) gives for r ∈ Rd , u(r ) = ln∑

y∈Ω exp (Ar + b)y −∑x∈Ωαx (Ar + b)x . The function v(s) = ln

∑y∈Ω exp sy −

∑x∈Ωαx sx is convex

on R|Ω| [45], p. 72. We see that u(r ) is a composition of a convex function withan affine mapping, i.e. u(r ) = v(Ar + b), and such functions are convex [45],p. 79.

2.D Proof of Lemma 2.16

We shall use Proposition 2.20 to prove Lemma 2.16, in which

Varµ[ f ] =12

∑x ,y∈Ω

f (x )− f (y)

2µxµy , (2.87)

( f , g)µ =∑x∈Ω

f (x )g(x )µx , (2.88)

and

‖µ‖2,ν =√√∑

x∈Ωµ2

xνx . (2.89)

Proposition 2.20 ([67], p. 2). On some Polish space Ω, let us consider a conserva-tive (continuous-time) Markov process denoted by X (t)t≥0 and with infinitesimalgenerator L . Let µ be a probability measure on Ω which is invariant and ergodicwith respect to Pt .

Assume that µ satisfies the Poincaré inequality Varµ[ f ]≤ −κ(L f , f )µ. Thenfor θ ∈ L1(µ) such that sup |θ |= 1, ε < 2Varµ[θ], and t > 0, assuming that theinitial distribution of Xs is ν,

P1

t

∫ t

0

θ (X (s))ds−∫θdµ

≥ ε≤ dν

dµ

2,µ

exp−

tε2

8κVarµ[θ]

. (2.90)

Before we can use Proposition 2.20 to prove Lemma 2.16, however, we needto verify all of its assumptions. We now verify these assumptions for continuous-time, reversible Markov processes with a product-form stationary distribution.Our method leverages a result for discrete-time Markov chains [50].

67


Define a graph G = (V, E), where V denotes the vertex set in which eachvertex corresponds to a state in Ω and E denotes the set of directed edges. Anedge e = (x , y) is in E if φ(e) = πxQx ,y = πyQy ,x > 0. Here, Q denotes thegenerator matrix of X (t)t≥0. For every pair of distinct vertices x , y ∈ Ω, choosea path γx ,y (along the edges of G) from x to y . Paths may have repeated verticesbut a given edge appears at most once in a given path. Let Γ denote the collectionof paths (one for each ordered pair x , y). Irreducibility of X (t)t≥0 guaranteesthat such paths exist. For γx ,y ∈ Γ define the path length by

(2.91) ‖γx ,y‖φ =∑

e∈γx ,y

1φ(e)

.

Also, let

(2.92) κ=maxe∈E

∑γx ,y∈Γ |e∈γx ,y

‖γx ,y‖φπxπy ,

and define f (e) = f (y)− f (x ) for e = (x , y) ∈ E. Then write

(2.93) Varπ[ f ] =12

∑x ,y∈Ω

∑e∈γx ,y

φ(e)φ(e)

12

f (e)2πxπy .

Use (i) the Cauchy-Schwarz inequality |x T y |2 ≤ x Tx · yT y and (ii) a change ofsummation variables to obtain

(2.94)

Varπ[ f ](i)≤ 1

2

∑x ,y∈Ω

πxπy

∑e∈γx ,y

1φ(e)

∑e∈γx ,y

φ(e) f (e)2

(ii)= 1

2

∑e∈E

φ(e) f (e)2∑

γx ,y∈Γ |e∈γx ,y

‖γx ,y‖φπxπy .

We then use (iii) the definition of κ, and (iv) reversibility of X (t)t≥0, as wellas the symmetry of φ(e), to write

(2.95)

Varπ[ f ](iii)≤κ

2

∑e∈E

φ(e) f (e)2

(iv)=κ

2

∑x ,y∈Ω

πyQy ,x ( f (y)2 − f (y) f (x ))

+κ

2

∑x ,y∈Ω

πxQx ,y( f (x )2 − f (y) f (x ))

= κ∑x∈Ω

∑y∈Ω

Qx ,y

f (x )− f (y)

f (x )πx .

68

2.D. Proof of Lemma 2.16

By definition of the infinitesimal generator L , we find that

(L f )(x ) = limt→0

1t

∑y∈Ω(etQ

x ,y f (y)− f (x )

= limt→0

1t

∑y∈Ω(I + tQ+O(t2)

x ,y f (y)− f (x )

=∑y∈Ω

Qx ,y f (y) =Qx ,x f (x ) +∑

y∈Ω\x

Qx ,y f (y)

=∑

y∈Ω\x

Qx ,y f (y)−∑

y∈Ω\x

Qx ,y f (x )

=∑y∈Ω

Qx ,y( f (y)− f (x )),

(2.96)

after which one can conclude that Varπ[ f ] ≤ −κ(L f , f )π. When consideringthe function θ (X (t)) = 1[X (t) = z], we find using the definition of the variancethat

Varπ[θ] =12

∑x ,y∈Ω

1[x = z]−1[y = z]

2πxπy ≤

|Ω|2

2. (2.97)

Starting from any state y ∈ Ω, i.e. νy = 1 and νx = 0 for x 6= y , it holds that

dνdµ

2,µ=∑

x∈Ω

νx

µx

2µx

12 =

1pπy≤

1pminx∈Ωπx

, (2.98)

since µ= π.

Thus far, we have proved that

P[|Π[n]x −πx (R[n−1])| ≥ e[n]]≤

exp− (e

[n])2(t[n]−t[n−1])4κ|Ω|2

(minx πx (R[n]))12

(2.99)

Lemma 2.16 is a consequence of the upper bound in (2.99), which we nextshow by calculating a lower bound on the smallest stationary probability, i.e.minx∈Ωπx (R[n]), and by providing an upper bound on κ as well. The latter ispossible because we know the operator is setting transition rates according tothe algorithm in (2.15).

69


2.D.1 Lower bound on π

Because (i) π has the product-form in (2.5), and by (ii) non-negativity of theexponential function,

(2.100)

minx∈Ω

πx (R[n])

(i)=min

x∈Ω

¦ exp (AR[n] + b)x∑y∈Ω exp (AR[n] + b)y

©

(ii)≥

minx∈Ωexp (AR[n] + b)x

|Ω|maxx∈Ωexp (AR[n] + b)x

.

Then by (iii) monotonicity of the exponential function, and (iv) Lemma 2.15,

(2.101)

minx∈Ω

exp (AR[n] + b)x

(iii)= exp

minx∈Ω

(AR[n] + b)x

≥ expminx∈Ωbx +min

y∈Ω

d∑i=1

Ay ,iR[n]i

©

≥ expminx∈Ωbx −max

y∈Ω

d∑i=1

|Ay ,i ||R[n]i |

(iv)≥ exp

minx∈Ωbx − max

i=1,...,d

|R[0]i |+ cg

n∑m=1

a[m]

maxy∈Ω‖Ay ,·‖1

.

Here, maxy∈Ω ‖Ay ,·‖1 denotes the maximum absolute row sum of A. Similarly,we derive the upper bound

(2.102)

maxx∈Ω

exp (AR[n] + b)x

≤ expmaxx∈Ωbx + max

i=1,...,d

|R[0]i |+ cg

n∑m=1

a[m]


,

so that ultimately,(2.103)

minx∈Ω

πx (R[n])

≥1|Ω|

expminx∈Ωbx −max

x∈Ωbx − 2 max

i=1,...,d

|R[0]i |+ cg

n∑m=1

a[m]


.

We conclude with the constants c1, c2 > 0 as in Theorem 2.11, that for n ∈ N+,

(2.104)minx∈Ω

πx (R[n])− 1

2 ≤ c1 expc2

n∑m=1

a[m].

70

2.D. Proof of Lemma 2.16

2.D.2 Upper bound on κ

From (i) bounding the definition of κ, and (ii) a change of variables, it followsthat

κ(i)≤∑e∈E

∑γx ,y∈Γ |e∈γx ,y

‖γx ,y‖φ(ii)=

∑x ,y∈Ω

∑e∈γx ,y

‖γx ,y‖φ ≤ |G|2|E| maxx ,y∈Ω

‖γx ,y‖φ.

(2.105)Recalling (iii) the definition of path length, (2.91), and that φ(z, v) = πzQz,v ,we can subsequently bound

‖γx ,y‖φ(iii)=

∑e=(z,v)∈γx ,y

1πzQz,v

≤|E|

minx∈Ωπx (R[n])minz,v∈Ω|Qz,v>0Qz,v.

(2.106)Recall that R[n] is used to set rates in the generator matrix Q. More precisely,

the controlled rates are µ=m←(R[n]), see Section 2.2.1. The non-controllablerates are kept constant, which we will capture using the finite constant cQ in The-orem 2.11. What remains is to bound the controllable rates, which correspondsto the minimum

minz,v∈Ω|Qz,v>0, ∃i∈1,...,d:m

←i (R[n])=Qz,v

Qz,v= mini=1,...,d

m←i (R[n]). (2.107)

Bounding (2.105) using (2.104) and (2.106), we find that

κ≤ c21 |G|

2|E|2exp (2c2

∑nm=1 a[m])

mini=1,...,dcQ, m←i (R[n])

= c21 |G|

2|E|2 exp (2c2

n∑m=1

a[m]) maxi=1,...,d

¦ 1cQ

,1

m←i (R[n])

©.

(2.108)

In order to bound the denominator, recall the definitions of ϕ(n) and ofH [n] above Theorem 2.11. We then have that m←i (R

[n]) ≥ exp (−ϕ(n)) for alli = 1, . . . , d and n ∈ N+, so that maxi=1,...,d1/m←i (R

[n]) ≤ exp(ϕ(n)). Then,

κ≤ c21 |G|

2|E|2 exp (2c2

n∑m=1

a[m])max¦ 1

cQ, exp(ϕ(n))

©

≤ c21 |G|

2|E|2 exp (2c2

n∑m=1

a[m])max¦ 1

cQ, 1

exp(1[ϕ(n)> 0]ϕ(n))©

≤ c21 |G|

2|E|2 max¦ 1

cQ, 1©

exp2c2

n∑m=1

a[m] +1[ϕ(n)> 0]ϕ(n).

(2.109)

71


In particular, we can write the bound in (2.109) as

(2.110) κ≤1

4|Ω|2c3exp

c4

n∑m=1

a[m] +1[ϕ(n)> 0]ϕ(n),

where c2, c3, c4 are as in Theorem 2.11.The vertices of G correspond to the states in the state space Ω, so |G|= |Ω|<

∞. Furthermore, the number of edges is bounded by the number of edges in acomplete graph, so |E| ≤ |Ω|(|Ω| − 1)/2 <∞. Bounding (2.99) using (2.104)and our last bound on κ, we find that there exist constants c1, c2, c3 ∈ (0,∞),such that

(2.111)

P[|Π[n]x −πx (R[n−1])| ≥ e[n]]≤ c1 exp

c2

n∑m=1

a[m]

− c3(e[n])2(t[n] − t[n−1])exp

−c4

n∑m=1

a[m] −1[ϕ(n)> 0]ϕ(n)

.

This completes the proof.

2.E Proof of Proposition 2.2

We define L[0] = 0, H[0] = 0, and the sequences

(2.112) L[n] =n∑

m=1

1m ln (m+ 1)

, H[n] =n∑

m=1

1m

for n ∈ N+.

Lemma 2.21.(a) 1− ln2+ ln (n+ 1)≤ H[n] ≤ 1+ ln n for n ∈ N+,

(b) 1/ ln 2− ln (ln 2) + ln (ln (n+ 1))≤ L[n] for n ∈ N+,

(c) L[n] ≤ 1/ ln 2+ 1/(2 ln 3)− ln (ln2) + ln (ln n) for n≥ 2,

(d) L[n−1] < H[n+1] for n ∈ N+.

Proof.(a) This follows from bounding, for n≥ 1, 1− ln 2+ ln (n+ 1) = 1+

∫ n

1 (1/(x +1))d x ≤

∑nm=1(1/m)≤ 1+

∫ n

1 (1/x)d x = 1+ ln n.

72

2.E. Proof of Proposition 2.2

(b) We now bound, for n ≥ 1, 1/ ln2 − ln (ln2) + ln (ln (n+ 1)) = 1/ ln2 +∫ n

1 (1/((x + 1) ln (x + 1)))d x ≤∑n

m=1(1/(m ln (m+ 1))).

(c) For n≥ 2,∑n

m=1(1/(m ln (m+ 1)))≤ 1/ ln 2+1/(2 ln 3)+∫ n

2 (1/(x ln x))d x =1/ ln 2+ 1/(2 ln 3)− ln (ln2) + ln (ln n).

(d) For n = 1,2,3,4, it follows from evaluation that L[0] = 0 < 3/2 = H[2],L[1] = 1/(ln 2)< 11/6= H[3], L[2] = 1/(ln 2) + 1/(2 ln 3)< 25/12= H[4], andL[3] = 1/(ln2) + 1/(2 ln3) + 1/(3 ln4) < 137/60 = H[5]. Now consider n ≥ 5.Define f (x) = (x − 1) ln x , g(x) = x + 1. Calculate f ′(x) = ln x + (x − 1)/xand note that it is a strictly increasing function of x . Next, calculate g ′(x) = 1and conclude that f ′(2) > g ′(2). Hence, f ′(x) > g ′(x) for x ≥ 2. Finally notethat f (5)> g(5), so that f (x)> g(x) for x ≥ 5. Therefore, for n≥ 5, L[n−1] −L[n−2] = 1/((n−1) ln n)< 1/(n+1) = H[n+1]−H[n]. Assume that it was alreadyestablished that L[n−2] ≤ H[n], in which case L[n−1] = (L[n−1]− L[n−2])+ L[n−2] <

(H[n+1]−H[n])+H[n] = H[n+1]. This sets up an induction argument, completingthe proof.

Let a[n], e[n] and t[n]− t[n−1] be as in Proposition 2.2. Conditions (2.14) aresatisfied, because

∞∑n=1

a[n] = limn→∞

L[n] ≥1

ln2− ln (ln2) + lim

n→∞ln (ln (n+ 1)) =∞, (2.113)

and

∞∑n=1

(a[n])2 =∞∑n=1

1/(n ln (n+ 1))2 ≤ 1+∞∑n=2

1/n2 <∞. (2.114)

Next, we examine condition (2.16). We have (i,ii) by Lemma 2.21 that for thefirst part

∞∑n=1

a[n]

n−1∑m=1

a[m]2=∞∑n=1

L[n−1]

n ln (n+ 1)

2 (i)≤∞∑n=1

H[n+1]

n ln (n+ 1)

2

(ii)≤∞∑n=1

1+ ln (n+ 1)n ln (n+ 1)

2=∞∑n=1

1n2+

2n2 ln (n+ 1)

+1

n2(ln (n+ 1))2<∞.

(2.115)

73


We now turn to verifying the second part of (2.16). We again utilize (iii,iv)Lemma 2.21 and (v) the fact that α > 0, to verify that

(2.116)

∞∑n=1

e[n]a[n] n−1∑

m=1

a[m]=∞∑n=1

L[n−1]

n1+α/2 ln (n+ 1)

(iii)≤

∞∑n=1

H[n+1]

n1+α/2 ln (n+ 1)

(iv)≤

∞∑n=1

1+ ln (n+ 1)n1+α/2 ln (n+ 1)

=∞∑n=1

1n1+α/2 ln (n+ 1)

+1

n1+α/2

(v)<∞.

All that remains is to check whether (2.17) holds for all c2, c3, c4 ∈ (0,∞).Define cL = 1/ ln2+ 1/(2 ln3)− ln (ln 2), so that L[n] ≤ cL + ln (ln n) for n ≥ 2by Lemma 2.21. Also, L[n−1] ≤ H[n+1] ≤ 1+ ln (n+ 1) for n ∈ N+. Thus,

(2.117) (2.17)≤ ec2cL

∞∑n=2

cL + ln (ln n)n ln (n+ 1)

(ln n)c2 exp−c3e−c4cL nδ−α(ln n)−c4

.

We now note that for any ε > 0, ln n≤ nε for all sufficiently large n. This boundis very important, because it allows us to lose the dependence on c4. That is,define m(ε) = infn ∈ N+| ln n≤ nε, and split and further bound

(2.118) (2.17)≤ cS + ec2cL

∞∑n=m(ε)


(ln n)c2 exp−c3e−c4cL nδ−α−c4ε

for some constant cS that is the partial summation up to and including summandm(ε)−1, and it is the tail that we have bounded. We can choose ε freely, and optto choose some ε ∈ (0, (δ−α− 1)/c4] so that δ−α− c4ε ≥ 1. This is possible,because (i) c4 <∞, and (ii) δ > 1+α by assumption. We bound

(2.119)

(2.17)≤ cS + ec2cL

∞∑n=m(ε)


(ln n)c2 exp−c3e−c4cL n

=: cS + ec2cL

∞∑n=m(ε)

S[n],

and utilize the ratio test, which reveals that limn→∞ |S[n+1]/S[n]| = e−c3 < 1,since 0 < c3 <∞. Hence the tail of

∑∞n=m(ε) S

[n] converges, and this impliesthat condition (2.17) is satisfied.

74

CHAPTER 3Wireless network control ofinteracting Rydberg atoms

by Jaron Sanders, Rick van Bijnen,Edgar Vredenbregt, and Servaas Kokkelmans

Phys. Rev. Lett. 112, 163001

75

3. W I R E L E S S N E T W O R K C O N T R O L O F I N T E R A C T I N G RY D B E R G AT O M S

Abstract

We identify a relation between the dynamics of ultracold Ryd-berg gases in which atoms experience a strong dipole blockade andspontaneous emission, and a stochastic process that models certainwireless random-access networks. We then transfer insights andtechniques initially developed for these wireless networks to therealm of Rydberg gases, and explain how the Rydberg gas can bedriven into crystal formations using our understanding of wirelessnetworks. Finally, we propose a method to determine Rabi frequen-cies (laser intensities) such that particles in the Rydberg gas areexcited with specified target excitation probabilities, providing con-trol over mixed-state populations.

76

3.1. Introduction

3.1 Introduction

Stochastic processes play a ubiquitous role in interacting particle systems. Astudy of the stochastic Ising model was initiated by Glauber in 1963 [68], andsimilar models are actively investigated in probability theory, often applied tovery different systems [11, 12]. The two seemingly disparate interacting particlesystems we study in this Letter, are a gas of ultracold Rydberg atoms [24] ac-companied by a dissipative mechanism, and a wireless random-access network,made up of for example electronic transmitters in communication networks [13].It turns out that their dynamics can be described, under certain conditions, withthe same equations. Indeed, Rydberg atoms exhibit a strong interaction, whilesimultaneously active transmitters would lead to interference at receivers, bothresulting in complicated large-scale system behavior.

Rydberg gases consist of atoms that can be in either a ground state or anexcited state with a high principal quantum number. When an atom is excited,the energy levels of neighboring atoms shift. This makes it unlikely for neigh-boring atoms to also excite, and we call this effect the dipole blockade [26,27]. The dipole blockade is at the basis of quantum information and quantumgate protocols [26, 69, 70], and also allows for a phase transition to orderedstructures [71]. Experimentally, the cNOT gate1 has been demonstrated [70],while also the first ordered Rydberg structures have been observed [73]. Re-cent experiments are geared towards leveraging the dipole blockade to createRydberg crystals, i.e. formations of regularly spaced excited atoms. A proposedmethod is to use chirped laser pules [74–76], and another utilizes a dissipationmechanism: specifically spontaneous emission [77].

Nowadays, transmitters in wireless networks share a transmission mediumthrough the use of distributed random-access protocols. We focus on wirelessnetworks operating according to the CSMA protocol [19], which lets transmit-ters autonomously decide when to start a transmission based on the level ofactivity in their environment, usually estimated through measurements of in-terference and signal-to-noise ratios. If too many neighbors are sensed to betransmitting, the transmitter postpones its activation and tries again at a randomlater point in time. We see that transmitters experience blocking effects similarto the Rydberg dipole blockade, which sparked our original interest to compare

1The cNOT gate is a quantum gate that is an essential component in the construction of aquantum computer. For further background, see [72].

77


their mathematical models [1]. Mathematical models of wireless networks werealready being studied because of our increasing demands on our communicationinfrastructures, and we focussed our attention on stochastic models of CSMAthat were originally considered in [13–15].

This Letter uses the fact that rate equations adequately describe the Rydberggas when spontaneous emission is introduced to the model [78], and we inter-pret the rate equations as Kolmogorov forward equations [53] that describe thetransient evolution of a stochastic model reminiscent of CSMA.

3.2 Model comparison

Regarding the Rydberg gas, we consider a gas of N atoms in the µ-Kelvin regime,to which we apply the frozen gas approximation by neglecting the kinetic energyof the system. The atoms are thus considered fixed at positions r i ∈ R3 fori = 1, . . . , N . The ultracold atoms are subjected to two lasers with associatedRabi frequencies Ωe,Ωr, respectively, that facilitate excitation from the groundstate |g⟩ to an intermediate state |e⟩, and from the intermediate state |e⟩ to aRydberg state |r⟩. We also assume that the intermediate state decays with rateΓ , through spontaneous emission. In principle, detuning of the laser frequenciescould be taken into account, but here we leave it out for simplicity.

The system description of a wireless random-access network is similar tothat of the Rydberg gas, but with different terminology. A wireless random-access network can be modeled as consisting of N transmitter-receiver pairs,and each transmitter can be either active (1) or nonactive (0). When active,a transmitter transmits data for an exponentially distributed time with mean1/µ. Similarly, a nonactive transmitter repeatedly attempts to become activeafter exponentially distributed times with mean 1/ν. Figure 3.1 summarizes ourmodeling assumptions thus far.

For a single atom, we can write down the optical Bloch equations. As in [78],we then conclude that if (i) the upper transition is much more weakly driventhan the lower one (Ωr Ωe), and (ii) the decay rate of the intermediate levelis much larger than the Rabi frequency driving between |e⟩ and |r⟩ (Ωr Γ ),that then the excitation dynamics are described using the rate equation,

(3.1)dp1(t)

d t= νp0(t)−µp1(t).

78

3.2. Model comparison

|g〉

|e〉

|r〉

Ωe

Ωr

Γnonactive

active

µ ν

Figure 3.1: (left) An atom can transition between a ground, intermediate and aRydberg state. (right) A transmitter can change between nonactive and active.

Here, p0(t) and p1(t) denote the probabilities that the atom is (effectively) inthe ground state or the Rydberg state, respectively. Furthermore,

µ=2ΓΩ4

r

(Ω2r − 2Ω2

e)2 + 2Γ 2(Ω2e +Ω2

r ), and ν=

Ω2e

Ω2r

µ (3.2)

denote the transition rates between the ground and Rydberg state. It is note-worthy that (3.1) also describes the time evolution of a single, noninteractingtransmitter. The p0(t) and p1(t) are then the probabilities that the transmitteris nonactive or active, respectively.

When dealing with many-particle systems, however, we have to take particleinteractions into account. The atoms in Rydberg gases, and the transmitters inwireless networks, interact with each other. Specifically, if an atom is in the Ryd-berg state, other nearby atoms experience a dipole blockade [76]. Transmittersthat detect high levels of interference and low signal-to-noise ratios (because oftheir neighbors) postpone their activation.

We will model the dipole blockade, as well as the interference constraintson transmitters, using a unit-disk interference model. The unit-disk interferencemodel involves the assumption that atoms (transmitters) within a distance R ofeach other cannot simultaneously be in the Rydberg state (active). For Rydberggases, this assumption is in line with measurements and simulations of paircorrelation functions between atoms in the Rydberg state, which show a sharpcutoff when plotted as a function of the distance between the atoms [28, 73].The collection of possible configurations is thus

Ω=x ∈ 0,1N

d(r i , r j)> R∀i 6= j:x i=x j=1

, (3.3)

79


and these configurations x = (x1, . . . , xN )T will be called feasible. The notationis such that if x i = 0 or 1, atom i is in the ground or Rydberg state, respectively.Similarly, x i = 0 or 1 if transmitter i is nonactive or active, respectively.

There are certainly practical differences between Rydberg gases and wirelessnetworks. In wireless networks, every transmitter can have its own activation(νi) and deactivation rate (µi). To achieve the same effect in Rydberg gases,we will assume that the two-step laser can be split into M N spots withradius S, and that each spot i = 1, . . . , M has a different laser intensity Ei . Eachlaser spot contains a cluster of atoms, and with this setup, the atoms withineach cluster may be subjected to a different Rabi frequency. We assume thatS R, so that we can treat each spot as being synonymous to one atom, and wewill replace the symbol M by N for notational convenience. Each atom (spot)i = 1, . . . , N will thus experience its own transition rates νi , µi . Figure 3.2summarizes the blockade effect, our assumptions on the laser spots, and theunit-disk interference model.

EiEj

R

S

Figure 3.2: The Rydberg blockade prevents atoms within a radius R from becom-ing Rydberg atoms (left). This interaction can be described using an interferencegraph where edges indicate which neighboring particles would block each other,which is part of the wireless network model (right). Active transmitters (black)prevent neighboring transmitters (red) from becoming active. Non-neighboringnonactive transmitters (white) can become active.

For both models, the probability of observing the system in state x ∈ Ω at

80

3.3. Rydberg crystals

time t, denoted by px (t), is described by the master equation

dpx (t)d t

=∑y∈Ω

Qy ,x py(t), (3.4)

where Q denotes a transition rate matrix. The master equation (3.4) can beinterpreted as a Kolmogorov forward equation, which characterizes a Markovprocess [17, 53].

The off-diagonal elements of Q ∈ R|Ω|×|Ω| describe the dynamics of thisstochastic process. Denoting the N -dimensional vector with a one in the ithposition by e i , we have that when the system is in state x ∈ Ω, it jumps to statesx + e i , i = 1, . . . , N , with rate Qx ,x+e i

= νi if x + e i ∈ Ω, and to states x − e i ,i = 1, . . . , N , with rate Qx ,x−e i

= µi if x −e i ∈ Ω. All other off-diagonal elementsof Q are set to zero, which (for the Rydberg gas model) means that we neglectmultiphoton processes. For completeness, we note that the diagonal elementsare given by Qx ,x = −

∑y 6=x Qx ,y . We conclude that the stochastic process de-

scribed by the generator matrix Q, which we denote by X(t)t≥0, is a modelfor wireless random-access networks, as well as Rydberg gases.

3.3 Rydberg crystals

We now investigate steady states of the Rydberg gas, using our understandingof wireless networks. The equilibrium fraction of time that the system spendsin state x is given by

πx (ν,µ) =1

Z(ν,µ)

N∏i=1

νi

µi

x i, x ∈ Ω, (3.5)

with Z(ν,µ) denoting the normalization constant. The equilibrium distributiondepends solely on the ratios νi/µi , and proving that it is in fact the equilib-rium distribution can be done by observing that it satisfies the detailed balanceequations [17], πxQx ,y = πyQy ,x , for all x , y ∈ Ω.

Consider the special case in which all particles make their transition at thesame rate, and set νi = ν and µi = µ for i = 1, . . . , N accordingly. When ν/µ→∞, the equilibrium probability of observing the system in state x ∈ Ω convergesto

πx (ν,µ) =1

Z(ν,µ)

νµ

∑Ni=1 x i→

1[x ∈ I ]|I |

, (3.6)

81


where I denotes the collection of maximum independent sets of Ω. In thepresent context, a maximum independent set is a configuration in which thelargest number of particles are active, i.e. in the Rydberg state. We call theseconfigurations dominant, because the probability of observing a dominant con-figuration x ∈ I , πx , is large compared to (i) the probability of observing aconfiguration y /∈ I , πy , when ν µ, and (ii) the probability of observing anybut the dominant configuration,

∑y /∈I πy , when ν Nµ.

The active particles in dominant configurations typically form patterns thatresemble crystal structures. Consider for instance an n×m lattice of particlesexhibiting nearest-neighbor blocking, where n, m ∈ N+. For such networks, theactive particles in the dominant configuration follow a checkerboard pattern, asillustrated in Figure 3.3. When both n and m are even, two dominant configu-rations exist, which we henceforth refer to as the even and odd configuration.

Figure 3.3: Dominant configurations in a (left) 9× 5 lattice and (middle, right)4× 4 lattice with nearest-neighbor blocking.

Our analysis reveals that when Ωr Ωe, the Rydberg gas spends more timein a dominant configuration than in another configuration. The time it takesfor the system to switch between different dominant configurations is relatedto the mixing time of the system, i.e. the time required for the Markov processto get sufficiently close to stationarity [51]. Depending on the topology, themixing time can be large when Ωr Ωe, implying that once the system is in adominant configuration, it tends to stay there for a long time. It is noteworthythat simulations of a driven dissipative Rydberg gas confirmed the formationof crystalline structures in [77, 79, 80], and here we have explained how suchformations appear using our connection to wireless networks.

We are also able to investigate the time τ it takes until the process reachesa dominant configuration. The hitting time τ of the dominant configuration

82

3.4. Control algorithm

is the first moment at which the system reaches the even or odd dominantconfiguration. The random variable τ is of interest, because it is a measure forhow long the experimentalist has to wait before a dominant configuration hasappeared.

To illustrate this, we have simulated sample paths of X(t) on even n×n latticetopologies. Histograms of the hitting time distributions for grids of several sizesare shown in Figure 3.4, as well as the normalized average number of excitedparticles. Note that the average hitting time increases as lattices become larger.

0 200 4000

0.1

t (µs)

P[τ=t]

0 5 100

1

t (µs)

E[∑

i2X

i(t)

N]4 × 4

6 × 6

8 × 8

Figure 3.4: Histograms of the hitting time distributions (left), and normalizedaverage number of excited particles (right) for lattices of sizes 4× 4, 6× 6, and8× 8. Here, Γ = 2π · 6MHz, Ωe,i = 2π · 3MHz and Ωr,i = 2π · 1 MHz.

3.4 Control algorithm

We now describe a wireless network algorithm in the context of Rydberg gases, todetermine Rabi frequencies (laser intensities) such that particles in the Rydberggas are excited with specified target excitation probabilities. The algorithm wasdeveloped in [22] to achieve maximum throughput in wireless networks in adistributed fashion, and was later generalized for implementation in product-form networks [10, Chapter 2]. In our Supplemental material §3.A–§3.B, weprovide a short discussion of the algorithm in its original context, and we explainthat the algorithm is solving an inversion problem that can be NP hard.

The wireless network algorithm can be applied to the Rydberg atoms by

83


iteratively setting

(3.7) |Ω[n+1]e,i |= |Ω[n]e,i |exp

−

12

a[n+1](θ [n+1]i −φi)

,

for atoms i = 1, . . . , N . Here, n ∈ N+ indexes each iteration, and the a[n] denotealgorithm step sizes that are typically chosen as a decreasing sequence. Theθ[n+1]i denote empirically obtained estimates of the probabilities of observing

atom i = 1, . . . , N in the Rydberg state, θi , and φi denotes the target probabilityof observing atom i in the Rydberg state. The algorithm in (3.7) seeks Ωopt

e suchthat θ (Ωopt

e ,Ωr) = φ.

In wireless networks, an estimate θi can be obtained through online observa-tion of a transmitter’s activity (see §3.A, Eq. (3.12)). Experimentally observingthe evolution of a particle system through time however is difficult. Instead,we can (i) determine an estimate θ of θ using simulation, or (ii) use repeatedexperimentation to determine an estimate θ of θ . With the latter approach, weforego our mathematical guarantee of convergence, but the design principlesthat guaranteed the convergence in the former method still hold. That is, weneed to improve the quality of θ as the number of iterations n increases.

For every nth iteration of the algorithm, we can for example reinitialize theprocess m[n] times and determine the state the process is in at some time T [n].Denoting these samples by X [n,s]

i (T [n]), with n ∈ N+ and s ∈ 1, . . . , m[n], wecan calculate

(3.8) θ[n]i =

1m[n]

m[n]∑s=1

1[X [n,s]i (T [n]) = 1], i = 1, . . . , N ,

which, for sufficiently large T [n] and m[n], provides an estimate of the equilibriumprobability that particle i is in the Rydberg state. Intuitively, we expect that T [n]

should be at least of the order of the mixing time (that is to say, the systemshould be close to equilibrium).

As an example, we focus on a system of i = 1, . . . , N atoms positioned on aline, that block the first b neighbors on both sides. We consider the problem ofdetermining Ωe such that each atom is excited with equal probability φ ∈ (0, 1).This problem is nontrivial because the atoms at the border have fewer neighborsthat block them and are therefore excited with higher probability. Moreover, thiseffect propagates through the system, which can be verified by an analytical

84

3.4. Control algorithm

evaluation of the probabilities of observing atom i in the Rydberg state,

θi(ν,µ) =∑x∈Ω

x iπx (ν,µ), i = 1, . . . , N , (3.9)

as shown in Figure 3.5.

1 2 3 4 5 6 7 8 90

1

Atom i

θ i(ν,µ

)

Figure 3.5: The θi(ν,µ) for N = 9, b = 1, and νi/µi = 10.

We consider this particular example because we can again utilize our con-nection to wireless networks and provide an analytical expression for Ωopt

e . Asshown in [81], we need to set

Ωopte,i

Ωr,i

2=

φ

1− (1+ b)φ

1− bφ1− (1+ b)φ

w(i)−w(1)(3.10)

for i = 1, . . . , N , in order to have θi(Ωopte ,Ωr) = φ for i = 1, . . . , N . Here, w(i) =

mini + b, N −max1, i − b denotes the number of other atoms that atom iblocks if it is excited.

In order to illustrate the algorithm applied to this system, we utilize thefollowing simulation procedure. We repeatedly simulate the Rydberg gas bygenerating sample paths X [n,s](t) using the generator matrix in (3.4). Subse-quently, we calculate an estimate of the excitation probabilities through (3.8),and update the Rabi frequencies according to the algorithm in (3.7). In every nthiteration of our algorithm, we set the maximum simulation time to T [n] = 250µs,produce m[n] = 25n2 samples, and choose step size a[n] = 100/(10+

pn). The

target excitation probability of the algorithm is set to φ = 1/6. The resultingRabi frequencies are shown in Figure 3.6, and approach the exact solution givenby (3.10),

Ωopte = (1,

p2,2, 2

p2,4, 2

p2, 2,p

2, 1)T · 2πMHz. (3.11)

85


The excitation probabilities approach the target φ, which can be verified byevaluating (3.9) after several iterations, as shown in Figure 3.7.

0 1 2 3 4 5 6 7 8 9 10

1

2

3

4

5

Iteration n

Ω[n

]e,i

(2π

MH

z)

1, 92, 8

3, 7

4, 6

5

Atomposition

Figure 3.6: Algorithm output when N = 9, b = 4, Γ = 2π · 6 MHz, and Ωr,i =2π · 1MHz. The dotted lines indicate Ωopt

e .

1 2 3 4 5 6 7 8 90

16

13

Atom i

θ i(ν

[n] ,µ)

n = 0

n = 3n = 10

Figure 3.7: The excitation probabilities θi(ν[n],µ) after iterations n= 0, 3, and10.

By manipulating excitation probabilities, we control the populations of mixedstates. This can be of interest to (for example) mixed state quantum computing,which lies in between classical computing and quantum computing based onpure, entangled states [82, 83]. Creating mixed states can also be a first steptowards efficient preparation of large qubit entangled states.

86

3.5. Conclusion

3.5 Conclusion

In conclusion, we studied the relations between a physical model of ultracoldRydberg atoms and a stochastic process that models certain wireless random-access networks. This allowed us to identify interesting connections betweenresearch fields in physics and mathematics, and to transfer techniques and in-sights to the realm of Rydberg gases. Our approach can be applied to many otherparticle systems and stochastic processes as well. Furthermore, the algorithmcan be applied to a much larger class of product-form networks, with differentadjustable parameters [9, 10, Chapter 2]. Whenever dynamical systems are welldescribed using rate equations, it can be worthwhile to explore possible relationswith stochastic processes and cross-pollinate ideas.

Supplemental material

3.A Wireless network algorithm

We propose using a wireless network algorithm to achieve target excitationprobabilities in a Rydberg gas. The algorithm will calculate Rabi frequencies sothat the probability that particle i is in the Rydberg state is equal to a targetprobability φi ∈ (0,1) that we can specify for every particle i = 1, . . . , N . Informula, the probability that particle i is in the Rydberg state is given by (3.9).The dependence on the system parameters ν, µ is emphasized by writing theequilibrium distribution as a function of ν and µ.

In the context of a wireless random-access network, this performance mea-sure is called normalized throughput and may in fact not be fair for end-users.We have for instance seen that when ν µ (a heavily loaded system), certaintransmitters are active for much larger fractions of time than other transmitters,recall (3.6). This leads to starvation effects and unfairness in wireless random-access networks, which has been studied in for example [42, 84–86]. In wire-less networks, it is therefore desirable to find system parameters νopt such thatθ (νopt,µ) = φ, where φ denotes fairer probabilities that transmitters are active.This problem is difficult to solve analytically, as well as numerically. For example,in order to evaluate (3.6), we need to determine all maximum independent setsof a graph, which is a well-known NP-hard problem.

87


Recent studies of the wireless network model have led to the developmentof distributed algorithms to solve θ (νopt,µ) = φ for νopt, without needing tonumerically evaluate the equilibrium distribution. Let 0 = t[0] < t[1] < . . .denote points in time, constituting time slots [t[n], t[n+1]]. At time t[n+1], theend of the (n + 1)-th time slot, each transmitter i = 1, . . . , N calculates theempirical estimate

(3.12) θ[n+1]i =

1t[n+1] − t[n]

∫ t[n+1]

t[n]1[X [n]i (t) = 1]d t

of the probability that it was active. Here, X [n](t) : [t[n], t[n+1])→ Ω denotes arealized sample path of the stochastic process that describes the wireless networkoperating with ratesµ andν[n] during time interval [t[n], t[n+1]). Keepingµ fixed,transmitters i = 1, . . . , N next update

(3.13) ν[n+1]i = ν[n]i exp

−a[n+1](θ [n+1]

i −φi), n ∈ N0,

where we denote the step size of the algorithm by a[n]. This update procedure isthen repeated. It is noteworthy that the right-hand side in (3.13) is independentof θ j and φ j for j 6= i, allowing each transmitter to base its decisions on localinformation only.

If the step sizes a[n] and time-slots are chosen appropriately, ν[n] convergesto νopt with probability one [10, 22]. The idea is to increase t[n+1] − t[n] as nincreases such that better estimates of θ [n] are obtained, while slowly decreasinga[n] to prevent poor decision-taking, but not too slow and be unresponsive. Theprecise form of the conditions depends on the specific modelling conditions. Ifthe set of feasible configurations is finite and if one projects the outcome of thealgorithm to a compact set, which are reasonable assumptions in practice due totypically having finite capacities and resources, it suffices to choose a[n] and t[n]

such that∑∞

n=1 a[n] =∞,∑∞

n=1(a[n])2 <∞ and

∑∞n=1 a[n]/(t[n]− t[n−1])<∞,

e.g. a[n] = n−1, t[n] − t[n−1] = n.

3.B Achievable region

A technical difficulty is determining whether there even exists a finite νopt suchthat θ (νopt,µ) = φ. If such νopt exists, we call φ achievable. We readily ob-tain the answer by again looking at known results for the wireless network

88

3.B. Achievable region

model [22], which have also later been generalized for product-form networks[9, Chapter 2]. In short, any

φ ∈∑

x∈Ωαx x

α ∈ (0, 1)|Ω|,αT1|Ω| = 1

(3.14)

is achievable. Here, 1|Ω| denotes the |Ω|-dimensional vector that contains allones. (3.14) is a convex hull of the configurations in Ω.

89

CHAPTER 4Sub-Poissonian statistics

of jamming limits inultracold Rydberg gases

by Jaron Sanders, Matthieu Jonckheere, and Servaas KokkelmansPhys. Rev. Lett. 115, 043002

91

4. SU B -PO I S S O N I A N S TAT I S T I C S I N U LT R A C O L D RY D B E R G G A S E S

Abstract

Several recent experiments have established by measuring theMandel Q parameter that the number of Rydberg excitations in ultra-cold gases exhibits sub-Poissonian statistics. This effect is attributedto the Rydberg blockade that occurs due to the strong interatomicinteractions between highly excited atoms. Because of this blockadeeffect, the system can end up in a state in which all particles areeither excited or blocked: a jamming limit. We analyze appropri-ately constructed random-graph models that capture the blockadeeffect, and derive formulae for the mean and variance of the num-ber of Rydberg excitations in jamming limits. This yields an explicitrelationship between the Mandel Q parameter and the blockade ef-fect, and comparison to measurement data shows strong agreementbetween theory and experiment.

92

4.1. Introduction

4.1 Introduction

Ultracold gases with atoms in highly excited states have attracted substantial in-terest over recent years, for example, for their potential application in quantumcomputing [26, 27, 87], and for the study of nonequilibrium phase transitions[71]. These atomic systems exhibit complicated spatial behavior due to strongvan der Waals or dipolar interactions between neighboring atoms, which hasbeen demonstrated through several experimental observations of reduced fluc-tuation in the number of excitations in ultracold gases of Rydberg atoms [30–35].

In these experiments, a laser facilitates excitation of ultracold atoms into aRydberg state. After some time t, information on the mean and variance of thenumber of excited particles X (t) is obtained by repeating counting experiments,and the Mandel Q parameter [88]

Q(t) =Var[X (t)]E[X (t)]

− 1 (4.1)

is calculated to quantify a deviation from Poisson statistics, since if X (t) is Pois-son distributed, Q(t) = 0. The experiments establish that Q(t) < 0 for t > 0,and X (t) is said to have a sub-Poisson distribution.

The observed negative Mandel Q parameter is attributed to the Rydbergblockade effect [26, 27]. There exist simulation techniques [80] and modelsbased on Dicke states [31] that numerically describe the Mandel Q parameter.Besides for a one-dimensional system with reversible dynamics [89], no closed-form expression appears to be available that describes the relation between theMandel Q parameter and the blockade effect.

Explicit formulae for the Mandel Q parameter are difficult to obtain, becausethe problem at hand is reminiscent of parking processes [36] and irreversiblecontinuum random sequential adsorption problems [37], which is also where theterm jamming limit comes from. The standard two-dimensional continuum ran-dom sequential adsorption problem is that of throwing disks of radius r > 0 oneby one randomly in a two-dimensional box, such that the disks do not overlap.Except for the one-dimensional variant, such problems are notoriously challeng-ing to analyze due to spatial correlations. One further question is whether suchstochastic processes are suited to explain effects occurring in ultracold Rydberggases, and if so, under what conditions. This matter is discussed in Ref. [3, Chap-ter 3], where a suitable stochastic process is provided based on rate equations

93


that adequately describe the Rydberg gas when an incoherent process (such asspontaneous emission) occurs [78].

This Letter adopts the stochastic process in Ref. [3, Chapter 3] that modelsthe Rydberg gas, and uses it to study the Mandel Q parameter in the jamminglimit which occurs when atoms only transition from the ground state to theRydberg state. The model includes the blockade effect through so-called in-terference graphs, and by considering specially constructed large Erdös–Rényi(ER) random graphs [38] that retain essential features of the blockade effect, weovercome the mathematical difficulties normally involved with having a spatialcomponent. The problem remains nontrivial though, and we point interestedreaders to the rigorous derivation of the necessary fluid and diffusion limits [5,Chapter 5]. This Letter explains how to use these theoretical insights in the con-text of Rydberg gases through less complicated heuristic arguments, and whiledoing so explicitly relates the mean and variance of the number of excitationsto the blockade effect.

4.2 Fitting the ER graph

We consider a gas of ultracold atoms in an excitation volume V ⊆ R3, andwe assume that each particle has its own distinct position. Each particle cango from a ground state to a Rydberg state, and a particle in the Rydberg stateprevents neighboring particles from also entering the Rydberg state. The densityof particles is assumed to be ρ, and the number of excitable particles N withinany region A⊂ V to be Poisson distributed with parameter ρA. This implies inparticular that in the absence of blockade effects, the number of excited particleswithin the excitation volume, X (t), will be Poisson distributed, as is the case inexperiments [30–35]. It also implies that the particles are uniformly distributedat random over the excitation volume.

The blockade effect will be modeled using the notion of a blockade radius r.This is in line with simulations and measurements of pair correlation functionsbetween atoms in the Rydberg state, which show a sharp cutoff when plotted as afunction of the distance between the atoms [28, 29]. Particles within a distancer > 0 are considered neighbors of each other, and neighbors each block theother if excited. We denote the number of neighbors of a particle i = 0, 1, . . . , Nwithin its blocking volume Vb,i by Bi . As a consequence of these assumptions,

94

4.3. Spatial dynamics and jamming limits

the number of neighbors of particle i is also Poisson distributed. Specifically,

P[Bi = b] =(ρVb,i)be−ρVb,i

b!, b = 0,1, . . . , (4.2)

if Vb,i is fully contained within V .We will study the number of excitations by examining the asymptotic behav-

ior of large ER random graphs. Each vertex of such a graph will represent oneparticle, so its set of vertices is V = 1, . . . , N. We draw an edge between twoparticles i and j if we consider i and j to be neighbors (particles that would blockone another). One can construct an ER random graph by considering every pairof vertices (i, j) once, and drawing the edge between i and j with probabilityp, independent from all other edges. In order to deduce information on X (t)through examining the ER random graph, we need to match the ER randomgraph model to the physical system, and we will do so by counting and matchingthe number of neighbors. Matching the models has to be done via the numberof neighbors, because there is no such notion as a physical position of a particlein an ER random graph. This principle, in fact, makes this mathematical modeltractable.

The number of neighbors BER,i of a particle i in the ER random graph isbinomially distributed, BER,i ∼ Bin(N − 1, p), so that for b = 0,1, . . . , N − 1,P[BER,i = b] =

N−1b

pb(1 − p)N−1−b, and E[BER,i] = (N − 1)p. When setting

p = c/N where c is some constant, we see that as N →∞, the distributionconverges to a Poisson distribution,

limN→∞P[BER,i = b] =

cbe−c

b!, b = 0,1, . . . . (4.3)

Comparing Eq. (4.3) to Eq. (4.2), we note that the limiting distribution is thesame if the average number of neighbors in the ER random graph, c, is relatedto the density and blockade volume as c = ρVb. By setting c = ρVb, we ensurethat the particles in the ER random graph have the same distribution of numberof neighbors as in the spatial problem when the number of particles N →∞.Figure 4.1 summarizes the construction.

4.3 Spatial dynamics and jamming limits

Let us now describe the spatial dynamics, illustrated in Figure 4.2. At time T0 = 0the laser is activated, and from that point onward excitations can occur. At a

95


Figure 4.1: (left) A spatial Poisson point process in which neighbors withinradius r block each other is used to choose appropriate parameters for (right)an ER random graph so that the particles have the same distribution of numberof neighbors as N →∞. Note that this identification procedure only matchesthe distribution of the number of neighbors, and does not entail a mappingbetween the specific particles in both models.

time T1 > T0, the first particle, 1, excites and enters the Rydberg state. Becauseof the Rydberg blockade, particle 1 will subsequently prevent all other particleswithin a radius r from also becoming excited. Later, at a time T2 > T1, a secondparticle, 2, excites, which cannot be within distance r of particle 1. Particle 2from that point onward also blocks particles within a distance r of itself. Thisprocess continues until some finite time TX (∞) <∞when all particles are eitherblocked or excited. The random number of excited particles 1≤ X (∞)≤ N isthen detected.

4.4 Exploration process and fluid limits

The spatial dynamics are mimicked when building the ER random graph throughan exploration algorithm [90] as follows. An unaffected particle is chosen uni-formly at random. It becomes excited , and simultaneously a random subsetof unaffected particles become blocked , to which we draw an edge. This re-peats until all particles are excited or blocked, and a jamming limit has beenconstructed. The Supplemental Material defines the exploration appropriately,

96

4.4. Exploration process and fluid limits

Figure 4.2: (left) A random first particle excites. (middle) Subsequently, randomsecond and third particles excite. (right) The process continues until all particlesare either blocked or excited, and the resulting state is a jamming limit.

see §4.A.To derive the Mandel Q parameter, we need a stochastic recursion for the

number of unaffected particles Um at each mth moment an excitation occurs.This is obtained when considering that when the (m+ 1)th excitation occurs,the number of unaffected particles decreases by (i) the one particle that excites,and (ii) a random number of unaffected particles that each is a neighbor of thenew excitation with probability p and thus now become blocked. Conditionalon there being N = n particles in the excitation volume, we thus have

Um+1 = Um − 1− Bin(Um − 1, p), U0 = n. (4.4)

We will now analyze this stochastic recursion, and identify the moment τ thenumber of unaffected particles is zero, i.e. Uτ = 0. Precisely at this moment, wehave that the number of excitations X (∞) = τ.

The Supplemental Material details the following steps, see §4.B–§4.C. FromEq. (4.4), we obtain a closed-form expression for E[Um] by invoking the towerproperty and giving an induction argument. Through decomposition, we subse-quently obtain an expression for Var[Um]. When scaling the probability of beingneighbors as p = c/n, the mean and variance converge to fluid limits, whichcan be seen by letting f ∈ [0, 1], and proving that as n→∞,

E[U[ f n]]

n→ u( f ),

Var[U[ f n]]

n→ v( f ). (4.5)

Here, [·] denotes rounding to the nearest integer, and the fluid limits are u( f ) =e−c f − (1− e−c f )/c, and v( f ) = (e−c f (1− e−c f )((1+ 2c)e−c f − 1))/(2c). Notethat these fluid limits are rigorously proven in Ref. [5, Chapter 5].

97


Consider now Figure 4.3 (left) and the following steps. The process Um hitszero when m ≈ f ∗n, with f ∗ = ln (1+ c)/c being the solution to u( f ∗) = 0.Therefore,

(4.6) E[X (∞)|N]≈ f ∗n=n ln (1+ c)

c.

To approximate the variance, calculate u′( f ) and note that u′( f ∗ + ε)≈ −1 forsufficiently small ε. Since Um is probably near 0 for m ≈ f ∗n, the fluctuationsin X (∞) will thus be of the order of

ÆVar[U[ f ∗n]]. Hence,

(4.7) Var[X (∞)|N]≈ Var[U[ f ∗n]]≈ v( f ∗)n=nc

2(1+ c)2.

Invoking the central limit theorem, we have for large fixed n that the numberof excitations is approximately normal distributed with mean n ln (1+ c)/c andstandard deviation

pnc/(2(1+ c)2). This limit result, Eq. (4.6), and Eq. (4.7)

are formally established by deriving diffusion limits in [5, Chapter 5].

4.5 Comparison to random sequential adsorption

We will now compare Eqs. (4.6) and (4.7) to simulations of the mean and vari-ance observed in the two-dimensional random sequential adsorption problemdescribed earlier, and with periodic boundary conditions. We consider h= 1µm,l = w = 400µm, r = 6.5µm, and ρ = 5 × 109cm−3, which are typical val-ues in magneto-optical traps, and correspond to n ≈ 800 and c ≈ 0.664. Fig-ure 4.3 (right) shows a histogram of the number of excitations, as well as theprobability density function of a normal distribution with mean n ln (1+ c)/cand variance nc/(2(1+ c)2). Compared to the simulation’s outcome, the expres-sions differ for this set of parameters (i) 2.6% for the mean, (ii) 2.5% for thevariance, and (iii) 0.015% for the Mandel Q parameter. Because the mean andvariance are both overestimated, the Mandel Q parameter happens to be moreaccurately approximated. The errors the approximation makes can be attributedto the fact that particles in the ER random graph model have no physical posi-tion, whereas particles in two-dimensional Poisson disk throwing processes doexhibit spatial correlations. Intriguingly the random graph, which has no spatialinterpretation, yields a good approximation.

98

4.6. Mandel Q parameter

1

0

1

Var[X(∞)|N ]

Var[U[f∗n]]

f

u(f

)

0 200 400 600 8000

6·10−2

x

dP[X(∞

)=

x]

Figure 4.3: (left) The fluid limit u( f ) together with a sample path of Um/n forn= 50. The dashed line indicates the tangent at f ∗, and the arrows indicate thetypical fluctuations. (right) Histogram of X (∞) in a two-dimensional randomsequential adsorption problem with precisely n = 800 particles, together withthe probability density function of a normal distribution with mean n ln (1+ c)/cand variance nc/(2(1+ c)2).

4.6 Mandel Q parameter

It is important to understand that the results thus far are conditional on therebeing N = n particles within the excitation volume. However, the number ofparticles within the excitation volume is random and Poisson distributed, specif-ically N ∼ Poi(ρV ). To obtain an unconditional expression for the mean andvariance, we can utilize the tower property,

E[X (∞)] = E[E[X (∞)|N]]≈ E[N] ln (1+ c)/c = ρV ln (1+ c)/c, (4.8)

and decomposition,

Var[X (∞)] = E[Var[X (∞)|N]] + Var[E[X (∞)|N]]

≈ E Nc

2(1+ c)2+ Var

N ln (1+ c)c

= c

2(1+ c)2+ ln (1+ c)

c

2ρV.

(4.9)

Recalling definition Eq. (4.1), the Mandel Q parameter in the jamming limit istherefore

Q(∞)≈c2

2(1+ c)2 ln (1+ c)+

ln (1+ c)c

− 1, (4.10)

99


which is exact in the ER case when ρV → ∞ [5, Chapter 5]. Note that Eq.(4.10) only depends on the average number of neighbors c, which in fact explainsobservations on simulated Mandel Q parameters [80] as we discuss in Ref. §4.D.

Let us also discuss the time dependency of the mean number of excitations.We incorporate time dependency by assuming that every unaffected particle ex-cites at rate λ, and specifically that Tm−Tm−1 ∼ Exp(λUm−1), which correspondsto modeling the Rydberg gas using rate equations [78] as discussed in Ref. [3,Chapter 3]. Under these assumptions, we obtain the time-dependent fluid limit[5, Chapter 5]

(4.11)E[X (t)|N]

n→ x(t) = λ

∫ t

0

u(x(s))ds.

After substituting u( f ) = e−c f −(1−e−c f )/c into Eq. (4.11), recalling that initiallyno particles are excited, and taking the derivative, we obtain the differentialsystem dx/dt = λ(exp (−cx(t))−(1−exp (−cx(t)))/c), with x(0) = 0, for x(t).This differential system has as its unique solution x(t) = ln (1+ c − ce−λt)/c,and in particular, we recover the mean fraction of excitations in the jamminglimit by calculating limt→∞ x(t) = ln (1+ c)/c.

4.7 Comparison to measurements

We now validate the model by comparisons with experimental data in Refs.[31, 32], which requires us to incorporate the notion of a detector efficiencyη ∈ [0, 1] into the model. The detector efficiency η can be interpreted as beingthe probability that a Rydberg atom is detected. Let Ii ∼ Ber(η) denote randomvariables that indicate whether each ith Rydberg atom is detected. The numberof detected Rydberg atoms is then given by XD(t) =

∑X (t)i=1 Ii . Assuming the

I1, . . . , IX (t) are independent, calculation shows that E[XD(t)] = ηE[X (t)], andVar[XD(t)] = η2Var[X (t)] + η(1− η)E[X (t)], see the Supplemental Material,§4.E. The detected Mandel Q parameter thus reduces to QD(t) = ηQ(t), seealso Ref. [30].

The experiments in Ref. [31] were on excitation volumes said to containρV = 8 × 103 ground-state atoms, and with a reported detector efficiency ofη= 0.40. Fitting

(4.12) E[XD(t)]≈ηρV ln (1+ c − ce−λt)

c

100

4.7. Comparison to measurements

to measurements of the number of detected excitations as a function of time [31,Fig. 1(a)], we obtain an excitation rate of λ = 14kHz, and average number ofneighbors of c = 2.7× 102. Figure 4.4 shows strong agreement between theoryand experiment.

0 1 20

10

20

30

t(µs)

E[X

D(t)]

Figure 4.4: The average number of detected excitations as a function of time,E[XD(t)], fitted to the measurement data in [31, Fig. 1(a)]. The fit results in anexcitation rate of λ= 14kHz, and average number of neighbors of c = 2.7×102.

Lastly, we will compare the model to a histogram of the number of detecteddark-state polaritons in Ref. [32]. The histogram displays sub-Poissonian statis-tics due to a blockade effect that is a result of the dominant Rydberg char-acter of the polaritons. Because of a partial overlap between the excitationlaser and the cigar-shaped atomic cloud, we will infer the size of the excita-tion volume using the density ρ = 5× 1017m−3 [32] as follows. The detectorefficiency is reported to be η = 0.4, and the histogram has a sample mean ofE[XD(∞)] ≈ 11. If we assume that the blockade regions are spherical, andsince the blockade radius r ≈ 5µm [32], we find that c = 4

3ρπr3 ≈ 2.6× 102.Using the formula for the mean number of detected Rydberg atoms, it followsthat V = cE[XD(∞)]/(ρη ln (1+ c)) ≈ 2.6× 10−15m3. The factor with whichthe density function of the Poisson distribution is scaled in Ref. [32, Fig. 4(a)]is ns ≈ 315. Figure 4.5 now compares the appropriately scaled probability den-sity function of a normal distribution with mean and variance as predicted bythe model to the histogram in Ref. [32, Fig. 4(a)]. The result QD ≈ −0.36 isconsistent with their observation that QD = −0.32± 0.04 in the density range2× 1017m−3 < ρ < 2× 1018m−3.

101


0 10 200

10

20

30

40

50

x

n s·dP[

XD(∞)=

x]

Figure 4.5: Histogram [32, Fig. 4(a)] of the number of detected dark-state po-laritons, together with the appropriately scaled probability density function ofa normal distribution with mean E[XD(∞)] and variance Var[XD(∞)]. Here,QD(∞)≈ −0.36, and the dashed line indicates the Poisson distribution.

4.8 Conclusion

This Letter derived closed-form expressions for the Mandel Q parameter in lim-iting large random graphs constructed to model the spatial problem. This ap-proach allowed us to derive explicit formulae for the mean and variance of thenumber of Rydberg excitations in the jamming limit, that turn out to be func-tions only of the average number of neighbors within the blockade volume. Thecomparison to measurement data of Refs. [31, 32] shows quantitative agree-ment between theory and experiment, and we conclude that the model capturesblockade effects observed in ultracold Rydberg gases.

Interesting future research would be to further explore the approximating re-lation between random graphs and spatial problems, particularly because higher-dimensional continuum random sequential adsorption processes are difficult toanalyze. To this end, the underlying stochastic recursions can be generalized.In fact, the derivation in Ref. [5, Chapter 5] covers a generalization of Eq. (4.4)in terms of the number of neighbors, and related more general stochastic re-cursions have been studied for configuration models [91]. These results canpotentially be used to describe geometrical features such as an inhomogeneous

102

4.A. Exploration algorithm

density and excitation intensity in an atomic cloud, but at increased complexity.Modifying Eq. (4.4) would also enable analysis of off-resonant excitation effectslike the growth of Rydberg aggregates [92, 93]. Such approaches can furtherextend the use of random graphs as an approximation to particle systems thatexhibit complicated interactions.

Supplemental material

4.A Exploration algorithm

The exploration can formally be described as follows. LetXm denote the set of ex-cited particles at time Tm, and letUm be the set of unaffected particles at time Tm.At time T0 = 0, no vertices are excited or blocked, soX0 = ; andU0 = V . At timeTm+1, a uniformly randomly chosen particle vm+1 ∈ Um excites, and starts block-ing random neighbors that were thus far unaffected, say um+1,1, . . . , um+1,b ∈ Um,so that Xm+1 = Xm ∪ vm+1, and Um+1 = Um\(vm+1 ∪ um+1,1, . . . , um+1,b).This stochastic process continues until the moment τ a jamming limit occurs,i.e. when Uτ = ; and the number of unaffected particles equals zero.

4.B Solving the stochastic recursions

The exploration algorithm provides the following stochastic recursions for thenumber of excited particles Xm = |Xm| and the number of unaffected particlesUm = |Um| at the moment of the m-th excitation,

Xm+1 = m+ 1,

Um+1 = Um − 1− Bin(Um − 1, p), (4.13)

with X0 = 0 and U0 = n. These recursions can be leveraged to determine themean and the variance of the number of unaffected particles at each momentm excitations have occurred, i.e. E[Um] and Var[Um]. To see this, let m≥ 1 andstart by noting that

Umd= Bin(Um−1 − 1,1− p). (4.14)

Utilizing the tower property, we find that

E[Um] = E[E[Bin(Um−1 − 1,1− p)|Um]] = (1− p)E[Um−1]− (1− p).(4.15)

103


Iterating and recalling that E[U0] = n, we obtain(4.16)

E[Um] = (1− p)mn−m∑

i=1

(1− p)i = (1− p)mn−(1− p)− (1− p)m+1

p.

A recursion for the variance can be found in a similar fashion by first decompos-ing

(4.17)

Var[Um] =Var[E[Bin(Um−1 − 1,1− p)|Um−1]]

+E[Var[Bin(Um−1 − 1, 1− p)|Um−1]]

=Var[(Um−1 − 1)(1− p)] +E[(Um−1 − 1)p(1− p)]

=(1− p)2Var[Um−1] + p(1− p)(E[Um−1]− 1),

and then recalling that Var[U0] = 0, E[U0] = n, so that after iterating,

(4.18) Var[Um] = p(1− p)2m−1(n− 1) + pm−1∑i=1

(1− p)2i−1(E[Um−i]− 1).

Substituting (4.16) into (4.18) and simplifying, we find that(4.19)

Var[Um]

=(p− 2)(1− p)m((n− 1)p+ 1) + (1− p)2m(1− (n− 1)(p− 2)p)− p+ 1

(p− 2)p.

4.C Determining fluid limits

When n→∞ and p = c/n, there exist fluid limits for E[Um] and Var[Um]. Tosee this, define

(4.20) u( f ) = limn→∞

E[U[ f n]]

n

for f ∈ [0, 1], with [·] denoting rounding to the nearest integer. Utilizing (4.16),we find that

(4.21)u( f ) = lim

n→∞

1−

cn

[ f n]−

1c

1−

cn

1−

1−

1c

[ f n]

= e−c f −1c(1− e−c f ).

104

4.D. Comparison to Petrosyan, 2013

Similarly, after defining

v( f ) = limn→∞

Var[U[ f n]]

n, (4.22)

and substituting (4.19), we find that

v( f ) =e−c f (1− e−c f )((1+ 2c)e−c f − 1)

2c. (4.23)

4.D Comparison to Petrosyan, 2013

Ref. [80] describes usage of semiclassical Monte Carlo simulations to studystationary states of the Rydberg gas in a two-dimensional system. There, particlesare positioned on points of a lattice with spacing a = 532nm, that fall within acircular excitation area with radius R, which is varied relative to the blockaderadius of r ≈ 1.905µm. Ref. [80] finds numerically that Q ≈ −0.84 for R ¦r. This independence on the system size is explained by the model, becausethe results indicate that the Mandel Q parameter only depends on the averagenumber of neighbors c, which in this simulation setup approaches a constantfor sufficiently large R. Modeling the blockade area as a hard circle of radius r,we have by Gauss’s circle problem that c + 1≈ 37 lattice points fall within theblockade area for sufficiently large R. Because the number of particles within theexcitation area N did not fluctuate between simulation instances, we can usethe conditional expressions for the mean and variance to estimate the Mandel Qparameter. This results in Q ≈ c2/(2(1+ c)2 ln (1+ c))− 1|c=36 = −0.87, whichis close to the simulation result.

4.E Incorporating detector efficiency

Assuming the I1, . . . , IX (t) are independent, the average number of detected Ryd-berg atoms is

E[XD(t)] = EX (t)∑

i=1

Ii

= E

EX (t)∑

i=1

Ii

X (t)= ηE[X (t)]. (4.24)

105


The variance of the number of detected Rydberg atoms is

(4.25)

Var[XD(t)] = VarX (t)∑

i=1

Ii

= Var

EX (t)∑

i=1

Ii

X (t)+E

VarX (t)∑

i=1

Ii

X (t)

= η2Var[X (t)] +EX (t)∑

i=1

Var[Ii]= η2Var[X (t)] +η(1−η)E[X (t)].

The detected Mandel Q parameter is therefore

(4.26) QD(t) =Var[XD(t)]E[XD(t)]

− 1= ηVar[X (t)]E[X (t)]

− 1= ηQ(t),

which completes the derivation.

106

CHAPTER 5Scaling limits

for exploration algorithms

by Paola Bermolen, Matthieu Jonckheere, and Jaron SandersarXiv preprint, 1504.02438

107

5. SC A L I N G L I M I T S F O R E X P L O R AT I O N A L G O R I T H M S

Abstract

We consider an exploration algorithm where at each step, a ran-dom number of items become active while related items get ex-plored. Using scaling limits of Markovian processes, we study thestatistical properties of the proportion of active nodes in time whenthe number of items tends to infinity. This rigorously establishes theclaims and heuristics presented in companion paper [4, Chapter 4].

108

5.1. Introduction

5.1 Introduction

Assume there exists a binary relation between items V = 1, . . . , N, to whichwe associate a relation graph where nodes are items such that two items areneighbors if they are related. Let At be the set of active items at time t (step t)and Bt the set of explored items. We assume that, initially, A0 = B0 = ;. Thenwe consider the following exploration process: (i) select It ⊆ V \ At ∪ Bt anddetermine its neighbors in the set of nonexplored items, Nt ⊆ V \ At ∪ Bt ∪ It,and (ii) update At and Bt by setting

At+1 = At ∪ It , and Bt+1 = Bt ∪Nt . (5.1)

Such exploration algorithms can be used to approximate the evolution of parkingprocesses [94] and as we shall show, random sequential adsorption processes.In case It is a single item at each step, the algorithm discovers in a greedymanner independent sets of the relation graph. If It is a subgraph, Nt is the setof nonactive, nonexplored neighbors of this subgraph at step t.

Exploration algorithms can be linked to communication procedures in com-munication networks. In communication network problems, the relation graphmight be the outcome of spatial effects, i.e. nodes interacting through a geome-try which can itself be random. This is certainly the case in wireless networksin which acceptable radio conditions have been defined based on the level ofadmissible interference between two competing nodes, so conditions that de-termine whether simultaneous communication is possible. These interferenceconstraints can be modeled using a hardcore interference graph, in which anedge is present between two nodes if their radio conditions would impede asimultaneous communication.

Relation graphs can also be used to describe blockade effects in complexsystems of interacting particles. Of particular interest to us are specially preparedgases that consist of ultracold atoms that can reach a Rydberg state from a groundstate. The essential feature of these particles is that each atom that is in itsRydberg state, prevents neighboring atoms from reaching their Rydberg state.This is similar in spirit to the interference constraints in wireless networks [3,Chapter 3], and the essential features of the blockade effect can therefore againbe described using interference graphs. This realization also allowed us to studystatistical properties of the proportion of atoms ultimately in the Rydberg statethrough the exploration algorithm defined above [4, Chapter 4]. There, it was

109


shown heuristically that the number of atoms ultimately in the Rydberg state isasymptotically described by a normal distribution with mean N ln (1+ c)/c andvariance Nc/(2(c+1)2). This paper rigorously studies these statistical propertiesresulting from the exploration algorithm.

The dimension needed to represent the exploration algorithm as a Markovprocess is in general equal to the size of the graph N , and this impedes the prac-ticality of performing computations. To overcome these difficulties, we assumethat a strong homogeneity assumption holds for the relation between items. Thisallows us to describe the dynamics using a one-dimensional Markov process,instead of the harder N -dimensional process. While this is a coarse simplificationfor many problems, it has the advantage that the analysis remains tractable. Wethen use classical tools of probability theory, specifically fluid limits and diffusionapproximations, to derive computable characterizations of the performance ofthese exploration algorithms. These techniques furthermore allow us to provefunctional laws of large numbers, as well as a law of large numbers (LLN) andcentral limit theorem (CLT) for the proportion of active items.

This paper is structured as follows. In Section 5.2, we study the case ofrandom sequential adsorption (RSA) under a homogeneity assumption. In Sec-tion 5.3, we provide a functional law of large numbers for the number of exploreditems. Section 5.4 subsequently provides a functional central limit theorem, andprovides error bounds when using the limit theorem as an approximation to afinite-sized system. Finally, Section 5.5 gives both a law of large numbers and acentral limit theorem for the eventual number of active items.

5.2 Random adsorption under a homogeneous relation

Assume from here on that precisely one item is selected in each step. Assumealso the following homogeneity assumption on the graph G induced by thebinary relation that has been assumed to exist between items.

Assumption 5.1. If (G1,G2) is a partition of G , then the distribution of the num-ber of neighbors of any i ∈ G2 depends only on the size of the partition: |G1|and |G2|.

While Assumption (5.1) is not valid in cases such as random geometricgraphs and random graphs with generic degree distribution, it is crucial to obtaina one-dimensional analysis. Assumption (5.1) can however be considered a

110

5.3. Functional law of large numbers

reasonable approximation for many systems, and it is for example satisfied byErdös–Rényi (ER) random graphs. We refer the reader to [94] for a study ofscaling limits in infinite dimension that applies to a larger class of problems.

If we let Zn denote the number of explored items at step n, so Zn = |An∪Bn|,we have that

Zn = Zn−1 + 1+ ξn, and Z0 = 0. (5.2)

Here, ξn denotes the number of neighbors that the selected item has at step n inthe remaining nonexplored portion of the graph. We note that Ref. [95] derivesfluid limits for a similar stochastic recursion that models wireless networks withstar configurations running an ALOHA-type random multiple-access protocol.

The distribution of ξn depends under Assumption (5.1) only on Zn−1, whichwe will denote by ξZn−1

with a slight abuse of notation. Assumption (5.1) alsoimplies that the process Znn∈N0

is a discrete Markov process that takes values in0, . . . , N, is strictly increasing, and has in N an absorbing state. The transitionprobabilities at step n of the process Znn∈N0

are therefore given by

px y(n) = P[Zn = y|Zn−1 = x] = P[ξx = y − x − 1] for y > x . (5.3)

If we now denote by pN (·, x) the distribution of the number of neighbors inG2 of any vertex i ∈ G2 given that (G1,G2) is a partition of G with |G1|= x , thetransition probabilities can be written as

px ,x+k+1(n) = P[ξx = k] = pN (k, x), with k ≥ 0. (5.4)

In case of the ER random graph, in which an edge exists between a pair ofnodes with probability p, the transition probabilities are given by the Binomialdistribution, i.e.

pN (k, x) =

N − x − 1k

pk(1− p)N−x−k−1. (5.5)

5.3 Functional law of large numbers

Given a partition (G1,G2) of G such that |G1|= x , consider the mean γN (x) andvariance ψN (x) of the number of neighbors in G2 of a given vertex i ∈ G2, i.e.

γN (x) =N−1∑k=0

kpN (k, x), ψN (x) =N−1∑k=0

(k− γN (x))2pN (k, x), (5.6)

111


and define γN = supx γN (x), ψN = supx ψN (x).Now consider the scaled process defined as the piece-wise constant trajectory

process of

(5.7) ZNt =

Z[tN]

Nfor all t ≥ 0. Here, [x] is the integer part of x . We now derive the law of largenumbers for ZN

t in Proposition 5.2. While the proof of convergence relies onclassical techniques [96], we leverage these tools to obtain error bounds alongthe way.

Proposition 5.2. If there exists a (CL)-Lipschitz continuous function γ [16] onR+ such that

(5.8) supx

γN (x)− γ x

N

≤ δN ,

then for p > 1 and T > 0,

(5.9) ‖ sups∈[0,T]

|ZNs − z(s)|‖p ≤

δN T +

1+ γN

N+κp‖M N

T ‖p

eCL T ,

where κp = p/(p− 1), and z(t) denotes the solution to the deterministic differ-ential equation

(5.10) z = 1+ γ(z), with z(0) = 0.

For p = 2, the bound reduces to ‖supt∈[0,T] |ZN (t)− z(t)|‖2 ≤ ωN , with ωN =

(δN T + (1+ γN )/N + 2ÆψN T/N)exp (CL T ).

Proof. Doob’s martingale decomposition [66] for the Markov process Znn∈N0

gives that for n≥ 0,

(5.11) Zn =n∑

i=0

(1+ γN (Zi)) +Mn.

Here, we have used that Z0 = 0, and Mn denotes a local martingale that isactually a global martingale since the state space is finite.

We will now examine the scaled random variable ZNt , for which

(5.12)

ZNt =

Z[tN]

N=

1N

[tN]∑i=0

1+ γN (Zi)

+

M[tN]

N

(i)=

1N

∫ [tN]

0

1+ γN (Zs)

ds+

M[tN]

N(ii)=

∫ [tN]N

0

1+ γN (ZuN )

du+M N

t ,

112

5.3. Functional law of large numbers

since we (i) view each trajectory as being path-wise continuous, and (ii) use thechange of variables u = s/N , and introduce the notation M N

t = M[tN]/N for ascaled martingale.

We can replace the integral∫ [tN]/N

0 · · ·du by the integral∫ t

0 · · ·du, whichintroduces an error ∆N ,t . Specifically, we can write

∫ [tN]N

0

1+ γN (ZuN )

du=

∫ t

0

1+ γN (ZuN )

du+∆N ,t (5.13)

where

∆N ,t =

∫ [tN]N

0

1+ γN (ZuN )

du−

∫ t

0

1+ γN (ZuN )

du. (5.14)

For large N such replacement has negligible impact, since independently of t,

|∆N ,t | ≤ supu∈[0,1]

1+ γN (ZuN ) [tN]

N− t≤ 1+ γN

N, (5.15)

where in the last inequality we have used that γN = supx γN (x) and |[tN] −tN | ≤ 1.

Using (i) the integral version of (5.10), the triangle inequality [57], and (ii)Lipschitz continuity of γ, as well as bound (5.15), we find that

sups∈[0,t]

|ZNs − z(s)|

(i)≤ sup

s∈[0,t]

∫ s

0

γN (ZuN )− γ(z(u))du+ |∆N ,s|+ |M N

s |

(ii)≤ CL

∫ t

0

supu∈[0,s]

|ZNu − z(u)|ds+δN t +

1+ γN

N+ sup

s∈[0,t]|M N

s |.

(5.16)Next, we define εN (T ) = sups∈[0,T] |ZN

s −z(s)| for notational convenience and toprepare for an application of Grönwall’s lemma [66]. Eq. (5.16) then shortensfor T > 0 to

εN (T )≤ δN T +1+ γN

N+ sup

s∈[0,T]|M N

s |+ CL

∫ T

0

εN (s)ds. (5.17)

Because δN T + (1+ γN )/N + sups∈[0,T] |M Ns | is nondecreasing in T , it follows

from Grönwall’s lemma that

εN (T )≤δN T +

1+ γN

N+ sup

s∈[0,T]|M N

s |eCL T . (5.18)

113


Using Minkowsky’s inequality for p ∈ [1,∞), strict monotonicity of exp (CL T )and δN T , and the triangle inequality, we find that

(5.19) ‖εN (T )‖p ≤δN T +

1+ γN

N+ ‖ sup

s∈[0,T]|M N

s |‖p

eCL T .

Finally, using Doob’s martingale inequality [66] for p > 1, we obtain

(5.20) ‖εN (T )‖p ≤δN T +

1+ γN

N+ κp‖M N

T ‖p

eCL T ,

completing the first part of the proof.For p = 2, this inequality can be further simplified by computing the increas-

ing process associated to the martingale. Note specifically that for l ≥ 0 we have

(5.21) E[(Ml)2] = E[⟨Ml⟩] = E

l∑i=0

Var[γN (Zi)]

where

(5.22)Var[γN (x)] =

N−x−1∑k=0

(k+ 1)2px ,x+k+1 −N−x−1∑

k=0

(k+ 1)px ,x+k+1

2

=ψN (x).

Therefore for the scaled martingale M Nt , we find by combining (5.21) and (5.22)

that for t > 0

(5.23) ‖M Nt ‖

22 = E[(M

Nt )

2] =E[M2

[tN]]

N2=

1N2

[tN]∑i=0

ψN (Zi)≤ψN t

N.

This completes the second part of the proof.

Corollary 5.3. If the distribution of the number of neighbors is such that δN → 0as N →∞, γN = o(N), and ψN = o(N), then the scaled process ZN

t convergesto z(t) in L1 uniformly on compact time intervals.

Corollary 5.4. If the number of initial items N is itself random and independentof the trajectory of Z , meaning that Z can be constructed (i) as a functional ofZ0 = N and (ii) of other random variables that are independent of N , then

(5.24) E

supt∈[0,T]

|ZN (t)− z(t)|≤ E[δN T] +E

1+ γN

N

+ 2eCL TE

√√ψN TN

.

114

5.4. Diffusion approximations with errors bounds

Example 5.5 (Sparse ER random graph with a Poisson number of vertices).Suppose that given N , the graph G = G (N , c/N) is a sparse Erdös–Rényi graph,i.e. pN (·, x) is the probability mass function of the binomial distribution Bin(N−x − 1, c/N) with c > 0. Additionally, suppose that N − 1 is Poisson distributedwith parameter h. The mean and variance of pN (., x) are then given by

γN (x) = (N − x − 1)cN

, ψN (x) = (N − x − 1)cN

1−

cN

. (5.25)

Let γ(x) = c(1− x). Condition (5.8) is then satisfied with δN = c/N , andas Lipschitz constant CL = c suffices. Moreover, γN , ψN ≤ c. The deterministicdifferential equation in (5.10) reads

z = 1+ c(1− z) = (1+ c)− cz, (5.26)

which can be explicitly solved, resulting in

z(t) =1+ c

c+z(0)−

1+ cc

e−t =

1+ cc

1−

1+ cc

e−t . (5.27)

Observe that limt→∞

z(t) = (1+c)/c > 1, implying the existence of a finite, constant

solution T ∗ such that z(T ∗) = 1, which we will leverage in Section 5.5.Furthermore, from Corollary 5.4 we obtain using Cauchy–Schwarz’s inequal-

ity [57] that there exists a constant C1 such that

E

supt∈[0,T]

|ZN (t)− z(t)|≤ (1+ 2c)E

1N

+ 2p

cTecTE 1p

N

≤ C1E 1

N

1/2= C1

1− exp(−h)h

1/2.

(5.28)

Note that as anticipated, E[supt∈[0,T] |ZN (t)− z(t)|]→ 0 as h→∞.

5.4 Diffusion approximations with errors bounds

We now proceed and derive a functional central limit theorem for the scalednumber of explored items ZN

t . The convergence proof relies similarly on classicaltechniques, which we again leverage to determine error bounds. To that end,we apply results of [97] which are based on results by Komlós–Major–Tusnády[98, 99]. The results in [97] allow one to construct a Brownian motion andeither a Poisson process or random walk on the same probability space. Since

115


we are concerned with discrete time, we need to consider the random walk case,see also [100]. In order to obtain explicit error bounds, we impose strongerassumptions on the transitions probabilities than would be needed when onlyproving convergence.

Proposition 5.6. If there exists a function p on (N,R+), and a sequence (εk)k=0,1,...

such that

(5.29)

|pN (k, [N x])− p(k, x)| ≤εk

N,

|p(x , x + k)− p(y, y + k)| ≤ Mεk|x − y|,∑k

k2|p(x , x + k)1/2 − p(y, y + k)1/2|2 ≤ M |x − y|2

and

(5.30)∑

k

kε1/2k <∞,

and if γ is twice differentiable with bounded first and second derivatives, thenthe process

(5.31) W Nt =p

N(ZNt − z(t))

converges in distribution towards Wt , the unique solution of the stochastic dif-ferential equation

dW (t) = γ′(z(t))W (t)dt +Æβ ′(t)dB1(t).(5.32)

Here, B1(t) denotes a standard Brownian motion, β(t) =∫ t

0 ψ(z(s))ds, and z(t)is the solution of (5.10). Furthermore,

(5.33) E[supt≤T|W N

t −Wt |]≤ Clog(N)p

N.

Proof. We adapt the results of Kurtz which were derived for continuous timeMarkov jump processes. For doing so, we can replace the Poisson processesinvolved in the construction of the jump processes by some random walks thatcan be used to construct discrete time Markov chains. We can then use exactlythe same steps as in [97], by first comparing the original process ZN to a diffusionof the form

(5.34) ZNt =

1N

∑l≤N

lBl(Nt∑0

pN (l, ZNs )ds),

116

5.5. LLN and CLT for the hitting time

that is a sum of a finite number of scaled independent Brownian motions Bl .Rewriting the inequalities in [97, (3.6)], and using a random walk version

of the approximation lemma of Komlós–Major–Tusnády [100], we obtain

Esupt≤T|ZN

t − ZNt |≤ C2

log(N)N

. (5.35)

This leads using the results of [97, Section 3] to

Esupt≤T|W N

t −Wt |≤ C3

log(N)p

N, (5.36)

which concludes the proof.

Example 5.7 (ER case – Continued). The relation between the binomial coeffi-cients and the Poisson distribution are well studied. Defining

pN (k, [xN]) =

N − [N x]− 1k

cN

k1−

cN

N−[N x]−1, (5.37)

and using (for instance) the Stein–Chen method [101], we have that

|pN (k, [xN])− p(k, x)| ≤cN

p(k, x), (5.38)

which shows that the assumptions of Proposition 5.6 are satisfied.

5.5 LLN and CLT for the hitting time

The exploration algorithm finishes at time

T ∗N = infτ ∈ N+|Zτ = N ≤ N <∞, (5.39)

which is a hitting time for the Markov process. Since the algorithm adds preciselyone node at each step, we have that the final number of active items is exactlyT ∗N , i.e. AT ∗N

= T ∗N . Because we wish to determine the statistical properties ofAT ∗N

, we will seek not only a first-order approximation for T ∗N , but also prove acentral limit theorem result as the initial number of items N goes to infinity.

Since the exploration process converges to the fluid limit z(t), we can antic-ipate that an appropriately scaled hitting time T ∗N converges to T ∗, the solutionto z(T ∗) = 1. This intuition is formalized in the LLN result for T ∗N/N in Proposi-tion 5.8.

117


Proposition 5.8. For all δ > 0,

(5.40) P T

∗N

N− T ∗

≥ δ≤

2ωN

δ.

Moreover if γ(x) is continuous, nonincreasing with γ(1) = 0, then there existsa constant C for sufficiently small δ so that

(5.41) T ∗N

N− T ∗

2≤ CωN := ΩN

Proof. Remark that if |z(s)− ZNs | ≤ δ/2 for all s > 0, that then

(5.42) T∗N

N−T ∗

≤(z− 1

2δ)−1(1)−(z+ 1

2δ)−1(1)

≤ |(T ∗+ 12δ)−(T

∗− 12δ)|= δ.

Here, the last inequality follows from the fact that z(s) = 1+ γ(z(s))≥ 1, since

(5.43)(z − 1

2δ)(T∗ + 1

2δ) = (z −12δ)(z

−1(1) + 12δ)

≥ z(z−1(1)) + 12δ−

12δ = 1= (z − 1

2δ)((z −12δ)

−1(1)).

Thus the first claim follows directly from the observation that the event

(5.44)¦ T

∗N

N− T ∗

≥ δ©⊆|z(s)− ZN

s | ≥12δ,

and then using (i) Markov’s inequality [66], and (ii) invoking Proposition 5.2,so that

(5.45) P T

∗N

N−T ∗

≥ δ (5.44)≤ P

|z(s)−ZN

s | ≥12δ (i)≤

2δE[|z(s)−ZN

s |](ii)≤

2ωN

δ.

Now (i) using that ZT ∗N/N = z(T ∗) = 1 together with (5.10) and (5.12), and

(ii) after expanding the integrals, we find that

(5.46)

T ∗NN− T ∗

(i)=

∫ T ∗

0

γ(z(s))ds−∫ T∗N

N

0

γN (ZsN )ds−MT ∗N

N

(ii)=

∫ T∗NN ∧T ∗

0

(γ(z(s))− γN (ZsN ))ds−MT ∗N

N

+

∫ T ∗

T∗NN ∧T ∗


N

T∗NN ∧T ∗

γN (ZsN )ds.

118


Then taking the absolute value and using the triangle inequality, it follows that

T∗N

N− T ∗

≤∫ T∗N

N ∧T ∗

0

|γ(z(s))− γN (ZsN )|ds+ |M NT ∗N /N|

+

∫ T ∗

T∗NN ∧T ∗

|γ(z(s))|ds+

∫ T∗NN

T∗NN ∧T ∗

|γN (ZsN )|ds.

(5.47)

Approximating γN by γ via (5.8), using Lipschitz continuity of γ, and recallingthat max(T ∗N/N , T ∗)≤ 1, we find that

T∗N

N− T ∗

≤ 2CL sups≤1|z(s)− ZN

s |+ 2δN + |M NT ∗N /N|+∫ T∗N

N ∨T ∗

T∗NN ∧T ∗

|γ(z(s))|ds. (5.48)

The continuity of γ(x) guarantees that there exist constants C1,ε > 0 suchthat (i) γ(z(s)) ≤ 1− ε for all s ≥ C1, and (ii) C1 < T ∗ − δ, provided that δ issufficiently small. There are now two possible cases: either (a) C1 ≤ T ∗N/N ∧ T ∗,or (b) T ∗N/N ∧ T ∗ < C1 < T ∗N/N ∨ T ∗. For convenience, we first split the integralaccording to

∫ T∗NN ∨T ∗

T∗NN ∧T ∗

|γ(z(s))|ds =

∫ T∗NN ∨T ∗

T∗NN ∧T ∗

|γ(z(s))|(1[s < C1] +1[s ≥ C1])ds. (5.49)

Then splitting further into case (a), we have that

∫ T∗NN ∨T ∗

T∗NN ∧T ∗

|γ(z(s))|1s < C1, C1 ≤

T ∗NN∧ T ∗

ds = 0, (5.50)

and∫ T∗N

N ∨T ∗

T∗NN ∧T ∗

|γ(z(s))|1s ≥ C1, C1 ≤

T ∗NN∧ T ∗

ds

≤ (1− ε) T∗N

N− T ∗

1C1 ≤

T ∗NN∧ T ∗

.

(5.51)

Next let C2 be a constant such that C2 ≥∫ C1

T ∗N /N∧T ∗ |γ(z(s))|ds. We can then, after

119


splitting further into case (b), bound

(5.52)

∫ T∗NN ∨T ∗

T∗NN ∧T ∗

|γ(z(s))|1s < C1,

T ∗NN∧ T ∗ < C1 <

T ∗NN∨ T ∗

ds

=

∫ C1

T∗NN ∧T ∗

|γ(z(s))|ds1 T ∗N

N∧ T ∗ < C1 <

T ∗NN∨ T ∗

≤ C21 T ∗N

N∧ T ∗ < C1 <

T ∗NN∨ T ∗

≤ C21

T ∗NN< C1

,

since if 1[T ∗N/N ∧ T ∗ < C1 < T ∗N/N ∨ T ∗] = 1, clearly T ∗N/N ∧ T ∗ < C1. Butby construction C1 < T ∗, so it must hold that T ∗N/N < C1 and thus 1[T ∗N/N <C1] = 1. Next, we bound

(5.53)

∫ T∗NN ∨T ∗

T∗NN ∧T ∗

|γ(z(s))|1s ≥ C1,

T ∗NN∧ T ∗ < C1 <

T ∗NN∨ T ∗

ds

≤ (1− ε) T∗N

N− T ∗

1 T ∗N

N∧ T ∗ < C1 <

T ∗NN∨ T ∗

.

Summarizing, there thus exists a constant C2 such that

(5.54)

T∗N

N− T ∗

≤ 2CL sups≤1|z(s)− ZN

s |+ 2δN + |M NT ∗N /N|

+ (1− ε) T∗N

N− T ∗

+ C21 T ∗N

N< C1

.

Now recall that if |z(s)− ZNs | ≤ δ/2, then |T ∗N/N − T ∗| ≤ δ. Moreover then

also T ∗N/N ≥ C1 since C1 < T ∗ −δ. Hence,

(5.55)¦ T ∗N

N< C1

©⊂|z(s)− ZN

s |>12δ.

Then by (i) collecting terms in and subsequently using (5.54), and then (ii)applying Minkowski’s inequality [57], we obtain(5.56)

ε T ∗N

N− T ∗

2

(i)≤ 2CL sup

s≤1|z(s)− ZN

s |+ 2δN + |M NT ∗N /N|+ C21

T ∗NN< C1

2

(ii)≤ 2CL‖sup

s≤1|z(s)− ZN

s |‖2 + 2δN + ‖M NT ∗N /N‖2 + C2

1 T ∗N

N< C1

2.

120


We now note that (iii) since f (y) = y2 is monotonically increasing for y ≥ 0and (iv) by Markov’s inequality, 1 T ∗N

N< C1

2= P

T ∗NN< C1

12 (5.55)≤ P[|z(s)− ZN

s |>12δ]

12

(iii)= P[|z(s)− ZN

s |2 > 1

4δ2]

12

(iv)≤

2δE[|z(s)− ZN

s |2]

12 =

2δ‖z(s)− ZN

s ‖2.(5.57)

Therefore,

ε T ∗N

N− T ∗

2≤2CL +

2C2

δ

‖sup

s≤1|z(s)− ZN

s |‖2 + 2δN + ‖M NT ∗N /N‖2. (5.58)

Thus by finally using Proposition 5.2 and (5.23), we have that there exist con-stants C3, C4 so that

ε T ∗N

N− T ∗

2≤ C3ωN + 2δN +

√√ψN

N≤ C4ωN , (5.59)

which concludes the proof.

Corollary 5.9. If the distribution of the number of neighbors is such that δN → 0as N →∞, γN = o(N), and ψN = o(N), then the proportion of active itemsT ∗N/N converges in L1 to T ∗.

The LLN in Proposition 5.8 provides us formally with a candidate, T ∗, aroundwhich to center T ∗N/N and subsequently prove the CLT result in Proposition 5.10.

Proposition 5.10. There exist constants C1, C2 such that the expectation of therandom variable

pN(T ∗N/N − T ∗) centered around −W ∗

T is bounded by

EpN

T ∗NN− T ∗

+WT ∗

≤ C1ω2N

pN +

ψNΩN +

ψN

N

12 + C2

log(N)p

N+

1+ γNpN

.(5.60)

Moreover, if the distribution of the number of neighbors is such that δN =o(1/p

N), γN = o(p

N), and ψN = o(N1/4), thenp

N(T ∗N/N − T ∗) converges inL1 to WT ∗ that is a centered Gaussian random variable with variance

σ2 = m(T ∗) = E[W 2T ∗], (5.61)

and where m(t) = E[W 2t ] solves the differential system

m= −2γ(z(t))m(t) + β , with m(0) = 0. (5.62)

121


Proof. First, recall that by (5.10) and (5.12), see (5.46),

(5.63)T ∗NN− T ∗ =

∫ T ∗

0


N

0

γN (ZsN )ds−M NT ∗N /N

,

Note furthermore that(5.64)

W NT ∗

(5.31)=p

NZN

T ∗ − z(T ∗) (5.12)=p

N∫ [T∗N]

N

0

(1+ γN (ZsN ))ds+M NT ∗ − z(T ∗)

=p

N∫ T ∗

0

(1+ γN (ZsN ))ds+M NT ∗ − z(T ∗)

+p

N∆N ,T ∗

where M NT ∗ = M[T ∗N]/N . Recall that the error ∆N ,T ∗ introduced by replacing the

upper integration boundary, can readily be bounded by |∆N ,T ∗ | ≤ (1+ γN )/N ,see (5.15).

Comparing (5.63) and (5.64), a subsequent natural series of steps wouldbe to (i) add comparison terms ±W N

T ∗ and use the triangle inequality, and then(ii) substitute (5.64), use the triangle inequality, and upper bound |∆N ,T ∗ | ≤(1+ γN )/N , after which we arrive at(5.65)pN

T ∗NN− T ∗

+WT ∗

(i)≤pN

∫ T ∗

0


N

0


+W N

T ∗

+ |WT ∗ −W NT ∗ |

(ii)≤pN

∫ T ∗

0


N

0


+p

N∫ T ∗

0

(1+ γN (ZsN ))ds+M NT ∗ − z(T ∗)

+ |WT ∗ −W NT ∗ |+

1+ γNpN

= term I + term II +1+ γNp

N.

We will now proceed and bound term I and II.Note that the expectation of term II can be directly bounded by Proposi-

tion 5.6, i.e. there exists a constant C2 such that

(5.66) E[term II] = E[|WT ∗ −W NT ∗ |]≤ E[sup

t≤1|Wt −W N

t |]≤ C2log(N)p

N.

122


Bounding term I requires more work. Using (5.64), the integral version ofz = 1+ γ(z), and the triangle inequality, we find that

term I≤p

N∫ T ∗

0

γN (ZsN )ds−∫ T∗N

N

0

γN (ZsN )ds+pN |M N

T ∗ −M NT ∗N /N|

= term Ia+ term Ib,

(5.67)

and will now proceed with bounding term Ia and Ib separately.

In order to bound term Ia, we add comparison terms±γ(ZsN/N) and±γ(z(s))and use the triangle inequality, so that1

term Ia≤p

N∫ T ∗

T∗NN

γN (ZsN )− γ ZsN

N

ds+pN

∫ T ∗

T∗NN

γ ZsN

N

− γ(z(s))ds

+p

N∫ T ∗

T∗NN

γ(z(s))ds

(5.68)Then by approximating γN by γ, using the Lipschitz continuity of γ, and upperbounding the first two integrands, we find that

term Ia≤p

NδN

T∗N

N− T ∗

+pNCL sups≤1

ZsN

N− z(s)

T∗N

N− T ∗

+p

N∫ T ∗

T∗NN

γ(z(s))ds

(5.69)

Taking the expectation and using the triangle inequality, it follows that

E[term Ia]≤p

NδNE T

∗N

N− T ∗

+p

NCLEsups≤1

ZsN

N− z(s)

T∗N

N− T ∗

+p

NE∫ T ∗

T∗NN

|γ(z(s))|ds.

(5.70)

1Note that we use the notation that∫ b

a = −∫ a

b when a > b.

123


Applying Hölder’s inequality [57],(5.71)

E[term Ia]≤p

NδN

T ∗NN− T ∗

2+p

NCL

sups≤1

ZsN

N− z(s)

2

T ∗NN− T ∗

2

+p

NE∫ T ∗

T∗NN

|γ(z(s))|ds,

and finally Propositions 5.2 and 5.8, we end up with

(5.72) E[term Ia]≤ ΩN

pN(δN + CLωN ) +

pNE

∫ T ∗

T∗NN

|γ(z(s))|ds.

In order to deal with the last term in (5.72), we will use a Taylor expansionof order zero around T ∗. Specifically, we write

(5.73) γ(z(s)) = γ(z(T ∗)) + c(s− T ∗) + R2 = c(s− T ∗) + R2,

where we have recalled that γ(1) = 0 by assumption and z(T ∗) = 1. Then, by (i)the triangle inequality, (ii) upper bounding the integrand, and (iii) evaluatingthe integral, there exists a constant C3 so that

(5.74)

pN

∫ T ∗

T∗NN

|γ(z(s))|ds(i)≤p

N

∫ T ∗

T∗NN

c|s− T ∗|+ |R2|ds

(ii)≤p

N

∫ T ∗

T∗NN

c T∗N

N− T ∗

+ |R2|ds(iii)≤ C3

pN T∗N

N− T ∗

2,

where for the second term we have used that |R2| = O((s − T ∗)2), and that|s− T ∗| ≤ 1 for s ∈ [T ∗N/N , T ∗]. Therefore, by Proposition 5.8,

(5.75)

pNE

∫ T ∗

T∗NN

|γ(z(s))|ds≤ C3

pNE

T∗N

N− T ∗

2

≤ C3

pN T ∗N

N− T ∗

2

2≤ C3Ω

2N

pN .

Ultimately bounding (5.72) using (5.75), we conclude that there exists a con-stant C1 such that

(5.76) E[term Ia]≤ ΩN

pN(δN + CLωN + C3ΩN )≤ C1ω

2N

pN .

124


To finish the proof we still need to bound the expectation of term Ib, thatis,p

NE[|M NT ∗ −M N

T ∗N /N|]. By (i) Cauchy–Schwarz’s inequality, (ii) definition of

the scaled martingale, (iii) calculating the increasing process similar to (5.21)–(5.23), and (iv) E[(Mt −Ms)2] = E[M2

t ]−E[M2s ] for t > s as a consequence of

Mt being a martingale, we find

pNE[|M N

T ∗N /N−M N

T ∗ |](i)≤p

NE[|M NT ∗N /N−M N

T ∗ |2]

12

(ii)=

1p

NE[|MT ∗N

−M[T ∗N]|2]12

(iii)=

1p

NE[< MT ∗N

−M[T ∗N] >]12

(iv)= E

1N

T ∗N∨[T∗N]∑

i=T ∗N∧[T ∗N]

ψN (Zi) 1

2.

(5.77)Then (v) upper bounding ψN (Zi)≤ ψN , (vi) adding compensation terms ±T ∗,applying the triangle inequality and upper bounding |T ∗N − [T ∗N]| ≤ 1, itfollows (vii) from Proposition 5.8 that

E[term Ib] =p

NE[|M NT ∗N /N−M N

T ∗ |](v)≤ E

ψN

T∗N

N−[T ∗N]

N

1

2

(vi)≤ψN

T ∗NN− T ∗

1+ψN

N

12 (vii)≤ψNΩN +

ψN

N

12.

(5.78)

Finally, we combine all bounds, resulting in

EpN

T ∗NN− T ∗

+WT ∗

≤ E[term Ia] +E[term Ib] +E[term II] +

1+ γNpN

≤ C1ω2N

pN +

ψNΩN +

ψN

N

12 + C2

log(N)p

N+

1+ γNpN

.

(5.79)If the distribution of the number of neighbors is such that δN = o(1/

pN),

γN = o(p

N) and ψN = o(N1/4), then ωN = o(1/N3/8) and ΩN = o(1/N3/8),and all the product-terms in (5.79) converge to 0 as N →∞. We have thusproven that under these conditions, the limit is a Gaussian random variable withvariance

σ2 = E[W 2T ∗]. (5.80)

Defining m(t) = E[W 2t ], and using Itô’s formula [66], note that

E[W 2t ] = E

∫ t

0

2WsdWs +12

2βt

= 2

∫ t

0

γ′(z(s))E[W 2s ]ds+ β(t), (5.81)

125


and hence m(t) satisfies the differential system

(5.82) m= −2γ(z(t))m(t) + β , with m0 = 0.

This finishes the proof.

Example 5.11 (ER case – Continued). For the ER graph γ(x) =ψ(x) = c(1− x)and z(t) = ((1+c)/c)(1−e−c t). Recall that δN = c/N , and γN , ψN ≤ c, thereforeall necessary conditions in Proposition 5.10 are met. Solving z(T ∗) = 1 givesT ∗ = ln (1+ c)/c. The differential equation for β(t) is given by

(5.83) β(t) =ψ(z(s)) = (1+ c)e−c t − 1,

and the solution to (5.62) is then

(5.84) m(t) = exp(−2c t)(1− exp(c t))(exp(c t)− 2c − 1)12c

,

ultimately leading to

(5.85) σ2 =m(T ∗)

(1− γ(1))2=

c2(c + 1)2

.

5.6 Conclusions

We have proven a functional law of large numbers and functional central limittheorem for the scaled number of explored items Zn/n= |An∪BN |/n on randomgraphs that satisfy a homogeneity assumption. The proof methodology reliedon the homogeneity assumption in that it allowed us to describe the explorationprocess using a one-dimensional process instead. The fluid limit and diffusionlimits of this one-dimensional process were then leveraged to prove a law oflarge numbers and central limit theorem for the eventual proportion of activeitems, limN→∞ |AT ∗N

|/N .

Notably, we have shown how our results apply to ER random graphs through-out the paper. This rigorously establishes the claims in [4, Chapter 4], where itwas shown heuristically that the number of atoms ultimately in the Rydberg stateis asymptotically described by a normal distribution with mean N ln (1+ c)/cand variance Nc/(2(c + 1)2).

126

5.A. Continuous-time result

Appendix

5.A Continuous-time result

Our arguments that have led to our results for discrete time can be used in asimilar fashion to obtain results for continuous time. We state the convergenceresults for fixed time intervals without proof.

Proposition 5.12. Suppose that there exist a function on R+, γ such that

γN (x) = γ x

N

+δN with δN →

N→∞0. (5.86)

and suppose that the function g(x) = (1− x)γ(x) is a (CL)-Lipschitz functionsuch that

gN (x) =1−

xN

γN (x) = g

xN

+αN with αN →

N→∞0. (5.87)

Then

E

supt∈[0,T]

|ZN (t)− z(t)|≤ (α′N + 2

ÆλCN T )exp (λCL T ), (5.88)

where α′N goes to zero with N , CN =ψNN +

(1+γN )2

N and z(t) is the solution of thefollowing (deterministic) differential equation

z = λ(1− z)(1+ γ(z)), with z(0) = 0. (5.89)

Suppose that there exists a function ψ on R+ such that

limN→∞

ψN (x)−ψ x

N

= 0. (5.90)

The process W Nt =p

N(ZNt − z(t)) converges in distribution towards W which

is the solution of the stochastic differential equation

dW (t) = λ−(1+γ(z(t)))+γ′(z(t))(1−z(t))

W (t)dt+

Æβ ′(t)dB(t), (5.91)

where B(t) is a standard Brownian motion, β(t) = λ∫ t

0 (1− z(s))(ψ(z(s))+(1+γ(z(s))2)ds, and z(t) is the solution of (5.89).

127

Part II

Large-scale systems

129

CHAPTER 6Scaled control

in the QED regime

by Guido Janssen, Johan van Leeuwaarden, and Jaron SandersPerformance Evaluation 70 (10), 750–769

131

6. SC A L E D C O N T R O L I N T H E QED R E G I M E

Abstract

We develop the many-server asymptotics in the Quality-and-Efficiency-Driven (QED) regime for models with admission control.The admission control, designed to reduce the incoming traffic inperiods of congestion, scales with the size of the system. For a classof Markovian models with this scaled control, we identify the QEDlimits for two stationary performance measures. We also derive cor-rected QED approximations, generalizing earlier results for the Er-lang B, C and A models. These results are useful for the dimensioningof large systems equipped with an active control policy. In partic-ular, the corrected approximations can be leveraged to establishthe optimality gaps related to square-root staffing and asymptoticdimensioning with admission control.

132

6.1. Introduction

6.1 Introduction

Many-server systems have the capability of combining large capacity with highutilization while maintaining satisfactory system performance. This potentialfor achieving economies of scale is perhaps most pronounced in the Quality-and-Efficiency-Driven (QED) regime, or Halfin-Whitt regime. Halfin and Whitt [52]were the first to study the QED regime for the GI/M/s system. Assuming thatcustomers require an exponential service time with mean 1, the QED regimerefers to the situation that the arrival rate of customers λ and the numbers ofservers s are increased in such a way that the traffic intensityρ = λ/s approachesone and

(1−ρ)p

s→ γ, γ ∈ R. (6.1)

The scaling (6.1) is effective because the probability of delay converges toa non-degenerate limit away from both zero and one. Limit theorems for other,more general systems are obtained in [102–106], and in all these cases, thelimiting probability of delay remains in the interval (0, 1). In fact, not only theprobability of delay, but many other performance characteristics or objectivefunctions are shown to behave (near) optimally in the QED regime, see forexample [107, 108]. An important reason for this near optimal behavior are therelatively small fluctuations of the queue-length process.

This can be understood in the following way. Let Xs(t) = (Qs(t) − s)/p

sdenote a sequence of normalized processes, with Qs(t) the process describingthe number of customers in the system over time. When Xs(t) > 0, it is equalto the scaled total number of customers in the queue, whereas when Xs(t)< 0,it is equal to the scaled number of idle servers. Halfin and Whitt showed for theGI/M/s system how under (6.1), Xs(t) converges to a diffusion process X (t)on R, that behaves like a Brownian motion with drift above zero and like anOrnstein–Uhlenbeck (OU) process below zero, and that has a non-degeneratestationary distribution. This shows that the natural scale of Qs(t)− s is of theorder

ps. More precisely, the queue length is of the order

ps, as well as the

number of idle servers.This paper adds to the QED regime the feature of state-dependent control,

by considering a control policy that lets an arriving customer enter the systemaccording to some probability depending on the queue length. In particular,a customer meeting upon arrival k other waiting customers is admitted withprobability ps(k), and we allow for a wide range of such control policies char-

133


acterized by ps(k)k∈N0with N0 = 0,1, . . . . An important property of this

control is that it is allowed to scale with the system size s. Consider for examplefinite-buffer control, in which new customers are rejected when the queue lengthequals N , so that ps(k) = 1 for k < N and ps(k) = 0 for k ≥ N . A finite-capacityeffect in the QED regime that is neither dominant nor negligible occurs whenN ≈ η

ps with η > 0, because the natural scale of the queue length is

ps. A

similar threshold in the context of many-server systems in the QED regime hasbeen considered in [109–112].

We introduce a class of QED-specific control policies ps(k)k∈N0designed,

like the finite-buffer control, to control the fluctuations of Qs(t) around s. To thisend, we consider control policies for which ps(x

ps)≈ 1− a(x)/

ps when x > 0.

Here, a denotes a nonnegative and nondecreasing function. While almost allcustomers are admitted as s→∞, this control is specifically designed for havinga decisive influence on the system performance in the QED regime. We alsoprovide an in-depth discussion of two canonical examples. The first is modified-drift control given by a(x) = ϑ > −γ, which is shown to effectively change theQED parameter γ in (6.1) into γ+ ϑ. The second example is Erlang A controlgiven by a(x) = ϑx , for which the system behavior is shown to be intimatelyrelated with that of the Erlang A model in which waiting customers abandonthe system after an exponential time with mean 1/ϑ.

Our class of QED-specific control policies stretches much beyond these twoexamples. In principle, we can choose the control such that, under Markovianassumptions, the stochastic-process limit for the normalized queue-length pro-cess changes the Brownian motion in the upper half plane (corresponding to thesystem without control), into a diffusion process with drift −γ− a(x) in statex ≥ 0. We give a formal proof of this process-level result.

We next consider the controlled QED system in the stationary regime andderive the QED limits for the probability of delay and the probability of rejection.Typically, such results can be obtained by using the central limit theorem andcase-specific arguments, see for example [52, 102, 103, 110]. However, we takea different approach, aiming for new asymptotic expansions for the probabilityof delay and the probability of rejection. The first terms of these expansions arethe QED limits, and the higher-order terms are refinements to these QED limitsfor finite s. This generalizes earlier results on the Erlang B, C and A models[113–115].

Conceptually, we develop a unifying approach to derive such expansions for

134

6.2. Many-server systems with admission control

these control policies. A crucial step in our analysis is to rewrite the stationarydistribution in terms of a Laplace transform that contains all specific informationabout the control policy. Mathematically, establishing the expansions requires anapplication of Euler–Maclaurin (EM) summation, essentially identifying the er-ror terms caused by replacing a series expression in the stationary distribution bythe Laplace transform. In this paper we focus on the probability of delay and theprobability of rejection, but it is fairly straightforward using the same approachto obtain similar results for other characteristics of the stationary distribution,such as the mean and the cumulative distribution function.

The paper is structured as follows. In Section 6.2 we introduce the many-server system with admission control and derive the stability condition underwhich the stationary distribution exists. In Section 6.3 we discuss in detail theQED scaled control. We introduce a global control for managing the overall sys-tem fluctuations, and a local control that entails a precise form of ps(k). For boththe global and local control, we derive the stability condition and the stochastic-process limit for the normalized queue-length process in terms of a diffusionprocess. In Section 6.4 we derive QED approximations for systems with globalcontrol. Hereto, we enroll our concept of describing the stationary distribution interms of a Laplace transform and using EM summation to derive the expansions.In Section 6.5 we derive QED approximations for local control, making heavyuse of the intimate connection with global control and the tools developed inSection 6.4. For demonstrational purposes we also provide some numerical re-sults for the Erlang A control. In Section 6.6 we discuss the potential applicationsof the results obtained in this paper.

6.2 Many-server systems with admission control

Consider a system with s parallel servers to which customers arrive accordingto a Poisson process with rate λ. The service times of customers are assumedexponentially distributed with mean 1. A control policy dictates whether or not acustomer is admitted to system. A customer that finds upon arrival k other wait-ing customers in the system is allowed to join the queue with probability ps(k)and is rejected with probability 1− ps(k). In this way, the sequence ps(k)k∈N0

defines the control policy. Since we are interested in large, many-server systems,working at critical load and hence serving many customers, the probability ps(k)should be interpreted as the fraction of customers admitted in state s+ k.

135


Under these Markovian assumptions, and assuming that all interarrival timesand service times are mutually independent, this gives rise to a birth–deathprocess Qs(t) describing the number of customers in the system over time. Thebirth rates are λ for states k = 0, 1, . . . , s and λ·ps(k−s) for states k = s, s+1, . . . .The death rate in state k equals min k, s for states k = 1, 2, . . . . Assuming thestationary distribution to exist, with πk = limt→∞ P(Qs(t) = k), it follows fromsolving the balance equations that

(6.2) πk =

π0(sρ)k

k! , k = 1, 2, . . . , s,

π0ssρk

s!

k−s−1∏i=0

ps(i), k = s+ 1, s+ 2, . . . .

Here

(6.3) ρ =λ

s, π−1

0 =s∑

k=0

(sρ)k

k!+(sρ)s

s!Fs(ρ)

with

(6.4) Fs(ρ) =∞∑n=0

ps(0) · . . . · ps(n)ρn+1.

6.2.1 Stability

The wide class of allowed control policies renders it necessary to carefully inves-tigate the precise conditions under which the controlled system is stable. From(6.2)–(6.4), we conclude that the stationary distribution exists if and only ifps(k)k∈N0

and ρ are such that Fs(ρ)<∞. Let

(6.5) Ps := lim supn→∞

(ps(0) · . . . · ps(n))1

n+1 ,

and set 1/Ps =∞ when Ps = 0. We then see that Fs(ρ)<∞ when

(6.6) 0≤ ρ <1Ps

.

For convenience, we henceforth assume that

(6.7) limρ↑1/Ps

Fs(ρ) =∞,

136

6.2. Many-server systems with admission control

so that the stationary distribution exists if and only if (6.6) holds. The caselimρ↑1/Ps

Fs(ρ)<∞ (as considered for example in [54, 116]) can also be con-sidered in the present context, but leads to some complications that distractattention from the bottom line of the exposition.

6.2.2 Performance measures

We consider in this paper two performance measures, viz. the stationary prob-ability Ds(ρ) that an arriving customer finds all servers occupied, and the sta-tionary probability DR

s (ρ) that an arriving customer is rejected. In terms of πk

and ps(k), these stationary probabilities are given by Ds(ρ) =∑∞

k=sπk andDR

s (ρ) =∑∞

k=sπk(1− ps(k)). Denoting the Erlang B formula by

Bs(ρ) =(sρ)s/s!∑s

k=0 (sρ)k/k!, (6.8)

we express Ds(ρ) and DRs (ρ) in terms of Bs(ρ) and Fs(ρ) as

Ds(ρ) =1+ Fs(ρ)

B−1s (ρ) + Fs(ρ)

(6.9)

and

DRs (ρ) =

1+ (1−ρ−1) Fs(ρ)B−1

s (ρ) + Fs(ρ). (6.10)

There are two extreme control policies. The first is the control that deniesall customers access whenever all servers are occupied, i.e. ps(k) = 0 for k ∈ N0.This is in fact the Erlang B system. Then, Fs(ρ) = 0 and (6.9), (6.10) indeedgive Ds(ρ) = DR

s (ρ) = Bs(ρ). The other is the control that allows all customersaccess, i.e. ps(k) = 1 for k ∈ N0, known as the Erlang C system. Equation (6.4)gives Fs(ρ) = ρ/(1−ρ) for 0≤ ρ < 1. Subsequently, (6.10) gives DR

s (ρ) = 0 (acustomer is never rejected) and expression (6.9) reduces to the Erlang C formula

Cs(ρ) =(sρ)s

s!(1−ρ)∑s−1k=0

(sρ)kk! +

(sρ)ss! (1−ρ)

. (6.11)

137


6.3 QED scaled control

To enforce the QED regime in (6.1) we henceforth couple λ and s according to

(6.12) ρ =λ

s= 1−

γp

s⇔ λ= s− γ

ps, γ ∈ R.

We next introduce two types of control, referred to as global and local control,both designed to reduce the incoming traffic in periods of congestion.

6.3.1 Global control

Recall (6.4) and let

(6.13) qs(n) := ps(0) · . . . · ps(n), n ∈ N0

be the coefficient of ρn+1 in Fs(ρ). For n ∈ N0, qs(n) is roughly equal to theprobability that a (fictitious) batch arrival of n customers is allowed as a wholeto enter the system, given that all servers are busy and that the waiting queue isempty. Since in the QED regime queue lengths are of the order

ps, it is natural

to consider control policies such that qs(n) scales with s in ap

s-manner as well.One way to achieve this is by choosing qs(n) of the form

(6.14) qs(n) = fn+ 1p

s

, n ∈ N0

for s ≥ 1, where f (x), henceforth referred to as scaling profile, is a nonnegative,nonincreasing function of x ≥ 0 with f (0) = 1. With global control we meanthat the admission control is defined through qs(n) in (6.14). A key example iswhat we have called modified-drift control, in which case

(6.15) ps(k) = p1ps , p ∈ (0,∞), qs(n) = p

n+1ps = f

n+ 1p

s

with f (x) = px .

It appears that many practical admission policies fit into the Ansatz (6.14), ordo so in a limit sense as s→∞. This is the case for the class of local control,as discussed next.

138

6.3. QED scaled control

6.3.2 Local control

While the global control is defined via qs(n), we also introduce a local control,that for each state k, defines the probability of admitting a new customer as

ps(k) =1

1+ 1ps a( k+1p

s ), k ∈ N0, (6.16)

with a(x) a nonnegative, nondecreasing function of x ≥ 0. A special case isErlang A control a(x) = ϑx , which gives ps(k) = 1/(1 + (k + 1)ϑ/s). In thiscase the stationary distribution is identical to that of an M/M/s + M system(or Erlang A model), with the feature that customers that are waiting in thequeue abandon the system after exponentially distributed times with mean 1/ϑ.Garnett et al. [102] obtained the diffusion limit for the Erlang A model in theQED regime, and the limiting diffusion process turned out to be a combinationof two OU processes with different restraining forces, depending on whetherthe process is below or above zero.

Note that setting a(x) = 0 leads to the ordinary M/M/s system consideredin [52] with in the QED regime as limiting process a Brownian motion in theupper half plane. Depending on a, i.e. the type of control, one gets a specificlimiting behavior in the upper half plane, described by Brownian motion, an OUprocess, or some other type of diffusion process with drift −γ − a(x) in statex ≥ 0. We give a formal proof of this process-level convergence in Section 6.3.5.

6.3.3 Connection between local and global control

There is a fundamental relation between local and global control. By substituting(6.16) into (6.13), rewriting the product and using Taylor expansion, we seethat

qs(n) = ps(0) · . . . · ps(n) = exp−

n∑k=0

ln1+

1p

sak+ 1p

s

= exp−1p

s

n∑k=0

ak+ 1p

s

+O

1s

n∑k=0

a2k+ 1p

s

, n ∈ N0.

(6.17)

For large s and under mild conditions on a, the last expression in (6.17) can beapproximated by

exp−∫ n+1p

s

0

a(y) dy +O 1p

s

∫ n+1ps

0

a2(y) dy

, (6.18)

139


which will be discussed in more detail in Section 6.5.1. We get the approximation

(6.19) qs(n)≈ fn+ 1p

s

, n ∈ N0,

where

(6.20) f (x) = exp−∫ x

0

a(y) dy, x ≥ 0.

The validity range and the approximation error in (6.19) depend on theparticular form of a, which will be discussed in detail in Section 6.5. Also, (6.20)implies that f and a are related as

(6.21) a(x) = −f ′(x)f (x)

, x ≥ 0.

From here onwards we assume that f in (6.14) and a in (6.16) are indeedrelated according to (6.20) and (6.21). We can then show that both local andglobal control have a similar impact on a system, characterized by

(6.22) ps(k)≈ 1−1p

sak+ 1p

s

, k ∈ N0.

In Section 6.2.2 we discussed how our class of control policies can coverthe entire range between the Erlang B model and the Erlang C model. Let usdemonstrate that for the modified-drift control described in (6.15) that admitsa customer when all servers are busy with probability p1/

ps, where p ∈ (0,1).

Figure 6.1 shows for fixed ρ = 0.99 the delay probability Ds(ρ) as a function ofp. Here we show both global control f (x) = px and the local control counterparta(x) = − ln p. Notice the relatively small difference between global and localcontrol, which would be even smaller for larger values of s.

6.3.4 Stability with control

Now that we have established the connection between global and local controlvia the relations (6.20) and (6.21), we next show that the stability conditionsfor the systems with these respective controls are similar as well.

Define the Laplace transform of the scaling profile f as

(6.23) L (γ) =∫ ∞

0

e−γx f (x)dx , γ > γmin,

140

6.3. QED scaled control

0 1

1

p

Ds(ρ)

Erlang B

Erlang C

global control

local control

Figure 6.1: The stationary probability of delay for global control f (x) = px andlocal control a(x) = − ln p for s = 10 and ρ = 0.99.

where γmin = infγ ∈ R|L (γ) < ∞. From (6.20), it follows that γmin =− limx→∞ a(x), and since a is nondecreasing, we have

limγ↓γmin

L (γ) =∞. (6.24)

In Appendix 6.A, we derive the following stability condition for global andlocal control in terms of γmin. For large s, the two stability conditions are almostidentical.

Proposition 6.1 (Stability conditions). Assume (6.20) and (6.21). The station-ary distribution (6.2) exists for

(i) the global control (6.14) if and only if 0 ≤ ρ < e−γmin/p

s = 1− γmin/p

s +O(1/s);

(ii) the local control (6.16) if and only if 0≤ ρ < 1− γmin/p

s.

6.3.5 Stochastic-process limit

We now derive using the local control in (6.16) a stochastic-process limit, whichprovides additional insight into the roles of the function a and the Laplace

141


transform L .Let Qs(t) denote the process describing the number of customers present in

the system over time. The subscript s is attached to all relevant quantities todenote their dependence on the size of the system. We obtain a scaling limit forthe sequence of normalized processes Xs(t) = (Qs(t)− s)/

ps. Let “⇒” denote

weak convergence in the space D[0,∞) or convergence in distribution. Thenext result is proved in Appendix 6.B.

Proposition 6.2 (Weak convergence to a diffusion process). Assume (6.1) and(6.16). If a is continuous and bounded on every compact subinterval I of R, andXs(0)⇒ X (0) ∈ R, then for every t ≥ 0, as s→∞,

(6.25) Xs(t)⇒ X (t),

where the limit X (t) is the diffusion process with infinitesimal drift m(x) given by

(6.26) m(x) =

(−γ− x , x < 0,

−γ− a(x), x ≥ 0,

and constant infinitesimal variance σ2(x) = 2.

Proposition 6.2 sheds light on the effect of the control ps(k) as s becomeslarge. It shows that for local control (6.16), which is asymptotically of the form(6.22), the process Qs(t) approximately behaves as s + X (t)

ps, where X (t) is

a diffusion process with drift −γ− a(x) for x ≥ 0 and an OU process with drift−γ− x for x < 0.

The stationary distribution of X (t) is easy to derive. Denote the probabilitydensity function of the standard normal distribution byφ(x), and its cumulativedistribution function by Φ(x) =

∫ x

−∞φ(u)du.

Proposition 6.3 (Stationary distribution of the diffusion process). The densityfunction ω(x) of the stationary distribution for X (t) is given by

(6.27) ω(x) =

(C(γ)φ(x+γ)φ(γ) , x < 0,

C(γ)exp(∫ x

0 m(u)du), x ≥ 0,

with C(γ) =∫∞

0 exp(∫ x

0 m(u)du)dx + Φ(γ)φ(γ)

−1. Moreover,

(6.28)

∫ ∞

0

ω(x)dx =L (γ)

L (γ) + Φ(γ)φ(γ)

.

142

6.4. QED approximations for global control

Proof. Since the diffusion process X (t) has piecewise continuous parameters, wecan apply the procedure developed in [117] to find the stationary distribution.This procedure consists of composing the density function as in (6.27) based onthe density function of a diffusion process with drift −γ− a(x) for x > 0 and ofan OU process with drift −γ− x for x < 0. The function C(γ) normalizes thedistribution.

Equation (6.28) follows after substituting (6.26) with a(x) = − f ′(x)/ f (x)into (6.27) and evaluating

∫ ∞

0

exp∫ x

0

m(u)du

dx =

∫ ∞

0

e−γx f (x) dx =L (γ), (6.29)

proving (6.28).

A natural approach now is to approximate the distribution of Qs(t) by thedistribution of s+X (t)

ps when s is large. In Section 6.4, specifically Theorem 6.6,

we show that (6.28) equals lims→∞ Ds(ρ), as expected, and we also derive themost relevant correction terms for finite s. Important here is that Ds(ρ) convergesto a value in the interval (0, 1) as s→∞, which confirms that the local control in(6.16) leads to a non-degenerate limit. In [116], s-independent control policieshave been considered for which Ds(ρ) has 1/

ps-behavior for large s.

6.4 QED approximations for global control

In this section we focus on global control defined by the scaling profile f andAnsatz (6.14). For this type of control there is a convenient manner of approxi-mating Fs(ρ) as s→∞ in terms of the Laplace transform of f . Define

γs = −p

s ln (1− γ/p

s), γ ∈ R. (6.30)

Utilizing (6.13) and (6.14) and recalling that ρ = 1−γ/p

s, we can write (6.4)as

Fs(ρ) =∞∑n=0

e−n+1p

s γs fn+ 1p

s

, (6.31)

This expression for Fs(ρ) is instrumental for our analysis. We apply EM summa-tion to (6.31), in order to replace the summation over n by an integral and anappropriate number of error terms. The approach is explained in Section 6.4.1,

143


and leads to the QED approximations for the stationary delay and rejectionprobability presented in Section 6.4.2. These approximations are demonstratedin Section 6.4.3 for several types of global control.

6.4.1 Euler–Maclaurin (EM) summation

We assume that the function f (x) in (6.14) is nonnegative, nonincreasing andsmooth, that is f ∈ C4([0,∞)), and that f (0) = 1. Furthermore, we assumethat for any γ > γmin, e−γx f ( j)(x) ∈ L1([0,∞)) and e−γx f ( j)(x)→ 0 as x →∞for j = 0,1, 2,3, 4.

We shall use the following form of the EM summation formula about whichmore details are collected in Appendix 6.C. Assume that g : [0,∞)→ R withg ∈ C2([0,∞)) and g( j) ∈ L1([0,∞)), j = 0, 1,2. Then

(6.32)∞∑n=0

gn+ 1p

s

=p

s

∫ ∞1

2p

s

g(x)dx +1

24p

sg ′ 1

2p

s

+ R,

where

(6.33) |R| ≤1

12p

s

∫ ∞

0

|g(2)(x)|dx .

When also g(4) ∈ L1([0,∞)), we have the asymptotically tighter bound

(6.34) |R| ≤1

384sp

s

∫ ∞

0

|g(4)(x)|dx .

By setting g(x) = e−γx f (x) for x ≥ 0 and γ > γmin, and using these formulas,we will now obtain several QED approximations.

6.4.2 Corrected QED approximations

We first present a result for Fs(ρ).

Theorem 6.4. With ρ = 1− γ/p

s and γmin < γ≤p

s,

(6.35) Fs(ρ) =p

sL (γs)−12+O

1p

s

,

where γs = −p

s ln (1− γ/p

s) and where O(1/p

s) holds uniformly in any compactset of γ’s contained in (γmin,∞).

144


Proof. We have from (6.31), (6.32) and (6.33) that

Fs(ρ) =p

s

∫ ∞1

2p

s

e−γs x f (x)dx +1

24p

s(e−γs x f (x))′

12p

s

+ R (6.36)

with

|R| ≤1

12p

s

∫ ∞

0

|(e−γs x f (x))(2)(x)|dx . (6.37)

Assume that γ is restricted to a compact set C contained in (γmin,∞). Thenγs is restricted to a compact set D contained in (γmin,∞) for all s ≥ 1 withp

s ≥ 2max|γ| | γ ∈ C. Hence

e−γs x f (x)− 1= O 1p

s

, 0≤ x ≤

12p

s, (6.38)

where O(1/p

s) holds uniformly in γ ∈ C . Therefore, we can replace the integralat the right-hand side of (6.36) by L (γs)− 1/(2

ps), at the expense of an error

O(1/s) uniformly in γ ∈ C . Furthermore,

(e−γs x f (x))′ 1

2p

s

= O(1), s ≥ 1, (6.39)

uniformly in γ ∈ C by smoothness of f . Finally, R= O(1/p

s) uniformly in γ ∈ Csince there is the bound

|R| ≤1

12p

s

∫ ∞

0

e−γs x(|γs|2 + 2|γs|| f ′(x)|+ | f ′′(x)|)dx (6.40)

in which γs ∈ D with f satisfying the assumptions made at the beginning ofSection 6.4.1.

Theorem 6.4 yields a simple and often accurate approximation of Fs(ρ),which we illustrate in Section 6.4.3. However, in the leading term

psL (γs), the

dependence on the number of servers s and the parameter γ is combined intothe single quantity γs. A more insightful result is given in Theorem 6.5 below,where the dependence of the approximating terms on s and γ is separated.

Theorem 6.5. With ρ = 1− γ/p

s and γmin < γ≤p

s,

Fs(ρ) =p

sL (γ) +M (γ) +O 1p

s

, (6.41)

145


where

(6.42) M (γ) =12γ2L ′(γ)−

12

,

and O(1/p

s) holds uniformly in any compact set of γ’s contained in (γmin,∞). Inleading order, the O(1/

ps) is given as N (γ)/

ps, where

(6.43) N (γ) =13γ3L ′(γ) +

18γ4L ′′(γ) +

112(γ− f ′(0)).

Proof. This result is obtained from (6.32) in a similar way as Theorem 6.4,

using now the estimate (6.34) of R, and approximating∫ 1/(2

ps)

0 e−γs x f (x)dxand (e−γs x f (x))′(1/(2

ps)) more carefully. In particular, we have

(6.44)

∫ 1/(2p

s)

0

e−γs x f (x)dx =1

2p

s+

18s( f ′(0)− γs) +O

1sp

s

and

(6.45)d

dx(e−γs x f (x))

12p

s

= γs − f ′(0) +O

1p

s

.

Furthermore, because γs = γ+ γ2/(2p

s) + γ3/(3s) + . . . for |γ| <p

s, we canapproximate L (γs) by(6.46)

L (γs)−L (γ) = (γs − γ)L ′(γ) +12(γs − γ)2L ′′(γ) +

16(γs − γ)3L ′′′(γ) + . . .

=γ2

2p

sL ′(γ) +

1s

13γ3L ′(γ) +

18γ4L ′′(γ)

+O

1sp

s

.

The O’s in (6.44)–(6.46) hold uniformly in γ in any compact set contained in(γmin,∞).

We use the following short-hand notations for approximations of Fs(ρ) andBs(ρ) as they occur in the performance measures Ds(ρ) and DR

s (ρ) in (6.9) and(6.10). We write (6.41) using (6.43) as

(6.47) Fs(ρ) =p

sL +M +1p

sN =

psL +M +O

1p

s

,

and we write the Jagerman approximation of Bs(ρ), see [116] and [118, Theo-rem 14], as

(6.48) Bs(ρ) =1p

sg +

1s

h+O 1

sp

s

.

146


Here, ρ = 1− γ/p

s, and

g(γ) =φ(γ)Φ(γ)

, h(γ) = −13

γ3 + (γ2 + 2)g(γ)

g(γ). (6.49)

The following results for Ds(ρ) and DRs (ρ) are proved in Appendix 6.D using

(6.47) and (6.48).

Theorem 6.6 (Corrected QED approximations). The stationary probability ofdelay satisfies

Ds(ρ) = T1(γ) +1p

sT2(γ) +O

1s

, (6.50)

where

T1 =gL

1+ gL, T2 =

(h+ g2)L + g(M + 1)(1+ gL )2

, (6.51)

and where O(1/s) holds uniformly in any compact set of γ’s contained in (γmin,∞).The stationary rejection probability satisfies

DRs (ρ) =

1p

sT R

1 (γ) +1s

T R2 (γ) +O

1sp

s

, (6.52)

where

T R1 =

1− γL1+ gL

g, T R2 =

1− γL1+ gL

h− γg

γL +M1− γL

− ghL + gM

1+ gL

, (6.53)

and O(1/sp

s) holds uniformly in any compact set of γ’s contained in (γmin,∞).

6.4.3 Examples

We now present several examples to illustrate Theorems 6.4, 6.5 and 6.6.

Modified-drift control (global)

Consider f (x) = px for x ≥ 0, with p ∈ (0, 1) fixed. Then, γmin = ln p, Ps = p1/p

s

and

Fs(ρ) =p1/p

s(1− γ/p

s)1− p1/

ps(1− γ/

ps)

,p

s(1− p−1/p

s)< γ≤p

s. (6.54)

Theorem 6.5 gives the approximation

Fs(ρ)≈p

sγ− ln p

−γ2

2(γ− ln p)2−

12

, γ > ln p. (6.55)

147


Erlang A control (global)

Let f (x) = px2for x ≥ 0, with p ∈ (0,1) fixed. In this case, γmin = −∞ and

Ps = 0. Also,

(6.56) L (γ) =1pαχ(γ/(2

pα)), γ ∈ R,

where α = − ln p and χ is Mills’ ratio, defined as χ(δ) = eδ2 ∫∞δ

e−y2dy for

δ ∈ R [119, §7.8]. Taking the derivative, we find that

(6.57) L ′(γ) =γ

2αpαχ(γ/(2

pα))−

12α

, γ ∈ R,

and we then obtain from Theorem 6.5 the approximation

(6.58) Fs(ρ)≈p

spαχ(γ/(2

pα)) +

14

γpα

3χ(γ/(2

pα))−

14

γpα

2−

12

for γ ∈ R. Note that the approximation Fs(ρ) ≈p

sL (γs) − 1/2 as describedin Theorem 6.4 can also be computed from (6.56) after recalling that γs =−p

s ln (1− γ/p

s).

Scaled buffer control (global)

Take a fixed η > 0 and set ps(k) = 1[k+ 1< ηp

s] for k ∈ N0. Thus for n ∈ N0,qs(n) = ps(n) = f ((n+ 1)/

ps), with f (x) = 1[x ∈ [0,η)] for x ≥ 0. It follows

that Ps = 0, γmin = −∞ and

(6.59) Fs(ρ) =psγ− 1

1−

1−

γp

s

bηpsc, −∞< γ≤

ps.

The function f is not smooth, and strictly speaking, Theorems 6.4 and 6.5do not apply. Still, we can calculate

(6.60) L (γ) =1γ(1− e−γη), L ′(γ) = −

1γ2(1− (1+ γη)e−γη), γ ∈ R,

and use the approximation that Theorem 6.4 would give, i.e.

(6.61) Fs(ρ)≈p

sL (γs)−12=

1− (1− γ/p

s)ηp

s

− ln (1− γ/p

s)−

12

.

148

6.5. QED approximations for local control

Alternatively, Theorem 6.5 gives the approximation

Fs(ρ)≈p

sL (γ)+12γ2L ′(γ)−

12=psγ−1(1−e−γη)−

12

e−γη(1−γη) (6.62)

for γ ∈ R.While (6.59) has a jump as a function of s at all s where η

ps is integer, its

approximations in (6.61) and (6.62) are smooth functions of s, if we considers ≥ 1 as a continuous variable. The averages of the approximations over s-intervals [(k/η)2, ((k + 1)/η)2] with integer k agree well with the average of(6.59) over these intervals. Thus, while Theorems 6.4 and 6.5 do not apply, theyyield approximations that perform well in an appropriate average sense. This isalso illustrated in Figure 6.2.

0 5 10

1

η

Ds(ρ)

Erlang B

Erlang C

Figure 6.2: The stationary probability of delay for s = 10 and scaled buffercontrol. The jagged, blue curve gives the exact value (6.59) and the red, smoothcurve pertains to approximation (6.62).

6.5 QED approximations for local control

A clear technical advantage of global control is that it leads to infinite-seriesexpressions for the performance measures Ds(ρ) and DR

s (ρ) that are directly

149


amenable to asymptotic analysis based on EM summation. This approach wasfollowed in Section 6.4 and led to Theorem 6.6. As argued in Section 6.3, insome practical cases it is more natural to work with the local control defined in(6.16). In this section we show that Theorem 6.6 also gives sharp approximationsfor local control. Indeed, in Section 6.3.3 it was argued that for local control,qs(n) ≈ f ((n + 1)/

ps) with f defined as in (6.20). Consider for instance the

modified-drift control in (6.15), in which case

(6.63)

Fs(ρ) =∞∑n=0

pn+1p

s

1−

γp

s

n+1

=p

sγ− ln p

−γ

γ− ln p−

12

ln pγ− ln p

2+O

1p

s

.

Here, the second equality follows from the QED approximation in (6.41). Thelocal counterpart follows from a(x) = − f ′(x)/ f (x) = − ln p and (6.16), forwhich

(6.64) Fs(ρ) =∞∑n=0

1

1− 1ps ln p

n+11−

γp

s

n+1=

ps

γ− ln p−

γ

γ− ln p.

The second equality in (6.64) follows from summation of a geometric series.Hence, for the example of modified-drift control, it can be seen from the closeresemblance of the last members of (6.63) and (6.64) that approximating localcontrol by global control yields sharp estimates in the QED regime.

In Section 6.5.1 we make formal the accuracy of the approximation qs(n)≈f ((n+ 1)/

ps) for a wide range of local controls. As it turns out, the approxi-

mation becomes asymptotically correct in the QED regime. Therefore, for localcontrols for which the Ansatz qs(n) = f ((n+ 1)/

ps) does not hold precisely, it

will give sharp approximations for the performance measures in the QED regime.For the example of Erlang A control, with a(x) = ϑx , this is demonstrated inSection 6.5.2.

6.5.1 Approximating local by global control

We first present a general result for all functions a considered in this paper. Theproof is given in Appendix 6.E.

150


Proposition 6.7 (Relation between global and local control). Assume that a(x)is nonnegative and nondecreasing in x ≥ 0. There is an increasing function ψ(s)of s ≥ 1 with ψ(s)→∞, s→∞, such that

qs(n) = fn+ 1p

s

1+O(s−

14 ), 0≤ n+ 1≤

psψ(s), (6.65)

where f is given as in (6.20).

We next illustrate Proposition 6.7 for a special case of increasing a. Let ϑ > 0and α≥ 0, and let a(x) = ϑ xα for x ≥ 0. Inspecting the proof of Proposition 6.7,case δ = 1/4, it is seen that ψ is found by requiring

an+ 1p

s

≤

12

s14 ,

∫ n+1ps

0

a2(x)dx ≤ s14 . (6.66)

Proposition 6.7 yields

qs(n) = exp −ϑα+ 1

n+ 1p

s

α+11+O(s−

14 ), 0≤ n+ 1≤

psψ(s), (6.67)

with

ψ(s) =min¦ 1

2ϑ

1αs

14α ,2α+ 1ϑ2

12α+1

s1

4(2α+1)

©. (6.68)

6.5.2 Erlang A control (local)

We now consider in detail Erlang A control in order to demonstrate our obtainedQED approximations. Erlang A control gives rise to a birth–death process thatis identical to the classical Erlang A model [102, 115]. It allows us to expressFs(ρ) in terms of the confluent hypergeometric function and to subsequentlyshow that an asymptotic expansion of the confluent hypergeometric functionleads to a QED approximation similar to (6.58). We thus consider the example

ps(k) =1

1+ (k+ 1) ϑs, k ∈ N0, (6.69)

which corresponds to α= 1 when a(x) = ϑxα for x ≥ 0, implying that f (x) =exp (−ϑx2/2) for x ≥ 0.

Proposition 6.8. Assume that ps(k) is given by (6.69). Then

Fs(ρ) =1ρ

M(1, s/ϑ, sρ/ϑ)− 1−ρ

, ρ ≥ 0, (6.70)

151


in which

(6.71) M(a, b, z) =∞∑n=0

(a)n(b)n

zn

n!, z ∈ C,

is the confluent hypergeometric function [119, Chapter 13], with (x)l the Pochham-mer symbol, i.e. (x)l = 0 for l = 1 and (x)l = x(x + 1) · . . . · (x + l − 1) for l ≥ 1.

Proof. For n ∈ N0,

(6.72) qs(n) =n∏

k=0

1

1+ (k+ 1) ϑs=(s/ϑ)n+2

(s/ϑ)n+2.

Therefore

(6.73) Fs(ρ) =∞∑n=0

qs(n)ρn+1 =

∞∑n=0

(s/ϑ)n+2

(s/ϑ)n+2ρn+1,

and (6.70) follows after some rearrangements.

In [119, 13.8(ii)] the asymptotics of M(a, b, z) is considered when b and zare large while a is fixed and b/z is in a compact set contained in (0,∞). With

(6.74) a = 1 , b = s/ϑ , z = sρ/ϑ

and s → ∞, while ρ = 1 − γ/p

s is close to 1, this is precisely the situationwe are interested in. Temme [120] gives a complete asymptotic series, and thisleads to the following result.

Proposition 6.9. As s→∞,

(6.75) Fs(ρ)∼2ϑ

12χ(γ/

p2ϑ)p

s+γ3p

2

3ϑ32

χ(γ/p

2ϑ)−γ2

3ϑ−

23

.

Proof. The first two terms of Temme’s asymptotic series [120] are as follows.Let ζ=

p2(ρ − 1− lnρ), where sgn(ζ) = sgn(ρ − 1). Then

(6.76)

M1,

sϑ

,sρϑ

∼ sϑ

12

expζ2s

4ϑ

¦ρU

12

,−ζ sϑ

12

+ρ −

ζ

ρ − 1

1

ζ

sϑ

12

U−

12

,−ζ sϑ

12©

152


with U the parabolic cylinder function of [119, Ch. 12]. In this particular case[119, §12.5.1, §12.7.1],

U1

2, z= e−

14 z2

∫ ∞

0

e−12 t2−zt d t, U

−

12

, z= e−

14 z2

. (6.77)

Since ρ = 1− γ/p

s,

ζ= −γp

s−γ2

3s+O

γ3

sp

s

, ρ −

ζ

ρ − 1= −

43γp

s+O

γ2

s

. (6.78)

Substituting in (6.76), together with (6.77), we get

Fs(ρ)∼ −1−1ρ+ sϑ

12

expζ2s

4ϑ

¦exp

−ζ2s4ϑ

∫ ∞

0

e−12 t2+ζ( s

ϑ )12 t d t

+ρ − ζ/(ρ − 1)

ζρ(s/ϑ)12

exp−ζ2s4ϑ

©,

(6.79)

so that

Fs(ρ)∼ −23+ sϑ

12

∫ ∞

0

e−12 t2+ζ( s

ϑ )12 t d t +O

γp

s

. (6.80)

Next

ζ sϑ

12 = −

γpϑ−

γ2

3pϑs+O

γ3

s

, (6.81)

and using this in (6.80), we get

Fs(ρ) =−23+ sϑ

12

∫ ∞

0

e−12 t2− γtp

ϑ d t

−13

sϑ

12 γ2

pϑs

∫ ∞

0

e−12 t2− γtp

ϑ td t +O γp

s

.

(6.82)

Finally, using that

∫ ∞

0

e−12 t2− γtp

ϑ d t =p

2χ(γ/p

2ϑ), (6.83)

∫ ∞

0

e−12 t2− γtp

ϑ td t = −χ ′(γ/p

2ϑ) = − 2γp

2ϑχ(γ/

p2ϑ)− 1

, (6.84)

we obtain the result.

153


It is instructive to rewrite the asymptotics (6.58) of Fs(ρ) in Example 6.4.3for the case that qn = f ((n+1)/

ps) and f (x) = px , in terms of ϑ = 2α= −2 ln p.

In doing so, (6.58) becomes

(6.85) Fs(ρ)∼2ϑ

1/2χ(γ/

p2ϑ)p

s+γ3

p2ϑ3/2

χ(γ/p

2ϑ)−γ2

2ϑ−

12

.

Observe the close resemblance between (6.85) and (6.75).

Numerical comparison

For Erlang A control, for which f (x) = px2and − ln p = α= ϑ/2, we have now

determined an exact expression and an asymptotic expression for Fs(ρ), givenin Proposition 6.8 and Proposition 6.9, respectively. Through (6.9) we thenobtain exact and asymptotic expressions for Ds. Furthermore, we can obtainapproximate values for Ds using the first-order and second-order approximationin Theorem 6.6. Table 6.1 shows a numerical comparison when using thesedifferent expressions for Ds.

From Table 6.1 we see that the precision of all approximations increase withs. We also see that the second-order approximation T1+ T2/

ps is more accurate

than the first-order approximation T1, particularly for moderate values of s.

6.6 Conclusions and outlook

We have introduced QED scaled control, designed to reduce the incoming trafficin periods of congestion, in such a way that the controlled many-server systemremains within the domain of attraction of the favorable QED regime. Thescaled control is chosen such that it affects the typical O(

ps) queue lengths that

arise in the QED regime. The class of many-server systems with QED controlintroduced in this paper contains the Erlang B, C and A models as special cases.For all cases we have derived so-called corrected QED approximations, whichnot only identify the QED limits as leading terms, but also provide correctionsthrough higher order terms for finite system sizes s. As a key example we took thestationary probability of delay, for which the corrected QED approximation readsDs(ρ)≈ T1(γ) + T2(γ)/

ps, as stated in Theorem 6.6. The technique developed

in this paper to obtain the corrected diffusion approximations can be easilyapplied to other characteristics of the stationary distribution, such as the meanand the cumulative distribution function.

154

6.6. Conclusions and outlook

ϑ=

1ϑ=

10ϑ=

100

sD

exac

ts

Das

ymp

sD

appr

oxs

Dex

act

sD

asym

ps

Dap

prox

sD

exac

ts

Das

ymp

sD

appr

oxs

10.

5934

30.

5727

70.

6258

20.

4941

50.

3930

50.

4852

80.

4759

10.

2917

20.

4107

6

20.

5543

70.

5434

20.

5773

00.

4138

90.

3470

40.

4079

70.

3809

30.

2352

50.

3149

8

40.

5265

20.

5209

20.

5430

00.

3513

70.

3122

50.

3533

00.

2986

20.

1928

30.

2472

6

80.

5069

10.

5041

00.

5187

40.

3073

20.

2865

80.

3146

50.

2322

60.

1617

20.

1993

8

160.

4931

30.

4917

20.

5015

80.

2783

00.

2679

20.

2873

10.

1822

90.

1392

50.

1655

2

320.

4834

30.

4827

30.

4894

60.

2595

60.

2544

80.

2679

80.

1471

70.

1231

50.

1415

7

640.

4766

00.

4762

50.

4808

80.

2473

50.

2448

70.

2543

20.

1240

70.

1116

90.

1246

4

128

0.47

178

0.47

160

0.47

481

0.23

924

0.23

802

0.24

465

0.10

961

0.10

354

0.11

267

256

0.46

837

0.46

828

0.47

053

0.23

375

0.23

316

0.23

782

0.10

068

0.09

776

0.10

421

512

0.46

597

0.46

592

0.46

749

0.23

000

0.22

970

0.23

299

0.09

506

0.09

367

0.09

822

1024

0.46

427

0.46

425

0.46

535

0.22

740

0.22

725

0.22

957

0.09

146

0.09

774

0.09

399

Tabl

e6.

1:N

umer

ical

com

pari

son

ofdi

ffer

ent

expr

essi

ons

ofD

sfo

rf(

x)=

exp(−ϑ

x2/2)

andγ=

0.1.

Her

e,D

exac

ts

isca

lcul

ated

usin

gPr

opos

itio

n6.

8,D

asym

ps

usin

gPr

opos

itio

n6.

9an

dD

appr

oxs

=T 1+

T 2/p

sus

ing

Theo

rem

6.6.

Furt

herm

ore,

for

alls

,T 1≈

0.46

017,

0.22

132

and

0.08

377

forϑ=

1,10

and

100,

resp

ecti

vely

.

155


Our corrected QED approximations pave the way for obtaining optimalityresults for dimensioning systems [107]. Consider for instance the basic problemof determining the largest load ρ such that Ds(ρ) ≤ ε with ε ∈ (0,1). Thedelay probability is a function of the two model parameters s and λ, and ofthe control policy. Denote this unique solution by ρ = ρopt and define γopt =p

s(1−ρopt). Asymptotic dimensioning would approximate Ds(ρ) by the QEDlimit T1(γ) that only depends on γ (and no longer on both s and λ). Hence, theinverse problem can then be approximatively solved by searching for the γ= γ∗such that T1(γ) = ε, and then setting the load according to ρ∗ = 1 − γ∗/

ps.

This procedure is referred to as square-root staffing, and the error |γopt − γ∗| iscalled the optimality gap. In future work, we will leverage the corrected QEDapproximations derived in the present paper to characterize the optimality gapsfor a large class of dimensioning problems.

Appendix

6.A Proof of Proposition 6.1

6.A.1 Proof of Proposition 6.1(i)

We assume that Fs(ρ), ρ = 1− γ/p

s, is of the form (6.4) with

(6.86) ps(0) · . . . · ps(n) = qs(n) = fn+ 1p

s

, n= 0,1, . . . ,

where s ≥ 1 and f (x) is a nonnegative and nonincreasing function in x ≥ 0with f (0) = 1. Furthermore, we recall the definition of γs = −

ps ln(1−γ/

ps) in

(6.30). The stability result of Proposition 6.1(i) is a consequence of the followinginequality.

Proposition 6.10. Forp

s(1− exp(−γmin/p

s))< γ≤ 0,

(6.87) eγs/p

s

∫ ∞

1/p

s

e−γs x f (x) dx ≤1p

sFs(ρ)≤ e−γs/

ps

∫ ∞

0

e−γs x f (x) dx .

Proof. We start by noting that

(6.88) γ >p

s(1− e−γmin/p

s) =: γmin,s ⇔ γs > γmin.

156

6.A. Proof of Proposition 6.1

We consider formula (6.31) for Fs(ρ). We have for γ≤ 0 and n= 0, 1, . . . frommonotonicity of f that

f (x)≥ fn+ 1p

s

, e−γs x ≥ eγs/

ps e−

n+1ps γs ,

np

s≤ x ≤

n+ 1p

s(6.89)

and

f (x)≤ fn+ 1p

s

, e−γs x ≤ e−γs/

ps e−

n+1ps γs ,

n+ 1p

s≤ x ≤

n+ 2p

s. (6.90)

Hence, from (6.89) for n= 0,1, . . .∫ (n+1)/

ps

n/p

s

e−γs x f (x) dx ≥1p

seγs/

ps e−γs

n+1ps fn+ 1p

s

, (6.91)

and from (6.90) for n= 0, 1, . . .∫ (n+2)/

ps

(n+1)/p

s

e−γs x f (x) dx ≤1p

se−γs/

ps e−γs

n+1ps fn+ 1p

s

. (6.92)

From (6.91) and (6.92) the two inequalities in (6.87) readily follow.

Proposition 6.10 shows that Fs(ρ) <∞ if and only if L (γs) <∞, whereit is used that f is nonnegative and bounded. By the definition of γmin and theassumption in (6.24) it follows that Fs(ρ) <∞ if and only if γs > γmin. Thenfrom (6.88) the equivalence in Proposition 6.1(i) follows.

6.A.2 Proof of Proposition 6.1(ii)

Let s = 1,2, . . . and consider the case −γmin = limx→∞ a(x) =: L <∞. Then,by monotonicity of a,

qs(n) =n∏

k=0

1

1+ 1ps a( k+1/2p

s )≥ 1

1+ 1ps L

n+1, (6.93)

and so Fs(ρ) =∞ when ρ ≥ 1 + 1ps L. Next, take 0 ≤ ρ < 1 + 1p

s L. From

a = a( k+1/2ps )≤ L, ρ < 1+ 1p

s L,

ρ

1+ 1ps a= 1−

1+ 1ps L −ρ

1+ 1ps a

+1p

sL − a

1+ 1ps a

< 1−1−

ρ

1+ 1ps L

+

1p

s(L − a).

(6.94)

157


Hence, we can find a K = 1, 2 . . . such that

(6.95)ρ

1+ 1ps a( k+1/2p

s )≤ 1−

12

1−

ρ

1+ 1ps L

, k > K .

Therefore,

(6.96)

Fs(ρ)≤K∑

n=0

ρn+1 +ρK+1∞∑

n=K+1

ρn−Kn∏

k=K+1

1

1+ 1ps a( k+1/2p

s )

<

K∑n=0

ρn+1 +ρK+1∞∑

n=K+1

1−

12

1−

ρ

1+ 1ps L

n−K<∞.

This proves the result for the case L <∞. The proof for the case L =∞ issimilar.

6.B Proof of Proposition 6.2

We will use Stone’s theorem [121], for which we need to verify that (i) the statespace of the normalized process converges to a limit that is dense in R and (ii)the infinitesimal mean and variance of Xs(t)t≥0 converge uniformly to m(x)and σ2(x), respectively.

Condition (i) is readily verified. The state space of Xs(t)t≥0 is given byΩs = (k − s)/

ps|k ∈ N0, and we see that for every x ∈ R and every ε > 0,

there exists an s > 0 such that miny∈Ωs|x − y|< ε.

To verify condition (ii), recall first that for any birth–death process Y (t)t≥0

with birth–death parameters λ(k) and µ(k) and associated states

(6.97) y (0) < y (1) < y (2) < . . . ,

the infinitesimal mean and variance are defined as

(6.98) m(y) = λ(e(y))(y (e(y)+1) − y (e(y)))−µ(e(y))(y (e(y)) − y (e(y)−1))

and

(6.99) σ2(y) = λ(e(y))(y (e(y)+1) − y (e(y)))2 +µ(e(y))(y (e(y)) − y (e(y)−1))2,

respectively. Here,

(6.100) e(y) = arg supk∈N0

y (k)|y (k) ≤ y

158

6.B. Proof of Proposition 6.2

is the label of the state closest to, but never above, y ∈ [y (0), y (∞)).

For each birth–death process Xs(t)t≥0, we have that

λ(k)s = λs1[k < s] +λs ps(k− s)1[k ≥ s], k ∈ N0, (6.101)

µ(k)s =mink, s, k ∈ N0, (6.102)

x (es(x)+1)s − x (es(x))

s = 1/p

s, x ∈ [−p

s,∞), (6.103)

and

es(x) = bs+ xp

sc, x ∈ [−p

s,∞). (6.104)

Because γ is fixed, (6.12) prescribes that we are scaling the arrival rate as λs =s− γ

ps. This yields

ms(x) =

(−γ+ s−bs+

psxcp

s , x < 0,

−γps(bs+p

sxc − s) + (ps(bs+p

sxc − s)− 1)p

s, x ≥ 0,(6.105)

and

σ2s (x) =

(s+bs+

psxc)

s − γps , x < 0,

ps(bs+p

sxc − s) + 1− γps(bs+p

sxc−s)ps , x ≥ 0.

(6.106)

By first Taylor expanding (6.16),

ps(k) =1

1+ 1ps a( k+1p

s )= 1−

1p

sak+ 1p

s

+O(s−1), k ∈ N0, (6.107)

and then substituting (6.107) into (6.105) and (6.106), we conclude that

ms(x) =

(−γ− bs+

psxc−sps , x < 0,

−γ− a bs+psxc−s+1p

s

+O(s−

12 ), x ≥ 0,

(6.108)

and σ2s (x) = 2 + O(s−

12 ) for all x . Because (bs +

psxc − s + 1)/

ps → x as

s→∞ and a is continuous and bounded on every compact subinterval I of R,we have that for every compact subinterval I of R, lims→∞ms(x) = m(x) andlims→∞σ

2s (x) = σ

2(x) = 2 uniformly for x ∈ I . This concludes the proof.

159


6.C Euler–Maclaurin (EM) summation

Let m= 1, 2, . . . , N = 1,2, . . . , and let h ∈ C2m([0, N + 1]). Then(6.109)

N∑n=0

h(n+ 1/2)−∫ N+1

0

h(x)dx =m∑

k=1

B2k(1/2)(2k)!

(h(2k−1)(N+1)−h(2k−1)(0))

−∫ N+1

0

B2m(x − 1/2)(2m)!

h(2m)(x)dx =m−1∑k=1

B2k(1/2)(2k)!

(h(2k−1)(N+1)−h(2k−1)(0))

−∫ N+1

0

B2m(x − 1/2)− B2m(1/2)(2m)!

h(2m)(x)dx .

Here

(6.110) B2k(1/2) = −(1− 2−2k+1)B2k, k = 1, 2, . . . ,

with B2k the Bernoulli numbers of positive, even order, see [119, §24.2 (i)] andB2m(x) = B2m(x − bxc) with B2m(x) the Bernoulli polynomial of degree 2m.Moreover, the two integrals in (6.109) involving B2m can be estimated as

(6.111)∫ N+1

0

B2m(x − 1/2)(2m)!

h(2m)(x)dx≤ |B2m|(2m)!

∫ N+1

0

|h(2m)(x)|dx

and

(6.112)

∫ N+1

0

B2m(x − 1/2)− B2m(1/2)(2m)!

h(2m)(x)dx

≤ 2(1− 2−2m)|B2m|(2m)!

∫ N+1

0

|h(2m)(x)|dx ,

respectively. These formulas follow from [122, Theorem 1.3] or [55, Theo-rem 2.1] (the latter reference containing a proof) for EM summation of h sam-pled at points n+ ν, n = 0,1, . . . , N . For the special case that ν = 1/2, a simpli-fication occurs due to B j(1/2) = 0 for j = 1,3, . . . . The bounds in (6.111) and(6.112) follow from [119, §24.12 (i) and §24.4.34], with a special considerationfor B2(x) = x2− x +1/6. We have B2 = 1/6, B4 = −1/30, B6 = 1/42, . . . . Whenm= 1, the series over k in the second form in (6.109) is absent.

The formula (6.109) is sometimes called the second EM summation for-mula, see [123, (5.8.18–19) on p. 154]. It distinguishes itself from the first

160

6.D. Proof of Theorem 6.6

EM summation formula, as appears in [119, §2.10 (i)], in that (i) half-integer,rather than integer samples of h are used at the left-hand side, (ii) absence of aterm 1

2 (h(N + 1/2) + h(1/2)) at the right-hand side, and (iii) smaller coefficientsB2k(1/2) in the series over k at the right-hand side. Hence, also see the commentin [123, (5.8.18–19)], the second EM formula is somewhat simpler in form andslightly more accurate when the remainder terms R are dropped than the firstEM formula.

In the main text, this formula is used for h(x) = g((x + 1/2)/p

s), with g ∈C2m([0,∞)) and m= 1 and 2 while assuming that

∫∞0 |g

(2m)(x)|dx <∞ andthat g(2k−1)(x)→ 0, x →∞, for k = 1 and k = 1, 2, respectively. Subsequently,the formula is used for functions g(x) of the form exp(−γs x) f (x), x ≥ 0.

6.D Proof of Theorem 6.6

We can write

Ds(ρ) =1+ Fs(ρ)

B−1s (ρ) + Fs(ρ)

= H1(Fs(ρ), Bs(ρ)) (6.113)

and

DRs (ρ) =

1+ (1−ρ−1)Fs(ρ)B−1

s (ρ) + Fs(ρ)= H1−ρ−1(Fs(ρ), Bs(ρ)), (6.114)

whereHa(x , y) =

1+ axy−1 + x

. (6.115)

Error propagation in Ds and DRs when both Fs(ρ) and Bs(ρ) are approximated

can be assessed using the following result.

Proposition 6.11. For a ∈ R and x ≥ 0, x + ∆x ≥ 0 and 0 ≤ y ≤ 1, 0 ≤y +∆y ≤ 1, it holds that

|Ha(x +∆x , y +∆y)−Ha(x , y)| ≤ |y(a− y)||∆x |+ |1+ ax ||∆y|. (6.116)

Proof. We have

∂ Ha

∂ x=

y(a− y)(1+ x y)2

,∂ Ha

∂ y=

1+ ax(1+ x y)2

. (6.117)

Therefore

maxx≥0

∂ Ha

∂ x

= |y(a− y)|, 0≤ y ≤ 1, (6.118)

161


and

(6.119) max0≤y≤1

∂ Ha

∂ y

= |1+ ax |, x ≥ 0,

and the result follows.

We insert the approximations (6.47) and (6.48) into (6.113), and we get

(6.120) Ds(ρ) = 1p

sg +

1s

h 1+

psL +M

1+ ( 1ps g + 1

s h)(p

sL +M )+O

1s

.

The approximation error O(1/s) here is obtained from using (6.116) with a = 1and with

x =p

sL +M = O(p

s), ∆x = O 1p

s

,(6.121)

y =1p

sg +

1s

h= O 1p

s

, ∆y = O

1sp

s

,(6.122)

where the O’s in (6.121)–(6.122) hold uniformly in any compact set of γ’s con-tained in (γmin,∞). Consequently, the O(1/s) in (6.120) holds uniformly inany compact set of γ’s contained in (γmin,∞). Expanding the expression at theright-hand side of (6.120), retaining only the terms O(1) and O(1/

ps), then

yields (6.50)–(6.51).In a similar fashion (6.52)–(6.53) is shown, although the computations are

rather involved. We must be a bit careful with T R2 because of the denominator

1 − γL that appears in (6.53). Recall that in the case that f (x) = 1, x ≥ 0,we have that 1 − γL = 0 = γL +M , and so T R

1 = T R2 = 0. In the case that

f (x0)< 1 for some x0 ≥ 0, it is easy to show from f (0) = 1, non-negativity anddecreasingness of f (x) that for any compact C ⊂ (γmin,∞)

(6.123) maxγ∈C

γL (γ)< 1.

This yields uniform validity of the O(1/sp

s) in (6.52) when γ is restricted to acompact subset of (γmin,∞).

6.E Proof of Proposition 6.7

We assume that a(x) is nonnegative and nondecreasing. Define

(6.124) a←(y) = sup x ≥ 0 | a(x)≤ y

162

6.E. Proof of Proposition 6.7

for y ≥ a(0). This generalized inverse function is continuous from the right at ally such that a←(y) is finite. Furthermore note that a(x)≤ y when x < a←(y).

Lemma 6.12. Let s = 1, 2, . . ., and denote for n= 1,2, . . .

Ss(n) =n∑

k=0

ln1+

1p

sak+ 1p

s

. (6.125)

Also, let δ ∈ (0,1/2). Then

0≤ Ss(n)−p

s

∫ n+1ps

0

ln1+

1p

sa(x)

dx ≤

12

sδ−12 . (6.126)

when n+ 1<p

sa←(sδ/2). Furthermore, define

A(x) =

∫ x

0

a2(u)du, x ≥ 0. (6.127)

Then, except in the trivial case a ≡ 0, A(x) is continuous and strictly increasingfrom 0 at x0 := sup x ≥ 0 | a(x) = 0 to∞ at x =∞. Furthermore,

0≤∫ n+1p

s

0

a(x)dx −p

s

∫ n+1ps

0

ln1+

1p

sa(x)

dx ≤

12

sδ−12 (6.128)

when n+ 1<p

sA←(s12−δ).

Proof. Since a(x) is nondecreasing in x ≥ 0, we have that Ss(n)/p

s is an upperRiemann sum for ∫ n+1p

s

0

ln1+

1p

sa(x)

dx , (6.129)

while

1p

sSs(n− 1) =

1p

sSs(n)−

1p

sln1+

1p

san+ 1p

s

(6.130)

is a lower Riemann sum for

∫ n+1ps

1ps

ln1+

1p

sa(x)

dx . (6.131)

163


It follows that

(6.132)

ps

∫ n+1ps

0

ln1+

1p

sa(x)

dx ≤ Ss(n)

≤p

s

∫ n+1ps

1ps

ln1+

1p

sa(x)

dx + ln

1+

1p

san+ 1p

s

≤p

s

∫ n+1ps

0

ln1+

1p

sa(x)

dx + ln

1+

1p

san+ 1p

s

,

where in the last inequality a(x)≥ 0 has been used. Now

(6.133) ln1+

1p

san+ 1p

s

≤

1p

san+ 1p

s

≤

12

sδ−12

when n+ 1<p

sa←(sδ/2). This yields (6.126).As for (6.128), we note that

(6.134) a(x)−1

2p

sa2(x)≤

ps ln

1+

1p

sa(x)

≤ a(x).

Hence,

0≤∫ n+1p

s

0

a(x)dx −p

s

∫ n+1ps

0

ln1+

a(x)p

s

dx ≤

∫ n+1ps

0

a2(x)2p

sdx ≤

s−δ

2

when n+ 1<p

sA←(s12−δ).

The proof of Proposition 6.7 follows now from Lemma 6.12 with δ = 1/4and taking ψ(s) =mina←(sδ/2), A←(s

12−δ).

164

CHAPTER 7Optimal Admission Control

for Many-Server Systemswith QED-Driven Revenues

by Jaron Sanders, Sem Borst,Guido Janssen, and Johan van Leeuwaarden

arXiv preprint, 1411.2808

165

7. OP T I M A L AD M I S S I O N CO N T R O L W I T H QED-DR I V E N RE V E N U E S

Abstract

We consider Markovian many-server systems with admissioncontrol operating in a Quality-and-Efficiency-Driven (QED) regime,where the relative utilization approaches unity while the number ofservers grows large, providing natural Economies-of-Scale. In orderto determine the optimal admission control policy, we adopt a rev-enue maximization framework, and suppose that the revenue rateattains a maximum when no customers are waiting and no serversare idling. When the revenue function scales properly with the sys-tem size, we show that a nondegenerate optimization problem arisesin the limit. Detailed analysis demonstrates that the revenue is max-imized by nontrivial policies that bar customers from entering whenthe queue length exceeds a certain threshold of the order of the typ-ical square-root level variation in the system occupancy. We identifya fundamental equation characterizing the optimal threshold, whichwe extensively leverage to provide broadly applicable upper/lowerbounds for the optimal threshold, establish its monotonicity, andexamine its asymptotic behavior, all for general revenue structures.For linear and exponential revenue structures, we present explicitexpressions for the optimal threshold.

166

7.1. Introduction

7.1 Introduction

Large-scale systems that operate in the Quality-and-Efficiency Driven (QED)regime dwarf the usual trade-off between high system utilization and shortwaiting times. In order to achieve these dual goals, the system is scaled so asto approach full utilization, while the number of servers grows simultaneouslylarge, rendering crucial Economies-of-Scale. Specifically, for a Markovian many-server system with Poisson arrival rate λ, exponential unit-mean service timesand s servers, the load ρ = λ/s is driven to unity in the QED regime in accor-dance with

(1−ρ)p

s→ γ, s→∞, (7.1)

for some fixed parameter γ ∈ R+. As s grows large, the stationary probabilityof delay then tends to a limit, say g(γ), which may take any value in (0,1),depending on the parameter γ. Since the conditional queue length distributionis geometric with mean ρ/(1−ρ)≈

ps/γ, it follows that the stationary mean

number of waiting customers scales as g(γ)p

s/γ. Little’s law then in turn impliesthat the mean stationary waiting time of a customer falls off as g(γ)/(γ

ps).

The QED scaling behavior also manifests itself in process level limits, wherethe evolution of the system occupancy, properly centered around s and normal-ized by

ps, converges to a diffusion process as s → ∞, which again is fully

characterized by the single parameter γ. This reflects that the system state typi-cally hovers around the full-occupancy level s, with natural fluctuations of theorder

ps.

The QED scaling laws provide a powerful framework for system dimension-ing, i.e., matching the service capacity and traffic demand so as to achieve acertain target performance level or optimize a certain cost metric. Suppose, forinstance, that the objective is to find the number of servers s for a given arrivalrate λ (or equivalently, determine what arrival rate λ can be supported with agiven number s of servers) such that a target delay probability ε ∈ (0,1) is at-tained. The above-mentioned convergence results for the delay probability thenprovide the natural guideline to match the service capacity and traffic volumein accordance with λ= s− γε

ps, where the value of γε is such that g(γε) = ε.

As an alternative objective, imagine we aim to strike a balance between theexpenses incurred for staffing servers and the dissatisfaction experienced bywaiting customers. Specifically, suppose a (salary) cost c is associated with eachserver per unit of time and a (possibly fictitious) holding charge h is imposed

167


for every waiting customer per unit of time. Writing λ= s− γp

s in accordancewith (7.1), and recalling that the mean number of waiting customers scales asg(γ)p

s/γ, we find that the total operating cost per time unit scales as

(7.2) cs+ hg(γ)p

sγ

= λc + cγp

s+ hg(γ)p

sγ

= λc +

cγ+ hg(γ)γ

ps.

This then suggests to set the number of servers in accordance with s = λ+γc,hp

s,where γc,h = argminγ>0(cγ+ hg(γ)/γ) in order to minimize the total operatingcost per time unit. Exploiting the powerful QED limit theorems, such convenientcapacity sizing rules can in fact be shown to achieve optimality in some suitableasymptotic sense.

As illustrated by the above two paragraphs, the QED scaling laws can beleveraged for the purpose of dimensioning, with the objective to balance theservice capacity and traffic demand so as to achieve a certain target performancestandard or optimize a certain cost criterion. A critical assumption, however, isthat all customers are admitted into the system and eventually served, whichmay in fact not be consistent with the relevant objective functions in the dimen-sioning, let alone be optimal in any sense.

Motivated by the latter observation, we focus in the present paper on theoptimal admission control problem for a given performance or cost criterion.Admission control acts on an operational time scale, with decisions occurringcontinuously whenever customers arrive, as opposed to capacity planning de-cisions which tend to involve longer time scales. Indeed, we assume that theservice capacity and traffic volume are given, and balanced in accordance with(7.1), but do allow for the value of γ to be negative, since admission controlprovides a mechanism to deal with overload conditions. While a negative valueof γ may not be a plausible outcome of a deliberate optimization process, inpractice an overload of that order might well result from typical forecast errors.

We formulate the admission control problem in a revenue maximizationframework, and suppose that revenue is generated at rate rs(k)when the systemoccupancy is k. As noted above, both from a customer satisfaction perspectiveand a system efficiency viewpoint, the ideal operating condition for the system isaround the full occupancy level s, where no customers are waiting and no serversare idling. Hence we assume that the function rs(k) is unimodal, increasing ink for k ≤ s and decreasing in k for k ≥ s, thus attaining its maximum at k = s.

168

7.1. Introduction

We consider probabilistic control policies, which admit arriving customerswith probability ps(k− s) when the system occupancy is k, independent of anyprior admission decisions. It is obviously advantageous to admit customers aslong as free servers are available, since it will not lead to any wait and drive thesystem closer to the ideal operating point s, boosting the instantaneous revenuerate. Thus we stipulate that ps(k− s) = 1 for all k < s.

For k ≥ s, it is far less evident whether to admit customers or not. Admit-ting a customer will then result in a wait and move the system away from theideal operating point, reducing the instantaneous revenue rate. On the otherhand, admitting a customer may prevent the system occupancy from falling be-low the ideal operating point in the future. The potential long-term gain mayoutweigh the adverse near-term effect, so there may be a net benefit, but theincentive weakens as the queue grows. The fundamental challenge in the de-sign of admission control policies is to find exactly the point where the marginalutility reaches zero, so as to strike the optimal balance between the conflictingnear-term and longer-term considerations.

Since the service capacity and traffic volume are governed by (7.1), the QEDscaling laws imply that, at least for γ > 0 and without any admission control,the system occupancy varies around the ideal operating point s, with typicaldeviations of the order

ps. It is therefore natural to suppose that the revenue

rates and admission probabilities scale in a consistent manner, and in the limitbehave as functions of the properly centered and normalized state variable(k − s)/

ps. Specifically, we assume that the revenue rates satisfy the scaling

conditionrs(k)− ns

qs→ r

k− sp

s

, s→∞, (7.3)

with ns a nominal revenue rate attained at the ideal operating point, qs a scalingcoefficient, and r a unimodal function, which represents the scaled reductionin revenue rate associated with deviations from the optimal operating point s.For example, with [x]+ =max0, x, any revenue structure of the form

rs(k) = ns −α−([s− k]+)β−−α+([k− s]+)β

+(7.4)

satisfies (7.3) when qs = smaxβ−,β+/2, in which case

r(x) = −α−([−x]+)β−1[β− ≥ β+]−α+([x]+)β

+1[β− ≤ β+]. (7.5)

169


Note that these revenue structures impose polynomial penalties on deviationsfrom the ideal operating point. Similar to (7.3), we assume that the admissionprobabilities satisfy a scaling condition, namely

(7.6) ps(0) · · · ps(k− s) = fk− sp

s

, k ≥ s,

with f a non-increasing function and f (0) = 1. In particular, we allow forf (x) = 1[0 ≤ x < η], which corresponds to an admission threshold controlps(k− s) = 1[k− s ≤ bη

psc].

In Section 7.2 we discuss the fact that the optimal admission policy is indeedsuch a threshold control, with the value of η asymptotically being determined bythe function r, which we later prove in Section 7.4. The optimality of a thresholdpolicy may not come as a surprise, and can in fact be established in the pre-limit (s <∞). However, the pre-limit optimality proof only yields the structuralproperty, and does not furnish any characterization of how the optimal thresholddepends on the system characteristics or provide any computational procedurefor actually obtaining the optimal value. In contrast, our asymptotic framework(as s→∞) produces a specific equation characterizing the optimal thresholdvalue, which does offer explicit insight in the dependence on the key systemparameters and can serve as a basis for an efficient numerical computation oreven a closed-form expression in certain cases. This is particularly valuablefor large-scale systems where a brute-force enumerative search procedure mayprove prohibitive.

Let us finally discuss the precise form of the revenue rates that serve as theobjective function that needs to be maximized by the optimal threshold. We willmostly focus on the average system-governed revenue rate defined as

(7.7) Rs(ps(k)k≥0) =∞∑k=0

rs(k)πs(k).

From the system’s perspective, this means that the revenue is simply governedby the state-dependent revenue rate rs(k) weighed according to the stationarydistribution, with πs(k) denoting the stationary probability of state k.

An alternative would be to consider the customer reward rate

(7.8) Rs(ps(k)k≥0) = λ∞∑k=0

rs(k)ps(k− s)πs(k).

170

7.1. Introduction

Here, rs(k) can be interpreted as the state-dependent reward when admitting acustomer in state k, and since this happens with probability ps(k−s) at intensityλ, we obtain (7.8). While this paper primarily focuses on (7.7), we show inSection 7.2.3 that there is an intimate connection with (7.8); a system-governedreward structure rs(k)k∈N0

can be translated into a customer reward structurers(k)k∈N0

, and vice versa.

7.1.1 Contributions and related literature

A diverse range of control problems have been considered in the queueing liter-ature, and we refer the reader to [124–128] for background. Threshold controlpolicies are found to be optimal in a variety of contexts such as [129–131],and many (implicit) characterizations of optimal threshold values have beenobtained in [132, 133], and [134]. For (single-server) queues in a conventionalheavy-traffic regime, optimality of threshold control policies has been estab-lished by studying limiting diffusion control problems in [135, 136], and [137].

The analysis of control problems in the QED regime has mostly focused onrouting and scheduling, see [138–141], and [142]. Threshold policies in thecontext of many-server systems in the QED regime have been considered in[109–111], and [112]. General admission control, however, has only receivedlimited attention in the QED regime, see for instance [143, 144]. These studiesspecifically account for abandonments, which create a trade-off between therejection of a new arrival and the risk of that arrival later abandoning withoutreceiving service, with the associated costly increase of server idleness.

In the present paper we address the optimal admission control problem froma revenue maximization perspective. We specifically build on the recent work in[6, Chapter 6] to show that a nondegenerate optimization problem arises in thelimit when the revenue function scales properly with the system size. Analysisof the latter problem shows that nontrivial threshold control policies are optimalin the QED regime for a broad class of revenue functions that peak around theideal operating point.

In Section 7.2 we present a fundamental equation which implicitly deter-mines the asymptotically optimal threshold. The subsequent analysis of thisequation in Section 7.3 yields valuable insight into the dependence of the op-timal threshold on the revenue structure, and provides a basis for an efficient

171


numerical scheme. Closed-form expressions for the optimal threshold can onlybe derived when considering specific revenue structures.

We will, for example, show that for linearly decreasing revenue rates, theoptimal threshold can be (explicitly) expressed in terms of the Lambert W func-tion [58]. We note that a linearly decreasing revenue structure has also beenconsidered in [134] for determining the optimal threshold kopt in an M/M/s/ksystem, and there also, kopt is expressed in terms of the Lambert W function.Besides assuming a static revenue and finite threshold k, a crucial differencebetween [134] and this paper is that our revenue structure scales as in (7.3),so that the threshold k is suitable for the QED regime. Our work thus extends[134], both in terms of scalable and more general revenue structures.

In terms of mathematical techniques, we use Euler–Maclaurin (EM) summa-tion [119] to analyze the asymptotic behavior of (7.7) as s→∞. This approachwas used recently for many-server systems with admission control in the QEDregime [6, Chapter 6], and is now extended by incorporating suitably scaledrevenue structures in Section 7.2. These ingredients then pave the way to de-termine the optimal admission control policy in the QED regime in Section 7.3.In Section 7.4, we use Hilbert-space theory from analysis, and techniques fromvariational calculus, to prove the existence of optimal control policies, and toestablish that control policies with an admission threshold which scales with thenatural

ps order of variation are optimal in the QED regime.

7.2 Revenue maximization framework

We now develop an asymptotic framework for determining an optimal admissioncontrol policy for a given performance or cost criterion. In Section 7.2.1 wedescribe the basic model for the system dynamics, which is an extension of theclassical M/M/s system. Specifically, the model incorporates admission controland is augmented with a revenue structure, which describes the revenue rateas a function of the system occupancy. Adopting this flexible apparatus, theproblem of finding an optimal admission control policy is formulated in termsof a revenue maximization objective.

172

7.2. Revenue maximization framework

7.2.1 Markovian many-server systems with admission control

Consider a system with s parallel servers where customers arrive according to aPoisson process with rate λ. Customers require exponentially distributed servicetimes with unit mean. A customer that finds upon arrival k customers in thesystem is taken into service immediately if k < s, or may join a queue of waitingcustomers if k ≥ s. If all servers are occupied, a newly arriving customer isadmitted into the system with probability ps(k−s), and denied access otherwise.We refer to the probabilities ps(k)k≥0 as the admission control policy. If wedenote the number of customers in the system at time t by Qs(t), and make theusual independence assumptions, then Qs(t)t≥0 constitutes a Markov process(see Figure 7.1 for its transition diagram). The stationary distribution πs(k) =limt→∞ P[Qs(t) = k] is given by

πs(k) =

(πs(0)

(sρ)k

k! , k = 1,2, . . . , s,

πs(0)ssρk

s!

∏k−s−1i=0 ps(i), k = s+ 1, s+ 2, . . . ,

(7.9)

with

ρ =λ

s, πs(0) =

s∑k=0

(sρ)k

k!+(sρ)s

s!Fs(ρ)

−1(7.10)

and

Fs(ρ) =∞∑n=0

ps(0) · · · ps(n)ρn+1. (7.11)

From (7.9)–(7.11), we see that the stationary distribution exists if and only ifthe relative load ρ and the admission control policy ps(k)k∈N0

are such thatFs(ρ)<∞ [6, Chapter 6], which always holds in case ρ < 1.

0 1

λ

1· · ·

λ

2s

λ

ss+ 1

ps(0)λ

s· · ·

ps(1)λ

s

Figure 7.1: Transition diagram of the process Qs(t)t≥0.

With k customers in the system, we assume that the system generates rev-enue at rate rs(k) ∈ R. We call rs(k)k≥0 the revenue structure. Our objective isto find an admission control policy in terms of the probabilities ps(k)k≥0 that

173


maximizes the average stationary revenue rate, i.e.

(7.12)to maximize Rs(ps(k)k≥0) over ps(k)k≥0,

subject to 0≤ ps(k)≤ 1, k ∈ N0, and Fs(ρ)<∞.

7.2.2 QED-driven asymptotic optimization framework

We now construct an asymptotic optimization framework where the limit laws ofthe Quality-and-Efficiency-Driven (QED) regime can be leveraged by imposingsuitable assumptions on the admission control policy and revenue structure. Inorder for the system to operate in the QED regime, we couple the arrival rateto the number of servers as

(7.13) λ= s− γp

s, γ ∈ R.

For the admission control policy we assume the form in (7.6), with f eithera nonincreasing, bounded, and twice differentiable continuous function, or astep function, which we will refer to as the asymptotic admission control profile.We also assume the revenue structure has the scaling property (7.3), with ra piecewise bounded, twice differentiable continuous function with boundedderivatives. We will refer to r as the asymptotic revenue profile. These assump-tions allow us to establish Proposition 7.1 by considering the stationary averagerevenue rate Rs(ps(k)k≥0) as a Riemann sum and using Euler–Maclaurin (EM)summation to identify its limiting integral expression, the proof of which can befound in Appendix 7.A. Let φ(x) = exp (− 1

2 x2)/p

2π and Φ(x) =∫ x

−∞φ(u)dudenote the probability density function and cumulative distribution function ofthe standard normal distribution, respectively.

Proposition 7.1. If r(i) is continuous and bounded for i = 0, 1, 2, and either (i)f is smooth, and ( f (x)exp (−γx))(i) is exponentially small as x →∞ for i = 0,1, 2, or (ii) f (x) = 1[0≤ x < η] with a fixed, finite η > 0, then

(7.14) lims→∞

Rs(ps(k)k≥0)− ns

qs= R( f ),

where in case (i)

(7.15) R( f ) =

∫ 0

−∞ r(x)e−12 x2−γx dx +

∫∞0 r(x) f (x)e−γx dx

Φ(γ)φ(γ) +


,

174


and in case (ii)

R(1[0≤ x < η]) =

∫ 0

−∞ r(x)e−12 x2−γx dx +

∫ η0 r(x)e−γx dx

Φ(γ)φ(γ) +

1−e−γηγ

, η≥ 0. (7.16)

Because of the importance of the threshold policy, we will henceforth usethe short-hand notations RT,s(τ) = Rs(1[k ≤ s+τ]k≥0) and RT(η) = R(1[0≤x < η]) to indicate threshold policies.

Example 7.2 (Exponential revenue). Consider a revenue structure rs(k) =exp (b(k− s)/

ps) for k < s and rs(k) = exp (−d(k− s)/

ps) for k ≥ s, and

with b, d > 0. Taking ns = 0 and qs = 1, the asymptotic revenue profile isr(x) = exp (bx) for x < 0 and r(x) = exp (−d x) for x ≥ 0, so that according toProposition 7.1 for threshold policies,

lims→∞

RT,s(bηp

sc) = RT(η) =Φ(γ−b)φ(γ−b) +

1−e−(d+γ)η

d+γ

Φ(γ)φ(γ) +

1−e−γηγ

. (7.17)

Figure 7.2 plots RT,s(bηp

sc) for a finite system with s = 8, 32, 128, 256servers, respectively, together with its limit RT(η). Here, we set b = 5, d = 1,and γ= 0.01. Note that the approximation RT,s(bη

psc)≈ RT(η) is remarkably

accurate, even for relatively small systems, an observation which in fact seemsto hold for most revenue structures and parameter choices. For this particularrevenue structure, we see that the average revenue rate peaks around ηopt ≈ 1.0.In Example 7.4, we confirm this observation by determining ηopt numerically.

An alternative way of establishing that the limit of Rs(ps(k)k≥0) is R( f ),is by exploiting the stochastic-process limit for Qs(t)t≥0. It was shown in [6,Chapter 6] that under condition (i) in Proposition 7.1, together with (7.6) and(7.13), the normalized process Qs(t)t≥0 with Qs(t) = (Qs(t)−s)/

ps converges

weakly to a stochastic-process limit D(t)t≥0 with stationary density

w(x) =

(Z−1e−

12 x2−γx , x < 0,

Z−1 f (x)e−γx , x ≥ 0,(7.18)

where Z = Φ(γ)/φ(γ) +∫∞

0 f (x)exp (−γx)dx . When additionally assuming(7.3), the limiting system revenue can be written as

R( f ) =

∫ ∞

−∞r(x)w(x)dx , (7.19)

175


0

0.2

0.4

RT,8(bηp

8c)RT(η)

RT,32(b4ηp

2c)RT(η)

0 2 4 6 8 100

0.2

0.4

RT,128(b8ηp

2c)RT(η)

η0 2 4 6 8 10

RT,512(b16ηp

2c)RT(η)

η

Figure 7.2: RT,s(bηp

sc) and RT(η) for s = 8, 32, 128, 256 servers.

the stationary revenue rate generated by the stochastic-process limit. So analternative method to prove Proposition 7.1 would be to first formally establishweak convergence at the process level, then prove that limits with respect tospace and time can be interchanged, and finally use the stationary behavior of thestochastic-process limit. This is a common approach in the QED literature ([52,102]). Instead, we construct a direct, purely analytic proof, that additionallygives insight into the error that is made when approximating Rs(ps(k)k≥0) byR( f ) for finite s. These error estimates are available in Appendix 7.A for futurereference.

With Proposition 7.1 at hand, we are naturally led to consider the asymptoticoptimization problem, namely,(7.20)

to maximize R( f ) over f ,subject to 0≤ f (x)≤ 1, x ∈ [0,∞), and

∫∞0 f (x)e−γx dx <∞.

The condition∫∞

0 f (x)e−γx dx <∞ is the limiting form of the stability condi-tion Fs(ρ) <∞, see [6, Chapter 6]. Also note that we do not restrict f to bemonotone. We prove for the optimization problem in (7.20) the following inSection 7.4.

176


Proposition 7.3. If r is nonincreasing for x ≥ 0, then there exist optimal asymp-totic admission controls that solve (7.20). Moreover, the optimal asymptotic ad-mission control profiles have a threshold structure of the form

f (x) = 1[0≤ x < ηopt], (7.21)

where ηopt is any solution ofr(η) = RT(η) (7.22)

if r(0) > RT(0), and ηopt = 0 if r(0) ≤ RT(0). If r is strictly decreasing in x ≥ 0,then ηopt is unique.

Recall that the optimality of a threshold policy should not come as a surprise,and could in fact be shown in the pre-limit and within a far wider class ofpolicies than those satisfying (7.6). The strength of Proposition 7.3 lies in thecharacterization (7.22) ofηopt. We refer to (7.22) as the threshold equation: it is apowerful basis on which to obtain numerical solutions, closed-form expressions,bounds, and asymptotic expansions for ηopt. Results for ηopt of this nature arepresented in Section 7.3.

Example 7.4 (Exponential revenue revisited). Let us revisit Example 7.2, wherer(x) = exp (bx) for x < 0 and r(x) = exp (−d x) for x ≥ 0. The thresholdequation, (7.22), takes the form

e−dηΦ(γ)φ(γ)

+1− e−γη

γ

=Φ(γ− b)φ(γ− b)

+1− e−(d+γ)η

d + γ, (7.23)

which we study in depth in Section 7.3.4. When b = 5, d = 1, and γ = 0.01,solving (7.23) numerically yields ηopt ≈ 1.00985, which supports our earlierobservation that ηopt ≈ 1 in Example 7.2.

The true optimal admission threshold τopt = arg maxτ∈N0RT,s(τ) is plotted

in Figure 7.3 as a function of s, along with the asymptotic QED approximationτopt ≈ bηoptpsc. We observe that the QED approximation is accurate, even fora relatively small number of servers, and exact in the majority of cases. This isreflected in Figure 7.4, which plots the relative optimality gap as a function of s.The relative optimality gap is zero for the vast majority of s values, and as lowas 10−2 for systems with as few as 10 servers.

We remark that when utilizing the asymptotic optimal threshold provided byProposition 7.3 in a finite system, the proof of Proposition 7.1 in Appendix 7.A

177


1 50 100 150 200 250

0

5

10

15

τopt, bηoptpsc

s

Figure 7.3: The true optimal admission threshold τopt as a function of s, togetherwith the (almost indistinguishable) QED approximation bηoptpsc.

100 101 102

10−6

10−4

10−2

100

RT,s(τopt)−RT,s(bηoptpsc)RT,s(τopt)

s

Figure 7.4: The relative error (RT,s(τopt)−RT,s(bηoptpsc))/RT,s(τopt) as a functionof s. The missing points indicate an error that is strictly zero. The errors thatare non-zero arise due to the QED approximation for the optimal admissionthreshold being off by just one state.

178


guarantees that Rs,T(bηoptpsc)− RT(ηopt) = O(1/p

s). In other words, a finitesystem that utilizes the asymptotic optimal threshold achieves a revenue withinO(1/

ps) of the solution to (7.20).

7.2.3 Customer reward maximization

In Section 7.1 we have discussed the difference between revenues seen fromthe system’s perspective and from the customer’s perspective. Although theemphasis lies on the system’s perspective, as in Section 7.2.2, we now show howresults for the customer’s perspective can be obtained.

Linear revenue structure

If revenue is generated at rate a > 0 for each customer that is being served, andcost is incurred at rate b > 0 for each customer that is waiting for service, therevenue structure is given by

rs(k) =

(ak, k ≤ s,

as− b(k− s), k ≥ s.(7.24)

When ns = as and qs =p

s, the revenue structure in (7.24) satisfies the scalingcondition in (7.3), with

r(x) =

(ax , x ≤ 0,

−bx , x ≥ 0.(7.25)

Consequently, Proposition 7.1 implies that

lims→∞

Rs(ps(k)k≥0)− asp

s=

a1+ γ Φ(γ)φ(γ)

− b

∫∞0 x f (x)e−γx dx

Φ(γ)φ(γ) +


, (7.26)

for any profile f , and Proposition 7.3 reveals that the optimal control is f (x) =1[0≤ x < ηopt] with ηopt the unique solution of the threshold equation (7.22),which with c = a/b becomes

ηΦ(γ)φ(γ)

+1− e−γη

γ

= c

1+ γ

Φ(γ)φ(γ)

+

1− (1+ γη)e−γη

γ2. (7.27)

179


We see that ηopt depends only on c. The threshold equation (7.27) is studiedextensively in Section 7.3.4. A minor variation of the arguments used there toprove Proposition 7.11, shows that

(7.28) ηopt = r0 +1γ

Wγe−γr0

a0

,

where W denotes the Lambert W function, and

(7.29) a0 = −γ− γ2 Φ(γ)φ(γ)

, r0 = cγ+1

γ+ γ2 Φ(γ)φ(γ)

.

Relating system-governed revenue to customer rewards

From (7.28), it can be deduced that for large values of γ, ηopt ≈ cγ (see theproof of Proposition 7.13, and for a discussion on the asymptotic behavior of thethreshold equation for general revenue structures, we refer to Section 7.3.3).Thus, asymptotically, the optimal threshold value is approximately equal to theproduct of the staffing slack γ and the ratio of the service revenue a and thewaiting cost b.

The asymptotic behavior ηopt ≈ cγ may be explained as follows. For eacharriving customer, we must balance the expected revenue a when that customeris admitted and eventually served against the expected waiting cost incurredfor that customer as well as the additional waiting cost for customers arrivingafter that customer. When the arriving customer finds τ customers waiting, theoverall waiting cost incurred by admitting that customer may be shown to behaveroughly as bτ/(γ

ps) for large values of γ. Equating a with bτ/(γ

ps) then yields

that the optimal threshold value should approximately be τopt ≈ cγp

s.

The stationary average revenue rate Rs(ps(k)k≥0) under revenue structure(7.24) is therefore the same as when a reward a > 0 is received for each admittedcustomer and a penalty bE[W ] is charged when the expected waiting time ofthat customer is E[W ], with b > 0. In the latter case the stationary averagereward earned may be expressed as in (7.8), where now

(7.30) rs(k) = a− b max¦

0,k− s+ 1

s

©

denotes a customer reward.

180


The system-governed revenue rate and the customer reward rate are in thiscase equivalent. To see this, write

Rs(ps(k)k≥0)

= aλ s−1∑

k=0

πs(k) +∞∑k=s

ps(k− s)πs(k)− bλ

∞∑k=s

k− s+ 1s

ps(k− s)πs(k).

(7.31)Then note that because the arrival rate multiplied by the probability that anarriving customer is admitted must equal the expected number of busy servers,and by local balance λπs(k)ps(k− s) = sπs(k+ 1) for k = s, s+ 1, . . . , we have

Rs(ps(k)k≥0) = a s−1∑

k=0

kπs(k) +∞∑k=s

sπs(k)− b

∞∑k=s+1

(k− s)πs(k)

=s−1∑k=0

akπs(k) +∞∑k=s

(as− b(k− s))πs(k)(7.24)=

∞∑k=0

rs(k)πs(k) = Rs(ps(k)k≥0).

(7.32)The optimal threshold in (7.28) thus maximizes the customer reward rate Rs

asymptotically as well, i.e., in this example by Proposition 7.3,

lims→∞

Rs(ps(k)k≥0)− ap

s= lim

s→∞

Rs(ps(k)k≥0)− ap

s= R( f ). (7.33)

In fact, for any system-governed revenue rate rs(k), the related customerreward structure rs(k) is given by

rs(k) =rs(k+ 1)

mink+ 1, s, k ∈ N0, (7.34)

because then

Rs(ps(k)k≥0) =s−1∑k=0

rs(k)λπs(k) +∞∑k=s

rs(k)λps(k− s)πs(k)

=s−1∑k=0

rs(k)(k+ 1)πs(k+ 1) +∞∑k=s

rs(k)sπs(k+ 1)

=s∑

k=0

rs(k− 1)kπs(k) +∞∑

k=s+1

rs(k− 1)sπs(k) = Rs(ps(k)k≥0),

(7.35)

using local balance, λπs(k) = (k+1)πs(k+1) for k = 0, 1, . . . , s−1 and λps(k−s)πs(k) = sπs(k+ 1) for k = s, s+ 1, . . ..

181


Proposition 7.5. For any system-governed revenue rate rs(k), the customer rewardstructure rs(k) in (7.34) guarantees that the average system-governed revenue rateRs(ps(k)k≥0) equals the customer reward rate Rs(ps(k)k≥0).

In particular, Proposition 7.5 implies that counterparts to Proposition 7.1and Proposition 7.3 hold for the customer reward rate Rs(ps(k)k≥0), assumingthe customer reward structure rs(k) is appropriately scaled.

7.3 Properties of the optimal threshold

We focus throughout this paper on maximization of the average system-governedrevenue rate. In Section 7.2 we have established threshold optimality and de-rived the threshold equation that defines the optimal threshold ηopt. In thissection we obtain a series of results about ηopt. In Section 7.3.1, we present aprocedure (for general revenue functions) to obtain an upper bound ηmax anda lower bound ηmin on the optimal threshold ηopt. Section 7.3.2 discusses ourmonotonicity results. Specifically, we prove that ηopt increases with γ ∈ R, andthat RT(0), RT(ηopt) both decrease with γ ∈ R. In Section 7.3.3, we derive asymp-totic descriptions of the optimal threshold for general revenue structures, even ifthe revenue structures would not allow for an explicit characterization. We provethatηopt ≈ r←(r(−γ)) as γ→∞, and thatηopt ≈ −(1/γ) ln (1− r ′(0−)/r ′(0+))as γ → −∞. In Section 7.3.4, we derive explicit characterizations of ηopt forlinear and exponential revenue structures.

From here on, we assume that r(x) is piecewise smooth and bounded on(−∞, 0) and (0,∞), and continuous at 0 with r(±0) = 1 = r(0). We alsoassume that r(x) is increasing on (−∞, 0] and decreasing on [0,∞), with0≤ r(x)≤ r(0) = 1. Revenue functions for which r(0)> 0 and r(0) 6= 1 can beconsidered through analysis of the scaled revenue function r(x) = r(x)/r(0).For notational convenience, we also define rL(x) and rR(x) as

(7.36) r(x) =

(rL(x), x < 0,

rR(x), x ≥ 0,

and introduce A=∫ 0

−∞ rL(x)e−12 x2−γx dx , and B = Φ(γ)/φ(γ). Note that RT(0)

= A/B.

182

7.3. Properties of the optimal threshold

Corollary 7.6. Under these assumptions, there exists a solution ηopt > 0 of thethreshold equation. This solution is positive, unless rL(x) = 1, and unique whenr ′R(x)< 0 for all x ≥ 0 such that rR(x)> 0.

Proof. Note that these assumptions on r are slightly stronger than in Proposi-tion 7.3. This corollary is directly implied by our proof of Proposition 7.3; seethe explanation between (7.100) and (7.101).

7.3.1 General bounds

Denote the inverse function of r by r←. The following bounds hold for generalrevenue structures, are readily calculated, and provide insight into the optimalthresholds. Later in Section 7.3.4 and Section 7.3.4, we illustrate Proposition 7.7for a linear revenue structure, and an exponential revenue structure, respectively.

Proposition 7.7. When r is strictly decreasing for x ≥ 0, ηmax = r←R (Rlower) ≥ηopt, and ηmin = r←R (Rupper)< ηopt. Here,

Rlower = RT(0) =

∫ 0

−∞ r(x)e−12 x2−γx dx

Φ(γ)φ(γ)

,

Rupper =

∫ 0

−∞ r(x)e−12 x2−γx dx +

∫ ηmax

0 e−γx dxΦ(γ)φ(γ) +

∫ ηmax

0 e−γx dx.

(7.37)

Proof. The assumptions on rR(x) imply that its inverse function r←R (y) exists,and that it is also strictly decreasing. It is therefore sufficient to provide upperand lower bounds on RT(η) that are independent of η.

For threshold control policies, the system revenue is given by (7.16). Also re-call that the optimal threshold ηopt solves the threshold equation, i.e., rR(ηopt) =RT(ηopt). By suboptimality, we immediately obtain Rlower ≤ RT(ηopt), and soηopt ≤ ηmax by monotonicity.

We will first derive an alternative forms of the threshold equation. For in-stance, rewriting (7.22) into

B +

∫ η

0

e−γx dxr(η) = A+

∫ η

0

r(x)e−γx dx , (7.38)

183


dividing by B, and using RT(0) = A/B gives

(7.39) r(η)− RT(0) = −r(η)

B

∫ η

0

e−γx dx +

∫ η

0

r(x)e−γx dx .

We then identify the right-hand member as being a result of an integration byparts, to arrive at the alternative form

(7.40)

r(η)− RT(0) = − r(x)

B

∫ x

0

e−γu duη

0+

∫ η

0

e−γx dx

= −∫ η

0

r ′(x)B

∫ x

0

e−γu du dx .

Let c(η) =∫ η

0 e−γx dx = (1− e−γη)/γ if γ 6= 0 and c(η) = η if γ = 0. Sincec(η) is increasing in η and −r ′(x)≥ 0 for x ≥ 0, we have for η≥ 0

(7.41) −1B

∫ η

0

r ′(x)c(x)dx < −1B

c(η)

∫ η

0

r ′(x)dx =1B

c(η)(1− r(η)).

Let η= η be the (unique) solution of the equation

(7.42) r(η)− RT(0) =1B

c(η)(1− r(η)).

Then

(7.43) r(η)− RT(0) =1B

c(η)(1− r(η))> −1B

∫ η

0

r ′(x)c(x)dx ,

and so 0< η < ηopt. We have from (7.42) that

(7.44) r(η) =RT(0) +

1B c(η)

1+ 1B c(η)

=A+ c(η)B + c(η)

.

From η < ηopt < ηmax = r←(RT(0)), we then find

(7.45) c(η)< c(ηmax),A+ c(η)B + c(η)

<A+ c(ηmax)B + c(ηmax)

= Rupper,

since 0< A< B, i.e. RT(0)< 1. This completes the proof.

184


7.3.2 Monotonicity

We next investigate the influence of the slack γ on the optimal threshold.

Proposition 7.8. The revenue RT(0) decreases in γ ∈ R.

Proof. Write r(−x) = u(x) so that 0≤ u(x)≤ 1= u(0) and

RT(0) =

∫∞0 u(x)e−

12 x2+γx dx

∫∞0 e−

12 x2+γx dx

, γ ∈ R, (7.46)

and calculate

dRT(0)dγ

=

∫∞0 e−

12 x2+γx dx

∫∞0 xu(x)e−

12 x2+γx dx

∫∞0 e−

12 x2+γx dx

2

−

∫∞0 u(x)e−

12 x2+γx dx

∫∞0 xe−

12 x2+γx dx

∫∞0 e−

12 x2+γx dx

2 .

(7.47)

The numerator can be written as

N =

∫ ∞

0

∫ ∞

0

(x − y)u(x)e−12 x2+γxe−

12 y2+γy dx dy. (7.48)

Suppose that u(x) = 1[0≤ x < a] for some a > 0. Then

N =

∫ a

0

∫ ∞

0

(x − y)e−12 x2+γxe−

12 y2+γy dx dy

≤∫ a

0

∫ a

0

(x − y)e−12 x2+γxe−

12 y2+γy dx dy = 0.

(7.49)

In general, we can write u(x) = −∫∞

0 1[0 ≤ x < a]u′(a)da = −∫∞

x u′(a)dawith u′(a)< 0, to arrive at

N =

∫ ∞

0

−u′(a)∫ ∞

0

∫ ∞

0

(x − y)1[0≤ x < a]e−12 x2+γxe−

12 y2+γy dx dy

da

≤ 0.(7.50)

This concludes the proof.

185


Proposition 7.9. The optimal threshold ηopt increases in γ ∈ R, and RT(ηopt)decreases in γ ∈ R.

Proof. By Proposition 7.8, we have that RT(0) decreases in γ ∈ R. Furthermore,for any η > 0, we have that

∫ η0 e−γx dx/B decreases in γ ∈ R. Consider the

alternative form (7.40) of the threshold equation. For fixed η, the left memberthus increases in γ, while the right member decreases in γ, since r ′(x)< 0. Thesolution ηopt of the threshold equation therefore increases in γ ∈ R.

To prove the second part of the claim, we recall that ηopt solves the thresholdequation, so RT(ηopt) = rR(ηopt). Since ηopt ≥ 0 is increasing in γ ∈ R, ourassumptions on r imply that rR(ηopt) is decreasing in γ ∈ R. Hence, RT(ηopt) isdecreasing in γ ∈ R as well.

Proposition 7.9 can be interpreted as follows. First note that an increase inγ means that fewer customers are served by the system, apart from the impactof a possible admission control policy. Then, for threshold control, an increasedγ implies that the optimal threshold should increase, in order to serve more cus-tomers. This of course is a direct consequence of our revenue structure, whichis designed to let the system operate close to the ideal operating point. A largeγ drifts the process away from this ideal operating point, and this can be com-pensated for by a large threshold ηopt. Hence, although the slack γ and thethreshold ηopt have quite different impacts on the system behavior, at a highlevel their monotonic relation can be understood, and underlines that the rev-enue structure introduced in this paper has the right properties for the QEDregime.

7.3.3 Asymptotic solutions

We now present asymptotic results for the optimal threshold in the regimeswhere the slack γ becomes extremely large or extremely small.

Proposition 7.10. When γ → −∞, and if the revenue function has a cusp atx = 0, i.e., r ′R(0+)< 0< r ′L(0−), the optimal threshold is given by

(7.51) ηopt = −1γ

ln1−

r ′L(0−)r ′R(0+)

+O

1γ2

.

186


Proof. We consider γ → −∞. From steepest descent analysis, we have for asmooth and bounded f on (−∞, 0],

∫ 0

−∞f (x)e−γx dx = −

f (0)γ−

f ′(0)γ2

+O 1γ3

, γ→−∞. (7.52)

Hence, it follows that

RT(0) =AB=− 1γ −

r ′L(0−)γ2 +O

1γ3

− 1γ +O

1γ3

= 1+r ′L(0−)γ

+O 1γ2

, γ→−∞. (7.53)

From the upper bound ηopt < r←(RT(0)) and r(0) = 1, r ′R(0+)< 0, we thus seethat ηopt = O(1/|γ|), γ → −∞, and so in the threshold equation, see (7.40),we only need to consider η’s of O(1/|γ|). In (7.40), we have

∫ x

0 exp (−γu)du=(1− exp (−γx))/γ. Using that 1/(γB) = 1+O(1/γ2), see (7.53), we get for theright-hand side of (7.40),

−1B

∫ η

0

r ′R(x)

∫ x

0

e−γu du dx =

∫ η

0

r ′R(x)(1− e−γx dx)dx1+O

1γ2

. (7.54)

Next,

r ′R(x) = r ′R(0+)+O1γ

, 1− e−γx = O(1), 0≤ x ≤ η= O

1γ

, (7.55)

and so

−1B

∫ η

0

r ′R(x)

∫ x

0

e−γu du dx = −r ′R(0+)1− e−γη − γη

γ+O

1γ2

. (7.56)

Furthermore, for the left-hand side of (7.40), we have

rR(η)− RT(0) = 1+ r ′R(0+)η+O 1γ2

−1+

r ′L(0−)γ

+O 1γ2

= r ′R(0+)η−r ′L(0−)γ

+O 1γ2

.

(7.57)

Equating (7.56) and (7.57) and simplifying, we find

r ′R(0+)(1− e−γη) = r ′L(0−) +O1γ

, (7.58)

and this gives (7.51).

187


If rL(x) is slowly varying, the optimal threshold is approximately given by

(7.59) ηopt ≈ r←R (rL(−γ))

as γ→∞. To see this, note that as γ→∞,(7.60)

RT(0) =

∫ 0

−∞ rL(x)e−12 x2−γx dx

∫ 0

−∞ e−12 x2−γx dx

=e

12 γ

2 ∫∞0 rL(−x)e−

12 (x−γ)

2dx

e12 γ

2∫∞

0 e−12 (x−γ)2 dx

≈ rL(−γ).

A full analysis of this result would be beyond the scope of this paper, and overlycomplicate our exposition. Instead, consider as an example rL(x) = exp (bx)with b > 0 small. We have as γ→∞ with exponentially small error(7.61)

RT(0) =Φ(γ− b)φ(γ− b)

·φ(γ)Φ(γ)

= e−bγeb2

2

1− b

φ(γ)Φ(γ)

+O(b2)= rL(−γ)(1+O(b2)).

When for instance rR(x) = exp (d x) with d > 0, we get that ηopt ≈ r←R (exp (−bγ+ b2/2)) = (b/d)γ− b2/(2d) with exponentially small error, as γ→∞. Further-more, the right-hand side in (7.40) is exponentially small as γ→∞, so that ingood approximation the solution to the threshold equation is indeed given by(7.59).

7.3.4 Explicit results for two special cases

We now study the two special cases of linear and exponential revenue structures.For these cases we are able to find precise results for the ηopt. We demonstratethese results for some example systems, and also include the bounds and asymp-totic results obtained in Section 7.3.1 and Section 7.3.3, respectively.

Linear revenue

We first present an explicit expression for the optimal threshold for the case ofa linear revenue function,

(7.62) rR(x) =1−

xd

1[0≤ x ≤ d], x ≥ 0,

and arbitrary rL(x). We distinguish between γ 6= 0 and γ= 0 in Proposition 7.11and Proposition 7.12 below.

188


Proposition 7.11. Assume γ 6= 0. Then

ηopt = r0 +1γ

Wγe−γr0

a0

, (7.63)

where W denotes Lambert’s W function, and

a0 = −γ2B +

1γ

, r0 =

d(B − A) + 1γ2

B + 1γ

. (7.64)

Proof. It follows from [145, Section 4] that B + 1/γ 6= 0 when γ 6= 0 so thata0, r0 in (7.64) are well-defined with a0 6= 0. From the threshold equation in(7.40), and rR(η) = 1−η/d when 0≤ η≤ d, we see that η= ηopt satisfies

1−η

d−

AB=

1d

∫ η

0

∫ x

0

e−γu du dx . (7.65)

Now ∫ η

0

∫ x

0

e−γu du dx =1γ

η−

1− e−γη

γ

, (7.66)

and this yields for η= ηopt the equation

γ(η− r0)eγ(η−r0) =

γ

a0e−γr0 (7.67)

with a0 and r0 given in (7.64). Note that equation (7.67) is of the form W (z)×exp (W (z)) = z, which is the defining equation for Lambert’s W function, andthis yields the result.

Proposition 7.11 provides a connection with the developments in [134].Furthermore, the optimal threshold ηopt is readily computed from it, taking carethat the branch choice for W is such that the resulting ηopt is positive, continuous,and increasing as a function of γ. For this matter, the following result is relevant.

Proposition 7.12. For rR(x) = 1−x/d with d > 0, and arbitrary rL(x), as γ→ 0,

ηopt =

√√√π2+ 2d

sπ2−∫ 0

−∞rL(x)e−

12 x dx

−sπ

2+O(γ). (7.68)

189


Proof. In the threshold equation in (7.40), we set ε = 1− A/B and use rR(x) =1− x/d, r ′R(x) = −1/d, to arrive at

(7.69) dε −η=1B

∫ η

0

∫ x

0

e−γu du dx .

Since

(7.70)

∫ η

0

∫ x

0

e−γu du dx =

∫ η

0

(x +O(γx2))dx =12η2 +O(γη3),

we obtain the equation

(7.71) dε −η=1

2Bη2 +O(γη3).

Using that ηopt < r←(A/B) = O(1) as γ→ 0, we find from (7.71) that as γ→ 0,

(7.72) ηopt =p

B2 + 2Bdε − B +O(γ) =Æ

B2 + 2d(B − A)− B +O(γ).

Finally (7.68) follows from the expansions

(7.73) B =sπ

2+O(γ), A=

∫ 0

−∞rL(x)e

− 12 x2

dx +O(γ),

as γ→ 0.

We may also study the regime γ→∞. Note that the following result coin-cides with the asymptotic behavior of ηmin and ηmax in Proposition 7.7.

Proposition 7.13. For rL(x) = ebx , and rR(x) = (d − x)/d, as γ→∞,

(7.74) ηopt = d1−Φ(γ− b)φ(γ)φ(γ− b)Φ(γ)

+O

1γ

e−12 γ

2.

Proof. The revenue structure implies that

(7.75) A=Φ(γ− b)φ(γ− b)

, B =Φ(γ)φ(γ)

,

∫ x

0

e−γu du=1− e−γx

γ.

Therefore, as γ→∞,

(7.76)AB=Φ(γ− b)φ(γ)φ(γ− b)Φ(γ)

,1B= O(e−

12 γ

2),

∫ x

0

e−γu du= O1γ

.

190


Substituting in the threshold equation (7.40), we find that as γ→∞,

1−η

d=Φ(γ− b)φ(γ)φ(γ− b)Φ(γ)

+O1γ

e−12 γ

2, (7.77)

which completes the proof.

Figure 7.5 displays ηopt given in Proposition 7.11 as a function of γ, togetherwith the bounds given by Proposition 7.7,

ηmax = d1−Φ(γ− b)φ(γ)φ(γ− b)Φ(γ)

,

ηmin = d1−Φ(γ− b)/φ(γ− b) +

∫ ηmax

0 e−γx dx

Φ(γ)/φ(γ) +∫ ηmax

0 e−γx dx

,

(7.78)

and asymptotic solutions of Proposition 7.10,

ηγ→−∞ = −1γ

ln (1+ bd), ηγ→∞ = d(1− e−bγ). (7.79)

Figure 7.5 also confirms the monotonicity of ηopt in γ established in Proposi-tion 7.9. Note also the different regimes in which our approximations are valid,and that the bounds of Proposition 7.7 are tight as γ→±∞.

Exponential revenue

Consider rL(x) arbitrary, and let rR(x) = exp (−δx) for x ≥ 0, with δ > 0. First,we will consider what happens asymptotically as δ ↓ 0 in the case γ= 0, whichshould be comparable to the case in Proposition 7.12. Then, we consider thecase γ= −δ, which like the linear revenue structure has a Lambert W solution.Finally, we consider what happens asymptotically when ε = 1− RT(x) > 0 issmall, and we check our results in the specific cases γ= −2δ, −δ/2 and γ= δ,which have explicit solutions.

Proposition 7.14. For γ= 0, as δ ↓ 0,

ηopt =

√√2(B − A)δ

−2A+ B

3+O

pδ

=

√√√ 2δ

sπ2−∫ 0

−∞r(x)e−

12 x2 dx

+O(1).

(7.80)

191


−5 −4 −3 −2 −1 0 1 2 3 4 50

1

ηmin

ηmax

ηγ→−∞ηγ→∞

γ

Figure 7.5: The optimal threshold ηopt, its bounds ηmin, ηmax, and its approxima-tions ηγ→±∞, all as a function of γ, when rL(x) = exp (bx), rR(x) = (d − x)/d,and b = d = 1. The curve for the optimal threshold has been produced with(7.63).

Proof. When γ= 0, the threshold equation reads

(7.81) eδη = 1+δ(B − A)1+ Aδ

+δη

1+ Aδ,

which follows from (7.38) with γ = 0. With δ > 0, η > 0, the left-hand sideof (7.81) exceeds 1 + δη + δ2η2/2, while the right-hand side is exceeded by1 + δη + δ(B − A). Therefore, the left-hand side of (7.81) exceeds the right-hand side if η > η∗, where η∗ =

p2(B − A)/δ. This implies that ηopt ≤ η∗, and

so we restrict attention to 0 ≤ η ≤ η∗ = O(1/pδ) when considering (7.81).

Expanding both sides of (7.81) gives

(7.82)1+δη+

12δ2η2 +

16δ3η3 +O(δ4η4)

= 1+δ(B − A)−δ2A(B − A) +O(δ3) +δη−δ2Aη+O(δ3η).

Cancelling the terms 1+δη at both sides of (7.82), and dividing by δ2/2 whileremembering that η= O(1/

pδ), we get

(7.83) η2 =2(B − A)δ

− 2ηA−13η3δ+O(1).

192


Therefore,

η= η∗1−

AδB − A

η−δ2

6(B − A)η3 +O(δ)

12 = η∗(1+O(

pδ)). (7.84)

Thus η = η∗ + O(1), and inserting this in the right-hand side of the middlemember in (7.84) yields

η= η∗1−

AδB − A

η∗ −δ2

6(B − A)η3∗ +O(δ)

12

= η∗ −Aδ

2(B − A)η2∗ −

δ2

12(B − A)η4∗ +O(

pδ)

= η∗ −2A+ B

3+O(

pδ),

(7.85)

which is the result (7.80).

Figure 7.6 draws for γ = 0 a comparison between rR(x) = exp (−∆x) andrR(x) = 1 −∆x . As expected, we see agreement when ∆ ↓ 0, and for larger∆ the exponential revenue leads to slightly larger ηopt compared with linearrevenues.

When γ= −δ, the threshold equation becomes

e−δη − RT(0) =φ(−δ)Φ(−δ)

η−

1− e−δη

δ

, (7.86)

or equivalently,

e−δη =1

Φ(−δ)φ(−δ) −

1δ

η−

1δ+Φ(−δ)φ(−δ)

RT(0), (7.87)

and the solution may again be expressed in terms of the Lambert W function.

Proposition 7.15. When γ= −δ, ηopt = r0 + (1/δ)W (δe−δr0/a0), with

a0 =1

Φ(γ)φ(γ) −

1δ

, r0 =1δ−Φ(γ)φ(γ)

RT(0). (7.88)

Proof. Immediate, since the standard form is e−δη = a0(η− r0).

In case α= (γ+δ)/δ 6= 0, 1, the threshold equation is given by, see (7.40),

e−δη−RT(0) =δ

B

∫ η

0

e−δx 1− e−γx

γdx =

δ

Bγ

1− e−δη

δ−

1− e−(γ+δ)η

γ+δ

, (7.89)

193


10−2 10−1 100 101

10−2

10−1

100

101

asymptotics

rR(x) = e−∆x

rR(x) = 1−∆x

∆

ηopt

Figure 7.6: The optimal threshold ηopt in the exponential revenue case rR(x) =exp (−∆x), and in the linear revenue case rR(x) = 1−∆x , as ∆ ↓ 0. In bothcases, r ′R(0+) = −∆. The leading-order behavior established in Proposition 7.14is also included.

After setting z = e−δη ∈ (0, 1], (7.89) takes the form

(7.90) z − RT(0) =1γB

1− z −

1− zα

α

.

Observe that the factor 1/γB is positive when α > 1, and negative whenα < 1. For values α= −1, 1/2, and 2, an explicit solution can be found in termsof the square-root function, see Proposition 7.24 in Appendix 7.B. In all othercases, the solution is more involved. In certain regimes, however, a solutionin terms of an infinite power series can be obtained, see Proposition 7.23 inAppendix 7.B.

For illustrative purposes, we again plot the optimal threshold ηopt as a func-tion of γ. It has been determined by numerically solving the threshold equation,and is plotted together with the bounds given by Proposition 7.7,

(7.91) ηmax = −1d

ln g(γ− b)

g(γ)

, ηmin = −

1d

ln g(γ− b) +

∫ ηmax

0 e−γx dx

g(γ) +∫ ηmax

0 e−γx dx

,

194

7.4. Optimality of threshold policies

and asymptotic solutions of Proposition 7.10,

ηγ→−∞ = −1γ

ln1+

bd

, ηγ→∞ =

bγd

, (7.92)

in Figure 7.7. Similar to Figure 7.5, Figure 7.7 also illustrates the monotonicityof ηopt in γ ∈ R, the different regimes our approximations and bounds arevalid, and how our bounds are tight as γ → ∞. We have also indicated theanalytical solutions for α= −1, 0, 1/2, and 2, as provided by Proposition 7.15and Proposition 7.24 in Appendix 7.B. The asymptotic width 1/2 of the gapbetween the graphs of ηopt and ηγ→∞ is consistent with the refined asymptoticsof ηopt as given below (7.61), case b = d = 1.

−3 −2 −1 −0.5 0 1 2 30

1

2

3

ηmin

ηmax

ηγ→−∞

ηγ→∞

γ

Figure 7.7: The optimal threshold ηopt, its bounds ηmin, ηmax, and its approxima-tions ηγ→±∞, all as a function of γ, when rL(x) = exp (bx), rR(x) = exp (−d x),and b = d = 1. The analytical solutions for α = −1, 0, 1/2, and 2 providedby Proposition 7.15 and Proposition 7.24 are also indicated. The curve for theoptimal threshold has been produced by numerically solving the threshold equa-tion.

7.4 Optimality of threshold policies

We now present a proof of Proposition 7.3, the cornerstone for this paper thatsays that threshold policies are optimal, and that the optimal threshold satisfies

195


the threshold equation. We first present in Section 7.4.1 a variational argumentthat gives an insightful way to derive Proposition 7.3 heuristically. Next, wepresent the formal proof of Proposition 7.3 in Section 7.4.2 using Hilbert-spacetheory.

7.4.1 Heuristic based on a variational argument

For threshold controls f (x) = 1[0≤ x < η] with η ∈ [0,∞), the QED limit ofthe long-term revenue (7.15) becomes (7.16). The optimal threshold ηopt canbe found by equating(7.93)

dRdη=

B + 1−e−γη

γ

r(η)e−γη −

A+

∫ η0 r(x)e−γx dx

e−γη

B + 1−e−γη

γ

2 =r(η)− RT(η)

eγηB + 1−e−γη

γ

to zero, which shows that the optimal threshold ηopt solves the threshold equa-tion (7.22), i.e. r(η) = RT(η).

For any piecewise continuous function g on [0,∞) that is admissible, i.e.such that 0≤ f + εg ≤ 1 and

∫∞0 ( f + εg)e−γx dx <∞ for sufficiently small ε,

define

(7.94) δR( f ; g) = limε↓0

R( f + εg)− R( f )ε

.

We call (7.94) the functional derivative of f with increment g, which can looselybe interpreted as a derivative of f in the direction of g, see [56] for background.Substituting (7.15) into (7.94) yields

(7.95)

δR( f ; g) =

B +

∫∞0 f e−γx dx

∫∞0 r ge−γx dx

B +


2

−

A+

∫∞0 r f e−γx dx

∫∞0 ge−γx dx

B +


2 .

Rewriting (7.95) gives

(7.96) δR( f ; g) =

∫∞0 g(x)e−γx

r(x)− R( f )

dx

B +∫∞

0 f (x)e−γx dx.

We can now examine the effect of small perturbations εg towards (or awayfrom) policies f by studying the sign of (7.96). Specifically, it can be shown

196


that for every perturbation g applied to the optimal threshold policy of Propo-sition 7.3, δR( f opt; g) ≤ 0, indicating that these threshold policies are locallyoptimal. Moreover, it can be shown that for any other control f , a perturbationexists so that δR( f ; g) > 0. Such other controls are therefore not locally opti-mal. Assuming the existence of an optimizer, these observations thus indeedindicate that the threshold control in Proposition 7.3 is optimal. We note thatthese observations crucially depend on the sign of r(x)− R( f ), as can be seenfrom (7.96). It is in fact the threshold equation (7.22) that specifies the pointwhere a sign change occurs.

Note that while these arguments support Proposition 7.3, this section doesnot constitute a complete proof. In particular the existence of optimizers stillneeds to be established.

7.4.2 Formal proof of Proposition 7.3

In the formal proof of Proposition 7.3 that now follows, we start by provingthat there exist maximizers in Section 7.4.2. This ensures that our maximizationproblem is well-defined. In Section 7.4.2, we then derive necessary conditions formaximizers by perturbing the control towards (or away from) a threshold policy,as alluded to before, and in a formal manner using measure theory. Finally, wecharacterize in Section 7.4.2 the maximizers, by formally discarding pathologicalcandidates.

With r : R→ [0,∞) a smooth function, nonincreasing to 0 as x →±∞, andγ ∈ R, recall that we are considering the maximization of the functional (7.15)with f : [0,∞)→ [0,1] measurable and with g(x) = f (x)e−γx ∈ L1([0,∞)).We do not assume f to be nonincreasing. Recall that A=

∫ 0

−∞ r(x)exp (− 12 x2−

γx)dx > 0, B = Φ(γ)/φ(γ)> 0, and let b(x) = e−γx for x ≥ 0. Then write R( f )as

R( f ) =A+

∫∞0 r(x)g(x)dx

B +∫∞

0 g(x)dx= L(g), (7.97)

which is considered for all g ∈ L1([0,∞)) such that 0 ≤ g(x) ≤ b(x) for0≤ x <∞. The objective is to maximize L(g) over all such allowed g.

For notational convenience, write

L(g) =AB

1+

∫∞0 s(x)g(x)dx

1+∫∞

0 Sg(x)dx

, (7.98)

197


where

(7.99) s(x) =r(x)

A−

1B

, S =1B

.

Recall that r(x) is nonincreasing, implying that s(x) ≤ s(0) for all x ≥ 0.When s(0)≤ 0, the maximum of (7.98) thus equals A/B, and is assumed by allallowed g that vanish outside the interval [0, supx ∈ [0,∞)|s(x) = 0]. Whens(0)> 0, define

(7.100) x0 = infx ∈ [0,∞)|s(x) = 0,

which is positive and finite by smoothness of s and r(x)→ 0 as x →∞. Notethat the set x ∈ [0,∞)|s(x) = 0 consists of a single point when r(x) is strictlydecreasing as long as r(x) > 0. But even if r(x) is not strictly decreasing, wehave s(x)≤ 0 for x ≥ x0. Because g(x)≥ 0 implies that

(7.101)

∫ ∞

x0

s(x)g(x)dx ≤ 0≤∫ ∞

x0

Sg(x)dx ,

we have

(7.102) L(g1[x ∈ [0, x0)])≥ L(g)

for all g. We may therefore restrict attention to allowed g supported on [0, x0].Such a g can be extended to any allowed function supported on [0, supx ∈[0,∞)|s(x) = 0 without changing the value L(g). Therefore, we shall insteadmaximize

(7.103) J(g) =

∫ x0

0 s(x)g(x)dx

1+∫ x0

0 Sg(x)dx

over all g ∈ L1([0, x0]) satisfying 0≤ g(x)≤ b(x) for 0≤ x ≤ x0, in which s(x)is a smooth function that is positive on [0, x0) and decreases to s(x0) = 0.

Existence of allowed maximizers

Proposition 7.16. There exist maximizers f opt ∈ F that maximize R( f ).

Proof. We will use notions from the theory of Hilbert spaces and Lebesgue in-tegration on the line. We consider maximization of J(g) in (7.103) over allmeasurable g with 0≤ g(x)≤ b(x) for a.e. x ∈ [0, x0].

198


For any g ∈ L1([0, x0]), the Lebesgue points of g, i.e., all x1 ∈ (0, x0) suchthat

limε↓0

12ε

∫ ε

−εg(x1 + x)dx (7.104)

exists, is a subset of [0, x0] whose complement is a null set, and the limit in(7.104) agrees with g(x1) for a.e. x1 ∈ [0, x0], [146].

The set of allowed functions g is a closed and bounded set of the separableHilbert space L2([0, x0]), and the functional J(g) is bounded on this set. Hence,we can find a sequence of candidates gnn∈N0

of allowed gn, such that

limn→∞

J(gn) = supallowed g

J(g)<∞. (7.105)

We can subsequently find a subsequence hkk∈N0= gnk

k∈N0such that hk

converges weakly to an h ∈ L2([0, x0]), [57]. Then

supallowed g

J(g)= limk→∞

J(hk) = limk→∞

∫∞0

r(x)A −

1B

hk(x)dx

1+ 1B

∫∞0 hk(x)dx

(i)=

∫∞0

r(x)A −

1B

h(x)dx

1+ 1B

∫∞0 h(x)dx

= J(h),

(7.106)

where (i) follows from weak convergence. We now only need to show that h isallowed. We have for any ε > 0 and any x1 ∈ (0, x0) by weak convergence that

12ε

∫ ε

−εh(x1 + x)dx = lim

k→∞

12ε

∫ ε

−εhk(x1 + x)dx ∈ [0, b(x1)], (7.107)

since all hk are allowed. Hence for all Lebesgue points x1 ∈ (0, x0) of h we have

limε↓0

12ε

∫ ε

−εh(x1 + x)dx ∈ [0, b(x1)], (7.108)

and so 0≤ h(x1)≤ b(x1) for a.e. x1 ∈ [0, x0]. This, together with (7.106) showsthat h is an allowed maximizer.

Necessary condition for maximizers

Proposition 7.17. For any maximizer f opt ∈ F , f (x) = 1 if r(x) > R(x), andf (x) = 0 if r(x)< R(x).

199


Proof. Let g be an allowed maximizer of J(g). We shall equivalently show thatfor any Lebesgue point x1 ∈ (0, x0) of g,

(7.109)

s(x1)(1+∫ x0

0 Sg(x)dx)

S∫ x0

0 s(x)g(x)dx> 1⇒ g(x1) = b(x1),

s(x1)(1+∫ x0

0 Sg(x)dx)

S∫ x0

0 s(x)g(x)dx< 1⇒ g(x1) = 0.

Let x1 ∈ (0, x0) be any Lebesgue point of g and assume that

(7.110)s(x1)(1+

∫ x0

0 Sg(x)dx)

S∫ x0

0 s(x)g(x)dx> 1.

Suppose that g(x1)< b(x1). We shall derive a contradiction. Let ε0 > 0 be smallenough so that

(7.111)12(g(x1) + b(x1))≤ min

x1−ε0≤x≤x1+ε0

b(x).

Along with g, consider for 0< ε ≤ ε0 the function

(7.112) gε(x) =

(g(x), x 6∈ [x1 − ε, x1 + ε],12 (g(x1) + b(x1)), x ∈ [x1 − ε, x1 + ε].

This gε is allowed by (7.111). Write J(g) as

(7.113) J(g) =C(ε) + Is(ε; g)D(ε) + IS(ε; g)

,

where(7.114)

C(ε) =

∫ y−ε

0

+

∫ x0

y+ε

s(x)g(x)dx , D(ε) = 1+

∫ y−ε

0

+

∫ x0

y+ε

Sg(x)dx ,

and

(7.115) Is(ε; g) =

∫ x1+ε

x1−εs(x)g(x)dx , IS(ε; g) =

∫ x1+ε

x1−εSg(x)dx .

200


We can do a similar thing with J(gε), using the same numbers C(ε) and D(ε)and g replaced by gε in (7.115). We compute

J(gε)− J(g)

=(C(ε) + Is(ε; gε))(D(ε) + IS(ε; g))− (C(ε) + Is(ε; g))(D(ε) + IS(ε; gε))

(D(ε) + IS(ε; g))(D(ε) + IS(ε; gε)),

(7.116)in which the numerator N(gε; g) of the fraction at the right-hand side of (7.116)can be written as

N(gε; g) =(Is(ε; gε)− Is(ε; g))D(ε)− (IS(ε; gε)− IS(ε; g))C(ε)

+ Is(ε; gε)IS(ε; g)− Is(ε; g)IS(ε; gε).(7.117)

Since x1 is a Lebesgue point of g, we have as ε ↓ 0

12ε

Is(ε; gε)→12

s(x1)(g(x1) + b(x1)),12ε

Is(ε; g)→ s(x1)g(x1), (7.118)

12ε

IS(ε; gε)→12

S(g(x1) + b(x1)),12ε

IS(ε; g)→ Sg(x1), (7.119)

while also

C(ε)→∫ x0

0

s(x)g(x)dx , D(ε)→ 1+

∫ x0

0

Sg(x)dx . (7.120)

Therefore,

limε↓0

N(gε, g)

=12(b(x1)− g(x1))

s(x1)

1+

∫ x0

0

Sg(x)dx− S

∫ x0

0

s(x)g(x)dx> 0

(7.121)by assumption (7.110). Then J(gε) − J(g) > 0 when ε is sufficiently small,contradicting maximality of J(g). Hence, we have proven the first relation in(7.109). The proof of the second relation is similar.

Characterization of maximizers

Proposition 7.17 does not exclude the possibility that a maximizer alternatesbetween 0 and 1. Proposition 7.18 solves this problem by excluding the patho-logical candidates.

201


Proposition 7.18. The quantity

(7.122) R( f ;η) =A+

∫ η0 r(x) f (x)e−γx dx

B +∫ η

0 f (x)e−γx dx.

is uniquely maximized by

(7.123) f (x) = 1[0≤ x ≤ ηopt],

with ηopt a solution of the equation r(η) = RT(η), apart from null functions andits value at any solution of r(η) = RT(η).

Proof. Assume that g is a maximizer, and consider the continuous, decreasingfunction

(7.124) t g(x1) = s(x1)1+

∫ x0

0

Sg(x)dx− S

∫ x0

0

s(x)g(x)dx ,

which is positive at x1 = 0 and negative (because g 6= 0) at x1 = x0 since s isdecreasing with s(0)> 0= s(x0). Let x2,g , x3,g be such that 0< x2,g ≤ x3,g < x0

and

(7.125) t g(x1) =

> 0, 0≤ x1 < x2,g ,

= 0, x2,g ≤ x1 ≤ x3,g ,

< 0, x3,g < x1 ≤ x0.

Note that x2,g = x3,g when s is strictly decreasing on [0, x0], and that s′(x) = 0for x ∈ [x2,g , x3,g] when x2,g < x3,g . According to [6, Chapter 6], we have(7.126)

g(x1) = b(x1), a.e. x1 ∈ [0, x2,g], and g(x1) = 0, a.e. x1 ∈ [x3,g , x0].

For an allowed h 6= 0, consider the continuous function

(7.127) J(h; x1) =

∫ x1

0 s(x)h(x)dx

1+∫ x1

0 Sh(x)dx, 0≤ x1 ≤ x0.

We differentiate J(h; x1) with respect to x1, where we use the fact that for anyk ∈ L1([0, x0]),

(7.128)d

dx1

∫ x1

0

k(x)dx= k(x1), a.e. x1 ∈ [0, x0].

202


Thus we get for a.e. x1 that

ddx1

J(h; x1)

=

Nh(x1)Dh(x1)

, (7.129)

where Dh(x1) = (1+∫ x0

0 Sh(x)dx)2, and

Nh(x1) = h(x1)Mh(x1) (7.130)

with

Mh(x1) = s(x1)1+

∫ x1

0

Sh(x)dx− S

∫ x1

0

s(x)h(x)dx . (7.131)

Now Mh(x1) is a continuous function of x1 ∈ [0, x0] with Mh(x0)< 0< Mh(0)since s(x0) = 0 < s(0) and h 6= 0. Furthermore, Mh(x1) is differentiable at a.e.x1, and one computes for a.e. x1,

ddx1

Mh(x1)

= s′(x1)

1+

∫ x1

0

Sh(x)dx. (7.132)

Since s is decreasing, the right-hand side of (7.132) is nonpositive for all x1 andnegative for all x1 with s′(x1)< 0.

Now let g be a maximizer, and consider first the case that s(x) is strictlydecreasing. Then x2,g = x3,g in (7.126). Next consider h = b in (7.127) andfurther. It follows from (7.132) that Mb is strictly decreasing on [0, x0], and soMb has a unique zero x on [0, x0]. Therefore, by (7.129) and (7.130), J(b; x1)has a unique maximum at x1 = x . Then, from (7.126) and maximality of g,x2,g = x = x3,g . Hence, J is uniquely maximized by

g(x1) = b(x1)1[x1 ∈ [0, x]], (7.133)

apart from null functions, with x the unique solution y of the equation

s(y)1+

∫ y

0

Sb(x)dx− S

∫ y

0

s(x)b(x)dx = 0. (7.134)

This handles the case that s is strictly decreasing.When s′ may vanish, we have to argue more carefully. In the case that x2,g =

x3,g , we can proceed as earlier, with (7.133) emerging as maximizer and x2,g =y = x3,g . So assume we have a maximizer g with x2,g < x3,g , and consider

203


h = g in (7.127) and further. We have that J(h = g; x1) is constant in x1 ∈[x3,g , x0]. Furthermore, from s′(x1) = 0 for x1 ∈ [x2,g , x3,g] and (7.130), wesee that J(h= g; x1) is constant in x1 ∈ [x2,g , x3,g] as well. This constant valueequals J(g), and is equal to J(b1[x1 ∈ [0, x2,g]]) since, due to (7.130), wehave J(g; ·) = J( g; ·) when g = g a.e. outside [x2,g , x3,g]. We are then againin the previous situation, and the solutions y of (7.134) form now a wholeinterval [y2, y3]. The maximizers are again unique, apart from their values forx1 ∈ [y2, y3] that can be chosen arbitrarily between 0 and b(x1).

7.5 Conclusions and future perspectives

The QED regime has gained tremendous popularity in the operations manage-ment literature, because it describes how large-scale service operations canachieve high system utilization while simultaneously maintaining short delays.Operating a system in the QED regime typically entails hiring a number of serversaccording to the square-root staffing rule s = λ/µ+γ

pλ/µ, and has the added

benefit that limiting performance measures can be described by elementaryfunctions of just the one parameter γ. Through the square-root staffing rule, γdetermines a hedge against variability or overcapacity, which is of the order ofthe natural fluctuations of the demand per time unit when the system operatesin the QED regime. Classical problems of dimensioning large-scale systems inthe QED regime can then be solved by optimizing objective functions solelydependent on γ.

Our paper adds a revenue maximization framework that complies withthe classical dimensioning of QED systems by constructing scalable admissioncontrols and revenue structures that remain meaningful in the QED regime(Proposition 7.1). As we have proven, our revenue framework naturally leadsto an optimal control that bars customers from entering when the queue lengthof delayed customers exceeds the threshold ηoptps, provided that ηopt satisfiesa fundamental threshold equation (Proposition 7.3). A detailed study of thisthreshold equation made it possible to characterize ηopt in terms of exact ex-pressions, bounds, and asymptotic expansions. The weak assumptions madethroughout this paper allow for application to a rich class of revenue structures,and an interesting direction for future work would therefore be the constructionof realistic revenue structures based on specific case studies, expert opinions, orcalibration to financial data.

204

7.A. Limiting behavior of long-term QED revenue

Let us finally discuss the fascinating interplay between the parameters γ andη, which suggest that they act as communicating yet incomparable vessels. Theoptimal threshold ηopt increases with the overcapacity γ. Since more overca-pacity roughly means fewer customers per server, and a larger threshold meansmore customers per server, we see that the optimization of revenues over thepair (γ,η) gives rise to an intricate two-dimensional framework in which thetwo parameters have radically different yet persistent effects in the QED regime.At the process level, the γ acts as a negative drift in the entire state space, whilethe η only interferes at the upper limit of the state space. Hence, while in thispaper we have treated γ as given, and mostly focused on the behavior of the newparameter η, our framework paves the way for two-dimensional joint staffingand admission control problems. Gaining a deeper understanding of this inter-play, and in relation to specific revenue structures, is a promising direction forfuture research.

Appendix

7.A Limiting behavior of long-term QED revenue

With rs(k) = r((k− s)/p

s) as in (7.3) and πs(k) = limt→∞ P[Qs(t) = k], (7.9),where ps and f are related as in (7.6), we compute for ρ = 1− γ/

ps > 0,

∞∑k=0

rs(k)πs(k) =

∑sk=0 r

k−sp

s

(sρ)kk! +

(sρ)s

s!

∑∞k=s+1 r

k−sp

s

ρk−s f

k−sp

s

∑s

k=0(sρ)k

k! +(sρ)s

s!

∑∞k=s+1ρ

k−s f

k−sps

. (7.135)

Dividing by the factor (sρ)s/s!, we obtain

∞∑k=0

rs(k)πs(k) =W L

s (ρ) +W Rs (ρ)

B−1s (ρ) + Fs(ρ)

. (7.136)

Here,

Bs(ρ) =(sρ)s

s!∑sk=0

(sρ)kk!

(7.137)

is the Erlang B formula,

Fs(ρ) =∞∑n=0

ρn+1 fn+ 1p

s

(7.138)

205


as in (7.11), and

W Ls (ρ) =

s∑k=0

rk− sp

s

s!(sρ)k−s

k!,(7.139)

W Rs (ρ) =

∞∑n=0

rn+ 1p

s

ρn+1 f

n+ 1p

s

.(7.140)

with superscripts L and R referring to the left-hand part k = 0,1, . . . , s andright-hand part k = s+ 1, s+ 2, . . . of the summation range, respectively.

From Jagerman’s asymptotic results for the Erlang B formula, there is theapproximation [118, Theorem 14]

(7.141) B−1s (ρ) =

psψ(γ) +χ(γ) +O

1p

s

with ψ(γ) = Φ(γ)/φ(γ) and χ(γ) expressible in terms of φ and Φ as well. ForFs(ρ) there is the approximation [6, Theorem 4.2],

(7.142) Fs(ρ) =p

sL (γ) +M (γ) +O 1p

s

,

with L (γ) =∫∞

0 f (x)exp (−γx)dx andM (γ) expressible in terms of L ′(γ).We aim at similar approximations for W L

s (ρ) and W Rs (ρ) in (7.139), (7.140).

We start by considering W Rs (ρ) for the case that r and its first two derivatives

are continuous and bounded in the two following situations:

(i.) f is smooth; f (y)exp (−γy) and its first two derivatives are exponentiallysmall as y →∞.

(ii.) f = 1[x ∈ [0,η]] with η > 0.

7.A.1 Asymptotics of W Rs (ρ)

In the series expression for W Rs (ρ), we have

(7.143) ρn+1 =1−

γp

s

n+1= e−

(n+1)γsps

with

(7.144) γs = −p

s ln1−

γp

s

= γ+

γ2

2p

s+ . . .> γ.

206


Hence, the conditions in case (i.) are also valid when using γs instead of γ.We obtain the following result.

Lemma 7.19. For case (i.) it holds that

W Rs (ρ) =

ps

∫ ∞

0

e−γs y r(y) f (y)dy −12

r(0) f (0) +O 1p

s

. (7.145)

For case (ii.) it holds that

W Rs (ρ) =

ps

∫ η

0

e−γs y r(y)dy−12

r(0)+bηp

sc−ηp

s−12

e−γsηr(η)+O

1p

s

.

(7.146)

Proof. We use EM-summation as in [6, Chapter 6, Appendix C], first instance in[6, (C.1)], case m= 1, with the function

h(x) = g x + 1

2ps

, x ≥ 0, and g(y) = e−γs y r(y) f (y), y ≥ 0, (7.147)

using a finite summation range n= 0, 1, . . . , N , where we take N = s in case (i.)and N = bη

ps − 3/2c in case (ii.). In both cases, we have by smoothness of h

on the range [0, N + 1] that

N∑n=0

hn+ 1

2

=

∫ N+1

0

h(x)dx + 12 B2

12

h(1)(N + 1)− h(1)(0)

+ R, (7.148)

where |R| ≤ 12 B2

∫ N+1

0 |h(2)(x)|dx . Due to our assumptions, it holds in both casesthat

12 B2

12

h(1)(N + 1)− h(1)(0)

+ R= O

1p

s

. (7.149)

In case (i.), the left-hand side of (7.148) equals W Rs (ρ), apart from an error that

is exponentially small as s→∞. In case (ii.), the left-hand side of (7.148) andW R

s (ρ) are related according to

W Rs (ρ) =

N∑n=0

hn+ 1

2

+ g

bηpscp

s

bηp

sc −ηp

s− 12

. (7.150)

The second term at the right-hand side of (7.150) equals 0 or g(bηp

sc/p

s)accordingly as η

ps − bη

psc ≥ or < 1

2 , i.e., accordingly as N + 1 = bηp

sc or

207


bηp

sc − 1. Next, by smoothness of h and g on the relevant ranges, we have(7.151)∫ N+1

0

h(x)dx =p

s

∫ N+3/2ps

12p

s

g(y)dy =p

s

∫ N+3/2ps

0

g(y)dy −12

g(0) +O 1p

s

.

In case (i.), we have that∫∞(N+3/2)/

ps g(y)dy is exponentially small as s→∞,

since N = s, and this yields (7.145). In case (ii.), we have

(7.152)

∫ N+3/2ps

0

g(y)dy −∫ η

0

g(y)dy =

∫ N+3/2ps

η

g(y)dy

=N + 3/2p

s−η

g bηpscp

s

+O

1s

=1p

s

ηp

s− 12

−ηp

s− 12

g bηpscp

s

+O

1s

,

and with (7.150), this yields (7.146). This completes the proof.

We denote for both case (i.) and (ii.)

(7.153) Lr f (δ) =

∫ ∞

0

e−δ y r(y) f (y)dy

with δ ∈ R such that the integral of the right-hand side of (7.153) convergesabsolutely. From (7.144) it is seen that, with the prime ′ denoting differentiation,

(7.154) Lr f (γs) =Lr f (γ) +γ2

2p

sL ′r f (γ) +O

1s

.

Thus we get from Lemma 7.19 the following result.

Proposition 7.20. For case (i.) it holds that

(7.155) W Rs (ρ) =

psLr f (γ) +

12γ

2L ′r f (γ)−12 r(0) f (0) +O

1p

s

.

For case (ii.) it holds that

(7.156)W R

s (ρ) =p

sLr f (γ) +12γ

2L ′r f (γ)−12 r(0)

+bηp

sc −ηp

s− 12

e−γηr(η) +O

1p

s

.

208


7.A.2 Asymptotics of W Ls (ρ)

We next consider W Ls (ρ) for the case that r : (−∞, 0]→ R has bounded and

continuous derivatives up to order 2. Using a change of variables, we write

W Ls (ρ) = r(0) +

s∑k=1

r−kp

s

s!s−k

(s− k)!ρ−k, (7.157)

and we again intend to apply EM-summation to the series at the right-hand sideof (7.157). We first present a bound and an approximation.

Lemma 7.21. We have for |γ|/p

s ≤ 12 and ρ = 1− γ/

ps,

s!s−k

(s− k)!ρ−k ≤ exp

−

k(k− 1)2s

+γkp

s+γ2k

s

, k = 1, 2, . . . , s, (7.158)

ands!s−k

(s− k)!ρ−k = Gs

kp

s

1+O

1s

P6

kp

s

, k ≤ s2/3, (7.159)

whereGs(y) = e−

12 y2+γy

1−

16p

sy3 +

12p

s(1+ γ2)y

, (7.160)

and P6(y) is a polynomial in y of degree 6 with coefficients bounded by 1 (theconstant implied by O(·) depends on γ).

Proof. We have for k = 1,2, . . . , s and |γ|/p

s ≤ 1/2, ρ = 1− γ/p

s,

s!s−k

(s− k)!ρ−k = ρ−k

k−1∏j=0

1−

js

= exp

k−1∑j=0

ln1−

js

− k ln

1−

γp

s

≤ exp−

k−1∑j=0

js+γkp

s+γ2k

s

= exp

−

k(k− 1)2s

+γkp

s+γ2k

s

,

(7.161)

where it has been used that − ln (1− x)≤ x + x2, |x | ≤ 1/2.On the range k ≤ s2/3, we further expand

s!s−k

(s− k)!ρ−k = exp

−

k−1∑j=0

js+

j2

2s2+O

j3

s3

+γkp

s+γ2k

s+O

ks3/2

= exp−

k(k− 1)2s

−k(k− 1)(2k− 1)

12s2+O

k4

s3

+γkp

s+γ2k

s+O

ks3/2

= exp−

k2

2s+γkp

s−

k3

6s2+

12(1+ γ2)

ks+O

ks3/2+

k2

s2+

k4

s3

(7.162)

209


On the range 0≤ k ≤ s2/3 we have

(7.163)k3

s2,

ks

,k

s3/2,

k2

s2,

k4

s3= O(1).

Hence, on the range 0≤ k ≤ s2/3,

(7.164)

s!s−k

(s− k)!ρ−k = exp

−

k2

2s+γkp

s

1−

k3

6s2+

12(1+ γ2)

ks

+Ok6

s4+

k4

s3+

k2

s2+

ks3/2

= G kp

s

1+O

1s

P6

kp

s

,

where P6(y) = y6 + y4 + y2 + y .

Proposition 7.22. It holds that

(7.165)

W Ls (ρ) =

ps

∫ 0

−∞e−

12 y2−γy r(y)dy +

12

r(0)

+

∫ 0

−∞e−

12 y2−γy

16

y3 −12(1+ γ2)y

r(y)dy +O

1p

s

.

Proof. With v(y) = r(−y), we write

(7.166) W Ls (ρ) = r(0) +

s−1∑n=0

vn+ 1p

s

s!s−n−1

(s− n− 1)!ρ−n−1.

By the assumptions on r and the bound in (7.158), the contribution of the termsin the series in (7.166) is O(exp (−Cs1/3)), s→∞, for any C with 0< C < 1/2.On the range n = 0,1, . . . , bs2/3c − 1 =: N , we can apply (7.159), and so, withexponentially small error,

(7.167) W Ls (ρ) = r(0) +

N∑n=0

vn+ 1p

s

Gs

n+ 1p

s

1+O

1s

P6

n+ 1p

s

.

By EM-summation, as used in the proof of Lemma 7.19 for the case (i.) asconsidered there, we have(7.168)

N∑n=0

vn+ 1p

s

Gs

n+ 1p

s

=p

s

∫ ∞

0

v(y)Gs(y)dy −12

v(0)Gs(0) +O 1p

s

,

210

7.B. Explicit solutions for exponential revenue

where we have extended the integration range [0, (N + 3/2)/p

s] to [0,∞) atthe expense of exponentially small error. Then the result follows on a change ofthe integration variable, noting that v(y) = r(−y) and the definition of Gs in(7.160), implying Gs(0) = 1.

The result of Proposition 7.1 in the main text follows now from (7.141),(7.142), Proposition 7.20 and Proposition 7.22, by considering leading termsonly.

7.B Explicit solutions for exponential revenue

Proposition 7.23. When ε = 1− RT(0)> 0 is sufficiently small,

ηopt = −1δ

ln (1−∞∑l=1

alεl), (7.169)

where for l = 2, 3, . . .,

a1 = 1, a2 =12(α+ β − 1), (7.170)

al+1 =1

l + 1

(lα+ (l + 1)β − 1)al + β

l−1∑i=2

iaial+1−i

, (7.171)

with β = (1−α)(1+ 1/(γB)) and the convention that∑l−1

i=2 = 0 for l = 2.

Proof. With ε = 1− RT(0) and w= 1− z, we can write (7.90) as

H(w) = w+1γB(w−

1α(1− (1−w)α)) = ε. (7.172)

Note that

H(w) = w+1γB(12(α− 1)w2 −

16(α− 1)(α− 2)w3 + . . .), |w|< 1, (7.173)

and so there is indeed a (unique) solution

w(ε) = ε +∞∑l=2

alεl (7.174)

211


of (7.172) when |ε| is sufficiently small. To find the al we let D = 1/(γB), andwe write (7.172) as

(7.175) (1+ D)w−1α

D+1α(1−w)α = ε, w= w(ε).

Differentiating (7.175) with respect to ε, multiplying by 1−w(ε), and eliminating(1−w(ε))α using (7.175) yields the equation

(7.176) (1−αε − βw(ε))w′(ε) = 1−w(ε).

Inserting the power series (7.174) for w(ε) and 1+∑∞

l=1(l + 1)al+1εl for w′(ε)

into (7.176) gives

(7.177)

1−αε +∞∑l=1

(l + 1)al+1εl −α

∞∑l=2

lalεl

− βε − β∞∑l=2

lalεl − β

∞∑l=2

alεl − β

∞∑l=2

alεl∞∑l=1

(l + 1)al+1εl

= 1− ε −∞∑l=2

alεl .

Using that

(7.178)∞∑l=2

alεl∞∑l=1

(l + 1)al+1εl =

∞∑l=3

l−1∑i=2

iaial+1−i

εl ,

it follows that a1, a2, a3, . . . can be found recursively as in (7.170)–(7.171), byequating coefficients in (7.177). The result (7.169) then follows from ηopt =−(1/δ) ln z = (1/δ) ln (1−w). The inequality β < 0 follows from the inequalityγ+φ(γ)/Φ(γ)> 0, γ ∈ R, given in [145, Sec. 4].

We consider next the cases α = −1, 1/2, and 2 that allow for solving thethreshold equation explicitly, and that illustrate Proposition 7.23.

Proposition 7.24. Let t = −γB/(1+ γB) > 0, and ε = 1− RT(0), for the cases(i) and (ii) below. The optimal threshold ηopt is given as

(7.179) ηopt = −1δ

ln (1−w(ε)),

where w(ε) is given by:

212

7.B. Explicit solutions for exponential revenue

(i) α= −1,

w(ε) =12

t√√(1+ ε)2 +

4εt− 1− ε

= ε −12

t∞∑k=2

(−1)kPk

1+ 2

t

− Pk−2

1+ 2

t

2k− 1εk

(7.180)

for |ε|< 1+ 2/t −p(1+ 2/t)2 − 1, and where Pk is the Legendre polynomial of

degree k,

(ii) α= 1/2,

w(ε) =2t

1+ γB

s1+

ε

t− 1

− tε = ε +

2t1+ γB

∞∑k=2

1/2k

εt

k(7.181)

for |ε|< t,

(iii) α= 2,

w(ε) = −γB +Æ(γB)2 + 2γBε = ε + γB

∞∑k=2

1/2k

2εγB

k(7.182)

for |ε|< 12γB.

Proof. Case (i). When α= −1, we can write the threshold equation as

w2 + t(1+ ε)w= tε. (7.183)

From the two solutions

w= −12

t(1+ ε)±

√√(12

t(1+ ε))2 + tε (7.184)

of (7.183), we take the one with the + sign so as to get w small and positivewhen ε is small and positive. This gives w(ε) as in the first line of (7.180), thesolution being analytic in the ε-range given in the second line of (7.180). To getthe explicit series expression in (7.180), we integrate the generating function

∞∑k=0

Pk(x)εk = (1− 2xε + ε2)−

12 (7.185)

213


of the Legendre polynomials over x from −1 to −1− 2/t, and we use for k =1,2, . . . that

(7.186) P ′k+1(x)− P ′k−1(x) = (2k+ 1)Pk(x), Pk+1(−1)− Pk−1(−1) = 0,

see [147, (4.7.29), (4.7.3-4)] for the case λ= 1/2.Case (ii). When α= 1/2, we can write the threshold equation as

(7.187) 2(1−w)12 = 2+ γBε − (1+ γB)w.

After squaring, we get the equation

(7.188) w2 + 22− (2+ γBε)(1+ γB)

(1+ γB)2w=

4− (2+ γBε)2

(1+ γB)2.

After a lengthy calculation, this yields the two solutions

(7.189) w=2γB

(1+ γB)21+

12(1+ γB)ε ±

√√1−

1+ γBγB

ε.

Noting that−1< γB < 0 in this case, and that w is small positive when ε is smallpositive, we take the − sign in (7.189), and arrive at the square-root expressionin (7.181), with t given earlier. The series expansion given in (7.181) and itsvalidity range follow directly from this.

Case (iii). When α= 2, we have γB > 0, and the threshold equation can bewritten as

(7.190) w2 + 2γBw= 2γBε.

Using again that w is small positive when ε is small positive, the result in (7.182)readily follows.

214

CHAPTER 8Optimality gaps

in asymptotic dimensioningof many-server systems

by Jaron Sanders, Sem Borst,Guido Janssen, and Johan van Leeuwaarden

arXiv preprint, 1511.01798

215

8. OP T I M A L I T Y G A P S I N A S Y M P T O T I C D I M E N S I O N I N G

Abstract

The Quality-and-Efficiency-Driven (QED) regime provides a ba-sis for solving asymptotic dimensioning problems that trade off rev-enue, costs and service quality. We derive bounds for the optimalitygaps that capture the differences between the true optimum andthe asymptotic optimum based on the QED approximations. Ourbounds generalize earlier results for classical many-server systems.We also apply our bounds to a many-server system with thresholdcontrol.

216

8.1. Introduction

8.1 Introduction

The theory of square-root staffing in many-server systems ranks among the mostcelebrated principles in applied probability. The general idea behind square-root staffing is as follows: a finite server system is modeled as a system in heavytraffic, where the number of servers s is large, whereas at the same time, thesystem is critically loaded. Under Markovian assumptions, and denoting theload on the system by λ, this can be achieved by setting s = λ+β

pλ and letting

λ →∞ while keeping β > 0 fixed, or alternatively setting λ = s − γp

s andletting s→∞ while keeping γ > 0 fixed. In both cases, the system reaches thedesirable Quality-and-Efficiency-Driven (QED) regime.

The QED regime refers to mathematically defined conditions in which bothcustomers and system operators benefit from the advantages that come withsystems that operate efficiently at large scale, which is particularly relevantfor systems in e.g. health care, cloud computing, and customer services. Suchconditions manifest themselves in a low delay probability and negligible meandelay, despite the fact that the system utilization is high. Properties of this sortcan be proven rigorously for systems such as the M/M/s queue by establishingstochastic-process limits under the aforementioned QED scalings [52]. The QEDregime also creates a natural environment for solving dimensioning problemsthat achieve an acceptable trade-off between service quality and capacity. Qualityis usually formulated in terms of some target service level. Take for instance theprobability that an arriving customer experiences delay. The target could be tokeep the delay probability below some value ε ∈ (0, 1). The smaller ε, the betterthe offered quality of service. Once the target service level is set, the objectivefrom the operator’s perspective is to determine the highest load λ such that thetarget ε is still met.

For the M/M/s queue, it was shown by Borst et al. [107] that such dimen-sioning procedures combined with QED approximations have certain asymp-totic optimality properties. To illustrate this, consider the case of linear costs,i.e. waiting cost are b per customer per unit time, and staffing cost are c perserver per unit time. Denoting the total cost by Kλ(s), it can be shown that whens = λ+ β

pλ and β > 0,

Kλ(s) = bλCλ(s)s−λ

+ cs = cλ+pλcβ +

bβ

Cλ(s)

(8.1)

with Cλ(s) the delay probability in the M/M/s queue. The first term cλ rep-

217


resents the cost of the minimally required capacity λ, while the second termgathers the cost factors that are all O(

pλ). Halfin and Whitt [52] showed that in

the QED regime Cλ(s) converges to a nondegenerate limit C0(β) ∈ (0, 1), so thatin the QED regime one only needs to determine β0 = arg minβcβ+ bC0(β)/β,and then set s0 = [λ+ β0

pλ] as an approximation for the optimal number of

servers sopt = argminsKλ(s). Borst et al. [107] called this procedure asymptoticdimensioning.

Based on the QED limiting regime, one expects that such approximate solu-tions are accurate for large relative loads λ. For the optimality gaps |s0−sopt| and|Kλ(s0)−Kλ(sopt)|, i.e. inaccuracies that arise from the fact that the actual systemis of finite size, Borst et al. [107] showed through numerical experiments thatthe approximation s0 performs exceptionally well in almost all circumstances,even when systems are only moderately sized. A rigorous underpinning for theseobservations was provided by Janssen et al. [114], who used refined QED ap-proximations to quantify the optimality gaps. The delay probability, for instance,was shown to behave as C0(β) + C1(β)/

pλ+O(λ−1), which in turn was used

to estimate the optimality gaps for the dimensioning problem in (8.1). Zhanget al. [115] obtained similar results for optimality gaps in the context of theM/M/s+M queue, in which customers may abandon before receiving service.

Motivated by the results in [114, 115], Randhawa [148] took a more abstractapproach to quantify optimality gaps of asymptotic dimensioning problems. Heshowed under general assumptions that when the approximation to the objec-tive function is accurate up to O(1), the prescriptions that are derived from thisapproximation are o(1)-optimal. The optimality gap thus becomes zero asymp-totically. This general setup was shown in [148] to apply to the M/M/s queuesin the QED regime, which confirmed and sharpened the results on the optimal-ity gaps in [114, 115] by implying that |Kλ(s0)− Kλ(sopt)|= o(1). The abstractframework in [148], however, can only be applied if refined approximations asin [114, 115] are available.

Such refined approximations were recently developed in [6, 7, Chapters 6, 7]for a broad class of many-server systems operating in the QED regime withλ = s − γ

ps, and equipped with an admission control policy and a revenue

structure. For a wide range of performance metrics, Ms(γ) say, these refinementsare of the form Ms(γ) = M0(γ) + M1(γ)/

ps + · · · . The method in [6, 7, Chap-

ters 6, 7] can deliver as many higher-order terms as needed, and generate allthe refinements obtained in [114, 115, 148]. Note that our problem formulation

218

8.2. Model description

differs from those encountered in typical perturbation analyses of optimizationproblems [149], since we consider expansions around the optimizer and notaround perturbations of system primitives.

In the present paper, we demonstrate how the results in [6, 7, Chapters 6, 7]can be leveraged to determine the optimality gaps of novel asymptotic dimen-sioning problems for a large class of many-server systems. Our main result(Theorem 8.1) provides generic bounds for the optimality gaps that becomesharper when more terms in the QED expansion for Ms(γ) are included.

8.2 Model description

8.2.1 Service systems with admission control and revenues

We consider many-server systems with s parallel servers, to which customersarrive according to a Poisson process with rate λ. Every customer requires anexponentially distributed service time with mean one. If a customer arrives andfinds k − s ≥ 0 customers waiting, the customer is allowed to join the queuewith probability ps(k− s) and is rejected with probability 1− ps(k− s). The totalnumber of customers in the system evolves as a birth–death process Xs(t)t≥0

and has a stationary distribution

πs(k) =

Z−1, k = 0,

Z−1 (sρ)k

k! , k = 1,2, . . . , s,

Z−1 ssρk

s!

∏k−s−1i=0 ps(i), k = s+ 1, s+ 2, . . . ,

(8.2)

where ρ = λ/s, Z =∑s

k=0(sρ)k/k!+ ((sρ)s/s!)Fs(ρ), and Fs(ρ) =

∑∞n=0 ps(0) ·

. . .·ps(n)ρn+1. The stationary distribution in (8.2) exists if and only if the relativeload ρ and the admission control policy ps(k)k∈N0

are such that Fs(ρ)<∞.Next, we assume that the system generates revenue at rate rs(k) ∈ R when

there are k customers in the system. The sequence rs(k)k∈N0will be called the

revenue structure. The stationary rate at which the system generates revenue isthen given by

Rs(γ) =∞∑k=0

rs(k)πs(k), (8.3)

which depends via the equilibrium distribution on the admission control policy.Ref. [7, Chapter 7] discusses the problem of maximizing the revenue rate overthe set of all admission control policies.

219


One advantage of considering general admission control policies and rev-enue structures is that one can study different service systems and steady-stateperformance measures through one unifying framework. For example, the equi-librium behavior of the canonical M/M/s/s, M/M/s, and M/M/s+M systemscan be recovered by choosing ps(k − s) = 0, ps(k − s) = 1, and ps(k − s) =1/(1+ (k − s + 1)θ/s), respectively. Here, θ corresponds to the rate at whichwaiting customers abandon from the M/M/s+M system. Similarly, the delayprobability Ds(γ) =

∑∞k=sπs(k) can be recovered by setting rs(k) = 1[k ≥ s],

the mean queue length Qs(γ) =∑∞

k=s(k − s)πs(k) is recovered when consid-ering rs(k) = (k − s)1[k ≥ s], and the average number of idle servers Is(γ) =∑s−1

k=0(s− k)πs(k) follows from rs(k) = (s− k)1[k < s].As a primary example we will consider a scenario where besides the waiting

cost b > 0 incurred per customer per unit time, a fee a > 0 is received for everyserved customer, and a penalty d ≥ 0 is imposed for rejecting a customer. Thelatter cost accounts for the degree of revenue loss from the admission controlpolicy. Denoting by DR

s (γ) =∑∞

k=s(1− ps(k − s))πs(k) the probability that anarriving customer is rejected, and by Ws(γ) =

∑∞k=s((k− s+ 1)/s)ps(k− s)πs(k)

the expected waiting time of an arriving customer, the total system revenue rateis given by

(8.4) Rs(γ) = aλ(1− DRs (γ))− bλWs(γ)− dλDR

s (γ).

By virtue of Little’s law λWs(γ) =Qs(γ) and λ(1− DRs (γ)) = s− Is(γ), and since

λ= s− γp

s, the revenue rate can equivalently be expressed as

(8.5) Rs(γ) = as+ dγp

s− (a+ d)Is(γ)− bQs(γ).

This scenario therefore corresponds to the revenue structure

(8.6) rs(k) =

(ak+ dγ

ps− d(s− k) k < s,

as+ dγp

s− b(k− s), k ≥ s.

8.2.2 QED scaling and refinements

We now discuss how to apply the QED scaling to obtain an asymptotic expan-sion for Rs(γ) for general revenue structures rs(k)k∈N0

, which we will exploitin Section 8.3 to characterize the asymptotic optimality gap. We impose thefollowing three conditions throughout this paper:

220

8.2. Model description

(i) The arrival rate and system size are coupled via the scaling λ= s− γp

s;

(ii) ps(0) · . . . · ps(n) → f ((n + 1)/p

s) where f (x) is either a continuous,nonincreasing function, or f (x) = 1[x ≤ η];

(iii) There exist sequences nss∈N+ , qss∈N+ with qs > 0, and a continuousfunction r(x) that satisfy the scaling condition (rs(k)− ns)/qs → r((k −s)/p

s).

It is proven in [6, 7, Chapters 6, 7] that under conditions (i)–(iii),

lims→∞

Rs(γ)− ns

qs= R0(γ), (8.7)

with

R0(γ) =

∫ 0

−∞ r(x)e−12 x2−γx dx +

∫∞0 r(x) f (x)e−γx dx

Φ(γ)φ(γ) +


. (8.8)

Here, Φ and φ denote the cumulative distribution function and probability den-sity function of the standard normal distribution. This asymptotic characteriza-tion of the revenue is leveraged in [7, Chapter 7] to prove that for many revenuestructures there exists an optimal admission control policy with a threshold struc-ture.

Moreover, the method used to obtain (8.8) in [6, 7, Chapters 6, 7] can beextended to derive an asymptotic expansion of the form

Rs(γ) = ns + qs

j∑i=0

Ri(γ)si/2

+O 1

s( j+1)/2

, (8.9)

which is shown in Appendix 8.A. We also provide closed-form expressions forthe first two terms R0(γ) and R1(γ) for arbitrary f (x) and r(x). The asymptoticexpansion in (8.9) is a crucial ingredient for determining the optimality gaps.

Let us discuss the asymptotic expansion in the context of (8.5). Denotingns = as and extracting a term qs =

ps yields Rs(γ) = ns +

psRs(γ) with

Rs(γ) = dγ− (a+ d)Is(γ)p

s− b

Qs(γ)ps

. (8.10)

Since our goal is to maximize Rs(γ) over γ, and because the term ns is constantand independent of γ, we only need to focus on the maximization of Rs(γ).

221


Recall (8.6) and note that the limiting revenue structure for the objectivefunction in (8.10) is given by r(x) = (a+d)x+dγ for x < 0, and r(x) = −bx+dγfor x ≥ 0. With an admission control policy f (x) = 1[x ≤ η] where η ≥ 0denotes the admission threshold, it follows from (8.8) that

(8.11) lims→∞

Rs(γ) = R0(γ) = dγ−(a+ d)

1+ γ Φ(γ)φ(γ)

+ b 1−(1+γη)e−γη

γ2

Φ(γ)φ(γ) +

1−e−γηγ

.

We prove in Appendix 8.A that lims→∞p

s(Rs(γ)− R0(γ)) = R1(γ) and providean explicit expression for R1(γ). We then have both a first-order approximationR0(γ) and a second-order approximation R0(γ) + R1(γ)/

ps for the objective

function Rs(γ).

8.3 General revenue maximization

For general objective functions (8.3), we now aim for solving the dimensioningproblem

(8.12) maxγ∈ΓRs(γ),

where we assume that Γ = [γl ,γr] is a compact interval contained in (γmins ,∞),

with γmins = infγ ∈ R|Fs(ρ)<∞. Denote the exact solution by

(8.13) γopts = argmax

γ∈ΓRs(γ).

We assume that an asymptotic expansion of the form (8.9) is available forRs(γ) and its derivative Rs

′(γ), which we can then use to approximate the ob-jective function. Hence we will consider

(8.14) γ j,s = arg maxγ∈Γ

¦R0(γ) + . . .+

R j(γ)

s j/2

©

as approximations for the exact solution γopts , which should be increasingly better

for larger j and/or s. Note that γ0,s = γ0 is independent of s.Denoting the ith derivative of a function g(x) by g(i)(x), we assume also

that R0(k)(γ), . . . , R j

(k)(γ) are bounded on Γ for k = 0,1,2, and that R j+1(l)(γ)

is bounded on Γ for l = 0,1. We furthermore assume that both the first-orderoptimizer γ0 and the exact optimizer γopt

s exist, are unique and lie in the interior

222

8.3. General revenue maximization

of Γ , and that R0(γ) is strictly concave on Γ and has a continuous derivativeR0′′(γ) on Γ . Finally, we assume that f (x) is such that

limγ↓γmin

∫ ∞

0

f (x)e−γx dx =∞, (8.15)

where γmin = infγ ∈ R|∫∞

0 f (x)e−γx dx <∞. Ref. [6, Chapter 6] discussesunder which conditions assumption (8.15) is satisfied, and it is satisfied forinstance for any admission control policy f (x) = 1[x ≤ η] with admissionthreshold η > 0, since

∫∞0 f (x)e−γx dx = (1− e−γη)/γ and γmin = −∞.

8.3.1 Optimality gaps

We now derive the optimality gaps for the general optimization problem in(8.12). Theorem 8.1 formalizes that the approximating solutions γ j,s are asymp-totically optimal through bounds for the optimality gaps, and that an approx-imation of order j yields a gap decay of order j + 1. With minor modifica-tions to the proof, the result also applies to minimization problems of the formminγ∈Γ Rs(γ).

Theorem 8.1. For j = 0,1, . . ., there exist constants M j and K j independent of sand s j ∈ N+ such that for all s ≥ s j ,

|Rs(γopts )− Rs(γ j,s)| ≤

qs M j

s( j+1)/2, |γ j,s − γopt

s | ≤K j

s( j+1)/2. (8.16)

Proof First, we will prove a monotonicity result, as well as the existence ofoptimizers.

Lemma 8.2. There is an s0 ∈ N+ such that for all s ≥ s0, the function R0(γ) +∑ ji=1 Ri(γ)/si/2 has a unique optimizer γ j,s ∈ Γ and a strictly decreasing derivative

R0′(γ) +

∑ ji=1 Ri

′(γ)/si/2.

Proof. Recall that γ0 lies in the interior of Γ and that R0′(γ) is strictly decreasing

on Γ by assumption, which implies that R0′(γl)> 0. We seek s1 ∈ N+ such that

for all s ≥ s1, R0′(γl) +

∑ ji=1 Ri

′(γl)/si/2 > 0. Note next that

1p

s

j∑i=1

|Ri′(γl)| ≥ −

j∑i=1

Ri′(γl)si/2

, s ∈ N+. (8.17)

223


For all s ∈ N+ for which R0′(γl) > (1/

ps)∑ j

i=1 |Ri′(γl)|, we have thus conse-

quently that R0′(γl)> −

∑ ji=1 Ri

′(γl)/si/2. We therefore pick

(8.18) s1 = ∑ j

i=1 |Ri′(γl)|

R0′(γl)

2£

to ensure that for all s ≥ s1, R0′(γl)+

∑ ji=1 Ri

′(γl)/si/2 > 0. A similar result holdsat γ= γr , i.e. we have R0

′(γr)< 0, and thus

(8.19) s2 = ∑ j

i=1 |Ri′(γr)|

R0′(γr)

2£

is such that for all s ≥ s2, R0′(γr) +

∑ ji=1 Ri

′(γr)/si/2 < 0. The function R0(γ) +∑ ji=1 Ri(γ)/si/2 thus has a unique optimizer γ j,s ∈ Γ .Finally we turn to proving the monotonicity property of R0

′(γ)+∑ j

i=1 Ri′(γ)/

si/2. By assumption, R0′′(γ)< 0 for all γ ∈ [γl ,γr]. Similar to before, set

(8.20) s3 = maxγ∈[γl ,γr ]

∑ ji=1 |Ri

′′(γ)|R0′′(γ)

2£,

and conclude that R0′′(γ) +

∑ ji=1 Ri

′′(γ)/si/2 < 0 for all s ≥ s3 and all γ ∈[γl ,γr]. Finish the proof by setting s0 =maxs1, s2, s3, and by noting that s0 isbounded.

Recall that the unique optimizer γopts exists, and lies in the interior of Γ .

Because γopts maximizes Rs(γ), we have therefore by suboptimality that

(8.21)

0≤ Rs(γopts )− Rs(γ j,s) =

qs

j∑i=0

Ri(γ j,s)

si/2− Rs(γ j,s)

−qs

j∑i=0

Ri(γopts )

si/2− Rs(γ

opts )+ qs

j∑i=0

Ri(γopts )

si/2−

j∑i=0

Ri(γ j,s)

si/2

,

and subsequently by expansion (8.9) that

(8.22)0≤ Rs(γ

opts )− Rs(γ j,s) = qs

j∑i=0

Ri(γopts )

si/2−

j∑i=0

Ri(γ j,s)

si/2

+ qs

R j+1(γopts )− R j+1(γ j,s)

s( j+1)/2+O

qs

s( j+2)/2

.

224

8.3. General revenue maximization

Since γ j,s maximizes∑ j

i=0 Ri(γ)/si/2, we have by suboptimality that the termwithin square brackets is negative, i.e.

j∑i=0

Ri(γopts )

si/2−

j∑i=0

Ri(γ j,s)

si/2≤ 0 (8.23)

for all s ∈ N0. Therefore, since qs > 0,

0≤ Rs(γopts )− Rs(γ j,s)≤ qs

R j+1(γopts )− R j+1(γ j,s)

s( j+1)/2+O

qs

s( j+2)/2

. (8.24)

Since R j+1(γ) is bounded on Γ by assumption, the first claim in (8.16) follows.

Corollary 8.3. There exist constants M j′ > 0 independent of s and s j

′ ∈ N+ suchthat for all s ≥ s j

′,

j∑

i=0

Ri′(γopt

s )

si/2−

j∑i=0

Ri′(γ j,s)

si/2

=

j∑i=0

Ri′(γopt

s )

si/2

≤ M j′

s( j+1)/2. (8.25)

Proof. Note that γ j,s the optimizer is of∑ j

i=0 Ri(γ)/si/2, and that consequently∑ ji=0 Ri

′(γ j,s)/si/2 = 0, which proves the leftmost equality.Next, we examine the asymptotic expansion of the derivative of Rs(γ), which

we have assumed is available and of the form (8.9). It follows that

qs

j∑i=0

Ri′(γopt

s )

si/2= R′s(γ

opts )− qs

R j+1′(γopt

s )

s( j+1)/2+O

qs

s( j+2)/2

(i)= −qs

R j+1′(γopt

s )

s( j+1)/2+O

qs

s( j+2)/2

,

(8.26)

since (i) γopts optimizes Rs(γ). Now (8.25) follows since R j+1

′(γ) is bounded onΓ .

We are now ready to establish the second claim in (8.16). Recall that for allj ∈ N+, and sufficiently large s, the function

∑ ji=0 Ri

′(γ)/s(i/2) is strictly decreas-ing in [γl ,γr], see Lemma 8.2. Note also that γ0,γopt

s ,γ j,s ∈ [γl ,γr]. The meanvalue theorem implies then that |

∑ ji=0 Ri

′(γopts )/s

(i/2) −∑ j

i=0 Ri′(γ j,s)/s(i/2)| ≥

m j,s|γopts − γ j,s| with

m j,s = − maxγ∈[γl ,γr ]

¦R0′′(γ) +

j∑i=1

Ri′′(γ)si/2

©, (8.27)

225


where we have also used that the function∑ j

i=0 Ri(γ)/s(i/2) is optimized by γ j,s.Combining with Corollary 8.3, it follows that

(8.28)M j′

s( j+1)/2≥

j∑i=0

Ri′(γopt

s )

si/2−

j∑i=0

Ri′(γ j,s)

si/2

≥ m j,s|γopts − γ j,s|,

which is almost the second claim in (8.16). What remains is to remove thedependency on s of m j,s.

To that end, remark that maxγ∈[γl ,γr ] R0′′(γ)< 0 by continuity and pointwise

negativity of R0′′(γ). Then, bound

(8.29)

m j,s = minγ∈[γl ,γr ]

¦−R0

′′(γ)−j∑

i=1

Ri′′(γ)si/2

©

≥ minγ∈[γl ,γr ]

−R0′′(γ)+

1p

smin

γ∈[γl ,γr ]

¦−

j∑i=1

Ri′′(γ)

s(i−1)/2

©

≥ minγ∈[γl ,γr ]

−R0′′(γ) −

1Æs j′

maxγ∈[γl ,γr ]

¦ j∑i=1

Ri′′(γ)

(s j′)(i−1)/2

©=: m j ,

and increase the value of s j′ if necessary to ensure that m j > 0.

Summarizing, we now have that M j′/s( j+1)/2 ≥ m j |γopt

s − γ j,s| for all s ≥ s j′.

Setting K j = M j′/m j completes the proof.

8.3.2 Dimensioning under a delay constraint

The approach of Section 8.3.1 can also be used to solve delay constrained di-mensioning problems. As an example, consider finding

(8.30) γopts = argγ∈Γ Ds(γ) = ε,

where ε ∈ (0,1). Since we have an asymptotic expansion of the form (8.9) forthe delay probability, see Appendix 8.A, instead of solving for γopt

s directly wecan obtain approximations γ j,s = argγ∈Γ j

∑ j

i=0 Di(γ)/si/2 = ε. The optimalitygaps can be calculated using a similar proof technique.

Corollary 8.4. For j = 0, 1, . . . there exist finite constants M j , K j > 0 independentof s and s j ∈ N+ such that for all s ≥ s j ,

(8.31) |Ds(γ j,s)− ε| ≤M j

s( j+1)/2, |γ j,s − γopt

s | ≤K j

s( j+1)/2.

226

8.4. Approaches to asymptotic dimensioning

8.4 Approaches to asymptotic dimensioning

8.4.1 Asymptotic revenue maximization with a threshold

We will now consider new approaches to asymptotic dimensioning in the con-text of optimizing the objective function (8.4). We start with considering therevenue maximization problem described in Section 8.2.1 while assuming thatthe threshold is fixed. This concretely requires us to maximize R0(γ) in (8.10)over γ given a fixed η <∞.

The accuracy of our asymptotic expansion as an approximation to the objec-tive function Rs(γ) is examined in Figure 8.1, which shows the function Rs(γ)with its first- and second-order approximations for a system of size s = 10. Weconclude that both approximations are remarkably accurate for this relativelysmall system. Near the optimizer γopt

s , the second-order approximation is almostindistinguishable from the objective function. The maximizer of the second-order approximation R0(γ) + R1(γ)/

ps is also closer to the maximizer of Rs(γ)

than the maximizer of the first-order approximation R0(γ) is. This illustratesthat including higher-order correction terms in the asymptotic expansion indeedreduces the optimality gap.

The absolute error |Rs(γ) − R0(γ) − R1(γ)/p

s| is plotted in Figure 8.2 asfunction of s for γ = 2. A fit is provided which confirms that the asymptoticexpansion is indeed accurate up to O(1/s), as suggested by the asymptotic ex-pansion in (8.9). The jumps in the data points are caused by rounding in theadmission control, since ps(k− s) = 1[k− s ≤ bη

psc].

We also examine optimality gaps in Figure 8.2, which shows first- and second-order optimality gaps. Again notice that jumps occur because of the roundingin the control policy. Furthermore, we have provided fits that confirm that theoptimality gap is of order O(1/

ps) when the asymptotic approximation is of

order O(1), and that the optimality is of order O(1/s) when the asymptoticapproximation is of order O(1/

ps).

8.4.2 Joint dimensioning and admission control

We now introduce joint dimensioning and admission control, which has not beenstudied in the QED literature. In [7, Chapter 7], it is proven that the admissioncontrol policy that maximizes the system’s revenue rate has a threshold struc-ture, and that for fixed γ <∞ there exists an optimal threshold level ηopt. In

227


−2 −1 0 1 2 3−1.5

−1

−0.5

0

R0(γ)

Rs(γ)

R0(γ) +R1(γ)p

s

γ

Figure 8.1: The function Rs(γ) as function of γ for a = 0.1, b = 1, and η= 2, fora system of size s = 10 with its first-order approximation R0(γ) (dashed curve)and its second-order approximation R0(γ) + R1(γ)/

ps (dotted curve).

Section 8.4.1, we fixed η <∞ and searched for γopt. We now consider theirjoint optimization, that is to find

(8.32) (γopt,ηopt) = arg maxγ∈R,η≥0

R0(γ,η).

We now illustrate numerically that joint dimensioning and admission controlprovides important improvements compared to optimizing over γ only as inSection 8.4.1. Table 8.1 displays solution pairs (γopt,ηopt) to the maximizationproblem in (8.32) for various ratios r1 = a/(a+d) and r2 = (a+d)/b. Note thatthese two ratios are sufficient to describe all possible optimization problems, i.e.that occur for different a, b, and d, which can for example be seen by dividing theobjective function in (8.11) by d and then noting that (a+d)/d = r2/(r2− r1r2)and b/d = 1/(r2 − r1r2). From Table 8.1, we see that the optimization problemis well-posed, since nondegenerate optimal pairs exist.

Table 8.2 contains the percentage improvements that can be achieved bysolving the joint dimensioning and admission control problem, compared tothe classical approach of finding γopt

∞ = argmaxγ≥0 Rs(γ,η =∞). We shouldnote that this concerns the maximization of the second-order term in (8.4), the

228


10 25 50 750

5·10−3

s

|Rs(γ)−

R0(γ)−

R1(γ)/p

s|

10 25 50 750

0.15

s

|γ0−γ

opt

s|,

and|γ

1,s−γ

opt

s|

Figure 8.2: (left) The absolute error |Rs(γ)− R0(γ)− R1(γ)/p

s| as function ofs for a = 0.1, b = 1, and η = 2, for a critically scaled system with γ = 2.Plotted also is the curve e(s) = c1 + c2/s

c3 with fit parameters c1 = 4.0 · 10−5,c2 = 7.2 · 10−2, and c3 = 1.1 (continuous). (right) The top data points give theoptimality gap |γ0 − γopt

s |, and the bottom data points |γ1,s − γopts |. The top fit

is for e(s) = c1 + c2/p

s with c1 = 2.3 · 10−4 and c2 = 4.9 · 10−1, the bottome(s) = c1 + c2/s with c1 = −1.0 · 10−5 and c2 = 1.3 · 10−3.

leading order cannot be influenced through optimization. Note also that γopt

can become negative.

8.4.3 Refined dimensioning

The QED refinements can also be used to derive refined dimensioning levels.The idea is that a higher-order asymptotic solution γ1,s can be expressed as afunction of the lower-order asymptotic solution γ0. To see this, consider thefollowing representation of γ1,s in the context of dimensioning under a delayconstraint as discussed in Section 8.3.2.

Theorem 8.5. For sufficiently large s, the first-order solution is given as γ1,s =

229


Table8.1:The

optimalthreshold,hedge

pair(γ

opt,ηopt)

fordifferent

ratiosof

a/(a+

d)and

(a+

d)/b.

a/(a+

d)(a+

d)/b1/5

1/41/3

1/21

23

45

0.1(1.9,0.4)

(1.9,0.5)(1.9,0.6)

(1.8,0.9)(1.6,1.6)

(1.4,2.9)(1.2,3.9)

(1.1,4.7)(1.1,5.5)

0.2(1.5,0.3)

(1.5,0.4)(1.5,0.5)

(1.4,0.7)(1.3,1.3)

(1.1,2.3)(1.0,3.1)

(0.9,3.8)(0.8,4.4)

0.3(1.2,0.3)

(1.2,0.3)(1.1,0.4)

(1.1,0.6)(1.0,1.1)

(0.8,2.0)(0.7,2.6)

(0.7,3.2)(0.6,3.7)

0.4(0.9,0.2)

(0.8,0.3)(0.8,0.4)

(0.8,0.5)(0.7,1.0)

(0.6,1.7)(0.6,2.3)

(0.5,2.8)(0.5,3.2)

0.5(0.5,0.2)

(0.5,0.2)(0.5,0.3)

(0.5,0.5)(0.5,0.8)

(0.4,1.5)(0.4,2.0)

(0.3,2.4)(0.3,2.8)

0.6(0.2,0.2)

(0.2,0.2)(0.2,0.3)

(0.2,0.4)(0.2,0.7)

(0.2,1.2)(0.2,1.7)

(0.1,2.0)(0.1,2.4)

0.7(-0.3,0.1)

(-0.3,0.2)(-0.3,0.2)

(-0.2,0.3)(-0.2,0.6)

(-0.2,1.0)(-0.1,1.4)

(-0.1,1.7)(-0.1,2.0)

0.8(-0.9,0.1)

(-0.9,0.1)(-0.9,0.2)

(-0.8,0.2)(-0.7,0.4)

(-0.6,0.8)(-0.5,1.1)

(-0.5,1.3)(-0.5,1.6)

0.9(-2.1,0.1)

(-2.1,0.1)(-2.1,0.1)

(-2.0,0.2)(-1.8,0.3)

(-1.5,0.5)(-1.4,0.7)

(-1.3,0.9)(-1.2,1.0)

Table8.2:Pairs

(γopt/γ

opt∞

,100·(R

opt−R

opt∞)/|R

opt∞|)

forratios

a/(a+

d)and

(a+

d)/b.

a/(a+

d)(a+

d)/b1/5

1/41/3

1/21

23

45

0.1(0.9,7)

(0.9,6)(0.9,5)

(0.9,3)(1.0,1)

(1.0,0)(1.0,0)

(1.0,0)(1.0,0)

0.2(0.8,13)

(0.8,11)(0.8,9)

(0.8,7)(0.9,4)

(0.9,2)(0.9,1)

(1.0,1)(1.0,1)

0.3(0.6,18)

(0.7,16)(0.7,14)

(0.7,11)(0.8,7)

(0.8,4)(0.9,3)

(0.9,3)(0.9,2)

0.4(0.5,23)

(0.5,22)(0.5,19)

(0.6,16)(0.6,11)

(0.7,8)(0.7,7)

(0.7,6)(0.7,5)

0.5(0.3,29)

(0.3,28)(0.4,25)

(0.4,22)(0.4,17)

(0.5,13)(0.5,11)

(0.5,10)(0.5,9)

0.6(0.1,36)

(0.1,34)(0.1,32)

(0.1,29)(0.2,23)

(0.2,19)(0.2,17)

(0.2,16)(0.2,15)

0.7(-0.2,44)

(-0.2,42)(-0.2,40)

(-0.2,37)(-0.2,31)

(-0.2,27)(-0.2,25)

(-0.2,23)(-0.2,22)

0.8(-0.6,54)

(-0.6,52)(-0.7,50)

(-0.7,47)(-0.8,42)

(-0.9,38)(-0.9,36)

(-0.9,34)(-1.0,33)

0.9(-1.5,67)

(-1.5,65)(-1.6,64)

(-1.8,62)(-2.0,58)

(-2.3,54)(-2.5,52)

(-2.6,51)(-2.6,50)

230


γ0 + γ0, where

γ0 =∞∑n=1

(−1)n

(n− 1)!

ddγ

n−1(A′(γ) + E′(γ))

γ

A(γ)

n+1(E(γ))n

γ=0

, (8.33)

and for small γ the auxiliary functions are defined as A(γ) = D0(γ+ D←0 (ε))− ε,and E(γ) = D1(γ+ D←0 (ε))/

ps.

Proof. We mimick the standard proof of Lagrange’s inversion theorem. Both Aand E are analytic in the neighborhood of γ= 0 by analyticity of Bs(γ) aroundγ = 0 and analyticity of F1(γ) in Reγ > γmin, with γmin < 0 by assumption.Also, A(0) = 0, A′(0) 6= 0.

By taking s sufficiently large, say s ≥ s0, we can arrange that there is an r > 0such that A′(0) + G′(0) 6= 0 and |E(γ)/A(γ) ≤ 1/2 for |γ| = r and s ≥ s0, whileγ = 0 is the only zero of F(γ) in |γ| ≤ r. By Rouché’s theorem, for any s ≥ s0

there is a unique γ0 in |γ| ≤ r such that A(γ0) + E(γ0) = 0. Thus the solutionγ1,s near γ0 of the equation D0(γ) + D1(γ)/

ps = ε is given as γ1,s = γ0 + γ0. By

Cauchy’s theorem, there is for γ0 the integral representation

γ0 =1

2πi

∫

|γ|=r

γ(A′(γ) + E′(γ))A(γ) + E(γ)

dγ. (8.34)

Since we have 0 6= |A(γ)| ≥ 2|E(γ)| on |γ|= r, it follows that

γ0 =∞∑n=0

(−1)n

2πi

∫

|γ|=r

γ(A′(γ) + E′(γ))(E(γ))n

(A(γ))n+1dγ. (8.35)

Due to analyticity of γ/A(γ), the term with n= 0 vanishes. For n= 1, 2, . . ., wefurthermore have that

12πi

∫

|γ|=r

γ(A′(γ) + E′(γ))(E(γ))n

(A(γ))n+1dγ

=1

2πi

∫

|γ|=r

1γn(A′(γ) + E′(γ))

γ

A(γ)

n+1(E(γ))n dγ

=1

(n− 1)!

ddγ

n−1(A′(γ) + E′(γ))

γ

A(γ)

n+1(E(γ))n

γ=0

.

(8.36)

This is the result in (8.33), and concludes the proof.

231


When using the first two terms in (8.33), we get for γ0 the approximation

(8.37) −EA′+

EE′

(A′)2−

A′′E2

2(A′)3−

3A′′E′E2

(A′)4+

E′′E2

(A′)3+

2E(E′)2

(A′)3,

with all functions evaluated at γ= 0. The first term gives an s−1/2-correction ofγ0, the second term gives an s−1-contribution, and other terms give contributionsof O(s−3/2) or smaller. Summarizing, we thus have that

(8.38) γ1,s = γ0 + γ0 = γ0 −1p

sD1(γ0)D0′(γ0)

+O1

s

.

Appendix

8.A Higher-order terms in asymptotic expansions

We now provide closed-form expressions for the asymptotic expansion in (8.9).We drop the dependence on γ for notational convenience, and prove the follow-ing result.

Theorem 8.6. As s→∞, (Rs − ns)/qs = R0 + R1/p

s+O(1/s), where

(8.39) R0 =W L

0 +W R0

B0 + F0, R1 =

W L1 +W R

1

B0 + F0−(W L

0 +W R0 )(B1 + F1)

(B0 + F0)2,

and

W L0 =

∫ 0

−∞r(x)e−

12 x2−γx dx ,

W L1 =

12

∫ 0

−∞( 1

3 x3 − (1+ γ2)x)r(x)e−12 x2−γx dx + r(0),

W R0 =

∫ ∞

0

r(x) f (x)e−γx dx ,

W R1 = −

12γ

2

∫ ∞

0

x r(x) f (x)e−γx dx − 12 r(0) f (0),(8.40)

as well as

B0 =Φ(γ)φ(γ)

, B1 =13

2+ γ2 + γ3 Φ(γ)

φ(γ)

,(8.41)

F0 =

∫ ∞

0

f (x)e−γx dx , F1 = −12γ

2

∫ ∞

0

x f (x)e−γx dx − 12 f (0).

232

8.A. Higher-order terms in asymptotic expansions

Proof. Note after substituting (8.2) into Rs =∑∞

k=0 rs(k)πs(k), that asymptoti-cally

Rs − ns

qs=

∑sk=0 r

k−sp

s

(sρ)kk! +

(sρ)s

s!

∑∞k=s+1 r

k−sp

s

ρk−s f

k−sp

s

∑s

k=0(sρ)k

k! +(sρ)s

s!

∑∞k=s+1ρ

k−s f

k−sps

. (8.42)

Dividing by the factor (sρ)s/s!, we obtain the form

Rs − ns

qs=

W Ls +W R

s

B−1s + Fs

, (8.43)

where we have introduced notation for the Erlang B formula,

Bs(ρ) =(sρ)s

s!∑sk=0

(sρ)kk!

, (8.44)

and we have defined

Fs =∞∑n=0

ρn+1 fn+ 1p

s

, W L

s =s∑

k=0

rk− sp

s

s!(sρ)k−s

k!, (8.45)

and

W Rs =

∞∑n=0

rn+ 1p

s

ρn+1 f

n+ 1p

s). (8.46)

In [6, 7, Chapters 6, 7], it is proven using Jagerman’s asymptotic expansions[118] for Erlang B’s formula that asymptotically,

W Ls =p

sW L0 +W L

1 +O 1p

s

, W R

s =p

sW R0 +W R

1 +O 1p

s

,

B−1s =

psB0 + B1 +O

1p

s

, Fs =

psF0 + F1 +O

1p

s

, (8.47)

with the coefficients as given in (8.40) and (8.41). After substituting theseasymptotic expansions into (8.43), we obtain

Rs − ns

qs=

W L0 +W R

0

B0 + F0·

1+ 1ps (W

L1 +W R

1 )/(WL0 +W R

0 ) +O( 1s )

1+ 1ps (B1 + F1)/(B0 + F0) +O( 1

s ). (8.48)

By then utilizing the Taylor expansion 1/(1+ x) = 1− x +O(x2), we obtain theresult.

233


Delay probability The delay probability Ds =∑∞

k=sπs(k) can be representedby rs(k) = 1[k ≥ s], recall (8.3). This corresponds asymptotically to r(x) =1[x ≥ 0]. introduce for convenience L =

∫∞0 f (x)e−γx dx . Then, W L

0 =W L1 =

0, W R0 = L , W R

1 =12γ

2L ′, F0 = L , and F1 =12γ

2L ′ − 12 . It follows that D0 =

L /(Φ(γ)/φ(γ) +L ) and

(8.49) D1 =12γ

2L ′

Φ(γ)φ(γ) +L

−L ( 1

3

2+ γ2 + γ3 Φ(γ)

φ(γ)

+ 1

2γ2L ′ − 1

2 )

( Φ(γ)φ(γ) +L )2.

Queue length The mean queue length Qs =∑∞

k=s(k − s)πs(k) can be rep-resented by rs(k) = (k − s)1[k ≥ s]. Scaling so that Qs/

ps =

∑∞k=s((k −

s)/p

s)πs(k), we see that the revenue structure can asymptotically be related tothe revenue profile r(x) = x1[x ≥ 0]. Therefore, W L

0 = W L1 = 0, W R

0 = −L′,

W R1 = −

12γ

2L ′′, F0 = L , and F1 =12γ

2L ′ − 12 . Thus Qs/

ps = Q0 +Q1/

ps +

O(1/s) with Q0 = −L ′/(Φ(γ)/φ(γ) +L ) and

(8.50) Q1 = −12γ

2L ′

Φ(γ)φ(γ) +L

+L ′( 1

3

2+ γ2 + γ3 Φ(γ)

φ(γ)

+ 1

2γ2L ′ − 1

2 )

( Φ(γ)φ(γ) +L )2.

234

Bibliography

[1] J. Sanders. “Interacting particles in wireless networks and Rydberg gasses”. Mas-ter Thesis. Eindhoven University of Technology, 2011.

[2] J. Sanders, S. C. Borst, and J. S. H. van Leeuwaarden. “Online network opti-mization using product-form Markov processes”. In: IEEE Trans. Autom. ControlPreprint (2015).

[3] J. Sanders, R. van Bijnen, E. Vredenbregt, and S. Kokkelmans. “Wireless NetworkControl of Interacting Rydberg Atoms”. In: Phys. Rev. Lett. 112 (2014), p. 163001.

[4] J. Sanders, M. Jonckheere, and S. Kokkelmans. “Sub-Poissonian Statistics ofJamming Limits in Ultracold Rydberg Gases”. In: Phys. Rev. Lett. 115 (2015),p. 043002.

[5] P. Bermolen, M. Jonckheere, and J. Sanders. Scaling Limits for Exploration Algo-rithms. Tech. rep. 2015.

[6] A. J. E. M. Janssen, J. S. H. van Leeuwaarden, and J. Sanders. “Scaled controlin the QED regime”. In: Performance Evaluation 70 (2013), pp. 750–769.

[7] J. Sanders, S. C. Borst, A. J. E. M. Janssen, and J. van Leeuwaarden. “Optimaladmission control for many-server systems with QED-driven revenues”. In: ArXive-prints (2014).

[8] J. Sanders, S. C. Borst, A. J. E. M. Janssen, and J. van Leeuwaarden. “Optimalitygaps in asymptotic dimensioning of many-server systems”. In: ArXiv e-prints(2015).

[9] J. Sanders, S. C. Borst, and J. S. H. van Leeuwaarden. “Achievable performancein product-form networks”. In: 2012 50th Annual Allerton Conference on Com-munication, Control, and Computing (Allerton). 2012, pp. 928–935.

[10] J. Sanders, S. C. Borst, and J. S. H. van Leeuwaarden. “Online optimization ofproduct-form networks”. In: 2012 6th International Conference on PerformanceEvaluation Methodologies and Tools (VALUETOOLS). 2012, pp. 21–30.

[11] T. M. Liggett. Interacting Particle Systems. Springer, 1985.

[12] T. M. Liggett. “Stochastic models for large interacting systems and related corre-lation inequalities”. In: PNAS 107 (2010), p. 16413.

235

B I B L I O G R A P H Y

[13] F. P. Kelly. “Stochastic Models of Computer Communication Systems”. In: J. Roy.Stat. Soc. B Met. 47 (1985), pp. 379–395.

[14] R. R. Boorstyn, A. Kershenbaum, B. Maglaris, and V. Sahin. “Throughput Analysisin Multihop CSMA Packet Radio Networks”. In: IEEE T. Commun. 35 (1987).

[15] A. Kershenbaum, R. R. Boorstyn, and M. S. Chen. “An Algorithm for Evaluationof Throughput in Multihop Packet Radio Networks with Complex Topologies”.In: IEEE J. Sel. Areas Commun. 5 (1987).

[16] P. Robert. Stochastic Networks and Queues. Springer, 2003.

[17] F. P. Kelly. Reversibility and Stochastic Networks. Cambridge University Press,2011.

[18] N. Abramson. “THE ALOHA SYSTEM: Another Alternative for Computer Com-munications”. In: Proceedings of the November 17-19, 1970, Fall Joint ComputerConference. 1970, pp. 281–285.

[19] L. Kleinrock and F. Tobagi. “Packet Switching in Radio Channels: Part I. CarrierSense Multiple-Access Modes and Their Throughput-Delay Characteristics”. In:IEEE Transactions on Communications 23 (1975), pp. 1400–1416.

[20] P. van de Ven. “Stochastic Models for Resource Sharing in Wireless Networks”.PhD Thesis. Eindhoven University of Technology, 2011.

[21] N. Bouman. “Queue-Based Random Access in Wireless Networks”. PhD Thesis.Eindhoven University of Technology, 2013.

[22] L. Jiang and J. Walrand. “A distributed CSMA algorithm for throughput andutility maximization in wireless networks”. In: Proceedings of the 46th AnnualAllerton Conference on Communication, Control, and Computing. 2008.

[23] A. Zocca. “Spatio-temporal dynamics of random-access networks: an interactingparticle approach”. PhD Thesis. Eindhoven University of Technology, 2015.

[24] T. F. Gallagher. Rydberg Atoms (Cambridge Monographs on Atomic, Molecular andChemical Physics). Cambridge University Press, 1994.

[25] R. P. Feynman, R. B. Leighton, and M. L. Sands. The Feynman Lectures on Physics.Pearson/Addison-Wesley, 1963.

[26] M. D. Lukin, M. Fleischhauer, R. Côté, L. M. Duan, D. Jaksch, J. I. Cirac, andP. Zoller. “Dipole Blockade and Quantum Information Processing in MesoscopicAtomic Ensembles”. In: Phys. Rev. Lett. 87 (2001), p. 037901.

[27] D. Comparat and P. Pillet. “Dipole blockade in a cold Rydberg atomic sample”.In: J. Opt. Soc. Am. B 27 (2010), A208–A232.

[28] F. Robicheaux and J. V. Hernández. “Many-body wave function in a dipole block-ade configuration”. In: Phys. Rev. A 72 (2005), p. 063403.

236

Bibliography

[29] P. Schauss, M. Cheneau, M. Endres, T. Fukuhara, S. Hild, A. Omran, T. Pohl, C.Gross, S. Kuhr, and I. Bloch. “Observation of spatially ordered structures in atwo-dimensional Rydberg gas”. In: Nature 491 (2012), p. 87.

[30] T. Cubel Liebisch, A. Reinhard, P. R. Berman, and G. Raithel. “Atom CountingStatistics in Ensembles of Interacting Rydberg Atoms”. In: Phys. Rev. Lett. 95(2005), p. 253002.

[31] M. Viteau, P. Huillery, M. G. Bason, N. Malossi, D. Ciampini, O. Morsch, E. Ari-mondo, D. Comparat, and P. Pillet. “Cooperative Excitation and Many-Body In-teractions in a Cold Rydberg Gas”. In: Phys. Rev. Lett. 109 (2012), p. 053002.

[32] C. S. Hofmann, G. Günter, H. Schempp, M. Robert-de-Saint-Vincent, M. Gärttner,J. Evers, S. Whitlock, and M. Weidemüller. “Sub-Poissonian Statistics of Rydberg-Interacting Dark-State Polaritons”. In: Phys. Rev. Lett. 110 (2013), p. 203601.

[33] N. Malossi, M. M. Valado, S. Scotto, P. Huillery, P. Pillet, D. Ciampini, E. Arimondo,and O. Morsch. “Full Counting Statistics and Phase Diagram of a DissipativeRydberg Gas”. In: Phys. Rev. Lett. 113 (2014), p. 023006.

[34] H. Schempp, G. Günter, M. Robert-de Saint-Vincent, C. S. Hofmann, D. Breyel,A. Komnik, D. W. Schönleber, M. Gärttner, J. Evers, S. Whitlock, and M. Wei-demüller. “Full Counting Statistics of Laser Excited Rydberg Aggregates in aOne-Dimensional Geometry”. In: Phys. Rev. Lett. 112 (2014), p. 013002.

[35] P. Schauß, J. Zeiher, T. Fukuhara, S. Hild, M. Cheneau, T. Macrì, T. Pohl, I. Bloch,and C. Gross. “Crystallization in Ising quantum magnets”. In: Science 347 (2015),pp. 1455–1458.

[36] M. D. Penrose. “Random Parking, Sequential Adsorption, and the JammingLimit”. In: Commun. Math. Phys. 218 (2001), pp. 153–176.

[37] J. W. Evans. “Random and cooperative sequential adsorption”. In: Rev. Mod. Phys.65 (1993), pp. 1281–1329.

[38] P. Erdös and A. Rényi. “On random graphs, I”. In: Publicationes Mathematicae(Debrecen) 6 (1959), pp. 290–297.

[39] F. P. Kelly. “Loss Networks”. In: Annals of Applied Probability 1 (1991), pp. 319–378.

[40] K. Jung, Y. Lu, D. Shah, M. Sharma, and M. S. Squillante. “Revisiting StochasticLoss Networks: Structures and Algorithms”. In: Proceedings of the 2008 ACMSIGMETRICS International Conference on Measurement and Modeling of ComputerSystems. 2008, pp. 407–418.

[41] F. Baskett, K. M. Chandy, R. R. Muntz, and F. G. Palacios. “Open, Closed, andMixed Networks of Queues with Different Classes of Customers”. In: J. ACM 22(1975), pp. 248–260.

237


[42] X. Wang and K. Kar. “Throughput modelling and fairness issues in CSMA/CAbased ad-hoc networks”. In: Proceedings of the 24th Annual Joint Conference ofthe IEEE Computer and Communications Societies (INFOCOM). 2005.

[43] S.-C. Liew, C. Kai, J. Leung, and B. Wong. “Back-of-the-Envelope Computationof Throughput Distributions in CSMA Wireless Networks”. In: IEEE InternationalConference on Communications, 2009. ICC ’09. 2009, pp. 1–6.

[44] P. Bremaud. Markov Chains: Gibbs Fields, Monte Carlo Simulation, and Queues.Springer, 1999.

[45] S. P. Boyd and L. Vandenberghe. Convex Optimization. Cambridge UniversityPress, 2004.

[46] G. Louth, M. Mitzenmacher, and F. Kelly. “Computational complexity of lossnetworks”. In: Theoretical Computer Science 125 (1994), pp. 45–59.

[47] H. J. Kushner and G. Yin. Stochastic Approximation and Recursive Algorithms andApplications. Springer, 2003.

[48] V. S. Borkar. Stochastic Approximation: A Dynamical Systems Viewpoint. Cam-bridge University Press, 2008.

[49] L. Jiang and J. Walrand. “Convergence and stability of a distributed CSMA algo-rithm for maximal network throughput”. In: Proceedings of the 48th IEEE Con-ference on Decision and Control, 2009 held jointly with the 2009 28th ChineseControl Conference. CDC/CCC 2009. 2009, pp. 4840–4845.

[50] P. Diaconis and D. Stroock. “Geometric Bounds for Eigenvalues of Markov Chains”.In: Ann. Appl. Probab. 1 (1991), pp. 36–61.

[51] D. A. Levin, Y. Peres, and E. L. Wilmer. Markov Chains and Mixing Times. AmericanMathematical Soc., 2009.

[52] S. Halfin and W. Whitt. “Heavy-Traffic Limits for Queues with Many ExponentialServers”. In: Operations Research 29 (1981), pp. 567–588.

[53] G. Grimmett and D. Stirzaker. Probability and Random Processes. Oxford Univer-sity Press, 2001.

[54] G. Falin and J. Artalejo. “Approximations for multiserver queues with balk-ing/retrial discipline”. In: Operations-Research-Spektrum 17 (1995), pp. 239–244.

[55] D. Elliott. “The Euler-Maclaurin formula revisited”. In: ANZIAM Journal 40(1998), pp. 27–76.

[56] D. G. Luenberger. Optimization by Vector Space Methods. John Wiley & Sons,1969.

[57] W. Rudin. Real and Complex Analysis. Tata McGraw-Hill, 1987.

238

Bibliography

[58] R. M. Corless, G. H. Gonnet, D. E. G. Hare, D. J. Jeffrey, and D. E. Knuth. “Onthe Lambert W Function”. In: Advances in Computational Mathematics. 1996,pp. 329–359.

[59] P. Marbach and J. Tsitsiklis. “Simulation-based optimization of Markov rewardprocesses”. In: IEEE Transactions on Automatic Control 46 (2001), pp. 191–209.

[60] P. Marbach and J. N. Tsitsiklis. “Approximate Gradient Methods in Policy-SpaceOptimization of Markov Reward Processes”. In: Discrete Event Dynamic Systems13 (2003), pp. 111–148.

[61] D. P. Bertsekas. Dynamic Programming and Optimal Control. Athena Scientific,1995.

[62] M. L. Puterman. Markov Decision Processes: Discrete Stochastic Dynamic Program-ming. John Wiley & Sons, 2009.

[63] M. T. Heath. Scientific Computing: An Introductory Survey. McGraw-Hill, 2005.

[64] A. Harel. “Convexity Properties of the Erlang Loss Formula”. In: OperationsResearch 38 (1990), pp. 499–505.

[65] M. J. Wainwright and M. I. Jordan. “Graphical Models, Exponential Families,and Variational Inference”. In: Found. Trends Mach. Learn. 1 (2008), pp. 1–305.

[66] J. M. Steele. Stochastic Calculus and Financial Applications. Springer, 2001.

[67] P. Cattiaux and A. Guillin. “Deviation bounds for additive functionals of Markovprocesses”. In: ESAIM: Probability and Statistics 12 (2008), pp. 12–29.

[68] R. J. Glauber. “Time dependent statistics of the Ising model”. In: J. Math. Phys.4 (1963), p. 294.

[69] D. Jaksch, J. I. Cirac, P. Zoller, S. L. Rolston, R. Côté, and M. D. Lukin. “FastQuantum Gates for Neutral Atoms”. In: Phys. Rev. Lett. 85 (2000), pp. 2208–2211.

[70] M. Saffman, T. G. Walker, and K. Mølmer. “Quantum information with Rydbergatoms”. In: Rev. Mod. Phys. 82 (2010), pp. 2313–2363.

[71] H. Weimer, R. Löw, T. Pfau, and H. P. Büchler. “Quantum Critical Behavior inStrongly Interacting Rydberg Gases”. In: Phys. Rev. Lett. 101 (2008), p. 250601.

[72] M. A. Nielsen and I. L. Chuang. Quantum Computation and Quantum Information:10th Anniversary Edition. Cambridge University Press, 2011.

[73] P. Schauss, M. Cheneau, M. Endres, T. Fukuhara, S. Hild, A. Omran, T. Pohl, C.Gross, S. Kuhr, and I. Bloch. “Observation of spatially ordered structures in atwo-dimensional Rydberg gas”. In: Nature 491 (2012), p. 87.

[74] T. Pohl, E. Demler, and M. D. Lukin. “Dynamical Crystallization in Dipole Block-ade of Ultracold Atoms”. In: Phys. Rev. Lett. 104 (2010), p. 043002.

239


[75] J. Schachenmayer, I. Lesanovsky, A. Micheli, and A. J. Daley. “Dynamical crystalcreation with polar molecules or Rydberg atoms in optical lattices”. In: NewJournal of Physics 12 (2010), p. 103044.

[76] R. M. W. van Bijnen, S. Smit, K. A. H. van Leeuwen, E. J. D. Vredenbregt, andS. J. J. M. F. Kokkelmans. “Adiabatic formation of Rydberg crystals with chirpedlaser pulses”. In: J. Phys. B: At. Mol. Opt. Phys. 44 (2011), p. 184008.

[77] M. Höning, D. Muth, D. Petrosyan, and M. Fleischhauer. “Steady-state crystal-lization of Rydberg excitations in an optically driven lattice gas”. In: Phys. Rev.A 87 (2013), p. 023401.

[78] C. Ates, T. Pohl, T. Pattard, and J. M. Rost. “Many-body theory of excitationdynamics in an ultracold Rydberg gas”. In: Phys. Rev. A 76 (2007), p. 013413.

[79] A. Hu, T. E. Lee, and C. W. Clark. “Spatial correlations of one-dimensional driven-dissipative systems of Rydberg atoms”. In: Phys. Rev. A 88 (2013), p. 053627.

[80] D. Petrosyan. “Two-dimensional crystals of Rydberg excitations in a resonantlydriven lattice gas”. In: Phys. Rev. A 88 (2013), p. 043431.

[81] P. M. van de Ven, J. S. H. van Leeuwaarden, D. Denteneer, and A. J. E. M. Janssen.“Spatial fairness in linear random-access networks”. In: Perform. Evaluation 69(2012).

[82] M. Siomau and S. Fritzsche. “Quantum computing with mixed states”. In: Eur.Phys. J. D 62 (2011), pp. 449–456.

[83] R. Blume-Kohout, C. M. Caves, and I. H. Deutsch. “Climbing Mount Scalable:Physical Resource Requirements for a Scalable Quantum Computer”. In: Found.Phys. 32 (2002), pp. 1641–1670.

[84] M. Durvy, O. Dousse, and P. Thiran. “Modeling the 802.11 Protocol Under Dif-ferent Capture and Sensing Capabilities”. In: Proceedings of the 26th IEEE Inter-national Conference on Computer Communications (INFOCOM). 2007.

[85] M. Durvy, O. Dousse, and P. Thiran. “Self-Organization Properties of CSMA/CASystems and Their Consequences on Fairness”. In: IEEE T. Inform. Theory 55(2009).

[86] D. Denteneer, S. C. Borst, P. van de Ven, and G. Hiertz. “IEEE 802.11s and thePhilosophers problem”. In: Stat. Neerl. 62 (2008).

[87] D. Jaksch, J. I. Cirac, P. Zoller, S. L. Rolston, R. Côté, and M. D. Lukin. “FastQuantum Gates for Neutral Atoms”. In: Phys. Rev. Lett. 85 (2000), pp. 2208–2211.

[88] L. Mandel. “Sub-Poissonian photon statistics in resonance fluorescence”. In: Opt.Lett. 4 (1979), pp. 205–207.

[89] C. Ates and I. Lesanovsky. “Entropic enhancement of spatial correlations in alaser-driven Rydberg gas”. In: Phys. Rev. A 86 (2012), p. 013408.

240

Bibliography

[90] R. M. Karp. “The transitive closure of a random digraph”. In: Random Struct.Algor. 1 (1990), pp. 73–93.

[91] P. Bermolen, M. Jonckheere, and P. Moyal. “The Jamming Constant of RandomGraphs”. In: arXiv:1310.8475 [math] (2013).

[92] I. Lesanovsky and J. P. Garrahan. “Out-of-equilibrium structures in strongly in-teracting Rydberg gases with dissipation”. In: Phys. Rev. A 90 (2014), p. 011603.

[93] A. Urvoy, F. Ripka, I. Lesanovsky, D. Booth, J. P. Shaffer, T. Pfau, and R. Löw.“Strongly Correlated Growth of Rydberg Aggregates in a Vapor Cell”. In: Phys.Rev. Lett. 114 (2015), p. 203002.

[94] P. Bermolen, M. Jonckheere, F. Larroca, and P. Moyal. “Estimating the SpatialReuse with Configuration Models”. In: CoRR abs/1411.0143 (2014).

[95] M. Frolkova, S. Foss, and B. Zwart. “Fluid Limits for an ALOHA-type Model withImpatient Customers”. In: Queueing Syst. Theory Appl. 72 (2012), pp. 69–101.

[96] R. Darling and J. Norris. “Differential equation approximations for Markov chains”.In: Probability Surveys 5 (2008), pp. 37–79.

[97] T. G. Kurtz. “Strong approximation theorems for density dependent Markovchains”. In: Stochastic Processes and their Applications 6 (1978), pp. 223 –240.

[98] J. Komlós, P. Major, and G. Tusnády. “An approximation of partial sums of inde-pendent RV’-s, and the sample DF. I”. In: Zeitschrift für Wahrscheinlichkeitstheorieund Verwandte Gebiete 32 (1975), pp. 111–131.

[99] J. Komlós, P. Major, and G. Tusnády. “An approximation of partial sums of inde-pendent RV’s, and the sample DF. II”. In: Zeitschrift für Wahrscheinlichkeitstheorieund Verwandte Gebiete 34 (1976), pp. 33–58.

[100] I. Berkes, W. Liu, and W. B. Wu. “Komlós–Major–Tusnády approximation underdependence”. In: Ann. Probab. 42 (2014), pp. 794–817.

[101] K. Teerapabolaan. “A bound on the binomial–Poisson relative error”. In: Inter-national Journal of Pure and Applied Mathematics 87 (2013), pp. 535 –540.

[102] O. Garnett, A. Mandelbaum, and M. Reiman. “Designing a Call Center with Impa-tient Customers”. In: Manufacturing & Service Operations Management 4 (2002),pp. 208–227.

[103] P. Jelenkovic, A. Mandelbaum, and P. Momcilovic. “Heavy Traffic Limits for Queueswith Many Deterministic Servers”. In: Queueing Syst. Theory Appl. 47 (2004),pp. 53–69.

[104] C. Maglaras and A. Zeevi. “Diffusion Approximations for a Multiclass MarkovianService System with "Guaranteed" and "Best-Effort" Service Levels”. In: Mathe-matics of Operations Research 29 (2004), pp. 786–813.

[105] A. Mandelbaum and P. Momcilovic. “Queues with Many Servers and ImpatientCustomers”. In: Mathematics of Operations Research 37 (2012), pp. 41–65.

241


[106] J. Reed. “The G/GI/N queue in the Halfin-Whitt regime”. In: Annals of AppliedProbability 19 (2009), pp. 2211–2269.

[107] S. C. Borst, A. Mandelbaum, and M. I. Reiman. “Dimensioning Large Call Cen-ters”. In: Operations Research 52 (2004), pp. 17–34.

[108] N. Gans, G. Koole, and A. Mandelbaum. “Telephone Call Centers: Tutorial, Review,and Research Prospects”. In: Manufacturing & Service Operations Management 5(2003), pp. 79–141.

[109] M. Armony and C. Maglaras. “On Customer Contact Centers with a Call-BackOption: Customer Decisions, Routing Rules, and System Design”. In: OperationsResearch 52 (2004), pp. 271–292.

[110] W. Massey and R. Wallace. An asymptotically optimal design of M/M/c/k queue.Tech. rep. 2005.

[111] W. Whitt. “Heavy-traffic limits for the G/H∗2/n/m queue”. In: Mathematics ofOperations Research 30 (2005), pp. 1–27.

[112] W. Whitt. “A Diffusion Approximation for the G/GI/n/m Queue”. In: OperationsResearch 52 (2004), pp. 922–941.

[113] A. J. E. M. Janssen, J. S. H. V. Leeuwaarden, and B. Zwart. “Gaussian Expansionsand Bounds for the Poisson Distribution Applied to the Erlang B Formula”. In:Advances in Applied Probability 40 (2008), pp. 122–143.

[114] A. J. E. M. Janssen, J. S. H. van Leeuwaarden, and B. Zwart. “Refining Square-Root Safety Staffing by Expanding Erlang C”. In: Operations Research 59 (2011),pp. 1512–1522.

[115] B. Zhang, J. S. H. van Leeuwaarden, and B. Zwart. “Staffing Call Centers withImpatient Customers: Refinements to Many-Server Asymptotics”. In: OperationsResearch 60 (2012), pp. 461–474.

[116] A. J. E. M. Janssen and J. S. H. van Leeuwaarden. “Staffing many-server systemswith admission control and retrials”. In: arXiv:1302.3006 [math] (2013).

[117] S. Browne and W. Whitt. J. Dshalalow’s Advances in Queueing Theory, Methods,and Open Problems: Piecewise-Linear Diffusion Processes. Taylor & Francis, 1995.

[118] D. L. Jagerman. “Some Properties of the Erlang Loss Function”. In: Bell SystemTechnical Journal 53 (1974), pp. 525–551.

[119] F. W. J. Olver. NIST Handbook of Mathematical Functions. Cambridge UniversityPress, 2010.

[120] N. M. Temme. “Uniform Asymptotic Expansions of Confluent HypergeometricFunctions”. In: IMA Journal of Applied Mathematics 22 (1978), pp. 215–223.

[121] D. L. Iglehart. “Weak convergence in applied probability”. In: Stochastic Processesand their Applications 2 (1974), pp. 211–241.

242

Bibliography

[122] J. Lyness. “The Euler Maclaurin expansion for the Cauchy Principal Value inte-gral”. In: Numerische Mathematik 46 (1985), pp. 611–622.

[123] F. Hildebrand. Introduction to Numerical Analysis. McGraw-Hill, New York, 1956.

[124] S. A. Lippman. “Applying a New Device in the Optimization of ExponentialQueuing Systems”. In: Operations Research 23 (1975), pp. 687–710.

[125] S. Stidham. “Optimal control of admission to a queueing system”. In: IEEE Trans-actions on Automatic Control 30 (1985), pp. 705–713.

[126] H. J. Kushner and P. Dupuis. Numerical Methods for Stochastic Control Problemsin Continuous Time. Springer Science & Business Media, 2001.

[127] S. P. Meyn. Control Techniques for Complex Networks. Cambridge University Press,2008.

[128] E. B. Çil, E. L. Örmeci, and F. Karaesmen. “Effects of system parameters on theoptimal policy structure in a class of queueing control problems”. In: QueueingSystems 61 (2009), pp. 273–304.

[129] P. de Waal. “Overload Control of Telephone Exchanges”. PhD thesis. KatholiekeUniversiteit Brabant, 1990.

[130] H. Chen and M. Z. Frank. “State Dependent Pricing with a Queue”. In: IIE Trans-actions 33 (2001), pp. 847–860.

[131] R. Bekker and S. C. Borst. “Optimal admission control in queues with workload-dependent service rates”. In: Probability in the Engineering and InformationalSciences 20 (2006), pp. 543–570.

[132] P. Naor. “The Regulation of Queue Size by Levying Tolls”. In: Econometrica 37(1969), pp. 15–24.

[133] U. Yildirim and J. J. Hasenbein. “Admission control and pricing in a queue withbatch arrivals”. In: Operations Research Letters 38 (2010), pp. 427–431.

[134] C. Borgs, J. T. Chayes, S. Doroudi, M. Harchol-Balter, and K. Xu. “The optimaladmission threshold in observable queues with state dependent pricing”. In:Probability in the Engineering and Informational Sciences 28 (2014), pp. 101–119.

[135] A. P. Ghosh and A. P. Weerasinghe. “Optimal buffer size for a stochastic processingnetwork in heavy traffic”. In: Queueing Systems 55 (2007), pp. 147–159.

[136] A. R. Ward and S. Kumar. “Asymptotically Optimal Admission Control of a Queuewith Impatient Customers”. In: Mathematics of Operations Research 33 (2008),pp. 167–202.

[137] A. P. Ghosh and A. P. Weerasinghe. “Optimal buffer size and dynamic rate controlfor a queueing system with impatient customers in heavy traffic”. In: StochasticProcesses and their Applications 120 (2010), pp. 2103–2141.

243


[138] R. Atar, A. Mandelbaum, and M. I. Reiman. “A Brownian control problem fora simple queueing system in the Halfin–Whitt regime”. In: Systems & ControlLetters 51 (2004), pp. 269–275.

[139] R. Atar. “Scheduling control for queueing systems with many servers: Asymptoticoptimality in heavy traffic”. In: Annals of Applied Probability 15 (2005), pp. 2606–2650.

[140] R. Atar. “A Diffusion Model of Scheduling Control in Queueing Systems withMany Servers”. In: Annals of Applied Probability 15 (2005), pp. 820–852.

[141] R. Atar, A. Mandelbaum, and G. Shaikhet. “Queueing systems with many servers:Null controllability in heavy traffic”. In: Annals of Applied Probability 16 (2006),pp. 1764–1804.

[142] I. Gurvich and W. Whitt. “Queue-and-Idleness-Ratio Controls in Many-ServerService Systems”. In: Mathematics of Operations Research 34 (2009), pp. 363–396.

[143] Y. L. Koçaga and A. R. Ward. “Admission control for a multi-server queue withabandonment”. In: Queueing Systems 65 (2010), pp. 275–323.

[144] A. Weerasinghe and A. Mandelbaum. “Abandonment versus blocking in many-server queues: Asymptotic optimality in the QED regime”. In: Queueing Systems75 (2013), pp. 279–337.

[145] F. Avram, A. J. E. M. Janssen, and J. S. H. V. Leeuwaarden. “Loss systems withslow retrials in the Halfin–Whitt regime”. In: Advances in Applied Probability 45(2013), pp. 274–294.

[146] G. Teschl. Topics in Real and Functional Analysis. 2014.

[147] G. Szegö. Orthogonal Polynomials. American Mathematical Society, 1939.

[148] R. S. Randhawa. “The Optimality Gap of Asymptotically-derived Prescriptionswith Applications to Queueing Systems”. In: arXiv:1210.2706 [math] (2012).

[149] J. Bonnans and A. Shapiro. Perturbation Analysis of Optimization Problems. SpringerNew York, 2000.

244

Summary

This thesis develops analysis techniques and optimization procedures that arebroadly applicable to large-scale complex systems. The focus is on probabilisticmodels of interacting particle systems, stochastic networks, and service systems,which are all large systems and display fascinatingly complex behavior. In prac-tice, these systems obey simple local rules leading to Markov processes that areamenable to analyses that shed light on the interplay between the local rulesand the global system behavior. Chapter 1 provides an overview of this thesis’scontent, and illustrates our approaches of analysis and optimization.

Chapter 2 deals with the topic Control and optimization of large-scale stochas-tic networks. There, we develop optimization algorithms that are applicable tothe whole class of product-form Markov processes. These algorithms can beimplemented in such a way that individual components of stochastic networksmake autonomous decisions that ultimately lead to globally optimal networkbehavior, and can for instance be used to balance a network of queues, i.e. toachieve equal average queue lengths. The algorithm does so by solving an inver-sion problem in an online fashion: every queue individually adapts its servicerate based on online observations of its own average queue length. The individ-ual queues do not need global network information (like the network structure),and even though all queues influence each other, the algorithm guarantees thatthe whole network achieves their common goal of a balanced operation.

Throughout Chapters 3, 4, and 5, we discuss the topic Ultracold Rydberg gasesand quantum engineering. Rydberg gases consist of atoms that exhibit strongmutual blockade effects, and this gives rise to an interacting particle systemwith intriguing complex interactions. These particle systems are investigatedin laboratory environments because of their application in quantum computingand condensed matter physics. Our research finds that in certain regimes, thecomplex interactions of these particles can be described using the stochastic

245

processes that also model the behavior of transmitters in wireless networks. Thisallows us to identify interesting connections between the fields of physics andmathematics, and to transfer techniques and insights from applied probability tothe realm of Rydberg gases. For example, the optimization algorithms for large-scale stochastic networks (described above) can be used to actively engineerthe atomic system, and by constructing special random graphs we can givetheoretical descriptions of statistical properties of the Rydberg gas.

Chapters 6, 7, and 8 cover the final topic Performance analysis and rev-enue maximization of critically loaded service systems. There, we consider large-scale Markovian many-server systems that operate in the so-called Quality-and-Efficiency-Driven (QED) regime and dwarf the usual trade-off between highsystem utilization and short waiting times. In order to achieve these dual goals,these systems are scaled so as to approach full utilization, while the number ofservers grows simultaneously large, rendering crucial Economies-of-Scale. Ourresearch extends the applicability of the QED regime by incorporating scalableadmission control schemes and general revenue functions. This also allows usto identify for a broad class of revenue functions exactly which nontrivial thresh-old control policies are optimal in the QED regime, yielding insight into therelation between the optimal control and revenue structure. By studying thesystem’s precise asymptotic behavior when nearing the QED regime, we are ableto also analyze the effectiveness of the QED regime as a framework for systemdimensioning.

About the author

Jaron Sanders was born in Eindhoven, The Netherlands, on November 6, 1987.He completed his pre-university secondary education at the Christiaan HuygensCollege in Eindhoven, and then started his studies in Applied Physics at Eind-hoven University of Technology in 2006. After obtaining his Bachelor degreewith highest honors (Cum Laude) in 2009, he continued his studies by nextpursuing a Master’s degree in Applied Physics as well as a Master’s degree inApplied Mathematics. In 2011, he obtained both Master’s degrees with highesthonors (Cum Laude). Jaron has been awarded two Dutch national prizes for hisMaster’s project: the Royal Holland Society of Sciences and Humanities awardedhim the ASML Graduation Prize for Mathematics in November 2012, and theNetherlands Society for Statistics and Operations Research awarded him theVvS+OR Master’s Thesis Award in March 2013.

In January 2012, Jaron started a PhD project at Eindhoven University ofTechnology in the Stochastic Operations Research group under the supervisionof Johan van Leeuwaarden and Sem Borst. His PhD research focused on thecritical scaling of service systems and the optimal control of stochastic systems,and he independently continued to research the topic of Rydberg gases. Hisresearch, presented in this dissertation, has led to a number of publications inleading scientific journals (IEEE Transactions on Automatic Control and PhysicalReview Letters) and exposure in popular magazines including Physics World andSTAtOR.

During his employment at Eindhoven University of Technology, Jaron at-tended several international conferences to present his work, among whichAllerton 2012 (Illinois, United States), Valuetools 2012 (Cargèse, France), Ap-plied Probability Society Conference 2013 (San José, Costa Rica), Performance2013 (Vienna, Austria), First European Conference on Queueing Theory 2014(Gent, Belgium), and Applied Probability Society Conference 2015 (Istanbul,

247

Turkey). Jaron also followed mathematics courses given by the Dutch Networkon the Mathematics of Operations Research, gave instructions on calculus, op-timization, and probability theory, provided professional service by conductingpaper reviews and chairing the Mathematics & Computer Science’s PhD Council,dabbled in arts different from mathematics by playing the saxophone, dancing,and photographing, and participated in numerous outreach activities throughscientific talks aimed at broad audiences.

Jaron will defend his PhD thesis at Eindhoven University of Technology onJanuary 28, 2016. After his defense, he will start as a post-doctoral researcherat the Royal Institute of Technology in Stockholm, Sweden.

Stochastic optimization of large-scale complex systems › ws › files › 13620787 ›...

Documents

Transcript of Stochastic optimization of large-scale complex systems › ws › files › 13620787 ›...