Semi-Infinite Programming: Recent Advances

Semi-Infinite Programming

Nonconvex Optimization and Its Applications

Volume 57

Managing Editor:

Panos PardalosUniversity of Florida, U.S.A.

Advisory Board:

l.R. BirgeNorthwestern University, U.S.A.

Ding-Zhu DuUniversity ofMinnesota, U.S.A.

C. A. FloudasPrinceton University, U.S.A.

r. MockusLithuanian Academy of Sciences. Lithuania

H. D. SheraliVirginia Polytechnic Institute and State University, U.S.A.

G. StavroulakisUniversity of Ioannina, Greece

The titles published in this seriesare listed at the end of this volume.

Semi-Infinite Programming Recent Advances

Edited by

Miguel A. Gobema and Marco A. L6pez Universidad de Alicante, Spain

Springer-Science+Business Media, B.v.

A C.I.P. Catalogue record for this book is available from the Library of Congress.

Printed on acid-free paper

All Rights Reserved© 2001 Springer Science+Business Media OordrechtOriginally published by Kluwer Academic Publishers in2001.Softcover reprint of the hardcover 1st edition 2001

No part of the material protected by this copyright notice may be reproduced orutilized in any form or by any means, electroni c or mechanical,including photocopying, recording or by any information storage andretrieval system, without written permission from the copyright owner.

ISBN 978-1-4419-5204-2 ISBN 978-1-4757-3403-4 (eBook)DOI 10.1007/978-1-4757-3403-4

To Rainer Hettich andWerner Oettli,

in memorian

Contents

Preface

Contributing Authors

Part I HISTORY

xi

xv

ON THE 1962-1972 DECADE OF SEMI-INFINITE PRO- 3GRAMMING: A SUBJECTIVE VIEWKen O. Kortan ek

1 Introduction: Origins of a theory 42 Generalized linear programming and the moment problem 73 Using the 1924 Haar result on inhomogeneous linear inequal-

ities 94 Introducing an infinity into semi-infinite programming 105 A classification of duality states based on asymptotic consis -

tency 216 Asymptotic Lagrange regularity 257 Applications to economics, game theory, and air pollution

abatement 268 Algorithmic developments: "Matching of the derivatives" 319 Epilog 33References 34

Part II THEORY

2 ABOUT DISJUNCTIVE OPTIMIZATIONIvan I. Eremin

1 Introduction2 Saddle points of disjunctive Lagrangian3 Duality framework4 An exact penalty function methodReferences

Vll

45

4548515557

Vlll SEMI-INFINITE PROGRAMMING. RECENT ADVANCES

3 ON REGULARITY AND OPTIMALITY IN NONLINEAR 59SEMI-INFINITE PROGRAMMINGAbdelhak Hassouni and Werner Oettli

1 Introduction 592 The linear case 603 The convex case 614 Convex approximants 665 The exchange method for semi -infinite convex minimization 686 Normal cones and complementary sets 71References 74

4 ASYMPTOTIC CONSTRAINT QUALIFICATIONS AND 75ERROR BOUNDS FOR SEMI-INFINITE SYSTEMS OFCONVEX INEQUALITIESWU Li and Ivan Singer

1 Introduction 752 Preliminaries 773 Asymptotic constraint qualifications. The sup-function method 804 Error bounds for semi-infinite systems of convex inequalities 905 Error bounds for semi-infinite systems of linear inequalities 95References 99

5 STABILITY OF THE FEASffiLE SET MAPPING IN CON- 101VEX SEMI-INFINITE PROGRAMMINGMarco A. LOpez and Virginia N. Vera de Serio

1 Introduction 1012 Preliminaries 1033 A distance between convex functions 1044 Stability properties of the feasible set mapping 105References 119

6 ON CONVEX LOWER LEVEL PROBLEMS IN GENERAL- 121IZED SEMI-INFINITE OPTIMIZATIONJan-J. Riickmann and Oliver Stein

1 Introduction 121,2 Thelocal topology of M 1233 A local first order description of M 1264 First order optimality conditions 1305 Final remarks 132References 132

Contents IX

7 ON DUALITY THEORY OF CONIC LINEAR PROBLEMS 135Alexander Shapiro

1 Introduction 1352 Conic linear problems 1363 Problem of moments 1454 Semi-infinite programming 1525 Continuous linear programming 155References 164

Part ITI NUMERICAL METHODS

8 TWO LOGARITHMIC BARRIER METHODS FOR CON- 169VEX SEMI-INFINITE PROBLEMSLars Abbe

1 Introduction 1692 A bundle method using e-subgradienrs 1703 Description of the barrier method 1724 Properties of the method 1755 Numerical aspects 1816 Numerical example 1827 A regularized log-barrier method 1858 Numerical results of the regularized method 1919 Conclusions 193References 194

9 FIRST-ORDER ALGORITHMS FOR OPTIMIZATION 197PROBLEMS WITH A MAXIMUM EIGENVALUE!SINGULAR VALUE COST AND OR CONSTRAINTSElijah Polak

1 Introduction 1972 Semi-Infinite Min-Max Problems 1993 Rate of Convergence of Algorithm 2.2 2064 Minimization of the Maximum Eigenvalue of a Symmetric

Matrix 2075 Problems with Semi-Infinite Constraints 2116 Problems with Maximum Eigenvalue Constraints 2167 Rate of Convergence of Algorithm 5.1 2168 A Numerical Example 2179 Conclusion 219References 219

x SEMI-INFINITE PROGRAMMING. RECENT ADVANCES

10 ANALYTIC CENTER BASED CUTTING PLANE 221METHOD FOR LINEAR SEMI-INFINITE PROGRAMMINGSoon-Yi Wu, Shu-Cherng Fang and Chih-Jen Lin

1 Introduction 2212 Analytic Center Based Cuts 2233 Analytic Center Cutting Plane Method for LSIP 2244 Convergence and Complexity 230References 233

Part IV MODELING AND APPLICATIONS

lION SOME APPLICATIONS OF LSIP TO PROBABILITY 237AND STATISTICSMarco Dall'Aglio

1 Introduction 2372 De Finetti coherence 2383 Constrained maximum likelihood estimation of a covariance

matrix 2464 LSIP in actuarial risk theory 247References 254

12 SEPARATION BY HYPERPLANES: A LINEAR SEMI- 255INFINITE PROGRAMMING APPROACHMiguel A. Goberna, Marco A. LOpez and Soon- Yi Wu

1 Introduction 2552 Separation in norrned spaces 2573 Strong separation of compact sets in separable norrned spaces 2624 Strong separation of finite sets in the Hadamard space 265References 269

13 ASEMI-INFINTEOPTIMIZATIONAPPROACHTOOPTI- 271MAL SPLINE TRAJECTORY PLANNING OF MECHANI-CAL MANIPULATORSCorrado Guarino Lo Bianco and Aurelio Piazzi

1 Introduction 2712 Cubic spline trajectory planning under torque and velocity

constraints 2753 A feasibility result 2774 Problem solution using an hybrid algorithm 2815 Penalty computation via interval analysis 2846 An Example 2907 Conclusions 293

Contents Xl

References 295

14 ON STABILITY OF GUARANTEED ESTIMATION 299PROBLEMS: ERROR BOUNDS FOR INFORMATIONDOMAINS AND EXPERIMENTAL DESIGNMikhail I. Gusev and Sergei A. Romanov

1 Introduction 2992 Rate of convergence of information domains for problems

with normally resolvable operator 3033 Optimal placement of sensors for nonstationary system: Du-

ality theorems 3104 Optimal sensor placement: the stationary case 3155 A sufficient number of sensors 318References 324

15 OPTIMIZATION UNDER UNCERTAINTY AND LINEAR 327SEMI-INFINITE PROGRAMMING: A SURVEYTeresa Leon and Enriqueta Vercher

1 Introduction 3272 Fuzzy sets 3293 Convex programming with set-inclusive constraints 3314 Fuzzy mathematical programming 3375 Linear semi-infinite programming 3416 Numerical Results 345References 346

16 SEMI-INFINITE ASSIGNMENT ANDTRANSPORTATION 349GAMESJoaquin Sanchez-Soriano, Natividad Llorca, StefTijs and Judith Timmer

1 Introduction 3492 Finite transportation and assignment games 3503 Semi-infinite assignment games 3534 Semi-infinite transportation problems and related games 3565 Final remark 362References 362

17 THE OWEN SET AND THE CORE OF SEMI-INFINITE 365LINEAR PRODUCTION SITUATIONSStefTijs, Judith Timmer, Natividad Llorca and Joaquin Sanchez-Soriano

1 Introduction 3652 Finite linear production situations 3663 Semi-infinite LP situations 3704 Finite LTP situations 3745 Semi-infinite LTP situations 379

XlI SEMI-INFINITE PROGRAMMING. RECENT ADVANCES

6 ConclusionsReferences

385386

Preface

Semi-infinite programming (SIP) deals with optimization problems in whicheither the number of decision variables or the number of constraints is finite.Hence SIP occupies an intermediate position between ordinary mathematicalprogramming and fully general optimization.

Although SIP theory is quite old -recall the classical work of Haar on linearsemi-infinite systems and the John's optimality conditions for SIP problems,published in 1924 and 1948, respectively- , the term appeared by the first timein a paper of Charnes, Cooper, and Kortanek (1962) devoted to duality in linearSIP. The last author also contributed significantly to the development of thefirst applications of SIP in economics, game theory, and air pollution control.Gustafson and Kortanek proposed -during the firsts 1970s- the first numericalmethods for the effective treatment of the SIP models arising in these applications. Two decades after its inception, the publication around 1980 of five booksconverted SIP in a mature and independent optimization chapter. These bookswere two volumes of Lecture Notes on Mathematics devoted to SIP -edited byHettich (in 1979), and by Fiacco and Kortanek (in 1983), respectively -, andthree monographs on linear SIP (by Tichatschke, 1981), on numerical methods in SIP and their applications to approximation problems (by Hettich andZencke, 1982), and another one on stability in SIP (by Brosowski, 1982).

Since then many papers are published every year on the theory, methodsand applications of SIP and its extensions. Consequently, SIP conferences areorganized regularly in order to communicate the recent advances in the field,establishing new challenges for researchers. SIP'96 took place in Cottbus,Germany, giving rise to the book Semi-Infinite Programming (R. Reemtsen andJ.-J. Riickmann, editors, Kluwer, 1998). The present volume is based upon thelast conference of this series , SIP'99, held in Alicante, Spain, and its purposeis to update the state-of-the art in a suggestive way, bringing the powerful SIPtools close enough to the potential users in different scientific and technologicalfields.

xiii

XIV SEMI-INFINITE PROGRAMMING. RECENTADVANCES

The volume is divided into four parts, devoted to the history, theory, methodsand applications (in the large sense of the term) of SIP and related topics,respectively. Part I contains a unique paper reviewing the first decade of SIP(1962-1972). His author, K.O. Kortanek -one of the main protagonists of thisperiod- dedicates his paper to one of the main contributors to the developmentof SIP, Rainer Hettich, who sadly passed away in June 2000.

Part II contains six theoretical papers. Three of them deal with convex SIPfrom different perspectives: constraint qualifications and error bounds (by W.Li and I. Singer), stability of the feasible set (M.A. Lopez and V. Vera deSerio), and regularity and optimality conditions (by A. Hassouni and W. Oettly,who also died during the preparation of this book). The remaining papersare focussed on different extensions of SIP: conic linear programming (by A.Shapiro), generalized SIP (by 1.-J. Riickmann and O. Stein), and disjunctiveprogramming (by I. Eremin).

Part III contains three works exclusively devoted to the numerical treatmentof different families of SIP problems and extensions. They propose an analyticcenter cutting plane method for linear SIP (S.- y- Wu, S.-Ch. Fang and Ch.Lin), two logarithmic barrier methods for convex SIP (L. Abbe), and first ordermethods for a class of problems which contains continuously differentiable SIPproblems as a particular case (E. Polak), respectively.

Finally, Part IV includes seven contributions on the connections betweenSIP and different fields. Those presenting numerical experiments are relatedto probability and statistics (by M. Dall'Aglio), optimization under uncertainty(by T. Leon and E. Vercher) and the design of mechanical manipulators (by C.Guarino Lo Bianco and A. Piazzi), respectively. Two of the remaining papersdeal with semi-infinite games (both of them authored by N. Llorca, J. SanchezSoriano, S. Tijs, and J. Timmer). Moreover, this part contains a paper on theseparation of sets in normed spaces (by M.A. Gobema, M.A. Lopez, and S.Y- Wu), and another one related to error bounds for information domains andexperimental design (by M.1. Gusev and S.A. Romanov).

We would like to express our gratitude to all the contributors for their highquality papers, and to the referees for their valuable reports . We are verymuch indebted to Dr. L. Canovas for his help in the careful preparation of thecamera-ready manuscript. Last but not least, our thanks to 1.R. Martindale andthe staff of Kluwer Academic Publishers for having accepted the edition of thisbook, and for their comprehension of the six months delay in the presentationof the manuscript. This delay is the consequence of the search of a compromise between two conflicting objectives: minimizing the publication time andmaximizing the quality of the book, through succesive corrections.

Alicante, May 2001.

Miguel A. Gobema and Marco A. Lopez

Contributing Authors

Lars Abbe. Department of Mathematics, University of Trier, Trier (Germany).

Marco Dall'Aglio. Dipartimento di Scienze, Universita "G. d' Annunzio",Pescara (Italy).

Ivan Eremin. Department of Mathematical Programming, Institute of Mathematics and Mechanics, Ekaterinburg (Russia).

Shu-Cherng Fang. Operations Research and Industrial Engineering, NorthCarolina State University, Raleigh (USA)

Miguel Angel Goberna. Departament of Statistics and Operations Research,University of Alicante, Alicante (Spain).

Corrado Guarino Lo Bianco. Dipartimento di Ingegneria dell'Informazione,Universita di Parma, Parma (Italy).

Mikhail I. Gusev. Institute of Mathematics and Mechanics, Ural Branch ofthe Russian Academy of Sciences, Ekateringburg (Russia).

Abdelhak Hassouni. Departement de Mathematiques et Informatique, Universite Mohammed V, Rabat (Morocco) .

Ken O. Kortanek. Department of Management Sciences, University of Iowa,Iowa (USA) .

xv

xvi SEMI-INFINITE PROGRAMMING. RECENTADVANCES

Teresa Leon. Departamcnt d'Estadfstica i Investigacio Opcrativa, Universitatde Valencia, Valencia (Spain).

Wu Li. Department of Mathematics and Statistics, Old Dominion University,Norfolk (USA).

Chih-Jen Lin. Department of Computer Science and Information Engineering,National Taiwan University, Taipei (Taiwan).

Natividad Llorca. Department of Statistics and Applied Mathematics, MiguelHernandez University, Elche (Spain).

Marco A. Lopez. Department of Statistics and Operations Research, Universityof Alicante , Alicante (Spain).

Werner Oettli. Fakultat fiirMathematik und Informatik, Universitat Mannheim,Mannheim (Germany).

Elijah Polak. Department of Electrical Engineering and Computer Sciences,University of California, Berkeley (USA) .

Aurelio Piazzi. Dipartimento di Ingegneria dell'Informazione.Universita diParma, Parma (Italy).

Sergei A. Romanov. Institute of Mathematics and Mechanics, Ural Branch ofthe Russian Academy of Sciences, Ekateringburg (Russia).

Jan-J. Riickmann. Institut fiir Mathematik, Technische Universitat Ilmenau,Ilmenau (Germany).

Joaquin Sanchez-Soriano. Department of Statistics and Applied Mathematics, Miguel Hernandez University, Elche (Spain).

Alexander Shapiro. School of Industrial and Systems Engineering, GeorgiaInstitute of Technology, Atlanta (USA) .

Ivan Singer. Institute of Mathematics, Bucharest (Romania).

Contributing Authors xvii

OliverStein. Lehrstuhl C fill Mathematik, RWTH Aachen, Aachen (Germany) .

StefTijs. Center and Department of Econometrics, Tilburg University, Tilburg(The Netherland).

Judith Timmer. Center and Department of Econometrics, Tilburg University,Tilburg (The Netherland).

Virginia N. Vera de Serio . Faculty of Economic Sciences, Universidad Nacional de Cuyo, Mendoza (Argentina).

Enriqueta Vercher. Departament d'Estadfstica i Investigaci6 Operativa, Universitat de Valencia, Valencia (Spain).

Soon-Yi Wu. Department of Mathematics, National Cheng Kung University,Tainan (Taiwan) .

Part I HISTORY

Chapter 1

ON THE 1962-1972 DECADE OF SEMI-INFINITEPROGRAMMING: A SUBJECTIVE VIEW*

Ken O. Kortanek

Department ofManagement Sciences, Tippie College of Business and the Program in Applied

Mathematical & Computational Sciences, University ofIowa, Iowa City, IA 52242 , USA

[email protected]

A bstra ct Severa l major themes developed during this, the apparent first of almost fourdecades of semi-infinite programming, are reviewed in this paper.

One theme was the development of a dual program to the problem of minimizing an arbitrary convex function over an arbitrary convex set in the n -spacethat featured the maximization of a linear functional in non-negative variablesof a generalized finite sequence space subject to a finite system of linear inequal ities . A characteristic of the dual program was that it did not involve any primalvariables occurring within an internal optimization.

A second major theme was the introduction of an "infinity" into systemsof semi - infinite linear inequalities, a manifestation of the "probing" betweenanalysis and algebra.

In finite linear programming there are four mutually exclusive and collectivelyexhaustive duality states that can occur, and this led to the third theme of developing a classification theory for linear semi-infini te programming that includedfinite linear programming as a special case.

The fourth theme was one of algori thmic development. Finally, throughoutthe decade there was an emphasis on applications, principa lly to Economics,Game Theory, Asympto tic Lagrange Regularity, Air Pollution Abatement, andGeometric Programming.

"This paper was developed in response to a request made by Professor S. H. Tijs at the Alicante Symposiumon Semi-infinite Programming in September, 1999. It is dedicated to the late Rainer Hettich, Friend,Mathematic ian, and University President. An earlier version formed the basis for a presentation at theEhrenkoUoquium fiir Herrn Prof. Dr. Rain er Hettich in Trier II February 2000.

3

M.A. Goberna and M.A. Lope z (eds.), Semi-Infinite Programming, 3-41.© 2001 K/uwer Academic Publishers.

4 SEMI-INFINITE PROGRAMMING. RECENTADVANCES

1 INTRODUCTION: ORIGINS OF A THEORY

A published statement about the origins of a theory of semi-infinite pro-gramming appears in [5, Introduction] :

In March, 1962 Charnes, Cooper, and Kortanek developed the theory of semiinfinite programming which associates the minimization of a linear function offinitely variables subject to an arbitrary number of arbitrary linear inequalities inthese variables with maximization of a linear function of infinitely many variablessubject to a finite system of linear inequalities. The notion of a generalized finitesequence space (defined to be the direct sum of the underlying field indexed bya given index set associated with the arbitrary system of linear inequalities) wasintroduced for the latter problem, and the dual structure of these programmingproblems was used to probe the borderline between properties which are purelyalgebraic and those in infinite programming which require topology with theprobing taking place in an infinite programming setting. See [II).

The authors started from the 1924 paper of A. Haar, [58], and defined thenotion of "Haar" (or "semi-infinite") programs having the features described inthe quotation above. In 1963 a paper was published by S. N. Tschernikow [85],having the translated title "On Haar's theorem about infinite linear inequalities".It is not known whether there exists an English translation of the journal in whichit appeared. More will be written about this important paper in a later section.In the meantime the pair of dual linear semi-infinite programs is given in thefirst definition.

Definition 1.1 (Dual semi-infinite programs) Let I be any set, termed theindex set. Given {Pd iET U Po C u», {Ci} iET C R, the following duality pairis constructed:

Primal Program

vp = inf uT Po, u E R m

subject to

Dual Program

subject to

x, ~ 0, with only finitely many x, =1= O.(1.1)

At various places throughout this paper we will refer to the following semiinfinite linear inequality system appearing in (1.1):

(1.2)

ON THE 1962-1972 DECADE OF SIP: A SUBJECTIVE VIEW 5

Later, in Definition 4.1 we shall be denoting the space of such A-functionsas R[Il. We define a fundamental convex cone with a slight abuse of standardnomenclature.

Definition 1.2 The moment cone of(1.1) is the convex set

The standard definition of the moment cone of(1.1) is

convex cone { ( Pi) }Ct iEI

(1.4)

Remark 1.1 The standard definition is also referred to as moment spacewith respect to {Pi, CiLEI in Karlin and Studden, [65].

The ordinary duality inequality takes on the following form here.

Lemma 1.1 (Duality Inequality) Assume u, A. respectively, are feasiblefor the primal and dual programs. Then

uT

Po;:::: I::CiAi.i EI

Hence,

ve >: VD. (1.5)

If the inequality in (1.5) is strict, then there is a duality gap. When the valuesare equal, the terminology duality equality is used.

Linear semi-infinite programming is a special case of infinite linear programming, which can best be seen when a duality pairing and compatible topologyare specified, see [5]. The power of semi-infinite programming stems from theclose relationship to ordinary finite linear programming, where no topologicalconsiderations need be made. The simplex method [25] works over any orderednumber field.

Remark 1.2 Generally, in this paper, we will term the program havingthe infinite linear inequality system as the primal program. However, there aretwo sections where this convention is reversed. In Sections 2 and 5 appearprograms having inequality systems that are termed "dual". The differencesin these conventions have to a substantial degree been minimized, while theduality formulations tend to follow their earlier presentations in the literature.

6 SEMI-INFINITE PROGRAMMING. RECENT ADVANCES

Remark 1.3 Perhaps from a probabilist's point of view semi-infinite programming problems are moment problems, equivalently formulated over Borelmeasures, see [65], [32], [52], and [57].

1.1 CONVEX PROGRAMS WITH CONVEX CONSTRAINTS

In 1962 linear semi-infinite duality was applied to convex programming [12] .The pair of dual problems involved (i), the minimization of an arbitrary convexfunction over an arbitrary convex set in the n-space and (ii), the maximizationof a linear functional in non-negative variables on a generalized finite sequencespace subject to a finite system of linear equations. The principal feature thatthis development brought to convex programming was a dual problem that didnot involve primal optimizing variables which usually occur within an internaloptimization. This duality feature had also been achieved in papers by Eisenberg [37] and Rockafellar [77]. The results appearing in [11] prompted the 1964claim made in [66], that they "in principle" include the duality results appearing in [37], [29], [31], and [77]. Relationships to the then cited "Kuhn-TuckerConditions" appeared in [16]. The convex programming duality results weregeneralized in Dieter [28] for infinite dimensional spaces. He also presentedan infinite linear program with a duality gap. Both he and Rockafellar [78]significantly developed the Fenchel contact-transformation approach to duality theory, although in the mid-60's it seems that Rockafellar restricted himselfto finite dimensions, as did A. Whinston in [90]. The powerful technique ofincluding perturbations in the convex programming formulation was developedin [78] building on the milestone paper ofDavid Gale, [44]. Gol'stein's generalized feasible solutions, [48], constructed as infinite sequences were advanced toremove "duality gaps" in finite dimensional convex programming. These wereanalogous to Duffin's subconsistent solutions introduced for the same purposein infinite linear programming, see [30] .

In this paper the review of the supporting hyperplane-duality constructionfor convex programming is delayed until Section 7.1, (7.4). The context thereis an economic application, and there is no loss in generality in the dual convexprogramming pair development.

1.2 APPLICATIONS TO GEOMETRIC PROGRAMMING

Applications to geometric programming were based upon the property oftransformable convexity of the geometric posynomial primal program. Thisgave rise to associated systems of supporting hyperplanes (see [34], [17], and[18]).

For completeness the posynomial dual geometric programming pair is presented below (see [31], [36], [33]) . New insights into classifying the duality

ON TilE 1962-1972 DECADE OF SIP: A SUBJECnVE VIEW 7

states of geometric programs appeared in [46], while algorithmic development'>appeared in [45] and [47]. New applications were also given in both CarnegieMellon theses, [45] and [83]. A general approach to classifying convex programs appears in Section 5.

k = 1, ... , p, where all ti > 0,

mo = 1, ml = no+ 1,

m2 = nl + 1, ..., m p = np-l + 1.

sup .n(*)6; Ii A~k~=l k=lno

s.t. L s, = 1i=lnp

L lSiaij = 0, j = 1,2, ..., mi=lnk

L ISr - Ak = 0, k = 1, ...,Pmk

lSi ~ 0, i = 1, ...,np .

(1.6)

Here {aij} are arbitrary real constants but {c.} are positive.

2 GENERALIZED LINEAR PROGRAMMING ANDTHE MOMENT PROBLEM

Let us briefly review a parallel duality development between Generalized Linear Programming (GLP) or Dantzig-Wolfe Decomposition and linear semiinfinite programming. Referring to the 1963 book of G. B. Dantzig [25], weobtain the following dual programming pair, where for convenience we retainthe same number of variables in the linear inequality system as in (1.1). Thenotation r = 1, m is typical Russian denoting, r = 1, ...m.

Program GLP

Let U r , r = 1, m, and - Um+l be real valued convex functions defined onan arbitrary convex set 5, and let bERm.

Find Z = sup Um+l (x) A for among x E 5 and A E R which satisfy

(2.1)


Program Dual GLP

Find VD = inf L:;.n=l Yr br from among Y E R m

which satisfy L:;.n=l Yrur(x) ~ Um+l (x) for all xES and Yr ~ 0, r = 1, m.(2.2)

Clearly Program Dual GLP is equivalent to the primal program in (1.1)under the data structure of the given coefficient functions . Constructing theformal semi-infinite dual program to Program Dual GLP yields the classicalmoment problem (see [65], [52], and [57]), which we term Program Moment.

Program Moment

Find M = sup L:xES Um+l (x) >.(x) from among >.(.) E R[S] (2.3)

which satisfy L:xEsur(x) >.(x) :c::; br, r = I ,m and >.(.) ~ O.

The generalized finite sequence space, R[S] , is introduced in Definition 4.1 ,in slightly more generality. The following result is a simple consequence of theconvexity assumptions.

Lemma 2.1 Assume Program GLP has afeasible point (x, >') with>. > O.Then Z=M.

Proof. If we view {x,>.(x) ,x E S} as a finite non-negative measure on S,then Program GLP is merely Program Moment restricted to one-point massmeasures. Hence, trivially, Z :c::; M.

On the other hand, assume {Xi, >'(Xi)} is any feasible for Program Moment.Let >'i = >'(Xi). Without loss of generality we may assume>' = L:i >'i > O.Since S is a convex set, it follows that x = L:i Xi >'d>' lies in S.

By convexity of U r , r = 1, m, and feasibility of (Xi, >'d, it follows that

Ur(X) = ur(2: Xi>'d>') :c::; 2: Ur(Xi) >'d>' :c::; bTl>', r = 1, m.i i

This means that (x, >') is feasible for GLP. Using the concavity of um+!, weobtain,

>'Um+dx) ~ x2: Um+dXi) >'d>' = 2: Um+!(Xi) >'ii

Since the feasible points were arbitrary for their respective programs, it followsthat Z ~ M . Hence, Z = M . 0

ON THE 1962-1972 DECADE OF SIP: A SUBJECTIVE VIEW 9

Under the convexity/concavity assumption on the coefficient functions, ageneralized linear program is equivalent to a moment problem.

3 USING THE 1924 HAAR RESULT ONINHOMOGENEOUS LINEAR INEQUALITIES

We begin by presenting the structure of a fundamental result on systems oflinear inequalities in a finite number ofvariables, (1.2). The history surroundingthis famous result of the great mathematician Alfred Haar is most interestingindeed, [58] .

Theore~ 3.1 . (Haar 1924 as Restated [58]) Let I bean~ set. Given.{Pi kl)U{Po} c R , ci , z E I,d E R. Assume that {Pi,CiLEllS compact tn R .Assume that m variables are required in the linear system uT Pi - Ci >0, for all i E I . Assume further that

uT Po - d 2: 0, whenever uT Pi - Ci 2: 0, for all i E I. (3.1)

Then there exists Ai 2: 0, i E I, with at most m + 1 nonzero, such that

uT Po - d 2: L (uT Pi - Ci) Ai.iEI

The original statement of the theorem was imprecise and therefore incorrect.Haar did not state the assumptions of interiority and compactness. Instead,he used the word "closed" instead of "compact", but his proof showed that heactually was using compactness. He also specifically assumed the existence ofan interior point for the given linear inequality system during the course of theproof, stating, "Otherwise we don't need m variables."

Several authors continued the oversight, e.g., [11], [12], [13] and [91, 1966Corollary 2.1 to the Theorem of A. Haar]. The interiority assumption shouldhave made explicit in the original 1962 definition of "Canonical Closure", see[66, Page 1-21]. Duffin and Karlovitz communicated the problem, see [35],and R. T. Rockafellar wrote a letter to A. Charnes about it in August 1965.

But the first known published recognition of the error appeared in a paper byS. N. Tschernikow [85] . While this paper is difficult to obtain, there is translation into German of a 1971 paper report ing on these results, see [87]. (Anotherinteresting but difficult-to-obtain paper by Tschernikow is [86].) Tschernikowwas the thesis advisor of Nikolay Astaf"ev of the Institute of Mathematics andMechanics, Ekateringburg, Russia. Tschernikow gave a necessary and sufficient condition for the Haar Theorem to hold, namely that the "moment cone",(1.3), be closed.


The idea of obtaining a canonically closed representation equivalent to agiven semi-infinite system was of interest during this time with Zhu's 1966index of degeneracy, [91], agreeing with the Charnes/Cooper/Kortanek 1965,[14], use of the smallest flat containing what we now call the constraint set.Here are some details.

In [14, page 114] the convex primal constraint set,

K = {u E R m IuT p,. > c · i E I}l _ l'

is contained in a smallest (n - r) flat T, 0 ~ r ~ n. Independently, in [91,footnote page 29] r is termed the index of degeneracy, with nondegeneratemeaning r = 0, or in Haar's terminology "m variables are not needed," astranslated in [14, pages 220, 223].

Definition 3.1 The consistent system uT Pi ~ Ci, i E I. is said to have theFarkas-Minkowski property iffor Po E Rm, d E R. (3.1) implies there exists

A E R[ I], A ~ 0 • such that uT Po - d ~ L (uT Pi - Ci) Ai for all u E R m .

iE[

(3.2)

4 INTRODUCING AN INFINITY INTOSEMI-INFINITE PROGRAMMING

Another major theme of this period was to investigate the introduction of"infinity" into semi-infinite programming, a manifestation of the so-called"probing" between topology (analysis) and algebra. A quotation by Hardy [60,APPENDIX IV The infinite in analysis and geometry] describes the dichotomy:

"The infinite in analysis is a 'limiting ' and not an 'actual ' infinite ."

Hardy goes on to discuss the infinite of geometry as an "actual" and notas a "limiting" infinite. Our focus was on the infinite in algebra and not ingeometry. It is time to be specific by starting with a definition that exhibits avery next level of extension of fundamental constructions appearing in finitelinear programming.

Definition 4.1 Let F be an arbitrarily ordered field. and let I be an arbitrary (index) set. Given {Pi}iEI U {Po} c r», {Ci}iEI C F. define thefollowing objects:

F[IJ = {A IA : 1-+ F , finitely many Ai =1= O} .. i.e.. the so-calledgeneralized finite sequence linear space of the set lover F,

(4.1)

ON THE 1962-1972 DECADE OF S1P: A SUBJECT1VE VIEW 11

Theorem 4.1 (Linear Independence with Extreme Points [13]) A is a convex set, and), i- 0 is an extreme point ofA ifand only if the nonzero coefficientsof). correspond to a set of linearly independent vectors.

Theorem 4.2 (Opposite Sign Theorem [13]) A is generated by its extremepoints if, and only, if for any a E F[l], LiE! Pi ai = 0 implies that thereexists r, s E I such that aT as < O.

These results are purely algebraic as manifested by the generality obtainedby using any ordered field. An opposite sign property algorithm for purificationto an extreme point solution was developed in 1963, first for ordinary linear programming and, almost simultaneously, for linear semi-infinite programming,see [20] .

Attempts at describing infinities as transfinite numbers were not appropriatebecause these numbers usually refer to Cantor's cardinal and ordinal numbers.Neither were transcendental numbers appropriate because these are numberssuch as 1f, which cannot be determined by an algebraic equation. Actually,in [9], A. Charnes and W. W. Cooper had termed the smallest ordered fieldobtained by adjoining an arbitrarily large element to the reals as the Hilbertfield, see also [10] . The underlying polynomial ring construction was sufficientto inject infinity as an "actual" into semi-infinite systems of linear inequalities.Here are the details .

Definition 4.2 Let R[O] denote the polynomial ring, R [8], consisting offinite degree, real coefficient polynomials in an indeterminate O. A non-Archimedean ordering is defined by requiring r < 0 for any real number r. Apolynomialp( 0) = L~=o ri Oi is positivelnegative] if the coefficient ofthe highest non-vanishing power of0 is positivelnegative]. The polynomial L~=l r i Oiis termed the infinite part ofp(0), which if not zero, is necessarily positive ornegative. Formal rational functions p(O)/q(O) ,q(O) i- Oform an ordered fielddenoted by R (0).

4.1 SEMI-INFINITE PROGRAMMING REGULARIZATIONS WITHINFINITE NUMBERS

Introducing an infinity, or a function of an infinity, is often suggested bythe behavior of introducing large real numbers and examining the effects oftheir further relaxations. Therefore, we initially construct real number regularizations of linear semi-infinite programs in order to transform an originalproblem pair into a problem pair that is consistent and bounded. We motivatethe construction with a program pair that has a duality gap:


Example 4.1 Let U be a positive real constant. Consider the dual pair

Primal

Vp = inf Ul

subject to

Ul ~ + U2r&- 2: 0, n = 1, ...

Ul 2: -U.

Dual

V D = sup L:n OAn- UII.

subject to

11., An 2: 0, for all n.

We see that 0 = vp > VD = -U, illustrating a duality gap.

Duffin and Karlovitz [35] introduced a sequential procedure with each successive approximation including the previous one, illustrated in the aboveexample with a sequence of linear programs, LP(N), with N inequalities,n ~ N, N = 1,2, .... But in the example, each LP(N) value is -U, "sticking" on the redundant inequality Ul 2: -U. Thus, in a trivial way the limitingvalue of the primal approximations agrees with the dual value VD .

A Semi-Infinite Programming Partial Regularization

Let U be a positive real constant, and let 1m denote the m x m identity matrixand em the m-vector of all ones. Let PR(U) denote the following program.

Vp(U) = inf uT Po

subject to UT P. > r~ i E 1% - "' ,

where vj , vi 2: 0, j = 1, n, Ai 2: 0, i E 1.

ON THE 1962-1972 DECADE OF SIP : A SUBJECTIVE VIEW 13

Theorem 4.3 ([ 14]) Assume that P R(U) isconsistent and that {(Pi, GiH iET

is compact. Then v p (U) = vD (U) , and the primal objective function value ofvp(U) is attained.

Remark 4.1 Adjoin the inequality -U2 ~ -U to Example 4.1 and denotethe program value of the new dual by "i» Then "b = 0, without attainment.

A Semi-Infinite Programming Full Regularization

Let M and U be positive real constants and 1m , em defined as before.

vp(M, U) = inf uoM + uT Po

subject to

subject to

We illustrate the classical Slater example under a full regularization but nowwith infinite numbers.

Slater

min xs.t, - (1 - x)2 ~ 0

SIP Version

min xs.t. 2(1- O')x ~ 1 - a 2 , o:S 0' :S 2,

where the optimal solution is x* = 1. We obtain an equivalent pair of dual semiinfinite programs by introducing a differential supporting hyperplane system.

Primal

min Mt+x

subject to

t~O

t + 2(1 - o:)x ~ 1 - 0:2,

0~0:~2

x~ -U

-x ~ -U.

Dual

sup L:a { (1 - 0:2).\a

-U.\+ - U.\-}

subject to/-L + L:a .\a = M

L:a {2(1 - O:).\a

+.\+ - .\-} = 1

/-L, .\a ~ 0, 0 s 0: ~ 2.

Referring back to Definition 4.2 set M = () and U = ()2. Set t* ~ and

x* = 0:* (the index point) = 1 - .Jo. A dual solution is defined next:

if 0: = 0:*

if 0: =1= 0:*

and X" =.\- = /-L = O.

A primal feasibility check yields t* +2(1 - 0: )x* = 1 - 0: 2 + (0: - 2~81)2 ~

1 - 0:2 , for 0 ~ 0: ~ 2.The common of objective function values is 1 - -ie . For any real number

r < 1, it follows that r < 1 - -ilf < 1. We note that the Lagrange multiplier isnow an infinite number, namely (). We conclude this section with a conjectureabout full regularizations with infinite numbers.

Conjecture 4.1 (1968 Solutions with Infinite Numbers) Assume that thesemi-infinite system u E R'"; uT Pi ~ Ci, i E I has a compact set of real coefficients. Consider the fully regularized, general dual programs ofSection 4.1having variables {uo, u} and {.\i, i E I, v+, v-} respectively. There existnon-Archimedean dual optimal solutions to the primal and dual full regularizations respectively, such that

where 0: : {I , 2, ...,m} -+ {O, 1, ...,m} and Y = ()rn +l, and where all variableslie in the base field R(()), introduced in Definition 4.2.

ON THE 1962-1972 DECADE OF SIP: A SUBJECnVE VIEW 15

Remark 4.2 A sketch of a proof has appeared in [66] and [15], usinggeometrical-type limiting arguments based upon Hardy's concepts of order,[59] . Rather than attempt to reconstruct these in this paper it is hoped that thereis sufficient motivation to develop modern proofs for this seemingly "pure"mathematical result. It therefore seems more appropriate to attach the invitinglabel, "conjecture".

4.2 ASYMPTOTIC SOLUTIONS TO SEMI-INFINITE LINEARINEQUALITY SYSTEMS WITH INFINITE NUMBERS

An asymptotic notion by definition involves convergence ofsome type, whichtakes us to the Hardy quotation of the infinite ofanalysis. The concept was tobe applied to systems of linear semi-infinite inequalities, thereby yielding adefinition of asymptotic consistency for (1.2) .

Duffin [30] introduced the concept of subconsistency into the mathematicalprogramming literature and developed an infinite programming duality theoryaround the new concept. Some specificity about general convergence conceptsapplied to (1.2) was provided in [5] and [61] by means of topological nets.

Definition 4.3 A net is an ordered triple (Sa, a E D , <) == Sa, where Sis a function on the domain D and D is directed by " < ", namely, it is:

1. Transitive, a < band b < c implies a < c,

2. Proper, a < band b < a implies a = b, and

3. Compositive, a, d E D implies that there exists c E D such that a <c and b < c.

The net Sa is eventually in a set X if there exists f3 E D such that f3 < 'Yimplies Sy E X. A net converges to a point P if Sa is eventually in eachneighborhood ofP.

It was rather cumbersome to work with topological nets , where often D wasthe set of integers or the collection of all finite subsets of an infinite index setI. Two simplifications were very much welcome at the time .

If the net D is the set of integers under the usual ordering, then {Uk IkE D}is an asymptotic solution to (1.2) if, and only if, limlc {ur~} 2: Ci for eachi E I, [61, Theorem 1]. Termed weak solutions, they are actually based on thetopology of pointwise convergence.

But an even more palatable characterization of asymptotic consistency for(1.2) occurred simultaneously, namely, every finite subsystem of (1.2) is consistent, see [62, Theorem 2].

Itis very interesting that the second condition was also studied by S. N. Tschernikow in a series of papers in the early 60's, which are generally not widelyavailable ([85], [86], and [87]). We present some of his results in Section 4.3.

Let us illustrate the implications for duality of these characterizations byextending Example 1.

Example 4.2 (Extended Example 4.1) Let

inf Ul subject to Ul ~ +U2~ ~ 0, and - Ul ~ n, for all n = 1, ...(4.2)

Every finite subsystem of (4.2) is consistent. This means that there are solutions to (4.2) in the polynomial ring R[O]2 with Ul having negative infinite part.In fact, a feasible solution having negative infinite part is u(0) = (-0, 02 ) .

This implies that (4.2) is asymptotically unbounded (AUBD), an example of[61, Theorem 9].

Actually, the asymptotic limiting behavior of (4.2) suggests adjoining boundson the variables having different powers in 0, e.g. ,

but there are other possibilities too. It would be interesting to study the dualregularizations based upon these additional inequalities, but this chore is left tothe interested reader.

We define a (double) sequence comprising an asymptotically feasible solu tion to the dual program for (4.2) as follows. Let a E R, 1 < a < 2, and letN be the set of positive integers. Consider R[N] according to Definition 4.1and introduce,

(4.3)

defined as follows .

0, if n =1= mfor n = 1, ...

m", ifn = m(4.4)

0, ifn =1= mfor n = 1, ...

mo:- 1 - 1, ifn = m(4.5)

For verifying asymptotic consistency it suffices to proceed as follows .


(4.6)

(4.7)

For each m the objective function value is m (ma - l - 1) , which tends to+00 as m does. This example illustrates duality state # 11, defined by bothprograms being inconsistent, both being asymptotically consistent, and bothbeing asymptotically unbounded, see Table 1.1 in Section 5, and [67] , [5,5.5],[57, CaseI1], [72], [63), [64], and [79] .

Lemma 4.1 (1971 with Jeroslow [62]) Assume for every finite subset J CI , that the system U E R ffl

, UT P i ~ Ci , i E J is consistent. Then the semiinfinite system u E R'" ; UT P i ~ Ci , i E I is asymptotically consistent.

Theorem 4.4 ([6 2]) Assume for every finite subset J C I, that the systemu E R ffl

, UT P i ~ Ci, i E J is consistent. Then the semi-infinite systemu E R[Or , uT Pi ~ Ci, i E I is consistent, where R[O] is the polynomialring in the indeterminate O. Solutions need only have polynomial componentsof degree not exceeding dim W , W = lin span {Pi li E I} .

Example 4.3 Generally, more than one power of the indeterminate is required, a manifestation that the result is not a trivial consequence of Fenchel 'sasymptotic cones of 1953, see [78] or Debreu's of 1959, [27, asymptotic cone].

Ul

-nUl n = 1, ...

Any finite system is consistent, but more than one power of 0 is required, e.g.,u(O) = (0 ,( 2 ) .

Lemma 4.2 (197 1 Lemma of the Alternative [62]) Assume for everyfinitesubset .J C I, that the system u E R ffl

, uT Pi ~ Ci, i E J is consistent. Theneither

1. u E R ffl, uT P i ~ Ci, i E I has a solution, or


2. there exists u =1= °contained in the linear subspace spannedby {Pi liE I},and uT Pi ~ Ofor all i E I.

4.3 ACHIEVING TIlE DUALITY EQUALITY IN LINEARSEMI-INFINITE PROGRAMMING WITH INFINITE NUMBERS

We refer back to the linear semi-infinite inequality system (1.2).

Lemma4.3 Assume that for a finite subset J of the index set I. the asso ciated finite linear inequality system (4.8) is inconsistent

Then.

( _: ) E M m+1 (of Defill ition 1.3).

Proof. Consider the finite linear program

v» = minu,t -t

(4.8)

(4.9)

t ~ 0.

for all j E J.(4.10)

Clearly, (6, 0) is feasible. If (u, t) were feasible with t > 0, then Tis feasiblefor (4.8), which is a contradiction. Hence, v» = 0, and the dual program isconsistent, i.e., there exist non-negative>. satisfying

(4.11)

o

Lemma4.4 Assume the semi-infinite dual program is consistent withfinitevalue vt» Then for any finite subset J c I , (4.8) is consistent.

ON THE /962-/972 DECADE OF SIP: A SUBJECnVE VIEW 19

Proof. If for some finite J c I the inequality system (4.8) were not consistent,then by Lemma 4.3 there exists XJ such that

L Pj s, = 6and L Cj s, ~ l.J J

Let>' be a feasible point to Program D. Extend XJ trivially to all of I andform >. + k Xwith k > O. Clearly,

L Ci (.xi+ k Xi) -+ 00 as k -+ 00,

I

contradicting vD being finite. 0

Theorem 4.5 Assume the dual program in (1.1) is consistent with finitevalue VD. Then there exists u(B) E R[B]m such that

U(B)TPi ~ Ci, for all i E I and u(B)TPo = VD E R. (4.12)

Proof. We will show that for any finite subset J c I, the following system isconsistent.

Suppose to the contrary that there exists a finite subset J such that

u E R m , uT Pj ~ Cj, for all j E J implies uT Po > VD,

(4.13)

(4.14)

noting that the left-side inequality system is consistent because Program D of(Ll) is.

Consider the following dual pair of finite linear programs.

Primal PJ

subject to

Dual DJ

subject to

Now (4.14) implies that PJ is consistent and bounded. Therefore, by finitelinear programming theory, there exists UJ feasible for PJ and >./ feasible forDJ such that

U)Po = LCj AI·jEJ

(4.15)

Using (4.14) gives :EjEJ Cj AI > VD, which is impossible because VD is

finite (D of (Ll) assumed to be consistent). In particular, :EjEJ Cj AI ~ VD.

Hence, the assertion (4.13) has been proved. Therefore, by Theorem 4.4, thereexists u(B) E Rm[B] such that

U(B)TPi ~ Ci, for all i E I and u(BfPo <VD. (4.16)

For completeness we derive the elementary duality inequality.Given any u(B) E R[B]m satisfying u(B)TPi ~ Ci, for all i E I, and A

feasible for Program D, it follows that for each i , Ai u(B)TPi ~ Cj Ai, sinceAi ~ O. Since the support of Ais finite, we obtain,

U(B)T Po = L u(B)T~ Ai ~ L ci Ai-iEI i EI

Since A was arbitrary D-feasible, it follows that u(B)TPo ~ VD. In particular, for u(B) introduced in (4.16) we obtain:

(4.17)

Combining (4.16) with (4.17) gives the desired conclusion establishing theduality equality, namely,

o

With the machinery we have developed we can now review and prove a veryinteresting result of Tschernikow.

Theorem 4.6 ([87, Satz 7.5]) Let I be an arbitrary index set. Assumethat the moment cone (1.3) is a closed set. If every finite subsystem of (1.2) isconsistent, then (1.2) itself is consistent.

Proof. System (1.2) is inconsistent if and only if

uT Pi - t Cj ~ 0, for all i E I , implies - t ~ O. (4.18)

ON TilE 1962-1972 DECADE OF S1P: A SUBJECTIVE VIEW 21

But (4.18) implies that

(4.19)

Assume to the contrary that (1.2) were inconsistent. Since M m+1 is closed,this implies there exists a finite subset J of I and non-negative AiEJ such thatfor this particular subset, J, (4.11) holds. Let UJ be a feasible solution forthe finite subsystem of (1.2), known to exist by assumption. We obtain thefollowing contradiction.

(4.20)

o

5 A CLASSIFICATION OF DUALITY STATES BASEDON ASYMPTOTIC CONSISTENCY

In finite linear programming we are familiar with the four mutually exclusiveand collectively exhaustive duality states that can occur. Let 's begin with a lookat how these are slightly modified for linear semi-infinite programming.

Theorem 5.1 ([16, Theorem 6]) Assume uT II ~ Ci , for all i E I has theFarkas-Minkowski property (3.1) when consistent.

Then precisely one of the following cases occurs:

1. P is inconsistent with vD finite or +00,

2. both inconsistent,

3. D inconsistent, Vp = -00,

4. both consistent, Vp = VD, and VD attained.

Remark 5.1 State 1 is illustrated with: min {a U IU t2 ~ t, t E [0, I]}.VD is finite when a = 0, while for a = 1, VD = +00. This theorem is also citedin J. P. Evans [39, Appendix], where a variation in the dual program in (1.1)involved infinite denumerable sums . The formulations arose in Markov decisionproblems. Evans showed that for canonically closed systems, generalized finitesequences suffice in the semi-infinite programming dual.

In Section 1 we observed that semi-infinite programming in a "linearoperator, topological" sense is a special case of infinite programming. In Section 4.2 we reviewed convergence concepts that were employed in definingasymptotic consistency, for example.


We make a very brief contact with the general structure of Duffin type,[30], but quickly return to finite dimensions, actually with no serious loss ofgenerality.

Let E and F be real linear spaces and A a linear mapping from E to F.Let K be a convex cone in E, and let E# denote the linear space of linearfunctionals on E, and c E E# , se F .

(I) sup., E E (c,x) (II) infyEE # (b, y)(5.1)

s.t, Ax = b, x E K . s.t, yAx 2: (c, x), for all x E K .

It is interesting that Duffin in [33, page 402] presented a general definitionof when two programs are dual, and Kretschmer in [76] expanded on thisconstruction while also presenting examples. The programs in (5.1) are dual inthis sense, but of course much more can be said about their duality properties(see [76], [28], and [5]).

We simplify now the topological requirements in order to review asymptoticconsistency and the resulting classification of duality states, without deviatingfrom the most general infinite linear programming classification results. Fortunately, we can do this already in the n - space, and even illustrate all dualitystates in 3 - space using a variation of a closed convex cone introduced by Fan,[42, Remark 3]. We therefore proceed with the analogous infinite programming operator notation, but of course, the operators are merely matrices in thisspecialization. At least in this way we are able to present the first classificationtable that was developed during the years 1968-1970.

Returning to Rm space we review the following dual pair as given in [4]:

(I.C) sup (c, x)

s.t. Ax=b,xEC,

(II.C·) infyEJlm (b, y)

s.t. ATy - c E C·,(5.2)

where cERn, bERm, A is m x n, C is a closed convex cone contained inR n , and C· = {z E R n I (z,x) 2: 0, \/x E K}, termed the dual cone.

The asymptotic extensions of Programs (I.C) and (II.C·) are constructedas follows:

vr = sup {limk (c, x k) I lim Axk = b} (5.3)

{xk}CK k

ON THE 1962-19 72 DECADE OF SIP: A SUBJECTIVE VIEW 23

The definition of the nonasymptotic states of primal or dual ED,UED, INCapply to programs P and D and are the usual one s. The behavior of the limiting objective function values for programs 1,11 in (5.3) and (5.4), respectively,determine particular du ality states as summarized in the Table 1.1. There, allinfinite values are defined in the sen se of standard limits. (No 00 + or - valuesare formally assigned). The nomenclature: ABD, AUBD, P AG,1AG denoterespectively: Asymptotically BD, Asymptotically UBD , Properly AG, Improperly AG, where v denotes the respective program value of the subscriptedprogram name. For either program, should there exist no asymptotic sequences,then that program is termed strongly inconsistent (8 1NG) , [4], [69] . See alsoTable 1.2.

ABD I AUBD PAC lAC

(I .C) is AC - 00 < V'J < +00 V'J = + 00 V'J> -00 V'J = - 00

(II .C· ) is AC - 00 < V'JT < +00 V'JT =-00 'ItJT < +00 V'JT = + 00

Table 1.1 Asymptot ically Consistent Duality States for (I .C), (II.C·)

Remark 5.2 The Duffin [30] infinite programming duality inequalities involving subconsistencies were generalized for finite dimensional convex programming by Rockafellar through the ingenious use of perturbations, see [78,Chapters 29, 30]. With the appropriate defin ition of a convex bifunction, G, op.cit., these general duality inequalities appear as:

(inj G)(O) = (d(supG*))(O) ~ (supG*)(O) = (d(inj G))(O). (5.5 )

Gol'stein in [48] employs asymptotically feasible solutions, termed generalized f easible solutions, to deliver (5.5) also for finite dimensional convexprogramming.

Theorem 5.2 ([4 ]) Of the 49 mutually exclusi ve and collectively exhaustive duality states given in Table 1.2, only 11 are possible , and are those denotedin Table 1.2 by positive integers. A zero means that the duality state is impossible. The duali ty states of Table 1.2 are summarized reading down by columnsfrom top to bottom as follows.


AC, PAC, CON, ABD, BDAC. PAC, CON, AUBD, BDAC. PAC, CON. AUBD, UBDAC, PAC, INC. ABDAC. PAC. INC. AUBDAC. lAC. INC, lACSINC, INC, SINC

for duality state 1for duality state 5for duality states 7.8for duality state 2for duality states 9,11for duality state 3for duality states 4,6.10

P AC SINCD R PAC lAC

U I CON INCA M ABD AUBD

L AL BD UBD ABD AUBD lAC SINC

C ABD B- 1 0 0 0 0 0 00 AD- D 0 0 0 2 0 0 0

P N UB UBD 0 0 0 0 0 3 4A A I ABD 0 5 0 0 0 0 0C C N AUBD 0 0 0 0 11 0 6

IAC C lAC 0 0 7 0 0 0 0SINC SINC 0 0 8 0 9 0 10

Table 1.2 Semi-Infinite Programming Duality States, circa 1969-1970

The origins of classifying semi-infinite or convex programs appeared in [4],[63], [67], [72], [64], and [69]. Actually, Duffin had developed the tools backin 1956 ([30]), when he introduced infinite homogeneous programs, termedhomogeneous derivants in [67J and [69].

In [57] the authors considered semi-infinite programs formulated over Borelmeasures, and gave infinite dimensional space examples of all eleven dualitystates. In [73] there was a concentration on chemical equilibrium problems .This was motivated by a paper by Bigelow, DeHaven, and Shapiro [6], where theauthors discovered chemical equilibrium problems having unbounded solutionsets.

A technical report and correspondence with Murray Schechter of LehighUniversity in 1970 (see [80]), led to joint work with A. S. Soyster (see [74]). In1971 a paper was published that used classification to characterize in! /minsup/max combinations, see [72].

The following classification table between two any dual programs is aboutthe "closest" obtainable to ordinary finite linear programming.


(D)

I INC 4

Table 1.3 Four Possible Duality States for the Linear Semi-Infinite Programming Dual PairAssuming Closure of the Moment Cone (1.3)

There is an asterisk for duality state 1, because unlike finite linear programming, the primal program value in (1.1) may not be attained. Below is anexample with common objective function value "in! = 0", necessarily attained in the dual program. Over the non-Archimedean polynomial ring, R[O](see Definition 4.2), the primal value is attained, e.g., u(O) = (0, 0).

Primal

inf Ul

subject to

Ul + t U2 ~ VI, t E [0, 1].

Dual

VB = sup 2:t VIAt

subject to

At ~ O,t E [0,1].

6 ASYMPTOTIC LAGRANGE REGULARITY

Consider the classical nonlinear problem:

z; = inf {f(x) I9i(X) ~ 0, i = 1, ...m}.

Assumptions:

(A-I) f(x) is pseudo-convex,

(A-2) 9i (x), i = 1, ...m are pseudo-concave and continuously differentiable,


(A- 3) the constraint set is nonernpty and for each i, :3 xb such that 9i (xb) > 0,

(A-4) if Z* is finite, :3 E > 0 for which

Z ( E) = inf {f(x) I9i(X) ~ -E, , i = 1, ...m }

is finite. Moreover, limE-to Z(E) = Z* .

Definition 6.1 G = (91, ...9n) is asymptotically Lagrang e regular ifwhenever Z* is finite there exists {x y , Ay } such that Ay ~ 0 for each 1/ and

(the ALR condition)

4. limy-too p(x y , K) = O. (distance from K)

Theorem 6.1 Under assumptions (A -I) through (A-4), G( x) is weaklyasymptotic Lagrange regular, namely properties 1, 2, and 4 hold.

Remark 6.1 Generalizations of the 1967, [40], paper were achieved in"Asymptotic Conditions for Con strained Minimization" by Fiacco and MeCormick in 1968, [43]. Related work by Beltrami [3] and by S. Zlobec in [92]and [93] also appeared during this period. The approach in [40] and [41] differsfrom the one developed by M. Guignard in [51].

7 APPLICATIONS TO ECONOMICS, GAMETHEORY, AND AIR POLLUTION ABATEMENT

7.1 EFFECTIVE CONTROL THROUGH COHERENTDECENTRALIZATION WITH PRE-EMPTIVE GOALS

The possibility of achieving correct decentralized behavior under assumptions of strict convexity had been conjectured during the early 60 's. A. Whinston 's thesis at Carnegie Mellon also provided some motivation for this study,see [88], as well as work of Baumol and Fabian, [2], and Arrow and Hurwicz,[1]; see also [89]. In [8] the overall division-separable problem is formulatedas follows, termed the Total Problem.

ON THE /962-/972 DECADE OF SIP : A SUBJECTIVE VIEW 27

Let ¢i[Bi and Gi] be convex [concave] functions with compatible domains,and define:

Program (T)

min ¢1(ud + ¢2(U2) + ... + ¢m(um)s.t.

(7.1)

The formal definition of the division k transfer-priced problem is the following one:

min ¢k(Uk) - Gk(Uk)TA*s.t. Bk(Uk) ~ bk.

(7.2)

The duality pair used to study (7.2) involved a system of supporting hyperplanes. For convenience consider the special convex program, which forms afoundation of several convex programs in [8].

min ¢(u) subject to uTPi ~ ci,i = 1,n. (7.3)

Supporting hyperplanes, indexed by a set A, were introduced for the closedconvex set {(z, u)lz - ¢(u) ~ O} (the epigraph in modem terminology) givingrise to the following pair of dual semi-infinite programs for proving certainproperties of the underlying convex programs.

Primal

inf z

subject to

T .-U Pi 2: q,t = 1,n.

Dual

subject to

n« 2: 0, ex E A, Ai 2: 0, i = 1, n.(7.4)

Theorem 7.1 ([8, Theorem 2]) Assume that an optimal solution exists forthe Total Problem (7.1) and that each <Pk is a strictly convex junction for eachk and all constraints are linear; i.e.•

Then there exists a price vector A· such that each sub-problem (Tk ) has aunique solution. which when taken together form an optimal solution to thetotal problem (T).

When strict convexity is not present, then additional information than pricesalone must be sent to the divisions in order to insure optimal division and hencetotal firm solutions. The notion of preemptive goals are introduced for thepurpose of achieving proper coordinated decentralized decisions in the nonstrictly convex case.

An application to the political economy of public school systems appears in[26]. Remarks on the "status" of separability in decentralization appeared in[22], with a further update in the partially survey-type paper, [68].

Remark 7.1 It has long been recognized that any nonlinear convex program such as (7.3) can be reformulated as an infinite linear program by use ofsupporting hyperplanes, see [38] and [33].

7.2 ON N -PERSON GAMES AND DUALITY THEORY

In the mid-60's there emerged a special interest in formulating solution concepts for n-person characteristic function games as mathematical programs.These models usually involved conditions on the excesses of coalition valueswith respect to payoff vectors. A class of solution concepts for characteristicfunction games was developed in Charnes and Kortanek [23] based upon theexcess functions.

ON THE 1962-1972 DECADE OF SIP: A SUBJEClIVE VIEW 29

Bondareva [7] was the first to use finite linear programming to characterizenonemptiness of the core. In [21] the duality theory of linear programmingwas also used to characterize the core of an n-person game via Shapley'sminimal balanced collections, [82], and to answer in the affirmative a Shapleyconjecture on the sharpness of proper minimal balanced collections. In [21] aproper operator M (.) was defined on coalitions to characterize the redundancyof certain coalition inequalities, see [70]. Basically, this operator associateswith each coalition the best weighted value among all coalitions which arebalanced with respect to the grand coalition. Schmeidler [81] defined a gamewith infinitely many players and extended the concept of the core and the conceptof a balanced game. He showed that an infinite player game for which the rangeof values is bounded has non-empty core if and only if it is balanced. He alsoextended the M(·) to infinite player games.

To briefly review the constructions at that time we begin with the definitionof a generalized game. The review follows the paper of Chames, Eisner, andKortanek, [19].

Definition 7.1 Given Van arbitrary vector space, and a subset X termedthe set ofcoalitions. A special member ofx' denoted by X o is called the grandcoalition. The following properties are required.

A. X spans V,

B. for each X in X there is a finite subset {Xl, X 2 , •••X n } C X, andnon-negative numbers {1]I, 172 , ...1]n} and 1]* such that

m

L 1]i Xi + 1]* X = X o·i= l

(7.5)

Property B states that each coalition can be incorporated in an expression ofXo as a weighted sum of coalitions, the weights being positive. If an orderingon V is induced by the convex cone spanned by X, then B becomes: X E Ximplies X o 2: 1]* X. There is no loss of generality under A. In addition anarbitrary function v from X to R is given, termed the payoff function. Thetriple (X, X o, v) is called a generalized game.

Definition 7.2 The collection of all subsets of X is denoted by S = 2x.

The set ofall real-valued junctions defined on S is denoted as usual by R[sJ,a generalized finite sequence space. A game is termed weakly balanced if

SUP1/E n[S) {LXcx ttx v(X) I Lxcx ttx X = X o, 1] 2: o} is finite.

(7 .6)


Primal

VI = inf >,(Xo)

subject to

>'(X) 2: v(X), for all X C X

>. E V# (linear functionals on V) .

Dual

VII = SUP7JE R[S) Lxcx rJX v(X)

subject to

Lxcx nx X = X o

(7.7)

The duality inequality is easily seen . Given>. E V#, rJ E R[Sj with rJ 2: 0:

>,(Xo) = x (L nx X) = L nx>'(X) 2: L nxv(X).xcx xcx xcx

Definition 7.3 An outcome ofa generalized game R[Sj is a member ofV#(a linear junctional on V) such that >,(Xo) = v(Xo). An outcome>' is said tobe in the core of the game iffor each X C x, >'(X) 2: v(X) . The game iscalled weakly balanced if the program value VII is finite.

A necessary condition for the core of a weakly balanced game to be nonempty is that v(Xo) = VII, as established [19].

Definition 7.4 Given a 'l/J C X an extended value junction M'IjJ V :-+[-00, 00] is defined by:

Let g = 2'IjJ .

M'IjJ(X) = -00 if there is no feasible rJ, +oofor unbounded program value .

Termed the M Operator, it is a concave function, positive homogeneousof degree 1. It is closely related to Schmeidler's operator introduced in [81].Without going into technical details this operator may be used to test whetheran inequality in the system, >'(X) 2: v(X), X C X is redundant.

7.3 NUMERICAL ASPECTS OF POLLUTION ABATEMENT

The U. S. Congress 1970 Clean Air Amendments required compliance withair quality standards for specified pollutants such as sulfur dioxide, particular

ON THE 1962-1972 DECADE OF SIP: A SUBJEC71VE ViEW 31

matter, hydrocarbons, and carbon monoxide. An air quality standard is a specified ambient concentration not to be exceeded anywhere in the air quality regionover a specified period of time. Our approach focused on relatively chemicallyinert pollutants such as the first two in the list above. The goal was to calculatean emission reduction policy which guarantees that the air quality standard is"met throughout the region" , while the costs caused by the forced reductionsare minimized. The quoted phrase led to modeling over a continuum, and adual pair of linear semi-infinite programs was developed similar in structure to(2.2).

A computational method was developed that takes into account the combined emissions from all known sources and incorporated the meteorologicalprocesses that cause pollutants to be dispersed between the sources of pollutionand ground level locations. In Allegheny County of Pennsylvania the policy toreduce the emission of sulfur dioxide as much as technically possible was compared with alternative policies, and different courses of action were evaluated.

Allegheny County adopted sulfur dioxide regulations which did not call forthe largest reductions possible but determined a policy which was less costlywhile still meeting the U. S. Federal air pollution standard. The emission reduction strategy was adopted by the Commonwealth of Pennsylvania and becamean integral part of the State's Implementation Plan, which was a requirementof the 1970 U. S. Clean Air Amendments and was ultimately approved by theU. S. Federal Government.

We choose not to review the detail of the modeling effort, but make citationsto [71], [49], and [55]. A concise description of the air pollution model appearedin the Chiu and Ward review of the 1979 book of W. Krabs, sec [24] and [75].At a later point in time there occurred additional reactions to this approach, see[84] and [50].

8 ALGORITHMIC DEVELOPMENTS: "MATCHINGOF THE DERIVATIVES"

Sven-Ake Gustafson visited Cornell University under the auspices of theCornell's Applied Mathematics Center in the Fall of 1968. In preparation forreviewing his numerical approach for solving semi-infinite programs, we returnto some dual formulations with a double purpose.

For the convenience of making a comparison with the equivalent momentprogram duality of Section 2, we introduce similar properties, except that wedelete the constraints Yr 2 0, r = 1, m. Program A corresponds to ProgramMoment (2.3), while Program B corresponds to Program Dual GLP, (2.2).

Let U r , r = 1, m, and -Um+l be continuously differentiable convex functions defined on a convex set S C R m and b CH":


Program A

subject to 'ExES Ur(X)A(X) = b- ,

Program B

A E R[S] A > 0, -,

r= I,m.

(8.1)

Find inf 'E~=l Yrbr, for all Y E R m,

subject to 'E~l Yrur(x) ~ Um+l (x), for all xES.

When Program A is consistent, an immediate elementary reduction is obtained from finite linear programming theory (or by Caratheodory's Lemma),namely, that no more than m points are needed in defining feasible points,A('), in Program A. This observation led to the development of necessary andsufficient conditions for primal and dual feasible points to be optimal, apparently appearing for the first time in Gustafson [52]. Building on Gustafson'sapproach, it is convenient to summarize these conditions in the following theorem.

Theorem 8.1 Let {Ai == A(Xi), x i E S, i = 1,m} and Y he dual feasihie solutions for Programs A and B respectively. Then these are optimal fortheir respective programs if and only if the following two conditions hold:

Complementary Slackness:Ai ['E~=l YrUr(Xi) - Um+l (xdl = 0, i = 1,m.

Local Minimality :'E~=l Yrur(x) - um+l(x) has a local minimum at Xi whenever Ai > O.

The Program A feasibility condition, 'E~l U(Xi)Ai = b, was added tothe above two conditions to form what Gustafson termed System NL. As a"stand-alone" system, the three conditions provided necessary conditions foroptimality. However, any solution to System NL which also satisfied Ai ~

0, i = 1, m, with Y being Program B-feasible, formed a pair of respectivedual optimal solutions. System NL provided an opportunity to bring nonlinearequations solvers to bear on linear semi-infinite programming. The earliestknown algorithm of this type appeared in Gustafson, [52, Section 4].

Gustafson visited Carnegie Mellon University at the outset of the 70's, andthe following papers emerged on the numerical treatment of semi-infinite programming: [54], [53], and [56].

ON THE 1962-1972 DECADE OF S1P: A SUBJECTIVE VIEW 33

9 EPILOG

As stated, 1962-1972 is the first of almost four decades of active researchand applications of semi-infinite programming. To provide some brief evidence of these activities , the following events are listed in an abbreviated form.Throughout this period there have been numerous talks at professional meetingsand universities.

1. Semi-infinite Workshop in Bad Honnef, Germany, 1978

2. Semi-infinite Programming and Applications, Austin Texas Symposium,1981

3. State of the Art Talk, "Semi-infinite Programming and Applications",XI International Symposium on Mathematical Programming, Bonn, Germany, 1982

4. Infinite Programming Symposium, Cambridge, United Kingdom, 1984

5. Cluster Section, Systems and Management Science by Extremal Methods, Research Honoring Abraham Charnes at Age 70, Austin, Texas,1987

6. Cluster Section, XIII World Conference on Operations Research, Lisbon,Portugal, 1993

7. Cluster Section, IV International Conference on Parametric Optimizationand Related Topics, Enschede, The Nederlands, 1995

8. Semi-infinite Programming Workshop, Cottbus, Germany, 1996

9. Cluster Section, XVI International Symposium on Mathematical Programming, Lausanne, Switzerland, 1997

10. Cluster Section, International Conference on Nonlinear and VariationalInequalities, Hong Kong, China, 1998

11. Semi-infinite Programming Symposium, Alicante, Spain, 1999

12. Cluster Section, EURO VII, Budapest, Hungary, 2000

13. Cluster Section, XVII International Symposium on Mathematical Programming, Atlanta , Georgia, 2000


Acknowledgements

The comments and corrections made by the two referees were invaluable forthe final preparation of this paper. Their persistent assistance to achieve accuracy together with their help to minimize the probable, perhaps unavoidable,"self-serving" nature of this history is most appreciated. In addition, both editors of this Volume provided important inputs to avoid additional embarrassingmisstatements. It will be up to the reader to decide upon the degree to whichthe author has succeeded in accuracy and objectivity.

References

[1] K. J. Arrow and L. Hurwicz . Decentralization and computation in resourceallocation. In P.W. Pfouts, editor, Essays in Economics and Econometrics,pages 34-104, University of North Carolina, Chapel Hill, N. C., 1960.

[2] W. J. Baumol and T. Fabian. Decomposition, pricing for decentralizationand external economies, Management Science, 11:241-261, 1964.

[3] E. 1. Beltrami. A constructive proof of the Kuhn-Tucker multiplier rule,Journal ofMathematical Analysis and Applications, 26:297-306, 1967.

[4] A. Ben-Israel, A. Chames, and K. O. Kortanek. Duality and asymptoticsolvability over cones, Bulletin of the American Mathematical Society,75:318-324, 1969. Erratum, Bulletin ofthe American Mathematical Society 76 (1970) 428 .

[5] A. Ben-Israel, A. Chames, and K. O. Kortanek. Asymptotic duality insemi-infinite programming and the convex core topology, Rendiconti DiMatematica (Rome), 4:751-761, 1971. Presented at the Conference onMapping Techniques and Problems University of Houston, November,1970 in honor of Professor David Bourgin.

[6] J. H. Bigelow, J. C. DeHaven, and N. Z. Shapiro. Chemical equilibriumproblems with unbounded solution sets, SIAM Journal on Applied Mathematics, 18:768-775, 1970.

[7] O. N. Bondareva. Nekotorye primeneniia metodor linejnogo programmirovaniia k teorii kooperativnykh igr., Problemy Kibernetiki, 10:119139, 1963. Translation of Title: Some applications of linear programmingmethods to the theory of cooperative games.

[8] A. Chames, R. W. Clower, and K. O. Kortanek. Effective control throughcoherent decentralization with pre-emptive goals, Econometrica, 35:294320, 1967.

[9] A. Chames and W. W. Cooper. The strong Minkowski-Farkas-Weyl theorem for vector spaces over ordered fields, U. S. Proceedings ofthe NationalAcademic ofSciences, 44:914-916, 1969.

ON THE 1962-1972 DECADE OF S1P: A SUBJECTIVE VIEW 35

[10] A. Chames and W. W. Cooper. Management Models and Industrial Applications ojLinear Programming, Volumes I and Il, Wiley, 1961.

[11] A. Chames, W. W. Cooper, and K. O. Kortanek. Duality, Haar programsand finite sequence spaces, U. S. Proceedings of the National Academyoj Sciences, 48:782-786, 1962.

[12] A. Chames, W. W. Cooper, and K. O. Kortanek. A duality theory forconvex programs with convex constraints, Bulletin ofthe American Mathematical Society, 68:605-608, 1962.

[13] A. Chames, W. W. Cooper, and K. O. Kortanek. Duality in semi-infiniteprograms and some works of Haar and Caratheodory, Manag ement Science, 9:208-228, 1963.

[14] A. Chames, W. W. Cooper, and K. O. Kortanek. On representation ofsemi-infinite programs which have no duality gaps, Management Science,12:113-121, 1965.

[15] A. Chames, W. W. Cooper, and K. O. Kortanek. On some nonstandardsemi-infinite programming problems. Technical report No. 45, CornellUniversity, Department of Operations Research, Ithaca, N. Y. , March1968.

[16] A. Chames, W. W. Cooper, and K. O. Kortanek. On the theory of semiinfinite programming and some generalizations of Kuhn-Tucker saddlepoint theorems for arbitrary convex functions , Naval Research Logisti csQuarterly, 16:41-51, 1969.

[17] A. Chames, W. W. Cooper, and K. O. Kortanek. Semi-infinite programming, differentiability, and geometric programming Part II, AplikaceMatematicky (Prague) , 14:15-22, 1969.

[18] Ai -Chames, W. W. Cooper, and K. O. Kortanek. Semi -infinite programming, differentiability, and geometric programming,- Journal oj Mathematical Sciences, 6:19-40, 1971. R. S. Varma Memorial Volume.

[19] A. Chames, M. 1. Eisner, and K. O. Kortanek . On weakly balanced gamesand duality theory, Cahiers du Centre d 'Etude de Recherche Operationnelle (Belgium), 12:7-21, 1970.

[20] A. Chames and K. O. Kortanek. An opposite sign algorithm for purification to an extreme point solution. O. N. R. Research Memorandum No. 84,Northwestern University, The Technological Institute, Evanston, Illinois,June 1963.

[21] A. Chames and K. O. Kortanek . On balanced sets, cores , and linearprogramming. Cahiers du Centre d'Etude de Recherche Operationnelle(Belgium), 9:32-43, 1967.

36 SEMI-INFlNI1E PROGRAMMING. RECENT ADVANCES

[22] A. Chames and K. O. Kortanek. On the status of separability and nonseparability in decentralization theory, Management Science: Applications, 15:B12-B14, 1968.

[23] A. Chames and K. O. Kortanek. On classes of convex and preemptivenuclei for ri-person games. In H. W. Kuhn, editor, Proceedings of thePrinceton Symposium on Mathematical Programming, pages 377-390.Mathematical Programming Society, Princeton University Press, 1970.

[24] C. K. Chui and J. D. Ward. Book review of "Optimization and Approximation" by W. Krabs, Wiley, 1979. Bulletin ofthe American Mathematical Society, 3:1056-1069, 1980.

[25] G. B. Dantzig. Linear Programming and Extensions, Princeton UniversityPress, 1963.

[26] O. A. Davis and K. O. Kortanek. Centralization and decentralization: thepolitical economy of public school systems, American Economic Review,61:456--462, 1971.

[27] G. Debreu. The01Y ofValue, Wiley, 1959. Cowles Foundation for Researchin Economics at Yale University, Monograph 17. Copyright renewed 1987for the Yale University Press .

[28] U. Dieter. Optimierungsaufgaben in topologischen Vektorraumen I: Dualitatsheorie, Z. Wahrscheinlichkeitstheorie verw, 5:89-117, 1966.

[29] W. S. Dom. Duality in quadratic programming, Quarterly Journal ofApplied Mathematics, 20:155-162, 1960.

[30] R. J. Duffin. Infinite programs, In H. W. Kuhn and A. W. Tucker, editors, Linear Inequalities and Related Systems, pages 157-170. PrincetonUniversity Press, 1956.

[31] R. J. Duffin. Dual programs with minimum cost, J. Society on Industrial& Applied Mathematics, 10:119-123, 1962.

[32] R. 1. Duffin. An orthogonality theorem of Dines related to moment prob lems and linear programming, Journal ofCombinatorial Theory, 2: 1-26,1967.

[33] R. J. Duffin. Duality inequalities ofmathematics and science, In NonlinearProgramming, pages 401-423. Academic Press, 1970.

[34] R. J. Duffin. Linearizing geometric programs, SIAM Review, 12:211-227,1970.

[35] R. J. Duffin and L. A. Karlovitz. An infinite linear program with a dualitygap, Management Science, 12:122-134, 1965.

[36] R. 1. Duffin, E. L. Peterson, and C. L. Zener. Geometric Programming Theory and Applications, Wiley, 1967.

ON IHE 1962-1972 DECADE OF SIP: A SUBJECTIVE VIEW 37

[37]

[38]

[39]

[40]

[41]

[42]

[43]

[44]

[45]

E. Eisenberg. Duality in homogeneous programming, Proceedings oftheAmerican Mathematical Society, 12:783-787, 1961.

E. Eisenberg. Supports of a convex function, Bulletin of the AmericanMathematical Society, 68:192, 1961.

J. P. Evans. Duality in Markov decision problems with countable actionand state spaces, Management Science, 15:626-638, 1969.

J. P. Evans and K. O. Kortanek . Pseudo-concave programming and Lagrange regularity, Operations R~search, 15:882-891, 1967.

J. P. Evans and K. O. Kortanek. Asymptotic Lagrange regularity forpseudo-concave programming with weak constraint qualification, Operations Research, 16:849-857, 1968.

K. Fan. Asymptotic cones and duality of linear relations, Journal ofApproximation Theory, 2:152-159, 1969.

A. V.Fiacco and G. P.McCormick. Asymptotic conditions for constrainedminimization. Technical Report RAC-TP-34O, Research Analysis Corporation, McLean, Virginia, 1968. The firm no longer exists.

D. Gale. A geometric duality theorem with economic applications, ReviewofEconomic Studies, 34:19-24, 1967.

W. Gochet, Computational Treatment ofSome Linear Programming Optimization Problems with Applications to Geometric Programming andProbabilistic Programming. PhD thesis, Carnegie Mellon University,Graduate School of Industrial Administration, Pittsburgh, Pennsylvania,June 1972.

[47]

[48]

[46]

[49]

W. Gochet , K. O. Kortanek, and Y. Smeers . On a classification schemefor geometric programming and complementarity. Technical report,Carnegie-Mellon University, Graduate School of Industrial Administration, Pittsburgh, Pennsylvania, October 1971.

W. Gochet and Y. Smeers. On the use of linear programs to solveprototypegeometric programs. CORE Discussion Paper No. 7229, University ofLouvain, Brussels, Belgium, November 1972.

E. G. Gol'stein. Theory ofConvex Programming, Translations of Mathematical Monographs, Vol. 36 of American Mathematical Society, Providence, 1972.

W. L. Gorr, S. -1\. Gustafson, and K. O. Kortanek. Optimal control strategies for air quality standards and regulatory policy, Environment andPlanning, 4:183-192, 1972.

[50] H. J. Greenberg. Mathematical programming models for environmentalquality control, Operations Research, 43:578-622, 1995.


[51J M. Guignard. Generalized Kuhn-Tucker conditions for mathematical programming problems in a Banach space , SIAM Journal on Control , 7:23224 1, 1969.

[52] S. -1\. Gustafson. On the computational solution of a class of generalizedmoment problems, SIAM Journal on Numerical Analysis , 7:343-357,1970.

[53] S. -1\. Gustafson. Nonlinear systems in semi-infinite programming, Series in Numerical Optimization & Pollution Abatement Technical ReportNo. 2, Carnegie Mellon University, School of Urban and Publi c Affairs,Pittsburgh, Pennsylvani a, July 1972.

[54] S. -1\. Gustafson and K. O. Kortanek. Numerical treatment of a classof semi-infinite programming problems, Institute of Physical PlanningTechnical Report No. 21, Carnegie Mellon University, School of Urbanand Public Affairs, Pittsburgh, Pennsylvania, August 1971.

[55] S. -1\. Gustafson and K. O. Kortanek. Analytical properties of somemultiple-source urban diffusion model s, Environment and Planning ,4:31-41 ,1972.

[56] S. -1\. Gustafson and K. O. Kortanek. Numerical treatment of a class ofconvex problems, Series in Numeri cal Optimization & Pollution Abatement Technical Report No.4, Carnegie Mellon University, School of Urban and Public Affairs, Pittsburgh, Pennsylvania, July 1972.

[57] S. -1\. Gustafson, K. O. Kortanek, and W. O. Rom. Non-chebysevianmoment problems, SIAM Journal on Numerical Analysis, 7:335-342,1970.

[58] A. Haar. Dber lineare ungleichungen, Acta Universitatis Szegedienis,2:1-14, 1924.

[59] G. H. Hardy. Orders of Infinity, Cambridge Tracts in Mathematics andMathematical Physics. Cambridge University Pres s, Cambridge, U. K.,1954.

[60] G. H. Hardy. A Course ofPure Mathematics, Cambridge University Press,U. K., tenth edition, 1958.

[61] R. G. Jcroslow and K. O. Kortanek. Algebraic Hilbert field characterizations of asymptotic duality states and optimal paths to infinity, Technicalreport , Carnegie-Mellon University, Graduate School ofIndustrial Administration, Pittsburgh, Pennsylvania , August 1970. Management SciencesResearch Report No. 215.

[62] R. G. Jeroslow and K. O. Kortanek. On semi-infinite systems of linearinequalities, Israel Journal of Mathematics, 10:252-258, 1971.

ON TilE 1962-1972 DECADE OF SIP: A SUBJECTIVE VIEW 39

[63] C. Kallina and A. C. Williams. Duality and solvability theorems overcones, Technical report, Mobil Research and Development Corporation,Princeton, N. J., August 1969.

[64] C. Kallina and A. C. Williams. Linear programming in reflexive spaces,SIAM Review , 13:350--376, 1971.

[65] S. Karlin and W. S. Studden. Tchebychev Systems: with Applications inAnalysis and Statistics, Wiley, 1966.

[66] K. O. Kortanek. Duality, Semi-Infinite Programming and Some AspectsofControl in Business and Econom ic Systems, PhD thesis, NorthwesternUniversity, Field of Engineering Science, Evanston, lllnois, June 1964.

[67] K. O. Kortanek. Compound asymptotic duality classification schemes,Technical report, Carnegie-Mellon University, Graduate School of Industrial Administration, Pittsburgh, Pennsylvania, November 1969. Management Sciences Research Report No. 185.

[68] K. O. Kortanek. Effective control through coherent decentralization inseparably and non-separably structured organizations, In R. Chisholm,M. Radnor, and M. F.Tuite, editors, Interorganizational Decision Making ,pages 70--82. Aldine (Chicago), 1972.

[69] K. O. Kortanek. On a compound duality classification scheme with homogeneous derivants, Rendiconti Di Matematica (Rome), 5:349-356, 1972.

[70] K. O. Kortanek and J. P. Evans. On the 'M-operator' and redundantinequalities of the core of a game. Technical report No. 43, Cornell University, Department of Operations Research, Ithaca, New York, February1968.

[71] K. O. Kortanek and W. L. Gorr. Numerical aspects of pollution abatement problems: opt imal control strategies for air quality standards. InM. Henke, A. Jaeger, R. Wartmann, and J. H. Zimmerman, editors, Proceedings in Operations Research, pages 34-58. Physic-Verlag (WurzburgWien), 1972.

[72] K. O. Kortanek and W. O. Rom. Classification schemes for the strongduality oflinear programming over cones, Operations Research, 19:15711585, 1971.

[73] K. O. Kortanek and A. L. Soyster. On classification schemes ofsome solution sets of chemical equilibrium problems. Institute of Physical PlanningTechnical Report No.6, Carnegie Mellon University, School of Urban andPublic Affairs, Pittsburgh, Pennsylvania, July 1970.

[74] K. O. Kortanek and A. L. Soyster. On refinements of some duality theorems in linear programming over cones, Operations Research, 20: 137142, 1972.


[75] W. Krabs. Optimierung und Approximation. B. G.Teubner, Stuttgart,Germany, 1979.

[76] K. S. Kretschmer. Programmes in paired spaces, Canadian Journal ofMathematics, 13:221-238, 1961.

[77] R. T. Rockafellar. Duality theorems for convex functions, Bulletin of theAmerican Mathematical Society, 70:189-192, 1964.

[78] R. T. Rockafellar. Convex Analysis. Princeton University Press, 1970.

[79] W. O. Rom. Classification Theory in Mathematical Programm ing andApplications. PhD thesis, Cornell University, Industrial Engineering andOperations Research, Ithaca, N. Y., June 1970.

[80] M . Schechter. Linear programs in topological vector spaces. Technicalreport, Lehigh University, Bethlehem, Pennsylvania 18015, 1970. Collegeof Arts and Science Report.

[81] D. Schmeidler. On balanced games with infinitely many players. Researchmemorandum no. 28, Hebrew University, Department of Mathematics,Jerusalem, Israel, 1967.

[82] L. S. Shapley. On balanced sets and cores, Naval Research LogisticsQuarterly, 14:32-43, 1967.

[83] Y. Smeers. Geometric Programming and Management Science. PhDthesis, Carnegie Mellon University, Graduate School of Industrial Administration, Pittsburgh, Pennsylvania, June 1972.

[84] R. E . Train and J . Carroll. Environmental Management and Mathematics,SIAM News, 7:2-3, 1974. In 1974 Russell Train was the Administrator ofthe Federal Env ironmental Protection Agency (EPA) and James Carrollwas a member of Office and Planning Management at the EPA.

[85] S. N. Tschernikow. 0 teoreme chaara dlja beskonetschnych sistem linejnych neravenctv, Uspechi matem. nauk, 113:199-200, 1963. Translationof Title: On Haar's theorem about infinite systems of linear inequalities.The journal Uspechi matem. nauk translates into Successes in Math. Sciences. As far we know, there exists no English translation of this journal.

[86] S. N. Tschernikow. Poliedraljno samknutye sistemy linejnych neravenstv,Doklady Akad. Nauk SSSR, 161:55-58, 1965. Translation: Polyhedrallyclosed systems of linear inequalities. Soviet Math. Doklady, 6:381-384.

[87] S . N. Tschernikow. Lineare Ungleichungen, chapter 7. Deutscher Verlagder Wissenschaften Berlin, 1971. Translation from the Russian Linejnyeneravenstva (Linear inequalities) published 1968 by Nauka in Moscow.

[88] A. Whinston. Price Coordination in Decentralized Systems. PhD thesis,Carnegie Mellon University, Graduate School of Industrial Administration, Pittsburgh, Pennsylvania, June 1962. Office ofNaval Re search MemoNo. 99.

ON TH E 1962-1 972 DECAD E OF SIP: A SUBJECTIVE VIEW 41

[89] A. Whinston. Price guides in decentralized organizations. In W. W.Cooper, H. J. Levitt, and M. W. Shelly, editors, New Perspectives in Organizational Research . Wiley, 1964.

[90] A. Whinston. Some Applications of the Conjugate Function Theory toDuality, In J. Abadie, editor, Nonlinear Programming, chapter 5. Wiley,1967.

[91] Y. J. Zhu. Generalizations of some fundament al theorems on linear inequalities, Acta Mathernaticae Sinica, 16:25-40, 1966.

[92] S. Zlobec. Asymptotic Kuhn-Tucker conditions for mathem atical programming problems in a Banach space, SIAM Journal on Control, 8:505512, 1970.

[93] S. Zlobec. Extensions of asymptotic Kuhn-Tucker conditions in mathematical programming, SIAM Journal on Applied Mathematics, 21:448460 , 1971.

Part II THEORY

Chapter 2

ABOUT DISJUNCTIVE OPTIMIZATION

Ivan I. Eremin

Departm ent ofMathemati cal Programming, Institute ofMathematics and Mechanics, S.Kovalevskaia

16. 620219 Ekaterinburg, Russia

[email protected]

Abstract In this paper we investigate the problems of disjunctive programming with an intinite array of components forming a feasible set (as their union) . The investigationcontinues a theme of the author's article [7] and describes original conceptualapproach to a) analysis of a saddle point problem for disjunctive Lagrangian functions , b) analy sis of dual relations for disjunctive programming problems, and c)technique of equivalent (with respect to argument) reduction of such problems tothe problems of unconstrained optimization .

1 INTRODUCTION

Many problems in mathematical optimization theory may be disintegrated,through suitable transformations, into a lot of sub-problems of more simplenature. In particular, continuous linear piece-wise programs [7] are of suchkind. They can be written as . max max {(c, X) IAjX ~ bj} or, equivalently,

J=t, ... ,m

as max {g(x) Ix E TI;tMj}, where g(x) = .max (c,Xj), x = {Xj} ,J=I ,.. .,m

Xj E Mj = {Xj IAjxj ~ bj}. A more general instance is sup {f(x) Ix E

TI Ma}, x = {xa} , where the function f(x) may be of the form sup fa(xa)aEO aEOor inf f a(xa) , and the index set n may be considered as infinite. In this article

aEOwe analyze such problems under a special point of view, which will be soonexplained.

45

M.A. Goberna and M.A. Lop ez (eds .), Semi-Infinite Programming, 45-58.© 2001 Kluwer Academic Publishers.

46 SEMI-INFINI1E PROGRAMMING. RECENT ADVANCES

We shall start with some definitions. Let {Fo(X)}oEn be a given set of

vector-valued functions defined on R" : x Fo) Fo(x) E Rmo. The inequality

sup lFo(x) lmax ::; 0, x ~ 0,oEn

will be called conjunctive, and the inequality

inf lFo(x)lmax < 0, x ~ 0,oEn

(1.1)

( 1.2)

will be called disjunctive. Here j· lmax denotes the discrete maximum operation;i.e., if zT = (Zl , ... , Zk) , then Izlmax = . max Zi. In both cases constraints

1=1, ... ,kx ~ 0 are included for convenience, in particular, for convenience of dualframework.

Define Mo := {x ~ 0 IFo(x) ::; O}. The solution sets u; and u., forthe systems (1.1) and (1.2) may be written respectively as M n = n M o and

oEnM u = U u.; The problem

oEn

Pn := sup {f(x) Ix E Mn } (1.3)

is very standard in mathematical programming (MP) theory. The problemsup {J(x) Ix EMu}; i.e.,

Pu : sup {J(x) I inf lFo(x) lmax ::; 0, x ~ O} (=: ,) (1.4)oEn

is known as a problem of disjunctive programming [1-5, 7, 9-16]. We areinterested in a special case of (1.4), namely:

sup {(c, x) I inf IAox - bolmax ::; 0, x ~ O}. (1.5)oEn

In what follows we shall use some Lagrangian functions associated with (1.3)and (1.4). If the set n is finite, n = {I, ... , m}, then the classical Lagrangian

mfor(1.3)iswell-known:f(x)- L:(Uj, Fj(x)) ,whereuT := (U1, ... , um) ~

j =l

O. Along with this, one can operate with a function Fo(x , u) := f(x). max (Uj , Fj (x)) too; the character of the connection between this functionl=l ,... ,m

and the problem

sup {J(x) IFj(x) ::; 0, j = 1, ... , m, x ~ O} (1.6)

is the same as the connection between it and classical Lagrangian; i.e., if the pair[x, u] denotes a saddle point of Fo(x, u) , then x E Arg (1.6) ; i.e., it belongs to

ABOUT DISJUNCTIVE OPTIMIZATION 47

the solution set of (1.6). This is a good reason to define a Lagrangian functionfor the general conjunctive instance (1.3) similarly:

Fn(x, u) := f(x) - sup(uo , Fo(x)),oEn

U o ~ O. (1.7)

Analogously, with a disjunctive problem (1.4) we shall associate a Lagrangianfunction

Fu(x, u) = f( x) - inf (uo, Fo(x)) ,oEn U o ~ O. (1.8)

Along with (1.7) and ( 1.8) we shall exploit also some modified Lagrangians:

F~(x, u) := f(x) - sup(uo , F:(x)) ,oEn

U o ~ 0,

Uo ~ 0;

where the super-index "+" means positive cut; i.e. , if zT = (Zl' .. . , Zk), then(zT)+ = (zt , .. . , zt), zt = max {O, Zi}. Emphasize that in (1.7)+ and(1.8)+ the set {uo ~ O} denotes a system of Lagrange multipliers, which,under some conditions, can be fixed at a level Ro > 0 such that the problem(1.3) is equivalent to

sup F~(x, R),x>o

and the problem (1.4) is equivalent to

supF~(x , R) ,x>o

(1.9)

(1.10)

with R = {Ro}oEn. This follows from well-known exact pen alty functionframework [6, 17].

We shall use the following notions of solvability of (1.4).

Definition 1.1 A problem (1.4) is said to be value-solvable. if its optimalvalue, is finite.

Definition 1.2 A value-solvable problem (1.4) is said to be Arg-attainable,if there exists Q: such that, = max {f(x) IFa(x) :::;: 0, x ~ O}. In this case avector ii E Arg max{J(x) Ix E Ma} will beoptimaljor(1.4) in the standardsense .

Definition 1.3 A value-sol vable problem (1.4) is said to be Arg-solvablein general, if there exists a sequence {XOk E Arg POk } converging to a vectorx such that f(x) =" where Po : sup {J(x) Ix E Mo } .

Clearly, a vector x may belong or not to a feasible set M u.

In this paper we shall consider the following topics:

a) a saddle point framework for disjunctive Lag rangians assoc iated with(1.4) and for (1.10);

b) a duality framework for disjunctive programming problems;

c) an exact penalty function technique for (1.4).

2 SADDLE POINTS OF DISJUNCTIVELAGRANGIAN

We define a saddle point [x, u] ~ 0 for the function (1.8) by means of thestandard pair of inequalitie s:

Fu(x , u) < Fu (x, u)Vx~O

< Fu (x, u).Vu~O

(2.1)

A saddle point for Fn (x, u) is defined in a similar way.

Theorem2.1 Let the pair [x ,u] ~ 0 be a saddle point fo r Fu (x , u) . Thenx E Arg (1.4) and

(2.2)

Proof. First, let us show that x E Mi, = U M a . Assume the contrary; i.e.,aEn

that x tt M u or, equivalently,

(2.3)

Take the right-hand side of (2.1):

The value l' is finite. If, in opposite, l' = -00, then, taking in relation (2.4) allUa = 0, we get the nonsense -00 ~ O. Next let us take an arbitrary positivenumber v > O. By (2.3), for any a there exists a posit ive component 'Yj (a )

of the vector Fa (e). Denote by 8j (a ) the corresponding component of thevector Ua' Because U a ~ 0 is arbitrary, it is possible to guarantee the inequality8j (a ) . 'Yj (a ) > C > 0, Va. Setting all the other components of the vectors Ua atthe value zero, and taking sufficiently large t > 0, we can obtain the inequality(u~ , Fa (x)) > 'Y for all a , where u~ := tUa ' Due to the arbitrariness of'Y > 0,the last inequality contradicts (2.4). Thu s, ii: EMu.

ABOUT DISJUNCTIVE OP71MlZATJON 49

Next we prove that l' = inf (uo, Fo{:t)) = 0; i.c., (2.2) is valid. Indeed,oEn

since x EMu, one has l' ~ O. But if l' < 0, then, taking in (2.4) U o = 0, wehave 0 > l' 2: 0, and this is a contradiction.

Finally we show that x E Arg (1.4). Since l' = 0, we can rewrite theleft-hand side inequality in (2.1) as:

(2.5)

For an arbitrary x EMu, the last term in (2.5) will be non-positive and,therefore, f{x) ~ f{x) for all x EMu , and x E Arg (1.4). 0

The following theorem can be proved reasoning as above.

Theorem 2.2 Let the pair [x, u] 2: 0 be a saddle point for Fn{x, u) . Thenx E Arg{1.3),and

sup (uo, Fo{x)) = O.oEn

(2.6)

Remark 2.1 Analogs of Theorems 2.1 and 2.2 are valid also for the modified Lagrangians F~ (x, u) and F~ (x, u). To verify these facts one can followthe scheme of the proof of Theorem 2.1.

It is known that, for a standard MP problem, the existence of a saddle pointof its Lagrangian is connected with the Kuhn-Tucker Theorem, proving it under some appropriate conditions; in particular, under conditions of convexityand constraint qualifications of some kind. Establishing similar results for theproblem (1.4), it would be expedient to use the same conditions for each of thesub-problems

sup {j{x) IFo{x) ~ 0, x 2: O}. (2.7)

Instead of it, we will simply the situation by supposing that, for each a E 0,the function Fo{x, u) := f{x) - (uo, Fo{x)) has a saddle point.

But this condition is not sufficient. Since the index set 0 may be infinite(e.g. continuum), we need somewhat like the following condition: the set M u(or U Arg (2.7)0) must be bounded.

oEnWe shall apply the second variant: let us assume that a ball S E R" exists

such that

Va: S n Arg (2.7)0 i: 0. (2.8)

If f{x) is continuous and (2.8) holds, then i:= sup{j(x)lx EMu} <+00; i.e, the problem (1.4) is value-solvable. Nevertheless it may be not Argattainable; i.e., the existence of x E M u such that f{x) = i could fail. That

is why we shall define below an optimal vector of a problem (1.4) as a limitpoint x of a converging sequence {xo . E Arg (2.7}0.}. Such a limit point

J =-J _

may belong or not to M u but, in any case, x E M u := U Arg (2.7}0, whereoEf!

the bar over set denotes its topological closure. Thus we define as the optimalset of problem (1.4)

{x E M If(x} = f} (= Arg (1.4}).

Let us present the list of all the conditions we introduced above:

1) f(x) is continuous over M u;

2) Va there exists [xa, ua] ~ 0 which is

a saddle point for f(x} - (ua, Fa(x));

3} (2.8) holds.

(2 .9)

Theorem 23 Assume that all the conditions (2.9) hold. If {XO j } ~ ii;

where X O . E Arg (2.7}0 ·, and f(x) = f (= opt (1.4)), thenJ J

Fu (x, u) < f(x) ,V'x:::,:o

(2.10)

Proof. We have (uo, Fo(xo)) = 0, Va, and f(x) - (uo, Fo(x)) :-:; f(xo) :-:;f(x), Vx ~ O. Consequently, sup [f(x) - (uo, Fo(x))] :-:; f(x). But the left

oEf!part of this inequality is equal to f (x) - inf (uo, Fo(x }). Therefore (2.10) is

oEf!valid. 0

Remark 2.2 . Theorem 2.3 is analog to Theorem 47.2 in [8], known for afinite index set n = {I, ... , m}.

Theorem 2.4 Assume that the conditions 1) and 3) from (2.9) hold. aswell as condition 2) for the function f (x) - (ua, Fet (x)). If x and u are thesame as in Theorem 2.3, then pair [x, u] is a saddle point for F~ (x, u) .

Proof. Let us prove the right-hand side inequality in the relation

F~ (x, u) < F~ (x , u)V'x:::,:o

< F~ (x, u),V'u:::,:o

(2.11)

which characterizes a saddle point[x, u].Using (1.8)+ (i.c., using the definitionof F~ (x , u)), one can rewrite this inequality as

(2.12)

ABOUT DISJUNCTIVE OPTIMIZATION 51

(2.13)

It is clear that the right-hand side of (2.12) is equal to zero. But {xo .} -7 X,J

and CUo " F;;' (xo · )) = 0, Vexj , and, therefore, the left-hand side is equal toJ J J

zero too; i.e., the desired inequality is valid.Consider the left-hand side inequality in (2.11). Rewrite it in detail :

f(x) - inf (uo , F: (x)) ~ f(x) - inf (ito, F: (x)).oEn vx~o oEn

When proving (2.12), we already proved that inf (uo , F; (x)) = O. ThereforeoEn

(2.13) takes the form of relation (2.5) proved above, which is valid for thefunction F; (x) too (see remarks to Theorems 2.1 and 2.2). The proof iscomplete. 0

Remark 2.3 The parameter ex, which plays the role of an index, enumerating the components of the disjunctive or conjunctive inequalities (1.1) , (1.2)(or of the systems of such inequalities), may be of divers nature; e.g., it can bea vector. In particular, instead of a function Fo (x), one can consider a vectorfunction F(x, y) of two vectorial arguments x E R" and y E R'", where yplays the role of the index ex and, consequently, n may be a subset of R'" ; e.g.,it can be a compact set. Then inequalities (1.1) and (1.2) take the form

sup IF(x, y)lmax ~ 0,yEn

inf IF(x, y)lmax ~ O.yEn

(2.14)

These inequalities have a deep maintenance. For example, a general gameof two persons with a zero sum may be reformulated as the first of them.Concerning the second inequality, a whole array of optimal control problemscan be reduced to it, y being interpreted as a time parameter t. One can prove hisown variants of Theorems 2.1 - 2.4 for optimization problems with constraintsof type (2.14) under appropriate conditions; e.g., under conditions ofcontinuityof F(x, y) in z = [x, y], compactness of n, etc.

3 DUALITY FRAMEWORK

We shall construct a duality framework for disjunctive programming problems by means of a general scheme, namely, using Lagrangian function, in ourcase the disjunctive Lagrangian function (1.8) .

Consider the problems

Pu : sup inf Fu(x, u) (=: 1),x~o u~o

and

P,* . inf supFu(x, u) (=: 1*),u .u~o x~o

(3.1)

(3.2)


and their analogs in the linear case

Lu: sup inf Lu(x, u) ,x~o u~o

and

(3.3)

L~: inf supLu(x, u), (3.4 )u~O x~O

where Lu (x , u) = (c, x) - inf .(ua , Aax - ba), Ua ~ O.aEI1

Lemma 3.1 The problems (3.1) and (1.4). with u; = {x ~ 0 Iinf !Fa (x)lmax ~ O}. share the same optimal value.aEI1

Proof. Indeed, consider the internal sub-problem from (3.1) and calculate itsoptimal value:

{f(x) , if x EMu,

inf[j(x) - inf (uo, Fo(x))] = . du~O oEI1 -00, If x 'F M u .

This immediately implies

'Y = sup {f(x) Ix EMu},

which is what we wanted to show. 0

We shall call the problem P~, introduced in (3.2), dual problem associatedwith Pu. Similarly, L~ will be the dual of Lu. According to Lemma 3.1 theproblem Li, is equivalent to the problem (1.5), which is a linear disjunctiveproblem.

Lemma 3.2 Both the problem L~ , i.e. (3.4), and the problem

inf {sup (bo, uo) I A~uo ~ c, Ua ~ 0, a E O} (3.5)u~O oEI1

have the same optimal value.

Proof. Let us rewrite the problem L~ in another form. First rewrite its Lagrangian Lu(x, u) :

Lu (x , u) = (c, x) - inf (uo, Aox - ba) = sup [(c, x) - (uo, Aox -a EI1 a EI1

bo)] = sup [(ba, ua) + (c - Ar Ua , x)].o EI1

ABOUT DISJUNCTIVE OPTIMIZATION

Then, performing the internal operation sup in (3.4), we get:x>o

This impliesinf sup Zi, (x, u) = opt (3.5).u::,:o x::,:o

D

53

Thus, we get (3.5) the dual of problem (1.5). It is interesting to observe thatthe original problem L u , (i.e. (1.5)) can be written in equivalent form as

sup {sup (c, x a) IAa X a ~ c: X a ~ 0, a E n}. (3.6)x::':O aEn

It makes it possible to recover the symmetry in the instances Li, and L~ . Operations inf and sup in (3.5), as well as sup and sup in (3.6) are commutative.

u::':o oEn x::':0 aEnConsequently, the problem (3.6) can be reduced to the problem of determiningan exact upper bound, i.e. sup, for the set of optimal values of the problems

aEn

L o : max {(c, x) Ix E Na } , (3.7)

where N a := {x ~ 0 IAax ~ bal. Analogously, the problem (3.5) can bereduced to the problem of determining an exact upper bound for the optimalvalues of the problems

L~: min {{ba, ua) IU a E N~},

where N~:= {u a ~ 0 IA?: U O ~ c}. If we assume all the problems (3.7) tobe solvable, then, according to the duality theorem in linear programming, onehas opt La = opt L~, and therefore sup opt La = sup opt L~.

a En aEnAny of the problems Lo and L~ may be solvable or not. Unsolvable (im-

proper) problems may be classified [8]:

1) No = 0, N~ =I- 0 corresponds to improper problems of the lst kind;

2) No =I- 0, N~ = 0 - the 2nd kind;

3) Na = 0, N~ = 0 - the 3rd kind.

If the feasible set of a certain problem is empty, then its optimal value usually is defined as -00 for sup-problems and +00 for inf-problems. In our case,according to the classification above, we have:


1) opt Lo. = -00, opt L~ = -00;

2) opt Lo. = +00, opt L~ = +00;

3) opt Lo. = -00, opt L~ = +00.

Theorem 3.1 Let the problem (1.5) ii.e. (3.6)) be value-solvable. andassume that there is no improper problem of3rd kind among the problems Lo..Then the problem (3.5) is also value-solvable. and opt (3.6) = opt (3.5).

Proof. Since the original problem has finite optimal value, there are no improper problems of 2nd kind among {Lo.}. Concerning the improper problemsof 3rd kind, they are forbidden by assumption. Thus, the real situation is asfollows: each of the problems Lo. is either solvable (with L~), or unsolvable,and then opt Lo. = opt L~ = -00. Consequently, the sets of optimal valuesfor {Lo.}o.En and {L~}o.En are the same, as well as their exact upper bounds.The proof is complete. 0

Theorem 3.2 Assume that some analog ofthe condition (2.8) holds. namely:there exists a ball 8 eRn such that

Va: 80. := 8 n Arg i.; # 0.

Then the problem (1.5) is Arg-solvable in general. the problem (3.5) is valuesolvable. and opt (1.5) = opt (3.5).

Proof. Since (2.9) holds, the property of value-solvability of (1.5) is immediate. Indeed, if, = opt (1.5), then one can take a convergent sequence{Xo.k E 80.k} -+ X, and {(c, Xo.k)} -+ (c, x) = , . But that is what wemean by saying that the problem (1.5) is arg-solvablc in general. The valuesolvability of the dual problem to (3.5) is then evident. 0

The problems (1.5) and (3.5) may be rewritten in a compact and symmetricway, when the set n is well ordered; i.e.,

n = {aI, a2 , .. . , a w , aW+I , "'}'

In fact, let us introduce the transfinite matrix 2{ and the vectors X, u, band c:

2{=

and the functions III (x) = sup (co.k' Xo.k), and 'IT* (u) = sup (bo.k , Uo.k). Thenk k

the problems (3.2) and (3.4) take the form

sup {Ill(x) I 2{ x ~ b, x 2: O},

ABOUT DlSJUNCnVE OPTIMIZATION 55

inf {\lJ*(u) I 2(T u 2: (5, u 2: O}.

Note that the first of these problems is not convex but the second one certainlybelongs to this category.

The scheme of the dual construction for the general problem of disjunctiveprogramming (1.4) may be the same as for the problem (1.5) . The dual objectstake the form (3.2) and (3.4), i.e. P~ and L~ (in our notation). The required dualrelation opt Pu = opt P~ is usually connected (and often coincides) with theexistence of saddle points of the function Fu (x , u) . If there exists a saddle pointfor F u (x , u) , then this dual relation will hold. Of course, one can apply anyconditions guaranteeing the existence of a saddle point for Fu (x , u) (as it wasmade in Theorem 2.4 for the modified function F~ (x, u). These conditionssurely allow to formulate duality theorems in the general case too. We omitall the detail s here, restricting ourselves to the linear case by methodologicalconsiderations.

4 AN EXACT PENALTY FUNCTION METHOD

Let us consider the question about the equivalence of problems (1.4) and(1.10), under an appropriate value of the parameter R.

Theorem 4.1 Assume that the Lagrangian Fu( x , u), associated with theproblem (1.4), has a saddle point [x , ul, U = {u a 2: O} . Ifu; 2: ua!or alla E 0, then

opt (1.4) = opt (1.10). (4.1)

Proof. According to Theorem 2.1 the following relations hold: x E Arg (1.4)and t : = inf (ua , Fa (x)) = O. Using the definition of saddle point and the

aE nequality t = 0, one has

vx 2: O. (4.2)

HenceF~ (x , R) = j(x) - inf (R a, F: (x)) :s j(x) + inf (ua , Fa (x)) -

aEn aEninf (Ra, F: (x)) :s j(x) + inf (ua , F: (x)) - inf (R a, F:(x)) :s j(x) .aEn aEn a En

Since this inequality holds for any x 2: 0, we get

sup F~ (x , R) :s j(x)(= opt (1.4)).x~o

Since the inverse inequality is trivial, the proof is complete. 0

More strong theorems, connecting the problem s (1.4) and ( 1.10), can beformulated under the following assumptions on the set 0 and the functions

Fo (x) =: cp(z), z = [a, x] :

1) Fo : R nF

o) R m

, }

2) n is compact subset of R k ,

3) cp(z) is continuous with respect to a.

(4.3)

Since cp(a, x) E Rm for any a, the dimension of the Lagrange vectors uomentioned above is m. Let 8 = [0, . .. , o]T E R'", 0 > 0, and choose thepenalty vectors Ro so that Ro ~ ito + 8.

Theorem 4.2 Assume that all the assumptions ofthe Theorems 4.1, (4.3)and the condition Ro ~ ito + 8 simultaneously hold. Then

Arg (1.4) = Arg (1.10). (4.4)

Proof. From the assumptions it is immediate to prove the equality (4.1). Remind the inequality obtained in Theorem 4.1:

F~(x, R) < JUt) + inf (ito , F;(x)) - inf (Ro , F;(x)) ~ JUt),'v'x~o oEn

(4.5)

where x E Arg (1.4). From this relation it follows that

opt (1.10) = sup F~(x, R) ~ J(x) = opt (1.4).

For x = x one has F~(x, R) = J(x), so that x E Arg (1.10) . Consequently,the inclusion Arg (1.4) C Arg (1.10) is valid.

Let us prove the inverse inclusion. Take any x E Arg (1.10) and substituteit in (4.5):

F~(x, R) ~ J(x) + inf (ito , F; (x)) - inf (Ro , F;(x)) ~ opt (1.4).oEn oEn

(4.6)

Since F~(x, R) = opt (1.4), one has

inf (ito, F;(x)) = inf (Ro , F;(x)) .oEn oEn

Taking an appropriate sequence {ak} -+ Q , we can rewrite (4.7) as

(4 .7)

{E:k > O} -+ O. (4.8)

Since

ABOUT DISJUNCfIVE OPTIMIZATION 57

2 {ua k ' r;(x)) + 81F:k {x)lmax,

it follows, from (4.8), that IFIk

(x) lmax :c:; C8k -+ O. Therefore 1Ft(x) Imax = 0

and Fo{x) :c:; OJ i.e. ,

x E u; = {x 20 I Fa{x) :c:; O} c u; = U u.;a En

Thus, we prove that x from Arg (1.10) is feasible for the problem (1.4) and provides an optimal valuefor f{x); i.e., x E Arg (1.4). Consequently, Arg (1.lO) CArg (1.4), and (4.4) is valid. 0

The analogs of Theorems 4.1 and 4.2 for the conjunctive problem (1.3) arevalid too. Let us write the problem (1.3) in detail:

sup {f{x) I sup lFa{x)lmax :c:; 0, x 2 O},

and consider the associated problem (1.9); i.e.,

sup {f{x) - sup (R a, F:{x))}.x~o aEn

(4.9)

(4.10)

Theorem 43 Let the function Fn (x , u) , i.e. (1.7), have a saddle point[x, ul, U = {ua 2 O}. and suppose that the conditions (4.3) hold. Then:i) If Va E n, u; 2 Ua , then

opt (4.9) = opt (4.1O).

ii) If Va E n we have R a 2 ua + 0 (o taken from Theorem 4.2), then

Arg (4.9) = Arg (4.10).

The proof of this statement is very similar to the proof of Theorems 4.1 and4.2. It is omitted since this article is focussed on the disjunctive problem.

Acknowledgment

This research was supported by Russian Found of Fundamental Researches(project codes 00-15-96041,99-01-00136).

References

[1] E. Balas. A note on duality in disjunctive programming, Journal oj Optimization Theory and Applications, 21:523- 528, 1977.

[2] E. Balas. Disjunctive programming, Annals ojDiscrete Mathematics, 5:351, 1979.


[3] E. Balas. Disjunctive programming: properties of the convex hull of feasiblepoints, MSSR , N 348, Carnegie Mellon University, Pittsburgh, Pennsylvania,1974.

[4] E. Balas, J. M. Tama, andJ. Tind. Sequential convexification in reverse convex and disjunctive programming, Mathematical Programming, 44A:337350, 1989.

[5] J. M. Borwein. A strong duality theorem for the minimum of a family ofconvex programs, Journal ofOptimization Theory andApplications, 31:453472, 1980.

[6] I. I. Eremin. About penalty method in convex programming, Kibernetika,4:63-67, 1967.

[7] I. I. Eremin. About some problems of disjunctive programming, YugoslavJournal ofOperations Research, 8:25-43, 1998.

[8] I. I. Eremin. Theory ofLinear Optimization, Eketerinburg publisher, Russia,1999.

[9] S. Helbig . Optimality criteria in disjunctive optimization and some applications, Methods ofOperations Research, 62:67-78, 1990.

[10] S. Helbig. Stability in disjunctive programming I: continuity ofthe feasibleset, Optimization, 21:855- 869, 1990.

[11] S. Helbig. Duality in disjunctive programming via vector optimization,Mathematical Programming, 65A:21-41, 1994.

[12] S. Helbig. Stability in disjunctive programming II: continuity of the feasible and optimal set, Optimization, 31:63- 93, 1994.

[13] H.Th. Jongen, J.-J. Riickmann, and O. Stein. Disjunctive optimization:critical point theory, Journal of Optimization Theory and Applications,93:321-336, 1997.

[14] V. Patcar, and I. M. Stancu-Minasian, Duality in disjunctive linear fractional programming, European Journal of Operations Research, 21: 101105, 1985.

[15] RD. Sherali, and C.M. Shetty. Optimization with Disjunctive Constraints,Springer-Verlag, 1980.

[16] F. Zafiridou. Parametrische disjunctive lineare optimierung, Diplom-Thesis, Frankfurt/Main, 1988.

[17] W.I. Zangwill. Non-linear programming via penalty functions, Management Science, 13A:344-358, 1967.

Chapter 3

ON REGULARITY AND OPTIMALITY INNONLINEAR SEMI-INFINITE PROGRAMMING

Abdelhak Hassouni! and Werner OetUP

1Universite Mohammed V. Departement de Mathematiques et lnformatique,

B.P. 1014, Rabat, Morocco

2Universitiit Mannheim, Fakultiitfiir Mathematik und lnformatik; D-6813 1 Mannheim , Germany

[email protected]

Abstract This paper deals with semi -infinite convex optimization problems. In particular,a regularity condition introdu ced recently by Guerra and Jimene z for the linear case is extended to the convex case, employing a suitable general ization ofHaar's condition. The convergence of an exchange process under such Haar-likeconditions is investigated.

1 INTRODUCTIONConsider a semi-infinite linear optimization problem,

min{ (ao, x) Ix EM},

where M:= {x E jRn I(at, x) + bt sO Vt E T} . Given x* EM, a standardoptimality condition for x* is furnished by the Karush-Kuhn-Tucker condition,namely

oE ao + cone convjc, It E T*},

where T* := {t E TI (at,x*) + bt = O}. This condition is easily seen tobe sufficient for optimality of x*, but to make it a necessary condition, someregularity assumption is usually needed. Recently in [8] for a special case ofthe above problem, derived from Chebyshev approximation, a simple regularitycondition was introduced. This condition requires essentially that M is not asingleton. To this has to be added in the general linear case a certain Haar-typecondition. Below we generalize this approach to convex semi-infinite optimization problems, introducing a suitable counterpart of Haar's condition. We

59

M.A . Goberna and M.A. Lopez (eds.), Semi-Infinite Programming, 59-74.© 2001 Kluwer Academic Publishers.


compare the resulting regularity condition with a standard regularity conditiondue to Slater. We also use our generalized Haar-type condition to establishthe validity of the exchange process for solving convex semi-infinite optimization problems. Finally we apply our approach to reprove a result about outernormals published recently in [3].

2 THE LINEAR CASE

Our starting point is a recent result by Guerra and Jimenez [8, Theorem 2.1]which can be viewed as giving a regularity condition for feasible sets definedthrough linear Chebyshev approximation. We extend their result to generalconvex semi-infinite programming problems. We discuss first the linear case.

Let T be a compact space . Let the mappings T 3 t H- at E lRn andT 3 t H- bt E JR be continuous. For all t E T let

be affine functions in a: Let ITI denote the cardinality of a finite subset T ~ T.We let

M := {x E lRn 11t{x) :::; 0 'Vt E T}.

Then we can prove

Theorem 2.1 Let z" E M , and let T* := {t E T I It{x*) = O} =1= 0.Assume that:

(H) for every subset T ~ T* with ITI :::; n the vectors at, t E T, are linearlyindependent.

Then the following statements are equivalent:

(i) M = {x*};

(ii) there exists T ~ T* with ITI = n + 1 such that the system

has no solution;

(iii) 0 E conv{at I t E T*}.

The result in [8] refers to the case when M has the form

M := {x E lRn I h2{t ) :::; P{x, t) :::; hI (t) 'Vt E T} ,

where T := [a, b] ~ lRI , andP{x, t) := L:~:~ Xktk is a polynomial ofdegree atmost n - 1 in i , with coefficients {xo, Xl , ... ,xn-d =: X E JR1I'. It is assumed

ON REGULARITY AND OPTIMALITY IN NONLINEAR SIP 61

that h2(t) < h 1(t) for all t E T , and therefore of the two constraints in Mcorresponding to a given t E T , at most one can be active at each x E M. Forthis reason it is easily seen that the Haar condition (H) is automatically satisfiedin this case, hence does not show up explicitly in the result. Condition (ii) ofTheorem 2.1 takes in [8] the form of an alternance condition, which howeverin this setting is equivalent with the present, Kolmogorov-type condition (ii) .

Formally speaking, there is no need to consider the linear case separately,since with suitable modifications everything carries over to the convex case.

3 THE CONVEX CASE

Now we extend Theorem 2.1 to convex, semi-infinite systems. Let C ~ lRn

be closed, convex, and nonempty. Let T be a compact space, having at leastn + 1 elements. For all t E T let it (.) : C -* lR be convex functions such thatt H- it (x) is upper semicontinuous on T for all x E C, and x H- it (x) is lowersemicontinuous on C for all t E T (lower semicontinuity and the closednessof C are not needed if T is finite). By 171 we denote the cardinality of a finite,nonempty subset 7 ~ T . Let

M := {x E C I ft(x) :::; 0 'It E T}.

Then we have

Theorem 3.1 Let x* E M, and let T* := {t E T I ft(x*) = O} I- 0,IT* I~ n + 1. Assume that:

(HI) for every 7 ~ T* with 171 :::; n the system

~ E C, !t(O < 0 'It E 7

has a solution ~;

(H2) for every 7 ~ T* with 171 = n + 1 the system

~ E C, !t(~) = 0 'It E 7

has no solution ~ I- x*.

Then the following statements are equivalent:

(i) M = {x*} ;

(ii) there exists 7 ~ T* with 171= n + 1 such that the system

~ E C, !t(~) < 0 'It E 7

has no solution;

(iii) there exist r ~ T* finite and real numbers Ut ~ 0 (t E r) with 2:tET Ut =1 and such that

I:Ut!t(O ~ 0 Ve E C.tET

It is easily seen that the Haar-like conditions (HI), (H2) are implied by (H),if C = IRn and It (.) = it(.) is affine as in Theorem 2.1. Therefore Theorem2.1 clearly is a special case of Theorem 3.1.

For the proofof Theorem 3. 1we need two standard results of convex analysis,which we quote as lemmata.

Lemma 3.1 (Helly) Let {CdiEK with IKI ~ n + 1 be a finite family ofconvex sets in IRn such that niEK Ci = 0. Then there exists J C K withIJI = n + 1 such that niEJ O, = 0.

For a self-contained proof see [4, p.33].

Lemma 3.2 (Fan, Glicksberg, Hoffman [5]) Let C be a convex set, and letIi, ... ,1m : C -+ IR be convex junctions such that

. max Ii (x) ~ 0 Vx E C.t=l ,... ,m

Then there exist real numbers Ul ~ 0, . . . , Um ~ 0, with 2:~1 Ui = 1, andsuch that

m

I:Udi(X) ~ 0 Vx E C.i=l

Two slightly different proofs of this are given in [5]. The geometry of thisresult is easily visualized: Let F := (Ii, ... ,1m) : C -+ IRm , and D :=F( C) +1R+. From the assumptions it follows that D is convex, and the origin isnot an interior point of D. The interior of D is nonempty. From the separationtheorem for convex sets there exists U E IRm , U i- 0, such that (u, d) ~ 0VdE D. This implies at once (u, F(x)) ~ 0 Vx E C, and (u, z) ~ 0 Vz ~ 0,thus U ~ O. The claim follows immediately.

For convenience let us single out one step in the proof of Theorem 3.1 alsoas a Lemma.

Lemma 3.3 Same assumptions as for Theorem 3.1, but without (HI) and(H2). If the system

x E C, It(x) < 0 Vt E T

has no solution, then the system

(3.1)

(3.2)

ON REGULARITY AND OPTIMALITY IN NONLINeAR SIP 63

has no solution either:

Proof. Let ~ be a solution of (3.2) . Since T* =I 0 we have ~ =I x*. T* iscompact, since we can write T* = {t E T I it (x*) 2: O} and t t-7 ft (x*) isupper semicontinuous. Then there cxists-y >°such that it (0 ~ -, Vt E T*.So the set TI := {t E T I ft(~) 2: -,/2} is compact and disjoint from T*.Since ft(x*) < °for all t in TI , there exists k i > °such that ft(x*) ~ -k i

for all t E TI . Also there exists k2 < 00 such that ft(~) ~ k2 for all t E T.For oX E ]0,1] set x>. := (1 - oX)x* + oX~. Then x>. E C , and for all oX E ]0,1]we obtain:

• ift E T \ TI , then ft(x>.) ~ (1 - oX)ft(x*) + oXft(~)

~ oXft(~) < -oX,/2 < 0;

• ift E T I , then ft(x>.) ~ (1 - oX)ft(x*) + oXft(e}~ (1- oX)(-kd + oXk2= -k, + oX(k l + k2 ) .

So there exists oX* E]O,l] such that, for all t E T, ft(x>.*) < 0, and x>.* is asolution of (3.1). 0

Proof of Theorem 3.1. We show (i) ===} (ii) ===} (iii) ===} (i).(i) ===} (ii) Assume that (i) holds. Then it is easily proved, since T* =I 0,that

system (3.1) has no solution. From Lemma 3.3, system (3.2) has no solution.Then for every fixed £ > °the family of sets

C(t) := {~ Eel II~ - x*1I ~ 1, ft(~) ~ -£} (t ET*)

has empty intersection. Since these sets are closed and contained in a compactball, there exist'> a finite subfamily having empty intersection. Since these setsare convex and contained in ~n, it follows from Lemma 3.1 that there existk among them, k ~ n + 1, having empty intersection. We may assume thatk = n + 1. So there exist td£)' . . . , tn+l(£) E T* such that the system

~ E C, II~ - x*1I ~ 1, it;(e)(O ~ -£ (i = 1, . .. , n + 1)

has no solution. Since T* is compact there exists a net E .J.. °such thatti(£) -t fi E T*, i = 1, .. . , n + 1 (some tt may be equal) . Then fromupper semicontinuity oft t-7 ft(O, the system

II~ - x*1I ~ 1, ftd~) <° (i = 1, .. . , n + 1)•has no solution. So we have found T ~ T* with ITI ~ n + 1 such that thesystem

has no solution. From (HI), this is only possible if Irl = n + 1. Therefore (ii)holds.

(ii) ==} (iii) Assume that (ii) holds with r ~ T* finite. Then

From Lemma 3.2, it follows that (iii) holds with the same r .(iii) ==} (i) Assume that (iii) holds . Then

I: Utft(~) ~ 0 'V ~ E C.tET

(3.3)

Without loss of generality we may assume that Ut > 0 'V t E r . If Irl < n + 1,then from (HI) there exists ~ E C with ft(~) < 0 'Vt E r ; so (3.3) would onlybe possible with Ut = 0 'Vt E r, a contradiction. Hence Irl ~ n + 1. Letx E M be arbitrary. Then ft( x) :S 0 'Vt E r. Therefore from (3.3) and Ut > 0,'V t E r, it follows that ft (x) = 0, 'V t E r . Then from (H2) it follows thatx = x*. Hence M = {z"}, and (i) holds. 0

If statement (iii) in Theorem 3.1 is true, then it is clear that system (3.1) doesnot have a solution . Conversely, we have shown in the proof of Theorem 3.1that inconsistency of (3.1) implies the validity of (iii). Assumptions (HI) and(H2) were not needed for this part of the proof. Therefore we obtain

Corollary 3.1.1 Same assumptions as for Theorem 3.1, but without (HI)and (H2). Then the following statements are equivalent:

(a) the system

x E C, ft(x) < 0 'Vt E T

has no solution;

(f3) there exist r ~ T* finite and real numbers Ut ~ 0 (t E r) with L:tET Ut =1, and such that

I:Utft(O ~ 0 'V ~ E C.tET

We consider now the semi-infinite convex minimization problem

(P) min{Jo(x) I x E C, ft(x) :S 0 'Vt E T} .

We let the feasible set

M := {x Eel ft(x) :S 0 'Vt E T}

obey the same hypotheses as introduced before in Theorem 3.1. The objectivefunction fo : C -+ lR is convex and lower semicontinuous (lower semicontinuity is not needed if T is finite). If x* E M is a solution of problem (P), thenthe non-validity of (iii) in Theorem 3.1 is a regularity assumption which ensures that x* satisfies the Karush-Kuhn-Tucker condition (KKT) below. Moreprecisely we have the following result, where C, T, ftO and M are as forTheorem 3.1 (but T* may be empty).

Theorem 3.2 Let x* EM and T" := {t E T I ft(x*) = O}. Assume thatcondition (iii) of Theorem 3.1 fails to be true. Then x* is a solution of (P) if,and only if,

(KKT) there exists a finite subset T ~ T* (T possibly empty) and real numbersAt ~ 0 (t E T) such that

fo(x*) :s fo(~) +L Atft(~) V ~ E C.tET

Proof. The "if" part is trivial and needs no regularity assumption. To provethe "only if" part, assume that x* solves (P). Then the system

~ E C, fo(~) - fo(x*) < 0, ft(~) < 0 "It E T,

has no solution f , So, from Corollary 3.1.1 we obtain real numbers Uo ~ 0,Ut ~ 0 (t E T) , where T ~ T* is finite, such that Uo + EtET Ut = 1 and

o:s uo(Jo(O - fo(x*)) +L Utft(~) V ~ E C.tET

If Uo = 0, then (iii) of Theorem 3.1 would be true, contradicting the hypothesis .So Uo > O. Dividing by Uo and setting At := utluo (t E T) we obtain (KKT). 0

From Theorem 3.1 we know that (HI), (H2) and M =1= {x*} together ensure the non-validity of (iii), and thus ensure that a solution x* of (P) satisfies(KKT). This would be the extension of the regularity condition introduced byGuerra and Jimenez. But from Corollary 3.1.1 we know that the non-validityof (iii) is equivalent with the existence of x E C such that ft(x) < 0 "It E T,and this is the classical Slater condition ensuring that a solution x* of (P) satisfies (KKT). The Slater condition therefore is less demanding than the abovecondition involving (HI), (H2). The latter therefore is only competitive if theHaar-like condition is easy to check; otherwise the Slater-condition is preferable. This results can be compared with Theorem 3.1 of Ben-Tal, Rosingerand Ben-Israel [1], by means of Helly-type theorem, the authors establish theequivalence of semi-infinite convex programs and certain finite subprograms,in the same context and with the Slater-condition.

4 CONVEX APPROXIMANTS

Let us now consider convex approximations of nonconvex functions. LetC ~ lRn be closed, convex and nonempty. Let f : C -+ lRbe a given function,x* E C, and Z := C - x*. We say that a function sp : Z -+ lR is a convexprederivative of f at x* if

I) cP is convex;

2) cp(O) = 0; and

3) lim sup )..-1 (f(x* + AZ) - f(x*)) ~ cp(z) for all Z E Z.>..j..O

Examples: (a) If f is convex, then cp(z) := f(x* + z) - f(x*) is a convexprederivative of fat z", since f(x* + AZ) - f(x*) ~ A(f(X* + z) - f(x*)) =)..cp(z) for 0 ~ ).. ~ 1. So convex functions can be used as their own prederivatives, without need to approximate them by "nicer" functions (like linear onesor sublinear ones).

(b) If f is locally Lipschitz, and if we set

cp(z) := fO(x*, z),

where fO(x*, z) := lim sup A-l(f(x+ AZ)- f(x)) is the Clarke's generalized>'.j..0

x-+x·directional derivative, then sp is a convex prederivative of f at x*, since fO (x*, .)is convex, and

lim sup X'<'{j'(z" + AZ) - f(x*))>'.j..0

~ lim sup A-1 (f(x + AZ) - f(x)) = fO(x*, z) = cp(z) .>..j.0

x-+x·

Now let

M := {x Eel ft(x) ~ ° Vt E T},

where ft : C -+ lR for all t E T, where T is compact, and t H ft(x) isupper semicontinuous on T for all x E C . Let x* E M, and let T* := {t ETift (x*) = O}. Then, T* is compact. Assume that the functions CPt : Z -+ lRare convex prederivatives of ft at x* for all t E T. Assume furthermore thatt H CPt (z) is upper semicontinuous on T for all Z E Z, and that the lim supoccurring in the definition of the prederivative is uniform in t E T for everyfixed Z E Z . Thus for every Z E Z and for every Q > 0, there exists 8 E ]0,1]such that for all A E ]0,8] and all t E T

A-1(ft(X* + AZ) - ft(x*)) ~ CPt(z) + Q. (4.1)

Then we have the following extension of Lemma 3.3:

Lemma 4.1 Assume that T* =I- 0. x* E M and U - x* is absorbing. Ifthe system

has no solution, then the system

Z E Z, CPt( z) < 0 \:It E T*

has no solution either.

(4.2)

(4.3)

Proof. Let Z be a solution of (4.3). Then z =I- O. There exists, > 0 such thatCPt(z) ::; -" \:It E T*. The set

T I := {t E T ICPt(z) 2: -,/2}

is compact and disjoint from T*. Therefore ft(x*) < 0 for all t E T I . So thereexists k l > 0 such that ft(x*) ::; -kl for all t E T I • Also there exists k2 < 00

such that CPt(z) ::; k2 for all t E T . For 0 < >. ::; 1 set (x := x* + Xz. Chooseo< a < ,/2. Then, there exists 0 E ]0, 1] such that, from (4.1),

ft((x) ::; ft(x*) + >'(cpt(z) + a) ,

for all x E ]0, 0] . Then for all >. E ]0, 0] we obtain:

• ift E T \ T I , then ft(6J ::; >.(-,/2 + a) < 0;

• ift E T I , then ft((x) ::; -k l + >.(k2 + a).

So, for all >. E ]0,0] sufficiently small , (x E C n U and

ft((x) < 0 \:It E T.

Thus (x is a solution of (4.2). 0

Now we return to the problem

(P) min{/o(x) I x E C, ft(x) ::; 0 \:It E T}.

The sets C and T, the functions (t, x) I-t ft(x), the functions (t , x) I-t CPt(x),and the set

M := {x Eel ft(x) ::; 0 \:It E T}

are as stipulated before Lemma 4.1. CPo : Z -+ lR is a convex prederivative of10 at x" , In addition, we assume that CPo, CPt are lower semicontinuous on Z.Then we have:

Theorem 4.1 Let x* E M be a local solution of (P). Let T* := {t E T Ift(x*) = O}, T* =1= 0. Then there exist 7 ~ T* finite (7 possibly empty) andreal numbers Uo 2: 0, Ut 2: 0 (t E 7), with Uo + 2:tET Ut = 1, and such that

o< uO'Po(z) + L Ut'Pt(z) \:I Z E Z.tET

Proof. Let x* be a local solution of (P). Then, with a suitable neighborhoodU [z"), the system

~ E en U(x*), 10(0 < 10(x*), ft(O < 0 \:It E T

has no solutionF. Subtracting a constant from 10, which does not affect 'PO, wemay assume that 10(x*) = O. Then from Lemma 4.1 the system

Z E Z, 'Po(z) < 0, 'Pt(z) < 0 \:It E T*

has no solution . From Corollary 3.1.1 (where we replace T* by T* U{O} =1= 0)there exist 7 ~ T* finite and real numbers Uo 2: 0, Ut 2: 0 (t E 7) such thatUo + 2:tET Ut = 1, and

UO'Po(Z) +L Ut'Pt(z) 2: 0 \:I Z E Z,tET

the claimed result. 0

Theorem 4.1 was motivated by a result of Giannessi [7, Theorem 1], andthen we can obtain the KKT conditions with the subdiferentia1 induced by theprederivative.

5 THE EXCHANGE METHOD FOR SEMI-INFINITECONVEX MINIMIZATION

It may be worthwile to point out that similar conditions as (H 1), (H2) are theessential prerequisite to permit a formal extension of the so-called exchangealgorithm from linear to convex semi-infinite minimization problems. Thisalgorithm was first described by Remez for continuous linear Chebyshev approximation. It is an iteration scheme of the type "outer approximation", wherethe subproblems have the same size in all iterations [2, pp. 247-253] . Detailsof the extension are as follows: We consider the problem

(P) min{fo(x) Ix E C, 1t(x) ~ 0 \:It E T}.

Here C ~ jRn is a closed convex set, T is sequentially compact and the functions10, It : C -T jR are convex for all t E T. The function (t, x) t-+ ft(x) iscontinuous on T x C and 10 is also continuous. Moreover we assume:

(Cl) for every T ~ T with ITI = n the system

!t(e) < 0 \It E T

has a solution ~ E C;

(C2) for every T ~ T with ITI = n the system

has at most one solution ~ E C;

(C3) for every T ~ T with ITI = n - 1 and for every x E C , the system

fo(~) < fo(x) , !t(e) < 0 \It E T

has a solution ~ E C.

A subset T ~ T with ITI = n is called a reference. It is sometimes convenientto consider a reference T as an ordered n-tuplet. thus T E T",

Assume there exists a reference TO ~ T such that the subproblem

min{fo(x) Ix E C, !t(x) ~ 0 \It E TO}

has a solution xO , say. Then all inequality constraints in (po) are active (i.e.,satisfied as equalities) in xO . Indeed. from convexity a constraint which is notactive at xO could be omitted without affecting the optimality of xO. But forless than n constraints the subproblem cannot have a solution, due to (C3). Thisproves the claim . From (C2) it follows then that xO is the only solution of (pO),and xO is uniquely determined as the solution of the system

If xO is feasible for the original problem (P), then it is optimal for (P), andwe terminate. Otherwise we can select an index sO E T such that fso(xO) > O.Then we consider the problem

If (pO) has no feasible points, then (P) has no feasible points, and we ter

minate. Otherwise (po) has an optimal solution . Indeed, since the solution of(pO) is unique, it follows from convexity ([6]) that the level sets of the objectivefunction fo, if restricted to the feasible region of (po) , ar..-:.bounded. Hence the

level sets of fo , if restricted to the feasible domain of (po) , are also bounded,

hence compact. Therefore (pO) has an optimal solution x l, say. Again from

(C3) it follows that out of the n + 1 inequality constraints in (pO) at least n

must be active in Xl. But not all n + 1 constraints in (pO) can be active in Xl,since otherwise we would have ft(x l) = 0 Vt E TO, hence xl = xO, which isimpossible since fso(xO) > 0, fso(x l) ~ O. So there exists exactly one indexrO E TO such that fro(xl ) < O. We may drop this inactive constraint withoutaffecting the optimality of xl. Altogether then, xl is the (unique) solution ofthe subproblem

and fo(x l) > fo(xO). Then the same reasoning as for TO and (pO) appliesto TI and (pI). Continuing in this way, and assuming that termination neveroccurs, we obtain sequences {XV} ~ C, {TV} ~ t», {sV} ~ T such that XV isa solution of

min{fo(x) Ix E C, ft(x) ~ 0 Vt E TV},

and ft(x v+I ) ~ 0 V t E TV , fo(x v +l) > fo(xV), fsv (XV+I ) ~ 0, fsv (XV) >O. Assume that the sequence {XV} remains bounded, thus having limit points.Then, if x* is a limit point of {XV}, there exists an infinite subsequence {v} ~ Nsuch that

Using convexity and condition (C 1) we can conclude that x* is a solution of

(P*) min {fo(x) Ix E C, ft(x) ~ 0 Vt E T*}.

Furthermore we obtain

ft(x**) ~ 0 Vt E T*, fo(x**) = fo(x*), fs·(x**) < O.

Thus x** is also a solution of (P*), hence x** = x*. The index s" E T upto now had only to fulfill fsv(xV) > O. We sharpen this to the requirement that

Then in the limit we obtain that

n-max ft(x*) ~ fs· (x*) = fs· (x**) ~ O.tET

So x* is feasible for (P), hence optimal for (P). Every solution of (P) mustalso be a solution of (P*) , hence equal to x*. Therefore x* is the only limitpoint of the sequence {XV}. Thus, collecting all the assumptions made so far,

ON REGULARITY AND OP11MALITY IN NONLINEAR SIP 71

we obtain the net result: XV converges to x*, the unique solution of (P), andfo(x V

) t fo(x*). D

If T is finite, then the method will terminate after finitely many steps, sincethe objective value fo(x V

) is strictly increasing at every step, thus none of thefinitely many references can occur twice . In this case (C 1) is not needed, andthe continuity requirements can be relaxed.

If (P) is an ordinary linear programming problem, then the method coincideswith the dual simplex algorithm of G. B. Dantzig.

6 NORMAL CONES AND COMPLEMENTARY SETS

Recently, Clarke, Ledyaev and Stem [3] have established a relationship between proximal normal cones to a closed set in jRn and those to the closure of itscomplement [3, Theorem 3.1]. The proof given in [3] requires a certain amountof nonsmooth analysis. Here we want to propose an alternative proof, whichrelies on our Corollary 3.1.1, hence employs, besides some proximal reasoning,only standard results from convex analysis.

We say that a cone K ~ jRn is pointed iff there exists "( E jRn such that(k ,"() < 0 for all k E K, k # O. All other notations and definitions are as in[3]. In particular,

• 8 is a proper, closed, and nonempty subset of jRn;

• 8 is the closure of the complement of 8 ;

• ds(u) := minxEs IIx - ull is the eucl idean distance of u to 8;

• projs(u):= {x E 8 I IIx - u] = ds(u)} is the set of nearest points tou in 8 (it is compact and nonempty).

If u rj. 8 and x E projs(u), then the vector u - x is an outer normal to 8 atx. The set of all nonnegative multiples of such normals to 8 at x is denoted

Nf(x),

the proximal normal cone to 8 at x. If x rj. 8, or if no outer normals at x exist,then by definition Nf(x) := {O}. ~

These definitions carryover to the closed set S. Furthermore, for all r > 0,

The result under consideration is the following [3, p. 203].

Theorem 6.1 Suppose that x E cl int (8). Assume that Nf (z) is pointedfor all r > 0 sufficiently small and all z sufficiently close to x . Then for every

e > 0,

N;(x) ~ cl conv { U -Nf(Y)}'Ily- xll<e

(6.1)

The proof is based on the following auxiliary result.

Lemma 6.1 Given r > 0 such that S; i= 0and w E S, let z E projsr (w).Then one has

oE cony {z -w, U (y - Z)}.YEprojs(z)

IfNf (z) is pointed, then one has

w - z E cony cone { U (y - Z)} .yEprojs(z)

Proof. I) We establish first that the system

(z - w,() < 0,

(y - z , () < 0 for all y E proj§(z),

(6.2)

(6.3)

(6.4)

(6.5)

has no solution ( E ~n.

Assume , for contradiction, that there exists ( with 11(\1 = 1 such that (6.4)and (6.5) are satisfied. Then from (6.4) follows obviously that

IIw - (z + tOil < IIw - zll for all t > 0 sufficiently small. (6.6)

We shall see that (6.5) implies

z + t( E S; for all t > 0 sufficiently small . (6.7)

(6.6) and (6.7) together contradict the hypothesis that z E projsr(w), and thisproves the inconsistency of (6.4), (6.5).

Assume (6.7) is not true. Then there exist sequences ti -!- 0, Yi E S such that

(6.8)

Since the Yi remain bounded we can take convergent subsequences and obtainin the limit

fi E S, liz - fill ::; r ,

ON REGULARITY AND OP71MALITY IN NONLINEAR SIP 73

hence y E projs(z). Then from (6.5) follows (y - z, () < 0, hence for all isufficiently large

(Yi - z, () < 0.

This implies that

liz + t( - Yill > liz - Yill

for all t > 0. Since liz - Yill 2: r ; in particular

and this contradicts (6.8). Thus (6.5) implies (6.7) .

2) From the inconsistency of (6.4), (6.5) it follows by Corollary 3.1.1 thatthere exist Yl, .. . , Yn+l E projs(z) and AO 2: 0, Al 2: 0, ... , An+l 2: °withn+l

L x, = 1 such thati=O

n+l

0= AO(Z - w) +L Ai(Yi - z).i=l

(6.9)

This yields (6.2)

3) Now assume that Nf (z) is pointed. So there exists (" E jRn such that

(k, (") < °for all kENf (z), k =I- 0. From z E projsr (w) follows ds(z) = r;therefore, if Y E projs(z), then Y - z E Nf (z), and Ily - zll = r. So weobtain

(y - z, (") < ° for all Y E projs(z).

If AO = °in (6.9), then

n+L

L Ai(Yi - z, (") = 0,i=1

(6.10)

and this contradicts (6.10). Hence AO > 0, and then from (6.9) follows (6.3). 0

Proof of Theorem 6.1. It is enough to consider the case x E bd S . Let °=I~ E Nf(x) . Then for some t > °sufficiently small one has w := x + t~ E Sand projs(w) = {x} .

For all r > °sufficiently small, S; =I- 0. Consider a sequence r; .J.. 0,Zi E projsr (w). Since x is the unique nearest point in S to w we have Zi -+ X •

•

Ify E proj§(zi), then Y- Zi E -N[(y) and lIy - zill = ri, hence lIy - xII < E

for i sufficiently large. So from (6.3),

W - Zi E cony { U -N[(Y)},lIy-xll <e

Since W - Zi -+ W - x = te, this gives (6.1). 0

Acknowledgments

The first-named author gratefully acknowledges the support received fromDAAD. The authors are grateful to the referees for his valuable comments andsuggestions.

References

[1] A. Ben-Tal, E. Rosinger and A. Ben-Israel. A Helly-Type Theorem andSemi-infinite Programming. In: C.V. Coffman and GJ. Fix, editors, Constructive Approches to Mathematical Models, pages 127-135, AcademicPress, 1979.

[2] E. Blum, W. Oettli. Mathematische Optimierung; Springer-Verlag, 1975.

[3] F. H. Clarke, Yu.S. Ledyaev andR. J. Stern. Complements, approximations,smoothings and invariance properties, Journal ofConvex Analysis, 4: 189219, 1997.

[4] H. G. Eggleston. Convexity, Cambridge University Press, Cambridge, 1969.

[5] K. Fan, 1. Glicksberg and A. J. Hoffman. Systems of inequalities involving convex functions, Proceedings of the American Mathematical Society,8:617-622, 1957.

[6] W. Fenchel. Uber konvexe Funktionen mit vorgeschriebenen Niveaumannigfaltigkeiten, Mathematische Zeitschrift, 63 :496-506, 1956.

[7] F. Giannessi , General optimality conditions via a separation scheme. In E.Spedicato, editor, Algorithms for Continuous Optimization, Kluwer, pages1-23,1994.

[8] F. Guerra and M. A. Jimenez. On feasible sets defined through Chebyshevapproximation, Mathematical Method'! of Operations Research, 47:255264 , 1998.

Chapter 4

ASYMPTOTIC CONSTRAINT QUALIFICATIONSAND ERROR BOUNDS FOR SEMI-INFINITESYSTEMS OF CONVEX INEQUALITIES

Wu LP and Ivan Singer-1Departm ent ofMathematics and Statistics. Old Dominion University, Norfolk VA 23529, USA

2 Institute ofMathematics, PO.Box J-764, RO-70700. Bucharest, Romania

wli@odu .edu, [email protected]

Abstract We extend the known asymptotic constraint qualifications (ACQs) and somerelated const ants from finite to semi -infinite convex inequality systems. We showthat, in contrast with the finite case, only some of these ACQs arc equivalent andonly some of these constants coincide, unless we assume the "weak Pshenichnyi Levin -Valadier property" introduced in (12). We extend most of the global errorbound results of [10] from finite systems of convex inequalities to the semi infinite case and we show that to each semi-infinite convex inequality system withfinite -valued "sup-function" one can associate an equivalent semi -infinite convexinequality system with finite-valued sup-function, admitting a global error bound.We give examples that the classical theorem of Hoffman [8] on the existence ofa global error bound for each finite linear inequality system, as well as a resultof [11] on global error bounds for finite differentiable convex inequality systemscannot be extended to semi -infinite linear inequality systems. Finally, we givesome simple sufficient conditions for the existence of a global error bound forsemi -infinite linear inequality systems.

1 INTRODUCTION

Let 9i : JRn ~ JR = (-00, +00) (i E 1) be a family of convex functions,where I is an arbitrary index set, assumed non-empty throughout this paper,and let us consider the system of "convex inequalities"

(i E 1). (1.1)

75

M.A. Goberna and M.A. Lopez (eds .), Semi-Infinite Programming, 75-!00.© ZOO! Kluwer Academic Publishers.


Throughout this paper we shall consider only the above framework, which issufficient for many applications. However, let us mention that some of ourresults and proofs can be extended to arbitrary (finite or infinite dimensional)nonned linear spaces X and to inequality systems (1.1) with convex functions9i : X -7 lR = [-00, +00] (i E I) .

In the sequel we shall assume, without any special mention, that the solutionset S of the system (I.!) is non-empty, that is,

( 1.2)

We shall often consider the important particular case when each 9i is affine,say

(x E R" , i E I), (1.3)

where ai E lRn, bi E lR and (ai,x) denotes the dot product of vectors in lRn

.

In this case, (1.1) becomes a system of linear inequalities

(1.4)

and (1.2) becomes

(1.5)

For the inequality system (1.1) and for any y in the boundary of S, we shalldenote by I(y) the set of active indices at y , that is,

I(y) := {i E II 9i(y) = O}, (1.6)

which will playa fundamental role in the sequel.

For finite convex inequality systems various "asymptotic constraint qualifications" (ACQs), and some related constants, have been introduced in theliterature, in view of applications to global error bounds for such systems. Recently, complete results on their inter-connections have been obtained in [10](see also the survey paper [15] and the references therein). In Section 3 we shallextend these ACQs (except two rather technical ones of [10]) and the relatedconstants to semi-infinite systems, replacing, when necessary, min and max byinf and sup respectively, and we shall study the inter-connections of the ACQsobtained in this way. For some of our proofs we shall use the "sup-functionmethod", consisting in the application of a result which is known for systemsformed by one single inequality, to the sup-function of a given semi-infinitesystem (for the proofs of some results on finite systems, this method has beenused e.g. in [4] and [14]). We shall show that, in contrast with the case of finiteinequality systems, for semi-infinite systems only some of the ACQs are equivalent, and only some of the corresponding constants coincide, but the assumption

ASYMPTOTIC CONSTRAINT QUALIFICATIONS AND ERROR BOUNDS 77

of the weak PLY property (see (2.19)) improves this situation. Furthermore,suggested by the main distance formula of [12], we shall introduce in Section3 a new constant C, which is useful for the study of global error bounds, sincein Theorem 3.4 we shall show that this constant C coincides with the constantoccurring in one of the asymptotic constraint qualifications studied in Section 3.Since the constant C involves the active index sets I(y) of the boundary pointsy of the solution set 5, we shall also introduce a larger constant Co independentof the sets I(y).

The main result of Section 4 is Theorem 4. I , which states that most ofthe global error bound results of [10] on finite systems of convex inequalitiesextend (sometimes with different methods) to the semi-infinite case. Furthermore, while if (1.1) admits no global error bound, then neither one of the twoknown linear representations does, in Theorem 4.3 we shall show that to eachsemi-infinite convex inequality system with finite-valued sup-function one canassociate an equivalent linear inequality system with finite-valued sup-function,admitting a global error bound.

Section 5 will be devoted to the study of error bounds for semi-infinite systems of linear inequalities (1.4). First we shall give an example showing thatthe classical theorem of [8] on the existence of a global error bound for eachfinite linear inequality system (1.4) cannot be extended to the semi-infinite case.Next, we shall give an example showing that the result of [11], according towhich for a finite convex inequality system (1.1) in which each 9i is a differentiable function, the existence of a global error bound implies that the system(1.1) satisfies the Abadie constraint qualification (see (2.8)), cannot be extendedeven to semi-infinite linear systems 0.4). Finally, in Theorem 5.1 we shall givesimple sufficient conditions for the existence of a global error bound for thelinear system (1.4), in terms of the components of the vectors ai.

2 PRELIMINARIES

In this section we recall some definitions and introduce some notations whichwe shall use in the sequel.

We shall consider lRn endowed with the Euclidean norm 11 ·11 = 11.11 2 and withthe topology induced by this norm. For a subset A of lRn we shall denote byA, int A and bd A the closure, the interior and the boundary of A, respectively ;by co A and co A the convex hull and the closed convex hull of A, respectively;by cone A and cone A, the convex cone and the closed convex cone, withvertex 0, spanned by A, respectively.

For two subsets A and B of llt", we shall denote by dist(A, B) the deviationfrom A to B (called also the upper Hausdorff distance from A to B) , defined

by

dist(A, B ) := sup dist (x , B) = sup inf IIx - yll. (2.1)xEA xEA yEll

For a closed convex subset Q of IRn and for y E bd Q, we shall denote byN (Q ;y) the normal cone to Q at y; we recall that

N(Q;y):= {a E IRnl (a,y) = max (a, z)} ,zE Q

(2.2)

where max denotes a supremum which is attained.For any convex functi on 9 : lRn --+ lR and any y E IRn we shall denote by

8g(y) the subdifferential of 9 at y; we recall that

8g(y) := {a E R"] (a, x - y) ~ g(x) - g(y) (x E lRn)} . (2.3)

For the inequality system (1.1), let us introduce, following [7], the notation

(y E bd S). (2.4)

The set N'(S;y) of (2.4) has been called in [5] " the cone of active constraintsat y".

Remark 2.1 a) In other words, for y Ebd S with I(y) i- 0, (2.4) meansthat

N'(S;y) = {?= Aiai!J~ I (y) , IJI < +00, x, ~ 0, ai E 8gi(y) (i E J)} ,IE J

(2.5)

where IJI denotes the number of indices in the set J.On the other hand, using the convention UiE0Mi = 0, (2.5 ) yields that

N'(S; y) = 0 (y E bd S, I(y) = 0). (2.6)

b) In the particular case when each gi is affine; i.e., of the form (1.3), whereai E IRn ,bi E lR (i E 1), ( 1.1) becomes a sys tem of linear inequalities (1.4)and we have 8gi(Y) = {\7gi(y)} = {ail (singleton) for eac h y E IRn, and

N'(S ;y) = { L iEJ Aiai lJ ~ I(y) , IJI < + 00, x, ~ 0 (i E J)}

= cone {a ihEl(Y) (y E bd S).(2.7)

We recall (see [12]) that the convex inequality system ( 1.1) is sa id to satisfy

a) the Abadie constraint qualification, or, briefly, the Abadie CQ, at a pointy Ebd S, if

N(S; y) = N'(S; y) ; (2.8)

b) the basic constraint qualification, or, briefly, the BCQ, at a point y EbdS. if

N(S; y) = N'(S;y); (2.9)

c) the Abadie CQ (respectively, the BCQ), if it satisfies the Abadie CQ(respectively, the BCQ) at all points y Ebd S.

Related to the BCQ are the convex Farkas-Minkowski systems, defined asfollows (see [12]): A linear inequality

(ao, x) ::; bo, (2.10)

where ao E lRn \ {O}, bo E lR, is called a consequence relation of the convexsystem (1.1), if every xES satisfies (2.10). The system (1.1) is called a FarkasMinkowski (or, briefly, an FM) system , if every linear consequence relation ofsystem (1.1) is also a consequence relation of some finite subsystem of (1.1).

Let us recall that for a convex inequality system (1.1) the function

G(x) := SUP9i(X)i EI

(2.11)

is called [7] the sup-junction of the system (1.1), and (1.1) is said to be a systemwith finit e-valued sup-junction, if

G(x) := SUP9i(X) < +00iEI

(2.12)

If (1.1) is a convex inequality system, then G : lRn -+ lR is a convexfunction and, clearly, the system (1.1) is equivalent to the "system" consistingof the single inequality

G(x) ::; 0;

we recall that two systems of convex inequalities, say (1.1) and

(2.13)

(k E K), (2.14)

are said to be equivalent, if their solution sets coincide. It is also obvious thatin the particular case when I is finite , (2.12) is satisfied; i.e., every finite systemof convex inequalities has a finite-valued sup-function.

We shall use the following notation:

Ia(x) := {i E 119i(X) = G(x)} (2.15)

Remark 2.2 a) If G(x) = 0, then xES and IG(x) = I(x) . If G(x) > 0,then x rt Sand I(x) n IG(x) = 0. If G(x) < 0 and G is finite-valued, thenx EintS and I(x) = 0, but IG(x) may be empty or not empty.

b) If G is finite-valued, then it is continuous, whence G(x) = 0 for allx E bdS. Hence, by a) above, if G is finite-valued, then

IG(x) = I(x) (x E bd S). (2.16)

In the particular case when G is the function (2.11), the condition of theexistence of some x' E lRn satisfying

G(x') < 0, (2.17)

is called the strong Slater condition for the convex system (1.1) (sec; e.g., [6]),since the C'usual"} Slater condition for (1.1) requires only the existence ofsome x' E lRn such that

gi(X') < 0 (i E I) ; (2.18)

i.e., of some x' E S with I(x') = 0.Let (1.1) be a convex inequality system with finite-valued sup-function. We

recall (see [12]) that the family {gil i E I}, or the system (1.1), is said to havea) the weak PLV property at a point y E lRn

, if

(2.19)

b) the PLY property at a point y E R", if

(2.20)

c) the weak PLVproperty (respectively, the PLV property), if it has the weakPLY property (respectively, the PLY property) at every y Ebd S.

3 ASYMPTOTIC CONSTRAINT QUALIFICATIONS.THE SUP-FUNCTION METHOD

For our extensions of the ACQs and some corresponding constants, fromfinite systems to semi-infinite systems of convex inequalities, we shall use thesame notations as those used in [10] in the case of finite systems.

For any bi E lR (i E 1) and any TJ > 0 we shall use the notations

(3.1)

and

(3.2)

Theorem 3.1 Let (1.1) be a system satisfying the strong Slater condition.with finit e-valued sup-junction. The f ollowing conditions are equivalent:

(ACQ 1) There exist posit ive constants t1 and 8 such that

dist (8, 8(b)) ::; t1 (lIbll oo := sup Ibi l ::; 8);iET

(3.3)

(ACQ2) There exist positive constants t1 and TJ such that

dist (8, 8( -TJ)) ::; t1j

(ACQ3) There exists a positive constant , such that

lIy - a] 1inf sup inf < -C>0 YEbdS ZES(-£ ) infiEI(-gi(Z)) - "

(3.4)

(3.5)

Each of these conditions implies the strong Slater condition (2.17).

Proof. The last statement is obvious, since inf 0 = + 00, and since the strongSlater condition is equivalent to 8( -TJ) i- 0 for suffi ciently small TJ > O.The equivalences (ACQI )<=>(ACQ2)<=>(ACQ3 ) can be proved with the "supfunction method". Indeed, for a finite system of inequalities, the theorem hasbeen proved in [10, Theorem I] . Appl ying this result to the system consistingof the single convex inequality G(x) ::; 0, where G is the sup-function (2.11)of (Ll), and using that -G(x) = infiET(-gi(X)) , we get the theorem. 0

Theorem 3.2 Let (1.1 ) be a system withfinite-valued sup-junction. and let, > O. Then the inequality (3.5) of (ACQ3) implies the following conditions.equivalent to each other:

(ACQ4) (1.1) satisfies the strong Slater condition (2.17) and

sup inf inf lI y - zil < ~.yEbd s£>o z ES (-£ ) infiEl( -gi(Z)) - ,

(ACQ9) We have

inf min lIall ~ , .yEbd S aE8G(y)

(3.6)

(3.7)

(ACQ II ) The strong Slater condition (2.17) is satisfied. and for each sequence {yd ~ bd 8 such that limk--t oo IIYkll = +00, the zero vector is not alimit point of any sequence {ad with ak E 8G(Yk) (k = 1,2, ...).

Proof. Again , we shall use the "sup-function method".The impl ication (ACQ3)::::}(ACQ4) follows by applying [10, Theorem 3], to

G(x) ::; 0 (note that more simply, it follows also directly from the well-knownfact that we always have "infc>o sUPYEbd S ~ SUPyEbd S inf£>o)".

82 SEMI-INFINITE PROGRAMMING. RECENrADVANCES

The equivalences (ACQ4){:}(ACQ9){:}(ACQII) follow by applying [10,Theorem 2], to G(x) ::; 0. 0

Remark 3.1 The implication (ACQ4)=>(ACQ3) is not valid, even when Iis finite, as shown by [10, Example 2], combined with [10, Theorems 2 and I].

Let us recall the following lemma of [10], which we shall use in the sequel.

Lemma 3.1 ([10, Lemma 2]) Let K be a subset oJJRn.lfthere exist x ElRn with IIxll = 1 and T > °such that

(a,x) ~ T (a E K),

then there also exists x' E cone K with IIx'll = 1, such that

(a,x') ~ T (a E K) .

(3.8)

(3.9)

Now we can prove the following theorem.

Theorem 3.3 Let (1.1) be a system with finite-valued sup-Junction and letT> 0. Then

a) Condition (ACQ9) ofTheorem 3.2 implies the following conditions, equivalent to each other:

(ACQ6) We have

inf sup inf min (a, ~) ~ T' (3.10)yEbd S XEN(S iY)\{O} iEI(y)aEagi(y) IIxll

where the min is attained since each ogi (y) is compact.(ACQ7) We have

inf sup inf min (a,~) ~ T' (3.11)yEbd S XERn\{O} iEI(y)aEagi(y) IIxll

(ACQ8) We have

I:iEJ x, ::; ~ II I:iEJ Aiai II(y E bd S, J ~ I(y) , IJI < +00, Ai ~ 0, ai E o9i(y) (i E I(y)) .

(3.12)

b) If the weak PLVproperty holds , then

(ACQ6) {:} (ACQ7) {:} (ACQ8) {:} (ACQ9). (3.13)

Proof. a) (ACQ9)=>(ACQ8): Assume that (ACQ9) holds and let y Ebd SandJ ~ I(y),IJI < +00, Ai ~ 0, ai E ogi(Y) (i E I(y)) be arbitrary, withI:iEJ x, i= O. Then

A'I:: I: t A'ai E co (UiEI(y)o9i(Y)) ~ oG(y)iEJ jEJ J

(for the last inclusion sec; e.g., [7, Lemma 4.4.1, p. 267]), whence, by (ACQ9),

which yields (ACQ8).For the case when I is finite, it has been observed in [10] that the implication

(ACQ6)=}(ACQ7) is obvious, since N(S; y) ~ IRn. For an arbitrary index set

I , the situation is the same.(ACQ7)=}(ACQ6): Assume that (ACQ7) holds, and let y Ebd Sand 0 <

E < ,. Then, by (ACQ7), there exists Xc E lRnwith IIxcII = 1, satisfying

(a, xc) ~ f - E (a E 8gi (y), i E I(y)), (3.14)

whence also

(3.15)

Hence, by Lemma 3.1, there exists x~ E cone [co (Ui EI(y)8gi (y))] ~ N(S;y)with IIx~1I = 1, such that

(3.16)

Consequently,

sup inf min (a,~) ~ 't .XEN(SiY)\{O} iEI(y) aE&gi(Y) IIxll

which yields (ACQ6).For finite I, the implication (ACQ7)=}(ACQ8) has been proved in [10]. For

an arbitrary index set I the argument is the same and we give it here for thesake of completeness. Condition (ACQ7) means that for any y Ebd SandE > 0 there exists Xc E IRnwith Ilxcll = 1, satisfying (3.14). Then, for anyJ ~ I(y), IJI < +00, and x, ~ 0, ai E 8gi (y) (i E J), with LiEJ x, =1= 0, wehave

whence (ACQ8) follows.


(ACQ8)=>(ACQ7) : Assume that (ACQ8) holds, and let Y Ebd S. Then, by(ACQ8),

Iiall ~ 'Y

and hence

(3.17)

But, by the well-known formula for the distance to a closed convex set, we have

dist (0, co (UiEI(y)8Yi(Y))) = sup inf (a, x) . (3.18)xERn\{o} aEro (UiEJ(y)09i(Y))

Ilxll=l

Hence, by (3.17) and (3.18), for each E > 0 there exists X e E lRn with IIxe li = 1satisfying (3.14), so (ACQ7) holds.

b) Assume that there hold the weak PLY property and (ACQ8), and letY Ebd S and a E 8G(y) . Then, by the weak PLY property, a = limm -+oo am,where am Eco(UiE1o(y)8Yi(Y)) (m = 1,2, ...). Thus, for each m we canwrite am = L:iEJm Ai,mai,m, with ai,m E 8Yi(Y), Ai,m ~ 0, L:iEJm Ai,m =1, i E Jm ~ Ia(Y) = I(y) (by Remark 2.2b» . Hence, by (ACQ8), we havelIam li ~ 'Y (m = 1,2, ...), whence Iiall ~ 'Y, which yields (ACQ9). Thus, bypart a), we obtain (3.13). 0

Remark 3.2 a) Since for a finite set I the PLY property always holds,from Theorems 3.2 and 3.3 we obtain again the known result (see [10, Theorem 2]) that in the particular case when I is finite, the asymptotic constraintqualifications (ACQ4), (ACQ6)-(ACQ9) and (ACQ11) are equivalent.

b) As we shall show in Example 3.2 below, the assumption that the weak PLYproperty holds cannot be omitted in Theorem 3.3 (for the implication (ACQ8)=>(ACQ9» .

In connection with the above asymptotic constraint qualifications, let usconsider the following constants (for finite systems (1.1) these have been introduced in [10], and now we extend them to arbitrary systems (1.1) , keeping the

notations used in [10] for the particular case of finite systems):

,8(ACQ3) := inf sup inf . lIy - zll , (3.19)e >OYEbdszES(-e) mfiEr(-gi(Z))

,8(ACQ4) := sup inf inf . lIy - zll, (3.20)yEbd S 00 zES( -c) mfiEI( -gi(Z))

,8(ACQ6):= inf sup inf min (a,~) (3.21)yEbd S XEN(S;y)\{O} iEI(y) aEogi(y) IIxll

,8(ACQ7):= inf sup inf min (a,~), (3.22)yEbd S XERn\{O} iEI(y) aEogi(y) IIxli

,8(ACQ8):= inf inf [c] , (3.23)yEbd S aEco (UiEJ(y)ogi(Y))

,8(ACQ9):= inf min lIall, (3.24)yEbd S aEoG(y)

where the minima are attained, since ogi(Y) and oG(y) are compact. From theabove results we obtain the following corollary.

Corollary 3.3.1 Let (1.1) be a system with finite-valued sup-function.a) We have

1 1,8(ACQ3) 2:: ,8(ACQ4) = ,8(ACQ9) 2:: ,8(ACQ8)

b) If the weak PLV property holds, then

1f3(ACQ7)

1,8(ACQ6)

(3.25)

1,8(ACQ4) = ,8(ACQ9)

1,8(ACQ8)

1,8(ACQ7)

1,8(ACQ6) .

(3.26)

As has been observed in [10, Section 5], the first inequality in (3.25) may bestrict, even when I is finite.

According to [12, Theorem 5.6(a)], if Xo E lRn\S and Yo is the projectionof Xo onto S, and if (1.1) satisfies the Abadie CQ, then

dist(xo, S) =

sup sup max. L Ai(ai, Xo - Yo}. (3.27)Jt;l(yo) ajEogj(Yo) (jEJ) Aj ~O (JEJ) iEJ1J1<+oo IILjEJ Ajajll=l

The distance formula (3.27) suggests to introduce the following new constant.

(3.29 )


Definition 3.1 Let (1.1) be a system of convex inequalities in IRn , withsolution set (1.2). We define the constant

Theorem 3.4 We have

1C = .B(ACQ8)'

Proof. Lety EbdS, J ~ I(y) , Aj ~ O,aj E agj(Y) (j E J), IILjEJ Ajajll =

1, LiEJ x, i- O. Then

and hence

1 1

infa E co (UiEJ (I/ )o9i(Y)) lIall ~ "LjEJ Ei~~ Ai aj ,, =~ x,

Consequently,

1.B(ACQ8) ~ C. (3.30)

Inorder to prove the opposite inequality, let y Ebd S and a Eco (UiEI(y)a9i(Y))be arbitrary. Then there exist J ~ I(y), Aj ~ 0, aj E agj(Y) (j E J) , withLjEJ Aj = 1, such that a = LjEJ Ajaj. Let

(i E J). (3.31)

Then J.Li ~ 0 (i E J) and IILiEJ J.Liadl = 1, whence, by (3.31) and (3.28),

_1_ = LiEJ Ai = L J.Li :::; C.

II all IILjEJ Ajajll iEJ

Consequently,

1 1-::-:""-:---:---=---:- = sup sup - < C,.B(ACQ8) yEbdS a Eco (UiEJ(I/ )o9i (y)) [c] -

which, together with (3.30), yields (3.29) . 0

Remark 3.3 In order to avoid the problem of the identification of the activeindex sets I(y), one can use the following constant:

Co := sup sup sup max. L x;yEbd S J';;I ajEogj(Y) (jEJ) Aj~O (JEJ) iEJ

1J1<+oo IIEjEJ Ajajll=l

For the constants C (of (3.28)) and Co we have, clearly,

C:::; Co.

(3.32)

(3.33)

Hence, if Co < +00 , which one can ensure in some cases, as we shall see inSection 5, then C < +00. The inequality (3.33) may be strict, even when I isfinite, as has been shown in [3] . Similarly to the above proof of Theorem 3.4,one can show that

1 1

Co = infYEbdSinfaEco(UiElogi(Y)) lIall = dist (O,UyEbdS co (UiEI 89i(Y)))'

(3.34)

Let us give now some examples of inequality systems with finite-valued supfunctions, which show that certain implications between the above conditionsdo not hold.

In the following example, the asymptotic constraint qualification (ACQ8),the Slater condition (2 .18) and the BCQ are satisfied, but (ACQ9) does not holdfor any constant (i.e. , (3(ACQ9) = 0), and the strong Slater condition as wellas the weak PLY property are not satisfied.

Example 3.1 Let n = 1, I = {I, 2, ...}, and for each i E I let

if x < -~z(3.35)

if x > -1.- z

Then 9 i(-1) = -t < 0 for all i E I, so the Slater condition (2.18) is satisfied,but

{0 if x < 0

G(x) = SUP9i(X) = if > 0iEI x I x _ ,

(3.36)

so the strong Slater condition (2.17) is not satisfied for any x' ; note also thatthe weak PLY property is not satisfied either. Furthermore, S = (-00,0],bd S = {O}, 1(0) = {i E II9i(X) = O} = I, and 8G(O) = [0,1], 89i(O) ={I} (i E I), whence dist(O, co (UiEI(0)89i(O))) = 1. Thus, condition (ACQ8)

is satisfied. and hence so are the equivalent conditions (ACQ6) and (ACQ7).Note also that N(S; 0) = [0, +00) = N'(S; 0), so the BCQ holds.

In the next example, (ACQ9) (whence also (ACQ8) and the strong Slatercondition) and the BCQ are satisfied. but we do not have the equality j3(ACQ8)= j3(ACQ9) and the weak PLY property does not hold. This also shows thatthe assumption of weak PLY property cannot be omitted in Theorem 3.3 (forthe implication (ACQ8)*(ACQ9)) and in the second equality of (3.26) .

Example 3.2 Let n = 1,1 = {1, 2, " ,}, and

Then

9;(X)~ {x-~

t

2x3x - ~

t

if x <-~I t Iif - -s- < x < ~t- -t

'f II x> T'

(3.37)

{X if x < °

G(x) = ~uP9i(X) = 3 ' f > °tEl X I X _ •

(3.38)

(3.39)

Furthermore, S = (-00,0], bdS = {O}, 1(0) = {i E l!gi(O) = O} = I, and8G(0) = [1,3], 8gi(0) = {2} (i E I), whence

j3(ACQ8) = dist (0,co (UiEI(O)8gi (0))) = 2

> 1 = dist (0, 8G(0)) = j3(ACQ9)

(therefore, in particular, we do not have the weak PLY property). Thus, thesecond inequality in (3.25) is strict, and for, = 2 (ACQ8) holds, but not(ACQ9) (which, however, holds for " = 1; we do not know whether it ispossible to have simultaneously (ACQ8) for some, > 0, but not (ACQ9) forany,' > 0) . At the same time, N(S; 0) = [0, +00) = N'(S; 0), so the BCQholds.

Let us also mention an example of a linear inequality system with some ofthe above properties.

Example 3.3 Let n = 1, 1 = {(1, k); 2; (3, k)1 k = 1,2, ... }, and

(al,k, x) = x, (a2' x) = 2x , (a3 ,k, x) = 3x (x E JR, k = 1,2, ...), (3.40)

Then

1 1bl,k = k' ~ = 0, b3,k = k (k=1,2, ...). (3.41)

if x <°if x 2: 0.

(3.42)

(3.43)


Hence G (-1) = -1, so the strong Slater condition is satisfied. Furthermore,S = (-00,0], bd S = {O}, 1(0) = {i E II (ai'x) = bd = {2}, and8G(0) = [1,3]' co {aihEl(o) = {2}, so there holds (3.39) (therefore, inparticular, we do not have the weak PLY property), whence the second inequalityin (3.25) is strict. At the same time, N(S; 0) = [0, +00) = N'(S;0), and thusthe BCQ holds.

Remark 3.4 M. A. Lopez has raised the question whether a FM systemsatisfying the strong Slater condition (2.17) has the weak PLY property. Example 3.3 above shows that the answer is negative even for linear inequalitysystems. Indeed, the inequality system (ai,x) ~ bi (i E I), with ai andbi of (3.40), (3.41) , is a FM system (since it is equivalent to the inequality(a2' x) ~ bz, and hence every consequence relation is also a consequence ofthe inequality (a2' x) ~ b2). A similar remark is valid also for the convexinequali ty system 9i (x) ~ 0 (i E 1), with 9i of Example 3.2 above .

In the next example, (ACQ9) (whence also (ACQ8) and the strong Slatercondition), the equality !3(ACQ8)= !3(ACQ9), and the BCQ are satisfied, butwe do not have the weak PLY property.

Example 3.4 Let n = 1,1= {1, 2, ...}, and

{

X if x < ~

9i(X) = 1 ~2x -..,. if x> ..,..

l - l

Then

{

X if x < 0G(x) = ~~r9i(X) = 2x if x ~ O. (3.44)

Hence G (-1) = -1 , so the strong Slater condition is satisfied. Furthermore,S = (-00,0], bd S = {O}, 1(0) = {i E II9i(X) = O} = I, and 8G(0) =[1 ,2], 89i(0) = {1} (i E I), so we do not have the weak PLY property.However, N(S;0) = [0, +00) = N'(S; 0), so the BCQ holds.

One can modify Example 3.4 so as to obtain a system satisfying (ACQ9),the equality !3(ACQ8) = !3(ACQ9) and the BCQ, and having the weak PLYproperty, but not the PLY property.

Example 3.5 Let n = 1, I = {1, 2, ...}, and let {Ci} be a sequence of realnumbers such that ~ = infiEI Ci < Cj < sUPiEI Ci = 1 (j E I) . Let us define

{CiX if x < 0

9i(X) = 2CiX if x ~ O. (3.45)

Then

{

I XG(x) = SUP9i(X) = 2

iEI 2xif x < 0if x ~ O.

(3.46)


Hence G (-1) = - ~, so the strong Slater condition is satisfied. Further

more , S = (-00,0], bd S = {O} ,1(O) = I, and 8G(0) = [~, 2]' 89i(0) =[ci,2q] (i E I), whence co (U iEI89i(0)) = (~, 2) . Thus, we have the weakPLY property, butnotthePLV property. Also, N(S; 0) = [0, +(0) = N'(S; 0),so the BCQ holds.

4 ERROR BOUNDS FOR SEMI-INFINITE SYSTEMSOF CONVEX INEQUALITIES

Given a convex inequality system (1.1), we shall consider now the problemof the existence of a "global error bound"; i.e., of a (finite) constant A > 0 suchthat

dist(xo, S) :s Asup 9i (xo)+iEI

(4 .1)

Using the sup-function G of (2.11), one can write (4.1) in the form

dist(xo, S) :s AG(XO)+ (4.2)

which shows that the results on error bounds for finite inequality systems involving only the function G extend automatically to semi-infinite systems ofinequalities, with the sup-function method. Note also that at the points Xo whereG(xo) = SUPiEI9i(XO) = +00, the inequality (4.1) holds trivially, with anyconstant A > O.

Theorem 4.1 Let (1.1) be a system ofconvex inequalities in JRn , withfinitevalued sup-junction and with solution set S. For any 'Y > 0, let us consider thefollowing statements:

(Cl) We have

inf inf sup max (a,~) ~ 'Y. (4 .3)yEbd S xEN(SjY)\ {oJ iEI(y) aE8gi(Y) IIxll

(C2) Let G'(y, x) be the directional derivative ofG at y in the direction ofx. We have

G'(y,x) ~ 'Yllxll (y E bd S,X E N(S;y)). (4.4)

(C3) We have

. 1dist (xo,S) :s - SUP 9i (XO)+

. 'Y iEI

Thena) We have

(4.5)

(C1) =? (C2) {:} (C3). (4 .6)

b) If the weak PLV property holds, then

(C 1) {:} (C2) {:} (C3). (4.7)

c) (ACQ6) (and hence anyone of the asymptotic constraint qualificationsgiven in Theorems 3.1-3.3) implies (Cl) (and hence also (C2) and the globalerror bound (C3)).

In each of these statements, one can take

I = (3(ACQ6). (4 .8)

Proof. a) (C1)=>(C2): If (C1) holds, then for any y Ebd S, x E N(S; y)\{O},we have

G' (y,~) = max (a,~) > sup max (a,~) > III xII aEoG(y) IIxli - iEI(y) aEo9i(y) IIxll-

(the first equality is a well-known formula of convex analysis), whence G'(y, x)= IIxli G'(y, rfxrr) 2: IlIxll ·

Furthermore, since G is finite-valued, whence continuous, we have y Ebd Sif and only if G(y) = 0, and therefore, when III = 1, (C2) is nothing else thancondition (b) of [14, Theorem 1]. Hence, the equivalence (C2){:}(C3) followsby applying the sup-function method to [14, Theorem 1].

b) We only have to prove the implication (C2)=>(C1), under the assumptionthat the weak PLY property holds. Now, by the linearity and continuity of themapping a ~ (a, rfxrr) , Remark 2.2b), the weak PLY property and (C2), we

obtain

x xsup max (a,-) = sup (a, -)

iEI(y) aEo9i(y) IIxli aEuiElc(I/)09i(y) IIxlix x

sup (a,-) = max (a,-)aEro (U iElC(I/)09i(y») [c] aEoG(y) IIxli

= G'(y, 11:11) 2: I (y E bd S, x E N(S; y)\{O}) ,

whence (Cl) follows.

c) By the well-known inequality "inf sup 2: sup inf", (ACQ6) implies (C1).Hence, by part a) above and by Theorems 3.1-3.3, the conclusion follows. 0

Remark 4.1 a) As shown by Example 3.2, the assumption of weak PLYproperty cannot be omitted in Theorem 4.1b), and (C2) does not imply (C1)with the same constant I (since in Example 3.2 for I = 3 (C2) holds, but not(C1): the constant on the left hand side of (C1) is I' = 2 < ,) . We do not

know whether it is possible to have simultaneously (C2) for some, > 0, butnot (Cl) for any,' > O.

b) In the particular case when I is finite, the PLY property holds, so Theorem4.1b) implies again the known result that the implication (C2)*(C I) holds true(see [10, the first part of Theorem 4]). Also, for I finite it is known thatanyone of the asymptotic constraint qualifications given in Theorems 3.1-3.3implies (C I) and hence also (C2) and the global error bound (C3) (see [10, thesecond part of Theorem 4]). We do not know under what conditions does theimplication (Cl)~ (ACQ6) hold, with the same constant, or with some otherconstant ,'.

c) The fact that (ACQ 11) (or, equivalently, (ACQ4) or (ACQ9)) implies theexistence ofa global error bound (4.1), follows also by applying the sup-functionmethod to a result of Auslender and Crouzeix [1, Theorem 2].

Remark 4.2 Assuming that (1.1) satisfies the Abadie CQ, one can alsogive another proof of Theorem 4.1c), in the equivalent form

. 1dist (xo, S) :s {J(ACQ8) ~~f 9i (xo)+ (4.9)

Indeed, by Theorems 3.1-3.3, anyone of the asymptotic constraint qualifications given in those theorems implies (ACQ8), so .B(AbQ8) < +00. Let

Xo E lRn\S. Then, for any J ~ I(yo) (where Yo is the projection of Xo onto

S) and )..j ~ 0, aj E 89j(Yo) (j E J), with IILjEJ )..jajll = 1, we have,

using also the definition (3.28) of C,

L )..i(ai, Xo - Yo) :s L )..i[9i(XO) - 9i(YO)] = L )..i9i(XO)iEJ iEJ iEJ

:s L)..i9i(XO)+:S CSUP9i(XO)+;iEJ iET

whence, by (3.27) and Theorem 3.4, we obtain (4.9).

Theorem 4.2 Let (1.1) be a system of convex inequalities in lRn, withfinite-valued sup-junction and with solution set (1.2).

a) If (1.1) satisfies the strong Slater condition (2.17), then it admits a localerror bound, namely

. IIxo -x'lIdlSt (xo,S) :s . f ( (')) sup 9i(XO)+

III iET -9i x iET(4.10)

b) If (1.1) satisfies the strong Slater condition (2.17) and the solution set Sis bounded, then (1.1) admits a global error bound (4.1).


Proof. For the case when III = 1, these results are due to Robinson ([17,Section 3]; for a), see also [2, Proposition 17]). The general case follows byapplying the sup-function method. 0

Remark 4.3 Part b) also follows by combining part a) and [13, Lemma 6and formulas (57) and (58)].

Let us observe now that, since Q ~ (3 implies Q+ ~ (3+, from [12, Proposition 5.2(b)] and the observation that a similar result holds also for the standardlinear representation

(a, x) ~ (a, y) - 9i(Y) (4.11)

(indeed, for the sup-functions G and G of (1.1) and (4.11), respectively, and forany x E lRn we have, by the definitions of the sup-function and of 89i(Y),

G(x) = sup {(a,x) - (a,y) + 9i(Y)} ~ sup 9i (x) = G(x)),YERn ,iEI,aEogi(Y) iEI

(4.12)

it follows that if the standard linear representation (4.11) or the linear representation

(y E bd S, i E I(y) ,ai E 89i(Y)) (4.13)

of (1.1) (see [12, Theorem 5.1 ]) admits a global error bound (4.1), then sodoes (1.1). Hence, if (1.1) admits no global error bound, then neither one ofthese linear representations does. Nevertheless, we have the following result:

Theorem 4.3 Let (1.1) be a convex inequality system with finite- valuedsup-junction. Then one can associate to (1.1) an equivalent linear inequalitysystem with finite-valued sup-junction, admitting a global error bound andsatisfying the BCQ, but not satisfying the strong Slater condition.

Proof. We shall show that the linear inequality system

(a, x) ~ (a, y) (y E bdS,a E N(S;y), lIall = 1) (4.14)

has the required pro~rties.

Let us denote by S1 the solution set of (4.14), that is,

81 := {x E lRnl (a, x) ~ (a, y) (y E bd S,a E N(S; y), lIall = I)} . (4.15)

First we shall show that S = S1. If XQ E S, then for any y Ebd Sand

a E N(S; y) we have (a, x) ~ (a,y), whence S ~ 81• On the other hand, the

opposite inequality 81 ~ S is proved as follows : Let XQ E 81• If XQ ~ S, then,

since 8 is closed and convex, by a consequence of the Brendsted-RockafellarTheorem (see; e.g., [16, proposition 3.20]), there exist ao E lRn and Yo E bd Ssuch that lIaoll = 1 and

(ao, Yo) = max (ao , z) < (ao,xo).zES

(4.16)

Then ~o E N(S; Yo), which, togeth:-r with (4.16), contradicts the assumptionXo E 8 1. This proves the inclusion 8 1 ~ 8 . Thus, (4.14) is a linear representation of 8.

Now, by the well-known duality formula for the distance to a closed convexset (see e.g. [9, p.62, formula (1)]) and the definition of normal cones, we have

dist (xo ,8) = SUPaERn ,llall= l {(a, xo) - SUPZES (a, z)}

~ sUPaEUYE bdsN (Sjy),llall= l ((a, xo) - sUPZES (a,z))+ (4.17)

= sUPYE bd S,aEN(Siy) ,llall=l (a, Xo - y)+ (xo E IRn) .

On the other hand., let Yo be the orthogonal projection of Xo onto 8. Then, bya well-known characterization of nearest points in closed convex sets (see e.g.[18, Theorem 5.1] ), there exists a vector ao E N(8; Yo) such that lIaoll = 1and (ao,xo - Yo) = IIxo - Yo II = dist (xo, 8), whence the last sup in (4.17) isattained for a = ao and it is equal to dist (xo, 8). Consequently, we obtain

dist (xo, S) = sup (a, Xo - y)+ (xo E lRn ) , (4.18)yE bd S,aEN(Sjy),llall=l

so (4.14) admits the global error bound A = 1.Furthermore, since the systems (1.1) and (4.14) are equivalent, their solution

sets 8 , and hence also their normal cones N(8; x) at each x E bd 8 , are thesame. In order to show that (4.14) satisfies the SCQ, we shall denote the coneN'(8;x) of this system by Nf(S;x).

Let us observe that (4.14) can be written in the form

(a~,x):s b~ (k E K), (4.19)

by taking

K :={(y,a)jyE bd8,aEN(8;y),lIall=1}, (4.20)

(4.21)((y ,a) E K) .a(y,a) = a, b(y,a) = (a, y)

Let x Ebd S. Then, by the definitions,

N(8;x) 2 Nf(S;x) =cone {a(y,a)}(y ,a)EI(x)

=cone {ailiall = 1,3y E bd 8, a E N(8; y), (a, x) = (a, y)} 2 N(8; x),(4.22)

which proves that the system (4.14) satisfies the BCQ.

Finally, by the above argument, for the sup-function G 1of the linear system(4.14) we have

sup ((a,x) - max (a,z))aEUyE bd S N(Siy) ,llall= l zES

dist (x, S) < +00 (x E lRn) ,

so G1 is finite-valued and there exist no x' E lRn satisfying (2.17). 0

(4.23)

Let us also raise an open problem. In [11] it has been proved that for a finiteconvex inequality system (1.1) in which each 9i is a differentiable function,the existence of a global error bound implies that the system (1.1) satisfiesthe Abadie constraint qualification (see (2.8)) . Therefore, it is natural to askwhether this result remains valid for semi-infinite convex inequality systems(1.1). In the next section we shall show that the answer is negative even in thecase when each 9i is an affine function (1.3). However, we do not know whetherunder some additional assumption one can obtain a positive answer. While inthe preceding sections the condition that (1.1) should have the PLY property hasturned out to be useful additional assumption for the extension of some resultsfor finite inequality systems to semi-infinite inequality systems, we do not knoweven for a semi-infinite linear inequality system whether the PLY property andthe existence ofa global error bound implies that the system (1.1) satisfies theAbadie constraint qualification. Note that in this problem we may assume thatthe strong Slater condition is not satisfied; i.e., G(x) := sUPiEr 9i(X) 2: 0 (x ElRn

) ; indeed, by [12, Proposition 4.2 (a)], the strong Slater condition, togetherwith the weak PLY property, imply the Abadie constraint qualification. Notealso that one cannot use the sup-function method to solve this problem, sinceeven when each 9i is affine , G need not be differentiable, so the result of [11]cannot be applied.

5 ERROR BOUNDS FOR SEMI-INFINITE SYSTEMSOF LINEAR INEQUALITIES

The first natural question is whether the classical theorem of [8] on theexistence of a global error bound for each finite linear inequality system (1.4)can be extended to the semi-infinite case . Now we shall show that the answeris negative.

Example 5.1 Let n = 1, I = {I, 2, ... }, and

(x E JR, i = 1,2, ...). (5.1)

(5.6)


Then S = (-00,0] , bd S = {OJ, N(S j 0) = [0, +00). Furthermore,

G(x) = s~P (~X - i;) ~ x2 (x E JR, i= 1,2, ... ), (5.2 )

so G is finite-valued. Also,

G(Xi) = X~ (Xi = ;, i = 1,2, .. .) , (5.3)t

and there exists no finite C > 0 such that

dist (Xi, S) = xi ~ Cx~ = CG(xd+ (Xi = ;, i = 1,2, ... ) , (5.4)t

so the inequality system gi(x) ~ 0 (i = 1, 2, ...) admits no global error bound.

Remark 5.1 There is no point in asking for the weak PLY property in thisexample, since 1(0) = {il (ai, 0) = bd = {il 0 = fr} = 0. However, let usadjoin {+oo} to I, and define a+oo = 0 = b+oo . Then the new index set,which we shall denote again by I, becomes compact, and for the new inequalitysystem gi(X) ::; 0 (i = 1,2, ..., +00) the assumption of upper semi-continuityof the PLY theorem is satisfied, so there holds the PLY (and hence the weakPLY) property. But, we have no global error bound, and the Abadie constraintqualification is not satisfied, since 1(0) = {+oo} and

cone{adiEI(O) = cone{a+oo } = {OJ i= [0,+00) = N(SjO). (5.5)

Now we shall show that for a linear inequality system (1.4) the existence ofa global error bound does not imply that the system (1.4) satisfies the Abadieconstraint qualification.

Example 5.2 Let 1= {O, 1,2, . .. }, and

{

X2 - 1 if i = 0,-Xl if i = 1,

gi(X) := Xl - 1 if i = 2,

-X2 - t if i = 3,4, ...

Then, as has been shown in [12, Example 1], S = {(XI,X2)/ 0 ~ XI,X2 ~ I},the family (5.6) does not satisfy the Abadie constraint qualification (indeed ,o E bd S, and N(Sj 0) = {(tl, t2)! tl ~ 0, t2 ~ OJ, while 1(0) = {i EII gi(O) = O} = {I}, whence N'(S; 0) = cone{(-1, On = {(-t, 0)It ~ O}).

Thus, it remains to show that we have a global error bound ; i.e., there existsa (finite) constant). > 0 such that (4.1), or equivalently, (4.2), holds, where Gis the sup-function (2.12). Now, by (5.6), for any X = (XI,X2) E JR2 we have

G(x) = max{x2 -1, -XI ,XI -1,suPi>3(-x2 -!)}- %

(5.7)

ASYMPTOTIC CONSTRAINT QUALIFICA110NS AND ERROR BOUNDS 97

Let us consider the finite linear inequality system

(i=1,2,3,4) , (5.8)

where 91,92 and 93 arc the same as in (5.6), and

(5.9)

Since (5.8) is a finite linear inequality system, by Hoffman's Theorem it hasa global error bound. But, clearly, the solution set and the sup-function for (5.8)arc the same as those for the initial system (5.6). Hence (5.6) has a global errorbound, too.

Remark 5.2 (a) In the above example we do not have the weak PLY property. Indeed, by the above, we do not have the Abadie constraint qualification.On the other hand, the strong Slater condition is satisfied, and hence the assertion follows from [12, Proposition 4.2 (a)] .

(b) The above argument suggests the following method to prove that a givensystem of linear inequalities (1.4), with solution set 8 and sup-function G,admits a global error bound: if we can find a finite linear inequality systemwhose solution set 8 1 is a subset of 8 and whose sup-function G 1 is less thanG, then (1.4) admits a global error bound. Indeed, the new finite system admitsa global error bound (by Hoffman's Theorem), say, A> 0, and we have

(Xo E R").(5.10)

However, it is easy to show that this method works only if 8 1 = 8, hence onlyif the solution set 8 of the initial (possibly infinite) linear inequality system(1.4) is a polyhedron (that is, an intersection of a finite number of half-spaces).Indeed, by

8 = {xjG(x) ~ a}, 8 1 = {xIG 1(x) ~ a}, (5.11)

and by G 1 ~ G, we must have 8 ~ 8 1. On the other hand, the above methodhas required that 8 1 ~ 8. Thus, we must have 8 1 = 8. Since we have required8 1 to be the solution set of a finite linear inequality system, 8 1 is a polyhedron,hence so is 8, which proves our assertion.

Finally, let us also give a positive result. In the case where each 9i is an affinefunction (1.3), 89i(Y) is the singleton {ad, for each y E m.n and i E I, andhence the constant Co of (3.32) becomes

Co = sup max L::Ai.JCI Aj ~o (jEJ) . J- II IEIJI<+oo L:j E J Ajaj 11=1

(5.12)

One can give simple sufficient conditions in order that Co < +00(and hence,by (3.33), C < +(0), in terms of

Theorem 5.1 a) If

ail := M > 0

and

(i E 1).

(i E I),

(5.13)

(5.14)

then Co < +00.b) If

(i E I;k = 1, ...,n), (5.15)

inf aik := m > 0,iEI,l:'Sk:'Sn

(5.16)

then Co < +00.

Proof. Since all norms on lRn are equivalent, we will show the result replacingthe Euclidean norm 11·11 = 11.11 2 on lRn by 11,11 00 , the Zoo-norm.

a) Assume that there hold (5.14) and (5.15). Then, for any J ~ I withIJI < +00and Ai ~ 0 (i E J), we have

IILiEJ Aiadloo = IILiEJ Ai(ail , .., ain)lloo

= II (LiEJ Aiail, .., LiEJ Aiain)1100 = max1:'Sk:'Sn ILiEJ AiaikI= max {(LiEJ AdM, max2:'Sk:'Sn ILiEJ AiaikI} = (LiEJ AdM.

(5.17)

Hence, if IILiEJ Aiadloo = 1, then LiEJ x, = ir· Consequently, Co < +00.b) Assume (5.16). Then, by (5.16) and (5.17), for any J ~ I with IJI < +00,

and Ai ~ 0 (i E J), we have

00

(5.18)

Hence, if IILiEJ Aiailloo+00. 0

1, then LiEJ Ai < ~ . Consequently, Co <

Acknowledgments

The research ofWu Li was partially supported by the National Science Foundation grant NSF-DMS-9973218.


The research of Ivan Singer was supported in part by the National Agencyfor Science, Technology and Innovation, grant nr. 4073/1998.

The authors wish to thank the referee for the careful reading of the manuscript,and for many valuable remarks which contributed to the improvement of thispaper.

References

[1] A. Auslender, and J.-P. Crouzcix. Global regularity theorems, Mathematicsof Operations Research, 13:243-253, 1988.

[2] H. Bauschke, J. M. Borwein, and W. Li. Strong conical hull intersectionproperty, bounded linear regularity, Jameson's property (G), and error boundsin convex optimization, Mathematical Programming, 86A:135-160, 1999.

[3] C. Bergthaller, and I. Singer. The distance to a polyhedron, Linear Algebraand its Applications, 169:111 -129, 1992.

[4] S. Deng. Global error bounds for convex inequality systems in Banachspaces, SIAM Journal on Control and Optimization , 36 :1240-1249, 1998.

[5] M. A. Goberna, andM. A. Lopez. LinearSemi-Infinite Optimization, Wiley,1998.

[6] M. A. Goberna, M. A. Lopez, and M. Todorov. Stability theory for linear inequality systems, SIAM Journal on Matrix Analysis and Applications,17:730-743, 1996.

[7] J .-B. Hiriart-Urruty, and C. Lemarechal, Convex Analysis and MinimizationAlgorithms. I, Springer-Verlag, 1993.

[8] A. J. Hoffman. Approximate solutions of systems of linear inequalities,Journal ofResearch ofthe National Bureau ofStandards, 49 :263-265, 1952.

[9] R. B. Holmes. A Course on Optimization and Best Approximation. LectureNotes in Mathematics 257, Springer-Verlag, 1972.

[10] D. Klatte, and W. Li. Asymptotic constraint qualifications and global errorbounds for convex inequalities, Mathematical Programming, 84A :137-160,1999 .

[11] W. Li. Abadie's constraint qualification, metric regularity and error boundsfor convex differentiable inequalities, SIAM Journal on Optimization, 7:966978, 1997.

[12] W. Li, C. Nahak, and I. Singer. Constraint qualifications for semi-infinitesystems of convex inequalities, SIAM Journal on Optimization, 11:31-52,2000.

[13] W. Li, and I. Singer. Global error bounds for convex multi functions andapplications, Mathematics ofOperations Research, 23:443-462, 1998.


[14] A. S. Lewis, andJ.-S. Pang. Error bounds for convex inequality systems. InJ.-P.Crouzeix , J.-E. Martfnez-Legaz and M. Volle, editors, Generalized Convexity. Generalized Monotonicity : Recent Results. pages 75-1 10, Kluwer,1998.

[15] J.-S. Pang. Error bounds in mathematical programming, MathematicalProgramming, 79A:299-332, 1997.

[16] R. R. Phelps. Convex Functions, Monotone Operators and Differentiability (2nd ed.), Lecture Notes in Mathematics 1364, Springer-Verlag, 1993.

[17] S. M. Robinson. An application of error bounds for convex programmingin a linear space, SIAM Journal on Control, 13:271-273, 1975.

[18] I. Singer. The Theory of Best Approximation and Functional Analysis,CBMS Regional Conference Series in Applied Mathematics 13, SIAM,1974.

Chapter 5

STABILITY OF THE FEASIBLESET MAPPING IN CONVEXSEMI-INFINITE PROGRAMMING

Marco A. Lopez! and Virginia N. Vera de Serio?1Department of Statistics and Operations Research. Faculty of Sciences. Alican te University.

Ora. San vicente de Raspeig sin. 0307J Alicante, Spain

2Faculty ofEconomic Sciences. Universidad Nacional de Cuyo. Centro Univers itario,

5500 Mendo za. Argentina

[email protected]. [email protected] .edu.ar

Abstract In this paper we approach the stability analysis of the feasible set mapping inconvex semi -infinite programming for an arbitrary index set. More precisely, weestablish its closedness and study the semicontinuity, in the sense of Berge, ofthis multivalued mapping. A certain metric is proposed in order to measure thedistance between nominal and perturbed problems. Since we do not require anystructure to the index set, our results cover the ordinary convex programmingproblem.

1 INTRODUCTION

Consider the convex semi-infinite programmin g problem (convex SIP, inbrief)

(P ) Inf g{x ) s.t. ft {x ) ~ 0, t E T ,

where all the involved functions, g and ft , t E T , are convex functions definedon the Euclidean space jRn that, for the sake of simplicity, we shall supposefinite-valued on the whole space. The constraint system of (F) will be represented by 0' ; i.e., 0' = { ft{x) ~ 0, t E T} . The solution set of 0' (also

101

M.A. Goberna and M.A. Lopez (eds.}, Semi-Infinite Programming. 101-120.© 2001 Kluwer Academic Publishers.


called feasible set of (P)) is denoted by F. The system (J is consistent if F isnon-empty.

If all the involved functions are linear; i.e., g{x) = c' x and ft (x) = a~x bt, t E T, with c, x, and at in ~n , and c' and a~ denoting the transposes of thecorresponding vectors, we obtain the linear SIP

(P) Inf c' x s.t. a~x::; bt , t E T,

where the constraint system is (J ={ a~x ::; bt , t E T}.

In different papers ([1], [2], [4], [5], etc.) the stability of the linear SIPproblem has been investigated in the case that T is an arbitrary (infinite) set,without any structure (so, the functions t f-7 at and t f-7 bt have no propertyat all). The approach followed in those references is based on the use of anextended metric between the so-called nominalproblem, (P), and the perturbedproblem

This metric, providing the uniform convergence topology on T, is given throughthe formula

where 11.1100

represents the Chebyshev norm. It gives rise to the maximum(supremum) of the perturbations, componentwise measured. The space of allthe linear SIP problems, with the same index set T, by means of this extendedmetric, becomes a metrizable space which, in fact, locally behaves as a completemetric space.

The present paper constitutes a convex counterpart of the theory developedin [2], [4] and [5]. The first obstacle we face is the definition of a convenientmetric in the convex setting. In Section 3 we propose a distance between theconvex problems (P) and (PI), with

{Pt} Inf gl{X) s.t. fl{x)::; 0, t E T,

which certainly extends (1.1) as far as

d{P, Pt} = max {8(g, gd,SUPtET o{ft, fl)} ,

and where the distance between the coupled functions (i.e., 9 and gl, fromone side, and it and Jl, from the other) yields the topology of the uniformconvergence of convex functions on compact sets.

The paper mainly focuses on the stability properties of the feasible set mapping. Although our approach is tipically convex in nature, occasionally we shall

STABILITY IN CONVEX SIP 103

rely on the linearization of the involved convex systems, obtained by means ofthe whole set of subgradients. The main section in the paper, namely Section4, provides a large list of characterizations of stability of (P) relative to theconsistency (i.e., the existence of feasible solutions), extending to the convexcontext results as [2, Theorem 3.1], [4, Theorem 3.1] and [1, Theorem 3.1]. Finally , some conditions given in [5], for the upper semicontinuity of the feasibleset mapping in linear SIP, are extended to the convex SIP.

2 PRELIMINARIES

Before stating some preliminary results on convex functions, we shall introduce the necessary notation, according to the Rockafellar style [8]. Given anon-empy set X of the Euclidean space W, by cony X we denote the convexhull of X. By dim X we represent the dimension of the affine hull of X, whichis represented by aff X. From the topological side, int X and c1 X representthe interior and the closure of X. respectively, whereas rint X is the symbolused for the relative interior. The Euclidean norm is symbolized by 11.11 , and Bis the corresponding open unit ball . Finally, Op denotes the null-vector in W,

and lim, should be interpreted as limr -+oo• By ~f) we represent the cone ofall the functions A : T 1---+ 114 such that At = 0 for all t E T except, maybe, fora finite subset of indices.

Let f be a convex function defined on ~n that is finite -valued; i.e., suchthat its effective domain, represented by dom(J) , is the whole space ~n. Inthis case there exists at least a subgradient at each point x E ~; i.e., a vectoru satisfying, for every y E ~n, f(y) ~ f(x) + u'(y - x). The set of allsubgradients of f at x, called subdifferential of f at x, is denoted by 8f(x).By 8f(~) we shall denote the range of 8f as a multivalued mapping; i.e.,8f(~n) := U{8f(x) I x E ~}.

The Fenchel conjugate of f is given by

r(u) := sup{u'x - f(x) Ix E IRn}.

dom(j) = ~ implies 1** = f, and

u E 8f(x) {::} x E 8r(u) {::} f(x) + j*(u) = u'x (2.1)

([8, Theorems 12.2 and 23.5]). The first equivalence above can be expressed as

8j(~n) = dom(8r) and 8j*(~n) = dom(8j). (2.2)

By means of 8 f (~n ) we get a linearization of the feasible set F of our convexSIP. In fact, it can easily be verified that F is also the solution set of the linearsystem

{u~x ~ u~y - ft(y) , (t , y) E T x ~n , Ut E 8ft(y)}.


Taking into account (2.1), this system can be reformulated as

(2.3)

Remark 2.1 The following alternative linearization of (T comes throughthe equality ft = it, t E T,

(2.4)

and it can be found, for instance, in ([3], §8.6). In fact, both linearizations, (TIJ

and (jL, are close from each other, according to the following relationship:

conv8f(lRn ) c dom(J*) c cl conv dj (R"). (2.5)

Actually, from Theorem 23.4 in [8] applied to 1*, and by (2.2), we may write

rintdom(f*) C dom(8j*) = 8f(lRn ) c dom(J*) c c1dom(J*). (2.6)

Since dom(J*) is convex, by taking closed convex hulls in (2.6), we get

cl conv dj'{R") = c1dom(J*) => dom(J*),

and the second inclusion in (2.5) is proved. From the central inclusion in(2.6), and taking convex hulls, we obtain the first inclusion in (2.5). In ourapproach we shall use linearization (TL, which has, in general, less constraintsthan (jL, making simpler most reasonings in the proofs. (TL is called standardlinear representation in [6], where another linear representation using a smallersubsystem is also considered, under certain special assumptions.

3 A DISTANCE BETWEEN CONVEX FUNCTIONS

The set of all the finite convex functions on jRn is endowed with the topologyof the uniform convergence on compact sets, which is obtained through thestandard distance 0 that we define below. In this complete metric space we alsohave that the convergence of {fT } to f is equivalent to the pointwise convergenceand to the epi-convergence of fT to f and, also, of f: to j*(see, for instance,Theorems 7.17 and 11.34 in [9]).

Let {Kp } bea sequence of compact sets in jRn such that K p C int K P+1 andIRn = U;l Kp- In particular, we could have considered K p = Pcl B,

Let f and 9 be two finite convex functions defined on jRn. Let Op be the(pseudo-)metric

Op (f ,g) := sup { If (x) - 9 (x) I},XEKp

for p = 1,2, . ... Then, let 8 be the distance defined through

8 (f ,g) := f ~ 8p (f ,g) .p=1 2P 1 + 8p (f , g)

(3.1)

Thi s metric gives the topology of the uniform convergence of convex functions on compact sets. In order to clarify its behavior, and for further references,we include the following technical lemmas which are known results.

Lemma 3.1 Let pE N ana let e > 0 be given. Then, there exists somep > 0 such that 8p (f , g) < c, f or any pair offinite convex function s I ana 9with 8 (f, g) < p.

Proof. Given c > 0 and p , let p > 0 such that Izi < 2Pp implies II':zI< c.

Then, IrtYl < 2Pp yields Iyl < e, because I':Z= Y when z = Tfy. Since

8 (f , g) 0, there exist pE N ana p > 0 such that8p (f , g) 0 be given and take p such that L i=p+1 2-1 < ~ . Choose

p > 0 such that Izi < p enta ils I I ~z l < ~ . Since 8d/,g) ::; 8p(J,g)

for i = 1,2, ... ,p, it follow s that 8p (f ,g) < P yields l ~iJ!t'J:g) < ~ , fori = 1,2, . .. , p oTherefore ,

{-- 1 8d/,g) ~ 18 (f ,g) ::;~ 2i 1 +8d/, ) + .LJ 2i < c.

1= 1 9 1= p+ 1

D

As an immediate corollary of these two lemmas, and since any compact set inlRn is contained in K p for p large enough, we have the following key property.

Proposition 3.1 Let In r = 1,2, .. . , ana I befin ite convex functions onlRn . Then, 8 (fr , J) -+ 0 if ana only if {lr} converges uniformly to I on anycompact subset of lRn .

4 STABILITY PROPERTIES OF THE FEASIBLE SETMAPPING

Once we have introduced, through (3. 1), a distance, 8(. , .}, in the space ofall the finite convex functions defined on lRn , we naturally define an associated

106 SEMI-INFlN17E PROGRAMMING. RECENT ADVANCES

extended distance in the space of all the convex SIP problems in IRn , whoseconstraint systems are indexed by the same set T. We propose the formula

d(P,Pd = max {8(g ,gd , SUPtET 8(ft , fl)} ,

where (P) and (Pd are the problems introduced in Section 1. In this way, ourproblem space becomes a metrizable space, with the same properties as in thelinear setting.

Since this section focuses on the stability of the solution set, it makes senseto consider, as parameter space, the set 8 of all the convex systems, in IRn ,

indexed by T. Thus, we shall neglect the objective function as far as it has noinfuence on the feasible set. In this way, if (7 = {ft(x) ~ 0, t E T} and(71 = {fl (x) ~ 0, t E T}, the (restricted) distance between these two systemswill be given by

(4.1)

(8 ,d) is a complete metrizable space. The convergence, in this space, (7T t--+(7, describes the uniform convergence, on T, of the associated functions f[.) (x)(considered as functions of t, and for every fixed x) to f(.)(x). Certainly, if(7T := {f[(x) ~ 0, t E T} converges to (7 = {ft(x) ~ 0, t E T} in (8, d),for each p > 0 there will exist an associated rp such that d( (7, (7T) 0 and any fixed point x, if p is such that x E K p , weapply Lemma 3.1 to conclude the existence of Px > 0 such that 8p(ft , fn < cwhenever 8(ft , fn < Px· Thus we get If [( x ) - ft(x) 1< s, if r ~ rpz; and forall t E T.

For the sake of completeness, we recall here the definitions of some wellknown concepts in the theory of multi valued functions. Let y and Z be topological spaces, and consider a set-valued mapping S : Y =1 Z. We say that Sis lower semicontinuous (lsc, in brief), in the Berge sense, at y E Y if, for eachopen set W C Z such that W n S(y) -# 0, there exist s an open set U C Y,containing y, such that W n S(y1) -# 0 for each y1 E U.

S is said to be upper semicontinuous (usc , in brief), again in the Berge sense,at y E Y if, for each open set W C Z such that S (y) C W, there exists an openneighborhood of yin Y, U, such that S(y1) C W for every y1 E U.

If both spaces verify the first axiom of countability, S is closed at y E dom Sif for all sequences {yT} C Yand {ZT} C Z satisfying limTyT = y, lim-z" = zand ZT E S(yT), one has z E S(y). We also say that S is closed on domS ifitis closed at every y E dom S.

Tue feasible set mapping, :F : 8 =1 IRn , assigns to each parameter (7 E 8its (possibly empty) solution set F. Obviously, dom:F is the subset of all the

consistent systems in 8 , which will be represented by 8 e. Throughout thissection we analyze the stability properties of F , starting with the closedness.

Proposition 4.1 F is closed on (ee, d).

Proof. Let us consider sequences {aT} C 8 e and [z"] C ~n such that a; --+a E 8 e , xT --+ X and xT E F (aT) == FT, r = 1,2, .... We have to prove thatx E F(a) == F .

Consider t E T fixed . Since x" EFT' one has f[ (z") ::; O. Now, let p besuch that K p contains {XT} . Then,

If[ (z") - ft (x) 1::; If[ (z") - ft (xT)1+ 1ft (z") - ft (x) 1

::; r5p (Jr, fd + 1ft (xT) - ft (x)1 ,

for r = 1,2,... . Hence, the facts that d (an a) --+ 0, xT --+ x, and thecontinuity of ft, all together, and applying Lemma 3.1, yield ft (x) = lim,f[ [z") ::; O. By moving t in T , it follows that x E F. Since a E 8 e has beenarbitrarily chosen, we conclude the closedness of F on (8 e, d). 0

The following theorem provides differently many characterizations of thelower semicontinuity ofF at a E 8 e . First, we state some definitions, extendedfrom the linear case.

The system a satisfies the strong Slater condition if there exist some x andsome p > 0 such that ft (x) ::; -p, for all t E T. Then, x is called a strongSlater (S5) point of a .

Consider the functional J : ~n --+ ~T, such that J (x) := fu (x), the latter

being a function of t. Here, the space ~T is endowed with the topology ofthe uniform convergence given by the sup norm, 11911

00:= SUPtET 19 (t)l . The

system a is regular if

(4.2)

where OT is the null function in ~T (see [7]). It follows immediately that (4 .2)is equivalent to the exi stence of some positive e such that whenever 11911

00< e,

the system

a l := {It (x) ::; 9t, t E T}

is consistent.The feasible set mapping F is said to be metrically regular at a if, for each

x E F , there exists a pair of positive real numbers, e and f3, such that

d (x ,Fd ::; f3 max [0,SUPtET fl (x)] , (4 .3)

for any system al = {Il (x) ::; 0, t E T} with d (a,ad < c, and whered(x,Fd = +00 if F, = 0.

:F is dimensionally (topologically) stable at (J" if there exists some c > 0such that dim (Ft} = dim (F) (FI homeomorphic to F, respectively) wheneverd ((J", (J"t} < c.

The following lemmas show that some of these properties are held at aconvex semi-infinite system (J" if and only if they apply for the correspondinglinearization (J"L .

Lemma 4.1 The convex system (J" and its corresponding linearization (J"L,

possess the same strong Slater points.

Proof. Let x be a strong Slaterpoint of (J" and let p > 0 be such that ft (x) ::::; - p,for all t E T . From the definition of the Fenchel conjugate, and for anyUt E 8Id}Rn) ,

It (Ut) 2: u~x - It (x) 2: u~x + p,

so, x is a strong Slater point of (J"L. For the converse, suppose that x is a strongSlater point of (J"L; i.e., u~x ::::; It (ud - p for some positive p and for allUt E 8ft (}Rn) , t E T. For each t E T, take some Ut E 8It (x). Then

It (tid + It (x) = u~x,

which gives

i.c., x is a strong Slater point of (J". 0

Lemma 4.2 The convex system (J" is regular ifand only if its correspondinglinearization (J"L is regular.

Proof. Suppose that (J"L is regular and let 9 (.) be any function in }RT . Considerthe perturbed system (J"I = {It (x) - 9t ::::; 0, t E T} . Here, Jl (x) = It (x) 9t and so 8Il (x) = 8It (x) and (Il)* (Ut) = It (ud +9t, for all x E }Rn andUt E 81l (}Rn) = 8It (}Rn) . Therefore, the linearization systems (J"L and (J"f'are

(J"L = {u~x ::::; It (ud, Ut E 8It (}Rn) , t E T} ,(J"f = {u~x::::; It (ud + 9t, Ut E 8It (}Rn) , t E T} .

The assumption that (J"L is regular gives that the latter system is consistent aslong as 119 (')11

00is small enough, which implies the consistency of (J"I . Hence

(J" is regular.On the other hand, suppose that (J" is regular and let b (', .) be any function

in }RTf- , where T D = {(t ,Ut), Ut E 8 It (}Rn) , t E T} is the index set of (J"L .

Consider the perturbed linear system

0- = {u~x::::; It (Ut) + b(t,ut) , Ut E 8ft (}Rn) , t E T} .

For each t in T, put

It is clear that IIg (-) 11 00 :S lib (',.) 11 00 , If lib (',.)11 00 is small enough, theregularity of 0" implies that the perturbed convex system 0"1 = {It (x) - gt :S 0,t E T} is consistent, and so is its linearization 0"[' = {u~x :S it (Ut) + gt,Ut E 8It (JR71' ) , t E T} . But F (O"f) c F ((j) and hence (j is consistent. Thisshows that O"L is regular as well. 0

Associated with O"L, two sets have to be considered. The first one is the set,in lRn ,

c:= U 8ft(lRn) ,

tET

whereas the second is the set, in lRn+! , formed by the coefficient vectors of allthe linear inequalities in O"L :

~ = tlJ{( I;(~t) ) , Ut E 8ft (lRn)}.

Theorem 4.1 Let 0" = { It (x) :S 0, t E T} E 8 c. Then, the followingstatements are equivalent:(i) F is lsc at 0".

(ii) 0" E int 8 c.

(iii) 0" is regular.(iv) On+! f/- cl conv D,(v) 0" satisfies the strong Slater condition.(vi) F is metrically regular at 0".

(vii) F = cl (Fss) , where Fss is the set of the strong Slater points of0".

(viii) For any sequence {O"r} C 8 converging to 0", there exists ro such thata- E 8 c ifr 2:: ro and. furthermore, F = limr 2:ro Fr , in the sense ofPainleviKuratowski.(ix) F is dimensionally stable at 0".

IfF is bounded, then the following statement can also be added to the list:(x) F is topologically stable at 0".

Proof. The proof will be accomplished in several steps.(i) '* (ii). It follows immediately from the definition of lower semi

continuity.(ii) '* (iii) . Lett: > 0 be such that 0"1 E 8 c whenever d (a,ad < c.

Apply Lemma 3.2 to get some p and some positive p such that op (h, h1) < P

implies 0 (h, h1 ) < e, for any convex functions hand h1. Let 9 (.) be any

function in ]RT with 119 (')1100

< p, and consider the perturbed system crl =Ui (x) :s; 0, t E T}, where Jl (x) = it (x) - 9t· Then, 8p (ft,in = 19t1 <p, for any t E T and, as a consequence, d (cr, crt} = SUPtET 8 (ft ,Jl) < c.Hence, crl is consistent and, then, a is regular.

(iii) ¢=::} (iv) ¢=::} (v). For linear systems, properties (iii) ,(iv) and(v) are known to be equivalent [4]. In view of the above Lemmas 4.1 and 4.2,and the fact that condition (iv) reads the same for the convex system a andits corresponding linearization crL , we also have these equivalences for convexsystems.

(v) ::::} (i). Let x be a strong Slater point of a associated with somep > 0. Let W be any open set with W n F =I- 0. There is no loss of generalityin assuming that x E W n F; otherwise we may replace it by some point of theform z = (1 -).) y + ).X, for some yEW n F and), E]O, 1] small enough.(Notice that z is also strong Slater because of the convexity of each ft). Lemma3.1 gives

lil (x) - it (x)1 < p, t E T,

for any system crlfollows that

Ul (x) :s; 0, t E T} with d (crl' o ) small enough. It

il (x) :s; it (x) + P :s; -p + P = 0,

which shows that x E W n Fl . Therefore, :F is lsc at cr.Observe that we have already shown the equivalence of the statements from

(i) to (v). This equivalence is used in the remaining steps of the proof.(i) - (v) ::::} (vi) . It is similar to the proof in the linear setting (see [3]),

but we include its sketch here in order to emphasize the main differences.Since (ii) holds at a, F l == F(o I) =I- 0 for every crl close enough to a, and

we can take Xl E F I such that d(x, F l ) := minxEFI IIx - xII = IIx - xlii. Weshall assume that x=I- Xl. (Otherwise the inequality (4.3) is trivially satisfied.)

It is straightforward that the linear inequality (x - xl)'X :s; (x - xl)'xl issatisfied by any point x in F l and, so, it is a consequent inequality of crf. Thus,we can apply the generalized Farkas' Theorem (see, for instance, [3, Corollary

3.1.2]) and conclude the existence of sequences {).r} C ]R~Cl) and {ILr} C lR.tsuch that

) = li:n { L L ).~l ( .u y~(un )+ ILr ( °t )}.tET ul E8 If (IRn)

(4.4)

Multiplying both sides by ( !1 ) one gets

Since

(un'X - Ul)*(u;) ~ Il(x) ~ SUPtET Il (X),

we obtain

(4.5)

where

We are assuming that (V) also holds and, so, there will exist x and p > °verifying It (x) ~ -p, for all t E T. Lemma 3.1 allows us to assert that

III (x) - It (x)1 < p/2, t E T,

for any system (Tl = {Il (x) ~ 0, t E T} with d (aI, (T) small enough. Itfollows that

Il (x) ~ It (x) + ~ ~ -p + ~ = -~, t E T;

i.e., x is still an SS point for the convex semi-infinite systems in a certainneighbourhood of (T.

Now, repeating the reasoning followed in [3, Theorem 6.1], we get

d (x, Ft} < (3 SUPtET Il (x),

with

(vi) =? (ii) . Pick any x E F and, associated with this point, choosesome positive (3 and £1 such that

d(x,Fl ) ~ (3 max [o,sUP/l (x)],tET

(4.6)

for any system 0-1 = Ul (x)::; 0, t E T} with d(o-,o-t} ::; €l. Now, Lemma3.1 provides a positive €2 such that

fl (x) ::; fdx) + 1 ::; 1,

for any t E T, as long as d (0-, o-t} ::; €2. Therefore, if d (0-, o-t} ::; min(€l' €2)then the right hand side in (4.6) is finite, which implies that F, i- 0 (otherwised (x, Ft} = (0). Hence 0-1 E 8 c·

(i) ::::} (vii). It is clear that c1Fss c F, because Fss c F and Fis closed. Suppose that there exists Xl E F\ c1 Fss and take an open setW such that Xl E Wand W n c1 Fss = 0. Since F is lsc at 0-, there issome positive e such that W n F, i- 0 for any o-l with d (0-, o-t} ::; e. Let0-1 = {It (X) + E ::; 0, t E T}; it is clear that F l C Fss . Since d (0-, 0-1) = €,it also follows that W n F; i- 0, contradicting the fact that W n c1Fss = 0.Therefore c1Fss = F.

(vii) ::::} (v). It is immediate because 0 i- F = c1Fss.(i) ¢::::::> (viii) . Assume that (i) holds and that {o-r} is a sequence of

systems in 8 converging to 0-. Since 0- E int 8 c (the equivalence between (i)and (ii) has already been established), there must exist ro such that F; i- 0,for every r ~ roo Then, the lower semicontinuity of F at 0- entails F C

liminfr 2:ro F; (see, for instance , [9, 5B]). Moreover, limsuPr>ro F; C F, asa consequence of the closedness of F. Finally, and by virtue-of the generalinclusion liminfr2:ro F; C limsuPr>ro Fr , we shall have limr2:ro F; = F.

Conversely, (viii) entails liminfO'~(T F(O') = F(o-). Thus, F is lsc at 0- ([9,

5B]).(i) - (viii) ::::} (ix). Because of (vii), we have

oi- rintF = rint(c1Fss) = rintFs s · (4.7)

Let us consider the sup-function f := SUPtET ft· Obviously, f is a properclosed convex function, and F = {x E ~n I f(x) ::; O}. If xE rint Fe Fs s,and its associated scalar is p, we have f(x) ::; -p < O. Then, Theorem 7.6 of[8] applies to conclude that aff F = aff dom(J) . Since f is continuous relativeto rint dom(J), in particular f will be continuous at x, and there will existJ.L > 0 such that

f(x) < -~, for all x E (x + J.Llffi) n aff F.

Applying Lemma 3.1, with p chosen in such a way that guarantees the inclusion (x+J.Llffi)naff Fe K p , we can ensure that Ifl (x) - ft (x)! < p/2, for allt E T ,everyx E (x+J.Llffi)naff F,andeverysystemo-l = {Jl (x)::; O,t E T}with d (0-1, a) small enough . Now we have, for all t E T,

fl(x) ::; fdx) + ~ ::; -~ + ~ = 0, if x E (x + J.Llffi) n aff F ,

and, therefore, dim F ::; dim Fl .Finally, we prove that the inclusion aff F c aff F I cannot be strict. Other

wise, there would exist a point

Xl E F I\ aff F .

Then , it must be xl fj. dom(f). If this were not the case, we would applyCorollary 7.5.1 in [8] to write

limf((l - A)XI + AX) = f(x) ::; -p,Atl

which is a contradiction because (1 - A)XI + AX fj. aff F, for every A E [0,1[and, consequently, f((l - A)XI + AX) > O.

Applying once again Lemma 3.1, with p chosen in such a way that xl E K p ,

we can ensure that Ifl (xl) - ft (xl) I < 1, for all t E T and every system0"1 = Ul (x) ::; 0, t E T} sufficiently close to 0". Now we have ft(x l ) <fl (xl) + 1, for all t E T and, so, one gets the following contradiction

+00 = f (Xl) = sup ft(x l) ::; sup fl (Xl) + 1 ::; 1.

tET tET

(ix) ::::} (ii). If F I and F have the same dimension, then F I i- 0whenever F is not void.

(x) ::::} (ix) . It follows immediately because the dimension is a topological invariant.

(ix) ::::} (x) , under the boundedness of F. In this case, the topological stability requires the boundedness of F I for every system 0"1 in a certainneighborhood of 0" (taken small enough to guarantee the same dimension forthe feasible sets corresponding to all the systems in the neighborhood). Thus ,if:F fails to be topologically stable at 0", a sequence {O"r} can be found suchthat lim, a- = 0" and with F; unbounded, for all r E N. But this is impossiblebecause (viii) ensures that F = limr~ro Fr , for a certain ro, and this leads usto the boundedness of the sets F; beyond a certain rl ~ ro (see, for instance,[9, Corollary 4.12]) . 0

Remark 4.1 In relation to the proof of (i) - (viii) => (ix), we do not have,in general, the inclusion Fss c rint(F), as the following example shows:

Example 4.1 Let us consider the system, in lR,

0" := {ft(x) := t Ixl - t - 1 ::; 0, t E R, }.

It can easily be verified that F = Fss = dom(f) = [-1, +1], where fSUPtET ft . Consequently, f is not continuous at the points -1 and +1, despitethat these points are 55 for 0". This is the reason for taking x E rint F in thispart of the proof.

114 SEMI-INFlN17E PROGRAMMING. RECENTADVANCES

Remark 4.2 In the linear case , if F is unbounded and On f/. c1 conv C,condition (x) is again equivalent to the remaining nine conditions ([2]). Thefollowing example shows that, in the convex setting, this is not true:

Example 4.2 The index set T is a singleton and CT := {i(x) == -x ~ O}.Obviously F = Rt is unbounded, any non-zero solution is an SS clement, ando f/. c1 conv C = {-I}. However, if we consider the sequence CTr := {ir (x) ~O}, where

I () {- x , if a: ~ r,

r x := x _ 2r if x > r, , T = 1,2, ... ,

it is obvious that the sequence {ir } converges pointwiscly to I and, so, lim, CTr =CT. Despite this fact, the associated feasible sets F; are bounded and, accordingly, not homeomorphic to F.

The analysis of the upper semicontinuity of:F starts with the following result,previously established for the linear SIP problem in ([5, Theorem 3.1]). Theproof, in the convex setting, follows the same reasoning, but now the closednessof :F straightforwardly yields the conclusion in the second part of the proof,avoiding any representation of Fr.

Theorem 4.2 Given CT = { It (x) ~ 0, t E T} E eo :F is usc at CT ifandonly ifthere exist two positive scalars, E and p, such that

F1\pc1B c F\pc1B,

for every CTI E e such that d (CT, CTl) < c.

The following results emphasize the consequences that the boundedness ofthe feasible set has in relation to its stability.

Lemma 4.3 If the solution set ofa consistent system CT is bounded. then :Fis uniformly bounded in some neighborhood ofCT.

Proof. Suppose that F C Pc1 B, for some positive p, and assume the existenceof sequences {CTd c e and {zk} C lRn such that d(CT,CTk) < 11k, zk E Fkand Ilzk II > k. Without loss of generality assume that

zk3Pllzkll-tz,

as k -t 00, and observe that IIzll = 3p. If we consider

3pAk = IIzkll;

for k > 3p, one has 0 < Ak < 1, and Ak -t 0 as k -t 00. Taking some x E Fand defining tc'i i> Akzk+(l- Ak)x,itturnsoutthatwk -t z+x,ask -t 00.

The convexity of each function It gives

It ( Wk) ~ )..kIt (Zk) + (1 - )..k) If (x)

~(l-)..k)It(X) .

By letting k -+ 00, we get

Idz + x) ~ Idx)

Since x E F, it follows that It (z + x) ~ It (x) ~ 0, for all t E T. Thereforez +x E F, but

liz + xII ~ IIzll - IIxll ~ 3p - p = 2p,

which is a contradiction with the fact that F C Pcl B, So we may conclude theexistence of some k such that F 1 C k cl (B) for any system al that satisfiesd(a,at} < 11k. 0

As corollary of the results above we get the following proposition:

Proposition 4.2 If the solution set of a consistent system a is bounded,then :F is usc at a.

If a = {It (x) ~ 0, t E T} is a consistent system in R then :F is alwaysusc at a, since the argument in [5, Example 3.3] applies also here, as far asonly the convexity of the involved sets is considered. Therefore, the followingresults are established for systems in JRll, with n ~ 2, and they extend theirlinear versions given in [5].

Theorem 4.3 Let a = {It (x) ~ 0, t E T} be a consistent system injRn,n ~ 2, such that C is bounded and different from {On}. Then, :F is usc at a ifand only if F is bounded.

Proof. In view of the last proposition, we only need to show the direct statement.Let /1. > 0 such that C C /1. cl B, Suppose that F is unbounded. For any pair ofpositive scalars E and p, take z E (bdF)\pdIm and y ~ F such that lIyll > Pand liz - yll < c//1. (since C :1= {On}, we have F :1= Rn). For each t E T, letVt E 8ft(y), and consider the system a 1 = {Ii (x) ~ 0, t E T} , where

Il (x) := It (x) + v~ (z - y) .

Then,

for all p = 1,2,3, . .. , and so 0 (Jl, It) < c. Hence d (a, ad < e as well.The convexity of It yields

Il (y) = It (y) + v~ (z - y) ~ Idz) ~ 0,

which implies that y E Fl. Therefore the inclusion F l \p cl B c F\p cl B doesnot hold. So, we may conclude that:F is not usc at a. 0

The last result in the paper constitutes a necessary condition for the uppersemicontinuity of :F when C is unbounded. It requires the correspondingconvex version of the notion of implicit fixed constraint introduced in ([5, §2]).

Given a = {It (x) ~ 0, t E T} E ee, the linear inequality a/x ~ b, a f=On , is said to be an implicit fixed constraint for a if we can find sequences{tk} c T , {Uk} C Jl4.,,{O}, such that limj, Ok = 00 and, for every x E jRn,

(4.8)

(4 .9)

Obviously, the implicit fixed constraint a'x ~ b is satisfied by every y E F .

Theorem 4.4 Let a = {It (x) ~ 0, t E T} E ee.n ~ 2, such that F andC are both unbounded. If:F is usc at a, then there exists some positive p suchthat. for every z E (bd F) \p cl B, there is an implicit fixed constraint which isactive at z ,

Proof. Assume that :F is usc at a and lets and p, 0 < c < 1, p > 0, besuch that F1\p cl B c F\p cl B for every al E e such that d (a, ad ~ c.Take z E (bd F) \p ellB and a sequence {zk} such that zk fj. F, II zk II > p andlimj, zk = z , For each k, k = 1,2,3, ... ., let T k = {t E T : It (zk) > O},and define

k { It (x) , t E T\Tk ,

It (x) := It (x) - It (zk) + It (z), t E Tk.

Then

and so

{

0 t E T\Tk,o(It,If) = 'ft(zk)-ft(z) tEn.

l+ft(Zk)-ft(Z) ,

The perturbed systems ak := {It (x) ~ 0, t E T} satisfy zk E Fk , whichimplies that d (a , ak) > c. Therefore , for each k, k = 1,2, .. . , there is some


tk E Tk such that 6 u.; It) > E. From (4.9), after some algebraic manipulations, it follows that

and so ak := lIukli -+ 00. Without loss of generality suppose that ak > 0 andthat

Now, let yk in the segment [z, zk] be such that

and notice that yk -+ z: Observe that each Fenchel conjugate satisfies

s; (uk) = (uk)' zk - ftk (zk)= (uk)' zk + (uk)' (yk _zk)= (uk)' yk,

which gives

Moreover, we have, for every x E jRn,

and

liminf(ak)-lh, (x) 2: lim(ak)-l{(uk)'x - ftk(uk)}

k k

= a'x - o'»,

Therefore, a'x ::; a'z is an implicit fixed constraint which is active at z. 0

The following example illustrates the generation of these implicit fixed constraints.

Example 4.3 Let us consider the following convex system, in JR2,

U := {t IXI + X21 - t S; 0, t ~ I} ;

i.e., ft(x) := t IXI + X2! - t and T = [1 , +00[. Here F = {x E JR2 IIXI + x21 S; I}, C = {u E JR2 I UI = U2 E JR }, and both sets are unbounded.

If<TI = Ul(x) S; 0, t E T} and we apply Lemma 3.1, withp = E = 1, therewill exist p > 0 such thatch (ft, fl) < 1, for all t E T, provided that d(<T, «i) <p. Next, and for such a system Ul, we shall show that PI \ cl B CF\ cl B, andTheorem 4.2 yields the upper semicontinuity of :F at a. Here, we are takingK p = p cl B, p = 1,2, ...

Actually, if we assume the existence of Z E PI\F and IIzlI > 1, a contradiction will arise . Since z rJ. F, IZI + z21 > 1 and, without loss of generality,we shall consider Zl + Z2 > 1.

If A := IIzll- l , then Y := AZ = AZ + (1 - A)02 E cl B, Since A < 1 andZ E PI, the convexity of Jl entails

fl(Y) S; >'fl(z) + (1 - >')fl(02) S; (1 - A)fl(02), for all t E T. (4.10)

Associated with each t E T, and because 81(ft, Jl) < 1, there exist scalars atand (3t such that latl < 1, l{3tl < 1, and

fl(y) = ft(y) + at and fl(02) = h(02) + (3t . (4.11)

Since YI + Y2 > 0, (4.10) and (4.11) lead us to

t(Yl + Y2 - 1) + at S; (1 - A)(-t + (3t) .

Dividing by t and taking limits for t ~ +00, one gets

Yl + Y2 - 1 S; -(1 - A);

i.e., Yl + Y2 S; >. implies Zl + Z2 S; 1, yielding a contradiction.Once we have established that :F is usc at a, we prove that the necessary

condition given in Theorem 4.4 holds, for any arbitrary p > O.Given Z E (bd F) \pcl B, we have IZI + Z2! = 1. If, for instance, Zl + Z2 =

1, it is quite obvious that Xl + X2 S; 1 is an implicit fixed constraint active at Z.Actually, if we take tk = ak = k, k = 1,2, ... , obviously

(ak)-l ftk (x) = k- l h(x) = IXI + x21- 1 ~ Xl + X2 - 1, for all X E ~.

Analogously, if Zl+Z2 = -1 , the implicit fixed constraint will be -Xl -X2 S;1.

Remark 4.3 The condition given in Theorem 4.4 is not sufficient for theupper semicontinuity of:F at a given convex system a. The last example in thepaper shows this fact.

Example 4.4 Consider the following convex system, in]R2,

a := {ft(x) ~ 0, t E T},

where ft(x) := max{-Xl, txt} and T = [1, +00[. Here F = {x E ]R21Xl = O}, C = {u E lR2

1 UI E [-1, +00[, U2=0} , and again both sets areunbounded.

Let us choose z E bd F and take any arbitrary X E ]R2. With tk = Qk =k, k = 1,2, ... ,

and Xl ~ 0 is an implicit fixed constraint active at z,If UI = Ul(x) ~ 0, t E T}, with

il(x) := max{ -Xl - C:, txt},

for a certain E > 0, we observe that F I = {x E ]R2 IXl E [-c:,0] }, andthe condition in Theorem 4.2 never holds. On the other hand, taking K p =p c1 B, p = 1,2, ... , we observe that op(ft, il) = e, for all t E T and forp = 1,2, ..., provided that E < 1. Consequently, o(ft, in = c:j(l + s) < eand, therefore, dia, uI) < E.

The conclusion is that we can approach the system a by a system UI so closeas we want, whereas the condition in Theorem 4.2 fails. Hence, F is not usc atu.

Acknowledgment

The research of Marco A. Lopez was supported by the spanish DGES, grantPB98-0975.

References

[1] M. Canovas, M. Lopez, J. Parra and M. Todorov. Stability and wellposedness in linear semi-infinite programming, SIAM Journal on Optimization, 10:82-98, 1999.

[2] M. Gobema and M. Lopez. Topological stability of linear semi-infiniteinequality systems, Journal of Optimization Theory and its Applications,89:227-236, 1996.

[3] M. Gobema andM. Lopez. LinearSemi-Infinite Optimization, Wiley, 1998.

[4] M. Gobema, M. Lopez and M.Todorov. Stability theory for linear inequalitysystems, SIAM Journal on Matrix Analysis and Applications, 17:730-743,1996.


[5] M. Goberna, M. Lopez andM. Todorov. Stability theory for linear inequalitysystems II: Upper semicontinuity of the solution set mapping, SIAM Journalon Optimization, 7:1138-1151, 1997.

[6] W. Li, C. Nahak: and 1. Singer. Constraint qualifications for semi-infinitesystems of convex inequalities, SIAM Journal on Optimization, 11:31-52,2000.

[7] S. Robinson. Stability theory for systems of inequalities. Part I: Linearsystems, SIAM Journal on Numerical Analysis, 12:754-769, 1975.

[8] R. Rockafellar. Convex Analysis, Princeton University Press, 1970.

[9] R. Rockafellar and R. B. Wets. Variational Analysis, Springer-Verlag,1998.

Chapter 6

ON CONVEX LOWER LEVEL PROBLEMSIN GENERALIZED SEMI-INFINITE OPTIMIZATION

Jan-J. Riickmann' and Oliver Stein?ITechnische Universitiit Ilmenau, lnstitutfiir Mathematik , 98684 llmenau, Germany

2RWTH Aachen, Lehrstuhl C fur Mathematik, 52056 Aachen, Germany

[email protected] .de, [email protected] .de

Abstract We give an introduction to the derivation of topological and first order propertiesfor generalized semi-infinite optimization problems. We focus our attention onthe case of convex lower level problems where the main ideas of the proofs canbe illuminated without the technicalities that are necessary for the treatment ofthe general non-convex case , and where we can obtain stronger results.

After the description of the local topology of the feasible set around a feasibleboundary point we derive approximations for the first order tangent cones to thefeasible set and formulate appropriate constraint qualifications of MangasarianFromovi tz and Abadie type . Based upon these results we show that the first orderoptimality conditions given by Riickmann and Stein ([23]) for the case of linearlower level problems also hold in the jointly convex case . Moreover we prove thatthe set of lower level Kuhn-Tucker multipliers corresponding to a local minimizerhas to be a singleton when the defining functions are in general position.

1 INTRODUCTION

This article gives a brief introduction to the derivation of topological properties and optimality conditions for so-called generalized semi-infinite optimization problems. These problems have the form

with

(CSIP) : minimize f (x) subject to x E M

and

M {x E Rnl g(x,y) ~ 0, Y E Y(x)}

Y(x) = {y E R'"] Vf(X,y) ~ 0, £ E L} .\21

M.A . Goberna and M.A. Lopez (eds.), Semi-Infinite Programming, \2\-134.© 200\ Kluwer Academic Publishers.


All defining functions I, 9, Vi, f E L = {I, ... , s} , are assumed to be realvalued and at least continuously differentiable on their respective domains.

As opposed to a standard semi-infinite optimization problem SIP, the possibly infinite index set Y(x) of inequality constraints is z-dependent in a GSIP.For a survey about standard semi-infinite optimization we refer to [4],[6],[20],and to other contributions in the present volume.

Some engineering applications which give rise to generalized semi-infiniteoptimization problems are robot design ([3],[7]) , reverse Chebyshev approximation ([13]), the time-minimal heating or cooling of a ball ([14]), and designcentering ([18],[19]). In finite optimization with uncertainty about parametersy from a fixed set Y, the worst-case formulation of inequality constraints givesrise to a standard semi-infinite problem (cf., e.g., the Savage minimax-regretcriterion in decision theory [31]). However, if the parameter set Y (x) is statedependent, then the worst-case formulation takes the form of a GSIP. Also inparametric standard semi-infinite problems aspects of generalized semi-infiniteoptimization come into play, since the unfolding of the feasible sets in theparameter possesses the structure of a feasible set from GSIP (cf. [12]). Inparticular, this leads to the generic phenomenon of so-called trap-door points(cf. [24]).

The growing interest in GSIP over the recent years resulted in various contributions on the structure of the feasible set M ([11],[25],[26],[30]) and on firstand second order optimality conditions ([7],[11],[13],[21],[22],[28],[29],[30]).

In the present article we concentrate on the case of regular, convex lower levelproblems without equality constraints, where the derivation of results is notveiled by technicalities, but the main ideas become transparent. We emphasizethat in the literature cited above optimality conditions for GSIP for the mostgeneral case of non-convex and non-regular lower level problems, includingequality constraints, can be found. Moreover, similar optimality conditions fornon-smooth problems of more general type than the ones presented here can bederived along the lines of [17, Chapter 5.2].

This article is organized as follows. In Section 2 we study the local topologyof M around a feasible boundary point and motivate the appearance of stablere-entrant corner points and local non-closedness by a projection formula. Firstorder approximations of M and constraint qualifications are given in Section 3They provide the basis for first order optimality conditions which are presentedin Section 4. We conclude this article with some remarks in Section 5.

ON CONVEX LEVEL PROBLEMS IN GENERALIZED SIP 123

2 THE LOCAL TOPOLOGY OF M

Since optimality conditions are well-known for points from the topologicalinterior of M, throughout this article we focus our attention on a given feasibleboundary point of M, i.e. a point x E M n 8M, where 8M denotes the topological boundary of M . Furthermore, we fix U to be some open neighborhoodofx.

The n-parametric so-called lower level problem corresponding to GSIP isgiven by

Q(x) : maximize g(x, y) subject to y E Y(x) .

We call the problem Q(x) convex, if the functions -g(x,'), ve(x, .), f E L ,are convex on R'". The main assumption of the present article is:

Assumption 2.1 The lower level problems Q(x) are convexforall x E U.

Under Assumption 2.1 a set Y(x) with x E U is said to satisfy the Slatercondition if there exists y* such that ve(x, y*) < 0 for all f E L.

Assumption 2.2 The set Y(x) is bounded and satisfies the Slater condition.

Note that, by continuity of the defining functions of Y(x), Assumption 2.2guarantees that all sets Y(x), x E U, satisfy the Slater condition, possibly aftershrinking U.

Associated with Q(x) are its optimal value function

ep(x) = max g(x, y)YEY(X)

and its set of optimal solution points

Y*(x) = {y E Y(x) Iep(x) ~ g(x,y) } .

It is easily seen that M and the lower level set {x E R"] ep(x) ~ O} coincide. Topological properties of M have been derived from this descriptionby Stein ([26]), whereas directional derivatives of ip have first been used byRiickmann and Shapiro ([21]) in order to establish first order necessary optimality conditions for GSIP. This approach was further generalized to secondorder optimality conditions by Riickmann and Shapiro ([22]), and to first ordersufficient conditions by Stein and Still ([29]).

Under Assumptions 2.1 and 2.2 the sets Y*(x) are non-empty and uniformly bounded on U by Lemma 2 in [9], so that the optimal value functionep(x) = maxyEy*(x) g(x, y) is well-defined and even continuous on U (cf.[9]).

This immediately yields the following result on the local topology of M:

Proposition 2.1 Under Assumptions 2.1 and 2.2 the following assertionshold:

(i) Each x E U with rp(x) < 0 lies in the interior ofM.

(ii) It is rp(x) = O.

(iii) The set M is closed with respect to U.

Proposition 2.1(ii) explains the common notation Yo(x) for Y*(x) since itis Y*(x) = {y E Y(x) I g(x, y) = O}. We illustrate the local topologicalstructure of M with one positive and one negative example, taken from [27].

Example 2.1 (Re-entrant corner point) For x E R2 consider the index set

and put g(x, y) = -y. Then we obtain

M = {x E R21 y ~ 0, Y E Y(x)}

M

////;~

Figure 6.1 A re-entrant corner point

Figure 6.1 illustrates that M is closed around the feasible boundary pointx = O. Note that in the present example M is non-convex, although all definingfunctions are linear. More precisely, M exhibits a so-called re-entrant cornerpoint at the origin.

Example 2.2 (Local non-closedness) For x E R2 consider the index set

and put again g(x, y) = -y. It is easily seen that now

M {x E R21 y ~ 0, Y E Y(x)}

= {x E R21 Xl ~ X2 , Xl ~ O} U {x E R21 Xl > X2} .

ON CONVEX LEVEL PROBLEMS IN GENERALIZED SIP 125X2

M

Figure 6.2 Local non-c1osedness

As depicted in Figure 6.2, M is the union of an open with a closed halfspace, although all defining inequalities are non-strict. This shows that Proposition 2.1(iii) does not have to hold if Y (i;) does not satisfy the Slater condition.

We remark that Y(x) = 0 for Xl > X2 in the preceding example, whereAssumption 2.2 is violated. For general GSIP, where neither Assumption 2.1nor 2.2 necessarily holds, each point X from the complement of the domain of theset-valued mapping Y is feasible. Losely speaking, this is due to the 'absenceof constraints' at z , and formally it is consistent with the usual agreementcp(x) = maxg g(x, y) = -00. However, even for finite-valued ip local nonclosedness of the feasible set can occur, as an example in [26] shows. There,only one of several components of Y(x) becomes void under perturbations ofx , so that Y stays non-void while failing to be lower semi-continuous.

It is well-known that re-entrant comer points as in Example 2.1 can also beachieved in finite optimization, even with a single Coo inequality constraintfunction . There, however, the local disjunctive structure of the feasible setis destroyed under small perturbations of the defining function (cf. [1]). Incontrast to this, in [26] it is shown that re-entrant comer points are stable inGSIP. Even the local non-closedness phenomenon, which does not have anyanalogue in finite or standard semi-infinite programming, is stable in GSIP (cf.[26]).

Instead of going into the details of this stability proof from [26], here we givea brief geometrical motivation along the lines of [23]. For the feasible set Mof an arbitrary GSIP, the following simple observation provides basic insightinto its topological features. In the product space R" x R?' define the sets

g {(x,y) ERn x Rffil g(x,y) ~ O},

Y ((x,y) ERn x Rffi\ Vt(x,y) ~ 0, l E L},

let AC denote the set complement of A, and denote by prxA the orthogonalprojection of A c R" x Rm to the space R". Note that Y is the graph of theset-valued mapping Y. The following lemma is a direct consequence of thedefinition of M :

Lemma 2.1 (cf. [23]) M = [prx(Y n gC)]C .

Assume for the moment that all functions 9 and Vi, .e E L, are affine-linearwith respect to (x, y). Then the set y n gc is the intersection of the closedpolyhedron Y with the open halfspace gc. Hence, the set prx(Y n gC) is theintersection of finitely many closed and open halfspaces, and its set complement M turns out to be the union of finitely many closed and open halfspaces .Therefore, re-entrant corner points and local non-closedness appear to be nondegenerate phenomena in generalized semi-infinite optimization problems. Fora precise treatment of this question we refer to [26].

3 A LOCAL FIRST ORDER DESCRIPTION OF M

In order to derive first order optimality conditions for GSIP it is necessaryto give local first order approximations of its feasible set. Following [16],we define the contingent cone r*(a, A) to a set A at a point a as follows:d E r*(a, A) if and only if there exist sequences (ti')lIEN and (£llI)lIEN suchthat

Moreover, we define the inner tangent cone I'(c, A) to A at a as: dE I'(c, A)if and only if there exist some T> 0 and a neighborhood D of d such that

a + t d E A for all t E (0, f), d ED.

Note that the cones r(x,M) and r*(x, M) are open and closed, respectively,and that r(x, M) c r*(x, M). In order to find explicit expressions for thetangent cones, or at least for approximations of them, we introduce the upperand lower directional derivatives of sp at a in direction d in the Hadamard sense.For a with Icp(a) I < +00 we put

1. cp(a + td) - cp(a)

cp~ (a, d) = irn sup ..:........:.----'--------'--'--'-t""O, d~J t

and

CP'... (a,d) 1· . f cp(a + td) - cp(a)

- 1m1n .t\,o, d~J t

sp is called directionally differentiable at a (in the Hadamard sense) if for eachdirection d it is cp'-t (a, d) = cp'- (a, d) . In this case we put

cp'(a, d) = limt\"O , d-td

cp(a + td) - cp(a)t

The following lemma is easily verified:

Lemma 3.1 For x E M n 8M it is

{d E R n I cp~ (x,d) < O} C r(s, M)

c r*(x,M)c {d E Rnl cp~(x, d) ::; O} .

With the Lagrangian

£(x,y,,) = g(x,y) - 'L,iW(X,y)iEL

we now define for y E Yo(x) the set of Kuhn-Tucker multipliers

KT(x) = {, ERBI, 2: 0, Dy£(x,y,,) = 0 , £(x,y,,) = O},

where DaF denotes the row vector consisting of the partial derivatives of afunction F with respect to the variable a. Note that in the definition of KT(x)the vanishing of the Lagrangian reduces to the usual complementarity condition,due to the fact that g(x, y) = 0 for y E Yo(x). Moreover, KT(x) does notdepend on y in the convex case (cf. e.g. [5]).

Due to the Slater condition the set KT(x) is a non-empty polytope. ByV(KT(x)) we denote the vertex set of KT(x); i.e., the set of points in KT(x)where the gradients of s of the active equality and inequality constraints arelinearly independent.

In the sequel we will use the following assumption:

Assumption 3.1 For each, E KT(x) the junction Dx£(x, y,,) is constant on Yo(x) .

Remarkably, Assumption 3.1 is already satisfied if the functions -g, Vi,

f E L, are jointly convex on U x R'", due to a result by Hogan ([9]).

The next theorem is essentially taken from [9]. Note that the last equality inpart (ii) is due to the vertex theorem of linear programming.

Theorem 3.1 Under Assumptions 2.1 and 2.2 the following assertionshold:

(i) The optimal value function ip is directionally differentiable at x with

cp'(x,d) max min Dx£(x, y ,,) dyEYo(x) 1'EKT(x)

min max Dx£(x, y,,) d1'EKT(x) yEYo(x)

jor all dE Rn.

(ii) IfAssumption 3.1 holds', then ip is directionally differentiable at x with

cp'(x, d) = min Dx£(x, y,,) d = min Dx£(x, y,,) d1'EKT(x) 1'EV(KT(x))

jor all d ERn, where y E Yo(x) is arbitrary.

The combination of Lemma 3.1 and Theorem 3.1 immediately yields thefollowing chain of inclusions:

Corollary 3.1.1 Let Assumptions 2.1 and 2.2 be satisfied.

(i) The following chain of inclusions holds:

u n {d E Rnl Dx£(x,y,,)d < O}1'EKT(x) yEYo(x)

{d E Rnl min max Dx£(x, y,,) d < O}1'EKT(x) yEYo(x)

c r(x,M)

c r*(x,M)

c {d E Rnl min max Dx£(x,y,,)d ~ O}1'EKT(x) yEYo(x)

U n {d E Rnl Dx£(x,y,,) d ~ O} .1'EKT(x) yEYo(x)

(ii) IfAssumption 3.1 is satisfied, then thefollowing chain ofinclusions holdsjor arbitrary y E Yo(x) :

U {d E Rnl Dx£(x,y,,) d < O} =1'EV(KT(x))

- {d E Rnl min Dx£(x, y,,) d < O}1'EV(KT(x))

C r(x,M)

c r*(x,M)

c {dERnl min Dx£(x,y,,)d~O}1'EV(KT(x))

U {d E Rnl Dx£(x,y,,)d ~ O} .1'EV(KT(x))

Corollary 3.1.1 indicates why a disjunctive structure is inherent in generalized semi-infinite optimization problems . We emphasize that the inner andouter approximations of the tangent cones differ only by the strictness of aninequality. In the general case of non-convex lower level problems this is notnecessarily true (cf. [27],[28]). A critical point theory for disjunctive optimization problems has been developed in [10].

We complete this section with the definition of appropriate upper level constraint qualifications under the given assumptions. Since the well-known Mangasarian Fromovitz condition at a point x E Mfm, with Mfin being the feasibleset ofa finite optimization problem, states that the cone f(x, Mfin) is non-empty,Corollary 3.1.1 suggests to use the following generalizations of the Mangasarian Fromovitz constraint qualification for the GSIP under consideration. For amultiplier t E KT(x) we define the conditions MFI(t) and MF2('n by

M F I (t): There exists a dE R n such thatDx.c(x, y, t) d < 0 for all y E Yo(x),

M F2 (t): There exists a d ERn such thatDx.c(x, y, t) d < 0 for some y E Yo(x),

as well as

CQI : There exists atE KT(x) such that M FI(t) is satisfied,

CQ2 : There exists atE V(KT(x)) such that MF2(t) is satisfied,

Due to Corollary 3.1.1, CQI guarantees that f(x, M) is not void wheneverAssumptions 2.1 and 2.2 are satisfied. Under the additional Assumption 3.1,the weaker condition CQ2 is sufficient for r(x, M) to be non-empty.

The following proposition shows that, like in finite programming, the Mangasarian Fromovitz type constraint qualifications CQI and CQ2 imply Abadietype constraint qualifications. The proof of this result (cf. [28]) relies heavilyon the sub-additivity of the optimal value function directional derivative cp'(x, .)in d, which is known to hold under Assumptions 2.1 and 2.2. We point out thatanalogous results do not hold in the general non-convex case .

Proposition 3.1 (cf. [28]) Let Assumptions 2.1 and 2.2 be satisfied.

(i) IfCQI holds at ii, then it is

f*(x,M) = {d E Rnl min max D x.c(x,y,,)d::; O}.'YEKT(x) yEYo(x)

(ii) If Assumption 3.1 is satisfied and CQ2 holds at ii , then for arbitraryy E Yo(x) it is

f*(x, M) = {d E Rnl min Dx.c(x, y,,) d ::; O} .'YEV(KT(x))

4 FIRST ORDER OPTIMALITY CONDITIONS

Now we tum our attention to necessary optimality conditions of first orderfor GSIP which we will derive from the different approximations of the feasibleset by first order tangent cones in Section 3. For the formulation of sufficientfirst order conditions we refer to [28], [29].

Recall that x E M is called local minimizer for GSIP if f(x) ~ f(x) forall x E M n U, possibly after shrinking U. The following observation formsthe basis for our derivation of first order optimality conditions:

Lemma 4.1 (cf. e.g. [8]) Ifx is a local minimizer for GSIP, then

{d E Rnl Df(x)d < O} n r*(x ,M) = 0. (4.1)

Subsequently we will replace I" (x, M) in (4 .1) with its approximations fromSection 3 in order to obtain explicit results.

The following first order necessary optimality conditions generalize resultsof Riickmann and Stein ([23, Theorem 4.2]) from the linear to the convex case.Theyalso strengthen conditions that Jongen, Riickmann, and Stein ([11 ,Theorem 1.1]), Riickmann and Shapiro ([21, Proposition 3.1]) and Stein([28, Theorem 3.5]) gave under more general assumptions about the lower levelproblem. By the term 'non-trivial multipliers' we mean that not all multipliersare vanishing.

Theorem 4.1 Let x E M n 8M be a local minimizer of GSIP and letAssumptions 2.1 and 2.2 be satisfied. Then the following assertions hold:

(i) For every 1 E KT(x) there exist yi E Yo(x) and non-trivial multipliersK ~ 0, Ai ~ 0, i = 1, ... ,p, p ~ n + 1, all depending on 1. such that

p

KDf(x) + LAiDx£(x,yi,1) = O. (4.2)i=l

Foreach 1 E KT(x) such that M F1(1) holds at x one can choose K = 1and p ~ n in (4.2).

(ii) IfAssumption 3.1 is satisfied. then for every 1 E V(KT(x)) there existnon-trivial multipliers K ~ 0, A ~ 0 , both depending on 1. such thatfor arbitrary y E Yo(x)

KDf(x) + ADx£(x,y,1) = o. (4.3)

For each 1 E V(KT(x)) such that MF2(1) holds at x one can chooseK = 1 in (4.3).

The idea for the proof of the first assertion in part (i) is to combine Lemma 4.1and Corollary 3.1.1(i) to obtain for each 1 E KT(x) the inconsistency of the

inequality system

Df(x)d<O, sTd<O forall SE{D~L:(x,Y,i) IYEYo(X)}.

The assertion then follows from a Gordan type theorem of the alternative asgiven in [2]. The second assertion of part (i) is shown with a Farkas typetheorem (cf. e.g. [8]). Part (ii) is treated analogously. A detailed proof ofTheorem 4.1 can be given along the lines of [28].

Under our convexity assumptions the optimality conditions in Theorem 4.1are much stronger than the ones in the general non-convex case (cf. the discussions in [27, 28]). In particular we obtain a family of optimality conditions(parametrized by the corresponding Kuhn-Tucker multipliers) which is certainly non-empty in Theorem 4.1(i) and even finite in Theorem 4. 1(ii). This canbe of particular importance in the formulation of numerical algorithms for thesolution of GSIP.

We conclude this section with the observation that KT(x) has to be a singleton at a local minimizer X, if the defining functions of GSIP are in general position. For Y E Yo(x) we define the lower level active index setLo(x, y) = {£ ELI W(x , y) = O} as well as Lo(x) = nyEyo(x) Lo(x , y) .

Corollary 4.1.1 Let ii: E M n 8M be a local minimizer of GSIP, letAssumptions 2.1.2.2, and 3.1 be satisfied, and let M F2 (i ) be satisfied at ii foraUi E V(KT(x)). Furthermore. let D f(x) f= oand let the vectors D x 9(x, y) ,Dx vt{x, y), £ E Lo(x), be linearly independent for some y E Yo(x) . ThenKT(x) is a singleton.

Proof. Let,l,,2 E V(KT(x)). Since Df(x) does not vanish and MF2(,l),M F2(T2) hold, Theorem 4.1(ii) yields the existence of >'1 , >'2 > 0 such that

- - >'1Dx.c(x, y"l) = >'2 DxL:(x, y , ,2) .

AS,i vanishes for any, E KT(it) and £ E L \Lo(x), rearranging terms resultsin

(>'1 - >'2) D x 9(x, y) + L (>'2,1 - >'111) Dx Vi(X, y) = 0,iELo(x)

so that the linear independence of the involved vectors yields >'1 = >'2 and,1 = ,2 . This means that the polytope KT(x) possesses only one vertex, andthus shows the assertion. 0

We note that by a result of Kyparisis [15] KT(x) is a singleton if and onlyif the so-called strict Mangasarian-Fromovitz constraint qualification holds inthe lower level problem Q(x) .


5 FINAL REMARKS

In [28] it is shown that Corollary 3.1.1, Proposition 3.1, and Theorem 4.1hold in a generalized form when the Slater condition is not satisfied in the lowerlevel problem. In this case the set ofKuhn-Tucker multipliers has to be replacedby the set of corresponding Fritz John multipliers. This is remarkable since theexplicit formulas for the directional derivative r.p'(x, d) from Theorem 3.1 donot always hold in this situation. However, since in generalized semi-infiniteoptimization one is only interested in a level set of the function ip, and not inthe function itself, the results about tangent cones and first order optimalityconditions can still be generalized from the non-degenerate to the degeneratecase.

Acknowledgement

The authors wish to thank the referee for valuable comments and for pointingout the reference [17].

References

[1] Th. Brocker and L. Lander. Differentiable Germs and Catastrophes, London Mathematics Lecture Notes Serie 17, Cambridge University Press, 1975.

[2] E.W. Cheney. Introduction to Approximation Theory, McGraw-Hill, 1966.

[3] TJ. Graettinger and B.H. Krogh. The acceleration radius : a global performance measure for robotic manipulators, IEEE Journal ofRobotics andAutomation, 4:60-69, 1988.

[4] M.A. Goberna and M.A. Lopez. Linear Semi-Infinite Optimization, Wiley,1998.

[5] E.G. Gol'stein. Theory of Convex Programming, Translations of Mathematical Monographs, Vol. 36, American Mathematical Society, 1972.

[6] R. Hettich and K.O. Kortanek. Semi-infinite programming: theory, methods, and applications, SIAM Review, 35:380-429, 1993.

[7] R. Hettich and G. Still. Second order optimality conditions for generalizedsemi-infinite programming problems, Optimization, 34:195-211, 1995.

[8] R. Hettich and P. Zencke. Numerische Methoden der Approximation undsemi-infiniten Optimierung, Teubner, Stuttgart, 1982.

[9] W.W. Hogan. Directional derivatives for extremal value functions with applications to the completely convex case, Operations Research, 21 :188-209,1973.

[10] H.Th. Jongen, 1.-1. Riickmann, and O. Stein. Disjunctive optimization:critical point theory, Journal of Optimization Theory and Applications,93:321-336, 1997.

ON CONVEX LEVEL PROBL£MS IN GENERALIZED SIP 133

[11] H.Th. Jongen, J.-J. Riickmann, and O. Stein. Generalized semi -infiniteoptimization: a first order optimality condition and examples, MathematicalProgramming, 83:145-158, 1998.

[12] H.Th . Jongen and O. Stein. On generic one-parametric semi-infinite optimization, SIAM Journal on Optimization, 7:1103-1137, 1997.

[13] A. Kaplan and R. Tichatschke. On a class of terminal variational problems. In 1. Guddat, H.Th. Jongen, F. Nozicka, G. Still, F. Twilt, editors,Parametric Optimization and Related Topics IV, pages 185-199, Peter Lang,Frankfurt a.M ., 1997 .

[14] W. Krabs. On time-minimal heating or cooling ofa ball, Numerical Methods ofApproximation Theory, 8:121-131, 1987.

[15] J. Kyparisis. On uniqueness of Kuhn-Tucker multipliers in nonlinear programming, Mathematical Programming, 32:242-246, 1985.

[16] P.-J. Laurent. Approximation et Optimisation, Hermann, Paris, 1972.

[17] E. Levitin. Perturbation Theory in Mathematical Programming, Wiley,1994.

[18] V.H. Nguyen and J.J. Strodiot. Computing a global optimal solution to adesign centering problem, Mathematical Programming, 53:111-123, 1992.

[19] E. Polak. An implementable algorithm for the optimal design centering,tolerancing and tuning problem, Journal ofOptimization Theory and Applications, 37:45-67, 1982.

[20] R. Reemtsen and J.-J. Riickmann (editors). Semi-Infinite Programming,Kluwer, 1998.

[21] J.-J . Riickmann and A. Shapiro. First-order optimality conditions in generalized semi-infinite programming, Journal of Optimization Theory andApplications, 101:677 -691, 1999.

[22] J.-J. Riickmann and A. Shapiro. Second-order optimality conditions ingeneralized semi-infinite programming, submitted.

[23] J.-J. Riickmann and O. Stein. On linear and linearized generalized semiinfinite optimization problems, Annals ofOperations Research, to appear.

[24] O. Stein. Trap-doors in the solution set of semi-infinite optimization prob lems. In P. Gritzmann, R. Horst, E. Sachs, R. Tichatschke, editors, RecentAdvances in Optimization, pages 348-355, Springer, 1997.

[25] O. Stein. The Reduction Ansatz in absence of lower semi-continuity, in J.Guddat, R. Hirabayashi, H. Th . Jongen, F. Twill, editors, Parametric Optimization and Related Topics V, pages 165-178, Peter Lang, Frankfurt a.M;2000.

[26] O. Stein. On level sets of marginal functions, Optimization, 48:43-67,2000.


[27] O. Stein. The feasible set in generalized semi-infinite programming. InM. Lassonde, editor, Approximation, Optimization and Mathematical Economics, pages 309-327, Physica-Verlag, Heidelberg, 2001.

[28] O. Stein . First order optimality conditions for degenerate index sets in generalized semi-infinite programming, Mathematics of Operations Research,to appear.

[29] O. Stein and G. Still. On optimality conditions for generalized semiinfinite programming problems, Journal of Optimization Theory and Applications, 104:443-458, 2000.

[30] a.-w.Weber. Generalized Semi-Infinite Optimization andRelated Topics,Habilitation Thesis, Darmstadt University of Technology, 1999.

[31] D.J. White. Fundamentals ofDecision Theory, North Holland, 1976.

Chapter 7

ON DUALITY THEORY OFCONIC LINEAR PROBLEMS

Alexander ShapiroSchool of Industrial and Systems Engineering,

Georgia Institute ofTechnology, Atlanta, Georgia 30332-0205, USA

[email protected] .edu

Abstract In this paper we discuss duality theory of optimization problems with a linearobjective function and subject to linear constraints with cone inclusions, referredto as conic linear problems. We formulate the Lagrangian dual of a conic linearproblem and survey some results based on the conjugate duality approach, wherethe questions of "no duality gap" and existence of optimal solutions are relatedto properties of the corresponding optimal value function. We discuss in detailapplications of the abstract duality theory to the problem of moments, linearsemi-infinite, and continuous linear programming problems .

1 INTRODUCTION

In this paper we discuss duality theory of conic linear optimization problemsof the form

Min (c, x) subject to Ax + bE K,xEC

(1.1)

where X and Yare linear spaces (over real numbers), C C X and KeY areconvex cones, bEY, and A : X -+ Y is a linear mapping. We assume thatspaces X and Yare paired with some linear spaces X' and y I

, respectively, inthe sense that bilinear forms (".) : X' x X -+ JR and (" .) : Y' x Y -+ JR aredefined. In other words, for any x* E X' and x EX, we have that (x*, .) and(', x) are linear functionals on spaces X and X', respectively, and similarly forthe pair Y and v'.

Many interesting examples of optimization problems can be formulated inthe above framework. We discuss, in particular, applications of the general

135

M.A. Goberna and M.A. Lopez (eds.), Semi-Infinite Programming, 135-165 .© 2001 Kluwer Academic Publishers.


theory to the problem of moments, semi-infinite programming, and continuouslinear programming problems.

There are numerous studies devoted to duality theory of optimization problems. We adopt here an approach which is based on conjugate duality pioneeredby Rockafellar [20], [22]. The standard way how the conjugate duality is developed in an infinite dimensional setting is based on pairing of locally convextopological vector spaces ([4], [5], [22]). We outline some basic results of thattheory in the next section. In particular, we indicate where exactly topologicalassumptions about the involved spaces are essential.

This paper is organized as follows. In Section 2 we discuss duality theory ofabstract conic linear problems. Sections 3, 4 and 5 are devoted to applicationsof the general theory. In Section 3 we discuss the classical problem ofmoments.In the setting of the problem of moments the space X is an infinite dimensionalspace of measures while the space Y is finite dimensional. The correspondingdual can be written as a semi-infinite programming problem which we discussin section 4. Finally, in section 5 we study the so-called continuous linearprogramming problems, where both spaces X and Y are infinite dimensional.

Recall that a vector space Y is said to be a locally convex topological vectorspace if it is equipped with a Hausdorff (i.e ., satisfying the separation axiom)topology which is compatible with the algebraic operations of Y, and such thatany neighborhood of 0 E Y includes an open, convex, balanced and absorbingsubset of Y . It is said that two locally convex topological vector spaces Yand Y' are paired locally convex topological vector spaces if their topologiesare compatible with the corresponding bilinear form (., .); i.e., the set of linearcontinuous functionals on Y coincides with the set {{y*,.) : y* E Y'} and theset oflinear continuous functionals on Y' coincides with the set {(., y) : y E Y} .If Y is a Banach space, then we can equip Y either with its strong (i.e., norm)or with its weak topology, and pair Y with its standard dual space Y* (ofcontinuous linear functionals) equipped with the weak star (weak") topology.The interested reader can look in almost any standard text book on functionalanalysis (e.g., [11]) for a theory of locally convex topological vector spaces.

We use" Min" and" Max " to denote the respective minimization and maximization operators. Their appearance does not automatically imply that thecorresponding minimum or maximum is attained.

2 CONIC LINEAR PROBLEMS

In this section we discuss a duality theory of the conic linear problem (1.1).We associate with the cone C its polar (positive dual) cone

C*:= {x* EX': (x*,x) 2': 0, \Ix E C},

ON DUALITY THEORY OF CONIC LINEAR PROBLEMS 137

and similarly with the cone K we associate its polar cone K* C yl. Furthermore , the bipolar of the cone K is defined as

K** := {y E Y : (y*, y) 2': 0, Vy* E K*} .

Note that at this point we do not introduce any particular topology in the considered spaces. We only require that the dual space X' be large enough such thatthe adjoint mapping of A does exist. That is, we make the following assumptionthroughout the paper.

(AI) For any y* E y' there exists unique x* E X' such that (y*, Ax) = (z", x)for all x EX.

Then we can define the adjoint mapping A* : y' -T X' by means of theequation

(y*,Ax) = (A*y* ,x), "Ix E X.

Existence of z" = A*y* in the assumption (AI) means that the space X' issufficiently large to include all linear functionals of the form x H- (y*, Ax) .Uniqueness of x* means that any two points of X' can be separated by a linearfunctional (-, x), x E X. That is, for any x* E X' there exists x E X such that(x*, x) =I- O.

Consider the Lagrangian function of problem (1.1)

L(x, y*) := (c, x) + (y*, Ax + b),

and the following optimization problem

Min {1jJ(X):= sup L(x, y*)} .XEC Y*E-K*

(2.1)

Let us observe that the above problem (2.1) is equivalent to the problem

Min (c, x ) subject to Ax + bE K**. (2.2)xEC

Indeed. if Ax + b E K**, then (y*, Ax +b) ::; 0 for any y* E - K*, and hencethe maximum of (y* , Ax + b), over y* E - K*, is zero. Therefore, in that case1jJ(x) = (c,x) . If Ax + b f/. K**, then (y* ,Ax + b) > 0 for some y* E -K*and, hence, 1jJ(x) = +00.

Note that the inclusion K C K** always holds. Therefore, the optimal valueof the problem (2.2) is less than or equal to the optimal value of the problem(1.1).

By changing the Min and Max operators in (2.1) we can calculate the dualof problems (1.1) and (2.2). That is, consider the problem

Max {r.p(y*) := inf L(X,y*)}. (2.3)Y*E-K* xEC


Since L(x,y*) = (c + A*y* ,x) + (y*,b), we have that cp(y*) = (y*,b) ifc + A*y* E C*, and cp(y*) = -00 otherwise. Therefore, problem (2.3) isequivalent to the following problem

Max (y* ,b) subject to A*y* + c E C*.Y'E-K'

(2.4)

We denote problems (1.1) and (2.2) by (P) and (P'), respectively, and referto them as primal problems. Problem (2.4) is denoted by (D) and referred toas the dual problem. By val(P), va1(P') and va1(D) we denote their respective optimal values, and by Sol(P), Sol(P') and Sol(D) their sets of optimalsolutions.

Let us remark that one can construct a dual problem by introducing a Lagrange multiplier for the constraint x E C as well. That is, the dual problem isconstructed by minim izing L(x, z" , y*) over x EX, where

L(x , z", y*) := (c, x) + (z" , x) + (y* , Ax + b).

This leads to the dual problem

Max (y* , b) subject to A*y* + c + x* = O. (2.5)x' E _CoY' E -K'

Clearly the above dual problem (2.5) is equivalent to the dual problem (2.4).

The dual problem (2.4) is also a conic linear problem. Its dual can be writtenin the form (1.1) but with the cones C and K replaced by C** and K**, respectively, and with A replaced by A **, provided A ** does exist. Therefore, thereis a symmetry between problems (P) and (D) (i.e., the dual of (D) coincideswith (P)), ifC = C**, K = K** and A = A**.

The following weak duality relations hold

val(P) ~ val(P') ~ val(D). (2.6)

Indeed, we already mentioned that the inequality va1(P) ~ val(P') followsfrom the inclusion K C K**. The other inequality val(P') ~ va1(D) followsby the standard min-max duality . Recall that the min-max problem (2.1) isequivalent to the problem (P'), and not to the original problem (P). In the subsequent analysis we deal with problem (P), while problem (P') is introducedonly in order to demonstrate this point.

The weak duality (2.6) is also not difficult to show directly. Let x be afeasible point of the problem (P) (of the problem (P')) and y* be a feasiblepoints of the problem (D) . We have then

(c,x) ~ (c,x) + (y*,Ax+b) = (c+A*y*,x) + (y*,b) ~ (y*,b). (2.7)


Since the inequality (c,x) ~ (y*, b) holds for any feasible pair x and y*,it follows that val(P) ~ val(D). It also follows from (2.7) that the equality(c,x) = (y* , b) holds iff the following complementarity conditions are satisfied

(y*,Ax+b) =0 and (c+A*y*,x) =0. (2.8)

The above complementarity conditions simply mean that a feasible pair (x, y*)is a saddle point of the Lagrangian. We obtain the following result.

Proposition 2.1 There is flO duality gap between the primal and dual problems. i.e.• val(P) = val(D), and both problems have optimal solutions iff thereexists a feasible pair (x , y*) satisfying the complementarity conditions (2.8).

Now let us associate with problem (1.1) the optimal value function

v(y) := inf{ (c,x) : x E C, Ax +Y E K}. (2.9)

By the definition v(y) = +00if the corresponding feasible set is empty. Clearlyv(b) = val(P). Since the primal problem (P) is convex we have that the(extended real valued) function v(y) is convex ([22]). It is also not difficult tosee that v(y) is positively homogeneous; i.e., v(ty) = tv(y) for any t > 0 andy EY.

The conjugate of v(y) is defined as

v*(y*) := sup{(y* ,y) -v(y)}.yEY

Let us calculate the conjugate function v*(y*) . We have

(2.10)

v*(y*) SUp{(y*,y) - (c,x) : (x,y*) E X x Y* , x E C, Ax + Y E K}sUPXECsUPAx+YEK{(y*,y) - (c,x)}sUPXEC sUPYEK{ (y*, y - Ax) - (c, x)}sUPXEC sUPYEK{ (y* ,y) - (A*y* + c, x)} .

We obtain thatv*(y*) = 0 ify* E -K* andA*y*+c E C*, and v*(y*) = +00otherwise. That is, v* (y*) is the indicator function of the feasible set of thedual problem. Therefore, we can write the dual problem (D) as

Max {(y*,b) -v*(y*)}.y*EY*

It follows that val(D) = v**(b), where

v**(y):= sup {(y* ,y) - v*(y*)}y*EY*

(2.11)

(2.12)

is the biconjugate of v(y) .So far we did not use any topology in the considered spaces. We make the

following assumption in the consequent analysis .

(A2) The spaces Y and Y' are paired locally convex topological vector spaces.

All consequent topological statements are made with respect to the consideredtopologies of Y and Y' .

Recall that the domain of an extended real valued function f : Y -t JR isdefined as

domf := {y E Y: f(y) < +oo},

and its subdifferential is defined at a point y, where f(y) is finite valued, as

8f(y} := {y* E y': f(z) - f(y) ~ (y* ,z - y), Vz E Y} .

It immediately follows, from the definitions, that

y* E 8f(y) iff j*(y*) = (y*, y) - f(y)·

By applying that to the function j**, instead of f, we obtain that y* E 8 j** (y)iff j***(y*) + j**(y} = (y*,y) . Now by the Fenchel-Moreau Theorem wehave that j*** = j* . Consequently we obtain (cf., [5, Lemma 2.4, p.52], [22,p.35])

8j**(y) = arg max {(y*,y) - j*(y*)}.y*EY*

(2.13)

This leads to the following result ([5], [22]).

Proposition 2.2 The following statements hold: (i) val(D) = v**(b), (ii)ifval.(D) is finite, then Sol(D) = 8v**(b).

Proof. We already showed that val.(D) = v**(b). In order to prove assertion(ii) we have to show that fj* is a maximizer of the right hand side of (2.12), fory = b, iff fj* E 8v** (b). This follows immediately from (2.13). 0

Note that assertion (i) in the above proposition does not depend on the assumption (A2), while assertion (ii) is based on (2.13) which in tum involvestopological properties of the paired spaces Y and Y' .

Consider the feasible set <Jl* of the dual problem (2.4) ; i.e.,

<Jl* = {y* E Y* : y* E -K*, A*y* + c E C*} .

We see that val(D) = sUPY*E* (y*, b). Since val(D) = v**(b), it follows that

v**(y) = sup (y*,y) ,u: E*

i.c., v** (y) is the support function of the set <Jl*.

By lsc v we denote the lower semicontinuous hull of the function v. That is,lsc v is the supremum of all lower semicontinuous functions majorized by v;i.e.,

lscv(y) = min {V(y), liminfv(Z)}.z-ty

The problem (P) is said to be subconsistent if lsc v(b) < +00. Of course,if (P) is consistent (i.e., its feasible set is nonempty), then v(b) < +00, andhence lscv(b) < +00. Therefore, if (P) is consistent, then it is subconsistent.By cl v we denote the closure of the function v,

cl v(.) '= {ISCV(')' if lscv(y) > -00 for all y E Y,. -00, if lsc v(y) = -00 for at least one y E Y.

By the Fenchel-Moreau Theorem we know that v** = cl v, and we have thatcl v(b) = lscv(b) iflscv(b) < +00, [22, Theorem 5]. Therefore, we have thefollowing result:

Proposition 23 The folio wing statements hold: (i) val(D) = cl v(b), (ii)if(P) is subconsistent, then val(D) = lscv(b) .

The above proposition shows that if (P) is subconsistent, then there is noduality gap between the primal and dual problems iff the optimal value functionv(y) is lower semicontinuous at y = b. Note that v(y) is lower semicontinuousat every y E Y iff its epigraph epi v := {(y, a) : a ~ v(y)} is a closed subsetofY X JR.

It may be not easy to verify lower semicontinuity of v(y) directly. Therefore,we discuss now conditions ensuring the "no duality gap" property and existenceof optimal solutions.

Proposition 2.4 Suppose that the space X is a topological vector space,the functional (c, .) is lower semicontinuous, the linear mapping A : X -+ Yis continuous, the cones C and K are closed in the respective topologies, andthe following, so-called inf-compactncss, condition holds: there exist a E JRand a compact set SeX such that for every y in a neighborhood ofb the levelset

{x EX: (c, x) ~ a, x E C, Ax + Y E K} (2.14)

is contained in S, and for y = b this level set is nonempty. Then Sol(P) isnonempty and compact and the optimal value function v (y) is lower semicontinuous at y = b.

Proof. The inf-compactness condition implies that there is a level set of (P)which is nonempty and compact. Since the objective function (c,·) is lower


semicontinuous, it follows that Sol{P) is nonempty and compact, and henceval{P) is finite. We can take a = val{P) . Since (c, .) is lower semicontinuousand Sol{P) is compact, it follows that, for any E > 0 there exists a neighborhoodN of Sol{P) such that (c, x ) ~ val{P) - E for all x E N . Consider the multi function M{y) which maps y E Y into the level set (2.14). The multifunctionM{y) is closed, M{b) = Sol{P) and for all y in a neighborhood of b we havethat M (y) is a subset of a compact set. It follows that M (y) is upper semicontinuous at y = b. and hence M (y) c N for all y in a neighborhood of b. Weobtain that for all y in that neighborhood of b the inequality v{y) ~ val{P) - E

holds. Since E > 0 was arbitrary it follows that v{y) is lower semicontinuousaty = b. 0

The inf-compactness condition implies that the problem (P) is consistent,and hence it follows from the lower semicontinuity of v{y) that val{P) =val{D). Note that the above proposition depends on the chosen topology of thespace X.

It is said that the function v{y) is subdifferentiable at y = b if v{b) is finiteand the subdifferential 8v{b) is nonempty. By convex analysis we have thatif v{y) is subdifferentiable at y = b, then v**{b) = v(b), and conversely ifv**(b) = v(b) and is finite, then 8v(b) = 8v**(b), [22] . This leads to thefollowing result:

Proposition 2.5 If the optimal value junction v (y) is subdifferentiable aty = b, then va1(P) = va1(D) and the set ojoptimal solutions oj(D) coincideswith 8v{b). Conversely, ifva1(P) = va1(D) and isfinite, then Sol{D) = 8v(b).

The above proposition shows that there is no duality gap between the primaland dual problems and the dual problem has an optimal solution iff v{y) issubdifferentiable at y = b. Yet it may be not easy to verify subdifferentiabilityof the optimal value function directly.

Let us consider the following set

M := {(y, a) E Y x JR : y = k - Ax, a ~ (c, x), x E C, k E K}.(2.15)

It is not difficult to see that M is a convex cone in the space Y x JR. Moreover,we have that a E JR is greater than or equal to (c,x) for some feasible pointx of the primal problem (1.1) iff (b, a) E M. Therefore, the optimal value of(1.1) is equal to the optimal value of the following problem

Min a subject to (b, a) E M . (2.16)

It follows that M is the epigraph of the optimal value function v{·) iff theminimum in the above problem (2.16) is attained for every b such that v(b)


is finite . This happens, in particular, if the cone M is closed (in the producttopology of Y x JR). Sometimes the following cone

M':={(y,a) EYxJR:y=k-Ax, a=(c,x) , xEC, kEK}(2.17)

is used rather than the cone M (see (3.6) and Proposition 3.1). Note that thecone M is closed iff the cone M' is closed

Proposition 2.6 Suppose that val(P) isfinite and the cone M is closed inthe product topology ofY x JR. Then val(P) = val(D) and the primal problem(P) IUJs an optimal solution.

Proof. Since M is closed we have that the set {a : (b, a) E M} is closed, andsince val(P) is finite this set is nonempty. Moreover, for any a in that set wehave that a ~ val(P). Therefore, this set has a minimal clement a. Let x bea corresponding feasible point of the primal problem (1.1) . Then by the aboveconstruction we have that a = val(P) and ii E Sol(P). This proves that theprimal problem (P) has an optimal solution.

Since M is closed we have that epi v = M , and hence epi v is closed. Itfollows that v(y) is lower semicontinuous, and hence val(P) = val(D) byProposition 2.3. 0

By convex analysis we know that if v(y) is finite valued and continuous aty = b. then v(y) is subdifferentiable at y = band 8v(b) is compact in thepaired topology of Y' (e.g., [II, p. 84]). In particular, if Y is a Banach spaceand Y* is its standard dual, then 8v(b) is closed and bounded in the dual normtopology of Y*. This leads to the following result:

Proposition 2.7 Suppose that the optimal value junction v(y) is continuous at y = b. Then val(P) = val(D) and the set ofoptimal solutions of (D) isnonempty. Moreover, ifY is a Banach space paired with its standard dual Y*.then the set ofoptimal solutions of (D) is bounded in the dual norm topologyofY*.

Suppose now that v(y) is bounded from above on an open subset of Y; i.e.,the following condition holds.

(A3) There exist an open set N C Y and c E JR such that v(y) ~ c for allyEN.

Note that the above condition holds iff the cone M, defined in (2.15), has anonempty interior in the product topology of Y x JR. In particular, condition(A3) holds if the cone K has a nonempty interior (in the considered topology ofY). Indeed,letNbeanopensubsetofY includedinK. Thenv(y) ~ (c,O) = °for any yEN.

(2.18)


Suppose that either assumption (A3) is satisfied or the space Y is finitedimensional. Then the optimal value function v (y) is subdifferentiablc at y = biff

1· . f v(b+ td) - v (b) \..I d YImlll > -00, v E

uo t

(e.g., [4, Proposition 2.134]) .Suppose that either one of the following conditions holds : (i) the space Y is

finite dimensional, (ii) assumption (A3) is satisfied, (iii) X and Y are Banachspaces (equipped with strong topologies), the cones C and K are closed and(c, ') and A: X ~ Y are continuous. Then the function v(y) is continuous aty = b iff

se int(domv). (2.19)

It is well known that a convex function over a finite dimensional space is continuous at every interior point of its domain. In a locally convex topologicalvector space this holds under assumption (A3) (e.g., [11]). The result that,under the above assumption (iii), condition (2.19) is necessary and sufficientfor continuity of the optimal value function v(y) at y = b, is due to Robinson[19]. Clearly we have that dom v = -A(C) + K . Therefore condition (2.19)can be written in the following equivalent form

-b E int[A(C) - K]. (2.20)

Suppose that either the space Y is finite dimensional or Y is a Banach space,and assumption (A3) holds. Then v(y) is continuous at y = b iff 8v(b) isnonempty and bounded (e.g., [4, Proposition 2.131]).

The above discussion leads to the following results. We assume that if thespace Y (the space X) is a Banach space, then it is equipped with its strong(norm) topology and is paired with its standard dual Y*, equipped with theweak* topology.

Proposition 2.8 Suppose that val(P) is finite and either the space Y isfinite dimensional or assumption (A3) is satisfied. Then thefollowing statementshold:(i) val(P) = val(D) and Sol(D) is nonempty iff condition (2.18) holds.(ii) Ifcondition (2.20) hold'}, then val(P) = val(D) and Sol(D) is nonempty,and, moreover, ifY is a Banach space, then Sol(D) is bounded.(iii) If the space Y is a Banach space and Sol(D) is nonempty and bounded,then condition (2.20) holds and val(P) = val(D).

Proposition 2.9 Suppose that X and Yare Banach spaces, the cones Cand K are closed, (c, ') and A : X ~ Yare continuous, and condition (2.20)holds. Then val(P) = val(D) and Sol(D) is nonempty and bounded.


It is said that the (generalized) Slater condition for the problem (1.1) issatisfied if

3 x E C such that Ax + b E int(K). (2.21)

Clearly (2.2l) implies (2.20) . The converse of that is also true if the coneK has a nonempty interior. That is, if K has a nonempty interior, then the(generalized) Slater condition (2.21) is equivalent to the condition (2.20) (e.g,[4, Proposition 2.106]).

In some cases there are equality as well as inequality type constraints involvedin the definition of the considered problem. That is, the feasible set of the primalproblem is defined by the constraints

(2.22)

(2.23)

(3.1)

where A, : X -+ Yi,i = 1,2,arelinearmappings,andC C XandK2 C Y2areconvex cones. Clearly such constraints can be written in the form of the problem(1.1) by defining the cone K := {O} X K 2 and the mapping Ax := (Alx, A2x)

from X into Y := YI X Y2 . In the case of such constraints and if the cones Cand K 2 have nonempty interiors, the regularity condition (2.19) can be writtenin the following equivalent form (cf., [4, Section 2.3.4]):

Al is onto; i.e., AI(X) = YI ;

3 x E int(C) such that Alx + bl = 0, A2x + b2 E int(K2 ) .

We have by Proposition 2.9 that if X and Yare Banach spaces, the cones C andK 2 are closed, (c, .) and Ai : X -+ Yi, i = 1,2, are continuous, and conditions(2.23) hold, then val(P) = val(D) and Sol(D) is nonempty and bounded.

3 PROBLEM OF MOMENTS

In this section we discuss duality theory of the following conic linear problems. Let ° be a nonempty set, F be a sigma algebra of subsets of 0, andrp(W),-IPt (w), ..., 'l/Jp(w), be real valued measurable functions on (0, F) . Consider the set M+ of all probability measures on the measurable space (0,F)such that each function ip , 'l/Ji , ..., 'l/Jp is j.t -integrable for all j.t E M+ . Let Xbe the linear space of signed measures generated by M+, and X' be the linearspace of functions f : ° -+ JR generated by the functions ip, 'l/Ji, ..., 'l/Jp (i.e., elements of X' are formed by linear combinations of the functions sp, 'l/Ji, ..., 'l/Jp).The spaces X and X' are paired by the bilinear form (scalar product)

(f, j.t):= ( f(w)dj.t(w).inSince in the sequel we deal with signed measures we say that a measure j.t E Xis nonnegative, and write j.t ~ 0, if j.t(:=:) ~ 0 for any :=: E F . If j.t ~ 0 and


p(st) = 1, then it is said that p is a probability measure. In that notation

M+ = {p EX: p ~ 0, p(st) = 1}.

Consider the problem

Max (cp, p) subject to Ap - b E K,p.EC

(3.2)

where C is a convex cone in X, K is a closed convex cone in JR'p , b =(b1, ••• , bp) E JR'p, and A : p I-t (('1h ,p), ..., ('l/Jp, p)) is the linear mappingfrom X into IR'p . In particular, suppose that the cone C is generated by a convexset A C M+ of probability measures on (st, .1") , and set 'l/Jl ( .) == 1, b1 = 1and K = {O}. Then problem (3.2) becomes

Maxp.EA JEp.[CP(w)]subject to JEp.['l/Ji(W)] = bi' i = 2, ... ,p,

(3.3)

where JEp. denotes the expected value with respect to the probability measure u.In case A = M+ the above problem (3 .3) is the classical problem of moments.For a discussion of the historical background of the problem of moments theinterested reader is referred to the monograph [16] and references therein.

Problem (3.2) is a conic linear problem of the form (1.1) with the "min"operator replaced by the "max" operator and the space Y := JR,P being finitedimensional. The space Y is paired with itself with respect to the standardscalar product in 1R,P denoted by ".". The Lagrangian of the problem (3.2) canbe written in the form

L(/-l, x) := (cp, p) - x· (A/-l - b) = (cp - A*x, p) + a:> b,

where A*x = Ef=l xi'l/Ji is the corresponding adjoint mapping from JR,P intoX' .

Therefore the dual of (3.2) is the conic linear problem

Recall that

p

Min b· x subject to "" xi'l/Ji - ip E C*.xE-K' L.Ji=l

C* := {f E X' : (J, /-l) ~ 0, VP E C}.

(3.4)

Suppose that the cone C is generated by a convex set A c X (written C =cone(A»; e.g., by a convex set of probability measures. Then, since (J, /-l) islinear in p , it follows that the condition "V p E C" in the above definition of C*


can be replaced by "'V I-" E A". Consequently, the dual problem (3.4) can bewritten as follows

p

Min b· x subject to L Xi('l/Ji, 1-") 2: (cp,I-") , 'V I-" E A. (3.5)x E- K* i= l

We refer in this section to (3.2) as the primal (P) , and to (3.4) (or (3.5)) asthe dual (D) problems, respectively. The above problem (3.5) is a linear semiinfinite programming problem. This is because the optimization space IRP isfinite dimensional while the number of constraints is infinite.

Consider the cone

Mp+1 := {(x, a) E mP+l : (x, a) = (AI-" - k, (cp ,I-")) , I-" E C, k E K}(3.6)

associated with the primal problem (3.2). The above cone MpH is a specification of the cone defined in (2.17) to the present problem. The next resultfollows by Proposition 2.6.

Proposition 3.1 Suppose that val(P) isfinite and the cone Mp+1 is closedin the standard topology of IRp+ 1• Then val(P) = val(D) and the primalproblem (3.2) has an optimal solution.

Of course, it is inconvenient to parameteri ze the inequality constraints in(3.5) by measures. Suppose that every finite subset of n is F-measurable,which means that for every wEn the corresponding Dirac measure 8(w) (ofmass one at the point w) belongs to M +. That assumption certainly holds if nis a Hausdorff topological (e.g., metric ) space and F is the Borel sigma algebraof n. Suppose further that C = cone(A) , where A c M + is a convex set suchthat 8(w) E A for every wEn. Then C* is formed by the nonnegative valuedfunctions , that is

C* = {f E X' : f(w) 2: 0, 'V wEn} . (3.7)

Indeed, since A c M + we have that the right hand side of (3.7) is includedin C*. Since (J,8(w)) = f(w) for any Dirac measure 8(w), wEn, we obtainthat the other inclusion necessarily holds, and hence (3.7) follows. Therefore,in that case we obtain that the dual problem (3.5) can be written in the form:

MinxE-K*

subject tob oxxl'ljJdw) + ..,+ xp'ljJp(w) 2: cp(w), 'Vw E n. (3 .8)

Note that if A = M+ and every finite subset of n is F-measurable , then itsuffices to take only measures with a finite support of at most p +1 points in thedefinition of the cone Mp+1, i.e., the obtained cone will coincide with the one


defined in (3.6). This follows from lemma 3.1 below. If moreover K = {O},then this cone can be written in the form

MpH = cone{(7/;l(w), ...,7/;p(W), cp(w)), wEn}. (3.9)

In that form the cone MpH and Proposition 3.1 are well known in the theoryof semi-infinite programming, [6, p. 79].

As it was shown in Section 2, the weak duality val(D) 2: val(P) alwaysholds for the primal problem (P) and its Lagrangian dual (D). Note that in thissection the primal is a maximization problem, therefore the inequality relationbetween the optimal values of the primal and dual problems is reverse from theone of Section 2. If the set n is finite, K = {O} and the set A is given by theset of all probability measures on n, then problems (3.3) and (3.8) are linearprogramming problems and it is well known that in that case va1(D) = va1(P)unless both problems are inconsistent. If n is infinite the situation is moresubtle of course. We apply now the conjugate duality approach described inSection 2.

Let us associate with the primal problem (3.2) the corresponding optimalvalue function

v(y) := inf{( - cp,f..L) : f..L E C, Af..L - Y E K}. (3.10)

(We use the infimum rather than the supremum in the definition of the optimalvalue function in order to deal with a convex rather than concave function.) Wehave here val(P) = -v(b) and by the derivations of Section 2 the followingresults follow:

Proposition 3.2 The following statements hold:(i) The optimal value function v(y) is convex.(ii) val(D) = -v**(b).(iii) If the primal problem (P) is subconsistent, then val(D) = -lscv(b).(iv) If val(D) isfinite , then the (possibly empty) set ofoptimal solutions of(D)coincides with -8v**(b).(v) Ifv(y) is subdifferentiable at b, then val(D) = val(P) and the set ofoptimalsolutions of (D) coincides with -8v(b).(vi) Ifval(D) = val(P) and is finite, then the (possibly empty) set ofoptimalsolutions of (D) coincides with -8v(b).

It follows from the assertion (iii) of the above proposition that in the subconsistent case there is no duality gap between the primal and dual problems iffthe optimal value function v(y) is lower semicontinuous at y = b. It followsfrom (v) and (vi) that val(D) = val(P) and the set of optimal solutions of(D) is nonempty iff val(D) is finite and v(y) is subdifferentiable at y = b.Moreover, since the space Y = JR,P is finite dimensional, we have that v (y) issubdifferentiable at b iff condition (2.18) holds .

Suppose now that the set n is a metric space, :F is the Borel sigma algebra of n, the functions 'l/J1' ... , 'l/Jp are bounded and continuous, and cp is uppersemicontinuous on n. Let us equip the space X with the weak topology (see,e.g., [3] for a discussion of the weak topology in spaces of probability measures). Under such conditions the mapping A : X -+ JRP is continuous and thefunctional ( - ip , .) is lower semicontinuous. We have then, by Proposition 2.4,that val(P) = val{D) if the inf-compactness condition holds. In particular,consider the moment problem, that is C = cone{A) where A is a convex setof probability measures, 'l/J1 (-) == 1 and b1 = 1. Suppose that the set A iscompact in the weak topology of X. Take a closed interval [a,,8] C JR suchthat 0 < a < 1 < ,8. Then the set S := UtE[a,.Bj tA is also compact and{Il E C : All - Y E K} C S for all vectors y such that their first component belongs to the interval (a, ,8). Therefore, if moreover the problem (P) isconsistent, then the inf-compactness condition holds. We obtain the followingresult:

Proposition 3.3 Consider the moment problem (i.e., A is a subset ofM+ ).Suppose that: (i) the set n is a metric space and F is its Borel sigma algebra, (ii)thejunctions 'l/J2, ..., 'l/Jp are bounded continuous and cp is upper semicontinuouson n, (iii) the problem (P) ;05 consistent, and (iv) the set A is compact in theweak topology of X. Then val(P) = val(D) and Sol(P) is nonempty.

Recall that, by Prohorov's Theorem, a closed (in the weak topology) set Aof probability measures is compact if it is tight, i.e., for any e > 0 there existsa compact set 3 c n such that 11(3) > 1 - E for any 11 E A. In particular, if nis a compact metric space, then the set of all probability measures on (n, :F),is weakly compact. Therefore, we obtain the following corollary ([14]).

Corollary 3.0.2 Consider the moment problem. Suppose that n ;05 a compact metric space, the junctions 'l/J2, ..., 'l/Jp are continuous and sp is upper semicontinuous, and the primal problem is consistent. Then there is no dualitygap between the primal and the corresponding dual problem and Sol(P) isnonempty.

Consider now condition (2.19). Since the objective function is real valued wehave that y E dom v iff the corresponding feasible set {Il E C : All - Y E K}is nonempty. That is,

domv = A{C) - K . (3.11)

Since the space Y is finite dimensional here, we have that the following conditions are equivalent: (i) v(y) is continuous at y = b, (ii) b E int(domv),and (iii) 8v(b) is nonempty and bounded. Because of (3.11), the conditionb E int(dom v) (i.e. , condition (2.19» can be written in the form

b E int[A(C) - K]. (3.12)


We also have that iflscv(b) is finite, then v(y) is continuous at b iff 8v**(b) isnonempty and bounded. Therefore we obtain the following results:

Proposition 3.4 If condition (3.12) holds, then val(D) = val(P), and,moreover, if the common optimal value ofproblems (D) and (P) is finite, thenthe set of optimal solutions of (D) is nonempty and bounded. Conversely, ifval(D) isfinite and the set ofoptimal solutions of(D) is nonempty and bounded,then condition (3.12) holds.

Sufficiency of condition (3.12) to ensure the property: "val(D) = val(P)and Sol(D) is nonempty and bounded", is well known ([12],[14]) . In fact theabove proposition shows that condition (3.12) is equivalent to that property,provided that val(P) is finite.

A condition weaker than (3.12) is that bbelongs to the relative interior of theconvex set A(C) - K . Under such condition v(·) is subdifferentiable at b, andhence val(D) = val(P) and the set of optimal solutions of (D) is nonempty(although may be unbounded), provided that val(D) is finite.

By the above discussion we have that if v(b) is finite, then 8v(b) is nonemptyand bounded iff condition (3.12) holds. We also have by convex analysis thatthe convex function v(.) is differentiable at b iff 8v(b) is a singleton, [20].Therefore, we obtain that v(·) is differentiable at b iff the dual problem (D) hasa unique optimal solution. Since v(·) is a convex function on a finite dimensionalspace it follows that there is a set S c IRP of Lebesgue measure zero such thatfor every b E IRP \ Seither v(b) = ±oo or v(y) is differentiable at b, [20] .Therefore, we obtain the following result.

Proposition 3.5 For almost every b E IRP (with respect to the Lebesguemea~ure)eitherval(P) = ±ooorval(P) = val(D) andSol(D) is a singleton.

The result that for almost every b such that val(D) is finite, the set Sol(D)is a singleton was obtained in [14].

Let us remark that the assumption that every finite subset of 0 is measurablewas used in the above arguments only in order to derive formula (3.7) for thepolar of the cone C and, hence, to calculate the dual problem in the form (3.8).The following lemma shows that if A = M+ and every finite subset of 0 is F measurable, then it suffices to solve the problem of moments (3.3) with respectto discrete probability measures with a finite support. This lemma is due toRogosinsky [23]; we quickly outline its proof for the sake of completeness.

Lemma 3.1 Suppose that every finite subset of 0 is F -measurable. LetIt, ..., In be measurable on (0, F) real valued functions, and let J.l be a nonnegative measure on (0, F) such that h, ...,In are u-integrable. Then thereexists a nonnegative measure J.l' on (0, F) with a finite support of at most npoints such that (Ii, J.l) = (Ii, J.l') for all i = 1, ... , n.


Proof. The proof proceeds by induction on n. It can be easily shown that theassertion holds for n = 1. Consider the set S c IRn generated by vectors ofthe form ((!I, J.L'), ..., (Jn , Ji)) with J.L' being a nonnegative measure on 0 witha finite support. We have to show that vector a := ((J1, J.L) , ••., (Jn, J.L) ) belongsto S. Note that the set S is a convex cone. Suppose that a rt. S. Then, by theseparation theorem, there exists c E IRn \ {O} such that c . a ::; c . x , for allxES. Since S is a cone , it follows that c -a ::; O. This implies that (J, J.L) ::; 0and (J, J.L) ::; (J, J.L') for any measure Ji ~ 0 with a finite support, wheref := L:~1 Cih In particular, by taking measures of the form J.L' = a8(w) ,a > 0, W E 0, we obtain, from the second inequality, that f(w) ~ 0 for allw E O. Together with the first inequality, this implies that (J, J.L) = O.

Consider the set 0' := {w EO: f (w) = O}. Note that the function fis measurable and, hence, 0' E F. Since (J, J.L) = 0 and f (.) is nonnegativevalued, it follows that 0' is a support of J.L; i.e., J.L(O') = J.L(O). If J.L(O) = 0, thenthe assertion clearly holds. Therefore suppose that J.L(O) > O. Then J.L(O') > 0,and hence 0' is nonempty. Moreover, that the functions Ii, 'i = 1, ... , n, arelinearly dependent on 0'. Consequently, by the induction assumption thereexists a measure J.L' with a finite support on 0' such that (Ii, J.L*) = (Ii, J.L'),for all i = 1, ... , n, where J.L* is the restriction of the measure J.L to the set 0'.Moreover, since J.L is supported on 0' we have that (Ii, J.L*) = (Ii, J.L) and, hence,the proof is complete. 0

Consider the problem of moments (3.3) with A = M+ . Suppose that everyfinite subset of 0 is F -measurable. Then it follows by the above lemma thatfor any nonnegative measure J.L satisfying the feasibility equations ('l1Ji, J.L) = bi,i = 1, ... ,p, (with 'l/Ji (-) =1 and b1 = 1) there exists a nonnegative measure J.L'with a finite support of at most p + 1 points satisfying the feasibility equationsand such that (cp, J.L') = (cp , J.L). Consequently it suffices to solve the problemof moments with respect to discrete probability measures with a finite supportof at most p + 1 points. In fact it suffices to solve it with respect to probabilitymeasures with a finite support of at most p points. Indeed, consider the problemof moments restricted to a finite subset {WI, ...,wm } of O. That is,

MaxaEJRm

subject to (3.13)

The feasible set of the above problem (3.13) is bounded, and hence, by standardarguments of linear programming, problem (3.13) has an optimal solution withat most p nonzero components provided that this problem is consistent.


It is also possible to approach the duality problem by considering the semiinfinite programming problem (3.4) as the primal problem. We discuss that inthe next section.

4 SEMI-INFINITE PROGRAMMING

In this section we consider the semi-infinite problem (3.8) and view it as theprimal problem (P) . We refer to the recent book [7] for a thorough discussion ofthe theory of linear semi-infinite programming problems. In order to formulate(3.8) in the general framework of conic linear problems we make the followingassumptions. Let X be the finite dimensional space IRP paired with itselfby thestandard scalar product in IRP, Y be a linear space of functions f : 0 -+ JR andY' be a linear space of finite signed measures on (0, F) paired with Y by thebilinear form (3.1). We assume that Y and y' are chosen in such a way that thefunctions ip, 'lPt, ..., 'l/Jp, belong to Y, the bilinear form (i.e., the integral) (3.1)is well defined for every fEY and I-" E y', and that the following conditionholds.

(BI) The space Y' includes all measures with a finite support on O.

In particular, the above assumption implies that every finite subset of 0 is Fmeasurable.

Let us consider the cones

c+(y) := {f E Y: f(w) ~ 0, Vw E O} and C+(y'):= {I-" E y': I-" ~ O} .

Clearly (J, 1-") ~ 0 for any f E C+(Y) and I-" E C+(Y') . Therefore, the polarof the cone C+(Y) includes the cone C+(Y'), and the polar of the cone C+(y')includes the cone C+ (Y). Assumption (B 1) ensures that the polar of the coneC+(y') is equal to the cone C+(Y). We also assume that C+(Y') is equal tothe the polar of the cone C+ (Y).

(B2) The cone C+ (y') is the polar of the cone C+ (Y) .

This assumption can be ensured by taking the space Y to be sufficiently "large".

Under condition (B2) the Lagrangian dual of the semi-infinite problem (3.8)can be written as follows

Max (cp,I-") subject to AI-" - b E K.j.l>-O

(4.1)

In this section we refer to (4.1) as the dual (D) problem. Assumption (B 1)ensures that the Lagrangian dual of (4.1) coincides with the semi -infinite problem (3.8), and furthermore by Lemma 3.1, it suffices to perform optimizationin (4.1) with respect to measures with a finite support of at most p + 1 pointsonly.


(4.3)

We have the following result by Proposition 3.4 (cf., [7, Theorem 8.1]) .

Proposition 4.1 Suppose that the assumption (B1) hold", and that theprimal problem (3.8) has a nonempty and bounded set of optimal solutions.Then vaI(P) = val(D).

Consider the optimal value function

associated with the problem (3.8). Clearly w(cp) is equal to the optimal valueof (3.8).

Suppose now that 0 is a compact metric space and the functions ip, 'l/JI' ... , 'l/Jp,are continuous. In that case we can define y to be the Banach space C(O) ofcontinuous real valued functions on 0 equipped with the sup-norm. The dualC(O)* of the space C(O) is the space of finite signed Borel measures on O.By equipping Y = C(O) and Y* = C(O)* with the strong (norm) and weak*topologies, respectively, we obtain a pair of locally convex topological vectorspaces. In that framework the assumptions (B 1) and (B2) always hold.

Note that the cone C+(Y) has a nonempty interior in the Banach spaceY = C(O). Note also that the norm of p. E C(O)*, which is dual to thesup-norm of C(O), is given by the total variation norm

1Ip.1I := sup p.(0') - inf p.(0").O'E:F O"E:F

Therefore, we have by Proposition 2.8 the following results (cf, [4, Section5.4.1]).

Proposition 4.2 Suppose that 0 is a compact metric space and that theoptimal value ofthe problem (3.8) is finite. Then thefollowing statements hold:(i) There is no duality gap between (3.8) and (4.1) and theset ofoptimal solutionsof(4.1) is nonempty iff the the following condition holds:

1· . f w(cp + t1J) - w(cp) u YlID III > -00, V1J E .

40 t

(ii) There is no duality gap between (3.8) and (4.1) and the set of optimalsolutions of(4.1) is nonempty and bounded (with respect to the total variationnorm) iff the folio wing condition holds: there exists x E -K* such that

(4.4)

Recall that in the problem ofmoments the function 'l/JI is identically 1. Therefore, if also 0 is compact and the functions are continuous, then by taking Xl

(4.5)


large enough condition (4.4) can be always satisfied. It follows that in suchcase there is no duality gap between the problem of moments and its dual. Thisresult was already obtained in Corollary 3.0.2.

We come back now to a discussion of the general case; i.e., we do not assumethat n is compact and the considered functions are continuous. For a given finiteset {WI, ... , Wn } C n consider the following discretization of (3.8)

MinxE-K' b· xsubject to Xl '!f;l (wd + ... + xp'!f;p(wd ~ cp(wd, i = 1, ... , n.

Wedenote the above problem (4.5) by (Pn ) , and make the following assumption.

(B3) The cone K is polyhedral.

Since the cone K is polyhedral, its dual K* is also polyhedral, and hence(Pn ) is a linear programming problem. The dual of (Pn ), denoted (D n ) , isobtained by restricting the problem (4.1) to discrete measures (with a finitesupport) of the form J1. = :E~=I AjO(Wj). Since the feasible set of (Pn ) includesthe feasible set of (P) and the feasible set of (Dn ) is included in the feasibleset of (D), we have that val(P) ~ val(Pn ) and val(D) ~ va1(Dn ) . Moreover,by the duality theorem oflinear programming we have that val(Pn ) = va1(Dn )

unless both (Pn ) and (D n ) are inconsistent.

Proposition 4.3 Suppose that the assumptions (B 1)and (B3) hold and thatval(P) is finite. Then the following statements hold:(i) val(P) = va1(D) ifffor any e > 0 there exists a finite discretization (Pn )

such that val(Pn ) ~ va1(P) - e.(ii) val(P) = val(D) and the dual problem (4.1) has an optimal solution iffthere exists a finite discretization (Pn ) such that va1(Pn ) = val(P).

Proof. Suppose that there exists a finite discretization (Pn ) such that va1(Pn ) ~

val(P) - e. We have that val(Pn ) = va1(D n ) , and hence

val(P) ~ va1(Pn ) + E = va1(Dn ) + e ~ val(D) + e. (4.6)

Since val(P) ~ va1(D), it follows that [valfP) - va1(D) I ~ e. Since e > 0was arbitrary, this implies that va1(P) = val(D).

Conversely suppose that va1(P) = va1(D). Since val(D) is finite, for anye > 0 the dual problem has an e-optimal solution J1.. By Lemma 3.1 there existsa measure J1.' with a finite support (of at most p + 1 points) such that AJL = AJL'and (cp, J1.) = (cp, J1.'). It follows that J1.' is also an e-optimal solution of the dualproblem. Let (Dn ) and (Pn ) be discretizations of the dual and primal problems,respectivc1y, corresponding to the support of J1.'. It follows that

(4.7)


which together with val(P) = val(D) implies that val(Pn ) ~ val(P) rr E, Thiscompletes the proof of assertion (i).

Assertion (ii) can be proved in a similar way by taking c; = 0. 0Since val(P) ~ val(Pn ) it follows that the condition "for any e > °there

exists a finite discretization (Pn ) such that val(Pn ) ~ val(P) - e" holds iffthere exists a sequence of finite subsets of n such that the optimal values of thecorresponding discretized problems converge to val(P). Linear semi-infiniteprogramming problems satisfying this property are called discretizable, andproblems satisfying the property that val (Pn ) = val (P), for some discretization(Pn ) , are called reducible in [7]. The results of Proposition 4.3 were proved in[7, Section 8.3] by a different method.

Recall that, by Proposition 2.5, val(P) = val(D) and Sol(D) is nonemptyiff the optimal value function w(e) is subdifferentiable at e= ip. Therefore,we obtain that w(e) is subdifferentiable at e= cp iff there exists a finite discretization of (P) with the same optimal value, i.e., iff (P) is reducible.

5 CONTINUOUS LINEAR PROGRAMMING

In this section we discuss the following class of optimization problems

Minx 101

c(tfx(t)dt (5.1)

s.t. lot M(s , t)x(s)ds ~ a(t) , t E [0,1]' (5.2)

H(t)x(t) ~ h(t), a.e. t E [0,1], (5.3)

x(t) ~ 0, a.e. t E [0,1]. (5.4)

Here c(t), a(t) and h(t) are column vector valued functions, defined on theinterval [0,1], of dimensions nl, n2 and n3, respectively, and M(s, t) and H(t)are matrix valued functions of dimensions n2 x nl and n3 x nl, respectively.The notation "a.e." means that the corresponding inequality holds for almostevery t E [0,1]; i.e., there is a set S C [0,1] of Lebesgue measure zero suchthat the corresponding inequality holds for all t E [0,1] \ S.

We refer to (5.1)-(5.4) as a continuous linear programming (CLP) problem. Continuous linear programming problems, with the constraints (5.2) and(5.3) mixed together, were introduced by Bellman ([2]). Duality theory of(mixed) continuous linear programming problems was discussed by several authors, notably in [15] and [8]. Duality theory of (CLP) problems in the aboveform is discussed extensively in [1], where such problems are called separablecontinuous linear programming problems.

Of course, in order to formalize the above (CLP) problem we need tospecify in what class of functions x(t) = (Xl(t), ..., Xn 1 (t))T we are looking


for an optimal solution. We work with the following Banach spaces: the spaceLIlO,1] of Lebesgue integrable on the interval [0,1] functions, and the space0[0,1] of continuous on [0,1] functions . Recall that the dual of LdO, 1] isthe space Loo[O, 1] of essentially bounded measurable functions, and the dualof 0[0, 1] is the space of finite signed Borel measures on [0,1], denoted by0[0,1]*. Note that 0[0,1]* can be identified with the space of continuous fromthe right functions 7jJO of bounded variation on the interval [0,1], normalizedby 7jJ(1) = O. By 0+ [0, 1] we denote the cone of nonnegative valued on [0, 1]functions in the space 0[0, 1], and by L~[O, 1] the cone of almost everywherenonnegative valued on [0,1] functions in the space Loo[O, 1].

By Loo[O, 1]* we denote the dual space of the Banach space Loo[O, 1]. Thisdual space can be identified with the linear space generated by finite additivemeasures p on [0, 1] of bounded variation such that, if 8 is a Borel subset of[0,1] of Lebesgue measure zero, then ",(8) = O. The corresponding linearfunctional y* E Loo[O, 1]* is given by the integral y*(y) = J0

1 y(t)dp(t) (see;e.g., [13, Chapter VI]). Moreover, every linear functional y* E Loo[O, 1]* canbe expressed uniquely as the sum of an "absolutely continuous" componenty; and a "singular" component y; . The "absolutely continuous" component isrepresentable in the form y;(y) = J01y(t)'f}(t)dt for some 'f} E LIlO,1], andthe "singular" linear functional y; has the property that the interval [0,1] canbe represented as the union of an increasing sequence of Borel sets 8k (i.e.,8 1 C 8 2 C ... C [0, 1] and Uk8k = [0, 1]) such that, for every k and anyfunction y E Loo[0,1] vanishing almost everywhere outside of 8k , it followsthat y;(y) = O. We denote by 5[0,1] the subspace of Loo[O, 1]* formed bythe singular functionals . By the above discussion we have that L 1[0, 1] can beidentified with a closed subspace of Loo[O, 1]*, and this subspace has a naturalcomplement formed by the linear space 5[0, 1] of singular functionals . Thedecomposition Loo[O, 1]* = LdO, 1] + 5[0, 1] was used by several authors instudies of duality properties of optimization problems (see [21] and referencestherein). We identify a functional y* E 5[0,1] with the corresponding "singular" finite additive measure, and denote by 5+ [0, 1] the cone of nonnegative"singular" finite additive measures.

We make the following assumptions about the data of the (0LP) problem.We assume that all components of h(t) and H(t) belong to Loo[O, 1]; i.e., theyare essentially bounded measurable functions, all the components ofc(t) belongto LdO, 1], and all the components of a(t) belong to 0[0, 1]. We also assumethat each component of M(s, t) is a continuous function such that. for every


s E [0, 1], it is of bounded variation as a function of t E [0, 1]. We need theseassumptions about M(s, t) in order to use integration by parts later on.

We can view then the (CLP) problem as a conic linear problem, of the form(1.1), if we define the spaces

X := Loo[O, l]n! and Y:= C[O, l]n2 x Loo[O, l]n3 (5.5)

(i.e ., X is given by the Cartesian product of n1 copies of the space Loo[O, 1]),and the cones C C X and KeY, and the linear mapping A : X --t YandbEY, by

C:= (L~[O, l])n! and K := (C+[O, l])n 2 x (L~[O, l])n 3 , (5.6)

(Ax)(t) := (ItM(s, t)x(s)ds, H(t)x(t)) and b(t) := (-a(t) , -h(t)).

(5.7)

Note that the above spaces X and Y, equipped with the corresponding product (say max) norms, become Banach spaces, and these Banach spaces are notreflexive. We also consider the spaces

X' := LdO, 1I" and y':= (C[O, 1]*t 2 x LdO, 1t 3• (5.8)

The space X is dual of the Banach space X'. Therefore X' and X can be viewedas paired spaces, equipped with the weak (or strong) and weak* topologies,respectively. The dual of the Banach space Y is the space

(5.9)

Therefore Y and Y* can be viewed as paired spaces, equipped with the strongand weak* topologies, respectively. Since LdO, 1] can be isometrically embedded into its bidual Loo[O, 1]*, the Banach space Y' can be isometricallyembedded into the Banach space Y*, and hence we can view Y' as a subspaceofY* .

The spaces L 1 [0, 1] and L oo [0, 1]can be paired by using their respective weakand weak* topologies, and hence Y and Y' can be paired by equipping themwith respective products of paired topologies. The above discussion suggeststhat we can construct a dual of the (CLP) problem either in Y* or Y' spaces.From the point of view of the duality theory developed in Section 2, it will bepreferable to work in the space y* which is the dual of the Banach space Y.However, it is inconvenient to deal with finite additive measures of the spaceLoo[O, 1]*. Therefore, we first consider the pairing Y and Y' .

Let us calculate now the dual of the (CLP) problem, with respect to thepaired spaces X ,X' and Y ,Y'. The corresponding Lagrangian can be written


as follows

L(x, n, T/) l1 c(t)Tx(t)dt -: [It M(s, t)x(s)ds _ a(t)] T d1r(t)

-l1 T/(tf[H(t)x(t) - h(t)]dt, (5.10)

where x E Loo[O, l]n 1 , T/ E LI[O, l]na, and 1r E (C[O, 1]*)n2 ; i.e., each component of 1r is a right continuous function of bounded variation on [0, 1], normalized by 1r(1) = 0, and the corresponding integral is understood to be aLebesgue-Stieltjes integral.

By interchanging the order of integration, and using integration by parts, weobtain

l1 [It M(s, t)X(S)ds]T d1r(t) = l1 x(tf [1 M(t, s)Td1r(s)dt =

-l1 x(t)T (M(t, tf1r(t) + [1 [dM(t, s)]T1r(s)) dt, (5.11)

and, hence,

L(x, 1r, TJ) = l1 T/(t)Th(t)dt + l1 a(t)Td1r(t) + l1 9-rr,1](tfx(t)dt,

(5.12)

where

91r,1](t) := c(t) + M(t, t)T1r(t) + [1 [dM(t, s)]T1r(s) - H(tfTJ(t),

(5.13)

and the components 'E-f;'1f/1ri(S)dMij(t,S), j = 1, ... ,n1, of the integral

term J/[dM(t, s)JT1r(s) are understood as Lebesgue-Stieltjes integrals withrespect to M (t, .). Note that if for every t E [0, 1] the function M (t, .) is

constant on [0,1], then f/ 1r(s)TdM(t, s) = °and, hence, the correspondingintegral term can be deleted.

It follows that the (Lagrangian) dual of the (CLP) problem can be writtenin the form

Max(1r,TJ)EYI l1 a(t)Td1r(t) + l1 h(tfTJ(t)dt (5.14)

subject to 9-rr,TJ(t) ~ 0, a.e. t E [0,1]' (5.15)

T/(t) ~ 0, a.e. t E [0,1], 1r ~ 0, (5.16)


(5.17)

(5.18)

(5.19)

wherc function gvj, (t) is defined in (5.13) , and 1f t °means that all componentsof 1f(t) are monotonically nondecreasing on [0,1] or, equivalently, that thecorresponding measures are nonnegative valued. We denote the above dualproblem (5.14)-(5.16) by (CLP') . For (CLP) problems with constant matricesM(s, t) and H(t), the dual problem (CLP') was suggested in [17],[18] .

The optimal value of the (CLP) problem is always greater than or equalto the optimal value of its Lagrangian dual (5.14)-(5.16); i.e., val(CLP) 2:val(CLP') . Moreover, we have, by Proposition 2.1, that if val(CLP) =val(CLP') and (CLP) and (CLP') have optimal solutions x and (n-,ij), respectively, then the following complementarity conditions hold

11

97l",1] (tfx(t)dt = 0,

11 [I tM(s, t)x(s)ds - a(t)]T dn-(t) = 0,

11

ij(t)T [H(t)x(t) - h(t)] dt = 0.

Conversely, if the above complementarity conditions hold for some feasible xand (n-, ij), then val(CLP) = val(CLP') and f and (n-, ij) are optimal solutionsof (CLP) and (CLP'), respectively. These complementarity conditions canbe written in the following equivalent form (cf., [17])

97l",1](tfx(t) = 0, a.e. t E [0,1], (5.20)

1tM(s, t) x(s)ds - a(t) = 0, t E suppljt}, (5.21)

ij(tf [H(t)x(t) - h(t)] = 0, a.e. t E [0,1], (5.22)

where supp(1f) denotes the support of the measure defined by the function n,

Let us consider now duality relations with respect to the pairing of X withX* := (Loo[O, 1]*)n 1 , and Y with Y* . The spaces X* and Y* are dual of therespective Banach spaces X and Y . Therefore, we equip X and Y with theirstrong (norm) topologies, and X* and y* with their weak* topologies. Wedenote by (CLP*) the optimization problem which is the Lagrangian dual of(CLP) with respect to such pairing. An explicit form of (CLP*) and a relationbetween the dual problems (CLP* ) and (CLP') will be discussed in a moment.Let us remark that, since (CLP*) is a Lagrangian dual of (CLP), we have thatval(CLP) 2: val(CLP*), and since X' and Y' can be identified with closedsubspaces of X* and Y*, respectively, we have that val(CLP*) 2: val(CLP'),and hence

val(CLP) 2: val(CLP*) 2: val(CLP') . (5.23)

It follows that there is no duality gap between the problems (CLP) and (CLP')if and only if there is no duality gap between the problems (CLP) and (CLP*)and val (CLP') = val(CLP*) . Therefore, the question ofwhether val (CLP) =val(CLP') can be separated into two different questions ofwhether val (CLP) =val(CLP*) and val(CLP') = val(CLP*).

Let us observe that the constraints (5.4) can be absorbed into the constraints(5.3) by adding to H(t) rows of the n1 x n1 identity matrix, and adding n1 zero

components to h(t) ; i.e., by replacing H(t) and h(t) with H(t) := [ Hjt) ]

and h(t) := [ h~t) ], respectively . This will not change the corresponding

dual problem (see the discussion of Section 2 about the dual problem (2.5)) .Consider the Lagrangian L*(x,1r,'TI,(J,r/,(J') of the obtained (CLP) problem with respect to the pairing of X ,X* and Y ,Y*. Here (J E 5[0, 1]n3 represents multipliers associated with singular measures of (Loo[O, 1]*)n3 , and'TI' E LdO,1]n1 and (J' E 5[O,1]n1 are multipliers corresponding to the constraints (5.4) . Compared with the Lagrangian L(x, 1r , 'TI), given in (5.10) and(5.12), the above Lagrangian can be written as follows

L*(x, 1r,'TI, (J,'TI',(J') = L(x,1r,'TI) -11[H(t)X(t)-h(t)fd(J(t)

-11

x(tf'TI'(t)dt -11

X(t)Td(J'(t). (5.24)

The dual problem (CLP*) is obtained by minimizing L * (x, zr, 'TI, (J, 'TI' , (J') withrespect to x E X and, then, maximizing with respect to 1r !'::: 0, 'TI ~ 0, (J !'::: 0,'TI' ~ 0, (J' !'::: 0.

Sinceg1T,T/ E LdO, 1]n1 and the space Lao [0, 1]* is the direct sum of the spacesLI[O, 1] and 5[0, 1], it follows that the dual problem (CLP*) is obtained from(CLP') by adding to it the term

Max r1h(t)Td(J(t) subject to - H(t)d(J(t) !'::: 0. (5.25)

uES+[O,lj n3 J0

Note that the above maximum is either zero or +00. This leads to the followingresult.

Proposition 5.1 Suppose that the problem (CLP) is consistent. Thenval(CLP') = val(CLP*) and Sol(CLP') = Sol(CLP*).

Proof. Since (CLP) is consistent, we have that val (CLP) < +00 and, sinceval(CLP*) ~ val(CLP), it follows that val(CLP*) < +00. Consequentlythe additional term given by the maximum (5.25) is zero. The result then followsby the above discussion. 0


The above proposition shows that an investigation of the duality relationsbetween the problems (CLP) and (CLP') can be reduced to a study of theproblems (C LP) and (C LP*) . Let us discuss now the question of "no dualitygap" between the problems (CLP) and (CLP*) .

Let v(y) be the optimal value of the problem (CLP). That is, y = (Yt ,Y2),u, E C[O, 1]n2 , Y2 E t.; [0, 1]n3 , and v(y) is the optimal value of the problem(5.1)-(5.4), with a( ·) and h(·) replaced by Yl (.) andY2('), respectively. We have,by Proposition 2.3, that if the problem (CLP) is subconsistent (in particular,consistent), then va1(CLP) = va1(CLP*) iff v(y) is lower semicontinuous aty=(a,h).

The cone K has a nonempty interior in the Banach space Y. This implies thatthe optimal value function v (y) is bounded from above on an open subset of Y ;i.e., that condition (A3) of Section 2 holds, and that the constraint qualification(2.20) is equivalent to the (generalized) Slater condition (2.21). The interiorof the cone C+[O, 1] is formed by functions cp E C[O, 1] such that cp(t) > °for all t E [0, 1], and the interior of the cone L~ [0, 1] is formed by functions'I/J E Loo[O, 1] such that 'I/J(t) ;:::: E for a.e. t E [0,1] and some E > 0. Thereforethe (generalized) Slater condition for the (CLP) problem can be formulated asfollows.

(Slater condition) There exists ii: E (L~[O, l])n1 and E > °such that:

It M(s, t)x(s)ds > a(t) , Vt E [0,1],

H(t)x(t) ;:::: h(t) + E, a.e. t E [0,1] .

(5.26)

(5.27)

Clearly the above Slater condition implies that the problem (CLP) is consistent.The above discussion together with Propositions 2.8 and 5.1 imply the followingresults:

Proposition 5.2 Suppose that the (generalized) Slater condition (5.26) (5.27) holds. Then va1(CLP) = va1(CLP') and, moreover, ifva1(CLP) isfinite , then the set ofoptimal solutions ofthe problem (CLP') is nonempty andbounded (in the norm topology ofY"). Conversely ifthe set ofoptimal solutionsofthe problem (C LP') is nonempty and bounded, then the (generali zed) Slatercondition holds.

In some cases (CLP) problems involve equality type constraints. For example, some (all) constraints in (5.2) and/or (5.3) can be equality constraints. Ofcourse, the equality constraints such as H(t)x(t) = h(t) can be split into theinequality constraints H(t)x(t) ;:::: h(t) and H(t)x(t) ::; h(t). Note, however,that for such split inequality constraints the (generalized) Slater condition cannever be satisfied. Nevertheless, the assertion of the above Proposition 5.2 still

holds if the (generalized) Slater condition is replaced by the regularity condition(2. 19), or its equivalent (2.20). Recall that (2.19) and (2.20) are equivalent toconditions (2.23), which can be written in a form similar to (5.26)-(5.27).

In the present case , since the cone K has a nonempty interior in the Banachspace Y , the Slater condition (5.26)-(5.27) is equivalent to the continuity (in thenorm topology ofY ) of the optimal value function v (y) at y = (a , h) . We shownow that a variant of the inf-cornpactness condition is sufficient for the lowersemicontinuity of the optimal value function. Recall that the Banach spaceX = Loo[O, 1]n1 is dual of the Banach space Lt[O, 1]n 1 and, therefore, X canbe equipped with the corresponding weak" topology. Moreover, if the feasibleset of the (CLP) problem is nonempty and bounded in the norm topologyof X , then it is compact in the weak" topology of X, and hence the problem(CLP) possesses an optimal solution. Note, however, that in the present casewe cannot apply the result of Proposition 2.4 in a direct way since the linearmapping A : X -+ Y, defined in (5.7), is not continuous with respect to theweak" topology of X and the norm topology of Y.

Let us consider the following condition. For y E Y , with y = (Yl, Y2) ,u, E C[O, 1]n2 , Y2 E Loo[O, 1]n3 , denote by (y) the set of all x E X satisfyingthe feasibility constraints (5.2)-(5.4), with functions a( ·) and h(·) replaced byYl (.) and Y2 (.), respectively. In that notation the set (a, h) denotes the feasibleset of the (CLP) problem.

(C'l) The feasible set of the (CLP) problem is nonempty, and there existsE > 0 such that the sets (y) are uniformly bounded for all y E Ysatisfying lIy - (a,h) II < c.

Proposition 5.3 Suppose that the assumption (C 1) holds. Then the (C LP)problem has an optimal solution, the optimal value junction v(y) is lowersemicontinuous at y = (a, h), and val (CLP) = val (CLP').

Proof. Since assumption (C 1) implies that the feasible set of the (CLP) problem is nonempty and bounded, it follows that (CLP) has an optimal solution.

By assumption (Cl) we have that the problem (eLP) is consistent. andhence it follows by Proposition 5.1 that va1(CLP') = val(CLP*). Therefore,in order to show that va1(CLP) = va1(CLP') it suffices to verify that v(y) islower semicontinuous, in the norm topology of Y, at the point Yo := (a, h).If lim infy-+ yOv(y) = +00, then clearly v(y) is lower semicontinuous at Yo.Therefore we can assume that lim infy-+ yOv(y) < +00. Consider a sequenceYn E Y along which the lower limit of v(y) is attained as y tends to Yo. Bycondition (Cl) we can assume that for every n E IN the corresponding feasibleset (Yn) is bounded and, since lim infy-+ yOv(y) < +00, is nonempty. Itfollows that the associated optimi zation problem attains its optimal value at a

point Xn EX; i.e., xn E <l?(Yn) and v(Yn) = (c, xn). Again by assumption(CI) we have that the sequence {xn } is bounded in X . Therefore, by passingto a subsequence if necessary, we can assume that {x n } converges in the weak*topology of X to a point xo. It follows that Xo is a feasible point of the (C LP)problem. Consequently we obtain

liminfv(y) = lim v(Yn) = (c,xo) 2: val(CLP) =v(Yo),y-tyo n-too

which proves the lower semicontinuity of v(y) at Yo. 0

The above condition (Cl) assumes that the feasible sets of the corresponding continuous linear programming problems are uniformly bounded for small(with respect to the sup-norm) perturbations of the right hand side functions.Condition (C 1) is slightly stronger than the condition that the feasible set of the(CLP) problem is nonempty and bounded. The later condition was used in [17],[18] for deriving the property val(CLP) = va1(CLP') under the additionalconditions that the matrices M(s, t) and H(t) are constant (i.e., independentof sand t), and the functions c(t) , a(t) and h(t) are piecewise analytic.

Let us consider the following condition.

(C2) The feasible setofthe (CLP) problemisnonempty, the matrixH = H(t)is constant, and the set

{x EX : Hx(t) 2: h(t) and x(t) 2: 0, a.e. t E [0, I]} (5.28)

is bounded in X .

Note that the above set (5.28) is the set defined by constraints (5.3)-(5.4).

Proposition 5.4 Assumption (C2) implies assumption (Cl).

Proof. For b E JRn3 consider the set

w(b) := {x E JRn l: Hx 2: b, x 2: O}.

Suppose that for some bo E JRn3 the set w(bo) is nonempty. Then by Hoffman's

Lemma in [10] there exists a constant r;, > 0, depending on H, such that

w(b) C w(bo) + r;,lIb - bollB, Vb E JRn3 , (5.29)

where B := {x E JRnl : IIxll ~ I}. Assumption (C2) implies that, for almostall t E [0,1], the sets w(h(t)) are nonempty and uniformly bounded. It followsthen by (5.29) that for any E > 0, the subsets of X defined by constraints(5.3)-(5.4), with h(t) replaced by Y2(t) satisfying IIY2 - hll ~ E, are uniformlybounded. This implies that the feasible sets <l?(y) are uniformly bounded for all


Y E Y such that IIY2 - hll is bounded. This shows that assumption (C2) impliesassumption (C1). D

By Propositions 5.3 and 5.4 we obtain that if assumption (C2) holds, thenval(CLP) = val(CLP') and Sol(CLP) is nonempty.

References

[1] E. J. Anderson and P. Nash. Linear Programming in Infinite-DimensionalSpaces. Theory and Applications, Wiley, 1987.

[2] RE. Bellman . Bottleneck problems and dynamic programming, Proceedings ofthe National Academic ofSciences ofthe USA, 39: 947-951, 1953.

[3] P. Billingsley. Convergence 0/ Probability Measures (Znd ed.), Wiley,1999.

[4] J.P. Bonnans and A. Shapiro. Perturbation Analysis ofOptimization Problems, Springer-Verlag, 2000.

[5] L Ekeland and R. Temam. Convex Analysis and Variational Problems,North-Holland, 1976.

[6] K. Glashoff and S.A. Gustafson . Linear Optimization and Approximation,Springer-Verlag, 1983.

[7] M. A. Gobema and M. A. Lopez. Linear Semi-Infin ite Optimization, Wiley, 1998.

[8] RC. Grinold . Symmetric duality for continuous linear programs, SIAMJournal on Applied Mathematics, 18: 84-97, 1970.

[9] R Hettich and K.O. Kortanek . Semi-infinite programming: theory, methods and applications, SIAM Review, 35: 380-429, 1993.

[10] A. Hoffman. On approximate solutions of systems of linear inequalities,Journal ofResearch ofthe National Bureau ofStandards, Section B, Mathematical Sciences, 49: 263-265, 1952.

[11] R.B. Holmes . Geometric Functional Analysis and its Applications,Springer-Verlag, 1975.

[12] K. IsH. On sharpness of Tchebycheff type inequalities, Annals 0/ the Instute 0/Statistical Mathematics, 14: 185-197, 1963.

[13] LV. Kantorovich and G.P. Akilov. Functional Analysis, Nauka, Moscow,1984.

[14] J.H.B. Kemperman . The general moment problem, a geometric approach,Annals ofMathematics and Statistics, 39: 93-122, 1968.

[15] N. Levinson. A class of continuous linear programming problems, Journal0/Mathematical Analysis and Applications, 16: 78-83, 1966.


[16] H.J . Landau (editor), Moments in mathematics, Proc. Sympos. Appl.Math., 37, Amer. Math. Soc. , Providence, RI, 1987.

[17] M. C. Pullan. A duality theory for separated continuous linear programs,SIAM Journal on Control and Optimization, 34: 931-965, 1996.

[18] M. C. Pullan. Existence and duality theory for separated continuous linearprograms, Mathematical Models and Systems, 3: 219-245 , 1997.

[19] S.M . Robinson. Regularity and stability for convex multivalued functions,Mathematics ofOperations Research, 1: 130-143, 1976.

[20] R.T. Rockafellar. Convex Analysis, Princeton University Press, 1970.

[21] R.T. Rockafellar. Integrals which are convex functionals. II, Pacific Journal ojMathematics, 39: 439-469, 1971.

[22] R.T. Rockafellar. Conjugate Duality and Optimization, Volume 16 ofRegional Conference Series in Applied Mathematics, SIAM, 1974.

[23] W.W. Rogosinsky. Moments of non-negative mass, Proceedings of theRoyal Society London, Serie A, 245: 1-27, 1958.

[24] J.E. Smith. Generalized Chebychev inequalities: theory and applicationsin decision analysis, Operations Research, 43: 807-825, 1995.

Part III NUMERICAL METHODS

Chapter 8

TWO LOGARITHMIC BARRIER METHODS FORCONVEX SEMI-INFINITE PROBLEMS

Lars AbbeDepartment ofMathematics, University ofTrier, D-54286 Trier, Germany

abbe@gksun01 .uni-trier.de

Abstract In the first part of the paper a logarithmic barrier method for solving convexsemi -infinite programming problems with bounded solution set is considered.For that the solution of a non-differentiable optimization problem by means ofa logarithmic barrier method is suggested. The arising auxiliary problems aresolved, for instance, via a bundle method. In the second part of the paper aregularized logarithmic barrier method for solving convex semi-infinite problemswith unbounded solution set is considered, which is based on the method from thefirst part. The properties and the behaviour of the presented methods are studiedand numerical results are given.

1 INTRODUCTION

In this paper we consider convex semi-infinite programming problems, i.e.,problems in which the objective function depends on finitely many variables andwhich contain infinitely many constraints. In practice many applications leadto semi-infinite programs. Caused by this fact particular methods for solvingsuch problems are developed. We refer the reader to the proceedings of Hettich[7] and Fiacco and Kortanek [3], as well as to Polak [15] and Reemtsen andRiickmann [16] for an overview.

Since several successful interior-point methods for finite problems have beeninvestigated during the last 15 years, it was natural to use ideas from such methods also in the field of semi-infinite programming, see, e.g., Ferris and Philpott[2], Kaplan and Tichatschke [10), Schattler [18], Sonnevend ([19], [20)), andTodd [21]. In this context several approaches are possible . For instance, onecan successively discretize the given semi-infinite problem and solve the arisingfinite discretized problems with an existing interior-point method.

169

M.A. Goberna and M.A. Lopez (eds.), Semi-Infinite Programming, 169-195 .© 2001 Kluwer Academic Publishers.


We study an approach where we directly attack the semi-infinite problemby an interior-point approach . In particular, we apply the logarithmic barrierapproach, which traces back to Frisch [6], to a non-differentiable reformulation(see Polak [15]) of a given convex semi-infinite problem. Thus we have nondifferentiable auxiliary problems and, for solving them, we extend a bundlemethod from Kiwicl [12]. In order to prove a convergence result for the obtainedalgorithm, we have to assume that the set of optimal solutions of the givenproblem is compact. Since many applications violate this assumption a secondalgorithm is studied. Therein we combine the logarithmic barrier approach withthe proximal point technique (introduced by Martinet [14]). Then we can provethat the regularized algorithm leads to a sequence which converges to an optimalsolution, while for the unregularized method we can only prove convergence tothe set of optimal solutions .

The paper is organized as follows. In Section 2 we start with a short description of the extension of Kiwicl's bundle method. In Section 3 the logarithmicbarrier method for solving semi-infinite programming problems with boundedsolution set is presented, while in the following section its properties are analyzed. Section 5 gives general hints for a numerical use of this method and inSection 6 a numerical example is analyzed. In Section 7 the logarithmic barrier method from Section 3 is coupled with the proximal point approach, and aconvergence result for the combined method is presented. Finally, in Section8, numerical results of the regularized method are given.

2 A BUNDLE METHOD USING e-SUBGRADIENTS

In this section we consider the problem

minimize f(x) s.t, xES (2.1)

with a convex function f : lRn -t lR, and a nonempty compact convex setS eRn. Let E ~ 0 be given. We assume that

• f(x) cannot be computed exactly, but an e-approximation j(x) can befound (in the sense of f(x) - E ~ j(x) ~ f(x) + c);

• for each xES an e-subgradient 9f(x) E 8ef(x) can be computed;

• the function f is Lipschitz continuous on S with Lipschitz constant L fsuch that 119f(x) 112 ~ L] for all xES.

These assumptions are similar to those of Kiwiel [12] (there e = 0). Thereforewe suggest a modification of Kiwiel's proximal level bundle method for solvingproblem (2.1). In Kiwicl's Algorithm 1 we merely replace all computations off by j and all computations of a subgradient by an s-subgradient. Linearizing

LOGARITHMIC BARRIER MFI1IODS FOR CONVf.-'X SIP 171

fin x k E S by

lead s to

Algorithm 2.1

Step 0 Give x l E S . the final tolerance Copt ~ O. a level parameter 0 < K. < 1and e ~ O. Set x~ := Xl, f2 p := 00. u: := minxES f1( x) . J1 := {I},k := 1, I := O. k(O) := 1.

Step 1 Set f:p := min{j(x k), f:;; l} , t::.k := f:p - fl~w ' If f:p = j(xk) setx k . - x k

Tee' - .

Step 2 If t::.k :s copt or 9f(xk) = 0 terminate; otherwise continue.

Step 3 If

minimize ~ IIx - x~ lI :

s.t. X E S , f i(x):s K.fl~w + (1 - K.) f :p (j E Jk)

(2.2)

is f easible. go to Step 5; otherwise continue.

Step 4 Set fl~w := minxEs maxi EJk f i (x). Choose x~ E {xi: j E Jk}. Setk (I + 1) := k , increase I by 1 and go to Step 1.

Step 5 Find the op timal solution x k+1of (2 .2) and its multipliers Aj such that

the set jk := {j E Jk : Aj > O} satisfies Ijk I<n.

Step 6 Calculate j(x k+1 ) and 9f(Xk+1) E Oef(xk+l).

Step 7 Select J: c Jk so that jk C J: . Set J k+1 := J: U {k + I}, X~+l :=

x~ . n: := n: Increase k by 1 and go to Step 1.

Since we compute f:p and fl~w with the approximation j of f the properties

f:p ~ 1* and n: :s 1* with 1* = minxES f(x), which are valid for Kiwiel's

Algori thm, arelost. But j(xi) ~ f (xi) - Eholds so that f:p ~ 1* - e follows.

M oreover, f i(x) = j(xi) + 9f(xi)T(x - xi) :s f( x) + e + c is true for allx E S and j E IN suc h that one can conclude fl~w :s 1* + 2c . Altogether wehave

rr E :s f~p - 1* :s c. -fl~w + 2c = t::.k + 2c, (2.3)

which is an equivalent for inequal ity (2.2) in [I 2]. Fro m th is point on we canfollo w the an alysis o f Ki wiel.

If the break in Step 2 is caused by 9f(xk ) = 0 for some k E IN, then x k

is an s-solution of (2.1) (cf., e.g., Theorem 1.1.5. p.94. in Hiriart-Urruty andLemarechal [8]). On the other hand if the break is caused by 11k ::; Copt theinequalities

j(x~eJ - r ::; Copt + 2c and f(x~eJ - r ::; Copt + 3c (2.4)

hold for the "record" point x~ec' i.e., x~ec is an (copt + 3c)-solution of (2.1).Due to the inexact input data we cannot expect a better solution.

Kiwiel's remarks concerning the algorithm in [12] can almost be copied.Particularly. the inequalities (2.3) and (2.4) of Kiwiel are valid. Also Lemmata3.1-3.4 just as Theorem 3.5 and Corollary 3.6 only need to be changed slightlydue to the use of (2.3) instead of the original inequality (2.2).

3 DESCRIPTION OF THE BARRIER METHOD

We consider the problem

minimize f(x) s.t. g(x, t) ::; 0 for all t E T.

In the sequel the equivalent formulation

minimize f(x) s.t.(P) maxg(x, t) ::; 0tET

is used and we suppose that the following assumptions are fulfilled:

(A 1) f : lRn~ lR is a convex function;

(A2) T C R1 is a compact set;

(A3) g(., t) is convex on R" for each t E T;

(A4) g(x,·) is continuous on T for each x E R";

(AS) for each x E JRn and each t E T an element of 8f(x) and an element ofthe subdifferential of g(., t) in x can be computed;

(A6) the set M opt := {x EM: f(x) = infzEM f(z)} is nonempty andcompact with M := {x E R" : maxtET g(x, t) ::; O};

(A7) thesetMo:= {x EM: maxtETg(X,t) < O} isnonempty;

(A8) for each h ~ 0 a subset Th of T is defined such that for each t E T thereexists a th E Th with lit - th 112 ::; h;

(A9) for each compact set S there exists a constant L~ with

(3.1)

LOGARITHMIC BARRIER METHODS FOR CONVEX SIP 173

for all xES and all tt , t2 E T.

Furthermore, for given J.L > 0 we define fp. : Mo -+ R by

fp.(x) := f(x) - J.L In (- maxg(x, t)) .tET

Now let us consider

Algorithm 3.1

• Give J.Ll > 0 and xo E Mo.

• For i := 1,2, . . . :

* Set xi,o := xi- 1 and select E:i,O > o.* For k := 0, 1, .. . :

a) Select ri,k > 0 such that

Si,k := {x E JRn : Ilx - xi,kll oo :s; ri,d C Mo .

b) Select hi,k ~ 0 and define h,k : Mo -+ JR by

l i,k(x) := f(x) - J.Li In (- max g(x, t))tETh; ,1c

where Th; ,1c is a set fulfilling (AB).

c) Select f3i,k ~ 0 and compute an approximate solution x i,k+lof the problem

. . . f ( ) s' kmlfllfnlze p.; X s.t. x E '

such that

- . k 1 E:i kAk(X~' + ) - min fp.;(x) :s; -2' + 2f3i ,k (3.2)

xES"1c

and h,k(xi,k+l) :s; h,k(xi,k) are true.

d) - Ifh,k(xi ,k)-li,k(xi,k+l):s; E:i,k/2andf(xi,k):s; f(xO)+2J.Li then set x i := xi,k, Si := s». ri := ri k» E:i := E:i k,, ,hi := hi ,k' stop the inner loop;

- ifh,k(xi,k)- h,k(xi,k+l) :s; E:i ,k/2andf(xi,k) > f(xO)+2J.Li then set E:i ,k+l := E:i,k/2 and continue the inner loop;

- ifh,k(xi,k) - li ,k(xi,k+l) > E:i,k/ 2 set E: i,k+l := E:i,k andcontinue the inner loop.

(3.3)


* Select 0 < J.Li+l < J.Li.

Regarding that Mo is an open set and x i,k E Mo holds by construction, therealways exists a positive radius Ti,k such that Si ,k C Mo. Thus the critical pointfor a practical realization of the presented method is whether there exists a pointxi,k+l which fulfills the given criterion. In order to ensure this we show that wecan use the bundle method from Section 2 for solving (P1l;,S; ,k) approximatelywith copt := ci,k/2.

It is known that f ll is convex on Mo for all J.L > 0 (see Fiacco and MeCormick [4], Lemma 11). Furthermore, Si ,k is obviously nonempty, convexand compact, and we use h,k as an approximation of f ll; with error f3i ,k. Then(3.2) follows immediately from (2.4) . In the following lemma the error f3i,k andan expression for a f3i,k-subgradient are specified. To formulate this lemma anadditional constant is needed. For each nonempty compact set DeMo wedefine a constant CD E R with

CD> max I 1 I- xED maxtET g(x, t)

and the property that D C D' c Mo implies CD :::; CD', Also we define forall x E lRn and all h ~ 0 the set of active constraints

T(x) := {S E T : g(x, s) = maxg(x, t)}tET

and its approximation

Th(x) := {S E Th : g(x, s) = maxg(x, t)} .iet;

Lemma 3.1 Let i.k and x E Si,k be fixed. Furthermore, let u E 8f(x)and u E 8(maxtETh;,k g(x, t)) be given. Iff3i,k ~ J.LiL~;,kCS;,khi,k then

and

(3.4)

Proof. See Abbe [1]. 0

The Lipschitz continuity of f ll; on Si ,k C Mo = int(domfll;) followsfrom Theorem 24.7 in Rockafellar [17]. Due to the same theorem, the subdifferentials of f and maxtETh . g(., t)) are bounded on s». Consequently,

. ,k

LOGARITHMIC BARRIER MEI1IODS FOR CONVEX SIP 175

the .Bi,k-subgradients given in (3.4) are bounded on s». Thus we can use thebundle method from Section 2 in order to determine the approximate optimalsolution xi ,k+l of (PJLi ,Si,k) with .Bi,k ~ lJ.iL~i .k CSi,khi,k. Additionally, in that

case we do not have to select .Bi,k explicitly since .Bi,k = lJ.iL~i ,k CSi,k hi ,k canbe used with the previously chosen hi,k.

Remark 3.1 If it is possible to determine maxtET g(x, t) exactly for eachfeasible solution x we can set hi ,k = 0 for all pairs i, k. Consequently d.j, = 0

is allowed (independent of the values L~i ,k ' CSi,k) such that h,k and f JLi areidentical. This leads to some simplifications in the algorithm above as well asin the following convergence analysis. Furthermore, in this case we can dropthe Assumptions (A8) and (A9).

4 PROPERTIES OF THE METHODIn this section we present conditions on the parameters of Algorithm 3.1 to

get a convergence result. For that we denote for given IJ. > 0

(PJL) minimize fJL(x) s.t, x E Mo

as well as min(P) = infxEM f(x) and min(PJL) = infxEMo fJL(x) . Lemma12 in Fiacco and McCormick [4] guarantees that (PJL) is solvable if (AI)-(A4),(A6) and (A7) are fulfilled. Moreover, we have

0::; f(x(J-L)) - min(P) ::; IJ. (4.1)

for all optimal solutions x{lJ.) of (PJL) (see, e.g., Wright [22]).

Lemma 4.1 Let i be fixed and 5: be an arbitrary optimal solution of(PJLi)'Moreover, let xi,k and xi,k+l be generated by Algorithm 3.1. Additionally, letthe estimate .Bi,k ~ lJ.iL~i ,k CSi,khi,k be valid. If the inequality

f-· (xi,k) - f-· (x i,k+l) < ci,k (4.2)z,k z,k - 2

is true, then

i k. { IIxi,k - 5:1100 } (0::; fJLi(x' ) - mm{PJLJ ::; max 1, Ci,k + 3.Bi,k). (4.3)

ri,k

Proof. The inequality 0 ::; f JLi (xi,k) -min{PJLi) is obvious since xi,k is feasibleby construction. Furthermore,

- . k 1 ci kJi,k{XZ

, + ) - insfk

fJLi{x) ::; -2' + 2.Bi,kxE t ,

is valid by construction. Using (4.2) and Lemma 3.1 we conclude

fJLi{xi,k) - inf fJLi{x)::; Ci,k + 3.Bi,k. (4.4)xESt ,k

X is an arbitrary optimal solution of (PJLi)' We first assume that x E s». Thenx is also an optimal solution of (P JLi ,Si.k) so that it follows from (4.4)

fJLi(xi ,k) - min(PJLJ = fJLi(xi,k) - fJL i(X) ~ ci,k + 3{3i ,k'

Thus the proposition is true in this case.Now, let us consider the case x fI. S i,k and define the line through xi,k and x

by

( ) ._ i,k S ( A i,k)'Y S .- x + IIxi,k _ xll

oox - x .

Due to x fI. Si,k we have IIxi,k - xll oo > ri,k > O. Therefore 'Y(ri,k) liesbetween xi,k and x on that line. Since f JLi is convex on Mo with minimizerx, one gets immediately fJLi(,(ri ,k)) ~ fJLJxi,k). Moreover, the equationIIxi,k - 'Y(ri,k)1100 = ri ,k is true so that 'Y(ri,k) E s». Altogether, using (4.4),we obtain

ikfJLi(x' ) - fJLi(,(ri,k)) ~ Ci,k + 3{3i,k'

Further, regarding the convexity of f JLi and 0 < ri,k/llxi,k - xll oo < 1, we have

and the proof is complete. D

Now a sufficient termination condition for the inner loop of Algorithm 3.1can be presented.

Proposition 4.1 Let i be fixed and ri,k ~ !:i > 0 be valid for all k. Moreover, let qi E (0,1), lSi > 0 be given. If

(4.5)

is true for all k, then the inner loop ofAlgorithm 3.J terminates after a finitenumber of steps.

Proof. The inner loop terminates after a finite number of steps if the inequalities(4.2) and

(4.6)

are both true. In the main part of the proof we assume that both inequalitiesnever hold together. In order to bring this to a contradiction we exclude thepossibility that (4.2) never holds in a first step.

i) Suppose that the inequality (4.2) never holds.

(4.7)


Then the algorithm generates an infinite sequence {Xi,k hand Ei ,k = Ei,O istrue for all kEN. By construction the estimate h,k(xi,k+l) ~ h,k(xi,k) isvalid for all k. Using this, (4.5) and Lemma 3.1 we infer for all k

o < h,k(xi,k) - h,k+l(xi,k+l) + qfc5i

(h,k(Xi'k) - ~lJic5i) - (h,k+l(Xi ,k+l) - tlJic5i) .

)=0 )=0

Thus

{Ji,k(Xi,k) - I: lJi c5i }

)=0 k

is a monotonically nonincreasing sequence which is bounded from below since

k-l

Ji,k(xi,k) - ~ lJi s. >j=o

>

00

fIJ.i (xi,k) - (3i ,k - ~ lJi c5ij=O

inf f IJ.i (x) - c5i - _1_ c5ixEMo 1- qi

for all k. Thus the sequence converges. Combining this and qfc5i~ 0 wecan find an index ko with

t. (xi,kO) - f-' (xi,ko+l) < t. (xi,kO) - t, (xi ,ko+l) + q~0c5' < Ei ,Oz,ko z,ko+l - z,ko z,ko+l z z - 4

and (3i,ko ~ qfO s, ~ Ei,0/4. Then another use of Lemma 3.1 leads to

t. (xi ,ko) - f-' (xi,ko+l)z,ko z,ko ~ Ji,ko(xi,kO) - !IJ.i(xi ,ko+l) + (3i,ko

< f- (i,kO) f- (i,ko+l) + R < Ei,O- i,ko x - i,ko+l x IJi,ko - 2 '

which contradicts our assumption, and we have an index k such that (4.2) isfulfilled.

ii) Suppose that the inequalities (4.2) and (4.6) never hold together.As in i) one can show that (4.7) defines a monotonically nonincreasing se

quence. Therefore, taking Lemma 3.1 and (4.5) into account, we infer

k

fIJ.i(xi ,k) - ~lJic5i ~ h,o(xi,o) ~ !IJ.i(Xi,O)j=o


for all kEN. Thus one has fJli(x i,k) ~ fJli(Xi,O) + oi!(1- qd and

xi ,k E Ni := {x E Mo : fJli(x) ~ fJli(Xi,O) + 1 ~ qi}

for all k. The set N, is compact (cf. [4, Lemma 12] ) so that the sequence{lIx i ,k - xllooh is bounded from above by a constant C ~ [ i' where x is anarbitrary optimal solution of (PJli)'

From the first part of the proof we know that there exists an index ko suchthat (4.2) holds for k = ko for the first time. Using the same arguments forall indices greater than ko we find an index k1 > ko such that (4.2) holdsfor k = k1 again. Repeating this procedure we get a strictly monotonicallyincreasing sequence {k j } such that (4.2) holds for all k = kj • Now we deducefrom Lemma 4.1 and (4.5)

o~ f Jli (xi,kj) - min(PJlJ ~ max { 1, IIXi'k~i:jxll oo} (Ci'kj + 3q:j Oi)

for j ~ O. It is simple to verify that Ci ,kj = (1/2)j ci,O . Combining this with

IIxi,kj - xlloo ~ C , kj ~ j as well as ri,k ~ [i > 0 for all k we have

os fp;(x"ki ) - min(Pp,) ~ ~ ( Gr"',0 + 3Oi q{)for all j ~ 0, hence

Since the sequence {xi ,kj} belongs to the compact set Ni, there exists an accumulation point z" of {xi ,kj} . From the last equation we obtain that x*solves (P /.Li). Due to the continuity of f there exists an index 3 ~ 0 with

f(xi,kJ) ~ f(x*) + J1.i . This and (4.1) lead to

o~ f(xi ,kJ) - min(P) ~ f(x*) + J1.i - min(P) ~ 2J1.i.

This contradicts our assumption, both inequalities (4.2) and (4.6) are true fork = k3 and the proof is complete. 0

Remark 4.1 The assumption Ti,k ~ !:.i > 0 is not used to prove that alliterates belong to the nonempty compact set Ni. Therefore there exists anr; > 0 such that the inclusion {z E R" : minxENi liz - xll oo ~ rn C Mo isvalid 'since Mo is an open set. Thus the selection ri,k ~ ri is possible for eachk such that Ti,k ~ [ i > 0 is no restriction for the algorithm. It still restricts thepractical computation of the radii ri,k of course,

With Proposition 4.1 we are able to control Algorithm 3.1 in order to makesure that each inner loop terminates and a well-defined sequence {xi} is generated.

Theorem 4.1 Let {r:i}, {&i} bepositive sequences. Additionally, let R > 0and qi E (0,1) be given for all i. Moreover. assume that (4.5) holds for all i, kand

(1) limi-too /-Li = 0;

(2) D :s; ri,k :s; Rfor all i , k ;

(4) limi-too &dri = O.

Then Algorithm 3.1 generates a sequence {xi}, which has at least one accumulation point and each accumulation point is an optimal solution of(P).

Proof. It is easy to see that the assumptions of Proposition 4.1 are satisfied foreach i E N. Therefore each inner loop terminates after a finite number of stepsand the algorithm generates a sequence {xi}. By construction this sequencebelongs to the level set {x E R" : f(x) :s; f(xO) +2/-Ld which is compact dueto (A6) (cf., e.g., [4, Corollary 20] ). Thus {xi} has an accumulation point andwe have to show that each accumulation point of {xi} is an optimal solution of(P).

Let x* be such an accumulation point of {ii} and let {xij } be a subsequenceof {xi} with limj-too xij = x* . By xj we denote an optimal solution of (PJ.'ij)'Using Lemma 4.1, (4.5) as well as 0 < qij < 1 we obtain

Furthermore, applying Theorem 25 in [4] and the compactness of level sets off , the sequence {xj} has an accumulation point. Without loss of generality weassume that {xj} is already convergent to the limit point x**. Using Theorem25 in [4] again we conclude that x** is an optimal solution of (P) and

Iim min(PJ.'i ') = min(P).J-too }

(4.9)

It is obvious that Ilxi j - xjll oo :s; IIxij - x** 11 00 + IIxj - x**lIoo. The first termof the right-hand side is bounded above, since all xi belong to a compact levelset of f. The second term of the right-hand side is also bounded from above

due to the convergence of the sequences involved. For this reason there existsa constant C with IIxi j - xj 1100 ~ C for all i. Together with (4.8) we have

o~ Illij (xij)

- min(Pll ij) ~ max {1, r~ } (€ij + 38ij) '

In view of Assumptions (2), (3) and (4) we obtain

o~ Iirn (Illi' (xi j) - min(Pll i .)) ~ 0J-too ) )

and from (4.9) follows

(4.10)

Iirn I(xi j) - Iirn j Ili ' (xij)J-too J-too)

j(x*) - min(P) ~ O. (4.11)

In the sequel we show that {xij} is not only a minimizing sequence but converges to an optimal solution of (P). For that we distinguish the two casesmaxtET g(x*, t) < 0 and maxtET g(x*, t) = O. One of these cases must bevalid since M is the closure of Mo and x* is an accumulation point of thesequence {xi} with xi E Mo for all i,

In the first case maxtET g(x*, t) < 0 is assumed to hold . Then (4.10) leadsdirectly to I(x*) = min(P), i.e., x* is an optimal solution of (P).

In the second case we assume maxtET g(x*, t) = O. Due to the continuityof I the equation limj-too I (xij) = I (x*) holds. Thus, regarding (4.10), thelimit point of I-tij In( - maxtET g(xi j , t)) exists and we have

Iim Pi· In (- maxg(xi j, t))J-too) tET

Since maxtET g(x* , t) = 0 there exists a io E INso that maxtET 9(xi j , t) > -1for all i ~ io. Thus we have I-tij In(- maxtET g(xi j, t)) < 0 for all i ~ ioand limj-too I-tij In (- maxtET g(xij, t)) ~ O. In combination with (4.11) wededuce j(x*) = min(P), i.e. x* is an optimal solution of (P) in this case aswell. 0

Remark4.2 Assumptions (3), (4) are a posteriori criteria since we do notknow €i and ri before the inner loop in step i terminates. But, e.g ., (3) can besatisfied if we change it into

(3') limi-too €i,Olr..i = O.

Of course this requires an a priori computation of Li'

If this is not possible we have to run step i of the algorithm with an arbitrary€i,O' When the inner loop terminates, we check whether €dri and 8dri satisfy

a decrease condition, e.g. geometric decrease. If at least one of them does notdo so, we repeat the step with smaller values for £i,O and/or Di. This procedureis finite for fixed i if we control the computation of the radii since the values ofri k can be bounded from below (see Remark 4.1).,

Remark 4.3 Up to now we only considered problems with one semiinfinite constraint. But with some changes we can transfer our method (andthe discussion) to problems like

minimize

s.t.f{x)x ERn, Ax = b,

9i{X,t) ~ 0 for all t E 1i (i = 1, ... ,1).

Thereby we demand, as usual in the convex optimization, A E R m x n withrank{A) = m . Also (A1)-(A9) should be valid in a similar formulation.

5 NUMERICAL ASPECTS

In this section we give some hints for a numerical use of Algorithm 3.1,especially how the required constants can be obtained. At first a possibility forcomputing a radius r > 0 is presented such that the box with radius r and givencentre in Mo is completely contained in Mo. The simplest way to find such aradius is a trial-and-error strategy, whereby only the edges of the considered boxhave to be checked. This procedure can be very costly so that we offer anothermethod in the following lemma. Before it is stated, we define a constant L~ foreach nonempty set 8 c R n with the properties

supsup sup IIvlll ~ L~.tET zES vE8g(z,t)

and 8 c 8' implies L~ ~ L~, . Furthermore , let us denote

Br{V) := {Z E R n: min liz - vlloo ~ r} .

vEV

(5.1)

for r > 0 and nonempty compact sets VeRn.

Lemma 5.1 Let x E Mo and f > 0 be given. Moreover, let h ~ 0 begiven such that

holds. Then the inclusion B; ({x}) c Mo is valid for all r > 0 with

. {A - maxtETh 9{X, t) - Lh}h}r < mm r , LX .

Br({x})(5.2)

(5.4)

(5.3)


Proof. Letz i Mo be given. One has to show IIz-xll oo > r . Ifllz-xll oo ~ fthen this follows immediately. If liz - xlloo < f holds we have

- ~~g(x, t) - L{x}h ~ IvT(x - z)1 ~ Laf({x})llx - zlloo

with v E 8g(z, t*), t* E T(z). Hence, we obtain liz - xll oo > r. D

A corollary allows to compute possible value s of r; in Remark 4.1.

Corollary 5.0.1 Let T E JR, I-L > 0 and f > 0 be given. Moreover, let flow

be a lower bound of f on N* and set

N* := {x E Mo: fll(x) ~ T}.

IfN* =I- 0 the inclusion Br(N*) C Mo is true for all r > 0 with

. {A e~(fIOW-T)}r < mm r, LX .

Bf(N.)

In the following lemma we discuss the calculation of Cs-

Lemma 5.2 Let the assumptions ofLemma 5.1 be fulfilled. Furthermore,let r > 0 be given such that (5.2) is valid. Then (3.3) is fulfilled with

1CBr({x}) := ( A ) Lt h LX .

- maxtETh 9 x, t - {x} - Br({x}{

Proof. From Lemma 5.1 it follows that B; ({x}) c Mo . Let x E B; ({z} ) begiven. Then we infer after a short calculation

- maxg(x, t) ~ - maxg(x, t) - Lt{x}h - La ({x})r > 0tET tETh r

and we have

I 1 I 1~ A t X = CBr({x})·maxtET g(X, t) - maxtETh g(X, t) - L{x}h - LBr({x}{

D

6 NUMERICAL EXAMPLE

In this section we present some numerical results of Algorithm 3.1. Weconsider the design of a perfect reconstruction filter bank as it was done byKortanek and Moulin [13]. Thus we deal with the linear semi-infinite problem

N-I

ro "mnurmze -"2 - L...J X m r2m+1m=O

N-I

s.r. g(x,w) := -1 - 2 L X m cos(2(2m + l)1fw) ~ 0, wE [0,0.5] ,m=O


with given constants r p (p = 1,3, ... , 2N -1)by means of the filter parameters.

Regarding the statements in [13] it is not difficult to verify that (Al)-(A7)are fulfilled. Moreover, using an equidistant grid, (A8) can be simply fulfilled.Thus it remains to specify the constant L~ to ensure that all assumptions hold.But this can be done by use of

N-l \8 IL~ :=41l"maxL(2m+1)lxml2:sup sup 8g

(x ,w) .XE S m=O XES wE[O,O.5] W

Analogously we can set

N-l

1

8 IL~ := 2N 2: sup sup L 8 9 (x ,w) ,

wE[O,O.5] xES m=O x m

which is required for the determination of the boxes. Thus , regarding Lemma5.2, we know all constants to start the algorithm. But if we choose equidistantgrids Th c T with h in accordance to (4.5), it becomes clear that hi,k decreases very fast which results in very large sets Thi,k' Therefore we investigatewhether we can select a (smaller) subset of Th, on which we have to look forthe maximum of g. For that let S := {z E R n : liz - xll oo ~ r} be arbitrarilygiven and maxtETh g(xO , t) with xO E S be known. Then we have

maxg(z, t) 2: maxg(xo, t) - L~lIz .; xOll oo - L~htETh tETh

for all z E Sand g(z, t) ~ g(xO,t) + L~lIz - xOll oo for all t E T after shortcalculations. From both estimates above we obtain that we have to consider alli E Tit with

in order to get an element of Tit (x). That means we can replace Tit by

'it := {l E Tit : 9 (xO, l) 2: maxg(xo, t) - 2L~r - L~h} (6.1)tETh

on S if xO is the centre of S.

Regarding this deletion rule we ran the algorithm. This was done with aC-implementation on a SparcSun-Computer. The arising linear problems aresolved via a Simplex-algorithm and the included quadratic problems are solvedwith a finite method of Fletcher (see [5]).

We consider the same examples as Kortanek and Moulin [13], namely

(6.2)


• AR(l)-process with rk = 0.95k;

• AR(2)-processwithp = 0.975,0 = 1f/3andrk = 2pcosOrk_1-p2rk_2with r = 1 r = 2eCO~O.o , 1 1+e '

• lowpass process with box-spectrum with is = 0.225 and rk = Sin2;7r;;.k .

Thereby we use the origin as starting point and the first barrier parameter 1-'1 = 1in each case. Then the algorithm starts for N = 4 and N = 10 with the additional settings 1-'i+1 = 0.2I-'i' 101,0 = 0.001 , E:i+l,O = 0.15E:i' Oi = 100000E:i,0,qi = 0.999 and, applying Lemma 5.1,

. _ . { - maxtETh g(xi,k, t) - L{Xi'k}h}r1,k - 0.9 mm 1, LX

Bl({xi ,k})

with h = hi ,k-1 if k > 0 or h = 0.0005 if k = O. Furthermore, regardingRemark 4.2, it is possible that the values of E:i,O and Oi are getting smaller duringthe i-th step which leads to a restart of the respective step. Additionally allvalues hi,k are computed as minimum of 0.0005 and the maximal value whichfulfills (4.5). Hence, the constants CS are given by Lemma 5.2. Then we obtainfor N = 4 the results summarized in Table 8.1. With respect to precision these

Process Xl X2 X3 X4 Coding gain

AR(l) 0.612048 -0.149279 0.045733 -0.008533 5.860

AR(2) 0.594990 -0.193611 0.059889 -0.042127 6.069

box-spec 0.613735 -0.169685 0.072194 -0.026933 4.884

Table 8.1 Results for N = 4

results are comparable to those of [13]. Furthermore, considering N = 10, weget the coding gains 5.943 for the AR(l)-process, 6.833 for the AR(2)-processand 9.869 for the box-spectrum . Again the precision is comparable to the resultsof Kortanek and Moulin [13].

Finally, let us have a closer look at the efficiency of the deletion rule. Forthat we consider exemplary the box-spectrum case for N = 10 in detail. Weobserve the values given in Table 8.2 where the second row shows the minimalcomputed h during the i-th step and the third row gives the average h-valueof this step. Further the last row contains the average ratio of IThi,k 1/IThi,k Iduring step i. These values show that our deletion process works very well inthe given example. A similar behaviour can be also observed in the other cases.


J1.i 2.0E -1 4.0E-2 8.0E-3 1.6E-3 3.2E-4 6.4E-5 1.3E-5

mink hi,k 5.0E-4 5.0E-4 5.0E-4 1.1E-4 3.1E-5 5.8E-6 1.1£-6 1.9E-7

ha v 5.0E-4 5.0E-4 5.0E-4 3.9E-4 7.2E-5 1.2E-5 1.7E-6 2.8E-7

'h/Th 1.00 0.83 0.47 0.42 0.41 0.22 0.08 0.04

Table 8.2 The deletion rule in the box-spectrum-case for N = 10

7 A REGULARIZED LOG-BARRIER METHOD

Up to now we considered semi-infinite problems under the Assumptions(Al)-(A9). In the sequel we want to replace (A6) by the weaker assumption

(A6') the set M opt = {x EM: j(x) = infzEM j(z)} is nonempty,

i.e., we do not claim compactness of the solution set which is essential in thefirst part (directly used in the basic framework of Fiacco and McCormick aswell as in the proof of Theorem 4.1). Therefore we have to look for a methodto treat semi-infinite problems under the changed assumptions, which uses asmuch as possible our former results.

We achieve this by applying the proximal point technique (see, e.g., Kaplanand Tichatschke [9]) so that we consider

(Prox) Minimize j(x) + ~ IIx - all~ s.t. x E M

with various prox-parameters s > 0 and various proximal points a . These problems are also semi-infinite problems, but they fulfill the Assumptions (Al)- (A9)since (A 1)-(A5) and (A7)-(A9) can be transferred from the original problemand (A6) is enforced by the additional quadratic term. Therefore we could useAlgorithm 3.1 for solving them. Due to the fact that the proximal point methodis a tool for approximating a given problem by a sequence of uniquely solvableproblems, we have to solve a sequence of problems of type (Prox) . But asAlgorithm 3.1 typically terminates with only an approximate solution anywaythere is little sense in solving each auxiliary problem of type (Prox) with ashigh an accuracy as possible. Particularly we will only realize one step of Algorithm 3.1 to compute an approximate optimal solution of (Prox) with fixedbarrier parameter, which is then used as the new proximal point. A practicalrealization of such a step requires the predetermination of the barrier and theprox parameter. From the classical logarithmic barrier approach it is knownthat the barrier parameter has to converge to zero, e.g. by reducing it from step

to step. But, due to the fact that the conditioning of the barrier problems isgetting worse with decreasing the barrier parameter, it makes sense to keep thisparameter fixed for a couple of steps. In order to permit a dynamical control,the choice of the barrier parameter is made dependent on the progress of theiterates in the last step. To avoid side effects which can influence this choice,we keep the prox-parameter 8 constant as long as the barrier parameter is notchanged. Merely the proximal point is updated more frequently. Altogether weobtain a multi-step-regularization approach (cf., e.g., Kaplan and Tichatschke[9]).

Algorithm 7.1

• Give J-t1 > O. xO E Mo. 0"1 > 0 and 81 with 0 < §. ~ 81 ~ S.

• For i := 1,2, ... :

* Set xi,o := x i- 1.

* For j := 1, 2, . .. :

- Select eij > O. setxi,j,O := xi,j-1 and define Fi ,j : Mo -+ lRby

Fij(X) := f(x) - J-ti In (- maxg(x, t)) + 8i IIx - xi,j-11l~ ., tET 2

(7.1)

- For k := 0, 1, .. . :a) Select Ti,j,k > 0 such that

Si,j,k := {x ERn: IIx - xi,j,klloo ~ Ti,j,k} c Mo.

b) Select hi,j,k ~ 0 and define Fi,j ,k : Mo -+ R by

Fi ,j,k(X) := f(x) - J-ti In (- max g(x, t))tEThi,i,k

+~ IIx - xi,j-11l~ ·

c) Select f3i j k ~ 0 and compute an approximate solutionxi,j,k+l ~j the problem

• .. D ( ) Si j kmtmmize L'i,j x s.t. x E ' ,

such that

- . . k+l Ei jFi ,j,k(Xt

,) , ) - mink Fi ,j(X) ~ -2' + 2f3i,j,kXES· ,J,

(7.2)

and p." .k(Xi,j ,k+l) < p'. .k(Xi ,j,k) are true.1,), _ 1,) ,

d) If

F:" (xi,j ,k) _ fr·· (x i,j,k+l) < Ci,j (7.3)1,),k 1,),k - 2

then set xi,j := xi,j,k, Si ,j := Si,j,k, ri,j := ri,j,k. stopthe loop in k, otherwise continue the loop in k.

- Ifllxi,j -xi ,j - 1 1l2~ (ji then set xi := xi,j ,ri := ri,j,j(i) :=

j and stop the loop in i. otherwise continue the loop in j.

* Select 0 < J.1.i+l < J.1.i , 0 < §. ~ Si+l ~ sand (ji+l > O.

We start the discussion of this algorithm with a short investigation of theauxiliary problems (~,j,k)' As in Algorithm 3.1 we want to solve these auxiliaryproblems with the bundle method from Section 2. Using the notes in Section3 it is easy to check that the assumptions of this bundle method are fulfilledif Fi,j,k is used as an approximation of Fi,j . Moreover, since the loop in kin the method above is the same as in Algorithm 3.1 with a slight change inthe stopping criterion, Lemma 3.1 is simply transferable to the new situation.Thus J.1.iL~i,j,,,CSi.i,,,hi ,j,k is an upper bound for the error which is caused

by using Fi,j,k instead of Fi ,j. Consequently we do not have to select (3iJ,kexplicitly since we can use (3i ,j,k = l-liL~i ,i ," CSi,i,"hi ,j,k with the predefinedhi ,j,k. Additionally in the case of exact determination of maxtET g(x, t) wecan set hi,j,k = (3iJ,k = 0 analogously to Remark 3.1

Now let us have a closer look at the loop in k, Analogous to Proposition 4.1the following result holds.

Lemma 7.1 Let i,j be fixed. Let 6i,j > 0 and qi,j E (0,1) be given . If

H·Lt C """h '"k < (3'"k < qk .6" .,-1 Si ,i," S"]' 1,), - 1,), - i,) 1,) (7.4)

for all k, then the loop in k ofAlgorithm 7.1 terminates after a finite number ofsteps.

With Fi,j(x) defined in (7.1) we denote

(R .)1,) minimize ~,j(x) s.t. x E Mo.

Lemma 7.2 Leti,j be fixed and xbe the optimal solution of(Pi,j). Moreover, let xiJ,k and xi,j,k+l be generated by Algorithm 7.1. Additionally, (7.4)holds. If

- " . k - .. k 1 ciJR "k(X 1,) , ) - R .k(X 1

,) , + ) < -1,), 1,) , - 2 (7.5)

then

0< F:- o(Xi,j ,k) - F:- o(x ) < max {I IIxi,j,k - Xll oo } (s, 0+ 3,800k) (7.6)

- Z,J Z,J - 'To 0 z,J z,J,z,J,k

and

2(E: ' . + 3,80 0k) 2(E: o0+ 3,8' 0k) }z,J z,J, z,J z,J,, .Si SiTi,j,k

(7.7)

Proof. Inequality (7.6) can be shown analogously as (4.3) in the proof ofLemma 4.1, so only .the second inequality needs to be proven.

Due to the strong convexity of Fi,j we have

(7.8)

and combined with (7.6) we deduce

Si Ilx _ xiJ,kl12 < max {I IIxi,j,k - xlloo} (E: o . + 3,80' ).

2 2-, Z,J z,J,kTiJ,k

At this point we distinguish two cases. We first suppose TiJ,k ~ IIxi,j,k - xlloo.Then it holds

2(E:iJ + 3,8i ,j,k)

Si

and (7.7) is valid in our first case.In the second case the inequality 1 < IIxi,j,k - xlloo/Ti,j,k is supposed to be

true. Then

Si Ilx _ xiJ,kl12 < IIxi,j,k

- xll oo (s, .+ 3,80' ) < IIxiJ

,k - xll2 (E: ' . + 3,800 )2 2 - Z,J z,J,k - Z,J z,J,k

~~ ~~

is valid. Hence, we infer

such that (7.7) is also valid in the second case. 0

Let us denote the Euclidean ball with radius r around Xc by Kr(xc ) . Thenwe can formulate a convergence result which is closely related to Theorem 1 inKaplan and Tichatschke [11], although the use of the discretization procedureand the logarithmic barrier method are here in reverse order.

Theorem 7.1 Let r ~ s.and x; E /RnbechosensothatMoptnKT/8(xc) i=0. Let x* E Mopt n K T/8(XC ) ' x E Mo n KT(xC ) and xO E Mo n K T/4(XC )

be fixed. Moreover, let 8i,j > 0, qi,j E (0, 1) and ai > 0 be given and assumethat (7.4) is true for all (i, i. k). Furthermore, assume that the controllingparameters of the method satisfy the following conditions:

(7.9)

(7.10)

(7.11)

and

(7.12)

Then

(i) the loop in k is finite for each (i,j);

(ii) the loop in j is finite for each i, i.e. j (i) < 00;

(iii) IIxi ,j - xc ll2< T for all (i,j) ; and

(iv) the sequence {xi} converges to x** E Mopt n KT(xC ) '

Here the constants are:

c~lIx-x*1I2 and c3:=f(x)-f_+cO+Cl

with f _ :s min(P),

co:= lIn (-%¥g(x, t)) I,and vE 8g(x, l), l E T(x).

ci := In (- maxg(x, t) + 2 11v1l2)tET

Proof. Our first proposition follows immediately from Lemma 7.1. The otherpropositions can be proven similarly to the proof of Theorem 1 in Kaplan andTichatschke [11]. But there are some essential estimates included which are deduced from arguments based on the differentiability of the involved functions .

Thus in the sequel we state analogous results which do not require differentiability.

Let us define

xiJ := arg min p,. .(x)xEM ia and :tJ := arg min {f(x) + Si IIx _ xi,j-lll~} .

xEM 2

Then, using the stopping criterion of the loop in k of Algorithm 7.1, (7.4), (7.7),(7.9) as well as the definition of xi,j, we infer

(7.13)

so that we obtain

f (xi,j) + ~ Ilxi,j - xi,j-lll~ - f (¥J) - ~ II¥J - xi,j-111: :s; Piwith (4.1). Furthermore, due to the strong convexity of f(x) +¥lIx- xi,j-lll~,

one has

~ Ilxi,j - ¥JII: :S;f (xi,j) + ~ Ilxi,j - xi,j-lll~ - f (:t,j) -II¥J - xi,j-111:analogously to (7.8). From both inequalities above it follows

S'II .'112d xi,j - ¥,) 2:S; Pi resp.

Combining this with (7.13) we obtain

(7.14)

Now, using (7.13) instead of (21) and (7.14) instead of (30), the proof can becompleted analogously to the proof of Theorem 1 in [11]. 0

Remark 7.1 The conditions on the parameters of the method require theirseparate adjustment to each example, which can be a fragile task when applyingthe multi-step procedure. Without this, parameters according to Theorem 7.1are easily chosen . At first (7.12) is automatically satisfied for each fixed T ~ 1if (Ii is sufficiently large. Furthermore, (7.11) holds for all T sufficiently largeif one guarantees that

00

L:ai < 00.

i=lConsequently (7.11) and (7.12) can be replaced by the given conditions aboveand T, (Ii need not to be specified explicitly.


8 NUMERICAL RESULTS OF THE REGULARIZEDMETHOD

In this section numerical results computed by Algorithm 7.1 are presented.For that we consider for fixed nand k E {1,... ,n - 2} the approximationproblem

min. Xn+2n

s.t. g(x, t):= ep(t) - L xmtm- l - Xn+l (tk + tk+1) - Xn+2 ~ 0m=l

for all t E [-1,2]

with

._ { tn ift E [-1 ,1]ep(t) .- max{1, t" - Pn(t)} ift E (1,2]

and Pn the normalized Chebyshev polynomial of degree n of the first kind.That means we want to approximate ip on [-1,2] by a polynomial based onthe functions 1, t, ... ,tn- l and tk + tk+l which are not linearly independenton [-1, 2]. Caused by this fact, the solution set is unbounded if it is nonemptyso that (A6) does not hold and we cannot use the unregularized algorithm forsolving this problem. Thus let us have a look at the weaker assumptions fora use of Algorithm 7.1. Regarding that (A3) holds due to Theorem 5.7 inRockafellar [17] the validity of (A 1)-(A5) as well as (A7) can be easily shownwhile we choose again an equidistant grid on T to fulfill (A8) . (A6') holdssince one optimal solution is given if Xl , ... , X n are the negative values of thewell-defined coefficients of the normalized Chebyshev polynomial Pn exceptthe leading coefficient 1, Xn+l = 0 and Xn+2 = 21- n . Thus it remains todetermine the constant L~, which is much more difficult than in the case ofthe filter problems considered in Section 6. Therefore we divide the interval[-1,2] into the two intervals [-1,1] and (1,2]. On [-1,1] one can use thedifferentiability of ep(t) - L~=l xmtm- l - Xn+l(tk + tk+l) w.r.t. t, so thatit is possible to determine a constant which fulfills (3.1) for all xES andall tl, t2 E [-1,1] by maximizing the absolute value of each addend of thederivative separately. Considering the interval (1,2] instead of [-1, 2] one hasto determine two constants, one for each possible constraint function. The casemax{1, tn - Pn(t)} = 1 leads to a polynomial such that it can be treated asit is done on [-1,1]. In the case max{1,t" - Pn(t)} > 1 we can use thewell -known recurrence scheme of the Chebyshev polynomials so that in factwe deal with a polynomial again. Altogether we can use the greatest of bothcomputed constants as L~ on (1,2]. Then the sum of the constants of both partsgives us L~ valid on [-1,2].


Consequently Algorithm 7.1 can be applied to solve the given approximationproblem. The radii are computed with Lemma 5.1 thus requiring the constantL~. Since cp(t) - ~~=1 xmtm- 1 - X n+1 (tk + tk+l) is linear in x we can

set L~ = 2 + maxtE[-1,2](~~~\ Itml + Itk + tk+ll). In order to ensureconvergence of the computed iterates, we have to adapt the parameters of themethod such that the assumptions of Theorem 7.1 are fulfilled .

We consider the case n = 6 (thus x E R 8 ) and k = 2 as example. Since theadaptation of the parameters is easier we do not want to use the multi-step technique so that the constants a; and r do not have to be specified in accordancewith Remark 7.1. Then the most difficult problem is to find a starting barrierparameter J.L1 which fulfills (7.10) with some x E Mo . But this problem canbe solved by running a few steps of the algorithm without multi-step but withparameters fulfilling (7.4). Then Lemma 7.1 ensures the finiteness of each step.In the example case we repeated a few steps with fixed barrier parameter J.L = 1and e = 0.001, 8 = 0.01 and xO = ... = x~ = 0, xg = 49. Further the radiiare computed by (6.2) with h = hi k-1 if k > 0 or h = 0.003 if k = 0, while,the constants CS are given by Lemma 5.2 again. The grid constants hi ,j,k arecalculated as large as possible by (7.4). After three steps we stopped this procedure whereby we merely changed the prox point and the accuracy parameter efrom step to step. The obtained result was used as starting point for Algorithm7.1 with barrier parameter J.L1 = 0.05 and the additional settings J.Li+1 = 0.3J.Li,£10 = 0.001, £i+l 0 = 0.25£i, bi = 10000£i 0, qi = 0.999, 81 = 0.01 and8i:1 = max{O.OOOOl, 0.58i} . Furthermore, th~ radii, the constants Os and thegrid constants hi,j,k are computed as mentioned above so that the assumptionsof Theorem 7.1 are fulfilled if we additionally use a similar restart procedureof particular steps as it is described in Remark 4.2 for Algorithm 3.1. Thenwe obtain the results summarized in Table 8.3 and the approximate solutionx = (0.031250,0, -0.380629,0.181871,1.500000,0, -0.181870,0.031261)with a distance of about 1.09E-5 to the solution set. Table 8.3 shows again theeffectivity of the deletion rule presented in Section 6. Addit ionally we noticethat the radius value is getting very small. This is typical for most problems sincethe optimal solutions are often located on the boundary of the feasible region.Moreover, since G:i depends directly on the fraction £dri and 8i is boundedaway from zero the assumptions of Theorem 7.1 enforce a faster convergenceof £i to zero . Thus, in regard to the machine precision, it is not recommendedto compute results for too small parameter values .

The last three columns of Table 8.3 contain some information which allow toestimate the computational effort of the method. The #LP-column contains thenumber of solved linear minimization problems and the #QP-column containsthe number of solved quadratic minimization problems. Both kinds of problemsarise in the bundle method presented in Section 2. The number of solved


J!.i d2(Xi, Mopt} Ti minh · ' k hay 11\1/IThl #LP #Qp #BPi,k ',J ,

5.0E -2 2.68E-l 8.6E -4 3.0E -3 3.0E -3 1.00 347 547 342

1.5E-2 1.52E-2 9.6E-5 6.7E-6 1.4E-3 0.46 1289 3445 809

4.5E-3 4.51E-3 2.7E-5 4.0E-6 1.7E-4 0.56 365 1108 210

1.4E-3 l.35E-3 8.2E -6 2.6E-5 5.5E -5 0.28 383 1113 207

4.1E-4 4.05E-4 2.5E -6 4.6E-6 1.4E-5 0.15 384 1151 199

1.2E-4 1.22E-4 7.8E -7 1.7E-6 3.7E-6 0.08 397 1213 192

3.6E-5 3.66E-5 3.0E -6 4.5E -7 l.lE-6 0.04 331 1018 145

l.1E-5 1.09E -5 9.OE-7 1.4E-8 3.3E -7 0.04 84 325 24

Table 8.3 Results for n = 6, k = 2

quadratic problems is equal to the number of inexact maximizations. Finally,in the #BP-column the number of considered boxes in step i is given. Thelarge number of successively considered boxes (and consequently of solvedlinear and quadratic problems) for barrier parameter 0.015 is caused by the factthat there occur two restarts in this step. And these restarts are induced byinsufficient accuracy values at this point.

9 CONCLUSIONS

We have shown that both proposed logarithmic barrier methods produce sequences leading to an optimal solution of the given convex semi-infinite problem. This is demonstrated with some numerical examples.

Moreover, we can prove similar results as Kaplan and Tichatschke [11] for therate of convergence (see Abbe [1]). The influence of the algorithm parameterson the performance of the methods is not analyzed here. This, combined withan extensive test of many more examples including real world problems, willbe future work.

Acknowledgements

(i) This research has been supported by the "Deutsche Forschungsgemeinschaft", through the Graduiertenkolleg "Mathematische Optimierung"at the University of Trier.

(ii) The author wish to thank the referee for the helpful comments and hints.


References

[1] L. Abbe. Two logarithmic barrier methods for convex semi-infinite programming problems, Forschungsbericht 99-30, Universitat Trier, 1999.

[2] M. C. Ferris and A. B. Philpott. An interior point algorithm for semiinfinite linear programming, Mathematical Programming, A43 :257-276,1989.

[3] A. V. Fiacco and K. O. Kortanek, editors. Semi-Infinite Programmingand Applications, University of Trier, Volume 215 of Lecture Notes inEconomics and Mathematical Systems, Springer, 1983.

[4] A. V. Fiacco and G. P. McConnick. Nonlinear Programming: SequentialUnconstrained Minimization Techniques, Wiley, 1968.

[5] R. Fletcher. A general quadratic programming algorithm, Journal of theInstitute ofMathematics and its Application, 7:76-91, 1971.

[6] K. R. Frisch. The logarithmic potential method of convex programming.Technical report, University Institute of Economics, Oslo, 1955.

[7] R. Hettich (editor). Semi-Infinite Programming, Volume 15 of LectureNotes in Control and Information Sciences, Springer, 1979.

[8] J.-B. Hiriart-Urruty and C. Lemarechal. Convex Analysis and Minimization Algorithms II, Springer, 1993.

[9] A. Kaplan and R. Tichatschke. Stable Methods for Ill-posed VariationalProblems, Akademie-Verlag, Berlin, 1994.

[10] A. Kaplan and R. Tichatschke. Proximal Interior Point Approach forConvex Semi-infinite Programming Problems. Forschungsbericht 98-09,Universitat Trier, 1998. Submitted to Optimization Methods & Software.

[11] A. Kaplan and R. Tichatschke. Proximal Interior Point Approach in Convex Programming, Optimization, 45:117-148, 1999.

[12] K. C. Kiwiel. Proximal level bundle methods for convex nondifferentiableoptimization, saddle-point problems and variational inequalities. Mathematical Programming, A69:89-109, 1995.

[13] K. O. Kortanek and P. Moulin. Semi-infinite programming in orthogonalwavelet filter design, In Reemtsen and Riickmann [16], pages 323-360,1998.

[14] B. Martinet. Regularisation d'inequations variationelles par approximations successives, Revue Francoise d'lnformatique et de RechercheOperationnelle, 4:159-180, 1970.

[15] E. Polak. On the mathematical foundations ofnondifferentiable optimization in engineering design, SIAM Review, 29:21-89, 1987.


[16] R. Reemtsen and J.-J. Riickmann (editors). Semi-Infinite Programming.Nonconvex Optimization and its Applications, Kluwer, 1998.

[17] R. T. Rockafellar. Convex Analysis. Princeton University Press, Princeton,1972.

[18] U. Schattler, An interior-point method for semi-infinite programmingproblems, Annals ofOperations Research, 62:277-301, 1996.

[19] G. Sonnevend. Applications of analytic centers for the numerical solution of semi infinite, convex programs arising in control theory. In H.-J.Sebastian and K. Tammer, editors, System Modelling ana Optimization,Volume 143 of Lecture Notes in Control and Information Sciences, pages413-422, Springer, 1990.

[20] G. Sonnevend. A new class of a high order interior point method for thesolution of convex semiinfinite optimization problems. In R. Bulirsch andD. Kraft, editors, Computational ana Optimal Control, pages 193-211,Birkhauser, Basel, 1994.

[21] M. J. Todd. Interior-point algorithms for semi-infinite programming,Mathematical Programming, A65:217-245, 1995.

[22] M. H. Wright. Interior methods for constrained optimization, Acta Numerica, 341-407, 1992.

Chapter 9

FIRST-ORDER ALGORITHMS FOROPTIMIZATION PROBLEMS WITH A MAXIMUMEIGENVALUE/SINGULAR VALUE COST AND ORCONSTRAINTS

Elijah PolakDepartment ofElectrical Engineering and Computer Sciences, University of California,

Berkeley, CA 94720, USA

[email protected]

Abstract Optimization problems with maximum eigenvalue or singular eigenvalue costor constraints occur in the design of linear feedback systems, signal processing,and polynomial interpo lation on a sphere. Since the maximum eigenvalue of apositive definite matrix Q(x) is given by maxllylI=l (y, Q(x)y), we see that suchproblems are, in fact, semi -infinite optimization problems. We will show that thequadratic structure of these problems can be exploited in constructing specializedfirst-order algorithms for their solution that do not require the discretization ofthe unit sphere or the use of outer approximat ions techniques.

1 INTRODUCTIONOptimization problems with maximum eigenvalue or singular eigenvalue

cost or constraints occur in a number of disciplines . For example, in the designof linear feedback systems , the suppression of disturbances can be modeled asthe minimization of the norm of the disturbance transmission transfer functionmatrix Gd(X,jW) over a specified range of frequencies, where x E lRn is thedesign vector and wEIR. is a frequency variable (see; e.g., [3], [1D. Since thenorm of Gd(x, jw) is its maximum singular value, we see that the minimizationof this norm can be expressed as the semi-infinite optimization problem:

minmax max (y, Q(x ,jw)y) ,xEX wEn Ilyll=1

197

M.A. Goberna and M.A. Lopez (eds.), Semi-InfiniteProgramming, 197-220.© 2001 Kluwer Academic Publishers.

(1.1)


where X c jRn is a constraint set, n = {WI, ... ,WN} is a grid of frequencies, Q(x,jw) = Gd(x,jW)*Gd(x,jw), and Gd(x,jw)* denotes the complexconjugate transpose of Gd(x, jw).

In a wide variety of signal processing applications, such as beam forming[5] and radar imaging [8], it is desirable to form an estimate of a covariancematrix from samples of a process. The true covariance matrix is known tobe positive definite and Hermitian Toeplitz. Let {Zl ... ZM} denote the set ofN x 1 observation vectors of the process I . Then the sample covariance matrixS is given by

(1.2)

where z" denotes the complex conjugate transpose of z. The desired estimateis then the N x N positive definite Hermitian Toeplitz matrix Rgiven by

R= arg max {-In(det R) - tr(R- 1S)} ,RE1I'+

(1.3)

(1.5)

(1.6)

where T" is the set of all N x N positive definite Hermitian Toeplitz matrices,and trt-) is the trace operator.

Now, any Hermitian Toeplitz matrix R can be parametrized in terms of a pairof vectors x = (x R, x r) E jRN X lRN as follows:

N

R(x) = L (X~QR,k + j X1Qr,k) , 0.4)k=l

where [xk + jx}, ... ,x~ + jxf] is the first row ofR (with real and imaginaryparts shown explicitly), and QR,k and Qr,k (k = 1, ... , N) are symmetric andskew symmetric matrices respectively.

Hence, problem (1.3) can be recast as a constrained semi-infinite optimization problem, as follows:

arg max { -In (detR(x)) - trR(x)-IS IX=(XR,Xl )ElRn x lRn

minllYII=1 (y, R(x)y) 2: o > 0, andN

R(x) = L (X~QR,k + j x1QI,k) } .k=l

The constrained maximum likelihood covariance estimate is then given by

N

R = L (X~QR,k + j X1Qr,k) .k=l

FlRST-ORDERALGORIGHMS FOR A CLA.SSOF OPTIMlZA110NPROBLt-MS 199

Our final example comes from the problem of choosing points on the unitsphere that minimize a bound on the norm of the polynomial interpolationoperator [15], [12]. This bound is minimized by finding the m points on theunit sphere S2 in 1R3 which maximizes the smallest eigenvalue of a symmetricGram matrix G, which is a nonlinear function of the angles between these points.For polynomials of degree at most p, a fundamental system of points on the unitsphere in JR3 consists of (p+1)2 points for which the only polynomial of degreeat most p vanishes at all points is the zero polynomial. For any fundamentalsystem, G is a symmetric positive definite m x m matrix where m = (p+ 1)2.Since a fundamental system can be parametrized in terms of a vector x E IRn ,

where n = 2m - 3, the problem of maximizing the smallest eigenvalue of Gcan be expressed as the following semi-infinite min-max problem:

max min (y, G(x)y) = - min max (y, -G(x)y). (1.7)xEIRn Ilyll=l xEIRn Ilyll=l

In this paper we will present two new specialized algorithms for the types ofproblem described above. These algorithms appear to have serious advantagesover existing algorithms (such as those described in [7], [14], for example, ormethods of outer approximations in [11]) when the matrix in the quadratic formis large (say at least 500 x 500)2.

2 SEMI-INFINITE MIN-MAX PROBLEMS

We begin by considering problems of the form

min'if;(x),xEIRn

where

'if;(x) = maxcp(x,y),yEY

(2.1)

(2.2)

where Y C IRffi is compact and ip : IRn x IRffi -7 IR is continuously differentiable. In particular, we will consider the case where

cp(x, y) = (y, Q(x)y), Y = {y E IRffi Illyll = 1}. (2.3)

Referring to [11, Section 3.1], we find a first-order optimality condition for(2.1) in terms of the set-valued map

~ { ( 'if;(x) - cp(x, y) ) }G'if;(x) = convYEY ,

\7xcp(x, y)(2.4)


where cony denotes the convex hull of the indicated set.

Theorem 2.1 ([11, p. 373)) Suppose that:

(a) The set-valued map G'ljJ(x),from jRn to the subsets ofjRn+l. is continuous in the Painleve-Kuratowski sense.

(b) Let the elements ofjRn+ 1 be denoted by e~ (~O, O. with ~o E lR, and.with 0 > O. let

let the optimality function

f:::" -O(x) = - _min q(O,

(,EG'ljJ(x)

and let

(2.5)

(2.6)

h(x) = (ho(x), h(x)) ~ - arg _ min q(e). (2.7)(,EG'ljJ(x)

Then

(i) The functions 0(-) and h(·) are continuous. and. for all x E jRn.O(x) ~ O.

(ii) For any x E jRn. the directional derivative satisfies the relation

sd'ljJ(x; h(x)) ~ O(x) - "2l1h(x) 11

2. (2.8)

(iii) Ifx E jRn is a local minimizer of'ljJ(.). then

oE G'ljJ(x), O(x) = 0, and h(x) = O. (2.9)

Furthermore. (2.9) holds if and only if 0 E 8'ljJ(x). where 8'ljJ(x)denotes the subgradient of'ljJ(·) at x.

We can now state formally the obvious generalization of the PshenichnyiPironneau-Polak Algorithm [10] (see Algorithm 2.4.1, p. 223, in [II)), whichsolves finite min-max problems. This algorithm has the following, non-implementable form for the problem (2.1).

Algorithm 2.1 Generalized Pshenichnyi-Pironneau-Polak Algorithm

Parameters. a, f3 E (0,1). 0> O.

FlRST-ORDERALGORIGflMS FOR A CLASS OF OPTIMIZATION PROBLEMS 201

Data. Xo E Rn .

Step O. Set i = 0.

Step 1. Compute ()i = ()(xd and hi = h(Xi)'

Step 2. If ()i = 0. stop. Else. compute the step-size

Ai = TEa:{{3k I '!f;(Xi + {3khd - '!f;(Xi) - {3ka()i ~ a}, (2.10)

where N = {a, 1,2,3, ...}.

Step 3. Set

(2.11)

replace i by i + 1 and go to Step 1.

The reason by which the Algorithm 2.1 is non-implementable for the problem (2.1) is that neither ()(Xi) nor h(Xi) can be computed exactly in reasonabletime. To obtain an implcmentable version of Algorithm 2.1, we must modify itso as to be able to use approximations to ()(Xi) and h(Xi). We will now developsuch an implementation which makes sense when cp(x,y) is of the form (2.3)and, possibly, a few other cases as well. The success of the new algorithmdepends on the following observation:

Theorem 2.2 Suppose that x E lRn is such that °rt G'!f;(x). 'Y E (0,1).and ~* E G'!f;(x). ~** rt G'!f;(x) are such that

(a) ~~* > 0;

(b) (\7q(~**), ~ - ~**} ~ OJor all ~ E G'!f;(x);

(c) q(~*) - q(~**) ~ 'Yq(~**).

Then.

(i) q(~**) ~ -()(x) ~ q(~*);

1 - .(ii) -l+-Y()(X) ~ q(~**) ~ - () (x ),

(iii) with h** = -~**.

- l:.'!f;(x;h**) = maxYEY{[cp(X, y) - '!f;(x)] + (\7xcp(x, y), h**)}

(2.12)


and

Proof. (i) Clearly, since q(.) is convex, it follows that

~** = argIpinq(~),~EH

where

(2.14)

Now, it follows from assumption (b) that G'ljJ(x) c H, and hence that

q(~**) = Ipinq(~) ::; -O(x) = _min q(~)::; q(~*), (2.16)~EH ~EGt/J(x)

which proves (i).

(ii) It follows from assumption (c) and (2.16) that

q(~**) ~ -O(x) -1'q(~**).

Hence it follows directly that

1 --1 +l'O(x) ::; q(~**) ::; -O(x),

(2.17)

(2.18)

which proves (ii).

(iii) Letp > II~**" be such that G'ljJ(x) c B(O,p) c Rn+1 (whereB(O,p) ~{~III~II ::; p}), and let

Then,

S=HnB(O,p). (2.19)

FlRST-ORDERA LGORIGHMS FOR A CLASS OF OPTIMIZATIONPROBLEMS 203

because the unconstrained min above yields

and, by definition, tu; = -~~**.

Now, with h** as above,

(2.21)

which proves (2.12).~ £::,.

Next, let Y(x) = {y E Y Icp(x, y) = 1/J(x)}. Then we see that

d1/J(x,h**) + ~lIh**1I2 = ml'tX (V'xcp(x,y),h**) + ~lIh**1I2YEY(X)

8 2= ml'tX ([cp(x, y) - 1/J(x)] + (V'xcp(x, y), h**}} + 2"lIh**1I

YEY(X)

::; max{[cp(x,y) - 1/J(x)] + (V'xcp(x ,y),h**}} + ~2I1h**112yEY

::; -q(~**),

which proves (2.13). D

(2.23)

Later, we will show that given a point x E IRn , points ~*' ~** as specifiedin Theorem 2.2, can be computed using either the Frank-Wolfe Algorithm [4],or the much more efficient Higgins-Polak Algorithm [6]. We will also showthat for the case where cp(x, y) and Yare defined as in (2.3), these algorithmscan be efficiently implemented using the fact that eigenvalues of a symmetricpositive-semidefinite matrix are relatively easy to compute. However, first westate an implementable modification of Algorithm 1 which uses such points:

Algorithm 2.2 Modified Generalized Pshenichnyi-Pironneau-Polak Algorithm

Parameters. a,{3" E (0,1),6> O.

Data. Xo E IRn .

Step O. Set i = O.

Step 1. Use the Higgins-Polak or Frank-Wolfe algorithm to compute a ~*i E

G'l/J(xd and a ~**i ¢ G'l/J(Xi) such that

(a) ~~*i > O.

(b) (V'q(~**d ,~ - ~**i) ~ O,Jorall ~ E G'l/J(xd,

(c) q(~*i) - q(~**d ~ 'Yq(~**d,

Step 2. Compute the step-size

k k k -Ai = max{,8 I 'l/J(Xi +,8 hi) - 'l/J(xd +,8 aq(~**d ~ O}. (2.24)

kEN

Step 3. Set

(2.25)


Lemma 2.1 Suppose that V'xcp(" .) is Lipschitz continuous on boundedset.i3 and that x E lRn is such that O(X) < O. Let p > 0 be such that O(x) ~0(x)/2,Jor all x E B(x, p). Then there exists a Lipschitz constant L E [6,00)such that.for all Xi E B(x , p), Ai is well defined by (2.24) and satisfies Ai ~,86/L.

Proof. Suppose that O(X) < 0 and that p > 0 is such that O(x) ~ 0(x)/2, forall x E B(x, p). Then, because the search directi ons h, computed by Algorithm2.2 for x E B(x, p) are uniformly bounded, there exists a Lipschitz constantL E [6,00) such that

IIV'xcp(x + sh(x) ,y) - V'xcp (x ,y) II ~ sLllh(x) II ,

for all x E B(x, p), Y E Y, and s E [0,1]. Hence , for any Xi E B(x, p) andAE [0,6/L],

'l/J(Xi + Ahi) - 'l/J(xd = max{[cp(xi ,y) - 'l/J(xd]yEY

+A(V'cp(Xi , y), hi) + Afo1 (V'CP(Xi + sAhi ,y) - V'CP(Xi ,y) , hi)ds}A2L

~ max{[cp(xi , y) - 'l/J(xd] + A(V'CP(Xi , y), hi)} + -2-lIhi 112

(2.26)yEY

~ Amax{[cp(xi ,y) - 'l/J(xd] + (V'CP(Xi , y), hi)} + -26 II hi 11 2

yE Y

~ -Aq(~**i)'

FIRST-ORDER ALGORIGHMS FOR A CLASS OF OPTIMIZATION PROBLEMS 205

Hence, for all AE [0, min{1, 81L }],

(2.27)

from which we deduce that the step-size satisfies Ai ~ {381L, which completesour proof. 0

To prove that Algorithm 2.2 is convergent, we will show that it has the Monotone Uniform Descent (MUD) property (see [11, p. 21]) and then make use ofTheorem 1.2.8 in [11].

Lemma 2.2 Suppose that 'Vxrp(·, .) is Lipschitz continuous on boundedsets and that x is such that O(X) < O. Then there exists a p > 0 and a r;, > O.such that.for all Xi E B(x, p) and Xi+l constructed byAlgorithm 2.2,

(2.28)

Proof. Let p > 0 be such that O(xd ~ O(X)/2, for all Xi E B(x, pl. Then itfollows from Lemma 2.1 that there exists an L E [8, 00) such that Ai ~ (381L,for all Xi E B(x, pl. Making use of this fact and of part (ii) of Theorem 2.2,we conclude that, for all Xi E Biii; p),

'I/J(xHd - 'I/J(Xi) < -Aiaq(~**i)

< a(36 (~ )--y;-q **i

< L0~'Y) O(Xi) (2.29)

< a(36 Oe)2L(!+'Y) X

l::.= -r;"

which completes our proof. 0

The following theorem is a direct consequence of Lemma 2.1, Lemma 2.2and Theorem 1.2.8 in [11].

Theorem 2.3 If {Xi}~o is a sequence constructed by Algorithm 2.2. thenevery accumulation point x of this sequence satisfies the first-order optimalitycondition O(x) = O.

3 RATE OF CONVERGENCE OF ALGORITHM 2.2

To establish the rate of convergence we will use the following hypotheses:

Assumption 3.1 We will assume that

(a) For every y E Y. cp(', y) is convex.

(b) 111e second derivative matrix CPxx(x, y) exists and is continuous.

(c) There exist °< m ~ M such that. for all x in a sufficiently large set.all y E Y, and all h E JR71.,

mllhll2 ~ (h,CPxx(x,y)h) ~ Mllhll2. (3.1)

(d) 8 E [m,M] holds.

In [11, p. 225], we find the following result:

Lemma 3.1 Suppose that Assumption 3.1 is satisfied. Let i; be the uniqueminimizer of 'ljJ( .). Then, for any x E lRn ,

'ljJ(i;) -7jJ(x) ~ iO(x) .m

(3.2)

Theorem 3.1 Suppose that Assumption 3.1 is satisfied. If {xd~o is asequence constructed by Algorithm 2.2, then

(3.3)

Proof. First, it follows from Lemma 3.1 and Theorem 2.2 (ii), that , for alli E N,

Next, in view of (3.1) and (2.12), and for any Xi and>' E [0 ,81M],

7jJ(Xi + >.hi) - 'ljJ(xd>.2M

~ max {[cp (Xi ,y) - 'ljJ(xd] + >'('\7xcp(Xi ,y), hi)} + -2-lIhiIl2yEY

~ xmax{[cp(xi' y) -7jJ(Xi)] + ('\7xcp(Xi, y) ,hi)} + -2811hi1l2yEY

~ ->'q(e**d·

(3.5)

Hence, for any Xi and>' E [0,8IM],

'ljJ(Xi +>'hd - 'ljJ(xd + >'aq(e**d ~ ->'(1 - a)q(e**d ~ 0, (3.6)

FIRST-ORDER ALGORIGHMS FOR A CLASS OF OPTIMIZATION PROBLEMS 207

which proves that Ai ~ fM.. Hence we have that

(3.7)

Now, it follows from (3.4) that

(3.8)

Combining (3.7) and (3.8) , we obtain that

(38a m ~'ljJ(xHd - 'ljJ(xd ~ M 8(1 +,) ['ljJ(x) - 'ljJ(xd)· (3.9)

Subtracting 'ljJ(x) - 'ljJ(xd from both sides of (3.9), we finally obtain

which completes our proof. 0

Remark 3.1 Note that when, = 0, we revert to the conceptual form of thealgorithm for which the rate of convergence is given in Theorem 2.4.5 of [11].We see that when, = 0, the expression (3.10) coincides with the correspondingexpression given in Theorem 2.4.5 of [11].

4 MINIMIZATION OF THE MAXIMUMEIGENVALUE OF A SYMMETRIC MATRIX

We now return to the special case where ep(.,.) and Yare given as in (2.3),with the matrix Q(.) at least once continuously differentiable. To show that Algorithm 2.2 is implcmentable for this case, we only need to show that the pointse*i and e**i can be computed using either the Frank-Wolfe algorithm [4] or themuch more efficient Higgins-Polak algorithm [6], [11] to minimize the functionq(e), defined in (2.5), over the set G'ljJ(x), defined in (2.4). Both of these algorithms depend on the computation of "support points" to the set G'ljJ(x), but theFrank-Wolfe algorithm is much simpler to explain, so we will restrict itself to it.

Modified Frank-Wolfe Algorithm (Computes points e* and e**)

Parameters. , E (0,1).

Data. eo E G'ljJ(x).

Step O. Set i = O.

Step 1. Compute a support point (i E G'IjJ(x) according to

Step 2. Compute the point

(4.2)

where

Step 3. If (;0 > 0 and

(4.4)

set ~. = ~i' ~•• = (I and exit.

Else, set hi = (i - ~i and go to Step 4.

Step 4. Compute the step-length

x, = argmin{q(~i + Ahi ) IA E [0, I]} . (4.5)

Step 5. Update: Set

(4 .6)

and go to Step 1.

The following result is a direct consequence of the fact (see Theorem 2.4.9in [11]) that if the Frank-Wolfe does not exit in Step 3 above, then the sequence{~}~o converges to the unique minimizer of q(.) on G'IjJ(x) .

Theorem 4.1 Suppose that x E lRn is such that O(x) < 0, then the Modi fied Frank- Wolfe Algorithm will compute the required points ~., ~•• in a finitenumber of iterations.

Proof. Suppose that x E lRn is such that O(x) < oand that the Modified FrankWolfe Algorithm does not exit in Step 3 after a finite number of iterations. Thenit follows from Theorem 2.4.9 in [11], that the sequence {~i}~O converges to theunique minimizer F' of q(.) on G'IjJ(x) (which is a support point by first-orderoptimality conditions), and the same holds for the sequence {(i}~O' Hencethe hyper-planes tl((i) converge to the hyper-plane tl(~·) and therefore thesequence {(;}~o also converges to (:. Since this implies that q(~i) -q((D~ 0,as i ~ 00, we have a contradiction, which completes our proof. 0

FIRST-ORDER ALGORIGHMS FOR A CLASS OF OPTIMIZATIONPROBLEMS 209

Clearly, neither the minimization of the quadratic function q(.) on the hyperplane 1£((i) in (4.2), nor the step-length calculation in (4.5) cause any difficulty.The only difficult operation in the Modified Frank-Wolfe Algorithm seems tobe the computation of the support (contact) point (i, according to (4.1). Wewill now show that, because of the quadratic form of the function ep(x, y), thiscomputation is quite simple.

Now, when ep(x, y) and Yare as in (2.3), the set G'IjJ(x) assumes the specificform:

I('IjJ(x) - (y, Q(x)y) lJ

_ (y, Ql(X)Y)G'IjJ(x) = convlIyl12 =1 : '

(y, Qn(x)y)

(4.7)

where Qi(x) = 8Q(x)j8xi.

It follows from (4.1) that any support point (i, associated with the point~i E G'IjJ(x), satisfies the relation

(i E argmin{ (V'q(~d,() I( E G'IjJ(x)}. (4.8)

Proposition 4.1 Suppose that G'IjJ(x) is defined as in (4.7). Let s =V'q(~i). let

n

R(x; s) = -Q(x) +L skQk(X), (4.9)k=l

andlety(x; s) beany uniteigenvectorofRix; 8)corresponding to Amin [R(x; s)].the smallest eigenvalue ofR(x; s). Then the point

(

'lj;(x) - (y(x; s), Q(x)y(x; s)) 1_ (Y(X; s), Qdx)y(x; s))(i = .

(y(x; s), Qn(x)y(x; s))

is a support point associated with the point ~i E G'IjJ(x).

Proof. Let

(4.10)

I('IjJ(x) - (y, Q(x)y) 1

S(x) = (y, Q~ (x)y)

(y, Qn(X)Y)

lIyII2

= IJ. (4.11)


Then, because the minimum of a linear function on the convex hull of a set isequal to the minimum of the linear function on the set itself, we have that

mm {(B,() I(E G'ljJ(x)} = min{(B,() I(E S}n

min{(y,[-soQ(x)+ LSkQk(X)]y) Illyll = 1} + sO 'ljJ (x)k=l (4.12)

n

Amin[-SOQ(x) + L skQk(X)] + s°'ljJ(x).k=l

The desired result now follows directly from (4.12). D

In view of Proposition 4.1, we can expand the description of the ModifiedFrank-Wolfe Algorithm as follows :

Expanded Modified Frank-Wolfe Algorithm(Computes points (* and (**)

Parameters. 'Y E (0,1) .

Data. (0 E G'ljJ(x).

Step O. Set i = O.

Step 1. Set Bi = V'q((i), set

n

R(X;Bi) = -Q(x) + LsfQk(X), (4.13)k=l

and compute a unit eigenvector y(x; Si) of the matrix R(x; Bi) corresponding to Amin[R(x; Bd].

Step 2. Set the support point (i to be

(

'ljJ(x) - (y(x; Si),Q(x)y(x; sd) 1_ (y(x; sd,Qdx)y(x; Si))(i = . .

(y(x; Si),Qn(X)Y(X; Si))

Step 3. Compute the point

where

(4.14)

(4.15)

FIRST-ORDER ALGORIGHMS FOR A CLASS OF OPTIMIZATIONPROBLEMS 211

Step 4. If (~O > 0 and

(4.17)

set ~* = ~i. ~** = (~ and exit.

Else. set iii = (i - ~i and go to Step 5.

Step 5. Compute the step-length

Ai = argmin{q(~i + Aiid IA E [0, I]). (4.18)

Step 6. Update: Set

(4.19)

and go to Step 1.

5 PROBLEMS WITH SEMI-INFINITECONSTRAINTS

We now tum to problems of the form

min{f(x) I 'ljJ(x) ~ O}, (5.1)

where f : ll~n -+ Ris continuously differentiable and 'ljJ(x) is defined as in (2.2),with Y C IRm compact, and ip : Rn x r -+ IR continuously differentiable.In particular, our algorithm will be applicable when Y and ep(., .) are definedas in (2.3).

The Polak-He Algorithm [10], [11, p. 260] (a phase I - phase II method ofcenters), solves smooth problems with a finite number of inequality constraints.It relies on the fact that the function F : IRn x IRn -+ lR, defined by

F(z; x) ~ max{f(x) - f(z) - w'ljJ(z)+, 'ljJ(x) - 'ljJ(z)+}, (5.2)

with the parameterw > 0 and, for any a E lR, a+ = max{O, a}, is a commonlyused vehicle for obtaining first-order optimality conditions for problem (5.1)from those for problem (2.1). The reason for this is given by the following result.

Proposition 5.1 ([11, p. 186]) Suppose that the junctions f (.) and ep(., .)are continuous and that x is a local minimizerfor (5.1). Then xis also a localminimizer for the problem:

min F(x;x).xEIRn

(5.3)


The conceptual extension of the Polak-He Algorithm to the problem (5.1),which has semi-infinite constraints, can be described as follows . Given thecurrent iterate Xi E ~n, the Polak-He Algorithm constructs the max functionF(Xi ; .) and applies one iteration of Algorithm 2.1 to its minimization. Then itreplaces i by i + 1 and the process is repeated, over and over again.

Note that F(x; x) = 0, for all x E JR1l, and hence the equivalent of theset-valued map G'IjJ(x), in (2.4), is seen to be set-valued map

{ (w'IjJ(x)+ ) { ( 'IjJ(x)+ - ep(x, y) ) }}

GF(x;x) ~ cony , convYEY ."V f(x) Vxep(x, y)

(5.4)

We also get the following equivalent of Theorem 2. 1:

Theorem 5.1 ([11, p. 381]) Suppose that:

(a) The set-valued map GF(x; x). from jRn to the subsets of jRn+l. iscontinuous in the Painleve-Kuratowski sense;

(b) Let the elements of jRn+l be denoted by t, ~ (eO, o. with eO E Rand.with 6 > O. let

- ~ ° 1 2q(e) = e + 26"el ,

let the optimality junction

8(x) ~ - _ min q(t,),~EGF(xjx)

and let

(5.5)

(5.6)

fI(x) = (Ho(x),H(x)) ~ - arg _ min q(t,). (5.7)~EGF(xjx)

Then.

(i) Thejunctions 8(·) and fI (.) are continuous. and.for all x E jRn.

8(x) $ O.

(ii) For any x E JWI.. the directional derivative. with respect to thesecond argument. satisfies the relation

FlRST-ORDERALGORIGHMS FOR A CLASS OF OPTIMI7ATIONPROBLEMS 213

(iii) If ii E IR.n is a local minimizer oft i .l ), then

oE GF(x; x), 8(x) = 0, and fI(x) = O. (5.9)

Furthermore, (5.9) hold'> if and only if 0 E 8:1F(x;x), where8:1F(x; x) denotes the subgradient ofF(x;·) at x.

It should now be clear that the straighforward extension of the Polak-Healgorithm is non-irnplementable because neither 8(Xi) nor H(Xi) can be computed exactly in reasonable time. To obtain an implementable extension of thePolak-He algorithm, we must modify it so as to be able to use approximations to8(Xi) and H(Xi) ' We will now develop such an implementation which makessense when rp(x, y) is of the form (2.3) and, possibly, a few other cases as well.The success of the new algorithm depends on the following observation, whichis a straightforward adaptation of Theorem 2.2, and hence is stated withoutproof:

Theorem 5.2 Suppose that x E IR.n is such that 0 rf. GF(x; z ), 'Y E (0,1),and i, E GF(x; z ), ~** rf. GF(x; x) are such that

(a) e~* > 0;

(b) (V'q(~**),~ - ~**) ~ OJorali ~ E GF(x;x);

(c) q(~*) - q(~**) ~ 'Yq(~**) .

Then,

(i) q(~**) ~ - 8 (x ) ~ q(~*) ;

(ii) -1~'Y8(x) ~ q(~**) ~ -8(x);

(iii) with h** = -e**,

FHx; x; h**) ~ max {(V' j(x), h**) - w7jJ(x)+,

max{rp(x, y) - 7jJ(x)+ + (V'xrp(x, y), h)}} + -26 I1 h** 112 (5.10)

yEY

~ -q(~**),

and

We can now state the implementable version of the Polak-He Algorithm andestablish its convergence properties:

Algorithm 5.1 Modified Generalized Polak-He Algorithm

Parameters. a.B, E (0,1). w,o > O.

Data. Xo E lRn .

Step O. Set i = O.

Step 1. Use the Higgins-Polak or Frank-Wolfe algorithm to compute a (*i E

GF(xjj xd and a ( ..i t/. GF(Xi; Xi) such that

(a) ~~*i > O.

(b) (\7q(~..d, ( - ~.. j) ~ O,Jor all ( E GF(Xij xd,

(c) q(~*d - q(~..i) :S ,q((..d.

Step 2. Compute the step-size

k k k-Aj = max{{3 IF(Xi; Xi + {3 hd + {3 o:q(~..d :S O}. (5.12)

kEN

Step 3. Set

(5.13)


Lemma 5.1 Suppose that 'V!(.) and 'Vxr.p(·, .) are Lipschitz continuouson bounded set.i3 and that x E JR7l. is such that 8(x) < O. Let p > 0 besuch that 8(x) :S 8(x)j2,Jor all X E B(x, p). Then there exists a Lipschitzconstant L E [0,(0) such that.for all Xi E B(x, p), Ai is well defined by (5.12)and satisfies Aj ~ {30 j L.

We omit a proof of Lemma 5.1, since it is virtually identical to the proof ofLemma 2.1 .

'Theorem 5.3 If{xd~o is a sequence constructed by Algorithm 5.1, thenevery accumulation point x of this sequence satisfies the first-order optimalitycondition 8(x) = O.

Proof. Since F(xj ;x i+d :S 0, for all i, by construction, it follows that if7jJ(xd :S 0, then

F(xj ;xj+d = max{f(xi+d - !(Xi), 7jJ(xi+d} :S 0, (5.14)

FIRST-ORDERALGORIGHMS FOR A CLASS OF OPTIMIZATIONPROBLEMS 215

and hence tP(xi+d :S °and, also, f(Xi+i) :S f(xd; i.e., that if for some io,Xio is feasible, then the rest of the sequence {Xi}~o remains feasible and thecost decreases monotonically after io.

If, on the other hand. 'I/J(Xi) > 0, then

F(Xi; xi+d = max{f(xi+d - f(Xi) - w'I/J(Xi), 'I/J(xi+d - 'I/J(xd} :S 0,(5.15)

and we see that the constraint violation function 'I/J(.) decreases, but the costfunction can increase in value.

For the sake ofcontradiction, suppose that {Xi}~o has an accumulation pointx such that 8(x) < 0.

Thus we need to consider two cases. First suppose that there is an io suchthat 'I/J(Xi) :S 0, for all i 2: io. Then, because the sequence {f(xd}~io ismonotone decreasing, and because it has an accumulation point f (x), we musthave that f(Xi) '\t f(x) , as i -7 00 (see Proposition 5.1.16 in [11]). Also, itfollows from Lemma 5.1, that there is a p > °and fJ :S L < 00, such that, for allXi E B (x,p), Ai 2: (3fJ/ L. Since {xd~o has a subsequence which convergesto x, it follows that there is an infinite number of indices i 2: i o, such that

(5.16)

Since (5.16) contradicts the fact that the monotone-decreasing cost sequence{f(Xi)}~io converges to x, we must conclude that 8(x) = 0.

Next suppose that 'I/J(Xi) > 0, for all i E N. Then, because the sequence{'I/J(Xi) }~o is monotone decreasing, and because it has an accumulation point'I/J(x), we must have that 'I/J(Xi) '\t 'I/J(x), as i -7 00 (see Proposition 5.1.16 in[11D. Again we conclude from Lemma 5.1 that there is an infinite number ofindices i such that

'I/J(Xi+d - 'I/J(Xi) < ~ -- L q(~**i)

< L0~'Y) 8(Xi)

< 2L(fi'Y) 8(x),

(5.17)

which contradicts the fact that 'I/J(Xi) '\t 'I/J(x), as i -+ 00. Hence we concludeagain that 8(x) = 0, which completes our proof. 0


Remark 5.1 Note that Theorem 5.3 does not guarantee that the accumulation points constructed by Algorithm 5.1 will be feasible. To obtain thisadditional property we must require that 0 t/. 8'l/J(x), for all x E IRn such that'l/J(x) > 0, as is also the case with the Polak-He Algorithm [6], as well as othermembers of methods of centers family of algorithms.

6 PROBLEMS WITH MAXIMUM EIGENVALUECONSTRAINTS

To conclude our presentation, we will consider briefly the case of problem(5.1) where 'l/J(x) and Y are defined as in (2.3), with the matrix Q(x) continuously differentiable. Again we propose to use either the Higgins-Polak [6]or the Frank-Wolfe [4] algorithm to compute the points e*i, e**i in Step 1 ofAlgorithm 5.1. Obviously, again, the main issue is that of computing a supportpoint to the set GF(Xi; Xi)'

When 'l/J(x) and Yare defined as in (2.3), the set GF(xj x), in (5.4), assumesthe form

Iol,() I('l/J(x)+ - (y, Q(x)y) 1))_ unp x + (y, Q1(x)y)

GF(Xjx) = COny ( ) ,conv . .\Jj(x) IlylI=1 :

(y, Qn(x)y)(6.1)

Hence, given any s E IRn+1 , with S= (sO, s), we see that

mm {(s,e) leEGF(x;x)} = min {sOw'l/J(x)+ + (s, \Jj(x)},n

s°'l/J(x)+ + min {(y, [-so[Q(x) + L~>kQk(X)]Y}} },Ilyll=1 k=l

(6.2)

which we recognize, again, as being a minimum eigenvalue problem. Thus,in this case, too, there are no impediments to the implementation of either theHiggins-Polak or the Frank-Wolfe algorithms.

7 RATE OF CONVERGENCE OF ALGORITHM 5.1

Referring to [11], we see that establishing the rate of convergence of thePolak-He Algorithm is a rather laborious process which we will not attempt toemulate in this paper. However, we believe that the following result is correct.

Theorem 7.1 Suppose that

FIRST-ORDERALGORIGHMS FOR A CLASS OF OPTIMIZATIONPROBLEMS 217

(a) Thefunctions f (.), and <p(', y), y E Y, are twice continuously differentiable. and the functions <p(', .), V'x<p(', .), <Pxx(" ') are continuous.

(b) The set {x I 'ljJ(x) < O} is not empty.

(c) There exist constants 0 < m ~ 0 ~ M < 00, such that, for allx , h E JR1I'.

(7.1)

and

(d) With x the unique solution of(5.1),

(7.3)

(e) {xd~o is a sequence constructed by Algorithm 5.1.

Then,

(i) Ifw > M / fL*a(3m, then there exists an io such that. for all i 2: io,'ljJ(Xi) ~ 0;

(ii) If'ljJ(xi) > O,Jorall i, then w ~ M/fL*a(3m and

lim sup 'ljJ(xi+d ~ 1 _ wfL*a.(3m . ~ 0, (7.4)i--+oo 'ljJ(xd M(l +,)

then 'ljJ(Xi) ~ O,Jor all i 2: iI, and

lim sup f(Xi+d - f~x) < 1 _ fL*a.(3m . (7.5)i--+oo f(Xi) - f(x) - M(l +,)

8 A NUMERICAL EXAMPLE

We will now present some numerical results for the problem (1.5).Inour example, in (l .5), we used the 4 x 4 complex sample covariance matrix

[

11.9644 10.8698 - j7.0581 3.8956 - j14.1565 -7.8576 - j12.5698 ]_ to.8698 + j7.0581 14.3893 12.0326 - jl1.2613 -0.6620 - j16.4748

S - 3.8956 + j14.1565 12.0326 + jl1.2613 19.6482 12.8970 - j15.6408-7.8576 + j12.5698i -0.6620 + j16.4748 12.8970 + j15.6408 21.8540

and we set a = 0.05.


1- AI~orith m:5,- - - AIJoo thm 4

16rr-- - - - - - r===== ==il

14

16,--- - - - - - ;=== = ==il

12

';( 10 ~"" .

8 :

6 ... .

100 200nerancns

300 4()() 1000 2000 3=compctancn al lcad

400C

Figure 9.1 Minimization of the objective function for the method of outer approximations(Algorithm 5) and the specialized Polak-He algorithm 5.1 (Algorithm 4)

I AlgorithmS••• Alrorithm-4 15 I A1sorilhn'lS

- - - Ahzorilhm 4

05

~ O~- _ . _ - - - - - - - - - - - - - - - _ . _ - - - - - -

- 0.5

- I

- 1.5

~ 0 ;••••• - ••"--- - - - ----1

-05 :

- I :

- 1.5 :

100 200 300Ilc=nllom

4()() 1000 2000 3000COlnputll io".lload

Figure 9.2 Constraint violation function for for the method of outer approximations (Algorithm5) and the specialized Polak-He algorithm 5.1 (Algorithm 4)

InAlgorithm 5.1, we set Q' = 0.5, (3 = 0.8, w = 4, / = 0.05, and 8 = 1, andwe initialized out computations with Ro = I . We solved the example problemusing both our new Algorithm 5.1 and, for comparison, the method of outerapproximations presented in [11, p. 464]. In our figures, the results of themethod of outer approximations are labeled as those of Algorithm 5.

Figure 9.1 shows plots of the values of !(Xi) versus the iteration number,as well as values of !(Xi) versus the approximate computational load, whileFigure 9.2 shows plots of the constraint violation function , 'I/J(Xi) for the twoalgorithms. Since a count of actual operations was not sensible for our unoptimized MATLAB code, the approximate computational load was determinedby counting the number of evaluations of !(x) . For problems with low dimensional decision variable z, such as the one we have presented, the performanceof these two algorithms are quite comparable. However, as the dimension ofthe decision variable increases, the empirically observed rate of convergence ofthe method of outer approximations becomes progressively slower, while the

FIRST-ORDER ALGORIGHMS FOR A CLASS OF OPTlMI7ATIONPROBLEMS 219

rate of convergence of Algorithm 4 is unaffected. Hence Algorithm 5.1 shouldbe the one of choice for large problems.

9 CONCLUSION

We have presented two new implementable semi-infinite optimization algorithms which are particularly useful for solving problems with maximumeigenvalue costs or constraints, as illustrated our numerical example. We havereason to believe that our algorithms are particularly competitive when the dimension of the decision vector is large.

In [2] we find another numerical example, which is particularly meaningful tothe Signal Processing community. That example shows that solutions obtainedusing Algorithm 5.1 lead to much more accurate signal spectrum identificationthan those obtained using other current techniques. This is particularly truewhen the spectra have multiple peaks.

Acknowledgements

(i) This work was supported by the National Science Foundation undergrant No. ECS-9900985

(ii) We wish to thank Dr. Linda Davis, of Lucent Technologies, Sydney,Australia, for supplying the numerical example.

Notes

1. Notation. We will denote components of a vector by superscripts (e.g.• x = (x t , ... , xn) and wewill denote elements of a sequen ce by subscripts (e.g.• {Xi}~o)'

2. Private communication: Dr. R. S. Womersley, Dept. of Applied Mathematics, University of NewSouth Wales, Sydney, Australia

3. The assumption of local Lipschitz cont inuity can be relaxed to continuity, but the proof of the Lemmais then a bit more difficult.

References

[1] S. P. Boyd, V. Balakrishnan, C. H. Barratt, N. M. Khraishi, X. Li, D. G.Meyer, and S. A. Norman. A new CAD method and associated archi tectures for linear controllers, IEEE Transactions on Automatic Control,33:268-283, 1988.

[2] L. Davis, and E. Polak. Maximum likelihood constrained covariance estimation, Submitted to IEEE Transactions on Signal Processing, Sept 2000.

[3] J. C. Doyle, and G. Stein. Multivariable feedback design concepts fora classical/modern synthesis, IEEE Transactions on Automatic Control,AC-26:4-16, 1981.


[4] M. Frank, and P. Wolfe. An algorithm for quadratic programming, NavalResearch Logistics Quarterly, 3:95-110, 1956.

[5] D. Gray, B. Anderson, and P. Sim. Estimation of structured covarianceswith applications to array beamforming , Circuits. Systems and Signal Processing, 6:421-447, 1987.

[6] J. E. Higgins, and E. Polak. Minimizing pseudo-convex functions on convex compact sets, Journal ofOptimization Theory and Applications, 65:128, 1990.

[7] A. S. Lewis, and M. L. Overton. Eigenvalue optimization, Acta Numerica,5:149-190,1996.

[8] M. I. Miller, D. R. Fuhrmann, J. A. O'Sullivan, and D. L. Snyder. Maximum likelihood methods for Toeplitz covariance estimation and radarimaging. In S. Haykin, editor, Advances in Spectrum Analysis and ArrayProcessing, Prentice-Hall, 1991.

[9] V. Pisarenko. The retrieval of harmonics from a covariance function, Geophysical Journal of the Royal Astronomical Society, 33:347-366, 1973.

[10] E. Polak, and L. He. A unified steerable phase I - phase II method offeasible directions for semi-infinite optimization, Journal ofOptimizationTheory and Applications, 61: 83-107, 1991.

[11] E. Polak. Optimization: Algorithms and Consistent Approximations,Springer-Verlag, 1997.

[12] I. H. Sloan. Interpolation and hyperinterpolation on the sphere , in Multivariate Approximation: Recent Trends and Results. In W. Hausmann, K.Jetter, and M. Reimer, editors, pages 255-268, Academic Verlag (WileyVCH) ,1995.

[13] D. Snyder, J. O'Sullivan, and M. Miller. The use of maximum likelihoodestimation for forming images of diffuse radar targets from delay-Dopplerdata, IEEE Transactions on Information Theory, 35:536-548, 1989.

[14] L. Vandenberghe,and S. Boyd. Semi-definite programming, SIAM Review,38:49-95, 1996.

[15] R. S. Womersley, and I. H. Sloan. How good can polynomial interpolationon the sphere be?, Technical Report. School of Mathematics, Universityof New South Wales, Sydney, Australia, 1999.

Chapter 10

ANALYTIC CENTER BASED CUTTING PLANEMETHOD FOR LINEAR SEMI-INFINITEPROGRAMMING

Soon-Yi Wu1, Shu-Cherng Fang? and Chih-Jen Lin"1Institute of Applied Math ematics, Nat ional Cheng-Kung University, Tainan Ztk), Taiwan, R.O.C.

20perations Research and Industrial Engineering, North Carolina State University, Raleigh,

NC 27695-7906, U.S.A.

3Department of Computer Science and Information Engineering, Nation al Taiwan University,

Taipei, Taiwan, R.O. C.

[email protected], fang@eos .ncsu .edu, cjlin@cs ie.ntu.edu.tw

Abstract In this paper, an analytic center based cutting plane method is proposed for solving linear semi-infinite programming problems. It is shown that a near optimalsolution can be obtained by generating a polynomial number of cuts.

1 INTRODUCTION

Consider a linear semi-infinite programming problem in the following form:

(LSIP)n

min LbiUi

i= ln

s.t. L uili{t) ~ g(t) , Vt E T,i=l

Ui 2: 0, i = 1, ... ,n,

(1.1 )

where T is a nonempty compact subset of R, Ii , i = 1, ... , n, and 9 are realvalued continuous functions defined on T . Its dual problem can be formulated

221

M.A. Goberna and M.A. Lopez (eds.), Semi-Infinite Programming, 221- 233.© 2001 Kluwer Academic Publishers.


in the following form :

(DLSIP) max -1g(t)dv(t)

s.t. -1 Ji(t)dv(t) ::; bi, i = 1, . .. , n ,

v E M +(T) ,

(1.2)

(1.3 )

where M +(T) is the space of all non-negative bounded regular Borel measuresdefined on T.

Under some regularity conditions, it can be shown that there is no dualitygap between (LSIP) and (DLSIP) [1]. According to the survey of [9], thereexist three basic solution procedures, namely, the cutting plane approach, discretization methods, and the methods based on local reduction . In addition, anentropic dual perturbation method was proposed in [11]. Among them, the cut ting plane approach [1],[2],[3],[7],[8] ,[10] and [12] is the mo st widely used andextensively studied. Basically, this approach solves a sequence of finite linearprograms in a systematic way and shows that the sequence of correspondingsolutions converges to an optimal solution of (LSIP ).

To be more precise, a traditional cutting plane method, in the kth iteration,considers the following linear problem with k constraints asso ciated with a set

Tk = {tl,t2 ,'" , td C T:

n

min LbiUii= ln

s.t. L Udi(tj) ::; g(tj), tj E Tk ,i= l

Ui 2: 0, i = 1,2, . .. ,n.

(1.4)

(1.5)

Its dual problem can be written as

k

max - Lg(tj)Vjj=l

k

s.t. - L fi(tj)vj ::; bi , i = 1,2, . . . ,n,j = l

Vj 2: 0, j = 1,2, ... , k.

(1.6 )

CU1TING PLANE METHOD FOR LINEAR SIP 223

After solving (LPTk) for an optimal solution Uk (or a relaxed one [2]), usually the maximizer tk +l (or a relaxed one [13]) of the function cp(uk , t) ==:E~=l ufh(t) - g(t) is used to either stop the process or to generate a new cutfor the (k + l)th iteration by noting the fact that if cp (uk, tk +d ~ 0, Vt E T ,then uk is feasible and optimal to (LSIP).

Alth ough the convergence of the traditional cutting plane method and its variants have been carefully studied, to the best of the authors' knowledge, therewas no formal analysis on the required number of cuts (i.e., iterations). In thispaper, motivated by the work of Goffin et al. [5, 6], we proposed a new cuttingplane method based on the concept of analytic centers. In this new framework,we are able to show that an e-optimal solution to (LSIP) can be identified bygenerating a polynomial number of cuts .

2 ANALYTIC CENTER BASED CUTS

The idea of using analytic center based cuts was recently proposed in [5]and [6] to study the convex (or quasiconvex) feasibility problem. This problemseeks an interior feasible solution to a given convex set defined by a (finite orinfinite ) system of convex inequalities:

S = {u E R n I hi(u) ~ 0, i E I}.

In the kth iteration, a polyhedral set nk = {u E R" I Ak u ~ ck} isinvolved. Its analytic center uk is defined to be the maximizer of the corresponding potential function

n

I:log sjj=l

with S == ck - Ak u being positive. This analytic center is then used either toterminate the iterations (if uk E int(S)) or to construct a "feasibility cut"

{u E R n IaI+lu = aI+luk}

with Ilak+lll = 1 such that S C {u E Rn I aI+lu ~ aI+lu k} . Subsequently,

we may move to the next iteration with

where

Under the assumptions that (l) the solution set S is contained in the cubenO = {u E Rn I0 ~ u ~ e}, where e is the n-vector with all ones; (2) the setint(S) contains a full dimensional ball with radius p < ~; and (3) there existsan oracle which for every u E nO either returns that U E int(S) or generates aseparating hyperplane such that S c {u I aT u ~ aTu} with lIall = 1, Goffinet at. ([5], [6]) showed that their algorithm terminates in at most k iterations,where k is the smallest integer satisfying the following inequality

2n+k n1 k >,.22 + 2nln(1 + BnT) - P

(see Theorem 3.1 of [5]).

3 ANALYTIC CENTER CUTTING PLANE METHODFOR LSIP

In this section, we extend the idea of analytic center based cuts to solve(LSIP). For easy references, given an optimization problem (P) , we denoteits optimal objective value by V(P), when it exists. Moreover, we denote thefeasible region of (LSIP) by U. If u* E U is an optimal solution to (LSIP),given E> 0, we define an E-neighborhood of u* by

In this case, any u E N€( u*) is an E-optimal solution of (LSIP), i.e., 0 ~

bT U - V(LSIP) ~ E. To simplify the derivation of the proposed algorithm,in this paper we assume that (LSIP) has a unique optimal solution u* and itsdual (DLSIP) has a unique optimal solution u",

Now we can describe the key concepts in designing the proposed algorithm.Because (LSIP) is not readily a convex feasibility problem, in addition togenerate "cuts for feasibility ", we also generate "cuts for optimality" when itis necessary. Consider that in the kth iteration with a current solution uk, thenecessity of generating an additional cut depends, like the traditional cuttingplane method, on the value of the function

n

rp(uk, t) == L ufJi(t) - g(t).i=l

If there exists tk E T such that rp(uk, tk) > 0, then uk is not feasible to(LSIP) and a "feasibility cut" is generated to move the next iterate closer tofeasibility. On the other hand, if no such tk can be found, then by checking someoptimality conditions, the algorithm either stops with an e-optimal solution of


(LSIP) or generates an "optimality cut" to move the next iterate closer tooptimality.

Inorder to simplify the derivation of the proposed algorithm, we impose fourcommonly seen technical conditions on the functions fi(t) and g(t): (i) fi(t),i = 1, ... , n, andg(t) are analytic functions defined on T; (ii) there exists f E Tsuch that Ji(f) > 0, Vi; (iii) g(t) does not lie in the linear space spanned by{Ji (t) I i = 1, , n}; and (iv) g' (t) does not lie in the linear space spanned by{f[(t) Ii = 1, ,n}.

Notice that the technical conditions immediately lead to the following resultwhich will be used later in the convergence proof:

Lemma 3.1 Let u* be the unique optimal solution oJ(LSIP). Under thetechnical conditions (i) and (iii), the equation

n

cp(u*, t) == L Ji(t)ui - g(t) = 0i=l

(3.1)

has only a finite number ofsolutions.

Proof. Suppose that the equation has infinite many solutions . With Ji(t), i =1, . . . , n, and g(t) being analytic functions, we know that g(t) is spanned byJi(t), i = 1, . .. , n . This causes a contradict ion to the technical condition(iii). D

Now we revisit the concept of checking for optimality in designing the proposed algorithm. Recall that this step is taken only when the current iterate ukis feasible, i.e., we cannot find any tk E T such that cp(uk, tk) > O. In thiscase , we let 0 > 0 be a small number and define the following set:

Bk (0) == {t E Tit is a local maximizer of cp(uk, t) such that

- 0 ~ cp(uk, t) ~ O}.

Note that when 0 is chosen to be sufficiently small (say 0 = 10-5) , then Bk(o)consists of all local maximizers that are "good approximates." Also note that,under the technical conditions (i), (iii) and (iv), like the previous lemma, we seethat Bk(O) is a finite set. To be more precise, since g(t) is an analytic function,g'(t) is also an analytic function. If there are infinitely many elements in Bk(O),the number of local maximizers of cp(uk, t) is also infinite. Actually, they arethe solutions to

n

L ff(t)uf - g'(t) = O.i=l

However, this causes a contradiction to the technical condition (iv).

(3.2)


Moreover, under the technical condition (ii), since each Ji(t) is continuous,there exists an interval in T with Ji{t) > 0 for all i and all t in the interval.When Bk(8) is a finite set, we can definitely find some t rt Bk(8) such thatJi(f) > 0 for all i. By using Tk == Bk(8) U {l}, we can construct (DLPTk)and use its optimal value to check for optimality of the current solution. Thefollowing lemma further addresses the issue on optimality check.

Lemma 3.2 Given any 8 > 0 and € > O. there exist two junctions 0"( E)and ,(E) with o-(E) -+ 0 and ,(E) -+ 0 as E-+ 0 such that if Eis small enough.for any uk E int(Nf(u*))•

.max luf - uil ~ ,(E) ~ _8_l=l, ...• n nM

and

Proof. Since u* is the unique optimal solution to (LSIP), it can be seen thatas E-+ 0, there exists ,(E) -+ 0 such that

.max luf - uil ~ ,(E), (3.3)l=l, ... ,n

for all uk E Nf(u*).Notice that

n n n

IL Ji{t)uf - g(t) - (L Ji(t)ui - g{t))1 = IL Ji(t)(uf - unl·i=l i=l i=l

Since Ji(t), i = 1, . . . , n, are continuous functions defined on a compact setT, there exists a positive M such that IJi(t)1 ~ M, \:It E T, i = 1, ... , nand

n n

ILJi(t)(uf - unl ~ L IJi{t)ll(uf - unl ~ Mn'(E). (3.4)i=l i=l

Since u* is optimal to (LSIP), we have

n

L Ji(t)ui - g(t) ~ 0, \It E T.i=l

By Lemma 3.1, there exist a finite number of tj E T, say j = 1, .. . ,m, suchthat

n

L Ji(tj)ui - g(tj) = 0, \Ij.i=l

(3.5)

CUrI'lNG PLANE METHOD FOR LINEAR SIP 227

From the complementarity slackness conditions of (LSIP), we know thatthe optimal solution 1I* of (DLSIP) must be a discrete measure concentratedon ti , ,t:n . Without loss of generality we may assume that lI*(tj) > 0, for

j = 1, ,m', and lI*(t:n'+j) = 0, for j = 1, . . . ,m - m'.For uk E N€(u*), we know that uk is feasible to (LSIP). It follows from

(3.4) and (3.5) that

n

-nM'Y(E) :'S L h(tj)uf - g(tj) :'S 0, j = 1, .. . ,m. (3.6)i=l

Because u* is the unique optimal solution of (LSIP) and 2:~=1 fi(t)ui - g(t)has local maxima at ti , . .. , t:n. When Eis sufficiently small, combining Lemma3.1, (3.6) and the fact that uk E N€(u*) , we know that 2:~=1 h(t)uf - g(t) haslocal maxima at tl, ' " .l-«. with

n

-nM'Y(E):'SLfi(tj)uf-g(tj):'SO, j=1, ... ,m. (3.7)i=l

Again, combining (3.7) and (3.4), and the feasibility of u*, we know that,for j = 1, .. , m,

n°~ Lfi(tj)ui - g(tj)i=l

n n

= L h (tj )uf - g(tj) + L h(tj)(ui - uf)i=l i=l

~ - 2nM'Y(E), (3.8)

Following equations (3.5) and (3.8), without loss of generality, we may assume that for j = 1, . . . ,m there exists a sufficiently small b.j(E) such thattj E NLlj(€)(tj), where NLlj(€)(tj) is a small neighborhood oftj with radiusb.j(E). - . ,

Thus for tj,] = 1, ... ,m , we have

(3.9)

where b.j(E) --t °as E--t 0, for j = 1, ... ,m'. Remember that the technicalcondition (ii) assumes that there exists some i E T such that Ji(f) > 0, fori = 1, ... ,n. Since Ji(t) is continuous on T, there exists an infinite subsetGeT such that h(t) > 0, for each t E G and i = 1, ... ,n. Now we choosea point i E G with i ~ '1' = {tl, '" ,tm'} and letT' = {tl,'" ,tm" f} andT* = {ti, . . . ,t:n,}. It is obvious that if we take E to be sufficiently small such

228 SEMI·INFINITE PROGRAMMING. RECENTADVANCES

that 8 ~ nM,(€), then for any fj,j = 1, ... ,m',n

-8:S -nM,(€}:S Lfi(fj)uf - g(fj):S O.i=l

Therefore, 'i' C Tk.Now we consider the following two problems (DLPr·) and (DLPf"):

(DLPr· )m'

max - L g(tj)Vjj=lm'

s.t, - Lfi(tj)Vj :S bi, i = 1,2,'" ,n, (3.10)j=lVj~0,j=1,2,··· ,m'.

m'

(DLPf") max- Lg(fj)vj - g(f)vm'+lj=lm'

s.t. - Lfi(fj)vj -!i(f)Vm'+l < bi, i = 1,'" ,n, (3.11)j=l

Vj ~ O,j = 1,'" , m' + 1.

It is clear that vi == i/" (tj), j = 1,... , m', forms an optimal solution for(DLPr.) and

V (DLPf" ) :S V(DLPTk) :S V(DLPr·). (3.12)

Here we can define a feasible solution v for (DLPf") with

{

vj, j = 1, . .. ,m',Vj = max ' (max(-bi-L:j~l f;(tj)fij,O») . _ m' 1 (3.13)

1=1,00.,n /;(t) , J - + .

Note that v is well defined as !i(f) > 0, i = 1, .. . , n. By equation (3.9),when € is sufficiently small, v satisfies

m' m'

-u(€) :S - Lg(fj)vj - g(f)vm'+l + Lg(tj)v*(tj) :S u(€),j=l j=l

for some function u( e) ~ 0 as € ~ O. Since

m'

- L g(fj )Vj - g(f)vm'+l :S V (DLPf" ) :S V(DLPrk) :S V (DLPT. ),j=l


we have

IV(DLPTk) - V(DLPT.)I ~ o-(f). (3.14)

Alsonotethatforuk E int(Nf(u*)),lbTuk-V(LS1P)1 ~ f. SinceV(LS1P) =

V(DLPT.), with equation (3.14), it follows that

IbTuk - V(DLPTk) 1 ~ IbTuk - V(LS1P) 1 + IV(LS1P) - V(DLPTk)1

~ f + o-(f),

where 0-(f) -+ 0 as f -+ O. This completes the proof. 0

With the previous analysis, defining

f(t) == [h (t), .. . ,fn(t)]T,

we can present the proposed algorithm as follows .

Algorithm 3.1

Step 0 Set k = O. Let 8 > 0 and € > 0 be very small numbers. Select e > 0to be sufficiently small (such that Lemma 3.2 holds). Set

AO = [-=1] ER2nxn,

cO = [~] E R2n,

1uO = 2"e ERn, and

sO = cO _ (AO)TuO = ~ [:] E R 2n.

Step 1 (Center computation) Find the analytic center uk oink = {u E R n IAku ~ ck}.

Step 2 Find tk E T such that cp(uk, t k) > O. If there is no such tk, go to Step4.

Step 3 (Feasibility cut generation) Generate a cutting plane:

{n f(tk)TU f(tk)Tuk}

H k = u E R I IIf(tk)1I = IIf(tk) II .

Set


and

Go to Step 6.

Step 4 (Stopping criteria) Find B k (8) and] rt. B k (6) such that Ji(f) > 0,Vi =1, ... ,no

Let Tk = B k(6) U {f} and Solve (DLPTk)'

lflV(DLPTk) - bT ukl ::; €+ 10, stop and output uk as an (€+€)-opt imalsolution to (LSIP ).

Step 5 (Optimality cut generation) Generate a cutting plane:

Set

and

Step 6 Set k to k + 1 and go to Step 1.

Notice that in the proposed algorithm, the cuts are written in a fonn to

explicitly emphasize that 1I11~~~:~1I11 = II ~II = 1. This matches with the

previous work in generating cuts in which Ilak+l11 = 1 is required.

4 CONVERGENCE AND COMPLEXITY

In this section, we first show that Ne(u*) always lies on the correct side ofthe cutting plane generated by the proposed algorithm.

Lemma 4.1 For any feasibility cut generated in Step 3 of the proposedalgorithm,

Proof. For any u E Ne(u*), since it is feasible to (LSIP), we have f(tk)T u g(tk) ::; 0. Also because it comes after Step 2, we further have f(tk)T uk g(tk) > O. Consequently, f(tk)T u ::; f(tk)T uk. D

CUTI1NG PLANE METHOD FOR LINEAR SIP 231

Lemma 4.2 Ifuk fj. N€(u*),for any optimality cut generated in Step 5 ofthe proposed algorithm,

(4.1)

(4.2)

For any u E N€(u*),

Thus for any u E N€(u*),

This completes the proof. D

Now we show that the proposed algorithm converges to a near optimal solution after generating only a polynomial number of cuts.

In parallel of Goffin et al. ([5], [6]), the following basic assumptions are alsomade:

Assumption 4.1 (a) The e-roptimal solution set N€(u*) of(LSIP) is contained in the cube nO = {u E Rn I0 ~ u ~ e], where e is an n-svector ofallones.(b) The set int(N€(u*)) contains afull dimensional ball with its radius p < !.(c) There exists an oracle which for every u E nO either returns that u Eint(N€(u*)) or generates a separating hyperplane {u E Rn I aTu = aTu}with IIall = 1 such that N€(u*) C {u E Rn IaT u ~ aTu}.

The final result of this paper is obtained as follows.

Theorem 4.1 With the four technical conditions and Assumption 4.1, givenany small 8 > 0 and € > 0, iff. is sufficiently small (such that Lemma 3.2 holds),then the proposed algorithm stops in no more than k iterations, where k is thesmallest integer satisfying

2n+k n1 k ~ 2 '2 + 2n In(1 + 8n2') P

Proof. When the algorithm stops, if uk E N€( u*) is the output solution, itmust be the first iterate that falls in N€(u*) . Otherwise, from Lemma 3.2, ifany previous iterate fell in N€(u*), the algorithm has already stopped. Since ukis the first iterate falling in N€(u*) and u 1, .• • ,uk- 1 are not in N€(u*), so the

conditions of Theorem 3.1 of [5] are met and consequently we have a specialcase of Goffin et al's result. Therefore, the proposed algorithm stops when orbefore equation (4.2) is satisfied.

Similarly, ifuk rt Nt{u*), there is no iterate before uk falls in the set N t{u*).Again, the conditions ofTheorem 3.1 of [5] are met and consequently it becomesa special case of Goffin et al. result. 0

The following simple example from [4, pp. 255-257] is used to illustrate thevalidity of Assumption 4.1 :

min 2UI + U2

tUl + {1 - t)U2 2: t - t 2, t E [0,1],

'Ill 2: 0, U2 2: O.

(4.3)

Note that in the original formulation, although the constraints 'Ill 2: 0 andU2 2: 0 were notexplicitly stated, they were actually implied. Here we explicitlyinclude them for consistency.

It is easy to check that the example has a unique solution located at u* =(1/9,4/9) which sits in the interior of the cube nO. Therefore, if to > 0 is chosen tobe small enough, N t {u*) satisfies the first condition of Assumption 4.1. Thenfor condition (b), since the feasible region U of the given problem is convexand the set {u IbTu* + to 2: bTu 2: bTu*} is also convex, we know Nt{u*) isconvex. Now let L = {u I uT = t{1, 1) + (1 - t)(1/9, 4/9), 0 < t < 1}.Since (1,1) is interior to U and U is convex, therefore every point in L isinterior to U. Consequently, there exists it ELand it is interior to the set{u I bTu* + to 2: bTu 2: bTu*}. This means it E int{Nt{u*)) and the linesegment between it and (1/9,4/9), excepting (1/9,4/9), are in the interiorof Nt{u*) . Hence int(Nt{u*)) contains a full dimensional ball with a radiusp < ~. For condition (c), first note that

{u I 'Ill 2: 0, U2 2: 0, bTu* + to 2: bTu 2: bTu*}

is a bounded convex set. As a subset of this set, N, (u*) is also a boundedconvex set. Therefore , for any it E nO- int(Nt{u*)), we can always find aplane passing through it with Nt{u*) lying on one side of it. Thus condition(3) is valid and Theorem 4.1 becomes applicable.

Acknowledgments

The research of S.-c. Fang has been supported by a National Textile Centerresearch grant.

The research ofC.-J. Lin has been supported by the National Science Councilof Taiwan, via the grant 89-2213-E-002-013.

CU7TING Pl.ANE METHOD FOR LINEAR SIP 233

References

[I] E. J. Anderson and P. Nash. Linear Programming in Infinite-DimensionalSpaces, Wiley, 1987.

[2] S. C. Fang and S. Y. Wu. An inexact approach to solving linear semi-infiniteprogramming problems, Optimization, 28:291-299. 1994.

[3] K. Glashoff and S. A. Gustafson. Linear Optimization and Approximation,Springer-Verlag, 1982.

[4] M. A. Goberna and M. A. Lopez. Linear Semi-Infinite Optimization, Wiley,1998.

[5] J. L. Goffin, Z. Q. Luo, and Y. YeoOn the complexity ofa column generationalgorithm for convex or quasiconvex feasibility problems. In W. W. Hageret al., editors, Large Scale Optimization, pages 182-191, Kluwer, 1994.

[6] J. L. Goffin, Z. Luo and Y. Yeo Complexity analysis of an interior-pointcutting plane method for convex feasibility problem, SIAM Journal onOptimization, 6:638--652, 1996.

[7] S. A. Gustafson and K. O. Kortanek. Numerical treatment of a class ofsemi-infinite programming problems. Naval Research Logistics Quarterly,20:477-504, 1973.

[8] S. A. Gustafson and K. O. Kortanek. Computation of optimal experimentaldesigns for air quality surveillance. In Jansen, P. J. et al. , editors, Quantitative Modelle fiir Okonomisch-Okologische Analysen, pages 43--60, VerlagAnton Hain, Meisenheim, Germany, 1976.

[9] R. Hettich and K. Kortanek. Semi -infinite programming: theory, methodand applications, SIAM Review, 35:380-429, 1993.

[10] H. C. Lai and S. Y.Wu. On linear semi-infinite programming problems: analgorithm, Numerical Functional Analysis and Optimization, 13:287-304,1992.

[II] c.J. Lin , S. C. Fang, andS. Y.Wu. An unconstrained convex programmingapproach for solving linear semi-infinite programming problems, SIAMJournal on Optimization, 8:443-456, 1998.

[12] K. Roleff. A stable multiple exchange algorithm for linear SIP. In Hettich , R., editor, Semi-Infinite Programming: Proceedings of a Workshop,Volume 15 of Lecture Notes in Control and Information Sciences, pages83-96, Springer-Verlag, 1979.

[13] S. Y. Wu, S. C. Fang, andC. J. Lin. Relaxed cutting plane method for solving linear semi-infinite programming problems, Journal of OptimizationTheory and Applications, 99:759-779, 1998.

Part IV MODELING AND APPLICATIONS

Chapter 11

ON SOME APPLICATIONS OF LSIPTO PROBABILITY AND STATISTICS

Marco Dall'Aglio

Dipartimento di Scienze, Un iversita "G. d'Annunzio ", Pescara, Italy

maglio@sci .unich.it

Abstract The duality results and the computational tools developed within the theory oflinear semi -infinite optimization can be successfully applied to several problemsin probability and statistics, including a subjective view on probability theorymaintained by de Finelli, a constrained maximum likelihood estimation problem,and some relevant topics in risk theory. This work is intended as an addendumto the review of LSIP applications contained in [5].

1 INTRODUCTION

As it often happens in operations research, an optimization problem shouldbe judged, among other important criteria, for the quantity and variety of applications this mathematical framework is able to work out. Two reasons thatexplain the recognized (and potentially huge) appeal of Linear Semi-InfiniteProgramming (LSIP) in the applied world are:

• LSIP is a very natural extension of finite LP, which is probably themost successful mathematical tool in thousands of applications, Often, a problem that is intrinsically "continuous" has been adapted tofinite LP through a discretization process, when LSIP was not availableon the market of optimization methods. The discretization in questioninevitably brings a degree of imprecision associated to the problem's solution. More importantly, this operation raises the level of arbitrarinessinherent to mathematical modeling, which, though unavoidable, mustbe kept as low as possible. LSIP helps overcome those difficulties byintroducing a "continuous" number of constraints.

237

M.t\. Goberna and M.A. LOpez (eds.), Semi-Infinite Programming, 237-254.© 200 1 Kluwer Academic Publishers.

238 St.MI-INFINITE PROGRAMMING. RECENTADVANCES

• The dual formulation of a (primal) LSIP problem in Haar's sense oftencoincides with a generalized moment problem, where the range of anintegral with respect to a measure satisfying a finite number of integralconstraints, is sought. Since the latter is quite frequent in many instancesof statistics and applied probability and - apart from very simple cases- it is difficult to solve with the sole aid of probabilistic tools, it isclear that LSIP, with its wealth of theory and algorithms, becomes anessential ally to tackle any serious application involving the generalizedmoment problem.

The first two chapters ofthe book [5] by Goberna and Lopez provide an extensive review of applications ofLSIP in the most diverse fields. Without forgettingthe rich interaction provided by duality theory, Chapter 1 shows several applications of the primal LSIP problem, including examples of approximation theory,pattern recognition, optimal control and statistics. Symmetrically, Chapter 2lists several usages of the dual, ranging from environmental policy-making, tologistics and statistics .

The aim of the present work is to provide a simple addendum to that work inthe direction of probability theory and statistics. Three other applications arepresented: Section 2 is an original contribution and explains how the subjectivetheory ofprobability is tightly connected with LSIP; Section 3 reports on a cleverusage of LSIP by Shaw and Geyer [13] to constrained maximum likelihoodestimation in statistical inference. Finally, Section 4 gives an account of theample work by Goovaerts, De Vylder and Haesendonck, finalized in a longseries of papers and in the book [6], on risk theory.

Following the classification principle adopted in [5], Sections 2 and 3 describe "primal" applications, while Section 4 focuses on a "dual" usage of LSIP.

2 DE FINETTI COHERENCE

In the subjective approach to probability proposed by de Finetti (with theinteraction and contribution of other researchers), the study of random entities includes all instances whose realization is unknown to the person who isexamining the entity in question.

From this perspective, the realm of applications of probability theory isextremely wide, encompassing not only situations where some symmetry ispresent - so that equal probability can be assigned to every occurrence or instances that can be repeated arbitrarily often - so to assign a frequencyto every occurrence - but any event which is unknown to the observer, eventhose that happened in the past. Moreover, as the term "subjective" suggests,different observers may have different information on the same random entity,and they may arrive to different probabilistic conclusions.

LINEAR SIP IN PROBABILITY AND STATISTICS 239

Let n be a set of states w which describe the possible occurrences in someaspect of the world on which the interest is focused. The set n is called thepossibility space. The states w are mutually exclusive (no more than one stateoccurs) and exhaustive (one state must occur) in n. Any subset of n is calledan event. We will focus on random entities whose outcome is numeric; i.e, anyreal valued function X on n. X is called a random quantity (r.q.).

We point out that, at this stage, no structure is required on n (while themost common axiomatization by Kolmogorov requires that a a-algebra is seton n). Also, it must be noted that in the original work by de Finetti randomquantities are introduced with no mention of an underlying space n. As a matterof fact, given any class of r.q.'s, it is always possible to specify an underlyingpossibility space that supports all the r.q.'s (see section 2.1.4 in [14]). Thisslight divergence from the original exposition is needed to better understandthe relationship between subjective probability and different kinds of linearprogramming problems.

In what follows, all r.q.'s are required to be bounded, i.e.

-00 < inf X(w) ~ sup X(w) < +00 .wEn wEn

Now, suppose that a person, say B (for booker - de Finetti, following Goodand Savage, uses the pronoun "you" instead to denote this person) wants tosummarize his degree of belief in the different values of a r.q. X with a singlenumber PX. According to B, the r.q. X and the number PX have the sameworth - in the sense that B is willing to exchange the random number Xfor the deterministic number px , and, conversely, to exchange PX for X. Inorder to make this notion operative, de Finetti devises the following bettingscheme: B is obliged to accept any bet with another person G (the gambler)with gain e (X(w) - px), where the real number PX is specified by Bandthe real number e is chosen by G. More precisely, if e > 0, then B is buyingthe random quantity eX and he is paying the price epx for it. The net payoff(gain or loss) of the bet is e (X(w) - px) and depends, of course, on whichrealization w of the possibility space n will occur. If, instead, e < 0, B isselling the random quantity - e X for the price -ePX . Once again, the netpayoff of the transaction will be e (X(w) - px).

It is important in this procedure that B is not deciding in advance whether heis going to buy or sell X, and therefore he cannot take any advantage in deviatingfrom the value PX that makes the payoffs e (X(w) - px) and -e(X(w) - px)equivalent to his judgment.

The number PX is called the prevision of X. If E is an event (E C n) thenthe probability of E is defined as the prevision of the indicator function of E.

Conceptual difficulties arise in the definition just given when the payoff is toohigh. These problems are linked with the utility of the amount paid or received.To overcome such questions, it is usually assumed that the constant c is chosenso that [c (X(w) - px)1 < C for every wand some threshold C > O.

A more recent approach allows a whole interval ofacceptable values [p~ ,Px1for the transaction ofX. We follow the original scheme by de Finetti to illustratehow the very founding ideas of subjective probability are intimately connectedwith semi-infinite programming techniques. We refer to [14] for a full accountof these recent developments.

2.1 DEFINITION OF COHERENCE

The choice of the value of the prevision for a r.q. X falls entirely upon thesubject B, but there is one important rule that needs to be observed in theassessment of Px: for no choice of c the bet must lead to a sure loss (or equivalently - to a sure gain) . That is, there is no real c and 10 > 0 such that

c(X(w) - px) ~-€ for every wEn.

It is straightforward to verify the equivalence of the last inequality to

inf X(w) ~ PX ~ supX(w),wEn wEn

(Cd

a very reasonable condition indeed. If this happens we say that the prevision Pis coherent. The same requirement becomes less trivial if it is used to verify theproper assessment of the previsions for a set of r.q.'s . Suppose B is examininga set X = (Xl, ... ,Xn ) of r.q.'s . When assessing a prevision Pi = PXifor each Xi, B is ready to accept a composite bet with total payoff given byEi Ci (Xi(W) - Pi) . This system of bets should not lead to a sure loss situation,where, for some 10 > 0,

n

L Ci (Xi(w) - pd ~ -10

i=l

for every wEn .

If this occurs, the previsions p = (PI, ... ,Pn) are incoherent for X coherent otherwise.

This definition can be easily extended to any (infinite) class of r.q.'s: Thecorresponding previsions are coherent if this property holds for any finite subsetof the class.

De Finetti states that coherence is all that theory can provide about previsions. Therefore, constructive methods to verify the coherence of a system of

previsions must receive great attention. In the case we are dealing with a finiteset X of r.q.'s accompanied by the corresponding previsions p, de Finetti asserts([4, Vol.1, Par. 4.1]) that p is coherent for X if and only if

P E (co (X(w) : wE 0))-,

where co A and A-denote, respectively, the convex hull and the closure of A.This geometrical description does not provide a constructive procedure to

verify coherence yet. Instead this tool is provided by linear programming andits extensions. Consider the following program in the real variables C1, ... ,enand t:

inf t

s.t. L:~=i q(Xi(W) - Pi) + t ~ 0, for all wE O.

We denote with v(P) the optimal value of problem (P). It is easy to verifythat the set of previsions p is incoherent (resp . coherent) for X if and only ifv(Pc ) = -00 (resp . v(Pc ) = 0) .

The cardinality of n determines the structure of (Pc). If Inl = +00 we aredealing with a semi-infinite linear programming problem. If Inl < 00 this is aclassical linear finite program.

2.2 HOW TO MEASURE INCOHERENCE

Suppose now that B has specified an incoherent set of previsions p and hewants to have an idea of the degree of incoherence in the overall specification.Schervish, Seidenfeld and Kadane [11] have recently proposed an index p whichranges between 0 and 1, the two bounds denoting a situation of coherenceand maximum incoherence, respectively. If the previsions are incoherent, thedefinition asserts that the competitor G may chose some constants ci, . . . ,ensuch that maximum payoff to B is negative:

n

V(Ci,'" ,cn ) := supI: q (Xi(w) - pd < 0 .WEfli= i

The value V can be made arbitrarily large in absolute value by a suitablechoice of the constants q. Instead, Schervish et at. consider a "constrained"loss . When Band G agree to bet on the r.q. Xi with the procedure seen above,G would like some sort of guarantee that B would be able to pay in case heloses. The maximum amount of money that B can lose in gamble i is


e(Ci) := - min {O,i~ Ci (Xi(W) - Pi)} .

Now, suppose that B is considering a different gambler, say Gi, for each beti and every gambler wants a guarantee that B can pay up in the worst possiblecase for him. Thus, an escrow account is introduced, where B must pay themaximum amount that he might lose in every gamble he is involved in. Theamount is given by

n

E(Cl,' " , cn) = L e(Ci) .i=l

The amount e(Ci) is called the booker's escrow for bet i , and E(Cl, ... , cn)is called the booker's composite escrow.

Schervish et al. [11] suggest a measure of incoherence which values howmuch B can be forced to lose for each unit of money paid into the escrowaccount when B has made incoherent previsions. Therefore they design anoptimization problem which computes the most unfavorable bet for B, subjectto the escrow account not exceeding one

p:= -infc V(cl , ... ,en)s.t. E(Cl, ' " ,cn) ~ 1.

The number p is called the extent of incoherence or maximum rate ofguaranteed loss. In case the previsions do satisfy coherency, then p = 0 (in thiscase V 2: 0 for every choice of the Ci, and V(O, ... ,0) = 0) (while p > 0 incase of incoherence). Also, since B cannot lose more than the amount put inthe escrow, p ~ 1 holds.

Problem (Pp) can be expressed as a linear program in the variables Ci, Zi,i = 1, .. . , n , and t:

p:= sups.t,

tLi Ci (Xi(w) - Pi) - t

Ci (Xi(W) - Pi) + ZiLi Zi

Zit

~O

2:0~1

2:02: O.

for all W

for all w, i

for all i

(KSS)

As before, when Inl = +00 this is a linear semi-infinite problem.Schervish et al. [11] also define another index of incoherence, called the

maximum rate ofguaranteed profit, where the escrow is paid by G against B .This index is non-negative, but unfortunately has no upper bound, which makesits interpretation more problematic.

LINEAR SIP IN PROBABILITYAND STAI1STICS 243

2.3 EXTENSION OF A COHERENT SYSTEM

As can be perceived from the very definition, the assessment of coherentprevisions grows more and more difficult with the number of r.q.'s considered.Suppose that Y is the only r.q. we are examining. Then, by (Cl ) , any numberpy comprised between the extremal values of Y defines a coherent prevision.Suppose instead that a system of coherent previsions p are already declared forthe set X. Then, the same number py which was coherent for Y alone may losethis property when Y is inserted in this larger system of r.q.'s. In fact, whilepy is coherent for Y and the elements of p are coherent for X, the previsions(py, p) may be incoherent for (Y, X) altogether.

An extreme case is given when Y is linearly dependent from the Xi'S; i.e.Y(w) = L:i.Bi Xi(W) for all wand some real numbers .Bi, i = 1, ... , n . Inthis situation, the only allowable value for a coherent prevision of Y is py =L: i.BiPi; i.e, coherence implies the linearity of previsions. More generally,an important result originally established by de Finetti, states that a wholeinterval of values [PI, Pu] is given, such that if py falls in this interval, thenthe set of previsions (py, p) are coherent for (Y, X). The values PI and Pucan be computed through the following linear problems in the real variables

Yo, Yl,··· ,Yn:

PI := SUPy Yo + L:iYiPi (Pds.t, Yo + L:i Yi Xi(W) ::; Y(w) , VWEO,

and

Pu := infy Yo + L:iYiPi (Pu)s.t. Yo + L:iYi X i(w) ~ Y(w) , VwEO.

A statement of this result in a context different from LSIP theory, togetherwith its proof, can be found in [15, Theorem 15.4.1].

In the original statement by de Finetti, previsions are replaced by events , butthe proof can be extended to the case considered here without much effort. Theoriginal result dates back to 1935 and is so important that it was later giventhe name of Fundamental Theorem of Probability by de Finetti himself. Theimportance of such result stems from the fact that this result yields a constructive, step-by-step procedure to actually build a coherent system of r.q.'s. Also,many inequalities in probability theory can be explained by the FundamentalTheorem. In [15, p. 278] , for instance, it is explained how the Markov and theChebyshev inequalities can be derived from de Finetti's result.

The usual remark about the cardinality of 0 applies. In the case 101 < +00the link between this theorem and finite linear programming was analyzed


in [1]. A more detailed discussion of the long history of this result and itsgeneralizations appears in [9].

2.4 SOME RESULTS ON DUALITY

As already pointed out, when n contains an infinite number of elements,problems (Pd and (Pu) are two particular cases of primal linear semi-infiniteprogramming problems, where both the objective function and the constraintsw.r.t. a finite number of variables are linear, and the number of constraints isallowed to be infinite. Following the theoretical development set forth in thebook by Gobema and Lopez [5], problems of this kind admit a dual formulationknown as the dual problem in Haar's sense, in which the number of constraintsis finite, while there may be an infinite number of variables. We examine (PL)first.

Let R~n) be the set ofall functions A : W H >.w E lR.t-, that vanish everywhereexcept on a finite subset of n, the so-called supporting set of A, denoted bysp A := {w En: >.w # O} = {WI, .. . ,Wk} for some integer k. The elements

of R~) are called the generalized finite sequences in lR.t- . Adopting the generalscheme illustrated in Chapter 2 of [5], the dual problem of (Pd is:

(D£)i = 1, . . . ,n,Pi,

1,

inf,\ 'l/J{A) = :EwEsp x >.w Y{w)s.t. :EwESP,\ >.w Xi{W)

:EwEsp x Aw

A E R~) .

Similarly (PL) admits a dual problem in Haar's sense (Du), which differsfrom (DL ) only in that inf,\ is replaced by sup,\.

The dual problems formulated above have a straightforward interpretation

in probabilistic terms. Any generalized finite sequence A E R~) such thatEWESP x Aw = 1 is known as an atomic probability measures on n, and

the expected value of a r.q. X with respect to this measure is E{X) =EWESP,\ Aw X{w). Therefore, the optimal value v{DL ) (resp . v{Du)) is thesmallest (resp. highest) expected value for Y with respect to all atomic probabilrty measures on n that verify the constraints E{Xi) = Pi, i = 1, ... , n.

A statement of the dual problem appears in [15, Comment 1, p. 277]. Herewe are interested in examining these results in the light of LSIP duality theory.

As shown by equation (2.1) in [5], the following weak duality inequalitiesalways hold

v{PIJ ~ v{DIJ and v{Du) ~ v{Pu) . (2.1)


To obtain two strict equality signs in the above equation, and therefore toestablish a strong duality link between (Pd and (DL) (resp. between (Pu ) and(Du», we resort to the available results on duality theory in LSIP.

LetM be the convex hull generated by the range ofther.q.'s Xi, i = 1, ... ,n,

M = co (X(w) : wEn) .

We also remind that all r.q.'s considered are bounded . Taking into accountthat the so-called first moment cone of the (consistent) systems given by (Pdand (Pu) is cone({1} x M) c ]Rn+l, an application of Theorem 8.2 in [5]yields the following results:

(i) If p E rint M (the relative interior of M), then strong duality holds:

and v(Du) = v(Pu)· (SD)

(ii) Suppose that p E rbd M (the relative boundary of M). Then

• a necessary condition for (SD) to hold is p E M;

• a sufficient condition for (SD) is the following : p E M, n compactsubset of]Rk , and all r.q.'s are continuous.

2.5 A NUMERICAL EXAMPLE

The following numerical example highlights the role played by linear semiinfinite programming in verifying coherence in not-sa-trivial situations. Acutting plane discretization method, similar to the conceptual Algorithm 1104.1in [5], is used throughout the computation of LSIP problems. All programsare implemented using some ad hoc routines written in the Mathematica 3.0programming language.

Suppose n = [0,1] and consider the following r.q.'s:

Xdw) - B(tS)w (1 - w)7

X 2 (w) - 2w

X ( ) 1 5 (1 )0 .53 w B(6,1.5)w - w ,

where B(2, 8) and B(6, 1.5) are two Beta integrals.Suppose that B has specified - after careful consideration - the following

set of previsions p = (0.9,0.1,0.5).


The first check on coherency reveals v(Pc) = -00. Therefore, p is incoherent for (Xl , X 2 , X 3 ) . We are now in the position to compute the rateof incoherence for the whole system. By solving problem (K88), we obtainv(K88) = 0.09188.

Suppose instead that another set of previsions is assigned to the same set ofr.q.'s, p' = (0.4, 0.75,0.6).

In this case v(Pc) = 0 and those new previsions are coherent. Now B isasked to provide a prevision for another r.q. Y(w) = -3w2 + 4w. In order forthe prevision py to be coherent with the other statements, py it must lie in therange [0.398353,0.83822] .

3 CONSTRAINED MAXIMUM LIKELIHOODESTIMATION OF A COVARIANCE MATRIX

An example of the fruitful usage of LSIP to solve constrained maximumlikelihood estimation problems in statistical inference, is illustrated in section1.7 of [5]. A more recent example is shown here.

Shaw and Geyer in [13] consider a variance component model (defined e.g.in [12]), where certain covariance matrices A( 9), whose clements are linearwith respect to a finite dimensional vector of parameters 9, are positive semidefinite . The maximum likelihood (ML) estimation method does not yield anyguarantee that A(9), the covariance matrix corresponding to the ML estimates

9of 8, shares the same property.This can be achieved by setting up a constrained optimization problem, where

the likelihood of the parameters is maximized subject to the requirement that theprincipal minors of A(8) be non-negative. These constraints, however, involvehigh-dimensional determinants and are very difficult to implement. It is muchsimpler to resort to the original definition of positive definiteness and considera problem such as:

suPos.t.

£(9)v' A(8) v ~ 0 , for every unit vector v,

8 and v being two vectors with equal dimension and £(8) being the likelihood(or the log-likelihood) of the model.

If £(8) is linear in the parameter vector, this is a LSIP problem. It is interestingto note that usually in LSIP models the (infinite) index set of the constraints is aninterval of the real line or a Cartesian product of intervals, while here the indexset is the "sphere" of all unit vectors of given dimension. This is an example ofclever modeling that reveals the flexibility and power of LSIP models.

A cutting plane algorithm, classifiable as conceptual Algorithm 11.4.1 in [5],is employed. At each step, after maximizing £ subject to the finite current setof constraints, corresponding to a finite number of unit vectors v, the eigen-


values of A( 8) are computed. If any eigenvalue is negative, the eigenvectorv associated with the most negative eigenvalue is used to form the new constraint v'A(8) v ~ 0 to be imposed, in addition to the existing ones, in the nextiteration.

4 LSIP IN ACTUARIAL RISK THEORY

The same betting scheme which is so important in the foundations of subjective probability, also plays a fundamental role in many actual economicapplications. For example, the notion of incoherence is intimately related tothat of arbitrage in mathematical finance.

Another important example is given by risk theory in actuarial sciences. Alsoin this instance LSIP plays an important role - quite different from its use insubjective probability.

In what follows, we outline how Goovaerts, De Vylder and Haesendoncksuccessfully applied what we now call semi-infinite programming techniquesto some important problems of actuarial sciences between 1982 and 1984. Infact they do not refer to LSIP literature (which was less developed at that time)nor to the geometric approach to the generalized moment problem set forth byKemperman (see e.g. [7] and [8]), and instead they develop a self containedtheoretical framework which allows them to gather a considerable amount ofinformation on the problem under investigation .

In order to explain their work, we must return to the probabilistic backgroundthey used, which is based upon the classical probability axioms by Kolmogorov(we refer to any basic book on probability theory for their explanation). Hereo is a probability space endowed with a a -algebra :F, containing all events(subsets of Q) of interest, and a probability measure P is defined on (0 , :F). Inthis context, a random variable (r.v.) X is any :F-measurable real valued functionon O. The expected value of a r.v. X is defined as E(X) = In X(w) dP(w),wherethe integral sign denotes a Stieltjes integral.

Consider a customer of an insurance company who pays a fixed amount ofmoney, called premium, to cover himself against a random damage which maybe too harmful for him to face on his own. Let the r,v. X describe the claim; i.e.,the monetary compensation for the damage. The computation of the premiummust take into account the following components:

• a theoretical premium, which is the deterministic equivalent of the random claim;

• an additional amount for commission and expenses of the insurancecompany.

Those two addends form the actual premium. A typical premium is givenby the following simple formula


prl (X) = (1 + A) E(X)

where A > 0 represents the portion due to commission and expenses. Otherpremiums frequently used take the variability of the claim into account. Forinstance

where Q > 0 and (T2(X) is the variance of X, or

pr3(X) = E(X) + ,B ess sup X(w),w

where ess sup X is the essential supremum of X (i.e., the supremum obtainedby neglecting sets of values of P-measure zero) .

For a discussion of the properties of the different premiums and a guide totheir use, the reader is referred to [2]. In all cases, the computation of theexpected value of the claim X plays a very important role.

Goovaerts et al. [6] addressed the problem of computing the premium of aclaim and, to adhere more closely to the actuarial practice, they make severalremarks on the topic.

First of all, a non-negative amount of money, up to a threshold t, is oftenretained from the payment of the claim X. Therefore, instead of consideringthe expectation of X, one should consider the expected value of the trimmedquantity (X - t)+ (where (a)+ equals a if a ~ 0 and 0 otherwise). In otherwords, the following value should be computed, instead of E(X):

prsl = E(X - t)+ = l (X - t)+ dP;

prsl is called the stop-loss premium. The transformation from X to (X - t)+,however simple, poses additional computational difficulties.

The case where the insurance company knows in every detail the elements(i.e. the probability measure P and the r,v. X) needed to compute the premiumis a very rare one. More common is the situation where, with some effort, somecharacteristics are known, such as the expected value (not the trimmed one ofthe stop-loss premium), or its variance, or the fact that something is knownabout the "shape" of P .

So, instead of the original premium, it is more reasonable to fix the knownfeatures and compute the range of the premiums among all claims X and measures P sharing those features. It is assumed here that X is known, while thelack of knowledge affects P .

(Bd')


More formally, let P be a class of probability measures on (0, F) sharingthe known features. The range of the premium is then given by the interval[pr~l,pr~l]' where

pr~l = inf { (X - t)+ dP and pr~l = sup r(X - t)+ dP . (Bd)PEP in PEPin

Goovaerts et al. study the very general case where the class P can be definedby a finite set of integral constraints.

It is well known that the r.v. X induces a probability distribution on (R, B),usually represented by its distribution function Fg . Therefore the expected ofX value can be expressed as E(X) = fIR x dFx(x) .

The problems (Bd) can be alternatively formulated as follows. Let V bea set of distribution functions on (R, B) that fulfill a finite number of integralconstraints. Instead of pr~l' the focus is on

pr~l = inf r(x - t)+ dF(x)FEV iIR

and, similarly, we proceed for the upper bound.The two approaches (Bd) and (Bd') are formally equivalent. The latter offers

a slightly better understanding of the problem's meaning, since, at best, one canobserve some features of the claim's distribution on lR. The former, however,allows a very general statement of duality theory and is therefore preferred bythe authors for a description of their general results .

Other difficulties about the computation ofthe claim pertain to the peculiari tyof the actuarial problem: the claim refers to a fixed period of time, say a year.The claim may refer to more than one unfavorable event, thus, often, it is moresuitable to model the claim as a sum of claims (with a random number ofaddends, each one represented by a r.v.).

More importantly, the premium is not computed for an individual customeronly, but it refers to the claims in a population of customers. Now, if thepopulation is homogeneous, this fact poses no problem. Otherwise we shouldconsider some sort of "average" of the individual claims. In this case, the notionof mixture is helpful.

Let (8,9) be another measurable space and let W be a probability measureon it. If Po(A) is a function on F x 8, such that Po(-) is a probability measureon (0, F) for almost all 0 E 8 and, moreover, Po(A) is Q-measurable for everyA E F, then

P(A) = Ie Po(A), dW(O) AEF, (Mix)


is a probability measure denoted mixture of {Po, (J E 8} (see [3, p. 190]).

The use of mixtures is often visualized by a two stage process: The actualclaim is determined by first picking out (with distribution W) the person whois sustaining the damage and, then, observing the realization of the claim concerning the given person (in a more abstract setting this is known as the "urn ofurns" scheme).

Mixtures are also very helpful in characterizing a probability measure. Anexample will be given later with the class of measures with unimodal density.

Goovaerts et at. [6] focus on the computation of the bounds defined in(Bd). In order to achieve this aim they define a general optimization problemsometimes known as the generalized moment problem.

Before proceeding, we briefly review the most relevant assumptions anddefinitions which are needed to define and solve the optimization problem.First of all, in order to develop the theory in its full extent (and, especially,duality theory) P will be replaced by a larger set M of non-negative finitemeasures (such that the measure of the whole space 0 does not necessarily addup to one) on which a normalizing condition is set, so that the search is reducedto probability measures only.

Moreover, the following technical assumptions are required:

• The a-algebra :F contains all singletons {w} of 0;

• M denotes a cone of non-negative measures (O,:F) that contains allatomic measures (the notion of atomic measure has been already definedin section 2.4);

• M is assumed to be convex.

Let now Ji : 0 -+ R, (i = 0,1, ... ,n) be a finite set of :F-measurablefunctions such that the integrals JJi(w) dJ.L(w) exist and are finite for all J.L E Mand all i. All integrals are taken over O. Also, let z = (Zl,." ,zn) be a ndimensional vector.

Goovaerts et at. tackle the following generalized moment problem. Find

Similarly the problem where sup is replaced by inf is also considered.

In all the ensuing applications, the authors set 10 = (X - t)+, h == 1 andZl = 1. Thus , the stop-loss premium always plays the role of the objective function, and M is always a subset of probability measures. The other constraintsusually pertain to the first moments of the measure.


The effective domain of the function u ; i.e., the set of vectors z for which theoptimization problem is properly defined, is:

dom u = {(/ t, du, .. . , / fn dp.) : J1- EM}In case there are only three constraints, or less, in problem (Dsl) and the

functions Ii have a "simple" structure, the authors propose to render u explicitlyas a function of z by computing, for every z E domou (the interior of dom u),the following geometrical problem:

where

u(z) = sup {a : (a, z) E Cu } ,a

Cu = co {(fo(w), fdw) , ... , fn(w)) : WED} .

(AS)

A convincing illustration of this method is to be found in pages 274-275 of[6]. This method and a considerable effort allow the authors to draw up a listof ready-to-use solutions (i.e. the analytic statement of u) for a considerableassortment of different cases.

In more general circumstances, the same authors propose the formulation ofa dual problem, together with a numerical recipe for its solution, which actuallyis a particular case of a LSIP procedure.

A dual problem for (Dsl) is given by

v(o) ~ yri. (t, V, Z, , t,V;!' (W) 2: fo(w) forcvcryw En), (P,il

and the following weak duality result always holds

u(z) :s v(z) .

Goovaerts et al. establish the following strong duality result ([6, Theorem 1,p. 271]) : ifz E dorn'iu and the concave function u is proper (i.e., v(z) is finitefor some z), then

u(z) = v(z) .

This result is obtained using specialized results in finite dimensional convexanalysis.

These authors also find sufficient conditions for the existence of an optimalsolution which is also n-atomic ([6, Theorem 12, p. 272]). If n c IRk, ncompact, :F is the Borel o-algebra, all Ii are continuous, u is proper and z Edom'lu, then u(z) admits an atomic solution having at most n atoms.

To solve (Psi) numerically, a sequence of discretized problems is considered. For any set J c n let vJ(z) be the optimization problem (Psi) inwhich the constraints are verified only for the points in J, and let EJ :={(fo(w),/dw), ... ,In(w)) : w E J}. Furthermore, let {Il,!2, ... } be anincreasing sequence of non-void subsets of n and set 10 = Uk h.

The authors prove ([6, Theorem I, p. 331]) that, when ETc is dense in Enand u(z) is proper, then

u(z) = lim UTIc (z) .k->+oo

Those conditions are fulfilled when 10 is dense in n c IRk, and all thefunctions Ii are continuous. If, moreover, z E dorn'iu, yk E IRn is a solutionfor UTIc (z), k = 1,2, ... , and yk -+ y, then y is a solution for u(z) .

In Section 5.3.4 of [6] the following case involving mixtures of distributionsis considered in detail. Let n = I , an interval of IR endowed with the usualBorel zr-algebra B, and fix m E I. The set M cu includes all measures J.L on Ithat can be written as

J.L(E) = Lcp(x) dx, for any E E B,

cp being some B-measurable function which is increasing (not necessarilystrictly) at the left of m, and decreasing (not necessarily strictly) at the right ofm. Every such J.L is called a continuous in-unimodal measure. The problemucu(z), where the generic Min (DsI) is replaced by M cu is considered here .

The key to solve U cu (z) is given by the fact that each continuous m-unimodalmeasure can be expressed in the form (Mix) as a mixture of uniform measureshaving one endpoint in m (the fact that here arbitrary non-negative measuresare considered does not change the essence of the computation).

For any 0 E I, denote by 10 the indicator function of the closed interval withextremes m and O. Thus, 10 = I[O ,m) if 0 < m, and 10 = l[m,O) if m < O.

It is easy to verify that any unimodal density cp can be written as a mixture ofrectangles 10 with the same measure W characterizing the mixture of measures(Mix).


A simple application of the Fubini Theorem (see e.g. [3, Theorem 2, p.186]) allows to solve U cu (z) both analytically and numerically. In fact, if weconsider for each i = 0, 1, . . . ,n

then the dual problem for U cu (z) becomes

A strong duality result follows ([6, Theorem 1, p. 280]): if U cu is proper,then ucu(z) = vcu(z), for all z E domoucu •

Similar results are given when the unimodal J.L is allowed to have a peak ofpositive mass concentrated in m . Also, other results pertain the shape of theoptimizing measure in M cu : similarly to the original problem (D s1)' undersuitable conditions, ucu(z) admits an optimal solution given by a finite linearcombination of rectangles.

Further examples of mixtures in the generalized moment problem (Ds1) arementioned in Section 6.3 of [6].

In Section 5.3.5 , the generalized moment problem where one or more integralconstraints are defined by inequalities is considered. This generalization is mostwelcome, since, when it comes to eliciting some features of J.L - expressible asgeneralized moments, it is often the case that a whole range of admissible valuesare available - the definition of a single value being too costly to acquire interms of time, money and other resources. The authors show that, whenever "= "is replaced by"::;" for the i-th constraint in (Ds1), the corresponding variableYi in the dual (Dsl), must be non-negative (a well known fact in finite LP).

Overall, the work by Goovaerts et al. is remarkable for its completenessand for its pioneering exploration of the relationship between the generalizedmoment problem and what is now known as the LSIP problem. Certainlya new reading of this work in the light of the most recent advances in thisbranch of operation research - with respect to both theoretical and algorithmicdevelopments - could further improve its appeal.

Acknowledgement

This work has been partially supported by CNR and COFIN-MURST 1999.


References

[I] G. Bruno and A. Gilio. Applicazione del metodo del simplesso al teoremafundamentale per la probabilita nella concezione soggettivistica, Statistica, 40:337-344, 1980.

[2] H. Biihlmann. Mathematical Methods in Risk Theory, Springer, 1970.

[3] Y.S. Chow and H. Teicher. Probability Theory (3rd ed.), Springer-Verlag,1997.

[4] B. de Finetti. Theory of Probability. A critical Introductory Treatment,Vol. 1 and 2, Wiley, 1974.

[5] M.A. Gobema and M.A. Lopez. Linear Semi-Infinite Optimization, Wiley,1998.

[6] MJ. Goovaerts, F. De Vylder and J. Haezendonck, Insurance Premiums.Theory and Applications, North Holland, 1984.

[7] J.H.B. Kemperman. The general moment problem: a geometric approach,Annals ofMathematical Statistics, 39:93-122, 1968.

[8] J.H.B. Kemperman. Geometry of the moment problem. In J. Landau,editor, Moments in mathematics, Volume 37 of Proceedings of Symposiain Applied Mathematics, pages 16-53, American Mathematical Society,1987.

[9] E Lad. Operational Subjective Statistical Methods. A Mathematical,Philosophical and Historical Introduction, Wiley, 1996.

[10] E. Regazzini. Subjective Probabilities. In N.L. Johnson, S. Kotz and C.B.Read, editors, Encyclopedia ofStatistical Sciences, Wiley, 1980.

[11] MJ. Schervish, MJ., T. Seidenfeld and J.B. Kadane . Two Measures ofIncoherence: How Not to Gamble if You Must, Technical Report No. 660,Department of Statistics, Carnegie Mellon University, 1998.

[12] S.R. Searle, G. Casella and C.E. McCulloch. Variance components, Wiley,1993.

[13] EH. Shaw and CJ. Geyer. Estimation and testing in constrained covariance component models, Biometrika, 84:95-102, 1997.

[14] P. Walley. Statistical Reasoning with Imprecise Probabilities, Chapmanand Hall, 1991.

[15] P. Whittle. Probability via Expectation (4th ed.) , Springer-Verlag, 2000.

Chapter 12

SEPARATION BY HYPERPLANES: A LINEAR SEMIINFINITE PROGRAMMING APPROACH

Miguel A. Goberna', Marco A. Lopez! and Soon-Yi Wu2

1Departm ent of Statistics and Operations Research, Faculty of Sciences, Al icante University,

O ra. San Vicente de Raspeig sin, 03071 Alicante, Spain

2Department ofMathemati cs, National Cheng Kung University, Tainan, Taiwan

[email protected], marco [email protected], [email protected] .edu.tw

Abstract In this paper we analyze the separability and the strong separability of two givensets in a real nonned space by means of a topological hyperplane. The existenceof such a separati ng hyperplane is characterized by the negativity of the optimalvalue of some related (infinite dimensional) linear optimization problems. In thefinite dimen sional case , such hyperplane can be effectively calculated by means ofwell-known linear semi-i nfinite optimization methods. If the sets to be separatedare compact, and they are contained in a separable nonned space, a conceptualcutting plane algorithm for their strong separation is proposed. This algorithmsolves a semi-infinite programming problem (in the sense that it has finitelymany constraints) at each iteration, and its convergence to an optimal solution,providing the desired separating hyperplane, is detailedly studied. Finally, thestrong separation of finite sets in the Hadamard space is also approached, and agrid discretization method is proposed in this case.

1 INTRODUCTION

Separation of finite sets in finite dimensional spaces is a basic tool in patternrecognition and other fields of applied mathematics. In [7], Rosen reported anapplication of separation by hyperplanes to weather forecasting, and observedthat the problem of finding a hyperplane separating strongly two finite setsin JRn could be formulated as a convex quadratic programming problem, Forthis kind of mathematical programming problems many standard methods areavailable. Recently, Botkin ([2]) has designed a randomized algorithm withexpected running time 0 {nn! (p + q)), where p and q are the cardinalities ofthe sets to be separated, The present paper discusses the separation of two sets

255

M.A.. Goberna and M.A. Lopez (eds.), Semi-Infinite Programming, 255-269 .© 200 1 Kluwer Academic Publishers.

by hyperplanes when either the cardinality of the two sets to be separated or thedimension of the space, or even both, are infinite. In many cases these separationproblems can be formulated as either a linear semi-infinite programming (LSIP)problem, or as an optimization problem for which an optimal solution canbe obtained as the limit of a sequence of optimal solutions of certain LSIPsubproblems.

We shall consider throughout the paper that the sets to be separated, Y and Z,are nonempty subsets of a given real normed space X whose topological dualspace is X*. We shall denote by 8 the null vector indistinctily in X and in X*(so, 8 = On if X = X* = }Rn). Given a nonempty set S of a normed space, wedenote by cony S, cone S, c1 Sand rint S the convex hull of S, the convex cone(also called wedge) spanned by S, the closure of S, and the relative interior ofS, respectively. Here, the topological operators are defined with respect to thetopology of the norm.

The reference cone associated with the linear inequality system {cp (Xj) ~

aj,j E J}, with data {Xj,j E J} c X and {aj,j E J} c lR, and unknowncp E X* , is

c1cone{(xj,aj),j E J; (8,-1)} c X x R

The extended nonhomogeneous Farkas Lemma (Theorem 2 in [8]) establishesthat the linear inequality sp (x) ~ a is a consequence of the above system,assumed to be consistent (in other words, the system has a nonempty solutionset), if and only if (x, a) belongs to the reference cone. From this result. andthe separation theorem, it can be proved that two consistent linear inequalitysystems have the same solution set if and only if their reference cones coincide.

An affine subspace H in X is called a topological hyperplane if there exist alinear functional cp E X* \ {8} and a scalar a such that H = {x E X I sp (x) =a}. We say that H separates Y and Z weakly if ip (y) ~ a ~ ip (z) for ally E Y and for all z E Z, and that H separates Y and Z strongly if there exists apositive scalar e such that ip (y) ~ a - c: < a +E ~ sp (z) for all y E Y and forall z E Z. Observe that Y and Z can be interchanged in the above definitions.

We associate with each pair of nonempty subsets of X, Y and Z, the linearsystem, with unknowns cp E X* and a E lR,

a = {cp (y) ~ a ~ ip (z), (y, z) E Y x Z},

as well as the following pair of infinite dimensional mathematical programmingproblems:

SEPARATION BY HYPERPLANES THROUGH LINEAR SIP 257

(P) inf v - us.t. <p(y) +u::; 0, Y E Y

<P (z) + v ~ 0, z E Z1I<p1l ::; 1<P E X* ,u E ~ v E~

and

(Pw ) inf <P (w)s.t. <P (y) + u ::; 0, y E Y

<P (z) + u ~ 0, z E Z1I<p1l ::; 1<pEX*,UE~

where the problem (Pw ) is associated with a fixed w EX. The optimal valuesof these problems are respectively denoted by v(P) and v(Pw ) .

We consider the product spaces X x Rand X x~x~ equipped with the norms

lI(x,8)1I = Vllxll2 + 82 and lI(x, 8, t)1I = VllXll2 +82+ t2 , respectively.Then, the corresponding topological dual spaces are X* x ~ and X* x ~ x ~the first one being the space of variables of (7 and (Pw), and the second onebeing the space of variables of (P) .

The structure of this paper is the following. In Section 2 we analyze theseparability and the strong separability of two given sets in a nonned space,emphasizing the role played by (7, (P) and (Pw ) in the computation of the desired separating hyperplanes. There we point out that, in the finite dimensionalcase, such hyperplanes can be effectively calculated by means of well-knownLSW methods. In Section 3 we propose a conceptual algorithm for the strongseparation of compact sets in separable nonned spaces. This algorithm solves asemi-infinite programming problem (in the sense that it has finitely many constraints) at each iteration. Finally, in Section 4 we consider the strong separationof finite sets in the Hadamard space .

2 SEPARATION IN NORMED SPACES

The first two results in this section provide geometrical characterizations ofthe weakly (strongly) separable pairs of sets.

Proposition 2.1 The following statements are equivalent:(i) There exists a topological hyperplane separating Y and Z weakly,(ii)

clcone{(-y,l),yEY; (z ,-l),zEZ}#XxlR. (2.1)

(iii) Given an arbitrary set W C X such that cone W = X , the optimalvalue v (Pw) is negative for, at least, one element w E W.

Proof. (i) {::} (ii) Obviously, there exists a topological hyperplane separatingY and Z if and only if the homogeneous system (J" has at least a nontrivialsolution. This is equivalent to assert that the reference cone of (J",

cl cone {( -y, 1,0) ,y E Y; (z, -1,0) ,z E Z; (8,0, -1)},

does not coincide with the reference cone of the trivial system

{cp(x) = O,X E X; a = O},

i.e.,

dcone{±(x,O,O),xEX; ±(8,1,0); (8,0,-1)}=XxlRxllL.

Consequently, (J" has at least a nontrivial solution if and only if (2.1) holds.(i) {::} (iii) Let v (Pw ) < 0, w E W, and let (cp, u) be a feasible solution

of (Pw) such that cp(w) < 0. Then sp =I 8 and {x E X I cp(x) = -u} is atopological hyperplane which separates Y and Z weakly.

Conversely, ifY and Z can be separated weakly by a topological hyperplane,there exists a nontrivial solution of (J" , say (cP', a) , such that cP' =I 8. Hence(cp ,u) := 1IcP'1I- 1 (cP', -a) is a feasible solution of (Pw ) ,whichever w E W weconsider. Since IIcpll = 1 = sup {lcp(x)1 : IIxll ::; 1} , we can take x'f' E X suchthat IIx'f'lI ::; 1andcp(x'f') > 0. From the assumption on W, we can write -x'f' =L .AwW, with .Aw ~ °for all w E W. Hence cp(-x'f') = L .AwCP(w) < 0,

wEW wEW

and there must exist w E W such that .Aw > °and cp(w) < 0, which entailsv (Pw) < 0.0

A practical consequence of (iii) is that, in lRn , the problem of calculating ahyperplane separating Y and Z weakly can be solved by means of at most n +1LSIP problems. In this space, the set W can be chosen in different ways so thatits cardinality is n + 1 (for instance, a basis of lRn and the opposite vector of itsalgebraic sum). From the proof it is clear that we can separate Y and Z fromany feasible solution (c, u) such that w' c < °(Le., it is not necessary to attainan optimal solution). Another feature of (Pw ) (and also of (P)) is that, when Yand Z are compact sets (in which case we get the so-called continuous model),the problem can be solved effectively by means of either grid or cutting planediscretization methods (see, e.g., Chapter 11 in [4]).

Proposition 2.2 (i) If there exists a topological hyperplane separating Yand Z strongly. then

(8,0) Et cony {( -y, 1) , Y E Y; (z, -1), z E Z} (2 .2)

or, equivalently,

(cony Y) n (cony Z) = 0. (2.3)

The converse statement holds ifX is a Banach space, both convex hulls cony Yand cony Z are closed, and at least one of the involved sets, for instance Y, iscompact.

(ii) There exists a topological hyperplane in X separating Y and Z stronglyifand only ifv (P) < O.

Proof. (i) Let sp E X*, a E IR and e > 0 such that

cp (y) :::; a - t < a + t :::; ip (z), for all Y E Y and all Z E Z. (2.4)

Assume that (2.2) is not held. Then, there exist nonnegative scalars AI, ... , Apand 'Y1, ... ,'Yq, and associated vectors Yi E Y,i = 1, ...,p, and Zj E Z,j =

p q

1, ... , q, such that 2: Ai + 2: 'Yj = 1 andi=l j=l

p q

(0,0) = 2:Ad -Yi, 1)+2:'Yj (Zj, -1) .i=l j=l

(2.5)

Ifwe take images through (cp, a), and combining (2.4) and (2.5), we obtain thecontradiction

p q

0= 2:Adcp (-Yi) + a) +2:'Yj (cp (Zj) - a) ~ e.i=l j=l

Hence (2.2) holds.Now we prove the equivalence between (2.2) and (2.3).

p

If (2.3) fails (i.e., if (cony Y) n (cony Z) =1= 0) we can write 2: AiYii=l

q

2: 'YjZj, for certain vectors Yi E Y,i = 1, ...,p, and Zj E Z,j = 1, ...,q,j=l

Pand associated nonnegative scalars AI, ... , Ap and 1'1,..., 'Yq, verifying 2: Ai =

i=lq

2: 'Yj = 1. Thus we havej=l

1 p 1 q

(0,0) = 2" 2:Ai (-Yi, 1) + 2" L 'Yj (Zj , -1)i=1 j=1

ECOny{(-y,l),yEY; (z,-l),ZEZ},

so that (2.2) fails.Conversely, if (2.2) fails, then (2.5) is true for certain nonnegative scalars

Al, ...,Ap and ,l" ""q, and vectors Yi E Y,i = 1, ... ,p, and Zj E Z,j =p q p q 1

1, ... , q, such that~ x, +jJ;l Ij = 1. In such a case i~ x, = jJ;l,j = 2' so

that

p q

22: AiYi = 22: IjZj E (cony Y) n (cony Z) ,i=l j=l

and (2.3) will fail.If X is a Banach space, Y is compact and cony Y is closed, statement (3) from

[6, §20.6] entails the compactness of cony Y. Then we shall apply the so-calledstrong separation theorem (see, for instance, Corollary F from [5, §11]).

(ii) Assume first that v (P) < O. Let (cp, u, v) be a feasible solution of (P)such that v - u < O. In this case cp =1= () and for every a E ] -U, -v[, {x E X Icp (x) = a} is a topological hyperplane which separates Y and Z strongly.

Conversely, let {x E X I cp (x) = a} be a topological hyperplane and £>0such that

cp (y) :s; a- e < a+ E :s; cp (z), for all Y E Y and for all Z E Z.

D fini cp E-a d £+a . . h k he rung cp = IIcpll' U = IIcpll' an v = - IIcpli ' It IS easy to c cc t at

(cp, u, v) is a feasible solution of (P) such that v - u = -2£/ II~II < O. Hencev (P) < 0.0

Remark 2.1 (a) According to Theorem C from [5, §19], the property ofthat each pair of disjoint closed closed convex subsets of a Banach space X, oneof which is bounded, might be separated strongly by a hyperplane characterizesthe reflexivity of X. In our proposition we are not requiring the reflexivity ofthe underlying space.

(b) From the proof ofthe previous result it is clear that the strong separationof Y and Z does not require the minimization of (P) until optimality. In fact,any feasible solution (cp, u, v) of (P) such that v - u < 0 is sufficient.

Let us discuss briefly the computation of strong separarting hyperplanes inthe case X = ~n. In order to calculate a hyperplane separating Y and Zstrongly it is convenient to use the Chebyshev norm in r, since

(P) inf v - us.t, y'c + u ~ 0, Y E Y

z'c + v ~ 0, z E Z-1 ~ Ci ~ 1, i = 1, ..., nC E Rn,u E R,v E JR,

becomes, then, an LSIP problem. This model was already considered in ChapterI of [4]. In particular, ifY and Z are finite, then (P) is an ordinary LP problem.This model is obviously preferable to the quadratic program proposed in [7].

Going back to problem (P), let us observe that, since (0,0,0) is a feasiblesolution, the optimal value will satisfy v (P) ~ O. On the other hand, if (<p, u, v)is an arbitrary feasible solution of (P), then we have

for all y E Y and for all z E Z. Therefore -d (Y, Z) ~ v (P) ~ 0 so thatv (P) is bounded. In particular, if d (Y, Z) = 0, then (P) is trivially solvable .In fact, we shall prove that (P) is solvable in either case.

Proposition 2.3 The problems (P) and (Pw ) are solvable; i.e., their optimal values are attained at some feasible solution.

Proof. We only give the proof for problem (P). Since v{P) ~ 0, the aggregation of the constraint v - u ~ 0 only provokes the elimination of dominatedfeasible solutions. Then (P) is solvable if and only if the modified problem

(P') inf v - us.t. v - u ~ 0

<P (y) + u ~ 0, Y E Y<P (z) + v ~ 0, z E Z1I<p1I ~ 1<P E X*, u E JR, v E R

is also solvable.First we prove that the feasible set, represented by F, of the program (P) is

weak* closed; i.e., closed in X* x R x R, X* endowed with the weak* topologycr(X*, X). Equivalently, we want to show that if (cp, 'ii, v) belongs to the weak"closure of F (which actually coincides with the strong closure, or the closurewith respect to the norm topology on X* x R x R, as a consequence of theconvexity of F), then (cp,'ii, v) E F. To this end, let us assume (cp, 'ii , v) tt. Fand proceed by looking for a contradiction.

Because (<p, u,v) belongs to the weak" closure of F, and the norm in X*is a weakly" lower semicontinuous function [3, p. 104, Corollary Iii], we canbe sure that 11<p11 ~ 1. Therefore, if (<p, u,v) fails to be feasible for (P), theremust exist e > 0 and, for instance, y E Y such that c. From theelement (y, 1,0) in X x JR x JR we shall define the weak" open set (i.e., openset in X* x JR x JR, X* endowed with the weak" topology a-{X*, X))

u = {( ip , u, v) E X* x JR x JR II(ip (y) + u) - (<p (y) +u)I < c} .

It is obvious that (<p, u,v) E U, with U n F = 0, which contradictsthat (<p, u,v) belongs to the weak* closure of F. Therefore (<p, u,v) E Fand F is weak" closed. Hence, the feasible set of program (P'), F' ={(cp, u , v) E Flv - u ~ O} , is weak" closed too.

Now let us assume that (cp, u , v) E F' . Then, taking arbitrarily Yo E Y andZo E Z, we have

-cp{zo) ~ v ~ u ~ -cp{Yo) ,

so that lui ~ p and Ivl ~ u, where f.t = max {Icp {yo)1 ,Icp {zo)I}. Moreover,

lI{cp,u,v)lI= sup Icp{x)+us+vtlII(x,s,t)119

< sup Icp {x)1 + sup lusl + sup IvtlII(x,s,t)119 II(x,s,t)119 II(x,s,t)119

~ IIcpli + lui + Ivl ~ 1 + 2f.t .

Since every ball in the dual space X* x JR x JR is weak" compact, F' is weak"compact. The objective functional of program (PI) is a weak* continuous linearfunctional, and this yields the solvability of (P') (and also the solvability of(P)). Therefore the objective function of (P) attains its minimum value at acertain point of F. D

Remark 2.2 According to Corollary §13A in [5] the optimal value will beattained at a certain extreme point of the feasible set.

3 STRONG SEPARATION OF COMPACT SETS INSEPARABLE NORMED SPACES

In this section we describe a conceptual cutting plane algorithm to solveproblem (P) when Y and Z are compact sets in X, which will be assumed to

be a separable nonned space. At each iteration of this algorithm, a subproblemof (P) , obtained by replacing the sets of indices, Y and Z, by finite subsets(grids), is solved. All these subproblems are posedinX* x]Rx]R, and all of themhave a finite number of constraints, so that they are semi-infinite programs if Xis infinite dimensional. Moreover, all these subproblems are solvable accordingto Proposition 2.3. If the algorithm does not stop at the current iteration, a pairof new elements of Y and Z are aggregated to the current grids ; i.e., two cutsare performed on the current feasible set. If the algorithm does not terminateafter a finite number of iterations, then it will generate an infinite sequence ofoptimal solutions of the corresponding semi-infinite programming subproblemscontaining a subsequence which converges to an optimal solution of (P).

Algorithm 3.1 (Cutting plane algorithm)

Step 1 Set k = 1, choose Yl E Y and Zl E Z, and set Y1

z, = {zr}.

Step 2 Find an optimal solution (cpk, Uk, Vk) ofthe subproblem

(Pk ) inf v - U

s.t. cP (y) + U ~ 0, Y E YkcP (z) + v ~ 0, z E ZkIIcpli ~ 1cP E X*, U E lR, v E ~

{yr} and

Step 3 Find aglobalmaxtmizerui.c, offk (y) overY,whereik (y) := CPk (y)+Uk ·Finda global minimizer Zk+l ofgk (y) over Z , wheregk (z) := CPk (z)+Vk·

Step 4 If fk (Yk+r) ~ °and gk (Zk+r) ~ 0, then stop: (cpk ' Uk ,Vk) is anoptimal solution for (P) . Otherwise, set Yk+l = Yk U {Yk+r} andZk+l = Zk U {Zk+l} , increment k~ k + 1, and go to Step 2.

In the case of finite termination at Step 4, two cases are possible accordingto Proposition 2.2:

If v» < Uk, then {x E X I CPk (x) = - Uk; Vk } is a topological hyper

plane separating Y from Z strongly.Alternatively; i.c., if v (P) = v(Pk) = Vk - Uk = 0, then there is no

topological hyperplane separating Y and Z strongly .

Proposition 3.1 Any infinite sequence generated by the cutting plane algorithm contains a subsequence which converges to an optimalsolution of(P) .

Proof. Let {('Pk, Uk, Vk)}~l be a sequence generated by the cutting planealgorithm. Since the sequence offeasible sets ofthe subproblems (Pt} , (P2) , ...is non-increasing for the inclusion, we have

VI - UI ~ ... ~ v» - Uk ~ ... ~ V (P) ~ O.

Hence,

so that the sequences {Uk}~==l and {Vk}~l are bounded.The ball B* := {<p E X* I II<pll ~ 1} is weak" compact and, since we are

assuming that X is separable, B* will be weak* sequentially compact (see, forinstance, Corollary 2 of [5, §12FJ). Thus, there exists a subsequence {<PkJ~lthat is weak* convergent to <p* E B*. Without loss of generality we can alsosuppose that the corresponding subsequences {Uki}~l and {Vki}~l convergeto real numbers u* and v*, respectively. We shall prove that (<p*, u*, v*) is anoptimal solution of (P) .

Let us proceed by proving that

<p* (Yk) + u* ~ 0, k = 1,2, ... (3.1)

If (3.1) is not true, then there must exist a natural number N and a positiveescalar E: such that sp" (YN) + u* - E: > O. Let m E {kil~l' m > N, suchthat

Then,

<Pm (YN) +Um > <p* (YN) + u* - E: > 0,

which contradicts <Pm (YN) +Um ~ 0 (observe that YN E Ym)·Similarly, it can be proved that

<p* (Zk) + v* ~ 0, k = 1,2, ... (3.2)

Now, and since the sequences {Yki+d~l and {Zki+d~l are confined in

the compact sets Y and Z, there will exist subsequences {Ykoo+I}OO and'] j=l


{ Zkij +l}:1 converging to points y* E Y and z* E Z, respectively. From

(3.1) and (3.2) we get '1'* (y*) + u* ~ 0 and '1'* (z*) + v* ~ o.Assume that (cp*,u*,v*) is not feasible for problem (P). Then, and since

11'1'* II ~ 1, we shall have, for instance,

'1'* (17) + u* > 0, for a certain fj E Y.

Now, from the definition of Yki .+l, one hasJ

s»: (Yki .+1) + v»: ~ cpki' (fj) + v»; j = 1,2, ... (3.3)J J J J J

Since

andlimj-too cpki ' (y*) = '1'* (y*), weconclude limj.xg, 'Pki ' (Yki '+d = '1'* (y*).J J J

Now, taking limits in (3.3),

'1'* (y*) +u* ~ '1'* (fj) +u* > 0,

which contradicts '1'* (y*) + u* ~ o.Thus, we have already proved that ('1'*, u*, v*) is a feasible solution of (P) ,

and we shall finish the proof by establishing its optimality for (P). In fact,since

we have v* - u* = v (P) ; i.e., ('1'*, u* ,v*) turns out to be optimal for (P). 0

Actually, the subproblems (Pk) are those problems arising in the strongseparation of finite subsets Yk and Zk of a certain separable normed space X .The final section in the paper shows a case where these problems can be solvedeffectively.

4 STRONG SEPARATION OF FINITE SETS IN THEHADAMARD SPACE

Let X = C (T) be the space of all real continuous functions on the compactmetric space T, equipped with the Chebyshev norm IIxlloo = max Ix (i)l. From

tETthe Riesz representation theorem, we know that X* = M (T) , the space ofall the finite signed Borel measures on T . Since T is a metric space, C (T) will

be separable, and every finite signed Borel measure on T is regular (see, forinstance, Example 2.37 in [1]).

In this section we consider that Y and Z are finite sets in X. According toProposition 2.2, there exists a hyperplane separating Y and Z strongly if andonly if v (P) < 0, where

(P) inf v - U

s.t, I f dJ-L + U ::; 0, fEYT

I gdJ-L + v ~ 0, 9 E ZT

IIJ-LII ::; 1J-L E M (T) ,U E JR, v E JR

is semi-infinite in the sense that the number of constraints is finite .In order to define a discretization algorithm for (P) let us denote by JR(T)

the linear space of the generalized finite sequences on T; i.e., the space of thosefunctions A : T --+ JR such that A vanishes everywhere except on the pointsof a finite set called the supporting set of A, that we shall denote by supp A.JR(T) can be interpreted as the subspace of discrete measures (concentrated atthe points of the finite set supp A). If A E JR(T), then A(h) = 'E Ath (t) for

tETevery hEX.

Algorithm 4.1 (Grid discretization algorithm) Let {Tkl~=l be an expansive sequence offmite subsets (grids) ofT such that U~ 1T k is dense in T.(Remember that T is a compact metric space and, so, it is separable.)

Step 1 Set k = l.

Step 2 Find an optimal solution (Ak,Uk,Vk) ofthe problem

(Pk) inf v- U

s.t. 'E At! (t) +u::; 0, fEYtET'E Atg (t) + v ~ 0, 9 E ZtETSUppA C Tk, IIAII ::; 1A E JR(T) , U E JR, v E lR

Step 3 /fv(Pk ) < 0, then stop: v(P) < °and the topologycal hyperplane

{k Uk + vk}H k = hEX I A (h) = - 2 separates strongly Y and Z.

Otherwise. increment k ~ k + 1, and go to Step 2.

SEPARATION BY HYPERPLANES l1JROUGlJ LINeAR SIP 267

Since 11>'11 = :L: I>'tl we can reformulate (Ii) as an LP problem:tET

(Pk) inf v - U

s.t, :L: {,t - 8d f (t) + U :s; 0, fEYtET:L: (rt - 8d 9 (t) + v ~ 0, 9 E ZtET:L: (rt + 8d :s; 1tETsupp I C Tk , supp 8 C Tk

s (T)I' o E lR+ ,U E ~ v E lit

If (rk,8k,Uk,Vk) is an optimal solution for (Pk), then (,k - 8k,Uk,Vk) is

the desired optimal solution for (Pk)'

Next we show that when the grid discretization algorithm does not terminate,we shall conclude that there is no hyperplane separating Y and Z strongly.

Proposition 4.1 lim V{Pk) = v (P) .k~oo

Proof. Since the grids Tk form an expansive sequence of subsets ofT, we have

(4.1)

Let (Ji, 'ii , v) be an optimal solution of (P) .Then, a sequence ofdiscrete measures {JLr }~1 can be found that weak" converges to Ji, and such that supp JLris finite and IIJLrll = 1, r = 1,2, .... .

The last assertion is a consequence of the following statements. From oneside, and as a consequence of the Krein-Milman Theorem ([6, Section 25.5]),the ball B* := {JL E M (T) I IIJL II :s; 1} is the weak" -closcd convex hull of theset of its extreme points, denoted by ext B* . From the other side

ext B* = {a8t 1 a E ~ [o] = 1, t E T} ,

where 8t(h) := h{t) , for every h E C(T) «(13.9) in [5]).Since conv(ext B*) is convex, its weak* closure coincides with the closure

with respect to the norm, Hence a sequence {JLr}~l C conv{extB*) can befound such that limr-too IIJLr - Jill = O. This fact entails that

JLr = 2: >'~8t , r = 1,2, ... ,tET

where >.r := (>.DtET E lR(T) and IIJLrll = :L:tET I>'tl = 1, r = 1,2, .. ..Moreover, for r = 1,2, .. .,

JLr{h) = 2: >'~8t(h) = 2: >'~h(t), for every hE C{T).tET tET

268 SEMI-INFINITE PROGRAMMING. RECf.rJTADVANCES

Since the convergence with respect to the norm is stronger that the weak*convergence, we conclude that {J.Lr }~l weak* converges to Ji.

Given an arbitrary E > 0, the weak* convergence will imply the existence ofro such that

IJi(h) - J.Lro(h) I == JfdJi - L A~Oh(t) < c/4, for every hEY U Z.T tET

(4.2)

(Remember that Y and Z are finite sets.)Moreover, the density of U~lTk , the continuity of the involved functions,

and the finiteness of the sets Y and Z, all together yield the existence of kro anda discrete measure Jik such that supp Jik C Tkr and

~ ~ 0

IJikro (h) - J.Lro(h) I< c/4, for every hEY U Z. (4.3)

Actually, we shall construct the measure Jik in the following way. Let usroconsidersupp J.Lro == {tl, t2, ..., tm } . The uniform continuity ofthe finite familyof functions Y U Z allows us to select, associated with each tk , k = 1,2, .., m,a point tk E U~lTk' such that

Ih(tk) - h(tk) I< c/4, for every hEY U Z, and k = 1,2, .. ., m .

Now, let us consider kro sufficiently large to guarantee that {tl, t2, ...,tm}C Tkro, and define

m

Jikro := LA~Z6tk'k=l

Hence,

m

IJikro (h) - J.Lro(h) I= L A~Z(h(tk) - h(tk))k=lm

::; L IA~Z Ilh(tk) - h(tk) I< c/4, for all hEY U Z.k=l

Next we pick scalars Ukro and Vkro such that (Jikro' Ukro,Vkro) is a feasible

solution of (Pkro)' In fact we can choose Ukro and Vkro in such a way that

for certain functions 10 E Y and go E Z. Then,

v (Pkro ) - v(P) ::; (Vkro - Ukro) - (v - U)

= ("fikro (10) - 71kro (go)) - (v - U)

::; 71kro(10) - 71kro (go) + ]1(90) - 71(10)

::; 171(go) - 71kro(90) 1+ 171(Jo) - 71kro(10) I< I71(go) - /-Lro (90)I+ I/-Lro (90) - 71kro(go) I+ 171(10) - /-Lro(Jo)1 + !/-Lro(Jo) - 71kro(10) I< (c/4) + (c/4) + (c/4) + (c/4) = c,

where we have applied (4.2) and (4.3) to get the last inequality.Since E > 0 is arbitrary, we already proved that v(Pk ) tends to v (P) . 0

Acknowledgments

(i) The authors wish to thank Dr. Mira, from Alicante University, for hisvaluable suggestions.

(ii) The research of M.A. Goberna and M.A. Lopez has been supported bythe Spanish DGES, Grant PB98-0975.

References

[1] J.E Bonnans and A. Shapiro . Perturbation Analysis ojOptimization Problems. Springer-Verlag, 2000.

[2] N.D. Botkin. Randomized algorithms for the separation of point sets andfor solving quadratic programs , Applied Mathematics and Optimization ,32:195-210, 1995.

[3] J.R. Giles. Convex Analysis with Applications in Differentiation ojConvexFunctions, Pitman, 1982.

[4] M.A. Goberna and M.A. Lopez. Linear Semi-Infinite Optimization, Wiley,1998.

[5] R.B. Holmes. Geometric Functional Analysis and its Applications,Springer-Verlag, 1975.

[6] G. Kothe. Topological Vector Spaces I, Springer-Verlag, 1969.

[7] J.B. Rosen. Pattern separation by convex programming, Journal ojMathematical Analysis and Applications, 10:123-134, 1965.

[8] YJ. Zhu. Generalizations of some fundamental theorems on linear inequalities, Acta Mathematica Sinica , 16:25-40, 1966.

Chapter 13

ASEMI-INFINTEOPTIMIZATIONAPPROACHTOOPTIMAL SPLINE TRAJECTORY PLANNING OFMECHANICAL MANIPULATORS

Corrado Guarino Lo Bianco and Aurelio PiazziDipartimento di Ingegneria dell 'Informazione, Universita di Panna

Parco Area delle Scienze IB1A - 43100 Parma -Italy

guar [email protected], [email protected]

Abstract The paper deals with the problem of optimal trajectory planning for rigid links industrial manipulators. According with actual industrial requirements, a techniquefor planning minimum-time spline trajectories under dynamics and kinematicsconstraints is proposed. More precisely, the evaluated trajectories, parametri zedby means ofcubic splines, have to satisfy jo int torques and end-effector Cartes ianvelocities within given bounds . The problem solution is obtained by means of anhybrid genetic/interval algorithm for semi-infinite optimization. This algorithmprovides an estimated global minimizer whose feasibility is guaranteed by the useof a deterministic interval procedure; i.e., a routine based on concepts of intervalanalysis. The proposed approach is tested by planning a 10 via points movementfor a two link manipulator.

1 INTRODUCTION

Motion planning for industrial manipulators can be handled with severalapproaches depending on the control target. An usual technique solves twodifferent aspects of the control problem separately: the trajectory planning andthe design of controllers to track the planned trajectory. This paper is devoted toapproach the former problem by means of an hybrid algorithm for generalizedglobal semi-infinite optimization.

Usually the aim is to obtain a movement that satisfies some optimality requirements. For example, in the case of redundant manipulators, the trajectorypath can be a priori assigned and, by taking advantage of the redundancy, the in-

271

M.A. Goberna and M.A. Lopez (eds.), Semi-Infinite Programming, 271-297.© 2001 Kluwer Academic Publishers.


verse kinematics can be optimized according to some given objective functions[28, 14, 17].

Under the assumption of assigned path, Bobrow et al. [2] and Shin andMcKay [35] proposed to plan minimum time trajectories taking into accountconstraints on the admissible joint torques/forces. The problem was posed inthe same way in both papers but was solved applying different approaches. Pathcurves were defined by a set of functions parametrized by a single variable (thedistance from the origin of the curve to the current position measured along thepath). Torque constraints were converted into constraints for the accelerationalong the path and, then, into constraints for the velocity. In both papers theauthors verified that minimum time motion can be obtained if the accelerationis always kept at its maximum value (positive or negative) compatibly with theconstraint on the torque. This consideration has permitted solving the optimization problem by searching proper switching instants for the acceleration. Suchswitching instants were found by means of different algorithmic approaches inthe two papers. Joint torques related to the optimal solution are discontinuousand, at each instant, at least one torque constraint is active.

Under the same assumptions, Shiller and Lu [34] extended the results of[2] and [35]. They converted the original problem into an equivalent reducedproblem where torque constraints were replaced with constraints on the maximum acceleration and velocity along the assigned path. Shiller and Lu alsoverified that, under particular conditions, the assumption made by other authorsto use always maximum acceleration to achieve minimum time trajectories canintroduce unnecessary chattering into the acceleration itself and can be evenunfeasible. Points and segments where this problem arises were denominatedby Shiller and Lu as singular points and singular arcs, respectively. Theseauthors pointed out that at singular points or arcs, in order to achieve the correct optimal feasible solution, an acceleration smaller than the maximal one isadopted and the velocity is maximized by sliding along the velocity limit curve.

The minimum time trajectory planning problem under specified path andtorque constraint has been also addressed by Pfeiffer and Johanni [25]. Theirmethod is similar to that used in [2, 35] but the solution is found by applying a different algorithm based on dynamic programming [1]. Pfeiffer andJohanni indicated one of the major limitations common to all minimum-timeapproaches: joint torques and forces generated by true minimum-time trajectories are intrinsically discontinuous. They proposed to mitigate this inconvenience by using a mixed performance criterion obtained by combining the usualtravelling time with the squares of joint torques and velocities. As a result, inalmost all positions of the optimal trajectory no one of the torque constraints is

TRAJECTORIES OF MECHANICAL MANIPULATORS VIA SIP 273

active . Obviously travelling time increases but a benefic smoothing effect onjoint torques is achieved.

All the above cited approaches basically exploit the problem structure arriving at highly tailored algorithms. Recently, various authors have suggested tooperate in a more general context by using algorithms for semi-infinite optimization. The point of view changes. Typical robotic planning problems areconverted into equivalent semi-infinite optimization problems and, then, solvedby using proper general purpose algorithms. This is the case, for example, ofHaaren-Retagne [11]. He proposed to convert dynamic (joint torques) or kinematic bounds (joint velocities, accelerations and jerks) of a robotic problem intoequivalent constraints for a semi-infinite problem whose performance index isgiven by the total traveling time. The traveling path is supposed to be assignedand parametrized by B-splines. The resulting generalized semi-infinite problemis converted, according with the scheme proposed by Marin [20], into a standardsemi-infinite problem. In [11], comparisons with the techniques proposed byMarin [20] and Shin and McKay [35] are also exposed.

A different approach to optimal trajectory planning (usually minimum-time)requires the assignment of prespecified via points . In such a way, the geometricpath is not completely defined a priori so that the optimization problem hasmore degrees of freedom. Number and position of via points may also dependon the obstacles to be avoided. Normally, via points are specified in the Cartesian space and then mapped, via inverse kinematics, into a set of joint spacepoints to be interpolated by suitably chosen smooth functions . An effective approach to minimum-time planning is given by the use of parameter optimizationmethods. Starting from the paper of Lin et al. [18], where a polyhedron localsearch technique was used to plan an optimal spline movement under kinematicsconstraints (velocity, acceleration, and jerk of all the joints were constrained),many efforts have been spent in this direction . For example, the same problemwassolved in [27] by devising a global optimization interval algorithm, whilea local gradient-based procedure for semi-infinite optimization was adopted in[4] to solve a more general problem with torque and joint velocity constraints.

In this paper, the problem of minimum-time trajectory generation is coupled with the fulfillment of two important specifications: limits are imposedon both joint torques and end-effector Cartesian velocities. The first (dynamic)constraint is justified by the limits of the torque exerted by the actuators. Thesecond (kinematic) constraint is introduced to avoid damaging the task of theend-effector tool whose Cartesian velocity, in many practical applications, cannot exceed a given operative maximum. By adopting a cubic spline parametrization for the trajectories, equivalent to that proposed in [18], and a full dynamicsmodel for the manipulator, the optimal trajectory planning problem is convertedinto a generalized semi-infinite nonlinear optimization problem whose cost in-

dex is the total travelling time, while the semi-infinite constraints take intoaccount, without conservativeness, the dynamics and kinematics requirements.The optimization problem is then solved by means of the genetic/interval algorithm presented in [6J, [10]. It is a global solver that combines a stochasticoptimization technique (a genetic algorithm) to minimize the cost index, witha deterministic optimization routine (an interval procedure) to handle the semiinfinite constraints. This hybrid algorithm permits obtaining an estimate ofthe global minimizer that is feasible with certainty (torque and Cartesian velocity constraints are always satisfied) and, since it is able to manage directlythe generalized problem, does not require to convert it into a standard semiinfinite problem . The same genetic/interval algorithm has been used to copeand solve several classic control engineering problems. For example, underproper hypotheses, it is possible to design optimal robust controllers for plantswith structured [6] and/or unstructured uncertainties (Hoo problems) [7, 8, 9]by reformulating the control problems into semi-infinite problems.

The proposed hybrid semi-infinite approach has various advantages with respect to those described in [2] and [35]. First of all it is more flexible. Forexample, the same algorithm can be used, by simply adding new constraints,to solve problems with further dynamic and kinematic specifications. Assigning via points instead of a fixed path adds degrees of freedom to the optimization problem in such a way that better performances may be achieved.The parametrization chosen for the splines guarantees the continuity of jointtorques and forces, thus reducing mechanical stresses and possible excitationof unmodelled dynamics .

In Section 2, the manipulator planning problem is posed and worked outto reformulate it as a generalized semi-infinite optimization problem. A related feasibility result is reported in Section 3 (Proposition 3.1). In Section 4,by means of a penalty method, the semi -infinite problem is converted into anequivalent unconstrained problem in order to apply the genetic/interval algorithm. Section 5 describes with details the penalty computation via intervalanalysis: a succinct exposition on inclusion functions is followed by the algorithmic description of an interval procedure, and its deterministic convergenceis established (Proposition 5.1). The potential effectiveness of the proposedapproach is tested in Section 6 by planning a 10 via points optimal trajectoryfor a two link planar manipulator.

Notation

In the following, vectors are indicated by means of lower case bold characters(e.g., q), while matrices are indicated by capital bold characters (e.g., M). Theabsolutevalueofa vectorofn elements be defined as Iql := [Iqll Iq21·· ' lqnIJT,while [q] < Igi means that !qil < Igil, i = 1,2, ... ,n. The euclidean norm


of q is denoted by IIqll. The set of real positive numbers and the set positiveintegers are denoted with J[~+ and N respectively.

2 CUBIC SPLINE TRAJECTORY PLANNING UNDERTORQUE AND VELOCITY CONSTRAINTS

Consider an m link robot and denote by p := [PI P2'" PmV E P cJRm the joint variable vector, belonging to the joint-space work envelope P.Let us assume that s via points have been assigned in the tool-configuration(Cartesian) space. These are mapped, via inverse kinematics problem, into sjoint knots of P. It was shown in [18] that, assuming continuity of velocitiesand accelerations, two free displacement knots must be added in order to exactlyinterpolate the given via points by cubic spline joint trajectories. The resultings + 2 knots can be represented by the data vectors (n := s + 1)

i . _ [i i i JT .- °1q .- ql q2' .. qm , ~ - , , . . . , n,

where ql and qn-l are the free displacement vectors. In particular, note that thecomponent q~ represents the displacement ofthe k-thjoint at the i -th knot. Thevectors of the joint velocities and accelerations at the i-th knot will be denotedby qi := [tif ti~ .. .ti:nV and qi := [ijf q~ ... ij:nJT , respectively. Velocities andaccelerations have to be considered assigned for the first and the last knot; i.e.,vectors qO, qn ,qO, qn are known given data.

Denote by h := [hI h2 . . , hnV E B := [v,+oo)n a vector of intervaltimes, where hi is the time required to move all the joints from the (i - 1)-thvia point to the i-th one, and v is a small positive number which is imposed inorder to avoid possible degeneracies at the implementation stage . The sum ofall the components of h is the total traveling time.

The i-th spline function for the k-th joint indicated by p1(t) with time t E[0, hi)' A convenient parametrization of p1(t), that guarantees the continuityof positions and velocities, is the following :

p1(t) := q1-1+ ti1- 1t

[3 . ' 1 1 . ' 1] 2+ h;(qj. - q1- ) - h/qj. + 2q1- ) t

[2 (i i-I) 1 ( 'i 'i-l)] 3+ - ht qk - qk + hr qk + qk t ,

i = 1, . . . ,n, k = 1,2, . . . ,m,t E [0, hi) . (2.1)

By imposition of the continuity of the acceleration in the resulting splinetrajectory, the following linear system of n + 1 equations can be written for


eachjoint(k=1,2, .. . ,m)

(2.2)

System (2.2) admits a unique solution, depending on h, in the unknowns{ql,q~, .. . ,q~-l,ql,qk-l} [18]. Therefore, the cubic spline p~(t) can beredenoted more explicitly as p~(t; h), and the i-th spline functions for all thejoints can be consequently introduced in vectorial form as:

pi(t;h) := [Pi(t;h)p~(t;h)·.·p~(t;h)]T,

t E [0, hil, i = 1,2, .. . , n . (2.3)

By neglecting friction, joint torques and forces can be evaluated by meansof the manipulator dynamic equation [3, p. 206]:

T = M(p) P + B(p)[pp] + D(p)[p2] + g(p), (2.4)

where T E jRm is the vector of the joint torques and forces, M(p) E jRffixmis the inertia matrix, B(p) E jRmxm(m-l)/2 is a matrix of Coriolis terms,D (p) E jRmxm is a matrix of centrifugal coefficients, g(p) is the vector of thegravity terms, and [pp] E jRm(m-l)/2 and [p2] E jRm are vectors composedwith velocities according to the definitions

and

Taking into account (2.3), the torque vector becomes a time function parametrizedby h, so that we define, congruently with (2.4) ,

Ti(t; h) := M(pi(t; h)) pi(t;h)

+B(pi(tj h))[pi(t; h)pi(t; h)] + D(pi(t; h))[pi(t; h)2]

+g(pi(t; h)), i = 1, ... , n, t E [0, hi] . (2.5)


An explicit relation between the joint variables and the Cartesian velocities ofthe end-effector can be expressed by means of the "geometric" jacobian matrixJ (p) [33] as follows:

[:] =J(p)p, (2.6)

where v := [vx vy vzjT and w := [wx wy wzjT are, respectively, the linearand the rotational velocities vectors of the tool frame affixed to the manipulatorend-effector. The joint trajectories are time functions parametrized by h, sothat (2.6) can be rewritten as

[yi.(t;h) ] ._ J (pi (t;h) )pi (t; h) ,wZ(t;h)

t E [0, hi], i = 1,2, ... ,n . (2.7)

Hence, a minimum-time movement under torque and tool velocity constraintscan be planned by solving the following semi-infinite optimization problem

(2.8)

subject to (i = 1,2, .. . ,n)

(2.9)

(2.10)

(2.11)

where T := [it f2 .. . fm]T E jR+ m is the vector of the imposed torque limits,while fJ E jR+ and w E jR+ are the linear and the angular velocity limits,respectively. Our aim is to find an estimated global minimizer of problem(2.8)-(2.11) hereinafter denoted by h" E B.

3 A FEASIBILITY RESULT

Problem (2.8)-(2.11) will be solved by means of an hybrid algorithm forglobal optimization (briefly described in the next sections). It is important toverify if a feasible solution exits. For this reason, a partial feasibility resultrelated to problem (2.8)-(2.11) is given in the following.

Proposition 3.1 Let the initial and final velocities and accelerations bezeros (<i0 = <in = 0, qO = qn = 0). Then problem (2.8)-(2.11) admits asolution if

Tk > maxI9k(p)l , k = 1,2, ... ,m,pEP

(3.1)

where 9k (p) denotes the k-tn component of the gravity vector g(p).

Proof. Problem (2.8)-(2 .11) admits a solution if there exists a feasible point hin the set B; i.e., an admissible point h that satisfies the semi -infinite problemconstraints (2.9)-(2.11).

First, we consider constraint (2.9). The first and the last equality of (2.2)permit expressing q1 and q~-l as functions of the sole unknown ti1 and q~-l

respectively, according to the following expressions

(3.2)

(3.3)

By replacing (3.2) and (3.3) into the remaining equations of system (2.2), weobtain a reduced linear system of order n - 1 that can be compactly written as

A(h)x = b(h) , (3.4)

o

2>.2 08>.3 2>.3 02>.3 8>.3 2>.3

where x := [£i1 £i~ . .. q~-lV is the reduced vector of the unknowns, whileA(h) E lR(n-l) x(n-l) is a proper tridiagonal matrix that depends only onvector h.

Choose a point h E B defined as h := ,X = [>' >. ... >.]T where>. is apositive real parameter. Next we will show that ,X is feasible for a sufficientlylarge >.. Consequently, with the choice of h , and taking into account that initialvelocities and accelerations have been set equal to zero , matrices A(h) andb(h) become

8>.24>.3

o0 0

A('x) = 0 2>.3 8>.3 2>.3 0

0 00 2>.3 8>.3 2>.3 0

0 2>.3 8>.3 4>.3

0 2>.2 8>.2

and

By scrutiny of system (3.4) for the chosen value of h we have

·i (') Pki )..3n- 6 Pki )..3n - 6 Pki 1 . 1 1qk A = det[A(,x)] = C

n)..3n - 5 = en 'X' ~ = ,..,n - , k = 1, . .. , m,

(3.5)

where Cn E N depends only on nand Pki E R Hence, it follows that

lim Iql(,x)I=0, i=1, ... ,n-1, k=1,2, ... ,m. (3.6)A-tOO

From (2.1), joint velocities and accelerations are given by (i = 1, ... , n, k =1, 2, . .. , m, and t E [0, )..D:

p1(t;,x) =

··i (t· ,x)Pk , [6 ( i i-i) 2(.i 2'i-i)]

)..2 qk - qk - 'X qk + qk

[12(i i-i) 6('i 'i-i)]+ - )..3 qk - qk + )..2 qk + qk t .

(3.7)

(3.8)

By virtue of (3.6) and above expressions (3.7) and (3.8) for any given e E 1R+there exists ).. = )..(e) E 1R+ such that (i = 1, ... , n):

Ilpi (t; ,x) II < e \It E [0,)..],Ilpi (t; ,x) II < e \It E [0,)..] .

(3.9)

(3.10)

Taking into account the boundedness, over P, of matrices M(p), B(p), andD(p) we have (i = 1, . . . , nand t E [0, >.D

IIM(pi(t; A)) pi(t; A) + B(pi(t;A))[pi(t; A)pi(t; A)]

+D(pi(t; A))[pi(t; A)2] II ::; M Ilpi(t; A) II+B II[pi(t;A)pi(t;A)]11 +D lI[pi(t;A)2]11, (3.11)

where M, B, and D are the appropriate real positive bounds for the matricesinvolved. From (3.9) and (3.10) we obtain

M Ilpi(t;A)II+B lI[pi(t;A)pi(t;A)]II+D II[pi(t;A)2]1I

< Me + B m(m - 1) e2 + D m e2

2

= (M +B m(m2-1) e +Dm c) e =: ,(c) c. (3 .12)

From (3.11) and (3.12) this inequality follows

IIM(pi(t; A)) pi(t; A) + B(pi(t; A))[pi(t; A)pi(t;A)]

+D(pi(t; A))[pi(t; A)2111 < ,(c) c. (3.13)

By considering definition (2.4) we deduce (i = 1, . .. , nand t E [0, >.D

We can write in turn

The hypothesis (3.1) permits choosing e such that

,(e)e < min (Tk - max 19k(P)I)k=l, ... ,m pEP

Therefore

(3.15)

(3.16)

,(e)e+maxI9k(p)1 <Tk k= 1, . .. ,m, (3.17)pEP

and, from (3.15), we finally conclude that A is a feasible point for constraint(2.9) since

Iri(t;A)1 < f Vt E [0,>.], i = 1, . .. ,n. (3.18)

Analogously, it is possible to prove that A is a feasible point also for the othertwo constraints. We assume the premises made for constraint (2.9) to be valid,so that inequality (3.9) holds for any given E > O.

The jacobian matrix J (p) can be divided into two parts to separate the linear and the rotational components of the velocity vector, so that (2.7) can berewritten as (i = 1,2, ... , nand t E [0, >.])

(3.19)

(3.21)

(3.22)

For any real robotic manipulator all the elements of the jacobian matrix arebounded for any pEP. By virtue of this property, appropriate norm boundsJ' and J" of matrices J' (p) and .r" (p) can be found for any PEP, so that wecan correctly write (i = 1,2, . " , nand t E [0, >.])

Ilvi(t;.\) II - IIi (pi(t; .\))pi(t; .\)11 ::; J' II pi (t ; .\)11 '

Ilwi(t;.\)11 - 1Ii'(pi(t;.\))pi(t;.\)II::; J" Ilpi(t;.\)II. (3.20)

By applying (3.9), equations (3.19) become (i = 1,2, ... , nand t E [0, >.])

II v i (t;.\)11 < J' s,

Ilwi(t; .\)11 < J" e,

and choosing e such that the following two inequalities hold simultaneously

J'e < fl,

J"e < w,it is possible to conclude that .\ is feasible because (i = 1,2, ... , nand t E[0, >.])

II v i (t ; .\)11 < fl,

IIwi (t ; .\)11 < w. (3.23)

If e satisfies simultaneously inequalities (3.16) and (3.22), the corresponding .\is a feasible point for the constrained problem (2.8)-(2.11). 0

4 PROBLEM SOLUTION USING AN HYBRIDALGORITHM

It is worth noting that (2.8)-(2.11) is formally a generalized semi-infiniteproblem. In fact. it can be recasted as

minf(h)hEB

(4.1)

subject to

9j(r;h) ~ 0 'Vr E lRj(h),j = 1,2, ... ,u.

where Yj (r; h) are continuously differentiable constraint functions, and lRj (h)denotes, in general, a multi-dimensional compact interval that depends on thesearch parameter h . For the problem at hand, the number of functions Yj (r; h)is directly correlated to the number of joints and time intervals of the problem.Taking into account that each torque constraint (2.9) must rewritten as (i =1,2, ... ,n)

in order to get differentiable functions, the total number of constraints is u =n(2m + 2).

The notation adopted in (4.1) is useful to show that the proposed hybrid algorithm can solve problems more general than (2.8)-(2.11); i.e., problems withr being a real vector and f(h) being a generic nonlinear function that does noteven need to bea continuous function. In the mathematical literature nonlinearsemi -infinite optimization has been treated with a variety of approaches, for example generalized gradient procedures, recursive nonlinear programming, etc.Good sources on the subject with extensive bibliography are the book of Polak[29], and the surveys of Hettich and Kortanek [13] and of Reemtsen and Gomer[31]. Specific algorithms that can solve (4.1) for the case of mono-dimensionalconstraints (i.e., lRj (h) is a compact real interval) that is actually our case for theminimum-time problem (2.8)-{2.1l) were presented by Jennings and Teo [15]and Teo et al. [37]. Both algorithms, using a constraint transcription methodbased on an integral representation, generate a converging sequence of finiteoptimization problems. These are equality constraints problems for [15] andpenalty-based unconstrained problems in [37] that intentionally rely on standardnonlinear programming; i.e., deterministic local optimization. In this sectionwe sketch a numerical approach to problem (4.1) based on the combined useof stochastic and deterministic global optimization (cf. the last paragraphs ofSection 5 for a comparison with [37]).

By defining o"j(h) := maxrEIRj(h) {Yj(r;h)}, problem (4.1) can be converted into an unconstrained problem by using the penalty method

min {f(h) + i: <'P(O"j(h))} ,hEB j=l

(4.2)

where the penalty function <1>(0") is defined as follows

{

0, if a E (-00,0],<1>(a):= M-M(T-a)2jT2, if aE(O,T],

M, if a E (T, +00) .(4.3)

When T -+ 0+ and M -+ +00 problem (4.2) is strictly equivalent to (4.1).For finite values of M > 0 and T > 0, monotonically better precisions areobtained for larger values of M and smaller values ofT.

The equivalent optimization problem (4.2) is solved by means of an hybridgenetic/interval algorithm originally proposed in [6, 10]. An interval procedureis used to evaluate the penalty terms of (4.2), while a genetic algorithm is usedto find the estimated global minimizer h".

The interval procedure is a special branch-and-bound algorithm based on theprinciples of interval analysis (an extension of the standard real analysis overthe arithmetics of real intervals [24]). Broadly speaking , interval procedures(algorithms) are deterministic routines that can be applied to linear or nonlinear optimization problems, assuring global convergence within an arbitrarilyprespecified precision [12, 30].

With the aim of improving the computational efficiency, our interval procedure does not determine the exact value of aj(h), but is tailored to directlycompute <1>(aj(h)) with the help of special accelerating devices to speed up theconvergence. This developed procedure can be considered as a generalizationof the interval positivity test presented in [26].

According to (4.2}, penalty terms evaluated by the interval algorithm, areadded to f(h) to obtain the overall objective function. The resulting unconstrained optimization problem is solved with a genetic algorithm; i.e., a stochastic technique that can deal with a variety of optimization tasks [5, 22]. Thischoice is justified by the need of a relatively fast procedure, again with theaim of limiting the computational time. A partially elitistic algorithm has beenimplemented so that, at each generation, the final population is composed by individuals randomly drawn from the previous generation mixed with individualsof the so called offspring population. The genetic algorithm uses a two-phasestechnique to approach the feasibility region [32]: until the feasible region isreached the objective function is exclusively given by the penalty terms (i.e.,the cost index is ignored), later the whole objective function is considered. Toincrease the convergence rate, procedures for the local improvement of the bestindividual of the current population have been introduced. The elaborated ge-

netic algorithm is a variant of the one presented in [21] and is described withdetails in [10].

The estimated global minimizer for the optimization problem (4.2) is givenby the individual that strictly satisfy all the semi-infinite constraints and hasthe best fitness over all the iterated generations. For the problem at hand, thismeans that the estimated global minimizer satisfies, with certainty, all the limitsimposed on the joint torques and on the tool Cartesian velocities.

5 PENALTY COMPUTATION VIA INTERVALANALYSIS

Penalty terms for the equivalent unconstrained problem are computed withan interval procedure. The interval algorithm evaluate directly «p(<T(h)) (forsimplicity, in this section the subscript j is dropped). It is an improvement ofthe procedure proposed in [10]: the program has been modified to acceleratethe convergence toward the solution. In the sequel, the improved algorithm isexposed and its convergence properties are investigated. A generic semi-infiniteconstraint function 9(r; h) is considered.

Define the set of real intervals as I := {[a, b] : a, b E lR, a ~ b}. In the following R, D ~ R"Il are used to denote finite multidimensional real intervalsor "boxes": for example, D := [41' dd x [fb,d2 ] x .,. x Wm, dm]. A midpoint of a box V is a vector whose i-th component is given by (Qi + di)/2:it individuates the "center" of a box and will be indicated as mid(V) E D. Itis usual to indicate the "width" of D as the widest edge of the box: w(V) :=

maxi=I" ",m {di - Qi}'

The vector h acts as a fixed parameter each time the interval procedure isinvoked by the genetic algorithm. For this reason the semi-infinite constraintfunction can be denoted in a compact way as 9h(r) == 9(r; h) and the searchbox is indicated by lRh == lR(h). The image set of lRh under function 9h(r) isdenoted by

the continuity of 9h(r) implies that 9h(lRh) E I. The global maximum valueof 9h over lRh will be denoted by 9h := cr{h) = maxrElRb {9h(r)}: 9h isthe upper endpoint of 9h(lRh ). Moreover lRh := {r" E lRh : 9h(r") = 9h}denotes the set of global maximizers which may have cardinality greater thanone. Lower and upper bounds of 9h are denoted by lb and Ub respectively.K pr e E N denotes the precision factor to be used by the termination test of theinterval procedure.


5.1 INCLUSION FUNCTIONS

An inclusion junction with respect to 9h is an interval-valued functionG h : {V : V ~ lRh} ~ I satisfying:

9h(V) ~ Gh(V) \/V ~ lRh .

Once an inclusion function is known, an upper bound of the global maximumof 9h(r) over V ~ lRh, denoted by ub(9h, V) , can be easily determined as theupper endpoint of Gh(V) , Interval analysis is a straightforward tool to get avariety of inclusion functions. The simplest of these is the so-called naturalinterval extension. Roughly speaking, it is obtained by evaluating a given formof 9h with the substitution of the usual arithmetic with the interval arithmetic.This is summarized as follows:

[a, b] + [e, d][a, b] - [e, d]

[a, b] . [e, d][a, b]/[e, d]

[a+ e, b + d] ;[a - d, b - e];

[min{ae, ad,be, bd}, max{ae, ad, be,bd}]; and

[a, b] . [lid, lie] if 0 tf- [e, d].

As an example consider 9h(r) = r? +5rlr2 +r2 - r1 and V = [-1,1] x [2,4].Hence , by using the natural interval extension we have Gh(V) = [-1,1]2 +5·[-1,1].[2,4]+[2,4]-[-1,1]3 = [0,1]+5'[-4 ,4]+[2,4]-[-1,1] = [-19,26].Other noteworthy inclusion functions are the "meanvalue forms" and "Taylorforms" ; both forms belong to the class of "centered forms" introduced in [23].For the purpose of the interval procedure to follow, the optimal meanvalue formof Baumann has been chosen as inclusion function for its sharp bounds andmoderate computational burden. An introduction to inclusion functions, withdetails on the Baumann form, can be found in [30].

5.2 THE INTERVAL PROCEDURE

By making use of operators borrowed from interval analysis, an algorithmfor the evaluation of the penalty function «P (a (h)) has been developed. It isan improved version of the algorithm appeared in [10]. More precisely, a moreversatile management of the List is obtained, and the algorithm is stronglytailored to the direct evaluation of «p(a(h)).

Step 1 Locally maximize the function 9h(r), with starting point mid(lRh) , toobtain r and set lb := 9h (r). [r E lRtt and 9h (r) ~ 9h (mid(lRh))]

Step 2 Set Ub = +00.

Step 3 If ub(9h, lRh )) > 0 initialize the list List inserting the pair (lRh, ub(9h, lRh)) .


Step 4 If lb 2': T then set ell (O"(h)) := M and terminate.

Step 5 If List is empty then set ell(O"(h)) := 0 and terminate.

Step 6 Set Ub := the second element of the first pair of List.

Step 7 If (Ub -lb) ~ T / K pre then set ell(O"(h)) := ell ((lb + ub)/2) and terminate.

Step 8 The first pairs ofList is popped out and its box, by halving on the largestedge , is split into boxes Vi and V 2•

Step 9 For i := 1,2 do

If9h(mid(Vd) > lbthen locally maximize thefunction pj, (r) withstarting point mid(Vi) to obtain r, and set h := 9h (F). rr E lRand 9h (r) 2': 9h (mid(Vi ) )] .

Step 10 For i := 1,2 do

If ub(9h, V i) 2': lb and ub(9h, Vi) > 0 then insert pair (Vi ,ub(9h, Vi)) in List, in such a way that the second elements ofthe pairs be placed in decreasing order.

Step 11 Discard from List, without perturbing the decreasing order, any pairsuch that its second element is smaller than lb.

Step 12 Go to Step 4.

Step 13 End .

The exposed procedure is based on the branch-and-bound principle wherethe bounding is done via inclusion functions, and the branching is done bysplitting the box which has the largest upper bound. In such a way it emerges,at the core of the procedure, an interval algorithm which can compute ell(O"(h))with arbitrary precision.

At Step s (1) and (9) a local maximization is simply performed with the steepest ascent method [19, see p. 214]. This accelerates the procedure convergencebecause it helps discarding portions of lRh not containing global maximizers(cf. Step 11).

5.3 CONVERGENCE ANALYSIS

The considered assumption of 9h(r) being continuously differentiable onthe compact~ implies that 9h (r) is Lipshitzian as well as continuous on ~ .

The following properties are essential to establish the main result of this section(Proposition 5.1).

Property 5.1 For any meanvalue form the following limit holds uniformlyfor V ~ lRh :

Property 5.2 The following limit holds uniformly for V ~ ~:

lim 9h(mid(V)) = max {9h(r)} .w(V)~O rEV

(5.1)

(5.2)

Property 5.1 is a well known result in the interval analysis literature, cf. forexample [30], and Property 5.2 is an obvious consequence of the continuity offunction 9h (r) over ~. In the following, considering the interval procedure,

we denote by lbi) and ubi) the values of variables lb and Ub, at Step 4, of the i-thiteration.

Lemma 5.1 At any stage ofiterations , the above interval procedure guar-

I * [lei) (i)]antees t tat 9h E b ' ub .

Proof. By virtue of Steps I and 9, lbi) is the maximum of all the 9h function

values computed till the i -th iteration. Obviously, the global maximum of 9h

must be greater than or equal to the current lb' so that inequality 9h ~ lbi) isensured.

Suppose that, at the i-th iteration, the list List composed as

L · - {(V(i) b( Dei»)) (V(i) ( Dei»)) (V(i) ( V(i»))}2st - l' u 9h, 1 , 2' ub 9h, 2 , ... , hi' ub 9h , hj

Steps 3 and 10 guarantee that, at any iteration i, the List is composed by boxes

such that ub(9h, V)i») > 0, j = 1,2, .. . , hi where hi indicates the current list

length . By considering also Steps 2 and 6 we conclude that uii ) > O. Twosituations could arise depending on the sign of the global maximum of 9h '

Let suppose that 9h > O. In this case, all the global maximizers are containedinto boxes ofList at any stage of iterations. In fact, there are only two conditions

that permit discarding a generic box V: ub(9h, V) ::; 0 or ub(9h, V) ::; lbi) (seeSteps 3, 10 and 11). In both instances we deduce ub(9h, V) < 9h' so that Vdoes not contain global max imizers. All boxes are inserted into the List suchthat their related upper bounds are placed with decreasing order (cf. Steps 10and 11). Hence

(') (') (') (')ub(9h,VIZ

) ~ ub(9h , V2z

) ~ ub(9h, Vaz

) ~ ••• ~ ub(9h, V hzi

) , (5.3)

and ubi) = ub(9h, vii)) owing to Step 6. Evidently, the upper bound ordering

implies that ubi) ~ Yh with certainty.

Now letsuppose that ef ~ O. In this case, being always verified that ubi) > 0,

it is straightforward to conclude that ubi) > 9h' 0

Proposition 5.1 For any T E JR+ and K pre E No the above intervalprocedure converges with certainty and computes (o-(h)) with arbitrarilygood precision.

Proof. The proof of Proposition 5.1 is divided into two parts: first we provethat, if the algorithm converges, it returns the correct value of (o-(h)) witharbitrarily good precision; secondly, it is shown that convergence is guaranteed.

First part:The interval procedure can stop owing to one of the three termination criteria

(Steps 4, 5 and 7).If the algorithm stops because of Step 4, Lemma 5.1 permits asserting that

9h ~ T so that, according with (4.3), the exact value of M for (o-(h)) isprovided.

If the algorithm stops because of Step 5, the given output is correct only ifYh ~ O. This can indeed be proved. If the List is empty, all the boxes havebeen eliminated. A generic box V can be discarded on account of the following

conditions: (1) Ub(Yh, V) ~ 0 , or (2) Ub(Yhl V) < lbi) . Let us indicate byII and I 2 the union of all the boxes discarded because of inequality (1) andinequality (2), respectively. Since the List is empty we have JRh = II UI2 • Byvirtue of Lemma 5.1 and condition (2), we can immediately conclude that setI 2 cannot include any global maximizer, so that they are all within II . On thebasis of condition (1), the global maximum of Yh over II is not positive so that

9h ~ O.Finally, if the algorithm stops because of Step 7 and taking into account

Lemma 5.1, the global maximum 9h is estimated as the midpoint between

l~i) and u~i) . In this case, the final distance between l~i) and ubi) can be setarbitrarily small because T and K pr e can be freely selected. Remembering

that 9h E [l~i), ubi)], we conclude that Yh and, consequently, (o-(h) ) can beevaluated with arbitrarily good precision.

Second part:Now we demonstrate that if the interval procedure does not halt at Step 4 or

5 then, necessarily, it halts at Step 7. The branching mechanism issued at Step8, in absence of the exit tests 4,5 and 7, permits writing

Iim w(Vii ) ) = 0 .1-+00

(5.4)

Property 3.1 implies that, for any given e > 0 there exists 8u > 0, such thatfor any V~i) satisfying w(V~i») < 8u it follows

Ub(gh' vii») - max {gh(r)} < erEV~')

or, equivalently, by virtue of Step 6,

(5.5)

On the other hand , it is possible to deduce from Property 5.1 that, for any

given e > 0, there exists 8m > 0 such that for any V~i) satisfying w(V~i») < 8mit follows

(5.6)

By considering that function gh (r) is evaluated at the midpoint of every box

inserted in the List (cf. Steps 1 and 9, it is clearly verified gh (mid(V~i»)) ~

l~i). Then from inequality (5.6) descends

(5.7)

Clearly limit (5.4) implies that w(Vii») can become arbitrarily small forsufficiently large values of i. For this reason, it is possible to find an i* E N

such that for any i ~ i* : w(V~i») < min {8u , 8m } . Finally, from (5.5) and(5.7), it is obtained

(i) _ l(i) < 2Ub b €, Vi ~ r,

and, imposing E = T/(2Kpr e ) , the above inequality becomes

(i) (i) \.J~ ~ ~ * .U b -lb <T/Kpre , v•• (5.8)

The interval procedure halts at Step (vii) not exceeding iteration i*, by virtueof statement (5.8). 0

It is worth comparing the proposed genetic/interval algorithm with the penaltybased constraint transcription algorithm of Teo, Rehbock, -and Jennings [37];


see more details on the related enforced smoothing constraint transcription inthe book of Teo et at. [36] that also reports a general treatment of semi-infinitecontrol problems.

Both approaches use a penalty method to transform the original semi-infiniteconstrained problems into finite unconstrained problems (cf. the second paragraph at the beginning of the Section 4). The unconstrained objective functionfor the Teo-Rehbock-Jennings (TRJ) algorithm is smooth (i.e., the associatedgradient is continuous), by virtue of the devised smoothing technique, whereasthe unconstrained function (4.2) is not smooth. On the other hand, the TRJalgorithm needs smoothness in the unconstrained problem because it is thensolved with a gradient method, whereas the proposed algorithm using a geneticalgorithm does not necessitate to compute the gradient.

The genetic/interval algorithm can be intrinsically used for generalized problems where lRj (h) is a multi-dimesional box that can explicitly depend on h,while on the contrary the TRJ algorithm only deals with mono-dimensionalconstraints. Moreover, the TRJ algorithm nominally requires a real intervalIRj (h) not depending on h, so that some artifices should be applied in order toextend the approach to generalized problems.

A crucial aspect of the comparison regards the feasibility of the solutionsprovided by the algorithms. The genetic/interval algorithm guarantees thefeasibility of the found minimizer because it relies on a deterministic globalmethod (the interval procedure) to compute the penalty terms . Applying theTRJ algorithm, feasibility can not be ensured with certainty, because the penaltycomputation uses a quadrature discretization in evaluating the related integralterm. As a consequence, in case of harsh constraints containing positive spikesfeasibility could be erroneously claimed.

From a computational viewpoint the TRJ algorithm provides definitely bettercomputation times than the genetic/interval algorithm. This is basically due tothe following reasons: (i) the efficiency of the gradient method ofTRJ algorithmto reach a local solution compared to the relatively heaviness of the geneticalgorithm to reach an estimated global solution; (ii) the fast computation of thepenalty terms using the quadrature points in the TRJ algorithm compared to theslower branch-and-bound interval procedure of the proposed algorithm.

6 AN EXAMPLE

The example proposed concerns a two-link (m = 2) mechanical arm withrevolute joints. The problem is the minimum-time planning under torque andvelocities constraints of a trajectory whose Cartesian path is, for example,schematically shown in Fig. 13.1.


T,~

I~~ .'

\':/;\\\.m,

Figure 13.1 Schematic representation of the two-link planar arm with the planned Cartesianpath.

xO =1.00 yO =-0.500 «=0.0000 qg = -1.5708

x 2 =1.00 y2 =-0.375 q~ =0.1253 q~ = -1.6804

x 3 =1.00 y3 =-0.250 q~ =0.2517 q~ =-1.7594

x 4 =1.00 y4 =-0.125 qt =0.3789 q~ =-1.8074

x5 =1.00 y5 = 0.000 qf =0.5054 q~ = -1.8235

x6 =1.05 y6 = 0.100 qr =0.5837 q~ = -1.7087

x7 =1.15 y7 = 0.200 q{ =0.6119 q~ =-1.4581

x 8 =1.30 y8 = 0.100 q~ =0.4263 q~ = -1.1040

x 9 =1.30 y9 = 0.050 q~ =0.3903 q~ = -1.1124

x ll=1.30 yll= 0.000 q~1=0.3526 q~l= -1.1152

Table 13.1 End-effector via points expressed in meters and equivalent jo int via points expressedin radiants.

The distal end of the second link is required to exactly cross some given viapoints (8 = 10, so that n = 11) on the Cartesian path. These via points areexpressed as Cartesian coordinates of the arm base frame (first two columns inTable 13.1).

By solving the inverse kinematics problem the Cartesian via points are converted into joint via points, and reported in the last two columns of Table13.1.


Note that the second and the penultimate knot points have not been imposed,being associated to the two free joint displacements. The joint variable vectoris p := [PI P2]T E P := [0,11"/2] x [-11", 0] . Closed form dynamic equationswere derived in [3, p. 204] under the hypothesis of masses concentrated at thedistal end of each link

1"1 := m21~(jh + ih) + m211 12 cos(P2)(2ih +P2) +

(ml + m2)lrpl - m21112 Sin(p2)p~

2m2h12 Sin(p2)PIP2 + m212g cos(PI +P2) +(ml + m2)11g cos(pt} ,

1"2 .- m21112COS(P2)Pl +m21112Sin(p2)pr+

m2hgcoS(pi +P2) + m21~(P1 + ii2), (6.1)

where ml and m2 are the link masses, 9 is the gravity acceleration, and 11 and12 are the link lengths. The arm parameters are listed in Table 13.2.

h = 1.0 m ml = 15.0 kg

12 = 0.5 m m2 = 7.0 kg

Table 13.2 Parameters of the two-link arm.

Analogously, closed form equations were derived for the linear and angularvelocities of the end-effector [3, p. 169]

IIvll := VZipr + 1i(PI+ Ii2)2 + 21112Pl(PI +Ii2) COS(P2) , (6.2)

W z := PI + P2 . (6.3)

Here, the vector of the interval times is defined as h := [hI h2 . • • h l1V EB := [0.02,1.0]11 (it has been fixed 1/ = 0.02 sec). Moreover, we consider thearm at rest in the initial and final positions (i.e., 40= 411 = 0; qO = ql1 = 0) .By virtue of Proposition 3.1, problem (2.8)-(2.11) admits a feasible solutionif f > [maxpEP{gt{p)}, maxpEP{g2(p)}V = [249.9, 34.3]T Nm. Withthe aim of planning the optimal trajectory with moderate actuator exertion,the torque limit vector is set to f = [260 50]T Nm, which is only slightlylarger than its minimum admissible value. The minimizer evaluated by thegenetic/interval algorithm is given in Table 13.3.The estimated global minimum travelling time is L:;;1 hi = 2.05009 sec.Fig. 13.2 shows that, at the optimal solution, the joint 1 torque constraint isactive into a wide time segment. Moreover, also the constraint on the maximumlinear velocity modulus is active (see Fig. 13.3). Finally, the planned optimal


hi =0.02000 s

h2=0.36429 s

hs=0.18419 s

h4=0.18386 s

hs=0.18423 s

h6=0.16735 s

h'7 =0.22310 s

hs=0.36539 s

hg=0.09945 s

hio= 0.23818 s

hil=0.02005 s

Table 13.3 Estimated global minimizer h" of the planned optimal trajectory.

~ml ~m2

300 r--~-~--~--~~-~~--,--., 75

250

200

150

100

62.5

50

37.5,' ..... --.

-, 25

12.5

I. 2 1.4 1.6 1.8 2

..'/' "- - - _.. -- - ~ ~ --..

-- Joint 1. - - - - -. ---Joint 2

0.2 0.4 0.6 0.8

50

Figure 13.2 Torque profiles for the two joints.

trajectory in the Cartesian plane is shown in Fig. 13.4, where crosses indicatesthe assigned via points.

The genetic/interval algorithm has been coded in C++ and uses, at the lowerlevel, the efficient PROFIL library [16].

7 CONCLUSIONS

In this paper a new method has been presented for the minimum-time trajectory planning of mechanical manipulators. The method, using a joint space


1.6 ,.-~---~~---~~---~---,--,

\.4

1.2

1.2 1.4 1.6 1.8 20.2 0.4 0.6 0.8

-- Ca rtesi an velocity (m/sec)- - - Angular velocity (roo/sec)

-0.2

-0.4L.-__~_~~_~_~~_~-=~~

o

0.8

0.6

0.4

0.2

o

Figure 13.3 Angular and Cartesian velocities of the end-effector.

0.3

0.2

0. \

0

-0. \ :¥-0.2 j-0.3 r-0.4 r-0.5 x

-0.60.95

,!

1.05 J.J 1.\5 1.2 1.25 1.3 1.35

Figure 13.4 Optimal trajectory in the Cartesian space. Crosses indicate the assigned via points

scheme with cubic splines, takes into account Cartesian velocity constraintsand torque constraints by inclusion of a full manipulator dynamic model. Theresulting problem is shown to be a semi-infinite nonlinear optimization problem for which an estimate of the global solution can be obtained by means of agenetic/interval algorithm. This estimate solution is guaranteed to be feasibledue to the deterministic interval procedure used by the hybrid algorithm.

First computational results highlight the effectiveness of the method andsuggest to apply and extend it in broader robotic planning contexts.

Ackowledgments

This work waspartially supported by MURST and ASI (Italian Space Agency) .


References

[I] R.E . Bellman and S.E. Dreyfus. Applied Dynamic Programming, PricetonUniversity Press, 1962.

[2] J.E. Bobrow, S. Dubowsky, and J.S . Gibson. Time-optimal control ofrobotic manipulators along specified paths, International Journal ofRobotics Research, 4:554-561, 1985.

[3] J,J . Craig. Introduction to Robotics: Mechanics and Control, AddisonWesley, 1989.

[4] A. De Luca, L. Lanari, and G. Oriolo. A sensitivity approach to optimalspline robot trajectories, Automatica, 27:535-539, 1991.

[5] D.E. Goldberg. Genetic Algorithms in Search, Optimization, and MachineLearning, Addison-Wesley, 1989.

[6] C. Guarino Lo Bianco and A. Piazzi. A hybrid genetic/interval algorithmfor semi-infinite optimization. In Proceedings ofthe 35th Conference onDecision and Control, pages 2136-2138, Kobe, Japan, 1996.

[7] C. Guarino Lo Bianco and A. Piazzi. Mixed H 2 / u.; fixed-structure control via semi-infinite optimization. In L. Boullart, editor, In Proceedingsof the 7th IFAC Symposium on Computer Aided Control Systems Design,pages 329-334, Pergamon, 1997.

[8] C. Guarino Lo Bianco and A. Piazzi. A worst-case approach to SISOmixed H 2/Hoo control. In Proceedings of the 1998 IEEE InternationalConference on Control Applications, pages 684-688, Trieste , Italy, 1998.

[9] C. Guarino Lo Bianco and A. Piazzi. A global optimization approach toscalar H 2/Hoo control, European Journal ofControl, 6:358-367, 2000.

[10] C. Guarino Lo Bianco and A. Piazzi. A hybrid algorithm for infinitely constrained optimization, International Journal ofSystems Science, 32:91

102,2001.

[11] E. Haaren-Retagne. A Semi-Infinite Programming Algorithm for RoboticTrajectory Planning, Phd thesis, Universitat Trier, Trier, Germany, 1992.

[12] E. Hansen. Global Optimization Using Interval Analysis, Marcel Dekker,1992.

[13] R. Hettich and K.O . Kortanek. Semi-infinite programming: theory, methods, and applications, SIAM Review, 35:380-429, 1993.

[14] B. Hu, K.L. Teo, and H. P. Lee. Local optimization of weighted jointtorques for redundant robotic manipulators, IEEE Transactions onRobotics and Automation, 11:422-425, 1995.

[15] L.S. Jennings and K.L. Teo. A computational algorithm for functionalinequality constrained optimization problems, Automaiica, 26:371-375,1990.


[16] O. Kniippel. PROFIL - programmer's runtime optimized fast intervallibrary, Technical Report 93.4, Technische Universitat Hamburg-Harburg(Germany), 1993.

[17] D. Li, A. A. Goldenberg, and J. W. Zu. A new method of peak torquereduction with redundant manipulators, IEEE Transactions on Roboticsand Automation, 13:845-853, 1997.

[18] Ci-S. Lin, P.-R Chang, and J.Y.S. Luh. Formulation and optimization ofcubic polynomial joint trajectories for industrial robots, IEEE Transactions Automatic Control, AC-28:1066-1074, 1983.

[19] D.G. Luenberger. Linear and Nonlinear Programming (Znd ed.).Addison-Wesley, 1989.

[20] S.P. Marin. Optimal parametrization of curves for robotic trajectory design, IEEE Transactions on Automatic Control, AC-33:209-214, 1988.

[21] R Menozzi, A. Piazzi, and F. Contini. Small-signal modeling for microwave FET linear circuits based on a genetic algorithm, IEEE Transactions on Circuits and Systems, Part I: Fundamental Theory and Applications, 43:839-847, 1996.

[22] M. Mitchell. An Introduction to Genetic Algorithms, MIT Press, Cambridge, MA, 1996.

[23] RE. Moore. Interval Analysis, Prentice-Hall, Englewood Cliffs, 1966.[24] R.E. Moore. Methods and Applications ofInterval Analysis, SIAM Press,

Philadelphia, 1979.

[25] F. Pfeiffer and R. Johanni. A concept for manipulator trajectory planning,IEEE Transactions on Robotics and Automation, RA-3: 115-123, 1987.

[26] A. Piazzi and G. Marro. Robust stability using interval analysis, International Journal ofSystems Science, 27:1381-1390, 1996.

[27] A. Piazzi and A. Visioli. Global minimum-time trajectory planning ofmechanical manipulators using interval analysis , International Journal ofControl, 71:631-652, 1998.

[28] R P. Podhorodeski, A. A. Goldenberg, and R. G. Fenton. Resolvingredundant manipulator joint rates and identifying special arm configurations using jacobian null-space bases, IEEE Transactions on Robotics andAutomation, 7:607-618, 1991.

[29] E. Polak. Optimization: Algorithms and Consistent Approximations.Springer-Verlag, 1997.

[30] H. Ratschek and J. Rokne. New Computer Methods for Global Optimization, Ellis Horwood Limited, Chichester, UK, 1988.

[31] R Reemtsen and S. Gomer. Numerical methods for semi-infinite programming: a survey. In R Reemtsen and J.-J. Riickmann, editors, Semi-InfiniteProgramming, pages 195-275, Kluwer, 1998.


[32] M. Schoenauer and S. Xanthakis. Constrained GA optimization. In Proceedings of the Fifth International Conference on Genetic Algorithms,pages 573-580, Urbana-Champaign, USA, 1993.

[33] L. Sciavicco and B. Siciliano. Modelling and Control ofRobot Manipulators, Advanced Textbooks in Control and Signal Processing, SpringerVerlag, 2000.

[34] Z. Shiller and H.-H. Lu. Computation of path constrained time optimalmotions with dynamic singularities, Journal ofDynamic Systems, Measurment and Control, 114:34-40, 1992.

[35] K. G. Shin and N. D. McKay. Minimum-time control of robotic manipulators with geometric path constriants , IEEE Transactions on AutomaticControl , AC-30:531-541, 1985.

[36] K.L. Teo, C,J . Goh, and K.H. Wong. A Unified Computational Approachto Optimal Control Problems, Longman Scientific and Technical, Harlow,UK,1991.

[37] K.L. Teo, V. Rehbock, and L.S. Jennings. A new computational algorithmfor functional inequality constrained optimization problems, Automatica,29:789-792, 1993.

Chapter 14

ON STABILITY OF GUARANTEEDESTIMATION PROBLEMS: ERROR BOUNDSFOR INFORMATION DOMAINS ANDEXPERIMENTAL DESIGN

Mikhail I. Gusev and Sergei A. RomanovInstitute of Mathemat ics and Mechanics. Ural Branch of the Russian Academ y of Sciences.

620066. ul. Kovalevskoi , 16. Ekaterinburg , Russia

[email protected], dezir@cily line.ru

Abst ract The two guaranteed estimation problems related to semi -infinite programmingtheory are considered. For abstract estimation problem in a Banach space westudy the dependence of information domains ([12), [14]) on measurement errorswith intensity of errors tending to zero. The upper estimates for the rate ofconvergence of information domains to their limit in the Hausdorff metric arcgiven . The experimental design problem for estimation of distributed systemwith uncertain parameters through available measurements is also considered inthe context of guaranteed estimation theory. For the stationary sensor placementproblem we desc ribe its reduction to a nonlinear programm ing problem. In thecase of sufficiently large number of sensors it is shown that the solution may beobtai ned by solving linear semi -infinite programming problem .

1 INTRODUCTION

A conventional approac h to the study of uncertain systems relates to the assumption that uncertain ty may be described as a random process with knowncharac teristics. In many applied problems, however, there may be a limitednumber of observations, incomplete knowledge of the data , and no available statistics whatever. An alterna tive approach to the uncertainty treatment,known as guaranteed (see , e.g. [12], [14]), is based on set-me mbership (un-

299

M.A. Goberna and M.A. Lopez (eds.), Semi-Infinite Programming. 299-326.© 2001 Kluwer Academic Publishers.


known but bounded) error description. In the problems considered here theset-membership description of uncertainty is employed.

We examine a guaranteed estimation problem in the framework of the following abstract scheme:

subject to

find z = Fw

y = Aw + ~, w E W, ~ E B.

(1.1)

(1.2)

Here A : X -+ Y represents an input-output operator for uncertain system,w E X is treated as an unknown but bounded input (disturbance), y E Y isconsidered as a result of measurements of system output, ~ E Y is a measurement error, F : X -+ Z, and X, Y, Z are assumed to be the real Banach spaces.The sets W C X, BeY represent a priori restrictions on uncertain parametersand measurement errors, further it is assumed that B = {~ : II~II ~ 8}. Theestimation problem consists in the estimation of an unknown value of z = Fwon the basis of available measurement y under constraints (1.2).

A rich variety of estimation problems may be treated in the context of theforegoing general scheme (see, e.g, [18]) . As an example consider here aninput-output operator for the following uncertain system

dxjdt = A(t, x) + B(t, x )u (t ), x(to) = xo,

with the measurement equation

y(t) = hdt, x(t)) + h2(t, x(t))u(t) + ~(t),

(1.3)

(1.4)

on a finite time interval t E [to, ttl . Here x(t) E H" is a state space vector, y(t) E H'" is a measurable output, and u(t) E RT is an unknown input(disturbance). Assume that the vector function A(t, x) and the matrix function B(t, x) are continuous in t , x, and Lipschitzian in x , and satisfy the conditions IIA(t, x)1I ~ Cdl + IIxll), IIB(t, x)1I ~ G2 for some G1, G2• Leth1(t, x), h2(t, x) be continuous in t, x . All a priori information on an unknowninitial state xo, disturbance u(t), and measurement errors ~ (t) are assumed tobe restricted to the relations xo E Xo, u(t) E P, 1~(t)1 ~ 8, to ~ t ~ t1,where XO c. u", PeRT are compact sets ; hereafter I~I (or 1~lm ) standsfor a norm in H'": Consider w = (xO,u('))' X = R" x L~, Y = L~ .The operator A, defined by equations (1.3) and (1.4), is a continuous operator.Assume that Fw = X(t1,xO,U( '))' where X(t1,xO,U( ')) denotes a trajectoryof system (1.3), corresponding to initial vector xO and disturbance u(·). In thiscase problem (1.1),(1.2) transforms into a standard state estimation problemwith unknown, but bounded, disturbances and measurement errors.

ON STABILITY OF GUARANTEED ESTIMATION PROBLEMS 301

In general, z can not be uniquely restored due to the fact that A has no inverseand, also, due to the presence of the measurement errors. Define the followingset:

Z.s(Y) := {Fw : fj = Aw + e,w E W, II~II ~ 8}, (1.5)

which is referred to, in guaranteed estimation theory, as the information domainrelative to measurement fj ([12], [14]). In this paper we study the dependenceof Z.s(Y) on measurement errors when the magnitude of errors tends to zero.This problem is closely related to the stability of guaranteed estimation problemwith respect to data disturbances.

Let us refine the problem statement. The set

Zo(y) := {z = Fw : Aw = y, wE W}, 0 .6)

will be referred to as the information domain relative to precise measurementof y . Further we assume that there exists w E W such that y = Aw; hence,Zo(Y) =1= 0.

Assume that W is a weakly compact subset of X. A connection betweenZo(Y) and Z.s(Y) follows from the next proposition in [4]:

Proposition 1.1 Let lIy - fjll ~ 8. Then Zo(Y) C Z.s(fj). If the operatorsA, F are weakly closed. and F is completely continuous, then h( Zo(y), Z.s (fj)) -+oas 8 -+ 0, where h denotes the Hausdorff distance.

Note that an operator A is said to be weakly closed (sequentially weaklyclosed) iff for every sequence W n E X the conditions W n ---->. w, AWn ---->. yimply the equality Aw = y. Here the sign ---->. denotes weak convergence in thecorresponding space. The considered example (1.3), (1.4) of state estimationproblem satisfies the assumptions ofProposition 1.1 if XO,P and Q are convexcompact sets.

The rate of convergence of h(Z.s(Y), Zo(Y)) to zero may be rather low whenboth X and Y are infinite-dimensional spaces . The situation reverses if atleast one of this spaces has a finite dimension. In the next section we establishthe estimates h(Z.s(fj), Zo(y)) = 0(8) for problems with normally resolvablelinear operator A. This class of estimation problems includes, for example,the systems with a finite numbers of observations and systems with finitedimensional space of uncertain parameters. Such dependence is found on thebasis of the results on duality in convex programming.

Note that the studied problem is closely related to the results on error boundsfor the sets ofconvex inequalities (see, e.g. [23], [10]). The stability of minimaxestimates in the guaranteed estimation theory, and their connection with theregularization algorithms for ill-posed problems ([24], [9]) have been studiedextensively in [15] and [16]. In [1], to derive the corresponding estimates, the


initial problem has been reduced to an equation with a normally resolvableoperator.

In many applied problems where uncertainty is inherent in it is possible toaffect the observation process. As for dynamical systems, the choice of controlplant inputs for the parameters identification gives a typical example of observations control. Another one concerns the measurement allocation problem,arising, for example, in environmental monitoring. Controlling the observations can reduce the estimation error. The measurement allocation problemsfor estimation of distributed system with uncertain parameters were studiedby many authors (see, for example, [3], [19], [22], [25], [26]) . These worksemploy a conventional approach based on stochastic description of unknowns.

The experimental design problem for the systems with set-membership description of uncertainty as applied to the abstract scheme (1.1), (1.2) may betreated as follows . Assume that the operator A depends on control parameteru E U affecting the measurement process so that y = A(u)w + ~ and thevalues of error bounds are fixed. The information domain Z6 depends hereon u : Z6 = Z6(Y,U). Let \11(0) be a real-valued function defined on thebounded subsets of Z such that \I1(Z6) characterize the accuracy of estimation.A frequently used function is, for example, the radius of a set defined by theequality

\I1(C) = inf sup IIx - yli.xEZ yEC

The experimental design problem then may be represented in the followingform

inf sup \I1(Z6(Y, u)),uEU y

where supremum is taken in all available values of the output. This type ofexperimental design problem for dynamic systems with magnitude bounds onuncertain items was considered in [5]. The optimality conditions for an abstractmeasurement optimization problem are given in [21]. The problems of opti mization of the trajectory of a movable sensor for distributed parameter systemwith integral restriction on uncertain parameters were examined in [13] and[11].

In this paper we consider a linear distributed-parameter system, describedby the diffusion equation in R2 with uncertainty in the right-hand side. It isassumed that all accessible information on the solution of the system is givenby the measurements performed by the sensors at finite number of points of theplane. The considered model of uncertainty and measurements errors is nonstochastic with set-membership description of the unknowns. We study theproblem of allocation of the sensors inside a given domain in order to ensurethe best possible estimate of the linear functional of the solution. The "duality"


results, which state that examined problem is equivalent to some impulsivecontrol problem, are given. Assuming that the unknown data do not depend ontime a theorem on sufficient number of sensors is proved and the reduction ofthe measurements allocation problem to a nonlinear programming problem isdescribed.

2 RATE OF CONVERGENCE OF INFORMATIONDOMAINS FOR PROBLEMS WITH NORMALLYRESOLVABLE OPERATOR

Consider the estimation problem (1.1), (1.2), assuming that A, F are linearcontinuous operators, W = {w EX : IIwllx ::; J.t}, B = {e E Y : lIelly ::;8}, u, 8 > O. In this case Zo{y), ZtS(Y) are convex closed bounded sets. Findthe values

zo{y) = p{z*IZo{Y)), ZtS(Y) = p{z*IZtS(Y)),

where z" E Z* , p{z* 18) denotes the value of support function of a set 8 c Z:

p{z*18) := sup{(z*,z)z: z E 8},

where (', ·)z represents a pairing between Z and Z*, Z* being an adjointspace. Denote I = F* z", where F* is the adjoint of F, zo{y) and ZtS(Y) maybe determined by solving the following convex programming problems

zo{Y) = max{(j,w)x : IIwllx ::; u; Aw = y}

and

ZtS(Y) = maxi (j, w)x : IIwllx ::; u , lIelly ::; u; Aw + e= v}·

We find ZtS(Y) and zo{Y) by solving the dual problems. The non-symmetricminimax-theorem[2],being applied to Lagrange functions for considered convex programming problems, yields

where

zo{Y) = inf{rpo{).) :). E Y*} ,

ZtS(Y) = inf{rptS{).) : ). E Y*} ,

(2.1)

(2.2)

rpo{).) = J.tIlA*)' - IlIx- + ().,y)y ,

rptS{).) = IIA*)' - IlIx- + 811).lIy- + ()., y)y,

where the values zo{y), ZtS(Y) are finite.A linear continuous operator A : X -+ Y is said to be normally resolvable,

if its range H := AX is closed in Y. If A is normally resolvable then A* isalso normally resolvable; hence, H* := A*Y* is closed in X* (see, e.g., [9]).

Lemma 2.1 Let A be a normally resolvable operator and y = Aw forsome w, IIwll < J.L. Then for each 1 = F* z* there exists the solution 5..(J) ofthe problem (2.1) such that

o~ zo(jj) - zo(y) ~ 26115..11,

provided that lIy - yll ~ 6.

(2 .3)

Proof. Further we omit subscripts in nonn notation. From the equality y = Awit follows that

cpo(>') = J.LIIA*>' - 111 + (A*>',w).

We show that infimum in (2.1) is attained for some 5... Denoting x = A *>., then

inf cpo(>') = inf 'l/Jo(x),>'EY' xEH*

where7/Jo(x) := J.LIIX - 111 + (x, w).

The inequality

1/1o(x) ~ J.L(lIxll-IIJID -lIxllllwll = J.L(llxll-lIwll)lIxll- J.Lllfll

implies that 1/1o(x) -t 00 as IIxll -t 00. Note that 'l/Jo(x) is lower semicontinuous, and that the set

{x E X* : IIxll ~ I} n H*

is compact with respect to the *-weak topology. This implies that there existsx E H such that

1/1(x) = inf 'l/Jo(x).xEH'

From the inequality

it follows that

IIxll < 2J.L IIf II .- J.L -/Iw/l

Let 5.. be a normal solution of the equation A*>. = x, that is

5.. E argmin{II>'/1 : A*x= s,xE Y*}.

(2.4)

From the definition of 1/10 it is clear that cpo(>') attains a minimum at 5... This,together with the inequality

yields inequality (2.3), provided that lIy - yll ~ 8. 0

Theorem 2.1 Let X, Y be the real Hilbert spaces. A : X -+ Y be anormally resolvable operator, and y = Aw for some ib, IIwll < J1. and lIy yll ~ 8. Then

h(Zo(y), Z6(Y)) = 0(8).

Proof. It is known [9] that for a normally resolvable operator A* a normalsolution of the equation A*). = x may be represented as follows :

where (A *) # stands for the linear bounded operator, which is the pseudoinverseto A*. This implies

For the Hausdorff distance h between the convex bounded sets Zo(y), Z6(Y)the equality

h(Zo(y), Z6(Y)) = sup{lp(z*IZo(Y)) - P(Z*IZ6(y))1 : IIz*1I = I} (2.5)

holds . The last relations together with inequality II!II ~ IIF* IIl1z* II yield

This completes the proof of the theorem. 0

Let us return to the estimation of the rate of convergence in the case ofBanach spaces X, Y. An operator A is normally resolvable if either X or Y isa finite-dimensional space . This enables us to arrive at the following assertion.

Theorem 2.2 Let X , Y the real Banach spaces at least one of which isfinite-dimensional. Let y = Awfor some w, IIwll < u, and lIy - yll ~ 8. Then

h(Zo(y), Z6(Y)) = 0(8) .

Proof. In view of the equality (2.5) and the inequality (2.4) it is sufficient toprove that there exists a constant C such that

where 5.(x) is the normal solution of the equation A*)' = ii , Under the conditions of the theorem the subspace H* = A*Y* is finite-dimensional. Let

Xl, ... , x k be a basis in H*. Denote as uiUt), i = 1, ... , k, the coordinates of iwith respect to this basis:

k

i = L Ui(i)xi.

i=l

Defining the functionk

,(i) := L IUi(i)l,i=l

it is clear that ,(i) is a norm on H* . Since H* is finite-dimensional, thereexists G1 such that ,(i) ~ Gdlill .

Denote by Ai an arbitrary solution of the equation A*A = xi,i = 1, oo.,k.Then A = :E~= 1 Qi(i) Ai is a solution of the equation A*A = ii : Hence,

k

115.(i)1I ~ IIAII m?X II AiII L IQi(i)1 ~ Gllill ,l~z~k .

z=l

where G = max1~i~k IIAi IlCl . 0

Remark 2.1 The assertions of the theorems hold true if W is a closed ballnot necessarily centered at zero.

Remark 2.2 Without the assumption y = Aw, IIwll < 1-", the rate of convergence may be arbitrarily low even if X, Y are finite-dimensional Banachspaces . Nevertheless for the case of Hilbert spaces X, Y, and a normally resolvable operator A, the estimate 0(81/ 2 ) for the rate of convergence is valid[7].

Presume that the system (1.3), 0.4) has a linear (in X, u) dynamics:

dx/dt = A(t)x + B(t)u(t) , x(to) = xo,

y(t) = C(t)x(t) + D(t)u(t) + ~(t),

and the measurements are performed at a finite number of points t ETC[to, ttl. Let the initial state xo, disturbance u(t), and the measurement error~ (t) be restricted by the magnitude constraints

xo E x", u(t) E P, to ~ t ~ t1, 1~(t)lm ~ 8, t E T,

where XO C Rn , PeRT are convex compact balanced sets, int.X'' =I0, intP =I- 0, intG denoting the set of all interior points of a set C. Letus take as X the set of pairs w = (xO, u(·)) E R" x L~ endowed with thefollowing norm

II(xo,u(.))1I = max{gxo(xo), max gp(u(t))} .to::;t91

Here gc(x) is a gauge (Minkowski) function of a set C; i.e.,

gc(x) = inf{a > 0: ax E C}.

Take the output Y = y(.) which satisfies the following requirement: there existan initial state x* E intX and a disturbance u*(t) E intP, to ~ t ~ t l, thatgenerate y(.) due to the system and measurement equations (under ~(t) == 0).In this case the assumptions of Theorem 2.2 are fulfilled. and the estimateh(Zo(y) , Zo(Y)) = 0(<5) takes place for each linear continuous operator F(for example, for F determined by the equalities Fw = x(B,xO,u('))' withBE [to, tl] be given, or Fw = u(·)).

If both X and Y are finite-dimensional spaces, and W is a convex polytope,then the equality h(Zo(y), Zo(Y)) = 0(8) holds without additional assumptions. It easily follows from Hoffman's error bound [8].

As an another example consider the state estimation problem for the following linear discrete-time uncertain system

with the measurement equation

Yk = CXk + DUk + ~k , k = 0,1, ..., N,

(2.6)

(2.7)

where Xk ERn, Uk ERr, Yk E R'"; k = 0, ... , N. Let the a prioriinformation on unknown disturbances Uk and measurement errors ~k be givenby the inclusions Uk E V, and the inequalities I~klm ~ 8, k = 0, ... , N. Itis necessary to estimate x N on the basis of available measurements of outputYk, k = 0,1, ...,N.

Define w = (xo , Uo, ... , UN), Y = (YI' ...,YN), .~ = (~o, ..., ~N), W XO x V x ... x V c Rn+r(N+I), II~II = IDaxO<k<N 1~lm. With w -(xo, Uo , ..., UN) given, the equalities - -

Yk = CXk + DUk' k = 0, 1, ...,N,

together with (2.6), define the linear operator A : Rn+r(N+l) -+ Rm(N+I)such that Y = Aw; the linear operator F : Rn+r(N+I) -+ H" is defined by theequality Fw = XN.

Let Zo(Y) be the set of all vectors z E Rn having the following property:there exist Xo E XO, Uk E V, I~k 1m ~ 8, k = 0, ... , N, generating a trajectory(xo, ..., XN) of (2.6), and an output fj = (Yo, ..., YN) such that

(2.8)


Assume that the sets V and Xo are convex polytopes. Let y = Aw for somewE W . Then

h(Z.s(Y), Zo(Y)) = 0(8)

under IYk - Yklm ~ 8, k = 0, ... , N. Note that the last estimate has an optimalorder.

The estimates derived here can be easily extended to the case when theoperator A is given with an error. Assume that instead of A we know A suchthat IIA - All ~ h. The information domain

Z.s,h(Y) = {Fw : (A + ~A)w + e= ii, IIwll ~ 11-, lIell ~ 8, II~AII ~ h}

is the set of all values of z = Fw that are consistent with the measurementresults and a priori constraints this set is nonconvex in general. Define also thefollowing convex domain

These domains are related by the following inclusions:

Zo(Y) C Z.s,h(Y) C Z.s,h(Y),

under lIy - ilil ~ 8, IIA - All ~ h.

Note that, in practice, one may construct only domain Z.s,h(Y) but not Z.s,h (Y),because the precise value of A is unknown. The inclusion (2.8) is used here toestimate the deviation of Z.s,h (Y) from Zo(y). For example, if one of the spacesX, Y is finite-dimensional, we have the following consequence ofTheorem 2.2

Theorem 2.3 Let Y = Aw, IIwll < 11-, lIy -1711 ~ 8, and IIA - All ~ h.Then the following equality holds

h(Z.s,h(Y), Zo(y)) = O(TJ) ,

where TJ = max {8,h} .

As an example, consider the problem of identification of a vector of unknownparameters {) E Rq for the system

x(t) = D({))rp(x(t)) + u(t) , x(to) = 0,

on the basis of observation of the trajectory

y(t) = x(t) + e(t), to ~ t ~ tl·

(2.9)

Here x(t), y(t), u(t) ERn, and u(·) E L~ is supposed to be known input. Theidentification problem is treated here under the following assumptions:

I) cp : R n -+ H" is K-Lipschitz function on Rn with respect to the norm

2) D(fJ) = DdfJ) + Do, where Dl(fJ) is an n x n matrix function linear infJ, and Do is n x n matrix; and

3) a priori restrictions on coordinates of fJ = (fJl, ... , fJq ) and measurement errors are given in the form of magnitude bounds: di ::; fJi ::; ei, i =1, ... , q, (di < e.), and le(t)ln ::; 6, to ::; t ::; t l .

The identification problem consists in the determination of information domain, consisting of unknown parameters fJ, compatible with the measurementresults.

DefiningfJ? = (ei +dd/2, i = 1, ...,q, fJo = (fJ~, ... ,fJg),

IIfJllq = max (ei - di)/2IfJil,l~l~q

then a priori restrictions on fJ may be written as IIfJ - 'l9°llq ::; 1.

Denoting w = fJ - fJo, and integrating equation (2.9) , we can describe adomain corresponding to precise measurements as follows:

Zo(Y) = {w : IIwllq ::; 1, D l (w) r ep(x(r»drito= «I>(x(·»(t), to ::; t ::; ttl + fJo ,

where y = x( ·) is a trajectory of the system (2.9), corresponding to unknown

actual value J of fJ, and the mapping «I> : L~[to , ttl -+: L~[to , ttl is definedby the equality

«Jl-(x(.»)(t) = x(t) - r u(r)dr _ (Dl(fJO) + Do) (t ep(x(r»dr.i~ i~

Because the precise value x(t) is unknown, all accessible information on fJ isrestricted to the following set

Zo(Y) = {w : IIwll q ::; 1, Dt{w) r ep(Y(r) - e(r»dr + e(t)ito= «I>(YO - eO)(t) , le(t)ln::; 8, to::; t ::; ttl + fJo,

where y = y(t) is a result of measurements of the trajectory x(t) .

Defining an operator A : Rq -+ L~[to, tl] by the equality

(Aw)(t) = Dl(w) r ep(x(r»dr, to::; t ::; tl,ito

then Amay be defined as follows :

(Aw)(t) = Dt{w) {t ep(Y{r))dr, to ~ t ~ tl.ito

Let IDxln be a norm of n x n matrix D

IIDlin = max{IDxln : Ixln ~ I},

and let

It is clear thatIIA - All ~ h:= MK{tl - to)8,

ifIy{t) - x{t)ln ~ 8, to ~ t ~ tt·

Note that

1I<p{y{·)) - <p{x{·)) II ~ (I + IIDt{'l9°) + DollnK{tl - to))8.

Hence, Z5(Y) C Z17(Y)' where

Z17(Y) = {w : IIwllq ~ 1, Dt{w) r ep{x{r))dr + 1J{t)ito= <p{x{·))(t), 11J{t)ln ~ a, to ~ t ~ td + 'l90

,

ifa ~ 2h + (I + IIDt{'l9°) + DollnK{tl - to))8.

Suppose that an unknown value J,generating the trajectory x{t), is such that

IIJ - 'l9°ll q < 1. From Theorem 2.2 and the inequality

h{Zo{Y),Z5(Y)) ~ h{Zo{Y)' Z17(Y)),

we get the estimate h{Zo{Y),Z5(Y)) = 0(8).

3 OPTIMAL PLACEMENT OF SENSORS FORNONSTATIONARY SYSTEM: DUALITYTHEOREMS

Consider the distributed field described by the solution to the following diffusion equation in R2

k

~ep + divwep + asp - J.L /::::,.ep = L Qi{t)8{x - ad, 0 ~ t ~ T, (3.1)t i=l


with initial and boundary conditions

cp(O, x) = cpo(x), cp(t, x) -+ 0 as Ixl -+ +00.

Here x = (Xl, X2), b..cp = p82

cp + p82

ip is the Laplace operator, 8(x) is theXl x 2

Dirac delta function, Qi (t) are measurable bounded functions on [0,T], and CT, I-"are given positive numbers. The vector function w = w(x) = (WI (x), W2(X))is assumed to satisfy the continuity equation

d· () () 0IVW = ~WI +~W2 = .

UXI UX2

For cpo(-) E L 2(R2 ) there exists an unique weak solution of the equation

(3.1).The system is disturbed at k given points on a plane al, ... ,ak. The pertur

bations values, described by the functions Qi(t) and an initial state are assumedto be unknown in advance. Suppose that the state ip is observed according tothe following measurement equations

Yj(t) = cp(t,bj) +~j(t), j = 1, ... ,8.

Here ~ (t) is the measurement error of the j-th sensor, and the points bl , . .. ,bs ,

describing the "sensors placement", have to be chosen inside a given domainf2*. Thus, the perturbations in the right hand side of the equation, the measurement errors, and initial system state CPo are assumed to be unknown. Allapriori information on unknowns is restricted to the inclusion

u:= {Qi('), i = 1, ... , k, ~j(')' j = 1, ... , s, cpoO} E U,

where U is a given subset of the space L~[O,T] x L~[O, T] X L 2(R2). A priorirestrictions can be represented, either in the form of the magnitude constraints

U={U:O~Qi(t)~Qi' i=l, ...,k,

l~j(t)1 ~ ej, j = 1, ... , 8, tl ~ t ~ tl, 0 ~ cpo(x) ~ rpo(x), x E R2},

or in the form of quadratic constraints

k T s TU = {u : I: I N(t)(Qi(t) - Qi(t))2dt + I: I M(t)(~j(t) - ~j(t))2dt

i=IO j=IO

+ I K(x)(cpo(x) - rpo(x))2dx ~ O},R2

where N(t), M(t), K(x) ~ a > O.

(3.2)


It is necessary to estimate the value of the integral

T

1=! ! rp(t, x)dtdxo n

on the basis of measurements of y(t), 0::; t ::; T . It can be shown [6] that thebest possible result of estimation may be achieved in the class of continuousaffine estimates h(y( ·))

T

h(y(·)) = !(z(t), y(t))dt + zo, z(·) E C[O, T], Zo E R.

o

We consider the following problem: how to allocate sensors inside domainfl* in order to reduce the guaranteed error of estimation? More precisely theproblem of sensors placement is as follows:

F(b1, • • • ,bs ) ~ min,bjEn-

F(b 1, • •• , bs ) = min q>(z(·), zo),z('),zo

T T<p(z(·), zo) = max I(z(t), y(t))dt + Zo - I I rp(t, x)dtdx ,

o 0 nwhere the last maximum is taken in all u = {Q(.), ~(.) <fi(.)} E U.

The discussion is motivated by the environmental monitoring problems [17].The diffusion equation (3.1) represents a two-dimensional model for the dispersion of air pollution with known air flow velocity w. The sources of pollutionai are assumed to be known, but their outputs Qi(t) are not. So the problemconsists of allocating sensors inside the region fl* in order to get more accurateestimate of the amount of pollution of the region fl.

The difficulty of the above problem consists in the implicit dependence of thefunctional F on its arguments: to get the value of F(b), b = (b1 , .. . , bs ) E fl*,one should solve the minimax optimization problem.

However the problem discussed may be put in a more convenient form if weproceed to an adjoint problem. The following theorem shows that consideredproblem is reduced to some optimal control problem for the adjoint distributedsystem.

Theorem 3.1 A sensor placement problem (3.2) for distributed system(3.1) is equivalent to the following optimal control problem

J~min, z( ·) EC([O,T]), bERs ,z(') ,b


-~ - divw'l/J + a'l/J - J.L ~'ljJ = 2:j=l Zj(t)O(X - bj) - Xn(x) ,

'ljJ(t, X)lt=T = 0, 'ljJ(t, x) ~ 0, [z]~ 00, divw = 0,

where

{1 under x E 0*

xn(x) = 0 under x f/. 0*

Proof. For the magnitude constraints J = J1, where

k T

f 1-J 1 = 2: 2"Qil'ljJ(t, ai)ldt

i=10s T

+ 2: ej f IZj(t)ldt + ~ f ep(x)I'ljJ(O,x)ldx,j=l 0 R2

and for the quadratic constraints J = h,

T kh = f 2: Ni-

1(t)'ljJ2(t, ado i=l

s+ ?= fOT Mj-

1(t)ZJ(t)dt + f K(x)-17/!2(O, x)dx.J=l R2

Here the control is a sum of s impulses whose placement and values have to bechosen to minimize functional J. The proof is based on the familiar schemeof transition to the adjoint equation. Multiplying both sides of (3.1) by thefunction 'ljJ and integrating one gets

T T T

JJ'ljJ ~~ dtdx +JJ'ljJ divwcp dtdx +JI a'ljJcpdtdxow ow OW

T T k-JJJ.L'l/J~cpdtdx = JI 'ljJ~ Qi(t)O(X - ad dtdx. (3.3)o R2 0 R2 z=l

By integrating by parts, applying Ostrogradski-Gauss and Green's formulas,and taking into account that divw = 0 and 'ljJ(t, x) -+ 0 as Ixl -+ 00, theequation (3.3) may be represented in the following form:

T

f f cp[-if -divw'ljJ+a'ljJ-J.L~'ljJ]dtdxo R2

T

= f f 'l/J(t, X)2::=l Qi(t)o(x-ad dtdxOR2

+ f[cp(O, x)'ljJ(O, x)-cp(T, x)'ljJ(T, x) ]dx.R2

Assume that '!fJ is the solution of the equation

with initial and boundary conditions

'!fJ(T, x) = 0, '!fJ(t, x) -+ °as Ixl-+ 00.

From (3.4) it follows that

T

I I <p(t, x) [L:.i=l Zj(t)8(x - bj) - xn(x)] dtdxo R2T

= I I '!fJL:7=1 Qi(t)8(x- ai)dtdx+ I <p(0, x)'!fJ(O, x)dx.ow R2

Thus

k Tq>(z(t), Zo) = max IL: I Qi(t) I '!fJ8(X- ai)dtdx

{Q(') '€( ') r,li(')}EU i=l 0 R2

s T I+ I <po'!fJ(O, x)dx + L: I Zj(t)ej(t)dt + Zo .R2 j=lO

Denoting

( ()Qi cfio(x)

Pi t) = Qi t - 2' Po = <po(x) - -2-'

then

k Tq>(z(t), Zo) = sup L: I Pi(t)'!fJ(t, ai)dt

IPi(t)I:5~ i=l 0

(3 .5)

s T+ sup L: I Zj(t)~j(t)dt + IGI + sup I po(x)'!fJ(O, x)dx

l€j(t)l:5ej j=l 0 Ipo(x)I:5r,lio(x)/2R2k T s T

= L: ~QiII'!fJ(t,ai)ldt+ I: Iejlzj(t)ldti=l 0 j=lO

where

+! ~cfio(x)I'!fJ(O, x)ldx + IGI,R2

k T

C=Zo+ L: ~QiI'!fJ(t,addt+ ~ I cfio(x)'!fJ(O,x)dx.i=l 0 R2

(3.6)


To complete the proof let us note that after minimization in Zo the last tenn in(3.6) vanishes. The proof for the case of the quadratic constrains on uncertaintyis analogous. 0

The differentiability and strong convexity of the functional h allow to simplify the studied optimal control problem.

Theorem 3.2 A sensor placement problem (3.2) for the case of quadratic constraints is equivalent to the following optimal control problem for thedistributed system (3.1)

J(b 1, •• • ,bs ) --+ mmb1, ..• .b,bj E 0*

k T s TJ = 2: J Ni-

1(t)'l/J2(t, ai)dt + 2: J Mj(t)p2(t, bj)dti=10 j=10

+ J K-1(x)'l/J2(0, x)dx,R2

(3.7)

1Jf + divpw + ap - J.L D.pk

= 2: Ni-1(t )'I/J (t ,ado(x - ad +K- 1(x)'I/J (0, x)o(t),

i=1-~ - divw'I/J + a'I/J - J.L D.'I/J = 2:j=1 Mj(t)p(t, bj)o(x - bj) - xn(x),

'I/J(T,x) = 0, p(O,x) = 0, 'I/J,p -+ ° as IIxll-+ +00.

4 OPTIMAL SENSOR PLACEMENT: THESTATIONARY CASE

Consider the case when the outputs Qido not depend on the time and w isconstant. Here the original problem is transformed into the static one in thesense that all considered values do not depend on the time:

(4.1)

ep(x) -+ ° as Ixl -+ 00.

The solution of the equation (4.1) may be represented in the form

k

ep(x) = L Qiepi(X),i=1

(4.2)

where cpi(X) is the solution of (4.1) with right-hand side being equal to 8(X-ai) :

cpi(X) = g(lx - ail), g(x) = 2;Ko(~x), and Ko(x) is the MacDonald

function00

Ko(x) = Je-XChYdy, x> 0,

o

where ch stands for the hyperbolic cosine function ch y = (eY + e-Y ) /2.For a stationary system the estimation problem takes the following form. On

the base of measurements

(4.3)

and a priori information on unknowns

0::; Qi ~ Qi' i = 1, ...,k, lejl::; ej, j = 1, ...,8,

denoted briefly as {Q, 0 E W, it is necessary to estimate the value

1= Jcp(x)dx = (Q, J),n

where Q = (Ql,'" , Qk)T, f = (iI, · · · ,fk)T, t, = Jcpi(x)dx .n

Hence the problem of sensors placement is as follows:

F(b1, .. . , bs ) = min <p(z, zo),ZER" ,zoER

<p(z, zo) = sup I(z, y) + Zo + (Q,J)I·{Q,nEW

Direct calculations show that

(4.4)

The further reasoning is based on the following known relation:

k k

min L IIi(x)1 = min L(ri + qi),xERn . XERn ,Ti,qi .

1=1 1=1

subject to

ri - qi = f(x) , ri ~ 0, qi ~ 0, i = 1, ... , k.

In view of this equality the studied problem may be rewritten as follows

minbj E 0*

r, q, t,P, Z

under constraintss

L <Pi (bj)Zj - Ii = ri - qi, i = 1, ..., k, Pj - tj = Zj, j = 1, ..., S,

j=I

r,q E Rt; P,t E R~; Z E RS,

or, after exclusion of {Z1' ... ,zs},

1 S

min min{-2(Q,r+ q) + Lej(pj +tj)} (4.5)b b r,q,t,p .1,·· ·, S 3=1

bj E 0*

under constraintsS

L<Pi(bj)(pj - tj) - Ii = ri -qi, i = 1,...,k, r.q E Rt, P,t E R~ .j=1

The minimization in {r, q, t,p}, in (4.5), constitutes a linear programmingproblem. In view ofthe duality theory oflinear programming, (4.5) is equivalentto the following.problem

mm max(f, y)b1 , •• • .b, YERk

bj E 0*

subject to

k

ILYi<Pi(bj)l=ej, j=l, ... ,s,i=1

I 1< Qi

Yi - 2' i = 1, ..., k.

Thus in the considered case it is necessary to minimize the function F(b) of2k variables, each value of this function is obtained as a result of the solution oflinear programming problem. The numerical simulation shows that the functionF(b) is multiextremal in general.

(5.2)


5 A SUFFICIENT NUMBER OF SENSORS

Here we treat the problem of sensor allocation, when the number s of sensorsis at least equal to the number ofsources k. In this case we consider the modifiedproblem statement for the stationary system by assuming that information canbe received on a system state function in the entire spatial domain n*. Let n* bea compact subset of R2 • Let us assume that the measurements of a system statefunction are performed at all points of n*, in accordance with the measurementequation

y(x) = cp(x) + ~(x) = (Q, <ll(x)) + ~(x), x E n*, (5.1)

where a measurement error ~(x) E C(n*), and uncertain parameters Qi, i =1, ... , k, are restricted by the inequalities

0::; Qi ::; Qi' i = 1,..., k, 1~(x)1 ::; u(x), "Ix E n*.

Here u(x) E C(n*) is a given function taking positive values, C(n*) denotes aspace ofcontinuous real-valued functions on n* , and <ll(x) = (CPl (x), ..., CPk (x)).It is necessary to estimate the value of

1=! cp(x)dx = ! (Q,<ll(x))dx,n n

on the basis of available information on y(x) . The minimal guaranteed valueof estimation error is achieved here for an affine estimate

h(y(.)) =! y(x)p(dx) + pO, pO E R,

n*

where a regular Borel measure ts on n* and a number pO are taken to be thesolution of the following minimax problem

iut sup !(Q, <ll(x))dx - h(y(x))p.,p. ~( '),Q n

Let Cu(n*) be a space of continuous functions on n* endowed with thefollowing norm

111011 = maxu- l(x)11(x)l·

xEn*

The set of all regular Borel measures is identified isometrically with the adjointspace C;(n*), where

IIpll = ! u(x)lp(dx)l·

n*


By IJ.L I we denote the total variation of J.L .Direct calculations give the following equality :

inf sup II(Q,«p(x))dx - h(y(X))1J.Lo ( (' ),Q n

=it~lt'Pi(X)J.L(dX) - hi + tu(x)IJ.L(dx) l.

Recall that f = (ft , ..., fk) ' where Ii = I 'Pi (x)dx, and denoten

Q = (Ql ' ..., Qk) , F(p., t ,p) = 1/2(Q, t + p) +! u(x)Ip.(dx)l·

n'

It is clear, that (5.2) is equivalent to the following semi-infinite convex programming problem

v = inf F(p., t ,p) ,t~O,p~O,J.L

subject to

! «p(x)p.(dx) - f = p - t.

n'

Consider a Lagrange function for problem (5.3), (5.4)

L(t,p, u; A) = ~ (Q, t + p) + I u(x) Ip.(dx) 1n'

+(A, f + p - t - I «p(x)p.(dx)), AE Rk .

n'

(5.3)

(5.4)

The minimax theorem [2] applied to L allows to assert that value (5.3) coincideswith the optimal value v* of the dual problem

subject to

Denoting

v* = max (f, A),

I(A,«p(x))1:S u(x ), \Ix E n*, IAi l :S~ , i = 1, .,' , k.

{2A' }7(A) = max max I(A, «p(x)) l/u(x) , max 1_ Z I ,

xE n' l ~i~k Q i

(5.5)

one observes that, is a norm in Rk .

Lemma 5.1 Let A* be a solution of the dual problem, and J.L E C*(O*),p, t E Ri be such that

Then

! (x)J.L(dx) - f = p - t.fl·

(5.6)

! (A*, (x) )J.L(dx) + (A*, t - p) = F(J.L, p, t)r(A*) (5.7)

fl·

if and only ifF(J.L,p, t) = v*.

Proof. By definition of A* we have (I, A*) = v* and ,(A*) = 1. Let theequality (5.6) holds. If F(J.L,p, t) = v*, then

!(A*, (x))J.L(dx) + (A*,t -p)fl·

= (I, A*) = v* = F(J.L ,p, t) = F(J.L,p, t)r(A*).

Conversely, if (5.7) holds, then

F(J.L,p, t) = F(J.L,p, t)r(A*)

= ! (A*, (x))J.L(dx) + (A*, t - p) = (I, A*) = v*.fl·

Thus, every solution J.L*, p*, t* of the problem (5.3) satisfies the equality

!().*, (x))J.L(dx) + (A*, r - p*) = F(J.L*, v', t*)r(A*). (5.8)

fl·

oThe proof of the theorem on the structure of the optimal measure J.L* is based

on the following result.

Theorem 5.1 ([20]) Let Yl, ... ,Yn be nonzero elements of the Banachspace X, and F : X* -+ R nisdefinedby the equality F(l) = (l(yd,· . . ,l(Yn)),l E X*. Assume that there exists a set D E X* such that

(a) IIlll = 1for each lED;

(b) the convex hull of D, convD, is dense in the unit ball S C X* in the*-weak topology; and

(c) if {lk}~l is an arbitrary sequence in X*, then there exists a subsequence {lkj} and 100 E D such that F(lkj) -T F{loo) as j -T 00.

Then, for each I E X*, there exist II, . . . , In E D and nonnegative numbersal,· ·· ,an such that F{l) = F(L::~=l aild and II L::~=l ailill ~ 11111·

Denote

r = {x E D*k

IL >';rpi(x)l/o-(x) = ,(>'*) = 1}.i=l

The next theorem defines the structure of the optimal measure.

Theorem 5.2 Let J.L* be the optimal measure for the problem (5.2). Then1J.L*I(D*\r) = 0; i.e., the measure J.L* is concentrated on the set I', Moreover,there exists a solution J.L* ofthe optimization problem (5.2) such that

/ y(x)J.L*(dx) = t J.LiY(xd, m ~ k.0* z=l

Proof. Assume that 1J.L*I(D*\r) > O. Then there exists closed set Be D*\rsuch that IJ.L*I(B) > O. From the inequalities

/(>.*,<])(x))J.L*(dx) + (>.*, r - p*) ~ max I(>.*, ~\x))1 /o-(x)IJ.L*(dx)1

xEB 0- X0* B

+ max 1(>'*, <])(x)) I / o-(x)IJ.L*(dx)1 + max 12>.; IL o,W+p!)'xEO*\B o-(x) iET Qi . T 2 z z

0* / B zE

and1(>'*, <])(x))1 1(>'*, <])(x)) I

max < max ,xEB o-(x) xEO* o-(x)

it follows that

[i», <])(x))J.L*(dx) + (>.*, r - p*)

0*

/

1 -< ,(>'*)( o-(x)IJ.L*(dx) I+ 2(Q ,P* + t*)).

0*

The last inequality contradicts (5.8). That proves the fist part of the theorem.

322 SEMI-INFlNI1E PROGRAMMING. RECENT ADVANCES

If r = 0 then IJL* I(0*) = 0, hence, JL* = O. In this case the assertion of thetheorem obviously holds. Further we assume that r =I- 0.

Define D C C;(O*) as follows: JL E D iff there exists x E 0* such thatJL( {x}) = o--l(!C), or JL( {x}) = -o--l(x) and JL(B) = 0 if x ~ B. Weidentify D with the subset of functionals I from C; (0*) :

1(J) = Jf(x)JL(dx) = ±o--l(x)f(x),

n'

f E C;(O*). (5.9)

Define a mapping F : C;(O*) -+ Rk by letting F(l) = (l(rpd, . . . , l (rpk)),where l(rpd = J rpi(x)JL(dx) .

n'We show that D satisfies the assumptions of Theorem 5.1. The assumption

(a) follows from the equality

11111 = Jo-(x)IJL(dx) I = 10--1(x)l· o-(x) = 1,

n'

arising from the definition of I = JL E D ; (c) follows directly from compactnessof 0* and continuity of 0-(x). To check (b) we need to prove that conyD isdense in the unit ball S C C;(O*) in the *-weak topology.

Let f E S . Then

f(f) = Jf(x)il(dx),

n'

where il E C;(O*) and 11111 = J o-(x) lil(dx)1 s 1.n*

Choose a neighborhood Of of f in the *-weak topology defined by elementsZl, . . . ,Zm from Co-(O*) and by a positive number e:

(5.10)

Let {Edi=l be a finite partition of the set 0* such that for each i = 1, ... ,m

v

!f(zd - LZi(17j)il(Ej)1 < e,j=l

where 'f/j is a point from Ej satisfying the inequality

o-('f/j)IMEj)1 s Jo-(x)IMdx)l·Ej

(5.11)

The existence of such partition follows from the continuity of functions Zi onthe compact set 0* and the properties of a countably additive measure. Define

lj E 0;(0*), j = 1, ... ,v, by the equalities

p,(Ej)lj(f) = f(T/j) a(T/j)Ip,(Ej) I

(if Jl(Ej) = 0 then we set lj = 0). Let

v

i = Llja(T/j) IJl(Ej) l.j=1

(5.12)

(5 .13)

From (5.11),(5.12) and (5.13) it follows that Il(zi) - f(zd I < to, i = 1, ..., m,that is f E 0[. By definition of lj, if lj =1= 0 then lj ED. Since

ta(T/j)Ip,(Ej)1 ~ j a(x)IJl(dx)1 ~ 1,3=1 n*

and the inclusion 1 E D implies -1 E D, the equality (5.13) means thati E convD. Thus, convD is dense in S; hence, the assumption (c) holds.

Recall that we identify a measure p.with a functional 1, andF(l) = (l( <PI), ••• ,l(<Pk)), where l(<pi) = I <pi(x)p.(dx). The problem (5.3) takes the form

n*

min (11111 + IIF{l) - fll),IEC;(n*)

where

k -

IIF(l) - fll = ?= ~i Il(<pd - hi and1=1

11111 = ja(x) Ip.(dx) I.n*

Let 1* = p.* be a solution of the problem. According to Theorem 5.1 therek

exists l' E 0;(0*), l' = L Ctjlj , such that F(l') = F(l*) and IIl'II ~ 111*11.j=1

Hence, l' is also a solution of the considered problem.Each lj corresponds to a measure p.j, having a single-point support. Hence

kthe support of the measure p.' = L Ctjp.j corresponding to l' contains no more

j=1than k points. This completes the proof of Theorem 5.2. 0

Theorem 5.2 asserts that there exists optimal measure which is concentratedat a finite number of points of the set 0*, and a number of optimal measureimpulses is no more than k. This theorem is an analogue of the Theorem of L.Neustadt for an impulsive control problem ([20]).

Thus, the sufficient number of sensors does not surpass the number of perturbation sources in the right hand side of the diffusion equation. The extra


sensors do not guarantee the better estimates. This property is peculiar to thegame-theoretic approach to the solution of sensor allocation problem employedhere. Note that additional sensors do not ensure a gain compared to k sensorsin the worst case realization of the unknowns. For a different realization of uncertain parameters and measurement errors an additional information providedby sensors may reduce the error of estimation.

For the case when there are k or more sensors a way of determining theoptimal coordinates of sensors is as follows, involving two stages. The first stageconsists in the solution of the semi-infinite linear programming problem (5.5) .

kAtthe second step it is necessary to find a set T = {x : 1:E !Pi(x )).i 10--1(x) =

i=1I} in the plane, all the sensors have to be placed inside this set. If T contains nomore than k points, these points uniquely determine the number of sensors andtheir optimal placement. Otherwise, it is necessary to solve a combinatorialproblem on allocating of k sensors among the points of r.

Acknowledgment

The research was partially supported by the Russian Foundation for BasicResearch, project no. 00-01-00646

References

[1] S.A. Anikin. Error estimate for a regularization method in problems of thereconstruction of inputs os dynamic systems, Computational Mathematicsand Mathematical Physics, 37:1056-1067, 1997.

[2] J.P. Aubin and L Ekeland. Applied Nonlinear Analysis, Wiley, 1984.

[3] W.H. Chen and J.H. Seinfeld. Optimal allocation of process measurements, International Journal ofControl, 21:1003-1014, 1975.

[4] MJ. Gusev. On the stability of solution ofthe inverse problems in controlsystem dynamics, Problems ofControl and Information Theory, 17:297310, 1988.

[5] M.l. Gusev. Measurement allocation problems in estimation of dynamical systems under geometrical constraints on uncertainty, Diff. Uravn.,24:1862-1870, (in Russian) 1988.

[6] M.l. Gusev. On the optimality of linear algorithms in guaranteed estimation. In A.B. Kurzhanski and V.M. Veliov, editors, Modeling Techniquesfor Uncertain Systems, Birkhauser, Boston, pages 93-110, 1994.

[7] M.L Gusev. On Stability of Information Domains in Guaranteed Estimation Problem. In Proceedings of the Steklov Institute ofMathematics,Suppl. I, MAIK "NaukalInterperiodika", pages 104-118,2000.


[8] A.J. Hoffman. On approximate solutions of systems of linear inequalities,Journal of Research of the National Bureau of Standards, 49:263-265,1952.

[9] V.K. Ivanov, V.V. Vasin and V.P. Tanana. Theory ofLinear Ill-Posed Problems and Applications, Nauka, Moscow, (in Russian) 1978 .

[10] D. Klatte and W. Li. Asymptotic constraint qualification and global error bounds for convex inequalities, Mathematical Programming, 8A:137160, 1999.

[11] E. K. Kostousova. Approximation of the problem of choosing an optimalcomposition of measurements in a parabolic system, U.S.S.R. Computational Mathematics and Mathematical Physics, 30: 8-17, 1990.

[12] A. B. Kurzhanski. Control and Observation under Uncertainty Conditions, Nauka, Moscow, (in Russian) 1977.

[13] A. B. Kurzhanski and A.Yu. Khapalov. Estimation of distributed fieldsaccording to results of observations. In Partial Differential EquationsProceedings of the International Conference in Novosibirsk 1983, pages102-108, (in Russian) 1986.

[14] A. B. Kurzhanski and I. Valyi. Ellipsoidal Calculus for Estimation andControl, Birkhauser, Boston, 1997.

[15] A.B. Kurzhanski and I.E Sivergina. On noninvertible evolutionary systems: guaranteed estimation and regularization problems, Dokl. Acad.Nauk SSSR, 314:292-296, (in Russian) 1990.

[16] A.B. Kurzhanski and I.E Sivergina. Method of guaranteed estimates andregularization problems for evolutionary system, Journal vichislit. matem.i mat. phiziki., 32:1720-1733, (in Russian) 1992.

[17] G.I. Marchuk, Mathematical Modelling and Environmental Problems.Nauka, Moscow, (in Russian) 1982.

[18] M. Milanese and J. Norton (editors). Bounding Approaches to SystemIdentification, Plenum Press, London, 1996.

[19] J. Nakamori, S. Miyamoto, S. Ikeda and J. Savaragi. Measurement optimization with sensitivity criteria for distributed parameter systems, IEEETransactions Automatic Control, AC-25:889-900, 1980.

[20] L. W. Neustadt. Optimization, a moment problem, and nonlinear programming, SIAM Journal ofControl, Ser. A, 2:33-53, 1964.

[21] V. Pokotilo. Necessary Conditions for Measurements Optimization. InA.B Kurzhanski and V.M. Veliov, editors, Modeling Techniques for Uncertain Systems, Birkhauser, Boston, pages 147-159, 1994.

[22] E. Rafajlowicz . Design of experiments for eigenvalue in identification indistributed parameter system, International Journal ofControl, 34:10791094, 1981.


[23] S.M. Robinson. An application of error bounds for convex programmingin a linear space, SIAM Journal on Control and Optimization, 13:271-273,1975.

[24] A.N. Tikhonov and V.Ya.Arsenin. Methods ofSolutions ofIll-Posed Problems , Nauka, Moscow, (in Russian) 1974.

[25] D. Ucinski. Measurement Optimization for Parameter Estimation in Distributed Systems , Technical University Press, Zielona Gora, Poland, 1999.

[26] D.l. Ucinski, J. Korbicz and M. Zaremba. On optimization of sensors motions in parameter identification of two-dimensional distributed systems,InProc. ZndEuropean Control Conference, pages 1359-1364, Groningen,The Netherlands, 1992.

Chapter 15

OPTIMIZATION UNDER UNCERTAINTY ANDLINEAR SEMI-INFINITE PROGRAMMING:ASURVEY

Teresa Leon and Enriqueta VercherDepartam ent d 'Estadistica i Investigacio Operativa. Universitat de Valencia.

Campus de Burjasot, Valencia, Spain

[email protected], [email protected]

Abstract This paper deals with the relationship between semi-infinite linear programmingand decision making under uncertainty in imprecise environments . Actually,we have reviewed several set-inclusive constrained models and some fuzzy programming problems in order to see if they can be solved by means of a linearsemi -infinite program. Finally, we present some numerical examples obtainedby using a primal semi-infinite programming method.

1 INTRODUCTION

Many mathematical model s for real world use constraints whose coefficientsare supposed to be fixed characteristics ofmodelled reality. Unfortunately theseparameters are, often, not known exactly because they are variable, unreliableor imprecise in a certain sense.

Several empirical phenomena may be related with this problem of nonexpressing accurately the values of the model parameters. The problem ofvagueness concerns the meaning of terms of a language used, for instance, toexpress our knowledge of a system. Imprecision is a problem of our inabilityto determine empirical quantities with unlimited accuracy. If we are asked towait for a moment, how long will we wait for? "A moment" is an indeterminateamount of time. When someone says something like "rich people should paymore taxes", what does he mean by "rich"? We should not confuse vaguenessand imprecision with another empirical phenomenon: the lack of knowledgewhich, many times is managed in terms of probabilities.

327

M .A. Goberna and M.A . Lopez (eds.), Semi-Infinite Programming, 327-348.© 2001 Kluwer Academic Publish ers.


Fuzzy sets theory allows us to deal with vagueness and imprecision on data.We could say that fuzzy sets were invented as a formal tool enabling the precise(or exact) manipulation of inexact (or imprecise) concepts. A tool, whichallows us to give a precise mathematical description of what are normally vaguestatements.

Some areas in mathematical programming model such problems by takingthe uncertainty on the parameters into account already in the phase of construction of the model: Stochastic Programming, Convex Programming withset-inclusive constraints, Fuzzy Mathematical Programming and PossibilisticProgramming.

Stochastic programming (see, for instance, Kall and Wallace [13] and Wetsand Ziemba [33]) is frequently used to supplement conventional mathematicalprogramming approaches in dealing with system impreciseness, but it requireslarge numbers of data for the identification of the probability distribution.

Sometimes the imprecise information may not be fully identified by probability theory, that is the exact values of the coefficients are not known withcertainty and the vagueness may not be of a probabilistic type. Therefore, ifthe decision maker must make an assignment of the decision variables beforethe exact parameter values are known, and he or she is able to model the inexact data by means of functional relations (see, for instance, Soyster [27] andTichatschke et al. [32]), then a set inclusive constraint model can be useful.As we have mentioned before, constructing a fuzzy programming model canalso help us to deal with vagueness and imprecision on data (Zimmermann [35]and Lai and Hwang [14]). The imprecise coefficients also may be modelledby means of possibility distributions; then possibilistic programming problemsare obtained.

Note that, for some instances that involve uncertainty, the optimal solutionis obtained by solving a finite linear programming problem . However, this isnot always the case and for some models no finite representation is apparentlypossible, so we should consider the semi-infinite programming methods as avery good alternative.

As most of the readers know, semi-infinite optimization deals with problemsof finitely many variables and infinitely many inequality constraints. There area number of motivating applications of semi-infinite programming (SIP) in awide variety of fields (see, for instance, Goberna and Lopez [10] and Hettichand Kortanek [11]). The treatment of such problems requires both theoreticaland numerical specific techniques (Reemtsen and Riickmann [25]).

Our main purpose is to review some connections between optimization under uncertainty and semi-infinite programming. Besides, we use our primalmethod for solving the semi-infinite linear programming problems that appear

OPTIMIZATION UNDER UNCERTAINTY AND LINEAR SIP 329

when such problems cannot be reduced to a finite one . The paper is organized asfollows. In Section 2 we introduce some notions and statements related to fuzzysets . In Section 3 we are concerned with the different optimization problemsthat model the uncertainty making use of the set-inclusion concept, whereas inSection 4 we focus on the problems that belongs to the class of fuzzy math ematical programming problems. In Section 5 we summarize our algorithmfor linear semi-infinite programming. Some numerical results in Section 6 willcomplete the paper.

2 FUZZY SETS

We will remind here only those notions and statements linked to the fuzzysets that we will use further on in the paper.

Let X denote the universal set. Zadeh [34] in his seminal work gave thefollowing definition: "A fuzzy set A in X is characterised by a membershipfunction /loA (x) which associates with each point in X a real number in the interval [0, 1], with the value of /loA (x) at x representing the "grade of membership"of x in A."

Definition 2.1 A fuzzy number A is a fuzzy set defined on the set of realnumbers IR characterized by means ofa membership function /loA (x). which isupper semi-continuous and fulfils the following conditions:(i) Normality. i.e. SUPxEJR/loA(X) = 1.(ii) Convexity, i.e. each of its a -cuts, So:(A) = {x EX: /loo:(x) ~ a},

is a convex set.

Note that each a -cut forms an interval [A~, A~], where A~ and A~ denotethe lower and upper limits of the a -level set So:(A), respectively. Figure 15.1shows the membership function of the fuzzy number A.

OI---L._--i, .L.-----l. ___

x

Figure J5.J The membership function of a triangular fuzzy number A .

In the paper we will use extended operations of the addition of two fuzzynumbers and ofthe multiplication ofa fuzzy number with a scalar. Let us remind


the definitions of those operations, which are a consequence of the extensionprinciple of Zadeh.

Definition 2.2 Let A, B fuzzy numbers and r E IR. Then:(i) A + B is a fuzzy number with the membership function J.LA+B(Z) =

sUPz=x+y min{J.LA(x), J.LB(Y)}, X,Y ,Z E IR.(ii) For r i= 0, r A is a fuzzy number with the membership function

J.LrA (z) = /LA (zlr), z E IR.(iii) Forr = 0, rA is zero, i.e. J.LrA(Z) = 1for Z = 0 and J.LrA(Z) = Ofor

Z i= o.

Now, let us briefly introduce some basic concepts related to decision makingin fuzzy environments . The next definition, due to Bellman and Zadeh (see[35]), is the definition of decision.

Definition 2.3 Thefuzzy objectivefunctionts) and the constraints are characterised by their membership function. Since we want to satisfy (optimise) theobjective function as well as the constraints, a decision in a fuzzy environmentis defined by analogy to nonfuzzy environments as the selection ofactivities thatsimultaneously satisfy objective functionts) and constraints.

According to the definition above and assuming that the constraints are 'noninteractive' , the logical "and" corresponds to the intersection. The 'decision'in a fuzzy environment can therefore be viewed as the intersection of fuzzyconstraints and fuzzy objective function(s) .

Remark 2.1 Due to the symmetric treatment given to objectives and constraints in Definition 2.3, it is clear that solving the multi-objective programmingproblem is as "easy" as solving a single objective problem.

Let us consider the next example in [35]:

Assume that our objective, finding a quantity a: that should be "substantiallylarger than 10", is characterized by the membership function:

J.LO(x) = { :1 + (z - 10)-2

if x ~ 10

if x > 10.

But we have a constraint: "x should be in the vicinity of 11", characterized bythe membership function:

1J.Ldx) = 1 + (x _ 11)4


The membership function, Il-D(X), of the decision is then

Il-D(X) = min{ll-o(x), Il-c(x)} ,

where

o if x :s 10

Il-D(X) =1

1 + (x - 10)-2if 10 :s x :s 11.75

ifx> 11.75.1

1 + (x - 11)4

Note that the interpretation of a decision as the intersection of fuzzy sets implies no positive compensation (trade-off) between the degrees of membershipof the fuzzy sets in question, if either the minimum or the product is used asoperator. On the opposite, the interpretation of a decision as the union of fuzzysets, using the max-operator, would lead to a full compensation of lower degreesof membership by the maximum degree of membership. So, neither the noncompensating "and", nor the fully compensatory "or", are always appropriateto model the aggregation of fuzzy sets representing managerial decisions.

Another approach for dealing with the accomplishment of the fuzzy constraints, in the context of mathematical programming, is to convert the comparison relation "2:/]=1 a'[jx :s bi', where aij and bi are fuzzy numbers, into a moretractable constraint, which establishes a ranking between two fuzzy numbers.The preference relation between fuzzy numbers based on their a-cuts can bedefined as follows:

Definition 2.4 Let A , B befuzzy numbers ana a E [0,1]. Then A :s B ifalia only if A~ :s Bh ana Ah :s B;:,Jorall hE [a, 1].

There is no universal ranking in fuzzy set theory. Depending on the comparison relation adopted, different fuzzy ranking methods can be introduced([14]).

3 CONVEX PROGRAMMING WITHSET-INCLUSIVE CONSTRAINTS

In order to model the situation in which the data are known to be containedin some given sets, described by functional relations , Soyster [27] introducedthe next mathematical programming problem with set-inclusive constraints:

(MPSIc) sup cT X

s. 1. X 1K1 + X2K2 + + x n K n ~ K

Xi ~ 0 i = 1,2, ,n,


where {Kj } are given nonempty convex sets in IRm, K is a closed convex set

in IRm , and + refers to addition of sets . Obviously, the feasible set is convex,and a vector x is feasible if and only if x1d 1+ X2d2 + ... + xndn E K for allpossible choices of di E Ki . That is a very conservative strategy to define afeasible solution (pessimistic approach) in opposite to the approach consideredin the generalized linear programming problem (Dantzig, [4]), wherein a vectorx is feasible if there exists a realization of the data, eli E Ki, i = 1,2, ... , n,such that Xldt + x2d; + ...+ xnd~ E K(b), u« ffim, where the resource setK takes on the form K = K(b) := {y E ffim : y:S b} .

3.1 INEXACT LINEAR PROGRAMMING

In [27] we can read that the term inexact (as suggested by K. O. Kortanek)applies to the situation in which the activity vectors for a linear program arenot known with certainty; all that is known is that the jth activity vector willbe a member of the convex set K]. Then problem (MPSIC) is a type of inexactlinear program. When the resource set K has the polyhedral structure K(b),it has been shown that problem (MPSIc) can be reduced to a finite linear optimization problem under certain conditions on the activity sets; as pointed out inTichatschke et al. [32], it is sufficient to assume that the sets {Ki} are bounded.Moreover, Tichatschke et al. [32] have studied the connections between generalized and inexact linear programming for the case K = K(b), bE ffim, bymeans of proper duality results, that extend the duality theorem for generalizedlinear programs, due to Thuente [31] . They give a classification of inexactoptimization problems that distinguishes between inexact problems with uncertainty in columns (Ib) :

(Ib) sup cT xs.t, x 1d1 + X2d2 + . .. + xndn :S b, Vdi E Ki,

Xi 2: 0, i = 1,2, ... , n ,

and or inexact problems with uncertainty in rows (Ia):

(Ia) sup cT x

s.t, afx:s bj, Vaj E Pj, j = 1,2"" ,m,Xi 2: 0, i = 1,2"" , n,

where K, C IRm, Pj C ffin are assumed to be given bounded sets. Also, forsome specializations of sets {Pj} an optimal solution for (Ia) can be obtainedvia an auxiliary linear programming problem. In particular, if the sets {Pj} areparallelepipeds ([31]) or polyhedral sets ([32]). When the activity sets {Pj}are bounded but they have not these special structures, the authors proposeto solve these inexact problems (Ia) by treating them as semi-infinite linearprogramming problems.


Inexact linear programming problems have been generalized in several waysby means of: (i) allowing uncertainty in the objective function, which is replacedby sup{inf cT x : c E C} where C ~ lRn is a closed convex set ([28], [6]);or (ii) using generalized resource sets K, for which the restriction "if Yl E Kand Y2 S tn, then Y2 E K" holds and the available resources are limited ([29]).Again, for some instances of these generalized programs the optimal solutionis obtained by solving a finite LP problem, but for the general case no finiterepresentation is apparently possible, and an alternative approach for obtaininga solution is to treat it by semi -infinite programming methods ([29], [32]).

3.2 FUZZY SET PROGRAMMING

In Negoita et al. [20] another class of (MPSIC) problem is introduced, forwhich K; and K are fuzzy subsets in R m and the binary operation + refers toaddition of fuzzy sets. This fuzzy mathematical programming problem involvesthe convolution of the n membership functions of the sets Ki, i = 1"" ,n.When these fuzzy sets are convex the authors show that the problem can bestated in terms of the a -cuts in the following form:

(FPsId maxs.t.

cTy

y1Sa(Kd + ... +YnSa(Kn) ~ Sa(K), a E [0,1],Yi 2: 0, i = 1, 2, .. . ,n,

where for each fixed a E [0, 1] an ordinary (MPSIC) problem is obtained, sinceeach level set is a convex set. Negoita et al. [20] suggested to select some gridof points on the interval [0,1] and solve the finite set-inclusive program that,under special assumptions concerning the range of the fuzzy subsets Ki, for1 SiS n, can be classified as an inexact programming problem.

Parks and Soyster [21] pointed out that the (FPslC) problem is a kind ofsemi-infinite set-inclusive program. They studied the relationship betweenfuzzy set programming and a special class of semi-infinite programming: concave/convex semi-infinite programs, abbreviated as C/CSIP, formulated in thefollowing way:

(C/CSIP) max cT xs.t, a(s)Tx S b(s), s E S

Xi 2: 0, i = 1,2 , .. . ,n,where S is a compact convex set. Here only one inequality, indexed by a continuous parameter, defines the constraints, the components of a(s) are strictlyconcave functions and b(s) is a convex function. This problem is characterizedby the fact that only one point exists where the constraint inequality is bindingat a non-zero optimum. Based on this property they proposed some alternativestrategies for solving it. In particular, if S is a subset of the real line they used


the solution of the corresponding dual problem and a sequence of grids overthe primal one to find an approximate solution.

The next example, suggested by Parks and Soyster, is a C/CSIP problem inS = [0,1] . But it is also an inexact problem with uncertainty in rows (Ia), wherethe activity set {P} is the closed curve defined by a(s) := (-(s + 1)2, - (s 2)2? for s E [0,1]' which has no finite representation:

Example 3.1 Let us consider the problem

min ~Xl +X2s.t, -(s + 1)2x l - (s - 2)2x2 ::; -1 for all s E [0,1]

Xl ~ 0, X2 ~ O.

This example, and also the other semi-infinite programming problems introduced throughout the paper, will be solved in Section 6.

The solution of a C/CSIP problem using the scheme suggested in [21] depends on the fact that only one inequality constraint is allowed. On the otherhand, if b(s) is a constant coefficient these problems can be considered as inexact.

Notice that the functional relations that define the components of a(s) andthe function b(s) come from the properties of the convex fuzzy sets K; andtheir corresponding membership functions. Then, an extension of the C/CSIPproblem can be considered in such a way that more than one inequality isallowed:

max cTxs.t. aj(s)Tx::; bj(s) for j = 1"" ,m

Xi > 0, i = 1, ... ,n,the components of each aj (s) being concave functions, and bj(s) a convexfunction. This extended C/CSIP is a generic semi-infinite linear programmingproblem and it can be solved, for instance, using the algorithmic scheme summarized in Section 5. The next example neither is a C/CSIP nor an inexactproblem.

Example 3.2 Let us consider the problem

max 3Xl + 2.5x2s.t. (3.5 - ~ s2)Xl + (6 - 2s2)X2 ::; 180 - 30s, for all s E [0,1]

(10 - s2)Xl + (7.5 - ~ s2)X2 ::; 320 - 20s, for all s E [0,1]Xl ~ 0, X2 ~ O.

Next we see that another class of fuzzy set programs can be also modelledas semi-infinite problems by means of the set-inclusion concept. From thereasoning above we have that depending on the type of the chosen membership

functions for Ki, i = 1, '" n, and K, the resulting optimization problem turnsout to be a linear semi-infinite program or not.

Now, let us consider an (FPSIc) problem and assume that the fuzzy subsetsKi, i = 1"" n, and K are fuzzy numbers. Let us consider the case that themembership function of each fuzzy subset is defined as follows:

{0 si x < 0

/-LK;(X) = hi(X) si x 2: 0,

where hi(X) is a strictly quasi-concave and non-increasing function over atolerance interval [0, ti], ti 2: 0 and hi(O) = 1, for i = 1"" ,n. Then,it is easy to show that the corresponding a-level sets can be expressed asSa(Ki) = [0, Kf(a)], for all a E (0,1], where Kf(a) = hi1(a) if hi isstrictly decreasing.

In this case, from the theory of convex analysis we have that, since themembership function of a fuzzy number is an upper semi -continuous andquasi-concave function, the condition above over /-LK; (x) excludes those quasiconcave functions having local maxima that are not global ones. That is, it doesnot allow us to associate the same degree of satisfaction to all the points in asubinterval [a , b] ~ 1R+, except if the lower point of the subinterval is zero.

Figure 15.2 illustrates different shapes of the membership functions that canbe associated to the fuzzy numbers K, or K.

xAUO'-------_--:::""'":-::~-xAU

a

o'-- --L__

Figure J5.2. Strictly quasi -concave and non-increasing membership functions.

Because of the non-negativity of the decision variables Yi, we have that thefeasible set of the problem (FPSIc) can be stated by means of the following setof infinite inequalities:

y1Kf(a) + Y2 K!j(a) + ...+ YnK~(a) :s; KU(a) , for all a E [0,1],

that corresponds to the pessimistic approach of the definition of feasibility.

In order to show a complete description of our proposal we introduce thefollowing example:

Example 3.3 Let us consider the fuzzy set programming problem

max Yl +Y2s.t, YIKI + Y2K2 ~ K

Yl ~ 0, Y2 ~ 0,

where the membership functions of K, , for i = 1,2, and K, are specified asfollows:

and

() {x2 - 2.5x + 1

/l-K\ X = 0

() {2VO.25 - x

/l-K2 X = 0

o~ x ~ 0.5otherwise,

o~ x ~ 0.25otherwise,

O~x<1

l~x~3

otherwise.

Theircorresponding a-cuts areSo(Kd ~ [0, ~ - V196 + a l So(K,) =

[0, 1~a']. and So(K) ~ [0,3 - 2aJ, aE [0, 1J.

Then, the maximizing solution y* of this example is given by solving thefollowing semi-infinite programming problem:

max Yl + Y2

s.t. (i- J"-ltr-+-o ) u, + ¥ Y2 ~ 3 - 20, 0 E [0,1],

u: ~ 0, Y2 ~ O.

In fuzzy programming literature other set-inclusive constrained models related to possibilistic programming appear ([14]) . They are linear programmingproblems in which the constraints coefficients may be imprecise with possibilistic distributions, that is the inexactness is modelled by means of fuzzyparameters. The posibilistic measure of an event might be interpreted as thepossibility degree of its ocurrence under a possibility distribution. Then possibilistic LP problems are:

(PLP) max cT xs.t. ailXl + ai2X2 + ' " + ainXn ~ bi ' i = 1,2",' ,m,

x ~O,

where A and b are restricted by a possibility distribution and the objectivefunction is deterministic. Using fuzzy numbers to represent the continuous


possibility distributions for fuzzy parameters allows us to state and solve (PLP)problems as a set-inclusive problem.

When all aij, bj are trapezoid fuzzy numbers, Ramik and Rimanek [23] usethe tetrad expression a = (m, n, a , (3), where m is the left main value and nis the right main value with complete membership; a is the left spread and{3 is the right spread, in order to establish a ranking preference between twotrapezoid numbers. In particular, for a = (m ,n, a, (3) and b = (p, q, T, 8), theyestablished that

a :s b~ {m:S p, m - a:S P - T , n:S q, n + {3 :s q + 8}.

Then, by means of the set-inclusion concept they obtained an equivalentfinite LP problem, whose optimal solution can be considered as the solution ofthe possibilistic linear programming problem.

4 FUZZY MATHEMATICAL PROGRAMMING

Through the paper we have been using several concepts of fuzzy logic, andwe have also talked about some kind of fuzzy programming problems (relatedto set inclusive constraints). In this section we briefly summarise some aspectsof fuzzy mathematical programming. We think it would be interesting for thosereaders non -familiar with the subject.

Let us consider the fuzzy formulation of the linear programming problem:

(FLP) max cT xs.t. Ax ;S b,

x ~ 0,

where C = (Cl ,' " ,cn ) E IRn, A = [ad E Mn x m and b = (bl , '" ,bm ) E

IRm , aij, Cj,bj are fuzzy numbers, ;S is a fuzzy inequality and x is a vector ofdecision variables. For x E IRn , the degree of satisfaction of the i-th fuzzyconstraint (Ax)i;S bi is established by a membership function.

Following the principle introduced in Definition 2.3, as strict optimizationand strict constraints are replaced by aspiration levels, we get a solution whichis "a good enough compromise" in terms of the aspiration levels given to theobjective and the constraints. In order to express such compromise the membership functions should be made explicit and properly defined. As each realnumber can be modelled as a fuzzy number the general system includes thecases in which the objective function is strict and some or all the constraints arestrict.

In the fuzzy LP formulation the most usual way to construct membershipfunctions is to use a linear form ([35]), which allows the model to be converted


into a traditional LP problem. When the implicit assumption that the degrees ofsatisfaction assigned to the objective and the constraints are equally importantis not realistic , there are some extensions to the formulations of the fuzzy LPmodels ([5]).

Now, let us summarise a method due to Carlsson and Korhonen ([3]) to solvea class of fuzzy linear programs in which the parameters of the model are knownwith only some degree of precision. The authors show that the model can beparametrized in such a way that the optimal solution becomes a function of thegrades of precision and illustrate their method with the next example:

max [1, 1.5)XI + [1, 3)X2 + [2, 2.2)X3s.t. [3,2)XI + [2,0)X2 + [3,1.5)X3 ~ [18,22),

[1,0.5)XI + [2, 1)x2 + [1, 0)X3 ~ [10,40),[9, 6)XI + [20, 18)X2 + [7,3)X3 ~ [96,110),[7, 6.5)XI + [20,15)x2 + [9,8)X3 :s; [96,110),Xl ~ 0,X2 ~ 0,X3 ~ O.

It is assumed that the user can specify the intervals [ao, ad for the possiblevalues of the parameters, where the first number represents "risk-free" valuesand the second number the "impossible" values. When moving from risk-freetowards impossible parameter values, we move from secure to "optimistic"solutions.

In this context. the authors suggest to use monotonically decreasing membership functions of the parameters Cj and bi and increasing ones for aij. Infact, they use the next membership functions, associated with each type ofparameters:

(a - a}j)!Laij(a) = (a9. -a~.)' i = 1,2,3,4, j = 1,2,3;

IJ IJ

1 - exp{-0.8(b - b})/(b? - b})}!Lbi(b) = (l-exp{-0.8}) , i=1,2,3,4j

1 - exp{3(c - c~ )/(cQ - c~)}

!LCj (c) = (1 _ eX;{3})J J, j = 1,2,3.

The precision of the optimal solution is evaluated through its membershipfunction, which is defined by

!La = min{ !Laij' !Lbi' !Lci' i = 1,2,3,4; j = 1,2,3},

which means that the best value of the objective, at a level of precision, can befound by using the parameter values at the same level of precision. Since theparameters can be presented as functions of the membership functions :


a.J· = a~ . + "a " (a9. - aL)• ~J'- 'J ~J ~J '

bi = b} + (In(l - Pbi (1 - exp{-0.8}) )(b? - b})) /0 .8,

Cj = cJ - (In(l - P Cj (1 - exp{3} ))(cJ - cJ)) /3,

and using that Paij = Pbi = PCj = P, it follows that the original fuzzy LP isequivalent to the next parametric programming problem, for P E [0,1] :

max cr (J-t) xi + C2(J-t) X2 + C3(J-t) X3

s.t. (2 + J-t)Xl + (2J-t)X2 + (1.5 + 1.5J-t)X3 :5 22 + 51n(I - 0.55J-t) ,(0.5 + 0.5J-t)Xl + (1 + J-t)X2 + J-tX3 :5 40 + 37.51n(I - 0.55J-t),(6 + 3J-t)Xl + (18 + 2J-t)X2 + (3 + 41-£)X3 :5 110 + I7.51n(I - 0.55J-t) ,(6.5 + 0.5J-t)Xl + (15 + 5J-t)X2 + (8 + J-t)X3 :5 110 + I7.51n(I - 0.55J-t) ,Xi ;::: 0, i = 1,2,3,

where Cl(p) = 1.5 - 0.167In(1 + 19.1p), C2(p) = 3 - 0.6671n(1 + 19.1p)and C3(p) = 2.2 - 0.067In(1 + 19.1p).

The authors proposed to carry out a series of experiments with various membership values P = 0,0.1"" , 1. Then, the decision-maker can get his optimalsolution under a pre-determined imprecision allowable. The objective functionhas value 30.25 if 'impossible' parameter values are used (p = 0), and value12.0 if 'risk-free' ones are used (p = 1).

Because of the relationship among the different grades of membership, itis easy to show that the feasible set of the problem cannot be modelled as thefeasible set of a semi -infinite programming problem. For instance, for the firstconstraint: if P = 0, we have that 2Xl + OX2 + 1.5x3 ~ 22, if P = 1, we havethat 3Xl + 2X2 + 3X3 ~ 18, and the second inequality implies the former.

It is known that if, in the formulation of (FLP) problem, each aij , bj is afuzzy number and the objective function is deterministic, then it can be statedand solved by means of specific ranking methods ([14]). Tanaka et at. [30]established a new formulation of the problem based on the ranking preferencegiven in Definition 2.4:

(FLP)o min cT xn

s.t . L aijXj ~o bi, i = 1, ... ,q,i=lXj ~ 0, j = 1, ... ,n.

When each fuzzy coefficient is a symmetric triangular fuzzy number theyshow that, for a given a level, a E [0, 1], which represents the degree ofoptimism of the decision maker, the problem is equivalent to an auxiliary finiteLP problem.


For the general case, i.e, for non-specific fuzzy numbers, Fang et al. [7]show that the (FLP)o problem with fuzzy coefficients can be converted into alinear semi-infinite program:

(LSIP) min cT Xn

s.t, L)aij)~Xj 2:: (bd~ 8 E [a, 1], i = 1,'" ,q,j=l

n

2)aij)~Xj 2:: (bi)~ 8 E [a,l], i = 1,," ,q,j=l

Xj 2:: 0, j = 1, ... ,n,

where (aij)~ and (aij) ~ denote the lower and upper limits of the level setsSo(aij) for 8 E [a, 1], analogously for bi. They propose a cutting plane algorithm for solving the (LSIP), and study the relations between extreme pointsand optimal solutions of the semi-infinite problem. They illustrate their methodwith the following fuzzy linear programming problem:

Example 4.1

min -Xl - 2X2 - 2X3s.t, [-2,2, 2]Xl + [-1,1, 1]x2 2::0 [-8,1,3)'

[-1, 1, I]X3 2::0 [-10,1,2] ,Xl 2:: 0, X2 2:: 0, X3 2:: 0,

where a E [0,1] and [y, f3, X] are triangular fuzzy numbers, where y is thecentral value, f3 is the left spread and X is the right spread. Notice that the fuzzynumbers corresponding to the RHS are non-symmetric. Its equivalent (LSIP)formulation for a given a-level is:

mm -Xl - 2X2 - 2X3s.t. (281 - 4)Xl + (81 - 2)X2 2:: 81 - 9, 81 E [a, 1],

(82 - 2)X3 2:: 82 - 11, 82 E [a, 1],-283Xl - 83X2 2:: -383 - 5, 83 E [a, 1],-84X3 2:: -284 - 8, 84 E [a, 1],Xl 2:: 0, X2 2:: 0, X3 2:: 0.

This problem neither is of the inexact type nor belongs to the class ofextendedC/CSIP problems. In fact, it is easy to show that this ranking relation does notimply a set-inclusive constraint approach.

Hu and Fang [12] study systems of infinitely many fuzzy inequalities withconcave membership functions that can be reduced to a SIP problem by meansof the tolerance approach and propose a cutting plane algorithm for solvingthose fuzzy mathematical programming problems.


5 LINEAR SEMI-INFINITE PROGRAMMING

Many algorithms for solving semi-infinite linear programming problemscould be used to solve the instances appearing in the paper. Reemtsen andGomer [24] gave a comprehensive survey of numerical methods for semiinfinite programming. In particular, for the linear case they described dualand primal exchange, interior-point and further methods . They analize thesetype of methods also comparing their efficiency and convergence results .

It can be shown that all the instances appearing in the paper attain theirrespective optima at extreme points. We propose to use our extension of thesimplex algorithm to the infinite-dimensional case to solve them. Our method,that we call Algorithm 5.1 ([18]), approaches the optimal solution through asequence of basic feasible solutions. Let us devote this section to summarisethe main ideas of the method.

We are concerned with the problem:

(LSIP) min.s.t.

cTx

fi(S)Tx 2: bi(S), S E Si , i E q := {1,2, ...,q},i, ~ Xj ~ Uj, j = 1,2, ...n,

where /i(s) := (fi1(s), /i2(S)" , '/in(s))T E lRn , for s E Si, and its components fij(S) and the functions bi(s) are n-times continuously differentiableon Si = [ai, .Bi], for i = 1"" , q. Any or all of the lower (respectively upper)bounds may be -00(+00).

Our procedure is simplex-type and based on the concept of extreme point.We would like to point out that until the appearance of the papers by Nash[19] and Anderson [2], in which the definition of extreme point was consideredfrom an algebraic viewpoint, all the simplex-like algorithms dealt with the dualof (LSIP) problem (see [8], [9] and [26]). A complete scheme of a simplextype procedure for the primal problem can be found in Anderson and Lewis[1], where the advantages of extreme point solution methods have also beendiscussed.

In constrast to finite linear programming, degeneracy is a frequently occurring phenomenon at (LSIP) problems. Our procedure is prepared to avoid thepractical disadvantages of using the schemes suggested in Anderson and Lewis[2] for this kind of extreme points. In fact, in the presence of degeneracy thesolution of a finite LP problem (see Theorem 5.4) allows us either to determinefeasible descent directions or to recognize an optimal solution for the (LSIP)problem.

In order to put the (LSIP) problem in standard form, we must introduceseveral slack-functions:


Zi(X, S) := fi(sfx - bi(S), i E q, s E Si, X E IRn.

We assume that the (LSIP) problem is consistent. Let (x, z) be a feasiblesolution of (LSIP), where Z = (ZI' Z2,'" ,Zq). Then Zi(X, s) ~ 0, s E Si. IfJl(X) = {j E {1,2,'" ,n} : Ij = Xj} and Ju(x) = {j E {1,2, '" ,n} :Xj = Uj}, then the set of active constraints is:

q

constr(x) = U{ s E S, : Zi(X, s) = O} U Jl(X) U Ju(x) .i=1

We assume that constr(x) has finite cardinality for all x E IRn and denoteU{=I{S E S, : Zi(X,S) = O} = {SI,'" ,sm}, the cardinals of Jl(X) andJu(X) being znj and rn., respectively. Ifsk E Zi(X) = {s E Si: Zi(X,S) = O},then d(Sk) denotes the degree of Sk as a zero of the slack function Zi(X, s).Furthermore, we define

t:= m + ml + mu + d(st} + ... + d(sm).

For the sake of simplicity, throughout the paper, we shall assume that Si = S,for all i.

The first part of Theorem 5.1 is a very useful characterization of extremepoint and the second one is a characterization of nondegenerate extreme point.

Theorem 5.1 Let (x,z) be a feasible point for (LSIP). Then (x,z) is anextreme point if and only if V (x) = {On}, where

V(x) =

n{y E IRn : fi(k) (spfy = 0, k = 0,··· ,d(sp) for each sp E Zi(X)}iEq

n{y E IRn : (ej)T y = 0 for j E J1(x) U Ju(x)}.

Moreover, (x, z) is a non-degenerate extreme point if and only if B(x) isinvertible. where B(x) is the (n x t)-matrix

B(x) = [II (si), ..., II(s.}, ..., fq(sm), ... , -ei, ... ,ek, ..., fF) (st}, ...,

f (d(sd )(s ) .. . f(d(sr))(s) ,(I)(S) f(d(Sm))(s)]1 1" 1 r , ... , Jq m t s •••, q m

The following result is an extension of a characterization of optimality fornon-degenerate extreme points in [2]. It is based on the existence of the inverseof the matrix B(x).


Theorem 5.2 A non-degenerate extreme point (z, z) is optimal ifand onlyif A ~ Om+m/+mu and p = On-(m+m/+mu ) • where (.x, p)T = B(x)-1C.

The optimality condition in Theorem 5.2 is not applicable for degenerateextreme points. In this case we use the next result, which is a Karush-KuhnTucker-type theorem, in order to establish a more general optimality check ([15]) useful for feasible solutions with some active constraints that satisfy aregularity condition.

Theorem 5.3 Let (x , z) be an optimal solution for (LSIP) and supposethat the Slater condition holds. Then there exist t1,'" .t-« E UiEqZi(X).iv .... .ik E Ji (x) U Ju(x) and non-negative numbers A1' ... , Am, //1, •••//k suchthat

r m

C = L Ai!I(td + ... + L Adq(td + L //jei + L //j( -ei).i=1 i=m-p jEJ/(x) jEJu(x)

OurAlgorithm 5.1 combines a simplex-type strategy with a feasible-directionscheme. The pivoting rules are based on the geometrical meaning of the simplexmethod for Linear Programming, they use the inverse of the matrix B(x), andtherefore they can only be applied for non-degenerate extreme points. Sometimes, a purification phase is necessary to proceed from a feasible solution to animproved extreme point. And finally, the descent rules may be used for everyfeasible solution, in particular for degenerate extreme points. Therefore, theyare more general than the pivoting ones. We will not describe exhaustivelyneither the pivoting rules nor the purification algorithm because the details canbe found in [16] and [18].

Our purification algorithm is well defined under the following rank assumption (RA), which is compatible with the eventual unboundedness of the (LSIP)problem:

Assumption 5.1 There exists at least oneindex i E qsuchthatrank{Ji(s) :s E S} = n .

Starting from a feasible solution, the purification algorithm generates a newfeasible point whose slack functions maintain all the zeros of the previousiterate . Moreover, it adds a new zero or increases the multiplicity of an existingone. The objective function value at the successive iterates does not get worse.The dimension of the subspace V(x), in Theorem 5.1, is iteratively reduced,and then either an extreme point is obtained in a finite number of iterations orthe algorithm concludes with the unboundedness of the problem.

Let us describe how our method obtains the descent directions to move from adegenerate extreme point. In order to avoid thejamming phenomena that appear

in the computation of the search directions in the feasible-direction methods (forinstance, [22]) it is necessary to introduce an assumption about the cardinalityof the local minima of Zi(X, .). It ensures that (LSIP) is tractable, and permitsus to take into account the gradients of these local minima in the linear problemthat reports on either the optimality of the current iterate or yields the searchdirection.

Assumption 5.2 For each x E lRn, the set of local minimizers mi(x) of

Zi(X, s) over S is finite.for i E q.

Theorem 5.4 Let (x, z) be a feasible point for (LSIP) and suppose thatthe Slater condition as well a~ Assumptions 5.1 and 5.2 hold. Then (x, z) isoptimal ifand only ifv(Q(x)) = 0, where v( .) is the optimal value ofthe linearproblem:

(Q(x)) min cT ds.t. fi(S)T d ~ -Zi(X, s), s E mi(x), i = 1, ... , q,°~ dj, for j E JI(X),

dj ~ 0, for j E Ju(x),-1 ~ dj ~ 1, j = 1, .. . , n.

It is important to note that ifthe current iterate (x, z) is a degenerate extremepoint, we use Theorem 5.4 both for checking its optimality and for improvingthe objective value. If v(Q(x)) < 0, then every optimal solution d* of theproblem is a descent direction at (x, z). However, in many cases the maximumstep-length from x along d* is zero. We will avoid this difficulty by constructinga new descent direction g such that:

Ji(s)Tg > -Zi(X,S), s E mi(x), i E q,gj > 0, for j E JI(X) and gj < 0, for j E Ju(x).

Solving a second linear program the method obtains g*, which is added tod* to construct the feasible descent direction d. In [16] it is proved that theexistence of such a direction 9 is ensured by the Slater condition. There wepresent a complete discussion of this subject for a less general but analogousproblem .

Let us recall the basic structure of the

Algorithm 5.1

Step 0 Let (x, z)be afeasible pointfor (LSIP). Find the set ofglobal minimizersofZi(X, s) on S.

Step 1 IfV (x) i- {o} apply a purification algorithm in order to determine abasic feasible solution (x+, z+) .


Step 2 Finding search directions step.

• If the matrix B (x+) is invertible. evaluate>. and p. If >. > 0 andp = O. STOP. Otherwise. find d through the suitable pivoting rule.and go to step 3.

• If the matrix B(x+) is non-invertible. determine the set of localminimizersmi(x+) , i E q. Computetheoptimalvaluev(Q(x+)) .Ifv(Q(x+)) = O. STOP. Otherwise. compute the search directiondE Argmin Q(x+). and go to step 3.

Step 3 Evaluate the maximum step-length Jl(x+) . Set x = x+ + Jl(x+)d. andgo to step 1.

The convergence of this algorithm has not been proved. But, it is known thatthis primal method generates points which are feasible and not have stabilityproblems. It requires to apply accuracy feasibility checks and compute maximalstep-sizes, but some very ill-conditioned tangent problems have been succesfully solved by using these techniques ([18]). For more details concerning tocomparisons with other methods see [17] and [18].

6 NUMERICAL RESULTS

In this section we use Algorithm 5.1 for solving the (LSIP) instances proposedthroughout the paper. Cases 1 and 2 refer to different pivoting rules for nondegenerate extreme points.

Example 6.1 (Example 3.1 revisited) Starting from the non-degenerateextreme point xl = (1,0) with Z\ (Xl) = {O}, d(O) = 0 and J/(xl) ={2}, it applies the pivoting rule (Case 1) and obtain the extreme point x2 =(0.333333, 0.166667),withZI (x2

) = {O},d(O) = 1. The direction (-7/24, 5/48),given by the pivoting rule (Case 2), leads to x3 = (0.266667,0.190476) withZI(X3 ) = {0.25}, d(0.25) = 1, a non-degenerate extreme point. Applying thepivoting rule (Case 2), it finds the optimal solution x* = (0.268245,0.189679),ZI(X*) = {0.2426}.

Example 6.2 (Example 3.2 revisited) Taking the starting solution xl =(0,0), with J/(xl) = {1,2}, the pivoting rule (Case 1) leads to x2 =(31.684387,0), a degenerate extreme point, with rnj (x2) = {0.9468} , Z2(X2) ={0.3156}, d(0.3156) = 1 and J/(x2) = {2}. The problem (Q(x2)) gives thedirection d = (-0.752515,1) but checks that the maximum step-length along dis zero, then it finds 9 = (-0.813129,1) and the algorithm moves along d+g tox3 = (16.225799, 19.747258), an improved extreme point. In fact, it finds theoptimal solution x* = (17.293524, 19.112040) in 14 iterations. The vector x*is a degenerate extreme point with Z\ (x*) = {0.32} and Z2(X*) = {0.3724}.


Example6.3 (Example 3.3 revisited) The starting point y l = (O,O),JI(yl) ={1,2}, is a basic feasible solution. The pivoting rule (Case 1) yields a nondegenerate extreme point: Y2 = (6,0), with Zl (y2) = {O}, d(O) = 0 andJI(y2) = {2}, it also provides the direction (-0.5,1) for finding the nondegenerate extreme point y3 = (3,6), with Zl (y3) = 0, d(O) = 1. Applying twice the pivoting rule (Case 2) it reaches the feasible solution y5 =(0,10.45244561), with JI(y5) = {1}. The direction (0,1), given by the purification algorithm, leads to y6 = (0,10.47213596), with Zl (y6) = {0.381966},d(0.381966) = 1 and JI(y6) = {1}, a degenerate extreme point. Sincev(Q(y6)) = 0, the algorithm stops at y6.

Example 6.4 (Example 4.1 revisited) Lets us consider Q = 0.6 (as suggested in Fang et al. [7]) and an arbitrary feasible starting point, say xl =(0,0,0), with Zi(X 1) = 0, for all i and JI(X 1) = {1, 2, 3}. xl is a nondegenerate basic feasible solution, applying twice the pivoting rule (Case 1)it reaches the non-degenerate extreme points x2 = (0,6,0), with Zl (x2) ={0.6}, d(0.6) = 0 and JI(X2) = {1,3}, and x3 = (0,6,7.428571), withZl(X3) = Z3(X3) = {0.6} , d(0.6) = 0 and JI(X3) = {1}, successively. Itchecks the optimality by means of Theorem 5.2 and returns the optimal solutionx· = x3 .

In summary, it is shown that some interesting mathematical programmingmodels under uncertainty can be reduced to linear semi-infinite optimizationproblems, and hence solved using this connection. This is another reason todedicate more effort to develop computationally efficient numerical methodsfor linear semi-infinite programs.

Acknowledgment

This work has been partially supported by the Ministerio de Educacion yCultura of Spain, Grant TIC98-1019.

References

[1] E.J. Anderson . A new primal algorithm for semi-infinite linear programming. In E. J. Anderson and A. B. Philpott, editors, Infinite Programming,pages 108-122. Springer Verlag, 1985.

[2] E. J. Anderson and A. Lewis. An extension of the simplex algorithm forsemi-infinite linear programming, Mathematical Programming, A44: 247269, 1989.

[3] C. H. Carlsson and P. Korhonen. A parametric approach to fuzzy linearprogramming, Fuzzy Sets and Systems, 20: 17-30, 1986.


[4] G. B. Dantzig. Linear Programming and Extensions, Princeton UniversityPress, 1963.

[5] D. Dubois and H. Prade. Fuzzy Sets and Systems: Theory and Applications,Academic Press, 1980.

[6] J.E. Falk . Exact solutions of inexact linear programs, Operations Research,24:783-787,1976.

[7] S. C. Fang, C. F Hu, H. F. Wang and S. Y. Wu. Linear Programmingwith fuzzy coefficients in constraints, Computers and Mathematics withApplications, 37: 63-76, 1999.

[8] K. Glashoff and S. A. Gustafson. Linear Optimization and Approximation.Springer-Verlag, 1983.

[9] M. A. Gobema and V. Jomet. Geometric fundamentals of the simplexmethod in semi-infinite programming, OR Spektrum, 10: 145-152, 1988.

[10] M. A. Gobema and M. A. Lopez . Linear Semi-Infinite Optimization.Wiley, 1998.

[11] R. Hettich and K. O. Kortanek. Semi-infinite programming: theory, methods and applications, SIAM Review. 35: 380-429, 1993.

[12] C. F. Hu and S. C. Fang. A relaxed cutting plane algorithm for solvingfuzzy inequality systems, Optimization. 45: 86-106, 1999.

[13] P. Kall and S. W. Wallace. Stochastic Programming, Wiley, 1994.

[14] Y.J. Lai and Ch. L. Hwang. Fuzzy Mathematical Programming. Methodsand Applications, Springer-Verlag, 1992.

[15] T. Leon and E. Vercher. An optimality test for semi-infinite linear programming, Optimization. 26: 51-60, 1992.

[16] T. Leon and E. Vercher. New descent rules for solving the linear semiinfinite programming problem, Operations Research Letters. 15: 105-114,1994.

[17] T. Leon, S. Sanmatfas and E. Vercher. Un metodo primal de optirnizacionsemi-infinita para la aproximacion uniforme de funciones, Qiiestiio, 22:313-335, 1998.

[18] T. Leon, S. Sanmatfas and E. Vercher. On the numerical treatment oflinearly constrained semi-infinite optimization problems, European JournalojOperational Research. 121: 78-91,2000.

[19] P.Nash. Algebraic fundamentals of linear programming. In E.1. Andersonand A. B. Philpott, editors, Infinite Programming, pages 37-52, SpringerVerlag, 1985.

[20] C. V. Negoita, P. Flondor and M. Suiaria. On fuzzy environment in optimization problems, Economic Computer and Economic Cybernetic Studiesand Researches. 2: 13-24, 1977.


[21] M. L. Parks and A. L. Soyster. Semi-infinite and fuzzy set programming.In A. V. Fiacco and K.O. Kortanek, editors, Semi-Infinite Programming andApplications, pages 219-235, Springer-Verlag, 1983.

[22] E. Polak and L. He. Unified steerable phase I-phase II of feasible directions for semi-infinite optimization, Journal of Optimization Theory andApplications, 69: 83-107, 1991.

[23] J. Ramik and J. Rimanek. Inequality relation between fuzzy numbers andits use in fuzzy optimization, Fuzzy Sets and Systems, 16: 123-138, 1985.

[24] R. Reemtsen and S. Gomer. Numerical methods for semi -infinite programming: A survey. In [25], pages 195-275, Kluwer Academic, 1998.

[25] R. Reemtsen and J. J Riickmann (editors). Semi-Infinite Programming,Kluwer, 1998.

[26] R. Roleff. A stable multiple exchange algorithm for linear SIP. In R. Hettich, editor, Semi-Infinite Programming, pages 83-96, Springer-Verlag, 1979.

[27] A. L. Soyster. Convex programming with set-inclusive constraints andapplications to inexact linear programming, Operations Research, 21: 11541157, 1973.

[28] A. L. Soyster. A Duality theory for convex programming with set-inclusiveconstraints, Operations Research, 22: 892-898, 1974.

[29] A. L. Soyster. Inexact linear programming with generalized resource sets ,European Journal of Operational Research, 3: 316-321,1979.

[30] H. Tanaka, H. Ichihashi and K. Asai. A formulation of fuzzy linear programming problems based on comparison of fuzzy numbers, Control andCybernetics, 13: 186-194, 1984.

[31] D. J. Thuente. Duality theory for generalized linear programs with computational methods , Operations Research, 28: 1005-10 11, 1980.

[32] R. Tichatschke, R. Hettich and G. Still. Connections between generalized,inexact and semi-infinite linear programming, ZOR-Method'i and Models ofOperations Research, 33: 367-382, 1989.

[33] R. 1. B. Wets and W. T. Ziemba (editors). Stochastic Programming. Stateofthe Art, 1998, Annals of Operations Research 85, 1999.

[34] L. A. Zadeh. Fuzzy sets, Information and Control, 8: 338-353, 1965.

[35] H. J. Zimmermann. Fuzzy set theory and its applications (3rd ed.), Kluwer,1996.

Chapter 16

SEMI-INFINITE ASSIGNMENTAND TRANSPORTATION GAMES

Joaquin Sanchez-Soriano', Natividad Llorca',Stef Tijs2 and Judith Timmer-1Department ofStatistics and Applied Mathematics, Miguel Hernandez; University, Elche Cam

pus, La Galia Building. Avda. del Ferrocarril, sin. 03202 Elche, Spain

2CeflfER and Department ofEconometrics and Operations Research , Tilburg University, P.O.

Box 90153, 5000 LE Tilburg, The Netherlands

[email protected]. [email protected]. [email protected]. [email protected]

Abstract Games corresponding to semi-infinite transportation and related assignment situations are studied. In a semi-infinite transportation situation, one aims at maximizing the profit from the transportation of a certain good from a finite numberof suppliers to an infinite number of demanders. An assignment situation is aspecial kind of transportation situation where the supplies and demands for thegood all equal one unit. It is shown that the special structure of these situationsimplies that the underlying infinite programs have no duality gap and that thecore of the corresponding game is nonempty.

1 INTRODUCTION

In 1972 Shapley and Shubik ([14]) introduced (finite) assignment games.These are games corresponding to an assignment situation where a (finite) setofagents has to be matched to another set of agents in such a way that the revenueobtained from these matchings is as large as possible. Since this introductiondifferent generalizations related to these games have been developed. Thepaper of Llorca, Tijs and Timmer ([8]) provides an infinite extension of thesegames. They introduce semi-infinite bounded assignment games in which oneset of agents is finite and the other is countably infinite and prove that thesegames have a nonempty core. That is, there exists an allocation of the maximalprofit over all the players such that any coalition of players cannot do betteron its own. Sanchez-Soriano, Lopez and Garda-Jurado ([12]) introduce finite

349

M.A. Goberna and M.A. Lopez (eds.), Semi-Infinite Programming. 349-363.© 2001 Kluwer Academic Publishers.


transportation games, which are based on transportation situations. Given a setof supply and demand points of a certain good, how much should be transportedfrom each supply point to each demand point to maximize the revenue from thistransportation plan? The arising transportation games can be seen as a finiteextension of the finite assignment games. Fragnelli et al. ([3]) and Llorca etal. ([8]) study games with a nonempty core arising from semi-infinite linearprogramming situations, where one of the factors involved is countably infinite,but the number of players is finite.

In this paper, we look at semi-infinite transportation problems where thenumber of suppliers is finite and the number of demanders is countably infinite.For each semi-infinite transportation situation we define a related assignmentproblem. With the help of the results in Llorca et al. [8], we show that semiinfinite transportation problems have no duality gap and the correspondingsemi-infinite transportation games have a nonempty core.

This work is organized in five sections. In the next section, we present finitetransportation and assignment games. Section 3 summarizes the main resultsand concepts for semi-infinite bounded assignment games that are needed tostudy semi-infinite transportation situations. In Section 4 we study transportation games arising from semi-infinite transportation problems in which thereis a countably infinite number of players of one type, the matrix of benefitsper unit is bounded and supplies and demands are positive integers. We showthat the corresponding primal and dual programs have no duality gap and provethat the games have a nonempty core . The proofs are based on an expansioncontraction procedure, which uses a semi-infinite assignment problem associated to the corresponding transportation problem generated by splitting thesupply and demand points. In the final section, we add a remark about the ideaof dropping the conditions that force the supplies and the demands to be naturalnumbers, in order to consider the transport of infinitely divisible goods.

2 FINITE TRANSPORTATION AND ASSIGNMENTGAMES

A (finite) transportation problem describes a situation in which demands atseveral locations for a certain good need to be covered by supplies from otherlocations. The transportation of one unit of the good from a supply point to ademand point generates a certain profit. The goal of the cooperating suppliersand demanders is to maximize the total profit from transport. For an exampleone may consider a large supermarket that has to supply its stores at variouslocations with bottles of wine stored in several warehouses.

More formally, let P be the finite set of supply points and Q the finite set ofdemand points . The supply of the good at point i E P equals Si units and the

SEMI-INFINITE ASSIGNMENT AND TRANSPORTATION GAMES 351

demand at point j E Q is dj units. Both Si and dj are (positive) integer numbersfor all i E P and j E Q, as we assume that the good is indivisible . The profit ofsending one unit of the good from supply point i to demand point j is tij, a nonnegative real number. All profits are gathered in the matrix T = [tij]iEP,jEQ'

Hence, a transportation problem can be described by the tuple (P, Q,T , s , d)where S = {Sil iEP and d = {dj }jEQ are the vectors containing respectivelythe supplies and demands of the good. For the sake of brevity we will use T' todenote the transportation problem (P, Q,T, s, d).

In the so-called Hitchcock-Koopmans transportation problem the goal is todetermine the number of units (Xij) to be transported from each supply pointi to each demand point j such that there are at most s, units transported fromsupply point i, the demand in each point j (dj) is satisfied and the maximalprofit (minimal cost) is achieved. This problem reads as follows:

{

LjEQ Xij ~ Si for all i E P }max L tijXij: L iEP Xij ~ dj f~r all j ~ Q

(iJ)EPxQ Xij E Z+ for all z E P, J E Q

where tij denotes the profit per unit transported from i to j and Z+ is the set ofnonnegative integers. It is well known that LiEP Si ~ LjEQ d j is a necessarycondition for feasibility of this problem. Moreover, if this condition holds withequality then the problem is called balanced. In a balanced transportationproblem all the constraints are equality constraints.

We are interested in relating each transportation problem to a cooperativegame with transferable utility (TV) . A cooperative TV-game can be describedby a pair (N, w) where N denotes the player set and w : 2N -t lR. is a functionthat assigns to each set of players S E 2N = {S'IS' eN} a real value w(S).For more details about cooperative game theory the reader is referred to thebooks by Curiel ([1]) and Owen ([11]) . Let S CPU Q. We would like todefine w(S) to be the value of the transportation problem related to the supplyand demand points in S. But if we consider the group of players {i} U Q forsome i E P then the related transportation problem

{

LjEQXij ~ Si }max L tijXij: Xij ~ dj for all j .E Q

jEQ Xij E Z+ for all J E Q

is feasible only if it satisfies the additional condition Si ~ LjEQ dj . Thiscondition should hold for all i E P. To avoid situations in which these additionalconditions do not hold, we will use another kind of transportation problem,described as follows : "how to achieve the maximal profit (minimal cost) whentransporting as much as possible from the supply points to the demand points?".


The transportation program becomes

{

L:jEQ Xij :S Si for all i E Pmax L tijXij: L:iEP Xij :S dj f~r all j ~ Q

(i ,j)EPxQ . Xij E Z+ for all 2 E P, J E Q}.

Notice that we are assuming tij ~ 0, for each i E P and j E Q. Using thetotal unimodularity of the system matrix ([9]) and the fact that the suppliesand the demands are integers, all extreme points of the set of feasible solutions[Xij](iJ)EP XQ are integer solutions ([13]) .

A transportation plan X = [Xij]iEP,jEQ is a matrix with integer entries whereXij is the number of units of the good that will be transported from supply pointi to demand point i. The maximal profit that the supply and demand points canachieve equals

Vp(1) = max { L tijXij: X is a transportation Plan} .(i ,j)EPxQ

A transportation plan X is also called a solution for T. Such a solution is anoptimal solution if L:(iJ)EPXQ tijXij = vp(T).

Given a transportation problem T, the corresponding transportation game(N, w) is a cooperative TV-game with player set N = P U Q. Let SeN,S =1= 0, be a coalition of players and define Ps = P n Sand Qs = Q n S .If S = Ps then there are no demand points present in S and therefore thesupply points in S cannot get rid of their goods. In this case the worth w(S)of coalition S equals zero. Similarly, if S = Qs then the demand points inS cannot receive any units of the good and w(S) = O. Otherwise, the worthw(S) depends upon the possible transportation plans. A transportation planX(S) for coalition S is a transportation plan for the transportation problemTs = (Ps, Qs, [tij] iEPs,jEQs' {SiliEPs ' {dj}jEQs) ' In this case

w(S) = max { L tijXij : X(S) is a transportation plan for S}(i ,j)EPsxQs

= vp(TS)

is the worth of coalition S .

One of the main issues in cooperative game theory is how to divide the totalprofit derived from cooperation. One way to share this profit among the playersin N is to do so according to an clement in the core. A precedent of this conceptis introduced in an economic context by Edgeworth ([2]) and the definition for


cooperative games is established by Gillies ([5]). The core of a cooperativegame (N, w) is the set

() {TT1>N 'L:iEN Zi = w(N) and }

C w = Z E ~ : 'L:iES Zi ~ w(S) for all SeN, S =I 0 .

If a core-element Z is proposed as a distribution of the total profit w(N) , theneach coalition S will get at least as much as it can obtain on its own because'L:iES Zi ~ w(S). So, no coalition has an incentive to disagree with thisproposal.

A special case of transportation problems occurs when all supplies Si anddemands dj equal 1. This kind of problem is called an assignment problem because in an optimal plan we either have that the whole supply of i E P istransported to one demand point or nothing is transported. This is like assigning supply points to demand points. For example, how should employees beassigned to jobs such that the total costs are minimized? Such an assignmentproblem is described by a tuple (M, W, A), where the sets M and W containrespectively the supply and demand points. The benefit of assigning i E M toJEW equals aij ~ 0, A = [aij] iEM,jEW.

In the next section, based on Llorca et al. ([8]), we summarize the mostrelevant results about semi-infinite assignment situations and correspondinggames.

3 SEMI-INFINITE ASSIGNMENT GAMES

In assignment situations we are interested in how to match , for example, afinite set of machines to a set of jobs such that we achieve the highest possiblebenefit. Consider a firm with a finite number of glass-cutting machines that canbe programmed to produce a vase. This firm can choose from an infinite numberof patterns (their designer are very productive!). The machines can produce allthese patterns, but with different (bounded) rewards. The marketing policy ofthe firm is to make unique vases. So, the firm has to tackle an assignmentproblem in which there is a finite number of one type (machines) and an infinitenumber of other type (possible designs). Its goal is to achieve the 'maximal'total benefits from matching the machines with the patterns.

A semi-infinite (bounded) assignment problem is denoted by a tuple (M, W,A), where M = {l , 2, "0' m} is a finite set, W = N, where N = {l, 2, ... } isthe set of natural numbers, and the nonnegative rewards aij are bounded fromabove, for all i EM, JEW. We will use A to denote the assignment problem(M,W,A).

An assignment plan Y = [Yij]iEM,jEW is a matrix with 0, l-entries whereYij = 1 if i is assigned to j and Yij = 0 otherwise. Each supply point will be


assigned to at most one demand point and vice versa, therefore 2:j Ew Yij ~ 1

and 2:i EM Yij ~ 1. Then

vp(A) = sup { L a ijYij : Y is an assignment Plan}(i,j)EMxW

is the smallest upper bound of the benefit that the supply and demand points canachieve. An assignment plan Y is also called a solution for A. Such a solutionis optimal if2:(i,j)EMxW aijYij = vp(A).

The corresponding assignment game (N,w) is the game with player setN = M U W. Hence, any supply or demand point corresponds to a player andany player corresponds to either a supply or a demand point. Let Ms = M n Sand Ws = W n S. Then the coalition S of players in N receives w(S) = 0if S = Ms or S = Ws because in these cases there is nothing to be assignedto. Otherwise, w(S) = vp(As) where As is the (semi-infinite) assignmentproblem (Ms,Ws, [aij] iEMs,jEWs) '

Relaxing the 0, I-condition of Yij to nonnegativity does not change the valueof the program, as the next lemma shows .

Lemma3.!

vp(A) = sup { L aijYij :

(i,j)EMxW

2:j EWOYfji

j ~111,. 2:Mi EM ~ij W~1, } .

Yij ~ or a z E ,J E

Proof. Let A* be the relaxed linear program

sup 2:(i ,j)EMxWaijYij

s.t. 2:j Ew Yij ~ 1, 2:i EM Yij ~ 1,Yij ~ 0 for alIi E M, JEW.

Obviously, vp(A) ~ vp(A*). We will show that vp(A) ~ vp(A*) - e for alle > O. For this define for each nEW the corresponding finite approximationproblem~ by

max 2:iEM 2:j=1 aijYij

s.t. 2:j=1 Yij ~ 1, 2:i EM Yij ~ 1,Yij ~ 0 for all i E M, j = 1, . . . ,n.

Let E > 0 and take a solution Y' of A * such that 2:(i,j)EMX W aijY~j ~

vp(A*) - c/2. Let nEW be such that 2:i EM 2:j=1 aijY;j ~ vp(A*) - c.It is well known that there exists an integer optimal solution Y" of the finite


program~ and that

n

2: 2:aijy~jiEM j=l

= max { 2:t aijYij :

i E M j=l

= : vp{An ) ,

L-j=l Yij ~ 1, L-iEM Yij ~ 1, }Yi j E {O, 1} for all i EM, j = 1, ... , n

where the finite assignment problem {M, {1, .. . ,n } , [aij] iEM,j=l, ... ,n) is denoted by An. Then

n n

vp{An ) = 2:2:aijy~j 2: 2:2: aijY~j 2: vp{A*) - ciEM j=l i EM j=l

where the first inequality follows from the fact that [Y~j] iEM,j=l, .. . ,n is a solutionof~. Together with u.l.A) 2: vp{An ) we conclude thatvp{A) 2: vp{A*)-c. 0

If we replace the condition Yij E {O, 1} by Yij 2: 0 then the dual program,with value vd{A) , of the problem that determines vp{A) equals

vd{A) = inf L-iEM Ui + L-jEW Vj

s.t, Ui + Vj 2: a ij , for all i E M , JEWUi , Vj 2: 0, for all i EM, JEW.

Let Od{A) be the set of optimal solutions of this dual problem. Both the primaland the dual program have an infinite number ofvariables and an infinite numberof restrictions. In general, 00 x co-programs show a gap between the optimalprimal and dual value. There is a large literature on the existence or absence ofso-called duality gaps in (semi-)infinite programs. See for example the booksby Glashoff and Gustafson ([6]) and Gobema and Lopez ([7]).

Semi-infinite assignment problems can be analyzed by finite approximationmatrices An E JRffi xn where An = [aij]iEM,j=1 ,2, ... ,n, and by means of the socalled hard-choice number of the matrix A. The following example illustratesthis last concept.

Example 3.1 Let M = {1, 2, 3}, W = Nand

A ~ [ Ii 3 3 3 3 3 33 8 2 9 2 2 ... ]1 3 5 4 9 11

2" 3 5 "6 .. .

For each agent i E M the choice set Ci consists of at most IMI elements inW, namely those that give the highest reward a i j when assigned to i , provided

they exist. If more than IM I agents in W satisfy this criterion then the choiceset contains only those IMI agents with the the smallest ranking number in W .

In this example, no matter to whom agent 1 E M is assigned, the resultingreward equals alj = 3. Hence, we take the three agents with the smallestranking number and C 1 = {I, 2, 3}.

If agent 2 E M is assigned to agent 1 E W then they obtain the maximalreward of 10. The second largest value is a25 = 9 and a23 = 8 is the thirdlargest value. Thus, C2 = {I , 3, 5}.

Finally, assigning agent 3 E M to agent 5 E W results in the maximalreward a35 = 4. However, there is no second largest value because a3n goesto 2 when n goes to infinity. So, this agent has C3 = {5}.

The hard-choice number n*(A) is the smallest number in N U {O} suchthat UiEMCi C {I, 2, .., n* (An. In this example we have n* (A) = 5.

The following theorem establishes that the primal and the dual problem havethe same value and there exists an optimal solution of the dual problem .

Theorem 3.1 ([8, Theorem 3.9]) Let (M, W, A) be a semi-infinite boundedassignment problem. Then vp(A) = vd(A) and Od(A) i= 0.

A sketch of the proof of the latter statement goes as follows. Take foreach n E N, n > n* (A), an element (u", vn ) of Od(An ) and remove allcoordinates of vn with index larger than n* (A) . The set of all those elements,which is in the finite dimensional space JlF xlRn*(A), is bounded. Without lossof generality, suppose that the limit, when n goes to infinity, of such a sequenceexists (otherwise take a subsequence) and denote this limit by (ti, n), With theaid of (ti, v) construct the vector (u, v) by taking u= ti and v is obtained fromvby adding an infinite number of zeros. Then (u, v) is an optimal dual solutionof the corresponding semi-infinite bounded assignment problem.

This theorem is of great importance for the next section. There we showthrough related semi-infinite assignment games that semi-infinite transportationproblems have no duality gap and the corresponding games have a nonemptycore.

4 SEMI-INFINITE TRANSPORTATION PROBLEMSAND RELATED GAMES

In this section we extend finite transportation problems to semi-infinite transportation problems. These are transportation problems where the number ofone type of agents (demanders or suppliers) is countably infinite. We assumethat Q = N, and that the profits tij are bounded from above, that is IITlloo < 00.

Once again, all the supplies and demands are positive integers . Therefore thecorresponding semi-infinite transportation situation has a finite value. For anexample, consider the situation of glass-cutting machines described in section


3. Assume that each machine can produce a finite number of vases. Now,the marketing policy of the firm is to make limited series of vases with certainpatterns. This situation can be seen as a semi-infinite transportation problemwhere the machines supply the vases and where limited series of each patternare demanded.

Corresponding to a semi-infinite transportation problem we define a semiinfinite transportation game (N, w) with player set N = P U Q. As before, theworth of coalition S equals zero, w(S) = 0, if S = Ps or S = Qs and

w(S) = sup { L tijXij : X(S) is a transportation plan for S}(i,j)EPs xQs

= vp(Ts)

otherwise.

Given a semi-infinite transportation problem T we construct a related semiinfinite assignment problem A(T) in the following way:

• Each supply point i E P is split into Si supply points named iI, i2, , is.,each with a supply of I unit. Hence, M = {ir : i E P, r E {I, , sd}.

• Each demand point j E Q is split into dj different players j1, j2, ... ,jdj, each with a demand of I unit. Therefore , W = {jc : j E Q, c E{I, ..., dj}}. Notice that W is a countably infinite set of players becauseQ=N.

• Define air, jc = tij for all ir E M, jc E W.

The next lemma deals with relations between solutions in T and A(T).

Lemma 4.1 Each solution for T determines a solution for A(T). andconversely. These solutions have the same value.

Before we prove the lemma, we give an example to illustrate a procedurethat we use in the proof.

Example 4.1 Consider the transportation problem T with P = {I, 2, 3},Q = Nand

2 1 2 1 1 1 dj2 1 2 3

11 0 2 1- =T3 3 2 dSi

358 SEMI-INFINITE PROGRAMMING. RECENT A DVANCES

A solution for T is the transportation plan

[

0 0 1 0X= 0 0 1 0

2 1 0 0~ ~ ~ :::]o 0 0 ...

with value L (i ,j )EP XQ tijXij = 11. The corresponding assignment problemA(T) hassupplypointsM = {11, 12, 21, 31, 32, 33} and demand points W ={11, 12,21,31 ,32,41 ,51 ,61 , ...}. From the solution X for T we construct asolution Y for A(T) where each cell in X with Xi j > 0 will corre spondto Xi j cells in Y with entry 1. The procedure goes as follows. We start withi = j = 1. If Xij i- 0 then we look for the smallest values for rand c suchthat both the points ir and jc are not assigned to any point, that is, row ir andcolumn jc in Y contain no entry equal to I so far. Define Yir ,jc = 1 . Continue

searching for new values rand c until L:~l L~~l Yir,jc = Xij . Repeat thisfor all (i, j) E P x Q with Xij i- 0 , where you first consider the first rowand first column in X , then the second row and second column, and so on. SetYir,j c = 0 for the remaining (ir , jc) E M x W . Following this procedure weobtain the assignment plan

11 12 I 21 31 32 41 I 51 61 71 I ...11 0 0 0 1 0 0 0 0 0 .. .12 0 0 0 0 0 0 0 1 0 .. .

21 0 0 0 0 1 0 0 0- 0 ...31 1 0 0 0 0 0 0 0 0 ...32 0 1 0 0 0 0 0 0 0 .. .33 0 0 1 0 0 0 0 0 0 ...

W

=Y

M

with value L(ir,jc)EM XW air,j cYir ,j c = 11. Conversely, given a solution Y for

A(T) , a solution X for T is given by Xij = L:~l L~~l Yir,j c for all i E P ,j E Q.

Proof of Lemma 4.1. Let X be a solution for T. Define the matrix Y E{O, I}M XW by

if {

(i) r E (Lq<j Xiq , Lq~j Xiq] ,

1 (ii) c E (Lp<i Xpj , Lp~i Xpj] andYir ,jc =(ii i) r - Lq<j Xiq = C - Lp<i Xpj ;

0 otherwise.

We show that Y is a solution of A(n. By definition Y ir Jc E {O, I} .

SEMI-INFINITE ASSIGNMENTAND TRANSPORTATION GAMES 359

Assume that Yir,jc = 1 and e < dj, that is, there exists a je' E W withd > e. Then

r - L Xiq = e - L Xpj < e' - L Xpj

«<: p e.

Next, consider (ir, j'd) with j' > j. If Xij' = 0 then Lq<j' Xiq =

Lq~j' Xiq, condition (i) cannot be satisfied and therefore Yir,j'd = O. Otherwise, if xij' > 0 then r ~ Lq~j Xiq ~ Lq<j' Xiq, where the first inequalityfollows from Yir,jc = 1. But then r f Lq<j' Xiq , condition (i) is not satisfied

for (ir,j'd) and so, Yir,j'd = O.We conclude that if Yir,jc = 1 then the remainder of row ir in Y (as of

column je) contains only entries equal to zero . Similarly, we can show that theremainder of column je (as of row ir) also consists of entries equal to zero.Hence, LjcEW Yir,jc ~ 1 and LirEM Yir,jc ~ 1. The matrix Y is a solutionof A(T) .

Finally, let Y be a solution of A(T). Define Xij = L;~l L~~l Yir,jc for alli E P, j E Q. Then, Xij is a non-negative integer and for all j E Q we have

S i dj dj Si

LXij = LLLYir ,j c = LLLYir,jciEP i EP r=l c=l c=l iEP r=l

dj dj

= L L Yir,jc~Ll=dj.c= l irEM c= l

The inequality holds because Y is a solution of A(T). Analogously, we canshow that LjEQ Xij ~ Si for all i E P. Hence, X is a solution of T. It readilyfollows that both solutions have the same value. 0

The following result is an immediate consequence of Lemma 4.1.

Lemma 4.2 Let T be a semi-infinite transportation problem and A(T)the corresponding assignment problem. Then vp(T) = vp(A(T)).

Recall that vp(T) is the value of the problem

sup { L t ijXij: X is a transportation Plan} .(i ,j)EPxQ

Similarly to Lemma 3.1 for semi-infinite assignment problems, we can showthat relaxing the condition Xij E N to Xij ::::: 0 will not change the value of the


problem. The dual problem D corresponding to this program is

inf {LSiUi + Ldjvj : Ui + Vj ~ tij , Ui,Vj ~ 0 for all i E P , j E Q}'iEP jEQ

We denote the value of this program by Vd(n and Od(n is the set of optimalsolutions of D . Similarly, we define for the related assignment problem A(n

(A(n) - inf {'""' . '""' . . Uir + Vjc ~ airJc, Uir , Vjc ~ 0 }Vd - 1 LJ U1r + LJ VJc

0 for all ir E M , jc E W .irEM jcEW

Let Od(A(n) be the set of optimal solutions of this infimum problem. Asis the case for semi-infinite assignment problems, semi-infinite transportationproblems have no duality gap , that is, vp(n = Vd(n and Od(T) is nonempty.

Theorem 4.1 Let T be a semi-infinite transportation problem. Then

(a) vp(n = Vd(n and

(b) Od(n =1= 0.

Proof. Theorem 3.1 states that Od(A(n) =1= 0, so, let (u,v) E Od(A(n) .Then, Uir + "i« ~ airJc = tij for all ir E M, jc E W. Thus for all i E P,j E Q,

S i dj Si dj Si dj

L L (Uir +Vjc) = dj L Uir + Si L vs« ~ L L tij = Sidjtijor=l c=l r=l c=l r=l c= l

Dividing both sides by Sidj gives

Si dj

L Uir/Si + L Vjc/dj ~ tijor=l c=l

Define Ui := ~~~l Uir/Si and Vj := L::~lVjc/dj. Then Ui ~ 0, Vj ~ 0, andUi +Vj ~ tij for all i E P, j E Q. Hence,

Vp(T) = vp(A(T)) = vd(A(T)) = L SiUi + L djvj ~ vd(T)iEP jEQ

where the first equality follows from Lemma 4.2, the second one from Theorem3.1, the third one from (u , v) E Od(A(n), and the last inequality follows fromthe definition of Vd(T). From duality theory we know that vp(n ::; Vd(n andtherefore

Vp(T) = L SiUi + L djvj = vd(T).i EP jEQ


We conclude that vp(T) = Vd(l) and (iL, v) E Od(T). 0

A concept related to the core is the so-called Owen set 1, which for transportation games is equal to

{N 3(u,v) E Od(T) such that Zk = SkUk }

Owen(1) = Z E lR : if k E P and Zk = dkVk if k E Q .

This set is not empty because Od(T) -1= 0. An element of the Owen set iseasy to find and it turns out to be an element of the core of the correspondingtransportation game as well.

Theorem 4.2 Let T be a semi-infinite transportation problem and (N, w)the corresponding game. Then, Owen(T) C C(w).

Proof. Let Z E Owen(T) and let (u, v) E Od(T) be such that Zk = SkUk ifk E P and Zk = dkVk if k E Q. Then

L Zi = L SiUi + L djvj = Vd(T) = vp(T) = w(N),iEN iEP jEQ

where the third equality follows from Theorem 4.1. Next, let SeN, S -1= 0.If S = Ps or S = Qs then LkES Zk ~ °= w(S) because Zk ~ °for allkEN. Otherwise, we know that Ui + Vj ~ tij for all i E P, j E Q, and thisholds in particular for all i E Ps, j E Qs. Thus

L Zi = L SiUi + L djvj ~ vd(Ts) = vp(Ts) = w(S).iES iEPs jEQs

We conclude that Z E C(w). 0

In general, the Owen set does not coincide with the core of a transportationgame, as the following example shows.

Example 4.2 Let T be a transportation problem with P = {I}, Q = N,and

2 I4 I 3 I~Si

I I

In this problem Owen(T) only contains (8; 2, 0, 0, ...). However, the point(10; 0, 0, ...) is an element of the core of the corresponding transportation game.Hence,Owen(T) is strictly contained in the core C(w).


5 FINAL REMARK

In our future research we will study semi-infinite transportation problemswhere supplies and demands are positive real numbers. The underlying idea isto consider infinitely divisible goods. One can think of using pipelines insteadof containers for the transportation ofpetrol. In this framework we consider twosemi-infinite transportation situations . The first one is such that the total demandfor the good is infinite and the individual demands are bounded from below, andin the second one the total demand is finite. In both cases, we will show thatthe corresponding semi-infinite transportation games have a nonempty core.

Acknowledgment

Judith Timmer acknowledges financial support from the Netherlands Organization for Scientific Research (NWO) through project 613-304-059.

Notes

1. Owen ([10]) presents a method to find a nonempty subset of the core of a linear production game .Gellekom et al. ([4]) names this set the 'Owen set' .

References

[1] I. Curiel, Cooperative Game Theory and Applications, Kluwer, 1997.

[2] F.Y. Edgeworth, Mathematical Psychics, Kegan, London, 1981.

[3] V.Fragnelli, F.Patrone, E. Sideri, and S. Tijs. Balanced games arising frominfinite linear models, Mathematical Methods of Operations Research,50:385-397, 1999.

[4] J.R.G. van Gellekom, J.A.M. Potters, J.H. Reijnierse, S.H. Tijs, and M.e.Engel. Characterization of the Owen set of linear production processes,games and economic behavior, 32:139-156, 2000.

[5] D.B. Gillies. Some theorems on N-person games . Dissertation, PrincentonUniversity, 1953.

[6] K. Glashoff, andS-A. Gustafson. Linear Optimization andApproximation,Springer-Verlag, 1983.

[7] M. A. Goberna andM.A. Lopez . Linear Semi-Infinite Optimization, Wiley,1998.

[8] N. Llorca, S. Tijs, and J. Timmer. Semi-infinite assignment problems andrelated games, International Game Theory Review, 2:97-106,2000.

[9] K.G. Murty. Linear Programming, Wiley, 1983.

[10] G. Owen. On the Core of Linear Production Games , Mathematical Programming. 9:358-370, 1975.


[11] G. Owen. Game Theory, Academic Press , 1995.

[12] J. Sanchez-Soriano, M.A. Lopez, and 1. Garda-Jurado. On the Core ofTransportation Games, Mathematical Social Sciences , 2000 .

[13] A. Schrijver. Theory of Linear and Integer Programming, Wiley. 1986.

[14] L.S. Shapley and S. Shubik. The Assignment Game I: The Core, International Journal ofGame Theory, 1:111-130, 1972.

[15] J. Timmer, N. Llorca and S. Tijs. Games Arasing from Infinite ProductionSituations, International Game Theory Review , 2:97-106, 2000.

Chapter 17

THE OWEN SET AND THE COREOF SEMI-INFINITE LINEARPRODUCTION SITUATIONS

Stef Tijs", Judith Timmer", Natividad Llorca" and JoaquinSanchez-Sorlano-lCentER and Department ofEconometrics, Tilburg University, P.O. Box 90153, 5000LETilburg,

The Netherlands

2Department of Statistics and Applied Mathematics, Miguel Hernandez University, Elche Cam

pu s, La Galia Building, Avda. del Ferrocarril, sin, 03202 Elche, Spain

[email protected], [email protected] , [email protected], joaquin@umh .es

Abstract We study linear production situations with an infinite number of production techniques. Such a situation gives rise to a semi-infinite linear program. Related tothis program, we introduce primal and dual games and study relations betweenthese garnes, the cores of these games and the so-called Owen set.

1 INTRODUCTION

Linear production (LP) situations are situations where several producers ownresource bundles. They can use these resources to produce various productsvia linear production techniques that are available to all the producers. Thegoal of each producer is to maximize his profit, which equals the revenue ofhis products at the given market prices. These situations and correspondingcooperative games are introduced in Owen ([4]). He showed that these gameshave a nonempty core by constructing a core-element via a related dual linearprogram. Samet and Zemel ([5]) study relations between the set of all coreelements we can find in this way and the core, and the emphasis in their study isplaced on replication of players. Gellekom, Potters, Reijnierse, Tijs and Engel

365

M.A. Gobernaand M.A. Lopez (eds.), Semi-Infinite Programming, 365-386.© 2001 KluwerAcademicPublishers.


([2]) named the set of all the core-elements that can be found in the same wayas Owen did, the "Owen set" and they give a characterization of this set.

More general are situations involving the linear transformation of products(LTP) introduced by Timmer, Borm and Suijs ([7]) because it is shown in thispaper that any LP situation can be written as an LTP situation. In an LTPsituation different producers may control different transformation techniquesand each of these techniques can have more than one output good. LTP situationsgive rise to LTP games, which also have a nonempty core.

A part of Fragnelli, Patrone, Sideri and Tijs ([1]) is devoted to the study ofsemi-infinite LP situations. These are LP situations where there is a countablyinfinite number of products that can be produced. Semi-infinite LTP situations,in which there is a countably infinite number of transformation techniques, areanalyzed in Timmer, Llorca and Tijs ([8]).

In this work we study the Owen set and the core of semi-infinite LP andLTP situations and relations between these two concepts. For this reason, weintroduce primal and dual games corresponding to the primal and dual programsof both semi-infinite LP and LTP situations. Using these primal and dual gameswe show that ifthese games have the same value then the Owen set is included inthe core and otherwise, they are disjoint. Our main result is that for both semiinfinite LP and LTPsituations the core of the corresponding game is nonempty ifthere exists a finite upper bound for the maximal profit obtained by the coalitionof all producers. Because LTP situations are more general than LP situations,the use of more sophisticated tools is required to show that the results for LPsituations also hold for LTP situations. This is why we analyze LP situationsbefore turning our attention to LTP situations.

This work is organized as follows. The Sections 2 and 4 present the most relevant results of respectively finite LP and LTP situations and their correspondinggames. Semi-infinite LP and LTP situations are introduced in the Sections 3and 5, respectively. Relations between the Owen set, the core and the primaland dual games are investigated and we show that the core is nonempty if thereexists a finite upper bound for the maximal profit obtained by the coalition ofall producers. Section 6 concludes.

2 FINITE LINEAR PRODUCTION SITUATIONS

Finite linear production (LP) situations describe situations with a set of producers, a bundle of resources for each producer and a set of linear productiontechniques that all the producers may apply. The resources are used in the various linear production techniques to produce some products that can be sold onthe market at given market prices. We assume that there are no costs involved.The goal of each producer is to maximize his profits . Producers are also al-

THE OWEN SET AND THE CORE OF SIL PRODUCTION SITUATIONS 367

lowed to cooperate and pool their resources. Such a coalition of producers alsomaximizes its profit given the joint resources . Cooperation pays off becausethe maximal profit of the group is at least as much as the sum of the individualprofits.

More formally, denote by N, Rand Qrespectively the finite sets ofproducers,resources and products. The technology matrix A E R~xQ , that is A E RRxQ

andAij ~ 0 for all i E Randj E Q,describes all the available linear productiontechniques in the following way. Each production technique produces oneproduct and you need A ij units of resource i E R to produce one unit ofproduct j E Q. The resources owned by the producers are described by theresource matrix B E R~x N where producer kEN owns Bik units of resource

i E R. Prices are denoted by the price vector c E R~ \ {O}. We assume thatthere is a positive quantity available of each resource, that is, for all resourcesi E R there is a producer k such that Bik > O. Furthermore, if there is a productj with a positive market price, then we do not allow for "output without input"and therefore there exists at least one resource i E R with Aij > O. Finally, allproducers are price-takers and all products can be sold on the market.

To maximize his profit, producer k needs an optimal production plan x E R~that tells him how much he should produce of each good. Not all productionplans are feasible since the producer has to take into account his limited amountof resources. The amount of resources needed in production plan x , Ax, shouldnot exceed the amount of resources of producer k, Be{k} ' where e{k} denotes

the k t h unit vector in R N with e{k},t = 1 if t = k and e{k},t = 0 otherwise.Furthermore, the production plan has to be nonnegative since we are only interested in producing nonnegative quantities of the products, and its profit equalsxT c. The following linear program maximizes the profit of producer k.

max {xTc lAx ~ Be{k} ' x ~ O}

Next to producing on their own, producers are allowed to cooperate. If acoalition S ofproducers cooperates then they put all their resources together andso, this coalition has the resource bundle Bes at its disposal, where es E R N

with esi = 1 if t E Sand eS,t = 0 if t f{. S. Given this amount of resources,the coalition wants to maximize its profit,

Ps: max {xTclAx ~ Bes , x ~ O},where Ps denotes the primal linear program for coalition S. The correspondingdual problem for this coalition, Ds, is the following program.

Ds: min {yTBes IAT y ~ c, y ~ O}

The vector y can be seen as a vector of shadow prices for the resources sincethe condition ATy ~ c can be interpreted as follows. If a company wants


to buy the resources Bes of coalition S and is willing to pay Yi per unit ofgood i E M then for any product j E Q, the value of the resources needed toproduce one unit of this product according to the prices in y should be at leastas large as the market price Cj. Otherwise, coalition S will not agree with thissale . Therefore, the program D S minimizes the value of the resources ownedby coalition S according to the shadow prices and subject to the restrictionsabove.

For ease of notation, let Fps and FdS denote the set of feasible solutions ofrespectively the primal and dual program for coalition S,

{x E RQ lAx::; Bes, x ~ O},{y ERR IAT Y ~ c, Y ~ 0 } ,

denote by wps and WdS the optimal values of the programs,

max {xTC Ix E Fps } ,

min {yTBes Iy E FdS} ,

and let Ops and OdS be the sets of optimal solutions,

{x E Fps /xTC = wps} ,

{y E FdS yTBes = WdS} .

The assumptions we made ensure that Fps, FdS, Ops and OdS are nonemptysets and wps and WdS exist and are finite. It follows from duality theory ([9, p.281]) that wps = WdS for all coalitions S.

We see that an LP situation can be described by the tuple (N, A, B, c).Corresponding to such a situation we define two games, (N, vp ) and (N, Vd) .The first one, (N, vp), is the well known LP game where vp(S) = wps for allcoalitions S. The second game, (N, Vd), is the game that gives each coalitionS the value of its dual program, Vd( S) = WdS.

If two producers cooperate then they can produce at least the amount thatthey can produce on their own, so, their joint profit will be at least as large as thesum of their individual profits. Similar reasoning shows that the highest profitwill be obtained if all the producers work together. But how should this jointprofit be divided among the producers? We could divide the profit accordingto a so-called core-allocation. The core of a game (N, v) , C (v), allocates theprofit in such a way that no coalition of producers has an incentive to startproducing on their own. More precisely,

C(v) = {x E RN LXi = v(N), LXi ~ v(S) for all seN} .

iEN iES

THE OWEN SETAND THE CORE OF SILPRODUCTION SITUATIONS 369

We define the core of an LP situation, Core(A, B, c), to be the core of thecorresponding LP game, Core(A,B,c) = C(vp ) . Owen ([4]) shows that LPgames are totally balanced, that is, the games themselves have a nonempty coreand so do all of their subgames. He obtains this result by showing that we caneasily obtain a core-element of an LP game as follows. Instead of solving theprograms Ps for all coalitions SeN in order to calculate vp(S) and the core,we only solve D N, the dual program of the grand coalition. Let y be an optimalsolution of D N. If each producer k gets the value of his resources according tothe shadow prices, yT Be{k}' then this distribution ofvalues is a core-allocation.The set of all core-allocations that we can obtain in this way, is called the Owenset corresponding to the LP situation (N, A, B, c):

This set has been studied extensively by Gellekom et al. ([2]) and they alsoprovide a characterization of the Owen set. Because the set OdN is nonempty,so is Owen(A, B, c). Furthermore, each vector in this setis an element ofC(vp )

and therefore Owen(A, B, c) C Core (A, B , c).

We end this section with an example of an LP situation and correspondingLP game.

Example 2.1 Consider the following LP situation. There are two producers, N = {I, 2}, two resources, two products and

Producer 1 owns nothing of the second resource (see the first column of theresource matrix B) and producer 2 owns nothing of the first resource. Since bothproducts require a positive amount of input of each of the two resources, a singleproducer cannot produce anything. Consequently, vp ({ I}) = vp ({2}) = O. Ifboth producers cooperate then they own a positive amount of each resource andthey have many production plans at their disposal, namely all plans x E FpN :

FpN = {x E R212xI + 2X2 :::; 6, Xl + 3X2 :::; 7, x ~ O}

The profit of such a production plan x is cT X = 3XI + 4X2 and so, the profitmaximization problem PN of the grand coalition equals max{3xI + 4X21 x EFpN } . The maximal profit WpN = 11 is attained in the plan x = (1,2)T, soOpN = {(I, 2)T}, and Core(A, B, c) = {(a, 11 - a)TI0 :::; a :::; 11}. For thedual game (N, Vd) it holds that Vd({i}) = vp ( {i}) = 0 for all i E N. The setof all feasible shadow prices for the grand coalition is the set


We want to minimize the value of the resources of coalition N according tothe shadow prices y, yTBeN = 6Yl + 7Y2, over all feasible shadow prices:min{6Yl + 7Y21 y E FdN} . The minimum WdN = 11 is attained in y =(5/4, 1/2)T and so 0dN = {(5/4, 1/2)T}. The Owen set, Owen(A,B, c) ={(15/2, 7/2)T}, consists of one point and we see that Owen(A, B, c) cCore(A ,B,c), Owen(A,B,c) =1= Core(A,B,c) .

3 SEMI-INFINITE LP SITUATIONS

If we extend the set Q such that it contains a countable infinite number ofproducts then we arrive at semi-infinite LP situations. Without loss of generalitywe may assume that Q = N = {I, 2, ... }, the set of natural numbers. Anexample of a production process with a countable infinite number of productsis the "process" of baking pancakes at home. Pancakes are made of milk, flour,eggs, salt, butter and perhaps a little sugar. If you have a recipe for bakingpancakes and you change the amounts of the ingredients slightly (e.g. you adda little flour or you use a little bit less milk) then you get another recipe forpancakes . This set of processes will be countable infinite if you require that allquantities should be integer multiples of one gram, for example.

A semi-infinite LP situation (N, A, B, c) thus has A E R~xQ , B E R~XN

and c E R~ with Q = N. As opposed to LP situations, we impose no furtherrestrictions on these variables since we want to keep our analysis as generalas possible. The restrictions we imposed in the previous section turn up bythemselves in the proof of Theorem 3.5. Because we have a countable infinitenumber of products, the linear programs, which determine the "maximal" profitsof the coalitions, and their dual programs are now semi-infinite linearprograms.The primal program for a coalition S of producers that determines its maximalprofit, now equals

PS: sup{xTcl Ax ~ Bes, x ~ O},

where we replaced the maximum by the supremum since the optimal value maynot be reached by any production plan x. This program contains an infinitenumber of variables xi - j E Q. Similarly, in the dual program

Ds : inf{yTBesl ATy ~ c, y ~ O}

we replaced the minimum by the infimum because we have an infinite numberof restrictions . The set of feasible dual solutions, FdS, may now be empty andthe same holds for the sets of optimal solutions Ops and OdS. The optimalvalues are

WpS = sup{xTcl x E Fps}

WdS = inf{yTBesl y E FdS}'

THE OWEN SET AND 111E CORE OF SIL PRODUCTION SITUA110NS 371

Once again we define two games, the LP game (N, vp ) and the dual game(N, Vd) where vp(S) = wps and Vd(S) = WdS . Notice that in this setting thevalues wps and WdS may take any nonnegative number including +00. Severalnice properties of these games are mentioned in the next theorem.

Theorem 3.1 Let (N, A, B , c) be a semi-infinite LP situation. Then

(a) FdS = FdN for all SeN;

(b) vp and Vd are monotonic games; and

(c) vp(S) ~ Vd(S) for all SeN.

Proof. First, by definition it holds that FdS = {y E RRIATY 2: c, y 2: O} =FdN for all SeN.

Second, let SeT c N be coalitions of agents. Then a game (N, v) ismonotonic if v(S) ~ v(T). Here, Bes ~ BeT implies that Fps C FpTand so vp(S) = sup{xTc] x E Fps} ~ vp(T). From the first part of thisproof it follows that FdS = FdT and together with Bes ~ BeT this givesVd(S) = inf{yTBesly E FdS} ~ vd(T) .

Third, let SeN be a coalition of agents. If FdS = 0 then vp(S) ~

00 = Vd(S). Otherwise, take feasible solutions x E Fps and y E FdS. ThenxTc = cTX ~ yT Ax ~ yT Bes and therefore vp(S) = sup{xTc]x E Fps} ~

inf{yTBesl y E FdS} = Vd(S), 0

We use these properties to prove the next results about the relations betweenthe Owen set and the cores of the LP and dual games.

Theorem 3.2 Let (N, A , B, c) be a semi-infinite LP situation. Then

(a) Owen (A , B, c) C C(Vd); and

(b) ijvp(N) = vd(N) then C(Vd) C C(vp).

Proof. To show the first item, if Owen(A, B, c) = 0 then we are finished.Otherwise, take an clement Z E Owen(A, B , c). Then there exists an optimaldual solution y' E 0dN such that z, = (y')TBe{i} for all i E N. So, L:iEN Zi =L:iEN(y')TBe{i} = (y')TBeN = vd(N) because y' E 0dN. It also holds thatL:iES Zi = (y')TBes 2: inf{yTBesl y E FdS} = Vd(S) where the inequalityfollows from y' E FdS. We conclude that Z E C(Vd)'

For the second item, we are finished if C(Vd) = 0. Otherwise take anclement Z E C(Vd)' By definition it holds that L: iENZi = vd(N) = vp(N).It also holds that L:iES Zi 2: Vd(S) 2: vp(S) where the first inequality followsfrom Z E C(Vd) and the second one from statement 3 in Theorem 3.1. Hence,Z E C(vp). 0

A corollary of this theorem is that if vp(N) = vd(N) , that is, there is noduality gap, then Owen(A, B, c) C Core(A, B, c). In the first part of the proof

we noticed that Owen(A, B, c) = 0may hold. The following example providesa semi-infinite LP situation where this is true and where the cores of the twogames are nonempty.

Example 3.1 Consider the semi-infinite LP situation (N, A, B, c) whereN is a set of one agent,

;2 :::],BeN = [ ~ ] ,and

2n . . . ].

Then

inf{yT BeNI ATy ~ c, y ~ O}inf{Y2lYl +n2Y2 ~ 2n, n = 1,2, ... , Y ~ O}0,

FdN =I 0 but OdN = 0. Consequently, Owen(A,B,c) = 0. However,vp(N) = 0 implies that Core(A,B,c) = C(vp) = {O} =I 0. Similarly wecan show that C(Vd) = {O}.

Two other relations between the Owen set and the core, depending on thevalues vp(N) and vd(N), are presented in the next theorem.

Theorem 3.3 Let (N, A, B, c) be a semi-infinite LP situation. Then

(a) ifvp(N) < vd(N) < 00 then Owen(A, B, c) n Core(A, B, c) = 0; and

(b) if vp(N) < vd(N) = 00 then Owen(A, B, c) = 0 and the coreCore(A, B, c) is nonempty.

Proof. Concerning the first item, if Owen(A, B, c) = 0 then the proof isfinished. Otherwise, let z E Owen(A, B , c) and take y E OdN such thatZi = yTBe{i} for all i E N. Then L:iEN Zi = L:iEN yTBe{i} = yTBeN =vd(N) > vp(N). Hence, Z ~ C(vp) = Core(A, B, c).

Secondly, since BeN contains finite quantities, vd(N) = 00 can occur onlyif FdN = 0. In this case, OdN = 0 and therefore Owen(A, B , c) = 0. Thelatter part of this statement, the nonemptiness of Core(A, B , c), will be shownin Theorem 3.5. 0

All the above relations between the Owen set and the core of a semi-infiniteLP situation can be summarized as follows.

Theorem 3.4 Let (N, A, B, c) be a semi-infinite LP situation.

(a) Ifvp(N) = vd(N) then Owen(A,B,c) c Core(A,B, c).

(b) Ifvp(N) < vd(N) then Owen(A, B, c) n Core(A, B, c) = 0.

Proof. The proof follows immediately from the Theorems 3.2 and 3.3. 0

As we stated in the second part of the proof of Theorem 3.3 there is one thingleft to show, namely that the core of a semi-infinite LP situation is nonemptywhenever the "profit" of the grand coalition is finite, vp(N) < 00.

Theorem 3.5 Let (N, A , B, c) be a semi-infinite LP situation where thecorresponding LP game (N, vp) has vp(N) < 00. Then Core(A, B, c) =I- 0.

Proof. This proof is an exhaustive list of all possible semi-infinite LP situationsthat we may come across . In each of these situations we will show that ifvp(N)is finite then Core(A, B, c) is a nonempty set.

First, suppose that BeN = 0, where 0 denotes the vector with each elementequal to zero. Thus, all the agents have no resources available . But then noproducer can produce a positive quantity of any product, so Fps = {O} for allcoalitions S and consequently vp(S) = O. In particular, vp(N) = 0 < 00 andCore(A,B,c) = C(vp ) = {(O, . . . ,On =I- 0.

What happens if BeN =I- 0 but every product needs a resource that is notavailable? Let h(t) describe for all resource vectors t E R~ those resources thatare available in a positive quantity, so, h(t) = {i E RI ti > O} . Denote by ejthe jth unit vector in RQ with ej,l = 1 if l = j and ej,l = 0 otherwise. Then Aejis a vector in R~ that describes how much we need of each resource to produceone unit of product j E Q. Thus, h(BeN) 1J h(Aej) for all j E Q meansthat each product j E Q needs some unavailable resources. Consequently, noproducer can produce a positive quantity of some product, Fps = {(O, 0, . .. )}and vp (S) = 0 for all coalitions S of producers. In particular, vp (N) = 0 < 00

and Core(A, B, c) = {(O, .. . ,on =I- 0.Assume now that BeN =I- 0 and that some products can be produced, that

is, h(BeN) ::J h(Aej) for some j E Q. All coalitions of producers wantto maximize their profit and therefore they will restrict their production tothe products that can be produced. So, without changing the values of thecoalitions we remove all products j E Q that cannot be produced, that is, forwhich h(BeN) 1J h(Aej), as well as all unavailable resources i E R, whichhave (BeN)i = O. For simplicity of notation, let (N, A, B, c) also denote thisreduced semi-infinite LP situation.

This brings us to the next situation where BeN > 0 and consequently,h(BeN) = M ::J h(Aej) for all j E Q. What happens if c = 0, pricesare zero? If all products have a price equal to zero then anything a producersells on the market will give him a revenue of zero. So, vp(S) = 0 for all

coalitions S of producers and in particular it holds that vp(N) = 0 < 00 andCore(A, B, c) -# 0.

If BeN > 0 and there is a product j E Q for which Cj > 0 then we can removeall products j for which Cj = 0 without changing any of the values vp(S). Thisholds because each coalition of producers will restrict its production to theproducts with a positive price.

This leads to BeN> 0 and C > O. If there exists a product j E Q that usesno resources, Aej = 0, then the producers can produce an infinite amount ofthis good, because it needs no input, and sell it at price Cj > 0 to obtain aninfinite profit. Hence, vp(N) = 00 and we may say that the producers are inheaven since they can take as much of the profit as they want.

Finally, we end up with BeN> 0, C > 0 and Aej -# 0 for all j E Q. Inthis case we use a theorem of Tijs ([6]) that says that we either have OdN = 0and vp(N) = vd(N) = 00 (heaven once again), or vp(N) = vd(N) < 00

and OdN -# 0. In the latter case Owen(A, B, c) -# 0, which implies thatCore(A, B , c) -# 0. 0

We may conclude from this theorem that if vp(N) < 00 then there exists acore-allocation, a division of the value vp(N) upon which no coalition Scanimprove. If we are in the heavenly situation vp(N) = 00, then we do not needshadow prices or core-allocations since any producer can get what he wantsfrom vp(N), even if it is an infinitely large amount.

4 FINITE LTP SITUATIONS

Another kind of linear production is described by situations involving thelinear transformation of products (LTP), where the "T" stands for the transformation of a set of input goods into a set of output goods. Timmer, Borm andSuijs ([7]) show that an LP situation is a special kind of LTP situation.

In LTP situations each transformation technique may have more than oneoutput good. Recall that each production process in an LP situation has only oneoutput good, namely its product. Furthermore, different producers may havedifferent transformation techniques at their disposal, while in an LP situationall producers use the same set of production techniques. LTP situations areintroduced in Timmer, Borm and Suijs ([7]) and defined as follows.

Let M be the finite set of goods and N the finite set of producers. Produceri E N owns the bundle of goods w(i) E R~ and we assume, as we do forLP situations, that all producers together own something of each good, that is,L iEN w(i) > O. We do make this assumption although in this model there neednot be a clear distinction between input and output goods. A good may be anoutput good ofone transformation technique while it is an input good of another

THE OWEN SET AND THE CORE OF SIL PRODUCJION SITUATIONS 375

technique. A transformation technique is described by a vector a E R M, forexample

if M contains four goods . Positive elements in such a vector a indicate thatthe corresponding good is an output of the transformation technique, negativeelements indicate input goods and zero means that the corresponding good isnot used in this technique. In this example, the first and third good are outputsof the transformation process, the fourth good is an input and the second goodis not used. More precisely, 3 units of the fourth good can be transformed into5 units of the first good and 1 unit of the third good. We assume that eachtransformation technique uses at least one good to produce another good, so, itcontains at least one positive and one negative element.

Denote by D, the finite set of transformation techniques of producer i EN.Then k E D; means that producer i can use technique ak • The set of alltransformation techniques is D = UiENDi. We assume that all producersare price-takers and that all goods can be sold at the exogenous market pricesp E R~ \ {O}. All transformation techniques are linear, so, 2ak means thattwice the amount of input is used to produce twice the amount of output withtechnique k. The factor 2 is called the activity level of technique k. Denoteby Y = (YkhED the vector of activity levels. Because we cannot reverse anytransformation process, all activity levels are nonnegative. The transformationmatrix A E R M x D is the matrix with transformation technique ak at columnk, Related to this is the matrix G E R~x D that describes which and how manyof the goods are needed as inputs in the various transformation techniques. Forall j E M and kED we have Gjk = gj = max{O, -aj}. From this it follows

that (ak + gk)j = max{a1, O}, so the vector ak + gk describes which and howmany of the goods are outputs in technique k . Thus, when technique k hasactivity level Yk ::::: 0 then the vector gkYk describes the amount of input goodswe need and (ak + gk)Yk describes the output of this transformation technique .

Consider first a single producer i E N. He should choose his activity vector Y such that the amount of goods he needs does not exceed the amountof goods he owns, Gy ::; w(i). Furthermore, this producer can only use hisown transformation techniques. Therefore Yk = 0 if k fj. Di , The amount ofoutput of the transformation techniques will be (A + G)y. We see that theproducer started with w(i) from which he uses Gy as inputs and he obtains(A + G)y as outputs, so he can sell the goods that remain after the transformation, w(i) - Gy + (A + G)y = w(i) + Ay, on the market. His goal is to


maximize his profit pT(w(i) + Ay) such that the activity vector y is feasible:

max{pT(w(i) + Ay)1Gy ~ w(i), y ~ 0, Yk = 0 if k fj. Di}.

Producers are also allowed to work together. When they cooperate then theywill pool their techniques and their bundles of goods. A coalition SeN ofproducers has the bundle w(S) = 2:iES w(i) at its disposal and it can use allthe transformation techniques in D(S) = UiESDi. The profit maximizationproblem of such a coalition is similar to that of a single producer and equals

max{pT(w(S) + Ay)1 Gy ~ w(S), y ~ 0, Yk = 0 if k fj. D(S)}.

When we want to determine the dual problem of this profit-maximization problem then the last constraint, Yk = 0 if k fj. D(S), gives some trouble. We willreplace this constraint by another one with the same interpretation. For this,define for all kED and SeN, S =1= 0

{3(S)k = {(X) ,k E D(S)o ,k fj. D(S).

This vector {3(S) gives an upper bound for the activity vector that can be chosenby coalition S and it implies that

{Yk = 0 if k fj. D(S) {::} { Y ~ {3(S)Y ~ 0 Y ~ O.

The (primal) maximization problem Ps for coalition S can thus be rewritten to

Ps: max{pT(w(S) + Ay)1Gy ~ w(S) , y ~ {3(S), y ~ O}.

Because of the vector {3(S) it is now very easy to determine the dual programDs of Ps (cf. [9]):

. {( )T () T ( ) I GT

ZM + ZD > ATp, }Ds: mm zM. + p w S + zD{3 S ZM ~ 0, z;;~ 0 .

The vector ATp E R D denotes the profits for all transformation techniquesper activity level. The matrix G is denoted in units of goods per activity level.Therefore, the vector ZM E R M is denoted in units of dollars per good an thevector ZD E R D in dollars per activity level. A nice interpretation for the vectorZM follows from the complementary slacknessconditions: if fl, ZM and ZD areoptimal solutions of the primary and dual programs of coalition S then

o= zt[w(S) - GfJ] ,

o= zb[{3(S) - fl](4.1)

(4.2)


and

Equation (4.1) is equal to EjEM ZM,j(W(S) - G'O)j = O. This sum of nonnegative elements is zero if and only if each element equals zero. So, for all goodsj EMit holds that ZM,j(W(S) - G'O)j = O. IfZMJ > 0 then w(S)j = (G'Okthe available amount of good j is precisely enough to cover the amount ofgoodj that is needed. From the objective function of the dual program Ds it followsthat an extra unit of good j will raise the profit by ZM because duality theorysays that the optimal values of Ps and Ds are equal. If, on the other hand, theamount of good j available is too large, w(S)j > (G'O}j, then ZM,j = 0: anextra unit of good j will not raise the profit. We can therefore think of ZM as thevector of prices that the coalition S of producers is willing to pay for an extraunit of the goods. We will call the vector ZM +p the vector of shadow pricesfor the goods of this coalition. The following theorem shows a nice result thatfollows from (4.2) .

Theorem 4.1 The equality i{;{3(S) = 0 holdsfor all optimal solutions(ZM,ZD) ofDs.

Proof. Because the set of feasible solutions of D s is the nonempty intersectionof a finite number of halfspaces that is bounded from below by the zero-vector,the program Ds can be solved and a minimum exists. Let (ZM, ZD) be anoptimal solution. By the complementary slackness conditions equation (4.2)holds and is equal to EkED ZD,k({3(S) - 'O)k = O. Again, this is a sumof nonnegative elements, so it should hold that ZD,k({3(S) - 'O)k = 0 for alltransformation techniques kED. IfZD,k > 0 then (3(S)k = 'Ok' The definitionof(3(S) implies that in this case (3(S)k = 0, so k fj. D(S). If (3(S)k > 'Ok, whichis equivalent to k E D(S), then ZD,k = O. We conclude that zD,k{3(S)k = 0for all transformation techniques kED. 0

For ease of notation let Fps and FdS denote the sets of feasible solutions ofrespectively the primal and the dual program for coalition S,

Fps = {y E RDIGy :S w(S), y :S (3(S), y ~ O},

FdS = {(ZM,ZD) E RM x RDIGTZM+ ZD ~ ATp, ZM ~ O,ZD ~ O},

denote by ups and UdS the optimal values of the programs,

UpS = max{pT(w(S) + Ay) 1y E Fps},

UdS = min{(zM + p)Tw(S) + Z];{3(S)I(ZM, ZD) E FdS} ,


and let Ops and OdS be the sets of optimal solutions,

OpS = {y E FpslpT(w(S) + Ay) = ups},

OdS = {(ZM, ZD) E FdSI (ZM +p)Tw(S) + zbf3(S) = UdS}'

The sets Fps, FdS, Ops and OdS are nonempty and the values ups and UdSexist and are finite. By duality theory it holds that ups = UdS for any coalitionS of producers.

An LTP situation will be described by the tuple (N, A, D, w, p) where w =(W(i))iEN' Corresponding to an LTP situation we define two cooperativegames. The first one, (N ,vp ) , is the LTP game as defined in Timmer, Borm andSuijs ([7]) where vp(S) = ups, the maximal profit that coalition S can obtain .The second one is the dual game (N, Vd) that gives each coalition S the valueof its dual program, Vd(S) = UdS.

The core of an LTP situation, Core(A,w,p), is defined as the core of an LTPgame, Core(A,w,p) = C(vp). Furthermore, we know that for all (ZM,ZD) E

°dN

vp(N) = vd(N) = (ZM +p)Tw(N) + zbf3(N) = (ZM +pfw(N),

where the last equality follows from theorem 4.1. Timmer, Borm and Suijs ([7]) show that ((ZM +p)TW(i))iEN E C(vp). Thus it follows from OdN =/: 0that LTP games are totally balanced : each LTP game has a nonempty coreand because each subgame is another LTP game, these subgames also havea nonempty core. Although G. Owen did not show that you can find a coreclement ofanLTP game via the dual program DN, we let Owen(A,w,p) denotethe set of all core-elements that we can find in this way:

From OdN =/: 0 it follows that Owen(A,w,p) =/: 0 and Owen(A,w,p) CCore(A, w,p) .

The following example of an LTP situation with its two corresponding gamesillustrates the concepts introduced in this section.

Example 4.1 Consider the following LTP situation. There are two producers, N = {l, 2}. They work with two goods in their transformation techniquesand

A = [-; -~] , w(l) = w(2) = [ ~ ] , p = [ ~ ] .

The first column of A contains the technique of producer 1 and the secondcolumn contains the technique of the other producer, so, D, = {i}, i E N.


When each of the producers works on her own then she will transform hersingle unit of the first good into respectively 2 and 3 units of the second good.This producer already owns a unitofthe second good and therefore vp ( {1}) = 3and vp ( {2}) = 4.

When the producers cooperate then they own w(N) = (2, 2)T and their setof feasible activity vectors is

Producer 2 has a more efficient transformation technique than producer I because it generates a larger profit from the same amount of input, namely 2dollars per activity level against 1 dollar per activity level for producer 1.

PN: rnax{4+Yl+2Y2IyEFpN}

The maximal profit UpN = 8 is attained in y = (0, 2)T, so OpN = {(0,2)T}.The core equals Core(A,w,p) = {(b, 8 - b)13 ~ b ~ 4}.

For the dual game (N, Vd) it holds that Vd( {1}) = 3 and Vd({2}) = 3. Theset of feasible solutions of D N is

FdN = {(ZM , ZD) E R~ x R~I ZM,l + ZD,l ~ 1, ZM,l + ZD,2 ~ 2}.

When we solve the program

DN: rnin{4 + 2ZM,1 + 2ZM,2 + OO(ZD ,l + ZD,2)1 (ZM' ZD) E FdN},

then we get 0dN = {((2,0), (O,O))} and UdN = 8 = UpN. Thus the Owenset consists of only one point, Owen(A,w,p) = {(4,4)} and is contained inCore(A, w,p)

5 SEMI-INFINITE LTP SITUATIONS

In this section we will study semi-infinite LTP situations where the set Dcontains a countable infinite number of transformation techniques. Without lossof generality we assume that D = {1, 2, 3, ... }. A semi-infinite LTP situation(N, A,D,w,p) thus has A E R M x D, w(i) E R~ for all i EN andp E R~.

As opposed to the previous section, we do not put any further restrictions on A,wandp.

Because of the infinite number of transformation techniques, the linear programs that determine the maximal profits of the coalitions and their dual programs are semi-infinite linear programs. Therefore, we will replace the maximum by the supremum in the definitions of Ps and ups and the minimum willbe replaced by the infimum in the definitions of Ds and UdS. As opposed tofinite LTP situations, the set of feasible dual solutions Fds may now be emptyand the same holds for the sets of optimal solutions Ops and OdS' The two

games (N, vp) and (N, Vd) are defined as before, so, vp(S) = ups for the LTPgame and Vd(S) = UdS for the dual game. In this semi-infinite situation thevalues ups and UdS can take any nonnegative value as well as +00.

The Owen set, as defined in the previous section, is based on the dual programfor the grand coalition:

. { T) T ( ) ICT

ZM + ZD > ATp, }DN: mf (ZM + p) w(N + zD{3 N ZM ~ 0, Z;;~ 0 .

In our definition of the Owen set we use that for finite LTP situations it holdsthat zb{3(N) = 0 for any optimal solution (ZM'ZD) of DN. But this propertyneed not hold for semi-infinite LTP situations. When UdN = 00 then an optimalsolution (zM, ZD) (if it exists) has zb{3(N) = 00 but when UdN < 00 thenzb{3(N) = O. For this reason we will define the Owen set only if UdN < 00:

Owen(A,w,p) = {((ZM + p)TW(i»iENI (ZM' ZD) E OdN}.

The next theorem states some nice properties of the LTP and dual games.

Theorem 5.1 Let (N, A, D, w, p) be a semi-infinite LTP situation. Then

(a) Fds=FdN!orallSCN;

(b) vp and Vd are monotonic games; and

(c) vp(S) 5, Vd(S)!orall SeN.

Proof. For the first item, by definition Fjj, = {(ZM' ZD) E R~ xRSI CTZM+ZD ~ ATp} = FdN for all SeN.

To show the second item, let SeT c N, then w(S) 5, w(T) and (3(S) 5,(3(T). So, Fps = {y E RDI Gy 5, w(S), y 5, (3(S), y ~ O} C FpTand therefore vp(S) = SUp{pT(w(S) + Ay)1y E Fps} 5, vp(T) . From thefirst part of this proof it follows that FdS = FdT and together with w(S) 5,w(T) and (3(S) 5, (3(T) this implies that Vd(S) = inf{(zM + p)Tw(S) +zb{3(S) I(ZM,ZD) E FdS} 5, vd(T).

Finally, for the third item, let S be a coalition of producers. If FdS = 0then vp(S) 5, 00 = Vd(S), Otherwise, take feasible solutions y E Fps and(ZM,ZD) E FdS. ThenpT(w(S) + Ay) = pTw(S) + yTATp 5, pTw(S) +yT(GTZM+ZD) = pTw(S)+z'frGy+zbY 5, pTw(S) +z'frw(S)+zT(3(S) =(ZM+p)Tw(S)+zb{3(S) and from this it follows thatu, (S) = sup{pF(w(S)+Ay)1 y E Fps} ~ inf{(ZM +p)Tw(S) + zb{3(S) I(ZM,ZD) E FdS} = Vd(S) ,o

Some relations between the Owen set and the cores of the LTP and dualgames are stated below.

THE OWEN SET AND THE CORE OF SIL PRODUCTION SlTUA110NS 381

Theorem S.2 Let (N, A, D, w,p) be a semi-infinite LTP situation. Thenthefollowing two relations hold:

(a) ljvd(N) < 00 then Owen(A,w ,p) C C(Vd)'

(b) ljvp(N) = vd(N) then C(Vd) C C(vp).

Proof. For item (a), if Owen(A, w, p) = 0then the result holds . Otherwise, letx E Owen(A,w,p). Then there exists a solution (z~,z~) E 0dN such thatXi = (z~ + p)Tw(i) for all i E N . By definition, L:iEN Xi = L:iEN(z~ +p)Tw(i) = (z~ + p)Tw(N) = (z~ + p)Tw(N) + (z~)T(3(N) = vd(N),where (z~)T(3(N) = 0 because vd(N) < 00. Second, (z~)T(3(N) = 0implies (z~)T(3(S) = 0 because (3(N) ~ (3(S). Also, (z~, z~) E OdN CFdN = FdS, where the last equality follows from Theorem 5.1(a). Thus,L:iES Xi = (z~ + p)Tw(S) = (z~ + p)Tw(S) + (z~)T(3(S) ~ inf{(zM +pfw(S) + zI;{3(S)I(ZM,ZD) E FdS} = Vd(S), Hence, X E C(Vd) '

For item (b), if C (Vd) = 0then we are done. Otherwise, take an element X EC(Vd). By definition it holds that L:iEN Xi = vd(N) = vp(N) . Furthermore,L:iES Xi ~ Vd(S) ~ vp(S) by Theorem 5.1(c). We conclude that x E C(vp). 0

A consequence of this theorem is that if vp(N) = vd(N) < 00 thenOwen(A,w,p) C Core(A,w,p) . We can now have Owen(A,w,p) = oeven ifvp(N) = vd(N), as the following example shows.

ExampleS.! Consider the semi-infinite LTP situation (N,A,D,w,p) ,where N is a set of one player,

Then

{TTl C

TZM + ZD > ATP }vd(N) = inf (ZM + p) w(N) + zD{3(N) ZM ~ 0, ZD ~ 0 '

{

k2zM,1 + ZM,2 + ZD,k }= inf ZM,l + 2ZM,3 + 5 + 00 L ZD,k ~ 2k, k = 1,2, ... ,

kED ZM ~ 0, ZD ~ 0

= 5,

where FdN i= 0, but OdN = 0 and this implies that Owen(A, w,p) = 0. Thereis no duality gap in this example because vp(N) = 5 = vd(N) .

In case of a duality gap, vp(N) < vd(N) , another relation between the Owenset and the core exists.

Theorem 5.3 Let (N, A , D ,w, p) be a semi-infinite LTP situation wherevp(N) < vd(N) < 00. Then Owen(A,w,p) n Core(A ,w,p) = 0.

Proof. The proof of this theorem goes analogously to the proof of the first partin Theorem 3.3. 0

Finally, we obtain the same result as for semi-infinite LP, namely, that ifvp(N) is finite in a semi-infinite LTP situation then the core is nonempty. Forthis, we need two intermediate theorems. The first one is a theorem by Karlinand Studden ([3]), which we translated to semi-infinite LTP situations.

Theorem 5.4 Suppose that vp(N) is finite and that Wj(N) > 0 for allj E M. Then there is no duality gap, vp(N) = vd(N), and the dual programD N has an optimal solution.

The second intermediate theorem shows that we have no duality gap, vp (N) =vd(N), and C(vp) =I- 0 if certain conditions hold.

Theorem 5.5 Ifw(N) E R~\ {O}, p E R~\ {O}, wj(N) = 0 =? gj = 0for all kED, pTak > 0 for all kED, ak fj. R~ for all kED andvp(N) < 00, then vp(N) = vd(N), OdN =I- 0 and C(vp) =I- 0.

Proof. If wj(N) > 0 for all j E M then together with vp(N) < 00 andTheorem 5.4 it follows that there is no duality g~ and there exists an optimaldual solution 2. Define x E R N by X i = (2 + p) w(i) for all i E N. We leaveit to the reader to show that x E C (vp ) .

If wj(N) = 0 for some j E M then define Mo = {j E M Iwj(N) = O}and M+ = {j EM Iwj(N) > O}. Then Mo =I- 0 and M+ =I- 0. Now theprimal problem ean be rewritten to

and similarly, we obtain for the dual problem

where we observe that the assumptions imply that for all kED there exists aj E M+ such that gj > O. Thus, the latter problem is feasible. Let ej denote

the jth unit vector in R M + , with e{ = 1 if 1= j and e{ = 0 otherwise. Define

THE OWEN SET AND THE CORE OF SILPRODUCTION SITUATIONS 383

the cone K 1 by

K 1 = cone (({gj} . ) , (ei) "EM) = R~+ ,JEM+ kED J +

where the last equality follow s from gj ~ 0 for all j E M +, kED. But then

{Wj(N)} jEM+ E int(Kt} = R~.t ,

where int (Kt} denotes the interior of the cone K 1, because wj(N) > 0 for allj E M+ . Together with vp(N) < 00 and Theorem 5.4 it follows once againthat vp(N) = vd(N) and there exists an optimal dual solution Z. To obtainan clement of the core C(vp ) , we define !fj = Zj for all j E M+ and !fj = 0

otherwise. Also, define x E R N by Xi = (!f+ p)T w(i) for all i E N. First,

LXi = L (!f+ pf w(i) = (!f+ p)T w(N)iE N i EN

= L zjwj (N ) + pTw(N) = vd(N) = vp(N).jEM+

Second, let SeN, S =1= 0, be a coalition of players. Notice that wj(N) = 0for some j E Mo implies that Wj(S) = 0 for all SeN because w(S)L:iES w(i). Then,

(!f+ p)T w(S) = pTw(S) + L ZjWj(S )jEM+

~ pTw(S) + inf { L ZjWj (S ) I L:jEM+ ~jZj ~ pT ak,

kED , }j EM+ Zj ~ 0, J E M +

= pTw(S) + sup {L pTakYk I L:.!;Efj 9jYk :s Wj(S) , j E M +, }kED Y-

~ pTw(S) + sup {pT Ay IGy :s w(S); Yk = 0 if k ~ D(S); Y ~ o}= vp(S).

We conclude that L:iES X i = (!f+ p)T w(S) ~ vp(S) and hence, X E C(vp). 0

With the help of these two theorems we prove our main result about semiinfinite LTP situations, which states that if there exists a finite upper bound forthe maximal profit that all producers together can obtain then the core of theLTP game is nonempty.

Theorem 5.6 Let (N, A, D ,w,p) be a semi-infinite LTP situation and let(N, vp) be the corresponding LTP game with vp(N) < 00. Then C( vp) =1= 0.


Proof. In this proof, we consider one-by-one all the possible semi-infinite LTPsituations that we may come across. In each of these situations we show thateither vp(N) = 00 or C(vp) t=- 0.

First, suppose that w(N) = O. This implies that w(S) = 0 for all coalitionsS. No coalition of producers can transform any goods or sell any on the market.Hence, vp(S) = 0 for all Sand C(vp) = {(O,... ,O)}.

Second, consider the situation where w(N) t=- 0 but every transformationtechnique k needs a good j for which wj(N) = O. Let h(t) describe for allbundles of goods t E R~ those goods that are available in a positive quantity, so,h(t) = {j EM I tj > O}. Then h(w(N)) 1J h(gk) for all kED means thateach technique k needs some unavailable goods. Consequently, no coalition Scan transform any goods. The only thing it can do is sell its goods at the marketand obtain vp(S) = pTw(S). From w(S) = LiES w(i) we derive that thecore consists of a single element, C(vp) = {(pTw(l), ... ,pTw(n))}, whereN={1,2, . .. ,n}.

Assume now that w(N) t=- 0 and that some transformation techniques canbe used because they only need goods that are available, h(w(N)) :J h(gk)for some kED. All the coalitions of producers want to maximize theirprofit and therefore they will restrict their transformation to those techniquesthat can be used because the other techniques will not generate any profit.Therefore, without changing the values of the coalitions we remove all thetransformat ion techniques k for which h(w(N)) 1J h(gk). If this removalimplies that D(S) = 0 for some coalition S then define vp(U) = pTw(U)for all U C S. For convenience, let (N, A, D ,w,p) also denote this reducedsemi-infinite LTP situation.

This leads us to the next situation where w(N) t=- 0, h(w(N)) :J h(gk) forall kED, and also p = O. If all the goods have a price of zero then vp(S) = 0for all coalitions S and consequently, C(vp ) = {(O, ... ,O)}.

Ifw(N) t=- 0, h(w(N)) :J h(yk) for all kED, p t=- 0 and pTak :s 0 for allkED then no transformation technique gives a positive profit. For all optimalsolutions Y E Ops it holds that pTakYk = 0 for all techniques k. Hence,vp(S) = pTw(S) for all coalitions Sand C(vp) = {(pTw(l) , . . . ,pTw(n))} .

Now assume that w(N) t=- 0, h(w(N)) :J h(gk) for all kED, p t=- 0 andpTak > 0 for some kED. In the previous situation we have seen that ifpTak :s 0 then in the optimum pTakYk = O. This technique k will not haveany influence on the profit and so, removal of these techniques will not change

THE OW/!'"N SET AND THE CORE OF SIL PRODUCTION SITUATIONS 385

the values of the coalitions. Also in this case, we define vp(U) = pTw(U) forall U C S if the removal implies that D(S) = 0.

In the next situation, we consider w(N) i- 0, h(w(N)) :J h(gk) for allkED, p i- 0, pTak > 0 for all kED and ak E R~ for some kED. Noticethat for this technique k we have ak E R~ \ {O}, because ak = 0 impliespTak = 0, which is in contradiction to pTak > 0. If ak E R~ then gk = 0,which means that technique k needs no input goods to generate the positiveprofit pTak . Consequently, the coalition N of all players will set the activitylevel Yk to infinity and so, vp(N) = 00 . The total profit is infinitely large. Wemay say that we are in heaven because all the producers can take as much ofthe profit as they want.

Finally, we consider w(N) i- 0, h(w(N)) :J h(gk) for all kED, p i- 0,pTak > 0 for all kED and ak fI. R~ for all kED. Notice that pTak >°implies that ak fI. R~ for all kED. Together with ak fI. R~ we getthat each vector ak contains at least one positive and one negative element.Each transformation technique needs at least one input good to produce at leastone output good. Now, two situations may occur. Either we have vp(N) =vd(N) = 00, heaven once again, or vp(N) < 00. In the latter case, Theorem5.5 shows that the core is a nonempty set. 0

6 CONCLUSIONS

We studied the Owen set, the core and relations between these two setsof two types of semi-infinite situations. These are situations involving linearproduction (LP) and those involving the linear transformation ofproducts (LTP).We showed that ifthe underlying primal and dual problems ofthe grand coalitionof players have the same value, that is, there is no duality gap, then the Owenset is a (possibly empty) subset of the core. Otherwise, the Owen set and thecore have nothing in common. In the case of LTP situations we had to excludesituations where the underlying dual problem takes the value infinite. Finally,we showed that if there exists a finite upper bound of the maximal profit thenthe core is a nonempty set.

After completing this study, some questions remain. Throughout the paperwe use cones consisting of real numbers like R N andR~. What would happenif we replace these cones by more general cones? How do the results changeif we consider an infinite number of producers (implying an infinite numberof production techniques)? And finally, what happens if we assume that theset of production techniques is no longer countable? We intend to study thesequestions in the near future.


Acknowledgment

Judith Timmer acknowledges financial support from the Netherlands Organization for Scientific Research (NWO) through project 613-304-059.

References

[1] V. Fragnelli, F. Patrone, E. Sideri, and S. Tijs . Balanced games arisingfrom infinite linear models, Mathematical Methods ofOperations Research,50:385-397, 1999.

[2] J.R .G. van Gellekom, lA.M. Potters, J.H . Reijnierse, S.H. Tijs, and M.e.Engel. Characterization of the Owen set of linear production processes,Games and Economic Behavior, 32:139-156, 2000.

[3] SJ. Karlin and WJ. Studden . Tchebycheff Systems: with Applications inAnalysis and Statistics, Interscience Publishers, 1966.

[4] G. Owen. On the core of linear production games, Mathematical Programming,9:358-370, 1975.

[5] D. Samet and E. Zemel. On the core and the dual set of linear programminggames, Mathematics ofOperations Research, 9:309-316, 1984.

[6] S.H. Tijs. Semi-infinite linear programs and semi-infinite matrix games,. Nieuw Archiejvoor Wiskunde, 27:197-214, 1974.

[7] J. Timmer, P. Borm, and J. Suijs. Linear transformation of products:games and economics , Journal of Optimization Theory and Applications,105:677-706, 2000.

[8] J. Timmer, S. Tijs and N. Llorca. Games arising from infinite productionsituations, International Game Theory Review, 2:97-106, 2000.

[9] W.L. Winston. Operations Research: Applications and Algorithms.Duxbury Press, Belmont, 1994.

Semi-Infinite Programming: Recent Advances

Documents

Transcript of Semi-Infinite Programming: Recent Advances