NZOR Volume 12 Number 273

July 1984

SOLUTION OF QUADRATIC PROGRAMMING PROBLEMS

SUSAN J, BYRNE

MASSEY UNIVERSITY PALMERSTON NORTH, NEW ZEALAND.

SUMMARY

Quadratic programming problems arise in a number of situations. Typical examples, e.g. portfolio selection, economic modelling, regression analysis} solution of non-linear programming problems, are briefly described. The methods that have been proposed for solving this problem are reviewed. The approach using a linear complementarity program is selected, and an efficient} numerically well-behaved procedure is developed, based on Lemke's algorithm and using orthogonal factorizations. A principal pivot version is also presented, which parallels Dantzig and Cottle’s solution procedure, but is applicable to the same very wide class of matrices processed by Lemke 's algorithm. A restart facility is developed for this version} which accelerates the solution of a sequence of related quadratic programs, by proceeding from the optimum of the last problem. The methods have been coded in Fortran IV and perform well. The process is easily extended to handle both upper and lower bounds on constraints and has proved acceptable for use in a general non-linear programming algorithm.

INTRODUCTIONQuadratic programming (QP) has been used in the formulation

and solution of a wide variety of operational research problems.The general problem is to minimize a quadratic function of many variables subject to a set of linear equality or inequality con­straints and possibly constraints on variable values. Usually, authors consider a simplified version of this problem when develop­ing solution procedures in which the constraints are either all equalities or all simple inequalities and variables are all non­negative. Frequently however, the more general case with upper and lower bounds on variable and constraint values is the real problem, and an extension of the method developed in this paper handles it in a simple and direct manner, unlike many other solution methods.

In this paper a number of typical applications of quadratic programming are described. They cover a wide range of areas. It is clear that the technique is of importance, and useful, provided good efficient routines exist to solve the problems so formulated. The next section, therefore, includes a discussion of various solution methods that have been proposed. As most methods are based on the Kuhn-Tucker conditions, a set of necessary and some­times sufficient, conditions satisfied by the optimal solution, these are presented briefly for completeness. It is apparent that no one method is clearly superior but that methods based on solvingManuscript received January 1984, revised June 1984.

74

the equivalent linear complementarity problem appear promising.The two chief methods of solving this problem are described, and the second method due to Lemke (1968) is chosen both because it is simpler and it has a wider applicability. Two methods based on this technique are developed which use efficient and numerically good procedures to obtain solutions. A method of solving a sequence of related problems is developed which uses much of the information from the previous problem in the sequence. A discussion of the methods developed concludes the paper.

APPLICATIONSOne of the major advantages of quadratic programming (QP) over

a linear programming (LP) approach is that it allows the modelling and investigation of interactions between variables, such as a demand change resulting from a price change, both of which would obviously affect total profits (or costs). In LP, it has to be assumed that price is independent of demand.

A brief survey of abstracts in IAORS for the years 1972-1978 revealed the following range of applications. A classical appli­cation which has received considerable attention is the problem of portfolio selection, first discussed by Markowitz (1959). The usual example is the problem of selecting a set of investments in stocks such that the expected rate of return is maximized, subject usually to a specified maximum variance in it, where the return on each stock is not fixed but has both an expected value and a vari­ance. Alexander (1977) discusses the selection of a market port­folio from a given set of risky assets and a specified risk-free borrowing and lending rate. The problem was solved using a sequence of quadratic programs, with the solution of these being found from the associated linear complementarity problem. Ohlson (1977) con­siders the problem when the means and variances of return are infinite and derives some weak sufficiency conditions to obtain a quadratic approximation solvable by quadratic programming. McCarl (1977) discusses the general portfolio selection problem where a selection must be made among portfolios with uncertain outcomes such as the return on agricultural crops (Scott and Baker, 1972) or stocks (Sharpe, 1970).

McCarl shows that a quadratic programming approach is better than using LP as it tends to allow for portfolio diversification to reduce risk as it considers not only the expected value of the return but also the variance-covariance relationships between stocks. He also indicates that a quadratic programming formulation could be appropriate for multiple objective programming problems, as in general, the goals to be achieved are not independent, and the QP can consider interactions and tradeoffs between the goals.

Regression analysis is amenable to a QP approach. In classical least squares regression it is possible to include linear constraints specifying a priori information about the particular problem.Although this is possible in standard regression analysis it is easier to consider in a QP formulation. An advantage, too, in modelling the problem as a QP is that it is necessary to consider the total problem structure, usually yielding rather more insight

75into the problem. Dent (1973) discusses the problem of making a least squares fit of functions constrained to be either non­negative, non-decreasing or convex and shows how all three of these may be modelled as QP problems.

Classical supply and demand response in a market is another application which may be usefully modelled by QP. Samuelson (1952) and Dorfman (1951) showed how to do it, and since then many workers have made contributions to the subject, see Judge and Takayama (1973) for a review. A similar problem is that of diminishing returns to scale for which a QP formulation is very straightforward, with the objective function involving the maximization of returns less cost of production which is an increasing function of quantity.

A range of other problems were also solved using either one or a sequence of QP's. Warner and Prawda (1972) discuss the schedul­ing of nursing staff for a hospital, a problem posed initially as a mixed integer QP but solved via a multiple-choice programming master problem with QP sub-problems. Glassey (1978) built an in- put-output 21 sector model of the US economy, extended to include consumer demands, imports, and exports as endogenous variables.By assuming consumption was linearly related to prices he found an equilibrium state by solving a QP. He then analysed the effect of Federal policies implemented to mitigate the large effects of the quadrupling of world crude oil prices in 1973-74, and showed it to have been effective, at least in the short term. Hall et al (1975) undertook studies into spatial equilibrium in US agriculture.Other applications in agriculture include models for both crop and livestock selection, generally as variants of the portfolio select­ion problem. Chia Shun Shih (1976) made a study of relative assessment for water quality management on a regional basis.

Apart from these obvious applications uses of quadratic pro­gramming, much interest in the past few years has centred on the use of sequences of approximating QP's in the solution of general non-linear programming problems. Fletcher (1972) described one such procedure which used sequences of quadratic programming sub­problems. Other authors since, such as Mayne and Polak (1978) have also considered this approach, with differences lying mainly in the definition of the QP sub-problem. This is currently an active area of research and has prompted the investigation of good reliable methods for solving a QP. As yet most codes available cannot handle large QP problems but with sufficient work this problem should not be insurmountable.

SOLUTION METHODSThis section includes a brief review of methods used to solve

the quadratic programming problem, typically represented asmin f° (x) = gTx + -jxT H x

Tsubject to G x ^ b (1 )

x > 0where

76and without loss of generality H may be assumed symmetric.

Most of the methods proceed by finding a point that satisfies the Kuhn-Tucker conditions, see Kuhn and Tucker (1951). These are a set of necessary, and in some cases sufficient, conditions for a point x* to be an optimal solution to the QP problem. Uniqueness of x* can only be guaranteed if the objective function f®(x) is strictly convex, or equivalently, if H, the Hessian matrix of second derivatives, is positive definite. If H is only positive semi-definite then the optimum value is unique but the optimum point may not be (it may be a line of optimal value). However the point so found by solving the Kuhn-Tucker conditions is certainly not a saddle point. The Kuhn-Tucker conditions may be stated as follows:If x* G Rn satisfies the constraints

GT x* > b (2)x* _> 0

which include at least one interior feasible point, and minimizes the objective function

0 T I Tf (x*) = g x* + £ x*1 H x*, (3)then there must exist vectors y* G Rn and A* G Rm such that

g + Hx* = y * + GA*GT x* _> b, X* > 0x* >_ 0, y* _> 0

T rn yA* (G x*-b) = 0, y* x* = 0.

(4.a) (4. b) (4. c) (4.d)

If, in addition, H is at least positive semi-definite then any x* satisfying (4.a) - (4.d) is the desired optimal solution. The methods considered here are all developed for the convex case, although some, such as Beale's may be applied to the general case.

Wolfe (1959) developed a method based on the simplex method for linear programming to solve the convex QP subject to equality constraints. He generates by means of a modified simplex pivoting procedure a sequence of feasible points that terminates in a point x* which satisfies the Kuhn-Tucker conditions. The modification is to ensure that the complementarity condition yT x = 0 is always satisfied. With H only positive semi-definite, finite convergence to the optimum cannot be guaranteed, but is usually obtained. The procedure is of the same order as solving an m + n dimensional LP problem and although slack variables can be added for inequality - constrained problems, the increase in dimension makes this approach less attractive.

Beale (1955, 1959) developed a method from the arguments of classical calculus. It is applied to equality-constrained problems and uses essentially a reduced gradient approach implemented using

77the simplex method. The tableau size varies as dual variables are added to the system but compares favourably with that used in Wolfe's method.

Dantzig (1961) and Van de Panne and Whinston (1964) both de­veloped another method using the simplex technique, and proceeding through sequences of "standard" and "non-standard" tableau. In the former, the complementarity conditions x = 0 and AT(GT x-b) = 0 are satisfied, whereas in the latter they are violated by one pair of primal-dual variables. The basis entry and exit rules are suitably modified to finally obtain an optimal point that also satisfies the Kuhn-Tucker conditions. The method is applicable only to convex problems with inequality constraints. The size of tableau is similar to that used in Wolfe's approach.

Methods based on active-set strategies have been developed by Goldfarb (1966, 1968, 1972) and Fletcher (1971). They may be con­sidered as extensions of Newton's method for minimizing an uncon­strained quadratic function. They are useful but computationally involved. Further development of these approaches has been report­ed by Best and Ritter (1976) and Gill and Murray (1978) .

All these methods require that first an initial feasible solution be found satisfying the constraints. Since this frequent­ly requires much computation (often, the solution of an LP problem), methods that avoid this have been investigated. Thiel and Van de Panne (1960) and Lemke (1962) both produced methods that started from the easily-obtained unconstrained minimum of the objective function and then proceeded to add constraints in such a way that finally a point x* was found which not only satisfied the con­straints but also gave a minimum for the objective function.Thiel and Van de Panne's method required more work than Lemke's, as they discarded all the prior information at each iteration. Neither however performed well if more than a few constraints were binding, i.e., satisfied exactly, at the optimum. Both techniques are applicable only if f® (x) is strictly convex.

A recent algorithm which also avoids the problem of maintaining feasibility is the dual approach of Goldfarb and Idnani (1983) , which uses Goldfarb's (1972) primal feasible-point algorithm on the dual of the quadratic program. An initial dual feasible point is easily obtained (the unconstrained minimum) and dual feasibility is readily retained. The description and implementation is some­what complicated, particularly as it is given in terms of the primal problem. Powell (1983) reported an extension of the method to the positive semi-definite case. Encouraging numerical results have been reported. However, as noted in Pang (1983) it is not yet clear how this approach compares with the method of primary interest in this paper, which is to solve the associated linear complement­arity problem using a simple pivotal scheme.

Lemke (1965) and Dantzig and Cottle (1967) , among others, both recognised that the Kuhn-Tucker conditions (4) for an optimal solution to the QP could be expressed as finding the solution to a linear complementarity problem, LCP as follows:

78

Find w, z £ Rm+n such that

where

w = Mz + qw > 0, z > 0w z = 0

m r :For quadratic programming, w, z, M and q are:

(5.a) (5. b) (5. c)

with

V 'h -g‘ X g, M = , z = q = (6)

f TG 0 A -bf = G x - b.

The linear complementarity problem is of interest both as a means of solving QP problems and also for solving such problems as finding the Nash equilibrium point for a bimatrix game, see Lemke and Howson (1964) and Cottle and Dantzig (1968), or examining equilibrium models for an open economy, see Mathiesen and Hansen (1979) and Scarf (1973). Methods of solving this LCP problem are now considered.

SOLUTION OF THE LINEAR COMPLEMENTARITY PROBLEMTwo chief methods have been developed to solve the LCP problem,

the difference lying in the pivotal scheme employed. The pair of variables (w^,x^) is called a complementary pair. If w and z satisfy both (5.b) and (5.c) they are complementary and feasible, and at least n of the w. and z. must be zero - the non-basic variables - with the remaining n - the basic variables - being either zero or positive. A pivotal scheme consists of a sequence of pivots between two variables w s and z , with some criterion that specifies the pivot pair at each step and a termination criterion.

In the method proposed by Dantzig and Cottle (1967), and Cottle (1968), a principal pivot scheme is used, that is, a pivot between the two members of one complementary pair. The idea is generalized to a block principal pivot to ensure that it can always be perform­ed. The method is a finite sequence of principal, possibly block, pivots. Each block pivot is a finite sequence of simple non­principal pivots. An initial feasible solution is not required as the method proceeds by successively reducing the number of negative Wj_ and z^, whilst retaining w^ z^ = 0 for each pair.

The method may be paraphrased as follows. Select the driving variable as the complement of the distinguished variable, which is any negative variable w ^ . Increase z^ from zero until it is blocked, i.e. either some positive basic variable is reduced to zero or the distinguished variable increases to zero. Replace the blocking variable by the driving variable, and in the latter case terminate the major cycle. Otherwise, let the complement of the

79

blocking variable be the new driving variable and increase it until a new blocking variable is found. Then pivot as above, and continue till the major cycle is completed. If all basic variables are now positive, then stop, else select another distinguished variable and commence the next major cycle.

The method applies to the problem with M positive definite and must be slightly modified for M positive semi-definite, as in con­vex quadratic programming. The theoretical approach of using block principal pivots is attractive, but results in a complicated implementation using major and minor cycles, and a method that applies only to a restricted class of M.

The method proposed by Lemke and Howson (1964), Lemke (1965), and Lemke (1968), is both simple and applicable to a wide class of matrices, M, and is the approach used here. Lemke augments the problem (5) by the addition of a non-negative artificial variable Zq and positive column Mg thus:

w = Mg Zq + Mz + qw > 0, z > 0, zQ > 0 (6)

wT z = 0, M q > 0.Obviously any solution of (6) with Zg = 0 is a solution of (5)

and the method proceeds to find such a solution. In such a solu­tion, the vectors w and z must have in total n zero elements, so they can be partitioned thus:

W 1 Z1w = , z =

W 2 Z2

w^ = 0, z^ _> 0W£ ^ 0, Z£ = 0 ,

and the problem reduces to finding the appropriate partition.On the first iteration, set z = 0 so w = MqZq + q and increase

Zg until w _> 0, with wr = 0. Pivot wr and Zg, and then use the complement, zr , of wr as the driving variable, increasing it from zero until some other variable decreases to zero. If no variable does, then Lemke (1968) and Eaves (1971) proved for a very wide class of matrices that (5) had no solution. Otherwise, if z q is first reduced to zero, by pivoting it with the driving variable the solution of (5) is obtained. Otherwise a new blocking variable is found and pivoted with the driving variable. The procedure is then repeated, and will terminate.

The full matrix M need not be transformed at each iteration, as only the transformed driving column, M'c and q-vector, q', are required at each iteration. The details are given in Appendix A 1 . By examining the partition of w, z, M and q at any iteration it

80becomes clear that it suffices to have some representation of a portion of M, that corresponding to w variables reduced to zero and z variables positive, available that allows solution of equa­tions such as x = P, in order to obtain the transformed driv­ing column and q-vector. Performing the pivot operation then reduces to updating this representation of M ^ by the addition or deletion of a row and column and then the computation of M' andq' •

Two possibilities for representing M-^ can be considered.One is to update the inverse M j- consecutively and the other is to update some factorization of M-q . It is well-known that the first procedure results in a poorly-behaved and often unstable numerical system. Two factorizations are possible, either triangular or orthogonal. Triangular factorization is only acceptable numerically if the order of pivoting to obtain it can be selected so as to minimize round-off errors. This is not possible here as the pivot order is specified by the solution order of the LCP. _Thus orthog­onal factorizations are used. They are of the form V M y = R where R is unit upper triangular and V is a square matrix sucn that v Td v = I for some positive diagonal matrix D. These behave very well numerically, see e.g. Wilkinson (1965), and can be updated from iteration to iteration using generalizations of techniques developed by Gill, Golub, Murray and Saunders, (1974) and Gill, Murray and Saunders (1975), using ideas mentioned briefly by Sargent (1978), and developed fully in Byrne (1980). It may be noted that only D and R need be retained so that the storage re­quirements are not excessive.

Using these techniques then, an efficient routine was developed to solve the LCP. Two possible disadvantages it has are that an apparent singularity can arise in the matrix R after some iteration. This will be removed at the next iteration, and in any case it is not serious, but just adds complications to the procedure. The more serious disadvantage may be seen by_noting that the artific­ial column Mg forms the first column of M^^, and thus forms an integral part of R. Any change in Mg will then involve a recompu­tation of R, which, although possible, is time-consuming. Such a situation arises in the solution of sequences of related quadratic programs e.g. in some non-linear programs. It seems desirable therefore to consider a principal pivot version of Lemke's routine, in which M q is retained separately, allowing easy restart, and no more difficulties arise in handling singular factorizations than already occur in the implementation outlined above.

A PRINCIPAL PIVOT VERSION OF LEMKE'S METHODPrincipal pivot methods such as that of Dantzig and Cottle

(1967) involve sequences of major and minor cycles and apply only to restricted classes of matrices. These requirements are neces­sary to avoid singular pivots. A procedure is developed here, following ideas suggested but dismissed in Sargent (1978), that uses only elementary principal pivots, thus remaining simple, but which is still applicable to the same class of matrices as Lemke's method. Its success lies in the use of orthogonal factorizations that exist even if M ^ is singular. Because of the relationship

81to Lemke1 s algorithm it can be guaranteed, that even if issingular, its nullity is only one.

In order to develop the method, consider the Lemke algorithm. Suppose that wr is found as the first blocking variable. Provided Mrr # 0, wr may be pivoted with zr , a principal pivot. To obtain the blocking variable for the next iteration would require Zg and zr to be pivoted to obtain the transformed column M'c and trans­formed q, q'. However, these two vectors can be easily represented in terms of the transformed principal pivot system so that the final pivot is not needed. A partitioning of the system may be made similar to that for the Lemke system and it is then seen that the required transformed vectors M'c and q 1 are easily obtained.

In fact, the procedure uses orthogonal factorizations which are arranged in such a way that if j is singular, the correspond­ing diagonal zero in the triangular matrix R occurs in the final row and column. As the maximum nullity of is one, there is at most only one such diagonal zero in R. Determining the transformed vectors for the test for the blocking variable can be presented in part as finding an expression for m of the m + 1 variables z q , z ;l in terms of one of them using the m equations

R0Z0 + Rzi + Rq = 0where R q and R are vectors. Even if Rj^ is zero, the process can be initiated using:

z„ = 0.z - R /R n •0 m mq mOColumn interchanges are used throughout this routine to ensure maximum numerical stability.

A principal pivot routine using the equivalent of elementary principal pivots but implemented using orthogonal factorizations may then be specified, and as it is another interpretation of Lemke1s routine it will process the same wide class of matrices. Again, the details and full development of the procedure are given in Byrne (1980).

A RESTART PROCEDUREIn many applications of quadratic programming it is necessary

to solve a sequence of quadratic programs in which neither H, the Hessian matrix nor GT , the constraint matrix changes. The changes are confined to the linear portion, g, of the objective function and the right hand side, b, of the constraints. Inspection of the equivalent LCP formulation (6) indicates that this corresponds to retaining M but changing q to q. Provided the changes in q are not too large, it is to be expected that the solution of the new problem should be similar to that for the old, at least in the sense that the optimal partition of w and z would be similar. Thus it would be useful to commence the solution of the new problem from the current partition of w and z with w^ = 0, Z2 = 0 rather than the usual initial partition with all z = 0. The matrix factoriz­ation R is available. Note that as the final iteration of the previous problem involved pivoting M^g out °f t*16 basis, a non-

82singular principal pivot is left and hence both R and M^i must be non-singular and invertible. All that needs to be done then is to obtain a new artificial column M q and vector R q = VM q and new transformed q-vector. These vectors may be readily obtained by solving a set of equations of the form r Td Rcji = -M^q^, and then computing vectors 5c such that x = x - M 2^y, and x = -M;q x . The details are given in Appendix A 2 . Having obtained the required vectors, it is a simple matter to reinitiate the Lemke procedure and continue the solution process from the previous optimal partition of w and z.

This routine is particularly useful in a number of solution methods for nonlinear programming problems which result in the solution of sequences of quadratic programs. Wilson (1963), Han (1976), Powell (1978), Garcia-Palomares and Mangasarian (1976), and Mayne and Polak (1978) have all proposed such methods.

DISCUSSIONThe two methods described in sections 4 and 5 were both imple­

mented using Fortran on a CDC Cyber 174 at Imperial College, London. ANSI standard Fortran was used as far as possible to ensure porta­bility of the programmes. The simple restart procedure was also implemented. They were tested on a variety of problems, some designed to ensure that the routines could correctly handle such possibilities as termination with no feasible solution, redundant constraints, nearly parallel constraints, strict complementarity not applying at the solution, i.e. w^ = = 0 for some i, and the minimum lying entirely inside the feasible region at the uncon­strained minimum.

Various other problems of sizes up to 35 variables and 20 constraints were solved. A set of ordinary LCP problems was taken from Mathiesen and Hansen (1979) and solved also. Three of them appear at the end of Table Tl. Performance was compared with an inverse-basis implementation of Lemke's method developed by Ravindram (1972) and it was found that despite the expected extra work to compute the accurate orthogonal factorization the two methods developed were generally better than that given by Ravindram (see Table Tl).

As was mentioned initially, often the constraints and/or variables are subject to both lower and upper bounds. These problems can be solved by considering each constraint as two separate constraints, a procedure which enlarges the problem dimen­sion rather significantly. Following Sargent (1978) a routine was developed that allowed solution of these problems with the only increase in size being in the vector q' and the selection of the blocking variable becoming more complex. This extension was imple­mented for both versions of the algorithm, and worked. The details are given in Byrne (1980). One of the versions has been incorpor­ated into an implementation of the non-linear programming algorithm described by Mayne and Polak (1978) and has proved successful.Other commonly available QP codes were tried but found to be un­satisfactory.

The methods of solution for convex quadratic programs developed

83here have been used successfully. The implementation using orthog­onal factorizations of just a submatrix appears to result in no undue computational penalty but does ensure good numerical perform­ance. An initial feasible solution is not required as in most other solution methods, which is a major advantage. The routines are suitable for the solution of convex quadratic programming problems which arise either directly or as part of the procedure to solve general non-linear programming problems.

ACKNOWLEDGEMENTThis work was carried out under the supervision of Professor

R.W.H. Sargent, Department of Chemical Engineering, Imperial College, to whom I am very grateful for his encouragement and assistance. Financial assistance in the form of a Sir Walter Mulholland Fellowship was greatly appreciated.

Problem No. N o . t Zero No. Time (secs)Size Vbles c/s tol. itns Ravindram LMKFACT * LMKPPI\

41 15 26 IQ'10 8 .205 .027 .03541 15 26 IQ'8 25 .633 .215 .24241 15 26 10-8 30 .758 .462 .50318 12 6 10-10 14 .081 .055 .06611 5 6 10-8 9 .024 .015 .02240 15 25 10-10 24 .613 .213 .24740 25 15 10-10 50 1,.179 .807 .85455 35 20 10-10 51 2..381 1.359 1.42423 8 15 10-8 25 .227 .201 .15118 _ _ 10-8 37 .240 .212 .21218 _ _ 10-8 45 .272 .248 .19918 _ _

001o 21 . 102 .093 .117

f constraints* orthogonal factorization version + principal pivot version

Table Tl. Results for single-bounded LCP problems

Appendix A 1 . Partitioning in Lemke's method.After a number of iterations suppose the vectors w and z can be partitioned:

84

(Al.l)

with w^ = 0, z^ > 0 and W 2 > o, oIICNN

Then an equivalent partition of M q , and

W 1 M 10 Mm ll M. ~ 1 lr 12wr = Mr0 Mrl Mrr Mr2w 2 M20 M 21 M 2r M 22

or in shortened form

‘fiu : *12 '

W 2 S 2i: *22

qrq 2

(A1.2)

(A1.3)

The corresponding transformed system is thus:

M ll M 12M21 M22

The systems are related by

= M 11M 12 = -M 12' M n q i = ~q i

(A1.4)

(A1.5)

**21 M 21^1l' M 22 M 22 + M 21M 12' q 2 q 2 + M 21q l *The blocking variable at each iteration is then found as that w . such that

-<3'M!1C

for all i : M! > 0.1C (A1.6)

Thus it suffices in order to select w' that q'and M' be computed from (A1.5) using r

M., M' = - M ' , 11 lc lc M n q i = -q i (A1.7)

M 2c M 2c + M 21M ic' q 2 q 2 + M 2lq l *

85

Appendix A 2 . Restart Procedure On completing the solution of the LCP

w = M q Zq + Mz + q Tw . > 0 , z _> 0, w z = 0

the vectors w and z are partitioned thus:

and the system (A2.1) may be written

W1 ■

II

M10 Z0 + M11

W2 M20 M21

12

*22

(A2.1)

w = W 1 , z = 'zl‘ , w x = 0, z2 = 0 (A2

W 2 z2

(A2.3)

w^ = 0, z^ >_ 0, Zq = 0 w2 > 0, z^ = 0.

The equivalent transformed system is

Zl' = r-M- 1wn M10 Z0 + Mi i wi +

W2 M20" M21 M11 M10 M21 M11 q2

(A2.4)

with = q* = -M 11 q l - °-1

q 2 q 2 M 21 M 11 q l - °= 0, w^ = 0, and z^ = 0 is omitted.

exists, but is never explicitly computed.A new problem

w = M, + Mz + q (A2.5)0, z > 0 , w z = 0

is to be solved. Select M q > 0 and consider the transformed system(A2.6)

_ZJL

II M„i.o. zo + M’1W 11 W 1 + q_l_

W2 M 2 0 M 2!M U q 2

86

where = “M^l q _

q 2 = q 2 M 2 i

Lemke1s algorithm proceeding from this augmented transformed system can be proved to either find a solution or demonstrate that no feasible solution exists and thus the elementary principal pivot developed does so also.In order to find Ro, it is necessary to have the original M q that would, under the transformations indicated by M;q , result in the vector M q . From (A2.6) the untransformed system corresponding to (A2.5) would be

(A2.7)_w_l_

II ~-M , M11 n io Z0 + ?JUL Z1 + ?JLW2 ^20 “ M21 **10 M 21 ^2

and hence

M 10 M 11 M 10M 20 M 20 M 21 M 10

(A2.8)

From (A2.6) q^ is givenThe other vectors required are q and Rr by L

q x = "M“J q xor equivalently, M^^ q^ = -q^.

TUsing the known factorization of = V D R, this becomesRT D R q x = - M ^ q 1 (A2.9)

which is solved by forward and back substitution for q^. isthen found using

= q: (A2.10)Rg is defined as Rq = V M^ q , and although V is unavailable, R is readily obtained by

Ro = V M 10 = -v M u M 10 = -R M 10 (A2.ll)All vectors required to recommence the principal pivot routine from the current partition for the new problem have now been obtained.Note that in general the columns of M ^ will have been permuted in­to an order different to their natural order, but the permutation matrices have been omitted here for clarity.

87REFERENCES

Alexander, G.J. (1977) An algorithm for the derivation of the capital market line. Man.Sci., 23 (11), 1183-1186.

Beale, E.M.L. (1955) On minimizing a convex function subject to linear inequalities. J.R. Stat. Soc., (B) 17, 173-184.

Beale, E.M.L. (1959) On quadratic programming. Naval Res. Log. Qtly 6 (3), 227-243.

Best, M.J. and Ritter K. (1976) An effective algorithm for quadratic minimiz­ation problems. Tech. Rept. No. 1691, Mathematics Research Centre, University of Wisconsin-Madison.

Byrne, S.J. (1980) Quadratic programming using complementarity and orthogonal factorizations. Ph.D. thesis, Imperial College, London.

Chia Shun Shih (1976) Relative assessment for regional water quality management. Comput. and Ops. Res. 3 (2/3), 145-155.

Cottle, R.W. (1968) The principal pivoting method of quadratic programming.In: Mathematics of the Decision Sciences, Vol. 11, Pt. 1 (Dantzig, G.B. and Veinott, A.F., Ed.) 144-162.

Cottle, R.W. and Dantzig G.B. (1968) Complementary pivot theory of mathematical programming. Lin. Alg. Applic. 1, 103-125.

Dantzig, G.B. (1961) Quadratic Programming - a variant on the Wolf e-Markowitz algorithm. Res. Rept. 2. Operations Research Centre, University of California, Berkeley.

Dantzig, G.B. and Cottle R.W. (1967) Positive (semi) definite programming.In: Non-linear programming (Abadie, J. Ed.). North-Holland, Amsterdam, 55-73.

Dent, W. (1973) A note on least squares fitting of functions constrained to be either non-negative, nondecreasing, or convex. Man. Sci. 20 (1), 130-132.

Dorfman, R. (1951) Applications of linear programming to the theory of the firm. Univ. of California Press, Berkeley.

Eaves, B.C. (1971) The Linear Complementarity Problem. Mar..Sci. 17 (9), 612-634.

Fletcher, R. (1971) A general quadratic programming algorithm. AERE Harwell Report T.P. 401.

Fletcher, R. (1972) An algorithm for solving linearly constrained optimization problems. Math. Prog. 2 (2), 133-165.

Garcia-Palomares, U.M. and Mangasarian, O.L. (1976) Superlinearly convergentquasi-Newton algorithms for non-linearly constrained optimization problems. Math. Prog. 11 (1), 1-13.

Gill, P.E., Golub, G.H., Murray, W. and Saunders, M.A. (1974) Methods of modi­fying matrix factorizations. Math. Comp. 28, 505-535.

Gill, P.E., and Murray, W. (1978) Numerically stable methods for quadratic programming. Math. Prog. 14, 349-372.

Gill, P.E., Murray, W. and Saunders, M.A. (1975) Methods for computing and modifying the LDV factors of a matrix. Math. Comp. 29, 1051-1077.

Glassey, (1978) Price sensitivity consumer demands in energy modelling - a quadratic programming approach to the analysis of some Federal agency policies. Man. Sci. 24 (9), 877-886.

Goldfarb, D. (1966) A conjugate gradient method for non-linear programming.Ph.D. thesis, Dept, of Chemical Engineering, Princeton University.

Goldfarb, D. (1968) Analogs of Newton's method for quadratic programming.Notices Am. Math. Soc., 15 (2) 400.

Goldfarb, D. (1972) Extensions of Newton's method and simplex methods for solving quadratic programs. In: Numerical methods for non-linear optimization (Lootsma, F.A., Ed.). Academic Press.

Goldfarb, D. and Idnani, A. (1983) A numerically stable dual method for solving strictly convex quadratic programs. Math. Prog. 27, 1-33.

Hall, et.al. (1975) Spatial Equilibrium in US agriculture. Siam Review 17 (2), 323-338.

Han, S.P. (1977) Superlinearly convergent variable-metric algorithms for general non-linear programming problems. Math. Prog. 11 (3), 263-282.

Judge, G.G., and Takayama, T. (1973) Studies in economic planning over space and time. North Holland.

Kuhn, H.W., and Tucker, A.W. (1951) Non-linear programming. In: Proceedings of the 2nd Berkeley Symposium on Mathematical Statistics and Probability, (Neyman, J. Ed.). University of California Press, Berkeley, 481-492.

Lemke, C.E. (1962) A method of solution for quadratic programs. Man. Sci.8 (4), 442-453.

Lemke, C.E. (1965) Bimatrix equilibrium points and mathematical programming.Man. Sci. 11 (7), 681-689.

Lemke, C.E. (1968) On complementary pivot theory. In: Mathematics of theDecision Sciences, Vol. 11, Pt. 1 (Dantzig, E.B., and Veinott, A.F. (Ed.), 95-114.

Lemke, C.E., and Howson, J.T. Jr. (1964) Equilibrium points of bimatrix games.J. Soc. InduSt. Appl. Maths. 12 (2), 413-423.

Markowitz, H.M. (1959) Portfolio Selection, Efficient Diversification of Investment. Cowles Foundation Monograph 16. John Wiley, New York.

Mathiesen, L., and Hansen, T. (1979) An equilibrium model for an open economy with institutional constraints on factor prices. Discussion paper 01/79. Norwegian School of Economics and Business Administration.

Mayne, D.Q., and Polak, E. (1978) A superlinearly convergent algorithm forconstrained optimization problems. CCD Public, No. 78/52. Imperial College, London.

89

McCarl, B.A., et.al. (1977) Quadratic programming applications. Omega 5 (1), 43-55.

Ohlson, J.A. (1977) Quadratic approximations of the portfolio selectionproblem when the means and variances of return are infinite. Man. Sci.23 (6), 576-584.

Pang, J.S. (1983) Methods for quadratic programming: a survey. Computers and Chemical Engineering 1 (5), 583-594.

Powell, M.J.D. (1978) Algorithms for non-linear constraints that use Lagrangian functions. Math. Prog. 14 (2), 224-248.

Powell, m .j .d . (1983) ZQPCVX: A Fortran subroutine for convex quadratic pro­gramming. Rept. no. DAMTP/1983/NA17, Dept, of Applied Mathematics and Theoretical Physics. University of Cambridge.

Ravindram, A. (1972) Alg. 431, A computer routine for quadratic and linear programming problems. CACM 15, (9), 818-820.

Samuelson, P.A. (1952) Spatial price equilibrium and linear programming.Am. Econ. Rev. 42, 283-304.

Sargent, R.W.H. (1978) An efficient implementation of the Lemke algorithm and its extension to deal with upper and lower bounds. Math. Prog. Study 7, 36-54.

Scarf, H. (1973) The computation of economic equilibria. Yale University Press.

Scott, J.T., and Baker C.B. (1972) A practical way to select an optimum farm plan under risk. Am. J. Agric. Econ. 54, 6 57-661.

Sharpe, W.F. (1970) Portfolio theory and capital markets. McGraw-Hill.

Thiel, H. and Van de Panne C. (1960) Quadratic programming as an extension of classical quadratic maximization. Man. Sci. 7 (1), 1-20.

Van de Panne, C. and Whinston, A. (1964) The simplex and dual method for quadratic programming. Opnl. Res. Qtly. 15 (4), 355-388.

Warner, D.M., and Prawda, J. (1972) A mathematical programming method forscheduling nursing personnel in a hospital. Man. Sci. 19 (4), 411-422.

Wilkinson, J.H. (1965) The Algebraic Eigenvalue Problem. Clarendon Press,Oxford.

Wilson, R.B. (1963) A simplical method for convex programming. Ph.D. disser­tation, Harvard University.

Wolfe, P. (1959) The simplex method for quadratic programming. Econometrica 27 (3), 382-398.