890_ftp

INTERNATIONAL JOURNAL FOR NUMERICAL METHODS IN ENGINEERINGInt. J. Numer. Meth. Engng 2003; 58:955–977 (DOI: 10.1002/nme.890)

On the Pareto optimum sensitivity analysis inmulticriteria optimization

W. H. Zhang∗;†

Sino–French Laboratory of Concurrent Engineering; Department of Aircraft Manufacturing Engineering;Northwestern Polytechnical University; P.O. Box 552; 710072 Xi’an; Shaanxi; China

SUMMARY

To analyse the trade-o� relations among the set of criteria in multicriteria optimization, Pareto optimumsensitivity analysis is systematically studied in this paper. Original contributions cover two parts: the-oretical demonstrations are �rstly made to validate the gradient projection method in Pareto optimumsensitivity analysis. It is shown that the projected gradient direction evaluated at a given Pareto optimumin the design variable space rigorously corresponds to the tangent direction of the Pareto curve=surfaceat that point in the objective space. This statement holds even for the change of the set of activeconstraints in the perturbed problem. Secondly, a new active constraint updating strategy is proposed,which permits the identi�cation of the active constraint set change, to determine the in�uence of thischange upon the di�erentiability of the Pareto curve and �nally to compute directional derivatives innon-di�erentiable cases. This work will highlight some basic issues in multicriteria optimization. Somenumerical problems are solved to illustrate these novelties. Copyright ? 2003 John Wiley & Sons, Ltd.

KEY WORDS: multicriteria optimization; Pareto optimum sensitivity analysis; gradient projectionmethod; directional derivative

1. INTRODUCTION

Considering the complexity and multidisciplinary nature of large-scale engineering designproblems in the real-life, multicriteria optimization is a rational approach for the design syn-thesis. With this procedure, structures and mechanical systems can be appropriately designedto satisfy a variety of task requirements or multidisciplinary performance demands. In multi-criteria optimization, con�icting among relevant criteria is a basic issue which makes it im-possible to �nd an optimum solution minimizing simultaneously all individual criteria. For

∗Correspondence to: W. H. Zhang, Sino–French Laboratory of Concurrent Engineering, Department of AircraftManufacturing Engineering, Northwestern Polytechnical University, P.O. Box 552, 710072 Xi’an, Shaanxi, China.

†E-mail: [email protected]

Contract=grant sponsor: National Natural Science Foundation; contract=grant number: 10172072Contract=grant sponsor: Aeronautical Foundation; contract=grant number: 00B53005

Received 17 October 2001Revised 23 May 2003

Copyright ? 2003 John Wiley & Sons, Ltd. Accepted 23 May 2003

956 W. H. ZHANG

this reason, a vector optimization problem has to be formulated and solved to produce a setof compromise solutions.Today, a variety of optimization methods are available and have been put in use (see

References [1, 2]). Among others, a conventional approach is to transform the original probleminto a scalarized one. The weighting method is a basic approach. However, as indicated byKoski [3], Messac et al. [4], the disadvantage and defectiveness of such a method are thatnon-convex Pareto optimum parts cannot be captured. To bypass this obstacle, Das and Dennis[5] established the normal boundary method. Alternatively, Athan and Papalambros [6], Chenet al. [7] suggested the compromise programming method in which a nonlinearly scalarizedobjective function with curvature is adopted. To enhance the reliability of the latter, Messacet al. [4] studied the selection of inherent parameters in order to attribute an appropriatecurvature to the objective function.Pareto optimum sensitivity analysis studied here is another important concern after the

solving of the optimization problem. In order to explore the Pareto optimum set for decision-makings, it is necessary to have a further insight into the neighbourhood of each solution.Pareto optimum sensitivity analysis re�ects relative variations among di�erent criteria. Thisinformation can be used to access quickly relative changes of design variables, constraints andcriteria when a Pareto optimum is shifted from one solution to another. Besides, this trade-o�information is essential to understand precisely how about the non-linearity, di�erentiabilityand discontinuity of the Pareto curve since the latter may be highly non-linear, non-smoothand even discontinuous in practice. For example, with known derivatives, the Pareto optimumcurve which is unknown in advance can be well approximated by �tting a few available Paretooptimum points. Within this scope, a comprehensive discussion was given by Tappeta et al.[8, 9]. In their work, a Pareto optimum sensitivity analysis procedure was implemented jointlywith the physical programming method. The concept of trade-o� matrix was introduced tocharacterize the in�uence of one criterion upon remaining ones. Likewise, the trade-o� matrixwas also used in the approximation of the Pareto curve. Meanwhile, Diaz [10], Tappeta et al.[8, 9] presented an interactive compromise programming procedure which employs sensitivityanalysis capabilities to update the aspiration levels. The SQP and gradient projection method(GPM) were adopted for sensitivity analysis, respectively. Later, Hernandez [11] evaluatedthe Pareto optimum sensitivity by solving a linear system associated with the set of activeconstraints. Recently, Fadel et al. [12] attempt to use Pareto optimum sensitivity for theevaluation of design robustness. A sensitivity measure using the so-called �-optimality conceptis applied for the preferred selection of robust design in bicriteria optimization.Basically, almost all formulations used by Tappeta et al. [8, 9], Diaz [10], Hernandez [11]

and even those by Sobieszczanski-Sobieski et al. [13] and Vanderplaats and Yoshida [14]for optimum sensitivity to problems’ parameters in single objective problems were performedunder the common assumption. It is assumed that the active constraint set remains unchangedfor the perturbed problem and that the strict complementary slackness holds at the currentoptimum. Tseng and Lu [15] had to appeal to the �nite di�erence computing to bypassthis assumption indirectly. In their work, the Pareto optimum sensitivity analysis is used tostudy the variation of optimum design with respect to the upper bounds of criterion-de�nedconstraints after the solving of multicriteria problems by the trade-o� method.As is well-known, the main di�culty associated with Pareto optimum sensitivity analy-

sis is the non-di�erentiability of the Pareto curve in the objective space. This is due to thechange of the active constraint set when a perturbation is slightly made. Relevant problems

Copyright ? 2003 John Wiley & Sons, Ltd. Int. J. Numer. Meth. Engng 2003; 58:955–977

PARETO OPTIMUM SENSITIVITY ANALYSIS 957

exist practically and readers can refer to the truss design with discontinuous Pareto curve dis-cussed in Reference [3]. Moreover, numerical tests given in References [16, 17] show that thenon-di�erentiability is inevitable when the same function is used both for the criterion def-inition and for the active constraint de�nition like in physical programming. In the recentwork of Zhang and Yang [18], it is revealed that the normal direction at a non-di�erentiableoptimum point on the Pareto curve will not be unique and that an interval of subgradientsexists, and can be obtained by linear programming.In this work, a new approach is proposed in consideration of the change of the active

constraint set. To evaluate directional derivatives and the normal of the Pareto optimum curvein non-di�erentiable cases, a modi�ed GPM method is proposed originally. In addition to this,a new active constraint strategy is established for the updating of the gradient matrix of activeconstraints. This new strategy is quite di�erent from the classical one that deletes the activeconstraint having the most negative Lagrangian multiplier value. To ensure the validity of theGPM method, theoretical demonstrations are given for the �rst time. Finally, numerical testsare used to validate the proposed method and the �nite di�erence scheme is used to justifysensitivity results obtained.

2. PARETO OPTIMUM SENSITIVITY ANALYSIS OF MULTICRITERIAOPTIMIZATION PROBLEMS

2.1. Basis of the multicriteria optimization

Before addressing the Pareto optimum sensitivity analysis, it is necessary to have a brief outlineof the multicriteria optimization problem. Consider the following mathematical programmingstatement:

Min F(X)

gj(X)60; j=1; : : : ; m(1)

where F(X) = [f1(X); f2(X); : : : ; fr(X)]T denotes a set of con�icting criteria to be mini-mized, gj(X) denotes the j-th constraint. Unlike scalar optimization problems, the solutionof the above problem is not unique and a compromise solution set exists. According to thePareto optimality, a feasible solution X∗ is de�ned as the Pareto optimum if there is no otherimproved feasible point X such that fk(X)6fk(X∗) with strict inequality for at least one con-dition. Theoretically, the optimality conditions satis�ed by the Pareto optimum are expressedas

r∑k=1wk∇fk(X∗) +

m∑j=1�j∇gj(X∗)=0 (2a)

�jgj(X∗)=0; �j¿0; j=1; : : : ; m (2b)

in which {wk} is a set of positive weightings satisfying the normalization equality∑r

k=1 wk =1.�j denotes the j-th Lagrangian multiplier and equals zero when the related constraint is in-active. In fact, it is easily observed that the above expressions are also the Kuhn–Tucker


958 W. H. ZHANG

Figure 1. Illustration of the Pareto curve in the objective space.

conditions of the following scalarized weighting problem:

Minr∑k=1wkfk(X)

gj(X)60 j=1; : : : ; m(3)

As illustrated in Figure 1, the compromise solution set in a bicriteria case, i.e. Pareto optimumset is geometrically represented by curve segments AB and CD in the objective space. Tangentlines correspond to sensitivities between f1 and f2 at points P and P′, respectively. For thenon-di�erentiable point P′, only directional derivatives exist. At point P, weighting vector{wk} represents the normal. In fact, according to the notion of the gradient projection method,Equation (2a) is physically equivalent to a zero-descent direction associated with the projectedgradient of the weighting sum. It can be then written in matrix form as

S=−r∑k=1wk∇fk −N[=Pr

(−

r∑k=1wk∇fk

)=0 (4)

that isr∑k=1wk∇fk +N[ = 0 (5)

where N= {∇gj}; ( j⊆ J; rank(N)= J ) is the gradient matrix whose columns consist of thegradients of J active constraints (J6M) at X∗;Pr = I−N(NTN)−1NT denotes the projectionoperator.

2.2. Pareto optimum sensitivity analysis

At a given Pareto optimum X∗, Pareto optimum sensitivity analysis consists in calculating thederivatives of criteria with respect to a certain one when the latter is perturbed. As illustrated inFigure 1, the derivative of one criterion f2 with respect to another f1 geometrically corresponds



to the tangent direction of the Pareto curve in the objective space. In this example, P is adi�erentiable Pareto optimum whereas P′ is a non-di�erentiable one with only directionalderivatives. B is non-di�erentiable but also discontinuous since a zero left-hand derivativeexists.Now, suppose that under the perturbation of the q-th criterion, the design variable vector

varies along the direction S. This variation can then be written as

�X=S� (6)

with � to be the step length. Therefore, the variation of the k-th criterion can be evaluatedby

�fk =∇fTk S� (7)

Now, the derivative quantifying the trade-o� relation between the k-th criterion and the q-thone is expressed as

dfkdfq

= lim�→0

�fk�fq

=∇fTk S∇fTq S

; k=1; r; k �= q (8)

Obviously, the derivative is independent of �. In the same way, derivatives of the constraintsand design variables with respect to the criterion fq are easily derived

dgjdfq

=∇gTj S∇fTq S

; j=1; : : : ; m (9a)

dxidfq

=Si

∇fTq S; i=1; : : : ; n (9b)

Meanwhile, if the perturbation �fq is prescribed a priori, the step length � can be evaluated as

�=�fq

∇fTq S(10)

As to the constraint variation, it can be estimated correspondingly as follows:

gj(X)= gj(X∗) +∇gTj�X= gj(X∗) +∇gTj S�fq

∇fTq S; j=1; : : : ; m (11)

2.3. Determination of tangent and normal directions of the Pareto curve

The above study points out that the basic work in sensitivity analysis is the determinationof the search direction S. Clearly, this direction has to be feasible in the design variablespace and its mapping in the objective space has to be the tangent of the Pareto curve. For aPareto surface, tangent directions are in�nite at a given optimum X∗, whereas S is a principaldirection associated with the descent direction of a certain criterion and sensitivities refer tothe directional derivatives along S. In the following work, it will be demonstrated that thedesired S satisfying relevant conditions can be derived by means of the GPM method.


960 W. H. ZHANG

Without loss of generality, suppose that a perturbation is made in order to reduce a pref-erence function as follows:

�=r∑k=1akfk (12)

Naturally, the projected descent direction associated with this function is

S=−r∑k=1ak∇fk −N\ (13a)

in which the vector of Lagrangian multipliers is de�ned by

\=−(NTN)−1NTr∑k=1ak∇fk (13b)

For example, consider a particular case where only the pth criterion involves in the preferencefunction with

ak =

{1; k=p

0; k �=p(14)

such that

S=−∇fp −N\ (15)

Two cases may exist in the evaluation of S by Equation (13a):

(i) If S is not a zero-vector, the multiplication of Equation (5) by S producesr∑k=1wk∇fTk S+ [TNTS=0 (16)

Due to the orthogonality between the projected direction S and the gradient matrix N of activeconstraints, the above expression can be further simpli�ed as

r∑k=1wk�fk =0 (17)

in which �fk =∇fTk S� refers to the k-th criterion variation along S with step length �. Itis worthwhile noting that � is just a scaling parameter having no in�uence on the directionof vector {�fk}. Equation (17) means that {�fk} is orthogonal to the normal vector {wk}.Hence, it follows that the mapping of the projected direction S de�ned in the design variablespace corresponds to the tangent direction {�fk} of the Pareto curve in the objective space.(ii) Secondly, if the search direction S obtained is a zero-vector. Namely

r∑k=1ak∇fk +N\=0 (18)

Equation (13b) has to be further investigated. Firstly, if all Lagrangian multipliers in \ relatedto active inequality constraints take non-negative values, it means that no descent directionexists, that will reduce � and that the current Pareto optimum is the minimum of �. Hence, the



Pareto optimum sensitivity analysis is trivial. In practice, this often happens at the extremityend points of the Pareto curve. Note that when equality constraints involve in the de�nitionof N in Equation (13b), no control is demanded to check the signs of associated Lagrangianmultipliers. Secondly, if the q-th Lagrangian multiplier in \ is negative (�q¡0), it means thatthe q-th active constraint will become inactive when a further reduction of � is done. Hence,a new modi�ed projected descent direction needs to be revaluated by dropping the gradient,denoted by nq, of the q-th active constraint from the matrix N. Suppose that �N is the reducedmatrix, a modi�ed projected direction �S will be then evaluated by

�S=−r∑k=1ak∇fk − �N �\ (19)

Now, let us prove that the mapping of �S also de�nes the tangent of the Pareto curve inthe objective space. In virtue of the matrix decomposition of N, one can write the followingexpressions:

[TNT �S= �[T �NT �S+ �qnTq �S (20a)

\TNT �S= �\T �NT �S+ �qnTq �S (20b)

In view of the orthogonality �NT �S=0 imbedded in Equation (19), it follows that

[TNT �S= �qnTq �S (21a)

\TNT �S= �qnTq �S (21b)

Based on Equation (21b), the multiplication of Equation (18) by �S gives rise to

r∑k=1ak∇fTk �S+ \TNT �S=

r∑k=1ak∇fTk �S+ �qnTq �S=0 (22)

namely

�qnTq �S=−r∑k=1ak∇fTk �S (23)

The multiplication of Equation (5) by �S with the retention of Equations (21) and (23)results in

r∑k=1wk∇fTk �S+ [TNT �S=

r∑k=1wk∇fTk �S+ �qnTq �S=

r∑k=1wk∇fTk �S−

(�q�q

)r∑k=1ak∇fTk �S=0 (24)

namely

r∑k=1

(wk − �q

�qak

)�fk =0 (25)


962 W. H. ZHANG

with

�fk =∇fTk �S� (26)

Here, � is added just to keep the conformity of the expression with Equation (17). Bynormalizing the weightings to the unit sum value, Equation (25) can be written in the samecompact form as Equation (17)

r∑k=1w∗k�fk =0 (27)

with normalized weightings to be

w∗k =

wk − (�q=�q)akr∑t=1(wt − (�q=�q) at)

=wk − (�q=�q)ak1− (�q=�q)

r∑t=1at

(28)

If the preference function de�ned by Equation (14) is used, the normalized weightings willbe reduced to

w∗k =

(wp − �q

�q

)/(1− �q

�q

); k=p

wk

/(1− �q

�q

); k �=p

(29)

Equation (27) points out that when an active constraint leaves the set of active constraintsand becomes inactive due to the perturbation, the normal direction vector of the Pareto curveat the same optimum will be shifted from {wk} to {w∗

k }. The shifting can be evaluated byusing the general expression (28). In other words, the Pareto optimum is identi�ed to be anon-di�erentiable point in the objective space provided that a shifting of weightings occurs.Because the non-di�erentiability depends uniquely upon the shifting of the weighting vector,the change of the active constraint set does not truly re�ect a non-di�erentiable optimumpoint of the Pareto curve. For example, if the q-th constraint is a degenerate inequality activeconstraint at the optimum, i.e. the Lagrangian multiplier �q=0 in Equation (28), then noshifting occurs between {wk} to {w∗

k }. This problem will be further illustrated by numericalexamples in the following section.Now, the remaining work is to prove that weightings obtained in Equation (25) consti-

tute indeed another set satisfying the Kuhn–Tucker optimality conditions of the multicriteriaproblem (1) at the same X∗. From Equation (25), we can write

r∑k=1

(wk − �q

�qak

)∇fk =

r∑k=1wk∇fk − �q

�q

r∑k=1ak∇fk (30)

In view of Equations (5) and (18), the above relation can be developed as

r∑k=1wk∇fk − �q

�q

r∑k=1ak∇fk =−N[+ �q

�qN\=−N

([− �q

�q\)

(31)



which can be further transformed intor∑k=1

(wk − �q

�qak

)∇fk +N

([− �q

�q\)=0 (32)

Similarly, weightings in the above expression can be normalized as those in Equation (28).As a result, Equation (32) can be written in a compact form

r∑k=1w∗k∇fk +N[∗=0 (33)

with

�∗j =

(�j − �q

�q�j

)/(1− �q

�q

r∑t=1at

); j �= q

0; j=q(34)

Therefore, Equation (33) validates the statement. To ensure the non-negative values of �∗j inEquation (34), the following relation must be satis�ed:

�j − �q�q�j¿0; j �= q (35)

Evidently, if each �j ( j �= q) takes a non-negative value, the above relation will be veri�edautomatically. On the other hand, if several �j’s are negative, we need then

�j�j6�q�q; j �= q (36)

This inequality indicates that when more than one active constraint have negative Lagrangianmultipliers, the q-th active constraint which becomes inactive and will be deleted from N hasto be identi�ed as such that has the maximum ratio

Maxj

{�j�j; �j¡0

}(37)

This active constraint strategy is quite di�erent from that used in the traditional GPM method.The latter identi�es the active constraint to be eliminated by taking the most negativeLagrangian multiplier with

Minj

{�j; �j¡0} (38)

Obviously, Equations (37) and (38) will be the same when only one active constraint withnegative �j exists. Therefore, the traditional GPM method is conditionally valid when appliedin Pareto optimum sensitivity analysis.On the other hand, It is important to remark that Equation (34) is an analytical expression

describing the inherent relationship between Lagrangian multipliers varying from {�j} to {�∗j }.Since �q¡0, all weightings {w∗

k } obtained in Equation (28) together with the Lagrangianmultipliers in Equation (34) are ensured to take only non-negative values. Therefore, it followsthat Equation (33) is indeed an alternative form of the Kuhn–Tucker optimality conditions


964 W. H. ZHANG

Figure 2. Flowchart of Pareto optimum sensitivity analysis.

for the same optimum satisfying Equation (5). To have a summary of the above presentation,a �owchart of the sensitivity analysis computing procedure is given in Figure 2.In this procedure, basically, the use of the weighting method in combination with the dual

optimization algorithm is an ideal approach which will provide directly weighting values andLagrangian multipliers in Step 2. Nevertheless, this is not obligatory and even not convenientfor non-convex problems. In fact, once the Pareto optimum X∗ is obtained by any availableoptimization method, a couple of non-negative sets {wk} and {�j} do exist and can be derivedfrom Kuhn–Tucker conditions (2) by linear programming [18] or other methods.Until now, the problem of having a constraint becoming inactive is thoroughly discussed.

For the case of having an inactive constraint becoming active, because sensitivity analysisrefers to function variations with respect to an in�nitesimal perturbation, an inactive constraint



becoming active has always a distance to traverse and such a distance is always greater thanthe in�nitesimal perturbation. Therefore, such a case does not need to be considered.

3. NUMERICAL EXAMPLES

3.1. Linear bicriteria optimization

Consider an illustrative linear bicriteria problem that is stated as

Min {f1(X) = x1; f2(X)= x2}g1(X) = −x1

4− x24+ 160

g2(X) = −x13

− x26+ 160

g3(X) = −x160g4(X) = −x260

Because each criterion is just de�ned by a proper design variable, the design variable spacecoincides with the objective space. Therefore, the Pareto curve can be easily plotted as thesegments AB and BC shown in Figure 3.Now, let us focus on the Pareto optimum sensitivity analysis at point B(2,2). Clearly, this

is a non-di�erentiable Pareto optimum point which can be obtained by the weighting methodwith weighting coe�cients w1 = 5

9 and w2 =49 . At point B, the �rst two constraints are active.

By de�nition, the gradient matrix is

N=

[− 14 − 1

3

− 14 − 1

6

]

From the Kuhn–Tucker optimality conditions, the Lagrangian multipliers related to bothactive constraints can easily be deduced as �1 = 4

3 and �2 = 23 . Firstly, sensitivity is

evaluated for the reduction of f1, i.e. �f1¡0. The projection of (−∇f1(X∗)) generatesS1 =Pr(−∇f1(X∗))= 0 with \=−(NTN)−1NT∇f1 = [−4; 6]T. The �rst negative componentof \ indicates that the �rst constraint will become inactive as shown in Figure 3. Therefore,its gradient should be taken away from N so that the reduced gradient matrix is �N=[− 1

3 ;− 16 ]T.

From Equation (19), the modi�ed descent direction equals �S1 = [− 15 ;25 ]T. Thus, sensitivity re-

sults with respect to f1 are then df2=df1 =∇fT2 �S1=∇fT1 �S1 =−2, dg1=df1 = 0:25, dg2=df1 = 0,dg3=df1 =−1, dg4=df1 = 2. From Equation (28), the critical weightings at B are found to be(w∗

1 ; w∗2 )= (

23 ;13 ). It can be observed that this is the normal vector of the Pareto curve de�ned

by the second active constraint in Figure 3. From Equation (34), the associated Lagrangianmultipliers are (�∗1 ; �

∗2 )= (0; 2).

In the case of �f1¿0, the search direction will be evaluated using the descent direction off2. It is found that \=[4;−6]T so that the second active constraint becomes inactive. Simi-larly, the projected direction is evaluated with �S2 = [12 ;− 1

2 ]T so that sensitivities correspond to

df2=df1 =∇fT2 �S2=∇fT1 �S2 =−1, dg1=df1 = 0, dg2=df1 =− 16 , dg3=df1 =−1 and dg4=df1 = 1. On


966 W. H. ZHANG

Figure 3. Pareto curve. Figure 4. Variation of the weightingcomponent w2.

the other hand, by means of Equations (28) and (34), one can obtain the shifting of weight-ings and Lagrangian multipliers, respectively. They are (w∗

1 ; w∗2 )= (

12 ;12 ) and (�

∗1 ; �

∗2 )= (2; 0).

Obviously, the weighting vector is the normal direction of the Pareto curve de�ned by the�rst active constraint in Figure 4. This example shows that sensitivity analysis, the shiftingof weightings and Lagrangian multipliers can be exactly evaluated by the proposed method atthe non-di�erentiable optimum. To have a direct understanding of the variation of the normaldirection (w1; w2) along the Pareto curve AB and BC, the variation of the weighting com-ponent w2 satisfying the condition w1 + w2 = 1 can be easily determined from the slope ofAB and BC and is plotted in Figure 4. At two end points A, C and point B, the weightingjumps mean that they are non-di�erentiable points and all weighting values within each jumpinterval will produce the same optimum when used in the weighting method.

3.2. Non-linear bicriteria optimization

This test was originally used by Li et al. [19] to study the approximation of the Paretooptimum curve. It is reconsidered here for the purpose of sensitivity analysis. The problemis stated as

Min {f1(X) = (x1 − 2)2 + (x2 − 1)2; f2(X)= x21 + (x2 − 6)2}g1(X) = x21 − x260g2(X) = 5x21 + x2610

g3(X) = x265

g4(X) = −x160By using the goal attainment method of the MATLAB optimization tool, the set of Paretooptimum solutions is evaluated and 65 discrete solution points are obtained. The optimumtrajectory is plotted as dot lines in the design variable space (see Figure 5) and objectivespace (see Figure 6), respectively. The variation of related weighting w2 from one point to



Figure 5. Description of the Pareto optimum solution set in the design variable space.

Figure 6. Pareto set in the objective space.


968 W. H. ZHANG

Figure 7. Variation of the weighting component w2 along the Pareto optimum trajectory.

another is depicted along the optimum trajectory in Figure 7. Here, attention is focused onfour singular points A(

√15=3; 53 ), B(1.24817,2.87956), C(0.4,5) and D(0,5).

At point A, only the �rst two constraints are active. Therefore, the gradient matrix N isde�ned as

N=

[2√153

10√153

−1 1

]

By using the Kuhn–Tucker optimality conditions, two pairs of extreme weighting vectorsexist with

(1) (w1; w2)= (0:86; 0:14) with (�1; �2)= (0; 0:0665)(2) (w1; w2)= (0:9072; 0:0928) with (�1; �2)= (0:4054; 0)

When �f1¡0, we have �2 =−0:1307¡0 so that �N=[2√15=3;−1]T. Consequently, the pro-jected direction is �S1 = [−0:13204;−0:3409]T. It can be veri�ed that this is the tangent direc-tion of the �rst constraint as shown in Figure 5. Based on (28), it can be veri�ed that the�rst set (w1; w2)= (0:86; 0:14) will be shifted to the second set. Conversely, when �f2¡0,we have �1 =−7:3889¡0 which leads to �S2 = [−0:05556; 0:71728]T as shown in Figure 5.In this case, the second weighing set will be automatically shifted to the �rst set. This jumpidenti�es the non-di�erentiability of point A in the objective space. Directional derivativescan be evaluated as

df2df1

=∇fT2 �S∇fT1 �S

=

{−9:7526; �f1¡0

−6:1428; �f1¿0



Note that the negative derivative results make sure the con�ict between both criteria. FromEquation (9a), constraint sensitivities are obtained

dg1df1

=

{0; �f1¡0;

−0:8315; �f1¿0;

dg2df1

=

{7:6525; �f1¡0

0; �f1¿0

and

dg3df1

=

{1:2753; �f1¡0;

0:6929; �f1¿0;

dg4df1

=

{−0:4939; �f1¡0

0:0536; �f1¿0

As to point B in Figure 5, only g2 is active with N=[12:4817; 1]T. One can see that it isnot only the Pareto optimum of the constrained problem but also the optimum of the sameproblem excluding g2 with �2 = 0. For di�erent perturbations associated with �f1¡0 and�f1¿0, two search directions S3 and �S4 will be evaluated by means of Equations (13a) and(19), respectively. Solutions of search directions are S3 =Pr(−∇f1)= [0:30884;−3:8548]T and�S4 =−∇f2 = [−2:49634; 6:24088]T as illustrated in Figure 5. Note that g2 will leave the activeconstraint set when �S4 is calculated. Sensitivities of criteria are then

df2df1

=∇fT2 S∇fT1 S

=

{−1:66019; �f1¡0

−1:66019; �f1¿0

Additionally, sensitivities of constraints are

dg1df1

=

{−0:30931; �f1¡0;

−0:45832; �f1¿0;

dg2df1

=

{0; �f1¡0

−0:91562; �f1¿0

and

dg3df1

=

{0:25775; �f1¡0;

0:22932; �f1¿0;

dg4df1

=

{0:02065; �f1¡0

0:09173; �f1¿0

Although S3 and �S4 are not aligned in the design variable space, derivatives of criteria remainthe same. This important result ensures that B is a di�erentiable point. In fact, this conclusioncan also be validated by checking the weighting variation. When �f1¿0, it can be foundfrom Equation (18) that g2 will become inactive with �2¡0. By putting �2 = 0 in Equation(29), we get

w∗1 =

(w1 − �2

�2

)/(1− �2

�2

)=w1

w∗2 =w2

/(1− �2

�2

)=w2

Because no shifting occurs for weightings, points B is a di�erentiable Pareto optimum. In thesame way, point C is studied. Because �3 = 0, �3¡0, point C is also a di�erentiable point


970 W. H. ZHANG

Figure 8. The 25-bar problem.

with the following search directions and derivatives:

�S5 = [3:2;−8]T and S6 = [−0:8; 0]T

df2df1

=∇fT2 S∇fT1 S

=

{−0:25; �f1¡0

−0:25; �f1¿0

Point D is an end point of the Pareto curve, where side constraints g3; g4 are active. Thecorresponding weightings are then (w1; w2)= (0; 1). The gradient matrix N is

N=

[0 −11 0

]

From Kuhn–Tucker conditions, we have �3 = 2, �4 = 0. For �f2¡0, we get S=0 from Equa-tion (13a). This solution is evident since f2 attains its minimum value at D. For �f1¡0,we obtain �3 =−8, �4 =−4 by means of Equation (18). As a result, g4 has to be eliminatedaccording to Equation (37). Hence, the search direction obtained is horizontal with �S7 = [4; 0]T

as shown in Figure 5. However, if we use the traditional GPM approach with Equation (38)by taking away the constraint having the most negative Lagrangian multiplier �3 =−8, anerroneous vertical search direction will produce.

3.3. The 25-bar problem

Figure 8 illustrates the 25-bar problem that is frequently used to validate the optimizationprocedure (see Reference [20]). The structure is subject to two load cases as listed in Table I.



Table I. Load de�nition.

Load components (lbf)

Load case Node X Y Z

1 1 1000 10 000 −50002 0 10 000 −50003 500 0 06 500 0 0

2 1 0 20 000 −50002 0 −20 000 −5000

Table II. Optimum solution and Pareto optimum sensitivity analysis.

Descent direction (s)(� �q¡0)

OptimumDesign variables section area GPM FDM(bar no.) (in2) method method

1 (1) 0.01 0 02 (2–5) 4.6739 −0:003886 −0:7583 (6–9) 6.5583 0.003180 0.624 (10,11) 0.01 0 05 (12,13) 0.01 0 06 (14–17) 1.2418 0.020256 0.0153367 (18–21) 3.8784 0.001351 0.0002358 (22–25) 4.8626 −0:000637 0.042198

Optimum weight W ∗=1155:13 (lbm)

Here, the problem is reformulated as a bicriteria optimization problem. Both the total weightof the truss denoted by W and the maximum of three displacement components (u; v; w) atnodes 1 and 2, denoted by �q, will be simultaneously minimized. To ensure the structuralsymmetry, eight design variables of section areas are de�ned by grouping bar elements aslisted in Table II. A lower bound is imposed for each design variable. Besides, an allowablestress ��=3000 psi is imposed for stresses in all bars. Initial data are as follows:

Young’s modulus: E=107 psi; Material density: �=0:1 lbm=in3

Lower bound of section areas: a=0:01 in2; Allowable stress: ��=3000 psi

Mathematically, the problem is stated as

Min{W (a); �q= Max

k=1;2; l=1;2{uk; l(a); vk; l(a); wk; l(a)}

}

�j; l(a)6 ��; j=1; 25

a6ai; i=1; 8


972 W. H. ZHANG

Symbols k and l denote the node number and load case, respectively. To study the trade-o� relation between the weight and the truss �exibility measured by the maximum nodaldisplacement, we will calculate the sensitivity between W and �q at the desired optimumsolution. Here, the di�culty is that �q is a non-di�erentiable function when compared withabove examples. By means of the MBF scheme proposed in Reference [21], it can be provedthat the actual problem can be transformed into an equivalently form as follows:

Min {W (a); �q}uk;l(a)6 �q; k=1; 2; l=1; 2

vk;l(a)6 �q; k=1; 2; l=1; 2

wk;l(a)6 �q; k=1; 2; l=1; 2

�j; l(a)6 ��; j=1; 25; l=1; 2

a6ai; i=1; 8

in which �q is regarded as an additional arti�cial design variable. Based on the �-constraintmethod, suppose now that �q=0:175 in is prescribed for the upper bound of nodal displace-ments. By using the dual approach with CONLIN approximation [22], a weight minimiza-tion problem is solved. The optimum solution is obtained as given in Table II. At this op-timum, displacement components v1;1 and v2;1 attain their upper bound simultaneously inthe �rst load case, and both components have also the same gradient due to the struc-tural symmetry. However, as listed in Table III, because the associated Lagrangian mul-tipliers take a zero-value, both constraints become degenerate and the optimum is an ir-regular point. Following Kuhn–Tucker conditions (2), weightings associated with W and�q are derived as w1 = 1; w2 = 0. In the meantime, stresses in bars 16, 25 attain the allow-able stress in the �rst load case, whereas stresses in bars 2, 5, 7, 8, 19, 20 attain theallowable stress in the second load case. Nevertheless, due to the structural symmetry, itis observed that the same stress gradients hold for bar pairs (2,5), (7,8) and (19,20). Ac-cordingly, only �ve among eight active stress constraints are retained to form the gradi-ent matrix N. Besides, three design variables numbered 1, 4, 5 attain the lower bound.As a result, there exist totally eight active constraints whose gradients are linearly inde-pendent and will be used to constitute the gradient matrix N. By using the dual approach, theLagrangian multipliers associated with these eight active constraints are provided automatically asa by-product after optimization. They are �=[3:0106E−2; 9:1991E−2; 8:6904E−2; 8:7608E−2;8:8358E−2; 8:8712; 4:5885; 8:9330]. Based on the GPM method, the search direction is nowevaluated. For a reduction of the weight, it concludes that the projection of the negative gra-dient of W gives rise to a zero-direction with all non-negative Lagrangian multipliers (�¿0).This implies therefore that the weight is no more reducible even with a bound relaxationof the displacement constraints. Conversely, if a reduction of the maximum displacementsis made by projecting the negative gradient of Y -component v1;1 or v2;1, a zero-direction isalso obtained but with negative Lagrangian multipliers being \=[−8:9989E−6;−2:7264E−5;−5:3219E−6;−1:0168E−5;−6:5811E−6;−7:8388E−4; 3:1628E−3;−1:1564E−3]. By meansof Equation (37), it turns out that the biggest value among all negative ratios (�j=�j) refersto the �rst stress constraint attached to bar 16. Therefore, the gradient of this constrainthas to be cancelled from N. Consequently, a modi�ed search direction is reevaluated and



Table III. List of active response constraints with associated Lagrangian multipliers.

Optimum Weight W Displacement �q List of active Lagrangianpoint (i) (lb) (in) response constraints multipliers (�)

1 1155.13 0.175 Displacements: v1;1, v2; 1 0:0000000E + 00Stresses: �16:1; �25:1; �2:2, 0:0000000E + 00�5:2; �7:2; �8:2; �19:2; �20; 2 3:01068425E − 02

9:19912364E − 024:34518871E − 024:34518871E − 024:38042233E − 024:38042233E − 024:41791559E − 024:41791559E − 02

2 1172.735 0.17 Displacements: v1;1, v2; 1 1:84754826E + 03Stresses: �2; 2, �5; 2, �7; 2, �8; 2, 1:84754826E + 03

�19; 2, �20; 2 3:35478362E − 023:35478362E − 022:50888437E − 022:50888437E − 023:20807300E − 023:20807300E − 02

3 1213.579 0.16 Displacements: v1;1; v2; 1 2:25729654E + 03Stresses: �2; 2, �5; 2, �7; 2, �8; 2, 2:25729654E + 03

�19; 2, �20; 2 3:10648197E − 023:10648197E − 022:11683644E − 022:11683644E − 022:95968917E − 022:95968917E − 02

4 1271.511 0.15 Displacements: 2:67736163E + 03v1; 2, v2; 2, v1; 2, v2; 2 2:67736163E + 03Stresses: �19;2, �20;2 1:50114296E + 03

1:50114296E + 032:94039349E − 032:94039349E − 03

5 1362.082 0.14 Displacements: 3:06680393E + 03v1;1, v2; 1, v1; 2, v2; 2 3:06680393E + 03

1:79656594E + 031:79656594E + 03


2:08339617E + 032:08339617E + 03


2:44486762E + 032:44486762E + 03


974 W. H. ZHANG

Table III. Continued.

Optimum Weight W Displacement �q List of active Lagrangianpoint (i) (lb) (in) response constraints multipliers (�)


2:90932144E + 032:90932144E + 03


3:51994813E + 033:51994813E + 03

Figure 9. Pareto optimum curve.

the solution is given in Table II. By using Equation (8), the criteria sensitivity equals thendW=d �q=−3345:6122. This solution can be validated by using �nite di�erence computing.To do this, the upper bound �q of displacement constraints will be relaxed and tightened,respectively. In the �rst case when �q=0:18 in is imposed, it turns out that the optimumsolution is insensitive to the relaxation and remains unchanged. This is consistent with thesolution obtained by GPM method. In the second case when the upper bound is tightenedwith �q=0:16 in, the perturbation causes the optimum solution to change with an increaseof �W =58:4492 lb and it is easily observed that the stress constraint associated with bar16 does leave the active constraint set and becomes inactive. The search direction de�ningthe variation of design variable vector is given in Table II. Although the latter is not com-patible with the previous one, the derivative evaluated by �nite di�erence approximation is�W=��q=−3368 with only a relative error of 0.7% with respect to the exact value.To validate further the proposed method, we study now the trade-o� relations between

W and �q at other prescribed values of �q. Optimum points are obtained in the same wayand plotted in Figure 9. At such points (2–9), active constraints are listed in Table III withrelated values of Lagrangian multipliers. Because the same multipliers correspond to thoseconstraints having the same gradient and upper bound values, the corresponding gradientshave to be countered and used one time in the formation of the gradient matrix N. At eachoptimum, search directions are evaluated by the GPM method for reduction perturbations ofW and �q, respectively. Each pair of search directions obtained is listed in Table IV. Both are



Table IV. Search directions obtained by the GPM method at di�erent optimum points.

(�W¡0)Optimum Search directionspoint (i) (�q¡0)

1 0 0 0 0 0 0 0 00 −0:003886 0.003180 0 0 0.020256 0.001351 −0:00064

2 0 10.13595 −8:30248 0 0 −70:5749 −3:4941 −53:8110 −0:00274 0.002246 0 0 0.019098 0.00094 0.01456

3 0 9.26089 −7:5850 0 0 −71:0197 −3:1957 −53:7110 −0:00205 0.001680 0 0 0.015730 0.00070 0.01189

4 0 −69:5067 −52:5741 0 0 −38.1401 3.55047 −21:2810 0.008317 0.006291 0 0 0.004563 −0:0004 0.00254

5 0 −59:8823 −44:6597 0 0 −60.9811 −79:569 −43:9120 0.00615 0.00459 0 0 0.00626 0.00818 0.00451

6 0 −59:88647 −44:66261 0 0 −60:98227 −79:5735 −43:91290 0.005308 0.003959 0 0 0.005405 0.00705 0.003892

7 0 −59:8899 −44:6650 0 0 −60:9827 −79:5768 −43:91310 0.004523 0.0033737 0 0 0.0046062 0.006010 0.003316

8 0 −59:8929 −44:66705 0 0 −60:9827 −79:5794 −43:91300 0.00380 0.00283 0 0 0.00387 0.00505 0.00278

9 0 −59:8955 −44:6687 0 0 −60:9823 −79:5815 −43:91250 0.00314 0.00234 0 0 0.003198 0.004174 0.002303

Figure 10. Sensitivity curve for trade-o� analysis.

aligned and oriented in opposite directions that produce the same sensitivity results as plottedin Figure 10. In this calculation, it is found that there are no active constraints to be eliminatedfrom the gradient matrix. Therefore, points (2–9) are all regular optima. In fact, accordingto the theory of the optimum sensitivity to the bound of active constraint [23], an alternativesensitivity analysis scheme is available for regular optimum points with non-degenerate active


976 W. H. ZHANG

constraints. For points (2–9), the trade-o� between W and �q can be analytically evaluated bymeans of relevant Lagrangian multipliers

dWd �q

=−J∑t=1�t

where J denotes the number of active displacement constraints. The sum is used because thesame upper bound �q is imposed for all active displacement constraints. In this way, it can beeasily veri�ed that the addition of related Lagrangian multipliers in Table III gives exactlythe same sensitivity results as those by the GPM method (see Figure 10).

4. CONCLUSIONS

This paper presents a new gradient-projection based Pareto optimum sensitivity analysismethod to study the trade-o� relations in multicriteria problems. Theoretical demonstrationsare made originally. It is shown that the gradient projection scheme constitutes the basis forthe determination of tangent and normal directions of the Pareto optimum curve. Depend-ing on the projection direction of the selected function, analytical relations are obtained toevaluate the weighting shifting that depicts directional derivatives in non-di�erentiable cases.Theoretically, it is revealed that the variation of non-degenerate active inequality constraint setis the cause of the non-di�erentiability. Thanks to the new active constraint strategy, an auto-matic identi�cation of the active constraint to be inactive is made possible and validated bynumerical solutions. In addition, the proposed method can also be generalized to study the de-pendence between the objective function and active constraints in single objective problems.Numerical applications indicate that the optimum sensitivity with respect to the constraintbound can be rigorously evaluated if the latter is considered as a design parameter.

ACKNOWLEDGEMENTS

This work is supported by the National Natural Science Foundation under the Grant No. 10172072 andthe Aeronautical Foundation under the Grant No. 00B53005. Numerical tests are carried out thanks tothe help of graduate students Tong GAO and Jihong ZHU.

REFERENCES

1. Eschnauer H, Koski J, Osyczka A. Multicriteria Design Optimization. Springer: Berlin, Heidelberg, New York,1990.

2. Stadler W. Multicriteria Optimization in Engineering and in Sciences. Plenum Press: New York, 1988.3. Koski J. Defectiveness of weighting methods in multicriterion optimization of structures. Communications inApplied Numerical Methods 1985; 1:333–337.

4. Messac A, Sundararaj GJ, Tappeta RV, Renaud JE. Ability of objective functions to generate points on non-convex Pareto frontiers. AIAA Journal 2000; 38(6):1084–1091.

5. Das I, Dennis JE. Normal-boundary intersection: a new method for generating the Pareto surface in nonlinearmulticriteria optimization problems. SIAM Journal on Optimization 1998; 8(3):631–657.

6. Athan TW, Papalambros Y. A note on weighted criteria methods for compromise solutions in multi-objectiveoptimization. Engineering Optimization 1996; 27:155–176.

7. Chen W, Wiecek MM, Zhang J. Quality utility—a compromise programming approach to robust design. Journalof Mechanical Design (ASME) 1999; 121:179–187.

8. Tappeta RV, Renaud JE. Interactive multiobjective optimization procedure. AIAA Journal 1999; 37(7):881–889.



9. Tappeta RV, Renaud JE, Messac A, Sundararaj GJ. Interactive physical programming: tradeo� analysis anddecision making in multicriteria optimization. AIAA Journal 2000; 38(5):917–926.

10. Diaz A. Interactive solution to multiobjective optimization problems. International Journal for NumericalMethods in Engineering 1987; 24:1865–1877.

11. Hernandez S. Post-optimal procedures for structural optimization. Advances in Engineering Software 2000;31:401–409.

12. Fadel G, Konda S, Blouin V, Wiecek M. �-optimality in bi-criteria optimization. AIAA Paper 2002; 1233:256–265.

13. Sobieszczanski-Sobieski J, Barthelemy JF, Riley KM. Sensitivity of optimum solutions of problem parameters.AIAA Journal 1982; 20(9):1291–1299.

14. Vanderplaats GN, Yoshida N. E�cient calculation of optimum design sensitivity. AIAA Journal 1985;23(11):1798–1803.

15. Tseng CH, Lu TW. Minmax multiobjective optimization in structural design. International Journal forNumerical Methods in Engineering 1990; 30:1213–1228.

16. Zhang WH, Yang HC. A study of the method for a certain type of multicriteria structural optimization problems.Computers and Structures 2001; 79(31):2741–2749.

17. Das I, Dennis JE. A closer look at drawbacks of minimizing weighted sums of objectives for Pareto setgeneration in multicriteria optimization problems. Structural Optimization 1997; 14:63–69.

18. Zhang WH, Yang HC. E�cient gradient calculation of the Pareto optimal curve in multicriteria optimization.Structural and Multidisciplinary Optimization 2002; 23(4):311–319.

19. Li YS, Fadel GM, Wiecek MM. Approximating Pareto curves using the hyper-ellipse. ISSMO=NASA=AIAAFirst Internet Conference on Approximation and Fast Reanalysis Techniques in Engineering Optimization,1999, http://www.aero.u�.edu/∼issmo/webconf.htm.

20. Fleury C, Schmit LA. Dual methods and approximation concepts in structural synthesis. NASA CR-3226,December 1980.

21. Zhang WH, Domaszewski M, Fleury C. An improved weighting method with multibounds formulation andconvex programming for multicriteria structural optimization. International Journal for Numerical Methods inEngineering 2001; 52(9):889–902.

22. Fleury C. First and second order convex approximation strategies in structural optimization. StructuralOptimization 1989; 1:3–10.

23. Luenberger DG. Introduction to Linear and Nonlinear Programming. Addison-Wesley: Reading, MA, 1973.


890_ftp

Documents

Transcript of 890_ftp