Evolutionary Bilevel Optimizationppsn2014.ijs.si/files/slides/ppsn2014-tutorial7-sinha.pdf14th...

14th September 2014, PPSN-2014, Ljubljana, Slovenia

Evolutionary Bilevel Optimization A. Sinha, P. Malo and K. Deb

Ankur Sinha, PhD Department of Information and Service Economy Aalto University School of Business Helsinki, Finland


Outline

Ø  Evolutionary Bilevel Optimization (EBO)

Ø  Genesis

Ø  Applications

Ø  Solution Methodologies

Ø  Results

Ø  Multiobjective Evolutionary Bilevel Optimization (MEBO)

Ø  MEBO with Lower Level Decision Uncertainty

Ø  Robust EBO

Ø  Conclusions


Single-level Optimization


Bilevel Optimization

A mathematical program that contains an optimization problem in the constraints.

-Bracken and McGill (1973)


Bilevel Optimization (cont.)

Multiple Lower Level Optimal

Solutions


Multi-level Optimization

•  Multi-level (L levels) optimization •  Two or more levels of optimization •  Nested structure

•  Bilevel: A more generic optimization concept than single-level optimization

Min F(x1,x2,…xL) s.t


Bilevel Optimization: A Genesis

•  All optimization problems are special cases of bilevel programming

An Extension of Mathematical Programming

•  Bilevel programs commonly appear in game theory when there is a leader and follower

Stackelberg Games

–Bracken and McGill (1973)

–Stackelberg (1952)


Bilevel Problems in Practice

•  Stability or equilibrium satisfaction •  Lower level task implements the stability or equilibrium criteria •  Dual problem solving in theoretical optimization

Mathematical programming: Functional feasibility

•  Leader is the first mover •  Leader needs to anticipate followers decisions •  Follower responds optimally to leaders decision

Stackelberg games: Leader-follower


Properties and Challenges

l  Bilevel problems are typically non-convex,

disconnected and strongly NP-hard

l  Solving an optimization problem produces one or

more feasible solutions

l  Multiple global solutions at lower level can induce

additional challenges

l  Two levels can be cooperating or conflicting


How does a bilevel optimization problem look?

Min(x,y) 3y + x Such that

y ∈ argmax(y) 2y Such that x + y ≤ 8, x + 4y ≥ 8, x + 2y ≤ 13, 1 ≤ x ≤ 6

Example

Bilevel Linear Optimization Problem


F

y

x

f

x1 x2

Bold Line - Induced Set (2,6) - Bilevel Solution

Solu

tion

to B

ileve

l Lin

ear O

ptim

izat

ion

Prob

lem

Example (Cont.)


Bilevel vs. Multi-objective

Min(x,y) 3y + x Min(x,y) -2y Such that

x + y ≤ 8 x + 4y ≥ 8 x + 2y ≤ 13 1 ≤ x ≤ 6

y

x

Pareto-optimal Set (Decision Space)

Bilevel Optimum

Bilevel linear optimization problem is now modified as follows:


Applications


Toll Setting Problem Authority's problem:

l  Authority responsible for highway system wants to maximize its revenues earned from toll

l  The authority has to solve the highway users optimization problem for all the possible tolls

Highway users' problem: l  For any toll chosen by the authority,

highway users try to minimize their own travel costs

l  A high toll will deter users to take the highway, lowering the revenues

Brotcorne et al. (2001)


Structural Optimization Upper level: Topology Lower level: Sizes and coordinates


Seller-Buyer Strategies

l  An owner of a company dictates the selling price and supply. He wants to maximize his profit.

l  The buyers look at the product quality, pricing and various other options available to maximize their utility

l  Mixed integer programs on similar lines have been formulated by Heliporn et al. (2010)

Max Profit (suppy, selling price, demand, other variables) Such that demand ∈ argmax { Utility | Lower Level Constraints} Upper Level Constraints

Heilporn et al. (2010)


Stackelberg Competition Competition between a leader and a follower firm (Duopoly) Leader solves the following optimization problem to maximize its profit

If the leader and follower have similar functions, leader always makes a higher profit.

- First mover’s advantage Can be extended to multiple leaders and multiple followers

Sinha, Malo, Frantsev and Deb (2013)


Stackelberg Competition (Results)

Oligopoly: Change in leaders’ profits when number of players (leader or follower) change in a market

Duopoly: Leader’s objective function in the presence of a follower, the function is well-behaved in the absence of the follower

Sinha, Malo, Frantsev and Deb (2013)


Forest Management

Upper Level: Find an optimal harvesting strategy, i.e. when to harvest and when not! (Binary Variables)

Lower Level: For a given s t ra tegy, determine the optimal harvesting level, i.e. how much to harvest! (Continuous Variables)

Sinha et al. (2014 working paper)

Similar to Benders’ Decomposition!


•  Task: Optimal management of an uneven aged spruce forest •  Problem Class: MINLP •  Size: Typically 50 binary variables, 5000 continuous variables

Past Status •  Computational Requirements: Satisfactory solution obtained in 1 week

Current Status •  Upper level: A customized GA •  Lower Level: SQP with multiple starting points

•  Computational Requirements: Better solutions obtained in less than 2 hours

Forest Management (Results)

Sinha et al. (2014 working paper)


Parameter Tuning

Upper Level: Find optimal parameters that maximize algorithm performance over a number of initial conditions

Lower Level: Run the optimization algorithm to find optimized solution

Sinha, Malo, Xu and Deb (2014)


Parameter Tuning (Results)

0 0.2 0.4 0.6 0.8 1 1.2 1.4 1.6 1.8 20

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

F

CR

BAPT Results

0 0.2 0.4 0.6 0.8 1 1.2 1.4 1.6 1.8 20

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

F

CR

BAPT Results

Sphere Problem BAPT: Different parameters obtained using BAPT in 21 runs F≈0.4, CR≈0.9

Schwefel problem BAPT: Different parameters obtained using BAPT in 21 runs F≈0.4, CR≈0.95



0 5 10 15 200

10

20

30

40

50

60

70

80

90

100

q

Low

er L

evel

Opt

imiz

atio

n C

alls

(Nor

mal

ized

)

0.6

0.8

1.0

0.4

0.2

0.0

Accuracy

Ø  A single evalua,on for each parameter vector was sufficient to solve the problem

Ø  Mul,ple evalua,ons at each point increased computa,onal expense with negligible improvement in accuracy

Parameter Tuning (Results)



Other Applications

l  Maximizing production in chemical industries where reactions

occur under equilibrium l  Defence applications like, strategic offensive and defensive force

structure design, strategic bomber force structure, and allocation of tactical aircraft to missions

l  Other applications include, optimal shape design, optimal operating configuration and optimal facility location

l  Wide applications in economics like Principal-Agency problems,

Taxation policies etc.


•  KKT conditions of the lower level problem are used as constraints (Herskovits et al. 2000) •  Lagrange multipliers increase the number of decision

variables •  Constrained search space •  Applicable to differentiable problems only

•  Another common approach: Nested optimization •  For every xu, lower level problem is solved completely •  Computationally very expensive

•  Discretization of the lower level problem •  The best solution obtained from discrete set for a given

xu is used as a feasible member at upper level

Solution Methodologies


Solution Methodologies (cont.)

•  Penalty based approaches •  Special forms of penalty functions have been used •  Lower level is usually required to be convex •  Penalty method requires differentiability

•  Branch and Bound techniques (Bard et al. 1982) •  Used KKT conditions •  Handled linear problems

•  Taking an approximation of the lower level optimization problem such that its optimum is readily available •  The optimal solutions from lower level might not be

accurate


Solution Methodologies (cont.) •  Evolutionary algorithms have also been used for bilevel

optimization •  Most of the methods are nested strategies •  Mathieu et al. (1994): LP for lower level and GA for upper level •  Yin (2000): Frank Wolfe Algorithm for lower level •  Oduguwa and Roy (2005): Proposed a co-evolutionary approach •  Wang et al. (2005):

•  Solved bilevel problems using a constrained handling scheme in EA

•  Method is computationally expensive, but successfully handles a number of test problems

•  Li et al. (2006): Nested strategy using PSO •  EA researchers have also tried replacing the lower level problems

using KKT (Wang et al. (2008), Li et al. (2007))


Why Evolutionary Approach? Ø  No implementable mathematical optimality conditions exist

(Dempe, Dutta, Mordokhovich, 2007) §  Lower level (LL) problem is replaced with KKT conditions and

constraint qualification (CQ) conditions of LL §  Upper level (UL) problem requires KKT of LL, but handling

LL-CQ conditions in UL-KKT becomes difficult §  Involves second-order derivatives in UL-KKT conditions

Ø  Moreover, classical numerical optimization methods require various simplifying assumptions like continuity, differentiability and convexity

Ø  Most real-world applications do not follow these assumptions Ø  EA’s flexible operators, direct use of objectives and population

approach help solve BO problems better


Niche of Evolutionary Methods

•  Usually, lower level optimal solutions are multi-modal •  Computationally faster methods possible through meta-

modeling etc. •  Other complexities (robustness, parallel implementation,

fixed budget) can be handled efficiently •  Many bilevel problems are multi-objective BO

•  Both levels require to find and maintain multiple optimal solutions •  EAs are known to be good for these cases


Bilevel Test Problems •  Controlled difficulty in convergence at upper and lower levels •  Controlled difficulty caused by interaction of two levels •  Multiple global solutions at the lower level for any given set of upper

level variables •  Clear identification of relationships between lower level optimal

solutions and upper level variables •  Scalability to any number of decision variables at upper and lower

levels •  Constraints (preferably scalable) at upper and lower levels •  Possibility to have conflict or cooperation at the two levels •  The optimal solution of the bilevel optimization is known


The objectives and variables on both levels are decomposed as follows:

SMD Test Problem Framework (Sinha, Malo & Deb, 2014)


Bilevel Test Problems

Table 1: Overview of test-problem framework componentsPanel A: Decomposition of decision variables

Upper-level variables Lower-level variablesVector Purpose Vector Purposexu1 Complexity on upper-level xl1 Complexity on lower-levelxu2 Interaction with lower-level xl2 Interaction with upper-level

Panel B: Decomposition of objective functionsUpper-level objective function Lower-level objective function

Component Purpose Component PurposeF1(xu1) Difficulty in convergence f1(xu1,xu2) Functional dependenceF2(xl1) Conflict / co-operation f2(xl1) Difficulty in convergence

F3(xu2,xl2) Difficulty in interaction f3(xu2,xl2) Difficulty in interaction

Properties for inducing difficulties:

1. Controlled difficulty in convergence at upper and lower levels.

2. Controlled difficulty caused by interaction of the two levels.

3. Multiple global solutions at the lower level for a given set of upper level variables.

4. Possibility to have either conflict or cooperation between the two levels.

5. Scalability to any number of decision variables at upper and lower levels.

6. Constraints (preferably scalable) at upper and lower levels.

Next, we provide the bilevel test problem construction procedure, which is able to inducemost of the difficulties suggested above.

3.1 Objective functions in the test-problem framework

To create a tractable framework for test-problem construction, we split the upper and lowerlevel functions into three components. Each of the components is specialized for inductionof certain kinds of difficulties into the bilevel problem. The functions are determined basedon the required complexities at upper and lower levels independently, and also by the requiredcomplexities because of the interaction of the two levels. We write a generic bilevel test problemas follows:

F (x

u

,x

l

) = F1(xu1) + F2(xl1) + F3(xu2,xl2)

f(x

u

,x

l

) = f1(xu1,xu2) + f2(xl1) + f3(xu2,xl2)

wherex

u

= (x

u1,xu2) and x

l

= (x

l1,xl2)

(3)

In the above equations, each of the levels contains three terms. A summary on the rolesof different terms is provided in Table 1. The upper level and lower level variables have beenbroken into two smaller vectors (see Panel A in Table 1). The vectors x

u1 and x

l1 are used toinduce complexities at the upper and lower levels independently. The vectors x

u2 and x

l2 areresponsible to induce complexities because of interaction. In a similar fashion, we decomposethe upper and lower level functions such that each of the components is specialized for a certainpurpose only (see Panel B in Table 1). At the upper level, the term F1(xu1) is responsible

Evolutionary Computation Volume x, Number x 5

Roles of Variables


Controlling Difficulty for Convergence Ø  Convergence difficulties can be induced via following

routes: Ø  Dedicated components: F1 (upper) and f2 (lower)

Ø  Example:

Multi-modal

Quadratic


Controlling Difficulty in Interactions

Ø  Interaction between variables xu2 and xl2 could be chosen as follows: §  Dedicated components: F3 and f3

Ø  Example:


Difficulty due to Conflict/Co-operation Ø Dedicated components: F2 and f2 or F3 and

f3 may be used to induce conflict or cooperation

Ø Examples: §  Cooperative interaction = Improvement in lower-level

improves upper-level (e.g. F2 = f2) §  Conflicting interaction = Improvement in lower-level

worsens upper-level (e.g. F2 = -f2) §  Mixed interaction = Both cooperation and conflict (e.g.

F2 = f2 and F3 = Σi (xu2i)2 – f3


Controlled Multimodality

Ø  Obtain multiple lower-level optima for every upper level solution: §  Component used: f2

Ø  Example: Multimodality at lower-level

Induces multiple solutions: x1

l1 = x2

l1

Gives best UL solution: x1

l1 = x2

l1=0


Difficulty due to Constraints Constraints are included at both levels with one or more of the following properties:

Ø  Constraints exist, but are not active at the optimum Ø  A subset of constraints, or all the constraints are active at the

optimum Ø  Upper level constraints are functions of only upper level

variables, and lower level constraints are functions of only lower level variables

Ø  Upper level constraints are functions of upper as well as lower level variables, and lower level constraints are also functions of upper as well as lower level variables

Ø  Lower level constraints lead to multiple global solutions at the lower level

Ø  Constraints are scalable at both levels


Test Problems: SMD1 – SMD3

•  Interaction: Cooperative •  Lower level: Convex (w.r.t. lower-level variables) •  Upper level: Convex (induced space)

SMD1:

•  Interaction: Conflict •  Lower level: Convex (w.r.t. lower-level variables) •  Upper level: Convex (induced space)

SMD 2:

•  Interaction: Cooperative •  Lower level: Multimodality using Rastrigin’s function •  Upper level: Convex (induced space)

SMD 3:



•  Interaction: Conflict •  Lower level: Multimodality using Rastrigin’s function •  Upper level: Convex (Induced Space)

SMD 4:

•  Interaction: Conflict •  Lower level: Complexity with Rosenbrock’s function •  Upper level: Convex (Induced Space)

SMD 5:

•  Interaction: Conflict •  Lower level: Infinitely many global solutions for any given xu •  Upper level: Convex (Induced Space)

SMD 6:



•  Interaction: Conflict •  Lower level: Convex (w.r.t. lower-level variables) •  Upper level: Multimodality

SMD 7:

•  Interaction: Conflict •  Lower level: Complexity with banana function •  Upper level: Multimodality

SMD 8:

•  Interaction: Conflict •  Lower level: Non-scalable constraints •  Upper level: Non-scalable constraints

SMD 9:



•  Interaction: Conflict •  Lower level: Scalable constraints •  Upper level: Scalable constraints

SMD 10:

•  Interaction: Conflict •  Lower level: Non-scalable constraints, multiple global solutions •  Upper level: Scalable constraints

SMD 11:

•  Interaction: Conflict •  Lower level: Scalable constraints, multiple global solutions •  Upper level: Scalable constraints

SMD 12:


A. Sinha, P. Malo and K. Deb

can be made, whether it is useful to solve the lower level optimization problem at all for a givenxu.

The upper level constraint subsets, Gb depends on xl, and Gc depends on xu and xl. Thevalues from these constraints are meaningful only when the lower level vector is an optimalsolution to the lower level optimization problem. Based on the various constraints which maybe functions of xu, or xl or both, a bilevel problem introduces different kinds of difficulties inthe optimization task. In this paper, we aim to construct such kinds of constrained bilevel testproblems which incorporate some of these complexities. We have proposed four constrainedbilevel problems, each of which has at least one or more of the following properties,

1. Constraints exist, but are not active at the optimum

2. A subset of constraints, or all the constraints are active at the optimum

3. Upper level constraints are functions of only upper level variables, and lower level con-straints are functions of only lower level variables

4. Upper level constraints are functions of upper as well as lower level variables, and lowerlevel constraints are also functions of upper as well as lower level variables

5. Lower level constraints lead to multiple global solutions at the lower level

6. Constraints are scalable at both levels

While describing the test problems in the next section, we discuss the construction proce-dure for the individual constrained test problems.

4 SMD test problems

By adhering to the design principles introduced in the previous section, we now propose a set oftwelve problems which we call as the SMD test problems. Each problem represents a differentdifficulty level in terms of convergence at the two levels, complexity of interaction between twolevels and multi-modalities at each of the levels. The first eight problems are unconstrained andthe remaining four are constrained.

4.1 SMD1

This is a simple test problem where the lower level problem is a convex optimization task, andthe upper level is convex with respect to upper level variables and optimal lower level variables.The two levels cooperate with each other.

F1 =∑p

i=1(xiu1)

2

F2 =∑q

i=1(xil1)

2

F3 =∑r

i=1(xiu2)

2 +∑r

i=1(xiu2 − tanxi

l2)2

f1 =∑p

i=1(xiu1)

2

f2 =∑q

i=1(xil1)

2

f3 =∑r

i=1(xiu2 − tanxi

l2)2

(12)

10 Evolutionary Computation Volume x, Number x

Upper and Lower Function Contours

Bilevel Test Problems

u2

with respect to lower level variableswith respect to upper level variables

u2u1at (x ,x ) = (−2,−2)

U: Lower level function contours

with respect to lower level variables

l2

l1

l2

l1

x

x

x

x

x

x

x

l2

l1

x

x

u1


u2

x

u2u1at (x ,x ) = (0,0)

S: Lower level function contours


l1x

l2x

l1

R: Lower level function contours

at (x ,x ) = (2,2)u1 u2

Q: Lower level function contours

at (x ,x ) = (2,−2)u1 u2

P: Upper level function contours

at optimal lower level variables


T: Lower level function contours

at (x ,x ) = (−2,2)u1

l24.3 4.1

4.34.5

5

5.5

5

11

4.54.3

4.1 15

15

1.51

0.50.1

5.5

500

20

30

8

1

4

500100500

100

4.1

4.5

55.5

115.5

4.5

4.34.1

4.5

55.5

115.5

4.5

4.3

11

5

5.5

5

100100

500

5.5

4.3

−0.4

−1−0.5 0 0.5 1 1.5−1.5

−1

−0.5

0

0.5

1

1.5

−0.2 0 0.2

−0.4 −0.2 0 0.2 0.4 0.6

0.8

1

1.2

1.4

1.6

0.4−1.6

−1.4

−1.2

−1

−0.8

−0.6

1.6

1.4

1.2

−4 −2 0 2 4

−4

−2

0

2

4

−0.2−0.4

1

−0.6

0

0.4

0.8

0.6 0.4

−1.6

0.2

−1.4

−1.2

−0.4 −0.2 0 0.2

−1

−0.8

−1.5

Figure 3: Upper and lower level function contours for a four-variable SMD1 test problem.

The range of variables is as follows,

xiu1 ∈ [−5, 10], ∀ i ∈ {1, 2, . . . , p}

xiu2 ∈ [−5, 10], ∀ i ∈ {1, 2, . . . , r}

xil1 ∈ [−5, 10], ∀ i ∈ {1, 2, . . . , q}

xil2 ∈ (−π

2 , π2 ), ∀ i ∈ {1, 2, . . . , r}

(13)

Relationship between upper level variables and lower level optimal variables is given as follows,

xil1 = 0, ∀ i ∈ {1, 2, . . . , p}

xil2 = tan−1 xi

u2, ∀ i ∈ {1, 2, . . . , r} (14)

The values of the variables at the optima are xu = 0 and xl is obtained by the relationship givenabove. Both the upper and lower level functions are equal to zero at the optima.

Figure 3 shows the contours of the upper and lower level functions with respect to theupper and lower level variables for a four-variable test problem. The problem has two upperlevel variables and two lower level variables, such that the dimensions of xu1,xu2,xl1 and xu2

are all one. Sub-figure P shows the upper level function contours with respect to the upperlevel variables, assuming that the lower level variables are at the optima. Fixing the upper levelvariables (xu1,xu2) at five different locations, i.e. (2, 2), (−2, 2), (2,−2), (−2,−2) and (0, 0),the lower level function contours are shown with respect to the lower level variables. This showsthat the contours of the lower level optimization problem may be different for different upperlevel vectors.

Figure 4 shows the contours of the upper level function with respect to the upper and lowerlevel variables. Sub-figure P once again shows the upper level function contours with respectto the upper level variables. However, sub-figures Q, R, S, T and V now represent the upper

Evolutionary Computation Volume x, Number x 11

Interaction: Cooperative Lower level: Convex (w.r.t. lower-level variables) Upper level: Convex (induced space) SMD1

with respect to lower level variableswith respect to upper level variablesu2u1at (x ,x ) = (ï2,ï2)

U: Lower level function contourswith respect to lower level variables

l2

u1 u2



with respect to lower level variablesT: Lower level function contours


x

at (x ,x ) = (ï2,2)u1 u2

R: Lower level function contours

l1

l2

l1

x

x

x

x

x

xx

l2

with respect to lower level variablesS: Lower level function contours

at (x ,x ) = (0,0)u1 u2

l1

l1x

x

u1

l2x

l1

at (x ,x ) = (2,2)u1 u2

Q: Lower level function contours

at (x ,x ) = (2,ï2)

u2

x

l2

4.5

20

8

14

30

11

4.5

4.3

5.5

4.1

54.5

5.5

115.5

4.54.1

4.3

5.5

100 100

55.5

500 100500 100

500

5.511

54.3

4.34.1

4.5

55.5

15

0.5

1

0.1

1.5

15

5

4.5

4.14.3

4.35

11

500

5.5

2

ï0.2 0 0.2 0.4ï1.6

ï1.4

ï1.2

ï1

ï0.8

ï0.6

0

ï0.2ï0.4

ï4 ï2 0 2

ï1.5

0.2 0.4 0.6

0.8

1

1.2

1.4

1.6

ï0.4ï0.2 0 0.2 0.4

4

ï4

0.6

ï1ï0.5 0 0.5 1 1.5ï1.5ï1ï0.5 0 0.5 1 1.5

0.8

0

1

1.2

1.4

1.6

ï2

ï0.4 ï0.2 0 0.2 0.4ï1.6

ï1.4

ï1.2

ï1

ï0.8

ï0.6

4

ï0.4





with respect to upper level variables

Q: Upper level function contours

at (x ,x ) = (2,−2)u1 u2



S: Upper level function contours

at (x ,x ) = (0,0)u1 u2


R: Upper level function contours

at (x ,x ) = (2,2)u1 u2


T: Upper level function contours

at (x ,x ) = (−2,2)u1 u2


U: Upper level function contours

at (x ,x ) = (−2,−2)u1 u2

x u1

xu2

xl1

xl2

xl1

l2

l1

l2

x

x

x

x

x

x

x

l1

l2

l1

l28.1

8.5

8.5

9

9.5

9.59

15

100500

8.1

8.5

9

9.5

8.5

500100

9 9.5

15

8.1

8.5

9

9.5

8.5

500100

9 9.5

15

20

30

8

1

4

0.10.5

1

1.5

15

15

8.5

8.5

9

9.5

9.59

15

100500

8.1

−0.4 −0.2 0 0.2 0.4−1.6

−1.4

−1.2

−1

−0.8

−0.6

−0.4 −0.2 0 0.2 0.4 0.6

0.8

1

1.2

1.4

1.6

−0.4 −0.2 0 0.2 0.4 0.6

0.8

1

1.2

1.4

1.6

−4 −2 0 2 4

−4

−2

0

2

4

−1.5 −1 −0.5 0 0.5 1 1.5−1.5

−1

−0.5

0

0.5

1

1.5

−0.4 −0.2 0 0.2 0.4−1.6

−1.4

−1.2

−1

−0.8

−0.6

Figure 4: Upper level function contours for a four-variable SMD1 test problem.

level function contours at different (xu1,xu2), i.e. (2, 2), (−2, 2), (2,−2), (−2,−2) and (0, 0).From sub-figures Q, R, S, T and V, we observe that if the lower level variables move away fromits optimal location, the upper level function value deteriorates. This means that the upper levelfunction and the lower level functions are cooperative.

4.2 SMD2

This test problem is similar to the SDM1 test problem, however there is a conflict between theupper level and lower level optimization task. The lower level optimization problem is onceagain a convex optimization task and the upper level optimization is convex with respect toupper level variables and optimal lower level variables. Since, the two levels are conflicting,an inaccurate lower level optimum may lead to upper level function value better than the trueoptimum for the bilevel problem.

F1 =∑p

i=1(xiu1)

2

F2 = −∑q

i=1(xil1)

2

F3 =∑r

i=1(xiu2)

2 −∑r

i=1(xiu2 − log xi

l2)2

f1 =∑p

i=1(xiu1)

2

f2 =∑q

i=1(xil1)

2

f3 =∑r

i=1(xiu2 − log xi

l2)2

(15)


xiu1 ∈ [−5, 10], ∀ i ∈ {1, 2, . . . , p}

xiu2 ∈ [−5, 1], ∀ i ∈ {1, 2, . . . , r}

xil1 ∈ [−5, 10], ∀ i ∈ {1, 2, . . . , q}

xil2 ∈ (0, e], ∀ i ∈ {1, 2, . . . , r}

(16)








at (x ,x ) = (2,−2)u1 u2




at (x ,x ) = (0,0)u1 u2



at (x ,x ) = (2,2)u1 u2



at (x ,x ) = (−2,2)u1 u2



at (x ,x ) = (−2,−2)u1 u2

x u1

xu2

xl1

xl2

xl1

l2

l1

l2

x

x

x

x

x

x

x

l1

l2

l1

l28.1

8.5

8.5

9

9.5

9.59

15

100500

8.1

8.5

9

9.5

8.5

500100

9 9.5

15

8.1

8.5

9

9.5

8.5

500100

9 9.5

15

20

30

8

1

4

0.10.5

1

1.5

15

15

8.5

8.5

9

9.5

9.59

15

100500

8.1

−0.4 −0.2 0 0.2 0.4−1.6

−1.4

−1.2

−1

−0.8

−0.6

−0.4 −0.2 0 0.2 0.4 0.6

0.8

1

1.2

1.4

1.6

−0.4 −0.2 0 0.2 0.4 0.6

0.8

1

1.2

1.4

1.6

−4 −2 0 2 4

−4

−2

0

2

4

−1.5 −1 −0.5 0 0.5 1 1.5−1.5

−1

−0.5

0

0.5

1

1.5

−0.4 −0.2 0 0.2 0.4−1.6

−1.4

−1.2

−1

−0.8

−0.6


level function contours at different (xu1,xu2), i.e. (2, 2), (−2, 2), (2,−2), (−2,−2) and (0, 0).From sub-figures Q, R, S, T and V, we observe that if the lower level variables move away fromits optimal location, the upper level function value deteriorates. This means that the upper levelfunction and the lower level functions are cooperative.

4.2 SMD2

This test problem is similar to the SDM1 test problem, however there is a conflict between theupper level and lower level optimization task. The lower level optimization problem is onceagain a convex optimization task and the upper level optimization is convex with respect toupper level variables and optimal lower level variables. Since, the two levels are conflicting,an inaccurate lower level optimum may lead to upper level function value better than the trueoptimum for the bilevel problem.

F1 =∑p

i=1(xiu1)

2

F2 = −∑q

i=1(xil1)

2

F3 =∑r

i=1(xiu2)

2 −∑r

i=1(xiu2 − log xi

l2)2

f1 =∑p

i=1(xiu1)

2

f2 =∑q

i=1(xil1)

2

f3 =∑r

i=1(xiu2 − log xi

l2)2

(15)


xiu1 ∈ [−5, 10], ∀ i ∈ {1, 2, . . . , p}

xiu2 ∈ [−5, 1], ∀ i ∈ {1, 2, . . . , r}

xil1 ∈ [−5, 10], ∀ i ∈ {1, 2, . . . , q}

xil2 ∈ (0, e], ∀ i ∈ {1, 2, . . . , r}

(16)


Interaction: Conflicting Lower level: Convex (w.r.t. lower-level variables) Upper level: Convex (induced space) SMD2

at (x ,x ) = (0,0)u1 u2

with respect to lower level variablesR: Lower level function contours

at (x ,x ) = (2,2)u1 u2

with respect to lower level variablesU: Lower level function contours

at (x ,x ) = (ï2,ï2)u1 u2


at (x ,x ) = (ï2,2)u1 u2

P: Upper level function contours Q: Lower level function contours

at (x ,x ) = (2,ï2)u1 u2at optimal lower level variableswith respect to upper level variables with respect to lower level variables

u2

x

x

l2

with respect to lower level variablesS: Lower level function contours

l1

l2

l1

l2

l1

x

x

x

x

xl2

l1x

l2x

l1x

x

u1x

7.55 5.5

4.14.01

4.5

5

5.5

0.5

11.5

7.55 5.5

4.1

4.5

4.01

5

5.5

0.1

5.5

8

20

4.5

5

4.5

4.01

5.5 5

4.1

4.5

5

4.1

30

41

4.5

4.01

5

0.2 0ï0.2ï0.4 ï0.2 0 0.2 0.4

2

4

6

8

10

12

14

16

18

20

ï0.4

ï4 ï2 0 2 4

ï4

ï2

0

2

4

ï1.5 ï1 ï0.5 0 0.5 1 1.5

0.5

1

1.5

2

2.5

3

3.5

4

ï0.4 ï0.2 0 0.2 0.4

0.1

0.2

0.3

0.4

0.5

0.6

ï0.4 ï0.2 0 0.2 0.4

0.1

0.2

0.3

0.4

0.5

0.6

20

18

16

14

12

10

8

6

4

2

0.4







at (x ,x ) = (2,−2)u1 u2




at (x ,x ) = (0,0)u1 u2



at (x ,x ) = (2,2)u1 u2



at (x ,x ) = (−2,−2)u1 u2



at (x ,x ) = (−2,2)u1 u2

x u1

x

xl1

xl2

xl1

l2x

x

x

x

x

l1

l2

l1

l2

l1

l2

x

x

u2

7.9

7.5

7

6.5

7.57 6.5

7.997.9

7.5

7

6.5

7.57 6.5

7.99

6.57

7.5

7.9

7.99

7.5

7

−1

−1.5

−0.5

−0.1

76.5

7.99

7.5

7

7.5

7.9

20

8

1

4

30

−0.4 −0.2 0 0.2 0.4

0.1

0.2

0.3

0.4

0.5

0.6

−0.4 −0.2 0 0.2 0.4

0.1

0.2

0.3

0.4

0.5

0.6

−0.4 −0.2 0 0.2 0.4

2

4

6

8

10

12

14

16

18

20

−1.5 −1 −0.5 0 0.5 1 1.5

0.5

1

1.5

2

2.5

3

3.5

4

−0.4 −0.2 0 0.2 0.4

2

4

6

8

10

12

14

16

18

20

−4 −2 0 2 4

−4

−2

0

2

4


level is convex with respect to upper level variables and optimal lower level variables.

F1 =∑p

i=1(xiu1)

2

F2 =∑q

i=1(xil1)

2

F3 =∑r

i=1(xiu2)

2 +∑r

i=1((xiu2)

2 − tanxil2)

2

f1 =∑p

i=1(xiu1)

2

f2 = q +∑q

i=1

(

(

xil1

)2 − cos 2πxil1

)

f3 =∑r

i=1((xiu2)

2 − tanxil2)

2

(18)


xiu1 ∈ [−5, 10], ∀ i ∈ {1, 2, . . . , p}

xiu2 ∈ [−5, 10], ∀ i ∈ {1, 2, . . . , r}

xil1 ∈ [−5, 10], ∀ i ∈ {1, 2, . . . , q}

xil2 ∈ (−π

2 , π2 ), ∀ i ∈ {1, 2, . . . , r}

(19)


xil1 = 0, ∀ i ∈ {1, 2, . . . , q}

xil2 = tan−1(xi

u2)2, ∀ i ∈ {1, 2, . . . , r} (20)

The values of the variables at the optima are xu = 0 and xl is obtained by the relationship givenabove. Both the upper and lower level functions are equal to zero at the optima. Rastrigin’sfunction used in f2 has multiple local optima around the global optimum, which introducesconvergence difficulties at the lower level.

Sub-figure P in Figure 7 shows the contours of the upper level function with respect to theupper level variables assuming the lower level variables to be optimal at each xu. Sub-figures Q,








at (x ,x ) = (2,−2)u1 u2




at (x ,x ) = (0,0)u1 u2



at (x ,x ) = (2,2)u1 u2



at (x ,x ) = (−2,−2)u1 u2



at (x ,x ) = (−2,2)u1 u2

x u1

x

xl1

xl2

xl1

l2x

x

x

x

x

l1

l2

l1

l2

l1

l2

x

x

u2

7.9

7.5

7

6.5

7.57 6.5

7.997.9

7.5

7

6.5

7.57 6.5

7.99

6.57

7.5

7.9

7.99

7.5

7

−1

−1.5

−0.5

−0.1

76.5

7.99

7.5

7

7.5

7.9

20

8

1

4

30

−0.4 −0.2 0 0.2 0.4

0.1

0.2

0.3

0.4

0.5

0.6

−0.4 −0.2 0 0.2 0.4

0.1

0.2

0.3

0.4

0.5

0.6

−0.4 −0.2 0 0.2 0.4

2

4

6

8

10

12

14

16

18

20

−1.5 −1 −0.5 0 0.5 1 1.5

0.5

1

1.5

2

2.5

3

3.5

4

−0.4 −0.2 0 0.2 0.4

2

4

6

8

10

12

14

16

18

20

−4 −2 0 2 4

−4

−2

0

2

4


level is convex with respect to upper level variables and optimal lower level variables.

F1 =∑p

i=1(xiu1)

2

F2 =∑q

i=1(xil1)

2

F3 =∑r

i=1(xiu2)

2 +∑r

i=1((xiu2)

2 − tanxil2)

2

f1 =∑p

i=1(xiu1)

2

f2 = q +∑q

i=1

(

(

xil1

)2 − cos 2πxil1

)

f3 =∑r

i=1((xiu2)

2 − tanxil2)

2

(18)


xiu1 ∈ [−5, 10], ∀ i ∈ {1, 2, . . . , p}

xiu2 ∈ [−5, 10], ∀ i ∈ {1, 2, . . . , r}

xil1 ∈ [−5, 10], ∀ i ∈ {1, 2, . . . , q}

xil2 ∈ (−π

2 , π2 ), ∀ i ∈ {1, 2, . . . , r}

(19)


xil1 = 0, ∀ i ∈ {1, 2, . . . , q}

xil2 = tan−1(xi

u2)2, ∀ i ∈ {1, 2, . . . , r} (20)

The values of the variables at the optima are xu = 0 and xl is obtained by the relationship givenabove. Both the upper and lower level functions are equal to zero at the optima. Rastrigin’sfunction used in f2 has multiple local optima around the global optimum, which introducesconvergence difficulties at the lower level.

Sub-figure P in Figure 7 shows the contours of the upper level function with respect to theupper level variables assuming the lower level variables to be optimal at each xu. Sub-figures Q,


Interaction: Cooperative Lower level: Multimodality using Rastrigin’s function Upper level: Convex (induced space)

SMD3


at optimal lower level variableswith respect to upper level variables

u2


at (x ,x ) = (ï2,ï2)u1 u2

T: Lower level function contours

at (x ,x ) = (ï2,2)u1 u2


at (x ,x ) = (0,0)u1 u2R: Lower level function contours

at (x ,x ) = (2,2)u1 u2

with respect to lower level variablesQ: Lower level function contours

at (x ,x ) = (2,ï2)u1 u2

x u1

x

xl1

xl2

xl1

l2x

x

x

x

x

l1

l2

l1

l2

l1

l2

x

x




with respect to lower level variables 30

8

14

20

2

10

1

0.12

11 2

3

10

5

915

25

6

9

664.1

5

915

25

6

9

664.1

5

915

25

6

9

664.1

5

915

25

6

9

664.1

ï4 ï2 0 2 4

ï4

ï2

0

2

4

ï1.5 ï1 ï0.5 0 0.5 1 1.5ï1.5

ï1

ï0.5

0

0.5

1

1.5

ï1.5 ï1 ï0.5 0 0.5 1 1.5 1

1.1

1.2

1.3

1.4

1.5

ï1.5 ï1 ï0.5 0 0.5 1 1.5 1

1.1

1.2

1.3

1.4

1.5

ï1.5 ï1 ï0.5 0 0.5 1 1.5 1

1.1

1.2

1.3

1.4

1.5

ï1.5 ï1 ï0.5 0 0.5 1 1.5 1

1.1

1.2

1.3

1.4

1.5


SMD4 A. Sinha, P. Malo and K. Deb

is convex with respect to upper level variables and optimal lower level variables.

F1 =∑p

i=1(xiu1)

2

F2 = −∑q

i=1(xil1)

2

F3 =∑r

i=1(xiu2)

2 −∑r

i=1(|xiu2|− log(1 + xi

l2))2

f1 =∑p

i=1(xiu1)

2

f2 = q +∑q

i=1

(

(

xil1

)2 − cos 2πxil1

)

f3 =∑r


l2))2

(21)


xiu1 ∈ [−5, 10], ∀ i ∈ {1, 2, . . . , p}

xiu2 ∈ [−1, 1], ∀ i ∈ {1, 2, . . . , r}

xil1 ∈ [−5, 10], ∀ i ∈ {1, 2, . . . , q}

xil2 ∈ [0, e], ∀ i ∈ {1, 2, . . . , r}

(22)


xil1 = 0, ∀ i ∈ {1, 2, . . . , q}

xil2 = log−1 |xi

u2|− 1, ∀ i ∈ {1, 2, . . . , r} (23)


Figure 8 represents the same information as in Figure 7 for a four-variable bilevel prob-lem. However, this problem involves conflict between the two levels, which make the problemsignificantly more difficult than the previous test problem. For this test problem if a lower leveloptimization problem is stuck at a local optimum for a particular xu, then it will end up havinga better objective function value at the upper level, than what it will attain at the true globallower level optimum. Therefore, even if another lower lower level optimization problem is suc-cessfully solved in the vicinity of xu, the previous inaccurate solution will dominate the newsolution at the upper level. This problem can be handled only by those methods which are ableto efficiently handle lower level multi-modality without getting stuck in a local basin.

4.5 SMD5

In this test problem, there is a conflict between the two levels. The difficulty introduced is interms of multi-modality and convergence at the lower level. The lower level problem containsthe Rosenbrock’s (banana) function such that the global optimum lies in a long, narrow, flatparabolic valley. The upper level is convex with respect to upper level variables and optimallower level variables.

F1 =∑p

i=1(xiu1)

2

F2 = −∑q

i=1

((

xi+1l1 −

(

xil1

)2)

+(

xil1 − 1

)2)

F3 =∑r

i=1(xiu2)

2 −∑r

i=1(|xiu2|− (xi

l2)2)2

f1 =∑p

i=1(xiu1)

2

f2 =∑q

i=1

((

xi+1l1 −

(

xil1

)2)

+(

xil1 − 1

)2)

f3 =∑r

i=1(|xiu2|− (xi

l2)2)2

(24)




is convex with respect to upper level variables and optimal lower level variables.

F1 =∑p

i=1(xiu1)

2

F2 = −∑q

i=1(xil1)

2

F3 =∑r

i=1(xiu2)

2 −∑r


l2))2

f1 =∑p

i=1(xiu1)

2

f2 = q +∑q

i=1

(

(

xil1

)2 − cos 2πxil1

)

f3 =∑r


l2))2

(21)


xiu1 ∈ [−5, 10], ∀ i ∈ {1, 2, . . . , p}

xiu2 ∈ [−1, 1], ∀ i ∈ {1, 2, . . . , r}

xil1 ∈ [−5, 10], ∀ i ∈ {1, 2, . . . , q}

xil2 ∈ [0, e], ∀ i ∈ {1, 2, . . . , r}

(22)


xil1 = 0, ∀ i ∈ {1, 2, . . . , q}

xil2 = log−1 |xi

u2|− 1, ∀ i ∈ {1, 2, . . . , r} (23)


Figure 8 represents the same information as in Figure 7 for a four-variable bilevel prob-lem. However, this problem involves conflict between the two levels, which make the problemsignificantly more difficult than the previous test problem. For this test problem if a lower leveloptimization problem is stuck at a local optimum for a particular xu, then it will end up havinga better objective function value at the upper level, than what it will attain at the true globallower level optimum. Therefore, even if another lower lower level optimization problem is suc-cessfully solved in the vicinity of xu, the previous inaccurate solution will dominate the newsolution at the upper level. This problem can be handled only by those methods which are ableto efficiently handle lower level multi-modality without getting stuck in a local basin.

4.5 SMD5

In this test problem, there is a conflict between the two levels. The difficulty introduced is interms of multi-modality and convergence at the lower level. The lower level problem containsthe Rosenbrock’s (banana) function such that the global optimum lies in a long, narrow, flatparabolic valley. The upper level is convex with respect to upper level variables and optimallower level variables.

F1 =∑p

i=1(xiu1)

2

F2 = −∑q

i=1

((

xi+1l1 −

(

xil1

)2)

+(

xil1 − 1

)2)

F3 =∑r

i=1(xiu2)

2 −∑r

i=1(|xiu2|− (xi

l2)2)2

f1 =∑p

i=1(xiu1)

2

f2 =∑q

i=1

((

xi+1l1 −

(

xil1

)2)

+(

xil1 − 1

)2)

f3 =∑r

i=1(|xiu2|− (xi

l2)2)2

(24)


Interaction: Conflicting Lower level: Multimodality using Rastrigin’s function Upper level: Convex (induced Space)


at optimal lower level variableswith respect to upper level variables

u2


at (x ,x ) = (ï2,ï2)u1 u2


at (x ,x ) = (0,0)u1 u2R: Lower level function contours

at (x ,x ) = (2,2)u1 u2

with respect to lower level variablesQ: Lower level function contours

at (x ,x ) = (2,ï2)u1 u2

x u1

x

xl1

xl2

xl1

l2x

x

x

x

x

l1

l2

l1

l2

l1

l2

x

x





at (x ,x ) = (ï2,2)u1 u230

8

14

20

0.1

1

11

2

2 2

3

5

4.1

5

5 5

6

6

6

6

6

6

5

4.1

5

5 5

6

6

6

6

6

6

5

4.1

5

5 5

6

6

6

6

6

6

5

4.1

5

5 5

6

6

6

6

6

6

5

ï4 ï2 0 2 4

ï4

ï2

0

2

4

ï1.5 ï1 ï0.5 0 0.5 1 1.5

0

1

2

3

4

5

ï1 ï0.5 0 0.5 1 3

4

5

6

7

8

9

10

ï1 ï0.5 0 0.5 1 3

4

5

6

7

8

9

10

ï1 ï0.5 0 0.5 1 3

4

5

6

7

8

9

10

ï1 ï0.5 0 0.5 1 3

4

5

6

7

8

9

10


Bilevel Evolutionary Algorithm Based on Quadratic Approximations

•  Recently proposed by Sinha, Malo and Deb (2013) for solving large instances of bilevel optimization problems

•  Based on two aspects 1.  Assumption that close upper level members have close

lower level optimal solutions 2.  Quadratic approximation of the inducible region

•  Method is highly efficient when compared against nested approaches and other contemporary evolutionary techniques

•  Tested on a recently proposed SMD test-suit and other standard test problems from the literature

BLE

AQ


BLE

AQ

Close upper level members are expected to have close lower level optimal solutions

Aspect 1


BLE

AQ

Approximation with localization

Aspect 2


Results

•  Mean (total function evaluations) results for ten bilevel test problems •  Comparison against the evolutionary algorithm of Wang et al. (2005,2011) Number of Runs: 21 Savings: Ratio of FE required by nested approach against BLEAQ


Results (cont.)

u  Following are the results for 10 variable instances of the SMD test problems (Sinha et al., 2014) using BLEAQ

u  Comparison performed against nested evolutionary approach

Number of Runs: 21 Savings: Ratio of FE required by nested approach against BLEAQ


−2 −1.5 −1 −0.5 0 0.5 1 1.5 2−2

−1.5

−1

−0.5

0

0.5

1

1.5

2

xu

4

xl4

True Relation

Gen. 1

Gen 150

Gen. 450

Gen. 550

Gen. 610

Figure 12: Quadratic relationship convergence.

also provides the median number of lower level calls, and the average number oflower level function evaluations required per lower level call. Table 10 comparesthe mean function evaluations at the upper and lower levels required by BLEAQagainst that required by WJL. WJl requires close to an order of magnitude timesmore function evaluations at the lower level, and two orders of magnitude timesmore function evaluations at the upper level, for most of the test problems. Thisclearly demonstrates the e�ciency gain obtained using the BLEAQ approach.It also suggests that the mathematical insights used along with the evolutionaryprinciples in the BLEAQ approach is helpful in converging quickly towards thebilevel optimal solution.

Table 8: Function evaluations (FE) for the upper level (UL) and the lower level(LL) from 11 runs of the proposed BLEAQ.

Pr. No. Best Func. Evals. Median Func. Evals. Worst Func. Evals.LL UL LL UL LL UL

TP1 14115 718 15041 780 24658 1348

TP2 12524 1430 14520 1434 16298 2586

TP3 4240 330 4480 362 6720 518

TP4 14580 234 15300 276 15480 344

TP5 10150 482 15700 1302 15936 1564

TP6 14667 230 17529 284 21875 356

TP7 234622 3224 267784 4040 296011 5042

TP8 10796 1288 12300 1446 18086 2080

30

SM

D1

Qua

drat

ic R

elat

ions

hip

Con

verg

ence

Quadratic approximation at optima (0,0) improves with increasing generations

Results on SMD1


Table 7: Accuracy for the upper and lower levels, and the lower level calls from11 runs of the proposed BLEAQ.

Pr. No. Median Median Median

UL Accuracy LL Accuracy LL Calls LL EvalsLL Calls

SDM1 0.006828 0.003521 528 202.47SDM2 0.003262 0.002745 522 176.09SDM3 0.009122 0.004364 603 225.91SDM4 0.006957 0.002716 574 146.39SDM5 0.004103 0.003773 513 243.68SDM6 0.000000 0.000000 167 50.24

0 50 100 150 200 250 300 350 400 450 500 55010

−4

10−2

100

102

F0

0 50 100 150 200 250 300 350 400 450 500 55010

−4

10−2

100

102

Upper Level Generations

f 0

Figure 10: Convergence plot for SMD1.

0 50 100 150 200 250 300 350 400 450 500 55010

−4

10−2

100

102

F0

0 50 100 150 200 250 300 350 400 450 500 55010

−4

10−2

100

102

Upper Level Generations

f 0

Figure 11: Convergence plot for SMD2.

approximations generated at various generations. It can be observed that theapproximations in the vicinity of the true bilevel optimum improves with in-creasing number of generations.

8.3 Results for constrained test problems

In this sub-section, we provide results for 8 standard constrained test problemschosen from the literature. We compare our results against the approach pro-posed by Wang et al. (2005). The reason for the choice of this approach as abenchmark is that the approach was successful in solving all the chosen con-strained test problems. The results obtained using the BLEAQ and WJL [25]approach has been provided in Tables 8, 9 and 10. Table 8 provides the mini-mum, median and maximum function evaluations required to solve the chosenproblems using the BLEAQ approach. Table 9 provides the accuracy obtainedat both the levels, in terms of absolute di↵erence from the best known solutionto a particular test problem. The numbers in the brackets provide the bestsolution known from the literature for each of the test problems. The table

29

Upper Level Func,on Value

Lower Level Func,on Value

Pro

gres

s pa

th fo

r the

elit

e m

embe

r

Convergence Plots on SMD1


Advanced Topics on EBO

Multi-objective EBO At least one level has multiple objectives

MEBO with decision-making One or both levels involve a decision maker

Robust EBO Uncertainty in at least one level


Bilevel Multi-objective Problems •  Bilevel problems may involve optimization of multiple objectives at one

or both of the levels •  Dempe et al. (2006) developed KKT conditions (difficult to implement) •  Little work has been done in the direction of multi-objective bilevel

algorithms (Eichfelder (2007), Deb and Sinha (2010)) •  A general multi-objective bilevel problem may be formulated as follows:


" Population structure " Lower and upper level

NSGA-II " Archiving

BLEMO (Deb and Sinha, 2009)


BLEMO Features: Different from a Pure Nested Procedure

•  A population of xu is initialized •  Lower level NSGA-II attempts to move towards different Pareto-

optimal fronts

•  Based on intermediate information, xu population is updated to move towards feasible and Pareto-optimal region of upper level problem •  Again, lower level NSGA-II attempts to move towards focused

areas on lower level Pareto-fronts •  And so on …


•  Lower level Pareto front depends on x •  Upper level Pareto-optimal front lies on

constraint G1 •  Maximum two solutions from each x •  Not all x in upper Pareto-optimal front

•  Solutions possible even below the upper level Pareto-optimal front

•  Dashed lines represent lower level Pareto fronts

•  Solid line represents upper level Pareto front

MEBO: Example 1

(Eichfelder, 2007)


Results: Example 1

Nu=400, Tu=200 Nl=40, Tl=40 Near PO solutions found

x

y2 y1


Lower vs Upper Level: Example 1 •  Identical function evaluations •  Large Tl means nested with less importance on

upper level •  Small Tl means less care on lower level

•  Tl=40 seems adequate

•  Refer Sinha and Deb (2009), Deb and Sinha (2010)

Gen = 2 Gen = 3


•  Only one point from a lower level PO front is on upper level PO front

•  Scalable to higher dimensional problems

•  15 variable version is also solved

(Deb and Sinha, 2009)

MEBO: Example 2


Results: Example 2 Close Pareto-optimal solutions are found by BLEMO

x

y1


Extensions to This Study

•  Difficult test problems suggested (Deb and Sinha, 2009, 2010) •  Allows an adequate evaluation for MEBO algorithms

•  Self-adaptive and local search based BLEMO suggested (Deb and Sinha, 2010) •  Much faster approach (H-BLEMO) •  No additional parameter

•  Interaction with the decision maker at upper level leads to substantial savings in function evaluations (Sinha, 2011)


DS-Test Problems

• S

cala

ble

in te

rms

of v

aria

bles

and

ob

ject

ives

• 

Con

trolla

ble

diffi

culti

es


Applications


Taxation Strategy

Leader: Government Maximize revenue from taxes Minimize Pollution

Follower: Mining Company Maximize Profit

•  Recently, there has been a controversy in Finland for gold mining in the Kuusamo region

•  The region is a famous tourist resort endowed with immense natural beauty

•  For any taxation strategy by the government, the mining company optimizes its own profits

Sinha, Malo and Deb (2013)


Preferred strategy: ~75% profit to the government, ~25% to company

a=2

a=3

a=4

a=1

−1000

−900

−800

−700

−600

−500

−400

−300

−200

−100

0 500 1000 1500 2000 2500 3000

0

3500 4000

Fig. 4. Approximate Pareto-frontier for the extended model takingall technologies into account.

RegionPreferreda=1

a=2

a=3

a=4

−600

−500

−400

−300

−200

−100

0 1000 1500 2000 2500 3000

0

3500 4000 500−1000

−900

−800

−700

Fig. 5. Approximate Pareto-frontiers for the individual technologiesconsidered one at a time.

1 2 3 4 58

10

12

14

16

18

20

22

Time Period

Quantit

y of M

eta

l Pro

duce

d

362.3 <= D <= 399.8

2650.0 <= R <= 2776.8

a=2

Fig. 6. Approximate Pareto-frontier for the extended model taking alltechnologies into account.

0 2 3 4 528

30

32

34

36

38

40

42

Time Period

Tax

per

unit

of m

eta

l

2650.0 <= R <= 2776.8

362.3 <= D <= 399.8

a=2


VIII. ACKNOWLEDGMENTS

The authors wish to thank the Academy of Finland andthe Foundation of the Helsinki School of Economics forsupporting this study.

REFERENCES

[1] E. Aiyoshi and K. Shimizu. Hierarchical decentralized systems andits new solution by a barrier method. IEEE Transactions on Systems,Man, and Cybernetics, 11:444–449, 1981.

[2] J. Bard. Practical Bilevel Optimization: Algorithms and Applications.The Netherlands: Kluwer, 1998.

[3] J. Bard and J. Falk. An explicit solution to the multi-level programmingproblem. Computers and Operations Research, 9:77–100, 1982.

[4] L. Bianco, M. Caramia, and S. Giordani. A bilevel flow model forhazmat transportation network design. Transportation Research PartC: Emerging technologies, 17(2):175–196, 2009.

[5] B. Colson, P. Marcotte, and G. Savard. An overview of bileveloptimization. Annals of Operational Research, 153:235–256, 2007.

[6] R. F. Conrad. Output taxes and the quantity-quality trade-off in themining firm. Resources and Energy, 3(3):207 – 221, 1981.

[7] R. F. Conrad and B. Hool. Resource taxation with heterogeneousquality and endogenous reserves. Journal of Public Economics,16(1):17 – 33, 1981.

[8] K. Deb and A. Sinha. An efficient and accurate solution methodologyfor bilevel multi-objective programming problems using a hybridevolutionary-local-search algorithm. Evolutionary Computation Jour-nal, 18(3):403–449, 2010.

[9] S. Dempe, J. Dutta, and S. Lohse. Optimality conditions for bilevelprogramming problems. Optimization, 55(5âAS6):505âAS–524, 2006.

[10] W. Halter and S. Mostaghim. Bilevel optimization of multi-componentchemical systems using particle swarm optimization. In Proceedingsof World Congress on Computational Intelligence (WCCI-2006), pages1240–1247, 2006.

[11] J. F. Helliwell. Effects of taxes and royalties on copper mininginvestment in british columbia. Resources Policy, 4(1):35 – 44, 1978.

[12] J. Herskovits, A. Leontiev, G. Dias, and G. Santos. Contact shapeoptimization: A bilevel programming approach. Struct MultidiscOptimization, 20:214–221, 2000.

[13] R. R. Pakala. A study on applications of stackelberg game strategiesin concurrent design models, 1993.

[14] S. Ruuska and K. Miettinen. Constructing evolutionary algorithmsfor bilevel multiobjective optimization. In 2012 IEEE Congress onEvolutionary Computation (CEC-2012), 2012.

Pareto-frontier for the leader’s problem

Leader’s profit

- Env

ironm

enta

l Dam

age

RegionPreferreda=1

a=2

a=3

a=4

−600

−500

−400

−300

−200

−100

0 1000 1500 2000 2500 3000

0

3500 4000 500−1000

−900

−800

−700

Fig. 4. Approximate Pareto-frontier for the extended model takingall technologies into account.

a=2

a=3

a=4

a=1

−1000

−900

−800

−700

−600

−500

−400

−300

−200

−100

0 500 1000 1500 2000 2500 3000

0

3500 4000


1 2 3 4 58

10

12

14

16

18

20

22

Time Period

Qu

an

tity

of

Me

tal P

rod

uce

d

362.3 <= D <= 399.8

2650.0 <= R <= 2776.8

a=2

Fig. 6. Approximate Pareto-frontier for the extended model taking alltechnologies into account.

1 2 3 4 528

30

32

34

36

38

40

42

Time Period

Ta

x p

er

un

it o

f m

eta

l

2650.0 <= R <= 2776.8

362.3 <= D <= 399.8

a=2


VIII. ACKNOWLEDGMENTS

The authors wish to thank the Academy of Finland andthe Foundation of the Helsinki School of Economics forsupporting this study.

REFERENCES

[1] E. Aiyoshi and K. Shimizu. Hierarchical decentralized systems andits new solution by a barrier method. IEEE Transactions on Systems,Man, and Cybernetics, 11:444–449, 1981.

[2] J. Bard. Practical Bilevel Optimization: Algorithms and Applications.The Netherlands: Kluwer, 1998.

[3] J. Bard and J. Falk. An explicit solution to the multi-level programmingproblem. Computers and Operations Research, 9:77–100, 1982.

[4] L. Bianco, M. Caramia, and S. Giordani. A bilevel flow model forhazmat transportation network design. Transportation Research PartC: Emerging technologies, 17(2):175–196, 2009.

[5] B. Colson, P. Marcotte, and G. Savard. An overview of bileveloptimization. Annals of Operational Research, 153:235–256, 2007.

[6] R. F. Conrad. Output taxes and the quantity-quality trade-off in themining firm. Resources and Energy, 3(3):207 – 221, 1981.

[7] R. F. Conrad and B. Hool. Resource taxation with heterogeneousquality and endogenous reserves. Journal of Public Economics,16(1):17 – 33, 1981.

[8] K. Deb and A. Sinha. An efficient and accurate solution methodologyfor bilevel multi-objective programming problems using a hybridevolutionary-local-search algorithm. Evolutionary Computation Jour-nal, 18(3):403–449, 2010.

[9] S. Dempe, J. Dutta, and S. Lohse. Optimality conditions for bilevelprogramming problems. Optimization, 55(5âAS6):505âAS–524, 2006.

[10] W. Halter and S. Mostaghim. Bilevel optimization of multi-componentchemical systems using particle swarm optimization. In Proceedingsof World Congress on Computational Intelligence (WCCI-2006), pages1240–1247, 2006.

[11] J. F. Helliwell. Effects of taxes and royalties on copper mininginvestment in british columbia. Resources Policy, 4(1):35 – 44, 1978.

[12] J. Herskovits, A. Leontiev, G. Dias, and G. Santos. Contact shapeoptimization: A bilevel programming approach. Struct MultidiscOptimization, 20:214–221, 2000.

[13] R. R. Pakala. A study on applications of stackelberg game strategiesin concurrent design models, 1993.

[14] S. Ruuska and K. Miettinen. Constructing evolutionary algorithmsfor bilevel multiobjective optimization. In 2012 IEEE Congress onEvolutionary Computation (CEC-2012), 2012.

Taxation strategies in preferred region.

a denotes technologies

Taxation Strategy (Results) Results

Sinha, Malo and Deb (2013)


•  Two levels of decision making •  Multiple objectives involved at

both the levels

Leader: CEO of a Company

Objectives: Max Company Profit

Max Product Quality

Followers: Department Heads

Objectives: Max Department Profit

Max Employee Satisfaction

CEO Department Head

A Company Scenario

Company Profit

Qua

lity

Dep

t. P

rofit

Employee Satisfaction

Follower

Leader

Zhang et al. (2007), Deb and Sinha (2009)


Weighted sum solution (Zhang et al, 2007) is an extreme solution

A Company Scenario (cont.)

Zhang et al. (2007), Deb and Sinha (2009)


MEBO under Lower Level Decision Uncertainty

•  Preference in LL Pareto front may not lead to UL Pareto solutions

•  Converse is not true

•  Raises interesting hierarchy among UL and LL decision-makers

We are currently pursuing these scenarios


Robust EBO

•  Uncertainty in LL variables may make LL optimum sensitive •  Find robust LL solution, but sacrifice in UL function value

•  Uncertainty in UL variables/parameters may make UL optimum sensitive •  Robust UL solution may not come from robust LL solution

•  Uncertainty in both LL and UL variables •  Create interesting scenarios, which we are currently pursuing


Conclusions

•  Bilevel problems are plenty in practice, but are avoided due to lack of efficient methods

•  Bilevel optimization received lukewarm interest by EA researchers so far

•  Population approach of EA makes tremendous potential •  Nested nature of the problem makes the task

computationally expensive •  Meta-modeling based EBO and its extensions show

promise •  More collaborative efforts are needed


References Ø  J.F. Bard. Practical bilevel optimization: Algorithms and applications. The Netherlands: Kluwer, 1998.

Ø  Brotcorne, M. Labbé, P. Marcotte, G. Savard and L. M. Houy. A Bilevel Model for Toll Optimization on a Multicommodity Transportation Network, Transportation Science, pages 345-358, 35(4), 2001.

Ø  S. Dempe. Foundations of bilevel programming. Kluwer, Dordrecht, 2002.

Ø  Dempe, S., Dutta, J., and Lohse, S. (2006). Optimality conditions for bilevel programming problems. Optimization, 55(5-6), 505–524.

Ø  K. Deb and A. Sinha. Constructing test problems for bilevel evolutionary multi-objective optimization. In 2009 IEEE Congress on Evolutionary Computation (CEC-2009), pages 1153--1160. IEEE Press, 2009.

Ø  K. Deb and A. Sinha. Solving bilevel multi-objective optimization problems using evolutionary algorithms. In Evolutionary Multi-Criterion Optimization (EMO-2009), pages 110--124. Berlin, Germany: Springer-Verlag, 2009.

Ø  K. Deb and A. Sinha. An efficient and accurate solution methodology for bilevel multi-objective programming problems using a hybrid evolutionary-local-search algorithm. Evolutionary Computation Journal, pages 403-449, 14(5). MIT Press, 2010.

Ø  G. Eichfelder. Solving Nonlinear Multiobjective Bilevel Optimization Problems with Coupled Upper Level Constraints, Preprint No. 320, Preprint-Series of the Institute of Applied Mathematics, Univ. Erlangen-Nürnberg, Germany, 2007.


References (cont.) Ø  Fliege, J. and Vicente, L. N. (2006). Multicriteria approach to bilevel optimization. Journal of

Optimization Theory and Applications, 131(2), 209–225.

Ø  Herskovits, J., Leontiev, A., Dias, G., and Santos, G. (2000). Contact shape optimization: A bilevel programming approach. Struct. Multidisc. Optimization, 20, 214–221.

Ø  G. Heilporn, M. Labbé, P. Marcotte and G. Savard. A parallel between two classes of pricing problems in transportation and marketing. Journal of Revenue and Pricing Management, pages 110-125, 9(1), 2010.

Ø  A. Sinha, P. Malo, A. Frantsev, and K. Deb. Multi-objective stackelberg game between a regulating authority and a mining company: A case study in environmental economics. In 2013 IEEE Congress on Evolutionary Computation (CEC-2013). IEEE Press, 2013.

Ø  A. Sinha, P. Malo, A. Frantsev, and K. Deb. Finding optimal strategies in multi-period multi- leader-follower stackelberg games using an evolutionary framework. Computers and Operations Research, 41(1), 374-385, 2014. 2014.

Ø  A. Sinha, P. Malo and K. Deb. Unconstrained scalable test problems for single-objective bilevel optimization. In 2012 IEEE Congress on Evolutionary Computation (CEC-2012). IEEE Press, 2012.

Ø  A. Sinha, P. Malo, and K. Deb. Efficient evolutionary algorithm for single-objective bilevel optimization. CoRR, abs/1303.3901, 2013.

Ø  A. Sinha, P. Malo, and K. Deb. Test Problem Construction for Single-Objective Bilevel Optimization. Evolutionary Computation Journal, MIT Press, 2014 (In press).


References (cont.) Ø  A. Sinha, P. Malo and K. Deb. Multi-objective Stackelberg Game Between a Regulating Authority and a

Mining Company: A Case Study in Environmental Economics. In 2013 IEEE Congress on Evolutionary Computation (CEC 2013), IEEE Press.

Ø  A. Sinha and K. Deb. Towards understanding evolutionary bilevel multi-objective optimization algorithm. In IFAC Workshop on Control Applications of Optimization (IFAC-2009), volume 7. Elsevier, 2009.

Ø  A. Sinha. Bilevel Multi-Objective Optimization Problem Solving Using Progressively Interactive EMO. In Proceedings of the Sixth International Conference on Evolutionary Multi-Criterion Optimization (EMO-2011), pages 269-284. Berlin, Germany: Springer-Verlag, 2011.

Ø  A. Sinha, P. Malo, P. Xu and K. Deb. A Bilevel Optimization Approach to Automated Parameter Tuning. In 2014 Genetic and Evolutionary Computation Conference (GECCO 2014).

Ø  Y. Wang, Y. C. Jiao, and H. Li. An evolutionary algorithm for solving nonlinear bilevel programming based on a new constraint-handling scheme. IEEE Transaction on Systems, Man and Cybernetics Part C: Applications and Reviews, pages 221–232, 32(2), 2005.

Ø  Yuping Wang, Hong Li, and Chuangyin Dang. A new evolutionary algorithm for a class of nonlinear bilevel programming problems and its global convergence. INFORMS Journal on Computing, 23(4):618–629, 2011.

Ø  G. Zhang, J. Liu, and T. Dillon. Decntralized multi-objective bilevel decision making with fuzzy demands. Knowledge-Based Systems, pages 495–507, 20, 2007.


Contact for further information: Ø  Ankur Sinha: [email protected] Ø  Pekka Malo: [email protected] Ø  Kalyanmoy Deb: [email protected]

Join the Club!

EBO Website: http://bilevel.org Ø  Matlab Codes Ø  Recent papers Ø  Introductory materials on EBO

Evolutionary Bilevel Optimizationppsn2014.ijs.si/files/slides/ppsn2014-tutorial7-sinha.pdf14th...

Documents

Transcript of Evolutionary Bilevel Optimizationppsn2014.ijs.si/files/slides/ppsn2014-tutorial7-sinha.pdf14th...