Dantzig-Wolfe Decomposition · Thomas Stidsen 3 DTU-Management / Operations Research Dantzig-Wolfe...

1Thomas Stidsen

DTU-Management / Operations Research

Dantzig-Wolfe Decomposition– Changing a problem ...

Thomas Stidsen

[email protected]

DTU-Management

Technical University of Denmark

2Thomas Stidsen


OutlineGeneral Background for Dantzig-Wolfe

Mathematical background

A 2-dim example

Multi Commodity Flow problem (the lastexercise)

Block-Angular Structure

3Thomas Stidsen


Dantzig-WolfeHistorically:

Dantzig-Wolfe decomposition was invented byDantzig and Wolfe 1961.

The method is so closely connected to columngeneration that they in some aspects may beconsidered to be identical.

Dantzig-Wolfe and Column-Generation is one ofthe most used methods for practical problems.

Notice that column generation and Dantzig-Wolfe

are used interchangeably ...

4Thomas Stidsen


From Appendix A we haveGiven the convex set X = {x|Ax ≤ b} can berepresented by the extreme points and extremerays of the convex set (Minkowski-Weyl’s Theorem):

X = {x|x =∑

p λp · xp +∑

r δr · xr}

where∑

p λp = 1, λp ≥ 0, δr ≥ 0 and where

p ∈ {1, . . . , P}, r ∈ {1, . . . , R} See Appendix A, p.

638

5Thomas Stidsen


Extreme raysGood news: In theory we need extreme rays torepresent arbitrary polyhedrons, but in reality weonly encounter this in Dantzig-Wolfe decompositionvery seldom. I have never seen a Dantzig-Wolfedecomposition where an extreme ray wasnecessary. Hence we will from here on simplyassume that we can just use extreme points.

6Thomas Stidsen


The importance of a good polytope

4

2

1 2 3 4

1

3

If we can achieve the smallest polytope in the figure,we can solve MIP problems with LP solvers !!!

7Thomas Stidsen


Given a Linear programMin:

cTx

s.t.:

A1x ≥ b1

A2x ≥ b2

x ≥ 0

8Thomas Stidsen


Changing representationDantzig-Wolfe decomposition is actually aboutchanging representation from a set of constraints toa set of extreme points. The good question is now,how many constraints should be replaced byextreme points ?

All: You tried that, in the first exercise, it doesnot really help ...

None: Well that does not change the problem !!!

Some: Yes, but why should that help ?

9Thomas Stidsen


Visualized

A1

A1’

1 2 3 4

1

3

4

2

A2

10Thomas Stidsen


The mathematical formulationMax:

x + 2y

s.t.:−x + y ≤

5

2(A1)

x + y ≤9

2(A1)

−x − y ≤ −1

2(A1)

−x − y ≤ −3

2(A1)

−x + y ≤ 1 (A2)

2x + y ≤ 13 (A2)

−x − 3y ≤ −7 (A2)

11Thomas Stidsen


Lets look at A1

A1

A1’

1 2 3 4

1

3

4

2

12Thomas Stidsen


Change representationNow we represent part, the A2 part, of theconstraints with the convex combination of extremepoints and extreme arrays:X = {x|x =

∑

p λp · xp}

where the extreme points p define the setX = {x|A2x ≥ b2}, BUT WHICH EXTREMEPOINTS ????

13Thomas Stidsen


Improving the bound of A1: A1’

−x + y ≤5

2(A1)

x + y ≤9

2(A1)

−x − y ≤ −1

2(A1)

−x − y ≤ −3

2(A1)

x, y ∈ R+(A1) OR x, y ∈ Z+(A1′)

14Thomas Stidsen


The LP improvement !

Objective

LP gap improvement

1 2 3 4

1

3

4

2

A2

A1

A1’

15Thomas Stidsen


InsertionMin:

cT (∑

p

λp · xp +∑

r

δr · xr)

s.t.:

A2(∑

p

λp · xp +∑

r

δr · xr) ≥ b2

∑

p

λp = 1

λp, δr ≥ 0

Notice: xp and xr are now constants and λp and δr

are now the variables

16Thomas Stidsen


Insertion without extreme raysMin:

cT (∑

p

λp · xp)

s.t.:

A2(∑

p

λp · xp) ≥ b2

∑

p

λp = 1

λp ≥ 0

17Thomas Stidsen


Insertion without extreme rays, matrix versionAssume that:

The A2 matrix is a m2 by n matrix, i.e. m2 rowsand n x variables.

We replace the values of the x variables withthe column vector of variables λp.

We have a new matrix X with n rows and pcolumns

18Thomas Stidsen


The transformed problemMin:

cTλp

s.t.:

A2X λp ≥ b2

1 λp = 1

λp ≥ 0

19Thomas Stidsen


WhereGenerally: x = Xλp.

The matrix AX = A2X is a matrix with m2 rowsand p columns (corresponding to the newvariables.

Each matrix element in the new matrix can befound as axm2,p = A1(m1)X(p)

A(m2): The m2’th row of the A2 matrix

X(p): The p’th column of the X matrix

This means that we can calculate for each addedcolumn the new resulting column in the masterproblem

20Thomas Stidsen


How can we find the extreme points of A1 ?We need two things:

Satisfy the constraints of A1 or A′1 (otherwise it

will not be an extreme points)

Calculate the reduced costs of the extremepoint in the A1 or A′

1 constraints, based on theoriginal costs and the dual variables from theA2 part.

21Thomas Stidsen


The sub-problemMin:

(c − π · A2)x − α

s.t.:

A1 · x ≥ b2

x, y ∈ R+(A1) OR x, y ∈ Z+(A′1)

22Thomas Stidsen


The sub-problemStopping criteria, if we cannot generate an extremepoint:(c − π · A2)x − α < 0Stopping criteria, if we cannot generate an extremeray:(c − π · A2)x < 0If we cannot generate either an extreme point or anextreme ray then we have the optimal solution.Of course, for maximization problems, the reducedprofits for extreme points and extreme rays isrequired to be positive.

23Thomas Stidsen


The column generation algorithmEnsure feasibility of the master problemrepeat

SOLVE min{cTx|A2X λp ≥ b, 1 λp = 1, λp ≥ 0}get πSOLVE

min{cred = (c − π · A1)x − α|A1 · x ≥ b1,x, y ∈ R+(A1)ORx, y ∈ Z+(A′

1)}calculate new column data:

cp = cTx

axm2,p = A2(m2)X(p)

until cred ≥ 0

24Thomas Stidsen


Guideline when decomposingThe guideline when performing Dantzig-Wolfedecomposition:

Find a problem which has Non-Integral property

Find a way to solve the sub-problem fast.

Examples of the combinatorial sub-problems: Theknapsack problem and the constrained shortestpath problem.

Much more about this in the next lecture !

25Thomas Stidsen


The Block-Angular StructureWe can use the structure in the following way:

The variables are linked together by the A1

matrix.

If we seperate the matrixes and are able tosolve the problems seperatly, we can solve theproblem by solving the subproblem for thematrix’es A′

1 and A′′2

26Thomas Stidsen


The Block-Angular StructureSuppose the optimisation problem has the followingstructure Given two convex sets, in two dimensions:

A1

A2’

A2’’

c

b

27Thomas Stidsen


The Block-Angular StructureNotice that the two sub-problems share theprices π ...

Sometimes it is beneficial to simply use astandard MIP solver.

You have already experienced the division intoseveral sub-problems in the multi-commodityflow exercise

28Thomas Stidsen


Multi-Commodity flowMaximize the flow AE and BE through anetwork with limited capacity.

An example could be: A producer of naturalgas, production at A and B. Supply thecustomer, who does not care where the gascomes from.

Actually this problem can be solved by the maxflow algorithm (how ?). This formulation waschosen in order to make the problem simple.

We will only consider simple paths.

Orientation is tricky !

29Thomas Stidsen


Multi-Commodity flow: The network

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

AB

C D

E

1

4

3

1

1

3

Demand1: A −> E

Demand2: B −> E

1

30Thomas Stidsen


AE matrix

AAE=

AB 1 1 1 1AC 1 1 1BC 1 1 1BD 1 1 1CD 1 1 1CE 1 1 1DE 1 1 1 1

31Thomas Stidsen


BE matrix

ABE=

AB 1 1AC 1BC 1 1BD 1 1CD 1 1 1CE 1 1 1DE 1 1 1

32Thomas Stidsen


c and bcp = 1

b =

AB 1AC 1BC 1BD 4CD 1CE 3DE 3

33Thomas Stidsen


The link-path problemMax:

cT (xAEp + xBE

p )

s.t.:

AAExAEp + ABExBE

p ≤ b

xAEp , xBE

p ≥ 0

How did we get here ?

34Thomas Stidsen


The arc-flow formulationMax: cT (yAE + yBE)

s.t.:

∑

j

xAE(ij) −

∑

j

xAE(ji) =

yAE i = A

−yAE i = E

0 otherwise

∀i

∑

j

xBE(ij) −

∑

j

xBE(ji) =

yBE i = B

−yBE i = E

0 otherwise

∀i

xAE(ij) + xAE

(ji) + xBE(ij) + xBE

(ji) ≤ b ∀{ij}

yAE, yBE, xAE(ij), x

BE(ij) ≥ 0

35Thomas Stidsen


CommentsThe arc-flow constraints correspond to apolytope (A1 · x = b1) which is replaced with thepath variables

Notice that each path variable corresponds to anumber of arc flow variables

We have two separate subproblems ...

We can easily calculate an arc-flow solutionbased on our paths ...

36Thomas Stidsen


Integer SolutionsAs I mentioned the last time, there are severalmethods:

Rounding up, not always possible ....

Solve the LP problem with column generation,and after the column generation algorithm hasfinished, the MIP model is solved using astandard solver.

Use standard branching in the original space

Branch and price, hard and quite timeconsuming ...

37Thomas Stidsen


Branch and Price: Branching ProblemIf we branch on the transformed variables, we havebig problems in "down branch", because thatvariable is generated again.

10S

S S

SSS SSSSS

S00

x3=0

x2=0

x1=1x1=0

111110101100011010001000

111001S

S

S

38Thomas Stidsen


Branch and Price: Branching in the original variabWe can perform standard branching, but we have todo it in another variable space ! This I will talk moreabout in the next lecture.

39Thomas Stidsen


BranchingSo called Ryan-Foster branching can be applied(for set-partitioning problems)Min:

∑

j

xj

s.t.:∑

j

ai,j · xj = 1 ∀i

xj ∈ {0, 1}

Further, notice that ai,j ∈ {0, 1}

40Thomas Stidsen


Ryan-Foster BranchingThe “end rule”: Each row will (in the optimal integersolution) be “covered” by EXACTLY one column(variable). When we have a fractional solution, thisend rule is broken: At least two rows will be coveredby two variables. So called Ryan-Foster branchingcan be applied (for set-partitioning problems)

41Thomas Stidsen


Ryan-Foster BranchingFinding the branching cuts, find two variables whichare fractional and which share at least one “cover”.

1

1

1

=

=

=

x1 x2==

0.3 0.7

1

1

1

42Thomas Stidsen


The branching rulesGiven two rows i1 and i2 in one branch requireai1,j′ = ai2,j′ for all variables, i.e. rows i1 and i2 arecovered by the same variables, and in the otherbranch: ai1,j′ 6= ai2,j′, i.e. rows i1 and i2 are coveredby different variables.

Dantzig-Wolfe Decomposition · Thomas Stidsen 3 DTU-Management / Operations Research Dantzig-Wolfe...

Documents

Transcript of Dantzig-Wolfe Decomposition · Thomas Stidsen 3 DTU-Management / Operations Research Dantzig-Wolfe...