Lecture.6. Table of Contents Lp –rounding Dual Fitting LP-Duality.

Lecture.6

Table of ContentsLp –rounding

Dual Fitting

LP-Duality

Linear Programming ProblemA linear programming (LP) problem is an optimization

problem in which we minimize or maximize a linear objective function subject to a given set of linear constraints.

Example:Minimize 3x1 − 5x2 + 3x3 + 2x4 subject to:

3x1 + 4x2 = 6−x3 + 2x1 − x2 ≥ 22x5 ≤ 3.5x3 + .5x4 = .8xi ≥ 0 for all i

SolutionsFeasible Solution

A feasible solution to a linear program is a solution that satisfies all constraints.

Optimal SolutionAn optimal solution to a linear program is a

feasible solution with the largest(smallest) objective function value for a maximization(minimization) problem.

Many optimization problems involve selecting a subset of a given set of elements.Examples:A vertex cover is a subset of vertices.A spanning tree is really a subset of edges.A knapsack solution is a subset of items.

Can be formulated as LPs with integrality constraints.

Integer ProgramAn Integer Program (IP) is an LP with Integrality

Constraints

Integrality Constraints: Some or all the variables are constrained to be integers.

Solving Linear/Integer Programming Problems

LPs can be solved efficiently (polynomially but slowly).IPs generally cannot be solved efficiently (it is NP hard). Some

specific IPs can be solved efficiently. Actually, their LP optimal is guaranteed to be integral.

Using Indicator Variables

Many selection problems can be formulated as IPs using indicator variables (or 0-1 variables).An indicator variable is defined for each element . A value of 1 indicating the selection of the element and a value of 0 indicating otherwise.

vertex cover Set Cover Knapsack

Few Examples are :

Example: Unweighted Vertex CoverVariables: {xv | v V }.∈

The IP:Minimize ∑ xv

s.t.xu + xv ≥ 1 (u, v) E,∀ ∈xv {0, 1} v V.∈ ∀ ∈

Example: KnapsackLet the item names be {1, . . . , n}.Variables: {xi | 1 ≤ i ≤ n}.The IP:Max ∑i cixi

s.t.∑i sixi ≤ K,

xi {0, 1} 1 ≤ i ≤ n.∈ ∀

Solving Linear/Integer Programming Problems

LPs can be solved efficiently (polynomially but slowly).IPs generally cannot be solved efficiently (it is NP hard).

Some specific IPs can be solved efficiently. Actually, their LP optimal is guaranteed to be integral.

LP Relaxation (Drop the integrality constraint)Example: Unweighted Vertex CoverThe IP:

Minimize ∑v xv

s.t.xu + xv ≥ 1 (u, v) E,∀ ∈xv {0, 1} v V∈ ∀ ∈ .

The LP relaxation:Minimize ∑v xv

s.t.xu + xv ≥ 1 (u, v) E,∀ ∈xv >= 0 v V∀ ∈ .

Example: Weighted Vertex Cover

Variables: {xv | v V }.∈The IP:

Min ∑Cv xv

where Cv : cost associated with vertex

xv : indicator variable

s.t: xu + xv ≥ 1 (u, v) E∀ ∈ xv {0, 1} ∈ v V∀ ∈

LP Relaxation (Drop the integrality constraint)Example: Weighted Vertex CoverThe IP:

Min ∑Cv xv

s.t: xu + xv ≥ 1 (u, v) E ∀ ∈ xv {0, 1} ∈ v V∀ ∈

The LP relaxation: Min ∑Cv xv

s.t: xu + xv ≥ 1 (u, v) E∀ ∈

xv ≥ 0 v V∀ ∈

LP roundingIf xv ≥ ½, round it up to 1

Else round it down to 0.

Here xv is the solution obtained from LP E.g:

LP: ¼ c1 + ½ c2 + ¾ c3 + 4∕5 c4

IP : c2 + c3 + c4

Claim 1: Solution Obtained is feasibleLet (u,v) ∈ E Since the solution of LP is feasible, values of xv , v ∈ V, satisfy

xu + xv ≥ 1 (1)

⇒ atleast one of xu and xv ≥ ½

Assume x’u and x’v be the solutions obtained after rounding, then at least one of them must be 1, i.e.

x’u + x’v ≥ 1

So the solution, obtained after rounding, is feasible.

Claim 2: C(S) ≤ 2LOPT

According to the strategy some of the variables have been increased to a maximum of double & some have been reduced to 0, i.e Cv’ <= 2Cv.

So,C(S): cost of solution obtained by IP

C(S) ≤ ∑v’ Cv’ xv’

≤ 2 ∑v Cv xv ( x’v ≤ 2* Xv )

= 2 LPOPT

Hence claim 2 follows

Set Cover ProblemA finite set (universe) U of n elements, U= {e1, e2,

…, en}, a collection of subsets of U i.e. S1, S2, …., Sk

with some cost, select a minimum cost collection of these sets that covers all elements of U.

IP:IP:• Indicator variable xs, xs ∈ {0,1}

xs =0 if set S is not picked

xs =1 if set S is picked

Min ∑s Cs xs

s.t.∑s:e belongs to S xs ≥ 1 ∀ e ∈ U xs = {0,1}

LP Relaxation:LP Relaxation:

Min ∑s Cs xs

s.t.∑s:e belongs to S xs ≥ 1 ∀ e ∈ U xs > 0

LP rounding for SCLet f denote the maximum frequency of any element in U Si

Find an optimal solution to LP-Relaxation

xs >1/f round it to 1

xs <1/f discard the set, i.e. round it down to 0.

ClaimsClaim 1: solution is feasibleClaim 2: It gives factor f approximation

Claim 1: Solution is feasibleLet,

ei ∈ U , 1≤i≤n S be the collection of subsets of U em : 1<m <n belongs to l subsets of S where 1<l<k

Since the solution of LP is feasible i.e. values of xs s ∈ S obtained satisfies xs1

+ xs2 + xs3

+ ….+ xsl >1 (1)

⇒ atleast one of xs1, xs2, xs3,…., xl >1/f ⇒ x’s1 + x’s2 + x’s3 +….+ x’l> 1

Where x’si is the solution obtained after rounding. Thus it is feasible.

Claim 2: Factor f approximationFor each set s ∈ Collection of picked sets(S), xs has been increased by a

factor of atmost f.

Let C(s): Cost of our solution

Therefore,

C(S) ≤ ∑s Cs x’s s ∀ ∈ S≤ f ∑s Cs xs ( x’s ≤ f* xs)= f LPOPT

Hence it is a factor ’f’ approximation.Note: f factor could be large. Later we’ll see a

technique of rounding that gives O(log n) factor.

Linear Programming - ExampleMinimize 8x1 + 5x2 + 5x3 + 2x4 subject to:

3x1 + 4x2 ≥ 63x2 + x3 + x4 ≥ 5

xi ≥ 0 for all i x = (2, 1,0, 3) is a feasible solution. 8*2 + 5*1 + 2*3 = 27 is an upper bound.

What is the Lower Bound?Minimize 8x1 + 5x2 + 5x3 + 2x4 subject to:

3x1 + 4x2 ≥ 63x2 + x3 + x4 ≥ 5

xi ≥ 0 for all i

LB: 8x1 + 5x2 + 5x3 + 2x4 ≥ 3x1 + 4x2 ≥ 6Better LB: 8x1 + 5x2 + 5x3 + 2x4 ≥ (3x1 + 4x2 ) + (3x2 + x3 + x4) ≥ 6+5 = 11

How to compute a good LBMinimize 8x1 + 5x2 + 5x3 + 2x4 subject to:

3x1 + 4x2 ≥ 6 ……………….y1

3x2 + x3 + x4 ≥ 5……………y2

xi ≥ 0 for all iAssign a non-negative coefficient yi to every inequality such that8x1 + 5x2 + 5x3 + 2x4 ≥ y1 (3x1 + 4x2 ) + y2(3x2 + x3 + x4 )Then, LHS ≥ 6y1 + 5y2.We are interested in finding yi’s such that RHS is maximum. This leads to our dual problem.

The corresponding dual for the given example will be:

max 6y1 + 5y2

such that,3y1 < 84y1 + 3y2 < 5y1 < 5y2 < 2

and, yi > 0 for all i

Thanks to Divya Narang(8), Gautam Pahuja(10), Harshi Verma(11), Monika Bisla(14)

Weak Duality TheoremTheorem: If x and y are feasible then,

>

Proof: > = >


n

1j=

jj xc n

1=i

ii y b

n

1j=

jj xc j

n

j

x)y a(m

1=i

iji,

1

i

m

i

y) xa(n

1j=

jji,

1

m

1=i

ii y b

Set Cover

xs is 1 iff set S in included in the cover.The Primal : Objective : min ∑ Cs xs

s.t > 1 U

xs = {0,1}

LP relaxation: xs > 0


eSeS

sx:

Introduce an indicator variable ye for each of the constraints in primal.The Dual : objective: max s.t < CSi

for i = 1 to k


iSee

ey:

iSee

ey:

Example S = { x, y, z, w}S1 = { x, y}S2 = { y, z}S3 = { x, w, y}Let xs1 , xs2 , xs3

be an indicator variable for S1 ,

S2 , S3 respectively.Let Cs1 , Cs2 , Cs3

is the cost of S1 , S2 , S3 respectively.


PrimalMin : Cs1

xs1 + Cs2 x2 + Cs3 x3

Subject toxs1

+ xs3 > 1 (yx)

xs1 + xs2

+ xs3 > 1 (yy)

xs2 > 1 (yz)

xs3 > 1 (yw)

xs1, xs2, xs3 > 0


DualMax: yx + yy + yz + yw Subject to

yx + yy < Cs1

yy + yz < Cs2

yx + yy + yw < Cs3

yx , yy , yz , yw > 0


From set cover via lp

Complementary Slackness Conditions

Relaxed Complementary Slackness Conditions

Example: Weighted Vertex CoverPrimal:Min ∑Cv xv

s.t: xu + xv ≥ 1 (u, v) E∀ ∈ xv {0, 1} ∈ v V∀ ∈

Dual:Max ∑ye

s.t: ∑e:e is incident on v ye < Cv v V∀ ∈

ye {0, 1} ∈ e E∀ ∈

Primal Dual Schema 1

U = empty, y = 0For each edge e = (u, v)

ye = min {c(u) − ∑e′:u∈e′ ye′ , c(v) − ∑e′:v∈e′ ye′ }U = U union argmin {c(u) − ∑e′:u∈e′ ye′ , c(v) − ∑e

′:v∈e′ ye′ }Output U

5

4

3

2 31

3

2

7

Thanks to Neha& Neha Katyal

5

4 (1)

3 (0)

Ye =3

For every edge pick minimum of two verticesMin{4,3} = 3 Set ye=3 U has vertex having red color

12 3

3

2

7


5(4)

4 (1) (0)

3(0)

Ye =1

Ye =3

Min{1,5} = 1 Set ye=1

12 3

3

2

7


5(4)

4 (1) (0)

3(0)

Ye =1

Ye =3

Min{1,0} = 0 Set ye=0Min{2,0} = 0 Set ye=0 Min{3,0} = 0 Set ye=0

Ye

=0Ye

=0Ye

=012 3

3

2

7


5(4)(0)

Ye =4

7(3) 4 (1) (0)

3(0)

Ye =1

Ye =3

Ye

=0Ye

=0Ye

=0

3

2


Ye =0

3

4 (1) (0)

3(0)

Ye =1

Ye =3

Ye

=0Ye

=0Ye

=0

5(4)(0)

Ye =4

7(3)

2


2 (0)

3 (1)

Ye =2

4 (1) (0)

3(0)

Ye =1

Ye =3

Ye

=0Ye

=0Ye

=0

5(4)(0)

Ye =4

7(3)

Ye =0

Red-colored nodes form a vertex-cover


Solution is feasibleTrivial, since the algorithm runs for every

edge.

Let e= (u,v) be an edge.Suppose if possible, none of the xu and xv has

been set to 1 i.e constraints corresponding to u and v have not yet gone tight and we have a ye that can be raised. That means the algorithm has not yet completed.

Solution is 2 factorFor every xv > 0, dual constraint is tight

(trivially).

For every edge e = (u,v), 1 < xu + xv < 2

Hence, by relaxed CSC, cost of the solution is at most twice the OPT.

Primal-Dual Schema 2 (Ignore)Raise the dual variables uniformly until one

or more of the constraints become tight. Freeze the dual variables contributing to these constraints. Set the corresponding primal variable to 1.

If more than one constraint becomes tight, take them one by one in an arbitrary order.

5

4

3

2 31

3

2

7


5

4

3 (1)

Ye =3/4

12 3

3

2

7

Ye =3/4Ye =3/4

Ye =3/4

Ye =3/4Ye =3/4

Ye =3/4Ye =3/4

5

4

3 (1)

Ye =3/4

12 3

3

2

7

Ye =3/4Ye =3/4

Ye =3/4

Ye =3/2

Ye =3/2Ye =3/2

Ye =3/2

5

4

3 (1)

Ye =3/4

12 3

3

2

7


Ye =3/4Ye =3/4

Ye =3/4

Ye =3/2

Ye =7/4Ye =7/4

Ye =3/2

Solution is feasibleLet e= (u,v) be an edge.Suppose if possible, none of the xu and xv has

been set to 1 i.e consraints corresponding to u and v have not yet gone tight and we have a ye that can be raised. That means the algorithm has not yet completed.

Solution is 2 factorFor every xv > 0, dual constraint is tight

(trivially).

For every edge e = (u,v), 1 < xu + xv < 2

Hence, by relaxed CSC, cost of the solution is at most twice the OPT.

Lecture.6. Table of Contents Lp –rounding Dual Fitting LP-Duality.

Documents

Transcript of Lecture.6. Table of Contents Lp –rounding Dual Fitting LP-Duality.