Efficient Solutions for 2-Variables-per-Constraint Integer Programming Problems

Efficient Solution for 2VIP Problems

1

Efficient Solutions for Efficient Solutions for 2-Variables-per-Constraint 2-Variables-per-Constraint Integer Programming ProblemsInteger Programming Problems

Reuven Bar-Yehuda &

Dror Rawitz

1999


2

Heirarchy of Relevant Problem Heirarchy of Relevant Problem TypesTypes

2-Satisfiability (2SAT)

Minimum Weight 2SAT

2-Variable-per-constraint Integer Programming

(2VIP)

Vertex Cover (VC)


3

Vertex CoverVertex Cover

Input: An undirected graph G(V,E)

Solution: A subset S V which includes a vertex in

each arc in E

“What about optimality?” Just wait…


4

2-Satisfiability2-Satisfiability

Input: A boolean formula in 2-CNF form,

with mm constraints and nn variables

)()()()()( 6554413121 xxxxxxxxxx

Solution: A vector of nn values for the nn variables, so that they satisfy all constraints

(FALSE, TRUE, FALSE, TRUE, FALSE, FALSE)


5

Vertex Cover as a Type of 2SATVertex Cover as a Type of 2SAT

Vertices Boolean Variables

)( ji xx

xi = TRUE

Selected Vertices Variables Assigned TRUE

Edges Simple OR constraints

i

i j

i

j

k xi, xj, xk


6

Feasibility Algorithm for 2SATFeasibility Algorithm for 2SAT

SAT (the general satisfiability problem) is NP-Complete.

3SAT (satisfiability of 3-CNF formulae) is NP-Complete.

…

But 2SAT is in P. This difference is the basis for our entire approach to the

approximation of 2VIP


7

Feasibility Algorithm for 2SAT Feasibility Algorithm for 2SAT (continued)(continued)

If all the variables are set, return SUCCESS

Select an unset variable xk

F1 F with the implications of xk=FALSE propagated

F2 F with the implications of xk=TRUE propagated

If both F1 & F2 cannot be satistfied return FAILURE

F3 the first of F1,F2 for which the propagation is completed without its being determined as a FALSE formula

return 2SAT-FEASIBILITY(F3)

2SAT-FEASIBILITY(F)

… 99% Pure Greed!


8

Feasibility Algorithm for 2SAT Feasibility Algorithm for 2SAT (correctness)(correctness)

Why does this work?

Observe an assignment effect propagation:

x1 = FALSE

)()()()()( 6554413121 xxxxxxxxxx

)()()()()( 6554432 xxxxxxx TTF

)()()()()( 655443 xxxxxx TTTF

x2 = TRUE


9


After the propagation is complete, we have:

)()()()()( 655443 xxxxxx TTTF

one side of OR set FALSE,

other side set TRUE

one side of OR set TRUE, other side unchanged

no changes to either side

satisfied constraints unaffected constraints


10


)()()()()( 655443 xxxxxx TTTF

satisfied constraints Unaffected Constraints

So… combine the values set here

with the values for this part

‘borrowed’ from an x1=TRUE feasible solution to create an x1=FALSE feasible solution.

But… every x1=TRUE feasible solution must also satisfy these

constraints!


11

Feasibility Algorithm for 2SAT Feasibility Algorithm for 2SAT (complexity)(complexity)

O(n) recursive calls O(m) constraits checked in the propagation process = O(mn) time

Each variable is set at most once Each constraint is checked at most twice Running the propagations in parallel binds the amount of ‘wasted’ time to the amount of time used to propagate adopted assignments.

Time Complexity - seemingly:

But, actually:

Hence O(2·(m+n))= O(m+n) time in total.


12

2SAT as an extension of Vertex Cover2SAT as an extension of Vertex Cover

)( ji xx

In Vertex Cover, we only have constraints of a single type:

Edges Simple OR constraints

i

)( ji xx ??? OR constraints with 1 NOT

i j?? )( ji xx

But there are 2 more types of constraints in 2SAT:

??? OR constraints with 2 NOTs

i j?? )( ji xx

j


13

Extending 2SAT: Minimum WeightExtending 2SAT: Minimum Weight

(MIN-2SAT) Given a weight vector w, find a vector x

which minimizes , subject to:

generalization)( ji xx k i k j ka x b x c 0, ba

)( ji xx k i k j ka x b x c 0,0 ba

)( ji xx k i k j ka x b x c 0, ba

generalization

generalization

k i k j ka x b x c 1,0ix ni ,........,1

mk ,........,1

2SAT:

2SAT:

2SAT:

n

iii xw

1


14

Extending MIN-2SAT: Any Integer ValueExtending MIN-2SAT: Any Integer Value

(2VIP) Given a weight vector w, find a vector x which

minimizes , subject to:

k i k j ka x b x c

iii uxl Nxi

ni ,........,1 ni ,........,1

mk ,........,1

n

iii xw

1

9876543210

9876543210


15

Feasible Polyhedron ConvexityFeasible Polyhedron Convexity

The polyhedron bounding the feasible solutions of an IP problem is convex.

Specifically, if the points (a,b),(c,d),(e,f),(g,h),(i,j) satisfy a constraint involving variables i and j …

…so do all the points on the line and within the triangle.

(a,b)

(c,d)

(e,f)

(g,h)

(i,j)


16

Extending the 2SAT Feasibility Algorithm: Extending the 2SAT Feasibility Algorithm: Partitioning the Solution SpacePartitioning the Solution Space

2SAT:unset variable xi

xi = TRUExi = FALSE

2VIP: iii ulx ,

2

, iiii

ullx

iii

i uul

x ,12

ii ul ,


17

Extending the 2SAT Feasibility Algorithm: Extending the 2SAT Feasibility Algorithm: Propagating Bound EffectsPropagating Bound Effects

(This illustration courtesy of Dror Rawitz )


18

Result: Feasibility Algorithm for 2VIPResult: Feasibility Algorithm for 2VIP

If l = u then

If l is a feasible solution, return SOLUBALE

The solution is infeasible; return INSOLUBLE

Select a variable xk for which lkuk

P1 P with the implications of propagated

P2 P with the implications of propagated

If both P1 & P2 are directly unsatisfiable return INSOLUBLE

P3 the first of P1,P2 for which the propagation is completed without its being determined as directly unsatisfiable

return 2VIP-FEASIBILITY(P3)

2VIP-FEASIBILITY(P)

2

, kkkk

ullx

kkk

k uul

x 1,2


19

Feasibility Algorithm for 2VIP Feasibility Algorithm for 2VIP (correctness)(correctness)

Q: Can choosing one ‘half’ of the solution space over the other ‘half’ rule out feasible solutions?

A: Of course.

Q: Can choosing one ‘half’ of the solution space over the other ‘half’ rule out ALL feasible solutions?

A: No! We can still ‘borrow’ parts of any feasible solution – y – for the discarded ‘half’, creating a feasible solution – y’ – for the selected ‘half’:

kkk

kkkk

kkk

k

yuu

uyly

lyl

ynk ', …


20


bounds for y – solution for

y’ – solution for

kkk

k uul

x 1,2

2

, kkkk

ullx

9876543210

k

kkk

k uul

x 1,2


21


Why is y’ a valid solution?

Examine any one of the m constraint (say, constraint k), involving variables i and j. One of the following holds:

We will see how in all 3 cases, (y’i,y’j) satisfy the constraint.

1. Both y’ variables are equal to the y variables.

2. One of the y’ variables differs from its y counterpart.

3. Both y’ variables differ from their y counterparts.


22


Case 1:

no changes to the y variables; since the y vector is a feasible solution, constraint k is satisfied by (y’i,y’j) .

' 'i i j jy y y y

lower bound for y’j

low

er b

ou

nd

fo

r y’

i

(yi ,yj) = (y’I ,y’j)


23


Case 2:

since yi is within the bounds for variable i, there must exist a value

vi such that (yi,vj) satisfy the constraint; we can now use the

convexity to draw the line between (y’i,y’j) and (yi,vj) within the

feasible polygon… the line crosses (yi,lj)=(y’i,y’j) .

' '

' '

( )

( )

i i j j

i i j j

y y y y

y y y y


low

er b

ou

nd

fo

r y’

i

(yi ,yj)

(yi ,vj)

we are demonstrating correctness for

' 'i i j jy y y y

(yi ,lj)


24


Case 3:

we again use the definition of the lower and uppoer bounds: there exist vi and vj so that (li,vj) and (vi,lj) satisfy the constraint.

Together with (yi,yj) , these points form a feasible triangle within

which we find (li, lj)= (y’i, y’j) .

' '( )i i j jy y y y


low

er b

ou

nd

fo

r y’

i

(yI ,yj)

(li ,vj)

we are demonstrating correctness for

' 'i i j jy y y y (vi ,lj)(li ,lj)


25

Feasibility Algorithm for 2VIP Feasibility Algorithm for 2VIP (complexity)(complexity)

Each lower/upper bound is set at most U times for every lower/upped bound update, at most m constraints are checked. Running the constraint propagations in parallel binds the amount of ‘wasted’ time to the amount of time used to propagate adopted lower/upper bound changes.

(Remembering 2SAT…)

Hence O(mU) time in total.

iii

luU max


26

Extending 2VIP Yet Again…Extending 2VIP Yet Again…

Q: How can we ensure we aren’t discarding all ‘good’ solutions by searching only one part of the solution space?

A: We must extend the problem again, slightly, so that we can ‘cut our losses’ when betting on one part of the solution space.

(E2VIP) Instead of minimizing , we minimize the

expression

for some , satisfying .

1

n

i iiw x

ˆ ˆ, nl u Q …

1 1

ˆ0 ,

ˆ ˆ ˆ ˆˆ ˆ ˆ( , , ) ( , , ) ,

ˆˆ ˆ,

i i

n n

i i i i i i i i i ii i

i i i i

x l

W x l u w x l u w x l l x u

u l u x

ˆ ˆl l u u


27

The W function is, in essence, the weight of the part of a vector x which intersects with the (non-integral) range

(i.e. the weight of the part x of between and ).

Decomposition of ‘Weights’Decomposition of ‘Weights’

ˆ ˆ( , , )

ˆ ˆ ˆ ˆ( , , ) ( , , )

W x l u

W x l m W x m u

ˆ ˆ( , )l u

Now we can use a partitioning of the solution space with some vector to achieve a sort-of-a-decomposition of weights, without changing the actual weight vector w: m

m

l u

l

u

m


28

Applying the Local Ratio PrincipleApplying the Local Ratio Principle

We can now apply the Local Ratio theorem:

If, for given we can find an so that every feasible solution, in both parts of the solution space (i.e. in the

solution space with the effects of either or

propagated) is an r-approximation w.r.t.

then every feasible solution that is an r-approximation

w.r.t. is also an r-approximation w.r.t. .

ˆ ˆ,l u m

ˆ ˆ( , )l mˆ ˆ( , )m u ˆ ˆ( , )l u

We are left with the task of finding a useful decomposition of , i.e. a useful vector.

,2

k kk k

l ux l

1,

2k k

k k

l ux u

mˆ ˆ( , )l u


29

Choosing a Useful ‘Weight’ DecompositionChoosing a Useful ‘Weight’ Decomposition

What effects the Choice of ?

We have partitioned the solution space, and will only be searching (for example) in the bounds effected by setting

(let’s call these bounds ).,2

k kk k

l ux l

,left leftl u

Intuitive choice: ˆ leftm l

any feasible solution in

would be a

1-approximation with

respect to , but…

,left leftl u

ˆ ˆ,l m

m


30


Problem: although we are sure there are feasible solutions in – the

only thing we know about their optimality is that the cost of in is

lower than that of (that’s why we would choose over

).

Conclusion: the selection of must take into account the possibility that the

optimal solution is in the discarded part of the solution space: .

We must make every solution in ‘pay’ enough in

so that it would still be worthwhile to only search for

solutions in .

( , )left leftl u

leftl rightl

m

( , )right rightl u( , )right rightl u

ˆ ˆ( , )l u

ˆ ˆ( , )l m( , )left leftl u

( , )left leftl u ( , )right rightl u


31


Solution:

Start with . Now increase the values of in

variables for which … until ‘pays’ in

as much as ‘pays’ in :

ˆ leftm l m

left rightk kl l rightl

ˆ ˆ( , )l m ˆ ˆ( , )l u

rightlleftl

ˆ leftm l ˆ leftm l

leftl


32


Loss:

We have added as much as to , which means the minimum feasible solution in is 2 times more expensive then (which is cheaper-or-equal to any feasible solution in ).

What have we achieved?

Gain:

Every solution in

now ‘pays’ in the range at least , so it can’t be more than 2 times cheaper, w.r.t , than the feasible solution we shall find in .

Thus any feasible solution in is a 2-approximation of the optimum solution

w.r.t the range .

ˆ ˆ( , , )leftW l l u m

ˆ ˆ( , )m uleftl

( , )left leftl u

ˆ ˆ( , , )leftW l l u

ˆ ˆ( , )l m

( , )right rightl u

ˆ ˆ( , )l mˆ ˆ( , )m u

ˆ ˆ( , )m uˆ ˆ( , )l m

Remember Dr. Bar-Yehuda’s ‘pay 2 dollars for 1’ rule-of-thumb!


33

If then l ; If then l

If l = u then:

if l is a feasible solution, return l

if l isn’t a feasible solution, return FAILURE

Select a variable xk for which lkuk

lleft,uleft l,u with the implications of propagated

lright,uright l,u with the implications of propagated

If both lleft,uleft and lright,uright are directly unsatisfiable return FAILURE

…

The Approximation Algorithm for 2VIPThe Approximation Algorithm for 2VIP

E2VIP-APPROX

2

, kkkk

ullx

kkk

k uul

x 1,2

ˆ ˆ( , , , )l u l u

l l l ˆu u u


34

The Approximation Algorithm for 2VIPThe Approximation Algorithm for 2VIP

…

If lleft,uleft is directly unsatisfible, return E2VIP-APPROX

If lright,uright is directly unsatisfible, return E2VIP-APPROX

If then

better left , worse right

else better right , worse left

Find so that and

If < lbest then lbest

Return E2VIP-APPROX

Note that calling E2VIP-APPROX(l,u,l,u) means

approximating our original 2VIP problem.

ˆ ˆ( , , , )left leftl u l uˆ ˆˆ ˆ( , , ) ( , , )left rightW l l u W l l u

ˆ ˆˆ max ,worsel m l l m ˆ ˆˆ ˆ( , , ) ( , , )worse bestW l l m W l l um

ˆ ˆ( , , , )best bestl u m u

m

ˆ ˆ( , , , )right rightl u l u


35

Approximation Algorithm for 2VIP Approximation Algorithm for 2VIP (correctness)(correctness)

At every level of recursion, we ‘cut out’ a piece of the solution space, and with it – a part of the cost of the solution.

Since we ensure that any feasible solution in the reduced range (i.e. ) is a 2-approximation in the cut-out range (i.e. ), we end up ‘cutting out’ the entire solution space, so that we are left with a single, trivial, feasible solution for lfinal=ufinal , and to it we add back the entire

solution space in the form of the ‘cut-outs’, while maintaining the 2-effectiveness.

Thus we end up with a solution which is a 2-approximation for the original problem.

ˆ ˆ( , )m uˆ ˆ( , )l m


36

Approximation Algorithm for 2VIP Approximation Algorithm for 2VIP (complexity)(complexity)

Unlike 2VIP-Feasibility, we can waste a lot of time propagating bound change effects in the discarded part of the solution space. Thus every single execution of the algorithm (not including the recursive call) takes O(mU) time. If we choose the xk’s in successive order (from x1 to xn

repeatedly), we will have a series of ‘runs’ over all n variables, with each i’th run cutting the range (lk,uk) by half

and taking O(nmU/2i) time. The number of runs is bounded by log(U).

log

0 2

U

ii

nmUHence = O(nmU) time in total.


37

GeneralizationsGeneralizations

We did not make use of the linearity of the constraints; we only used the covexity of the feasible polyhedron. We can therefore generalize the problem to any ‘axis-convex’ constraints.

We did not make use of the linearity of the cost function. We can generalize the problem to (more or less) any non-negative monotone weight function.


38

The End.The End.

Efficient Solutions for 2-Variables-per-Constraint Integer Programming Problems

Documents

Transcript of Efficient Solutions for 2-Variables-per-Constraint Integer Programming Problems