ILP solvers

140
Prologue LP Solvers Adding Intergality Constraints Special Cases of ILP Epilogue ILP Solvers Dalya Gartzman Tel Aviv University A glance at the backstage Dalya Gartzman Tel Aviv University ILP Solvers

description

Short review of different Linear Programming and Integer Programming solvers.

Transcript of ILP solvers

Page 1: ILP solvers

Prologue LP Solvers Adding Intergality Constraints Special Cases of ILP Epilogue

ILP Solvers

Dalya Gartzman

Tel Aviv University

A glance at the backstage

Dalya Gartzman Tel Aviv University

ILP Solvers

Page 2: ILP solvers

Prologue LP Solvers Adding Intergality Constraints Special Cases of ILP Epilogue

LP vs. ILP

I Consider the following LP Standard Form

maximize ctx

subject to Ax ≤ b, x ≥ 0

where x ∈ Rn, A ∈ Mm×n(R).I If a solution exists, it can be given by many polynomial time

algorithms (in the size of the input), that we will review todayvery briefly.

I Unfortunately, once we add the integrality constraints

x ∈ Zn

this problem, like many others in combinatorial optimization,becomes NP − complete.

Dalya Gartzman Tel Aviv University

ILP Solvers

Page 3: ILP solvers

Prologue LP Solvers Adding Intergality Constraints Special Cases of ILP Epilogue

LP vs. ILP

I Consider the following LP Standard Form

maximize ctx

subject to Ax ≤ b, x ≥ 0

where x ∈ Rn, A ∈ Mm×n(R).

I If a solution exists, it can be given by many polynomial timealgorithms (in the size of the input), that we will review todayvery briefly.

I Unfortunately, once we add the integrality constraints

x ∈ Zn

this problem, like many others in combinatorial optimization,becomes NP − complete.

Dalya Gartzman Tel Aviv University

ILP Solvers

Page 4: ILP solvers

Prologue LP Solvers Adding Intergality Constraints Special Cases of ILP Epilogue

LP vs. ILP

I Consider the following LP Standard Form

maximize ctx

subject to Ax ≤ b, x ≥ 0

where x ∈ Rn, A ∈ Mm×n(R).I If a solution exists, it can be given by many polynomial time

algorithms (in the size of the input), that we will review todayvery briefly.

I Unfortunately, once we add the integrality constraints

x ∈ Zn

this problem, like many others in combinatorial optimization,becomes NP − complete.

Dalya Gartzman Tel Aviv University

ILP Solvers

Page 5: ILP solvers

Prologue LP Solvers Adding Intergality Constraints Special Cases of ILP Epilogue

LP vs. ILP

I Consider the following LP Standard Form

maximize ctx

subject to Ax ≤ b, x ≥ 0

where x ∈ Rn, A ∈ Mm×n(R).I If a solution exists, it can be given by many polynomial time

algorithms (in the size of the input), that we will review todayvery briefly.

I Unfortunately, once we add the integrality constraints

x ∈ Zn

this problem, like many others in combinatorial optimization,becomes NP − complete.

Dalya Gartzman Tel Aviv University

ILP Solvers

Page 6: ILP solvers

Prologue LP Solvers Adding Intergality Constraints Special Cases of ILP Epilogue

Why is this so hard?

LP ∈ P

I Every inequality represents a half-plane, thus their intersectionis a convex polytope, which has polynomially-many edges andvertices.

I The solution is a vertex of the polytope, hence finding it inpolynomial time is not surprising.

ILP ∈ NP

I Following the same heuristics, this time the convex polytopecreated may have exponentially many facets, edges andvertices.

I Reduction from maximum− clique (0− 1 ILP is also in NP).

Dalya Gartzman Tel Aviv University

ILP Solvers

Page 7: ILP solvers

Prologue LP Solvers Adding Intergality Constraints Special Cases of ILP Epilogue

Why is this so hard?LP ∈ P

I Every inequality represents a half-plane, thus their intersectionis a convex polytope, which has polynomially-many edges andvertices.

I The solution is a vertex of the polytope, hence finding it inpolynomial time is not surprising.

ILP ∈ NP

I Following the same heuristics, this time the convex polytopecreated may have exponentially many facets, edges andvertices.

I Reduction from maximum− clique (0− 1 ILP is also in NP).

Dalya Gartzman Tel Aviv University

ILP Solvers

Page 8: ILP solvers

Prologue LP Solvers Adding Intergality Constraints Special Cases of ILP Epilogue

Why is this so hard?LP ∈ P

I Every inequality represents a half-plane, thus their intersectionis a convex polytope, which has polynomially-many edges andvertices.

I The solution is a vertex of the polytope, hence finding it inpolynomial time is not surprising.

ILP ∈ NP

I Following the same heuristics, this time the convex polytopecreated may have exponentially many facets, edges andvertices.

I Reduction from maximum− clique (0− 1 ILP is also in NP).

Dalya Gartzman Tel Aviv University

ILP Solvers

Page 9: ILP solvers

Prologue LP Solvers Adding Intergality Constraints Special Cases of ILP Epilogue

Why is this so hard?LP ∈ P

I Every inequality represents a half-plane, thus their intersectionis a convex polytope, which has polynomially-many edges andvertices.

I The solution is a vertex of the polytope, hence finding it inpolynomial time is not surprising.

ILP ∈ NP

I Following the same heuristics, this time the convex polytopecreated may have exponentially many facets, edges andvertices.

I Reduction from maximum− clique (0− 1 ILP is also in NP).

Dalya Gartzman Tel Aviv University

ILP Solvers

Page 10: ILP solvers

Prologue LP Solvers Adding Intergality Constraints Special Cases of ILP Epilogue

Why is this so hard?LP ∈ P

I Every inequality represents a half-plane, thus their intersectionis a convex polytope, which has polynomially-many edges andvertices.

I The solution is a vertex of the polytope, hence finding it inpolynomial time is not surprising.

ILP ∈ NP

I Following the same heuristics, this time the convex polytopecreated may have exponentially many facets, edges andvertices.

I Reduction from maximum− clique (0− 1 ILP is also in NP).

Dalya Gartzman Tel Aviv University

ILP Solvers

Page 11: ILP solvers

Prologue LP Solvers Adding Intergality Constraints Special Cases of ILP Epilogue

Why is this so hard?LP ∈ P

I Every inequality represents a half-plane, thus their intersectionis a convex polytope, which has polynomially-many edges andvertices.

I The solution is a vertex of the polytope, hence finding it inpolynomial time is not surprising.

ILP ∈ NP

I Following the same heuristics, this time the convex polytopecreated may have exponentially many facets, edges andvertices.

I Reduction from maximum− clique (0− 1 ILP is also in NP).

Dalya Gartzman Tel Aviv University

ILP Solvers

Page 12: ILP solvers

Prologue LP Solvers Adding Intergality Constraints Special Cases of ILP Epilogue

Why is this so hard?LP ∈ P

I Every inequality represents a half-plane, thus their intersectionis a convex polytope, which has polynomially-many edges andvertices.

I The solution is a vertex of the polytope, hence finding it inpolynomial time is not surprising.

ILP ∈ NP

I Following the same heuristics, this time the convex polytopecreated may have exponentially many facets, edges andvertices.

I Reduction from maximum− clique (0− 1 ILP is also in NP).

Dalya Gartzman Tel Aviv University

ILP Solvers

Page 13: ILP solvers

Prologue LP Solvers Adding Intergality Constraints Special Cases of ILP Epilogue

So what can we do?

I Stick to small problems. For large problems use an estimate.

I Some algorithms first ignore the integrality constraints andsolve the LP (Relax).

Then they introduce the integrality constraints in anintelligent way to receive the optimal integer solution, withbetter performance than the naive way, yet still exponential.

I Under certain conditions, the convex polytope is ofpolynomial size.

Identify these occasions, and solve the usual way.

Dalya Gartzman Tel Aviv University

ILP Solvers

Page 14: ILP solvers

Prologue LP Solvers Adding Intergality Constraints Special Cases of ILP Epilogue

So what can we do?

I Stick to small problems. For large problems use an estimate.

I Some algorithms first ignore the integrality constraints andsolve the LP (Relax).

Then they introduce the integrality constraints in anintelligent way to receive the optimal integer solution, withbetter performance than the naive way, yet still exponential.

I Under certain conditions, the convex polytope is ofpolynomial size.

Identify these occasions, and solve the usual way.

Dalya Gartzman Tel Aviv University

ILP Solvers

Page 15: ILP solvers

Prologue LP Solvers Adding Intergality Constraints Special Cases of ILP Epilogue

So what can we do?

I Stick to small problems. For large problems use an estimate.

I Some algorithms first ignore the integrality constraints andsolve the LP (Relax).

Then they introduce the integrality constraints in anintelligent way to receive the optimal integer solution, withbetter performance than the naive way, yet still exponential.

I Under certain conditions, the convex polytope is ofpolynomial size.

Identify these occasions, and solve the usual way.

Dalya Gartzman Tel Aviv University

ILP Solvers

Page 16: ILP solvers

Prologue LP Solvers Adding Intergality Constraints Special Cases of ILP Epilogue

So what can we do?

I Stick to small problems. For large problems use an estimate.

I Some algorithms first ignore the integrality constraints andsolve the LP (Relax).

Then they introduce the integrality constraints in anintelligent way to receive the optimal integer solution, withbetter performance than the naive way, yet still exponential.

I Under certain conditions, the convex polytope is ofpolynomial size.

Identify these occasions, and solve the usual way.

Dalya Gartzman Tel Aviv University

ILP Solvers

Page 17: ILP solvers

Prologue LP Solvers Adding Intergality Constraints Special Cases of ILP Epilogue

So what can we do?

I Stick to small problems. For large problems use an estimate.

I Some algorithms first ignore the integrality constraints andsolve the LP (Relax).

Then they introduce the integrality constraints in anintelligent way to receive the optimal integer solution, withbetter performance than the naive way, yet still exponential.

I Under certain conditions, the convex polytope is ofpolynomial size.

Identify these occasions, and solve the usual way.

Dalya Gartzman Tel Aviv University

ILP Solvers

Page 18: ILP solvers

Prologue LP Solvers Adding Intergality Constraints Special Cases of ILP Epilogue

So what can we do?

I Stick to small problems. For large problems use an estimate.

I Some algorithms first ignore the integrality constraints andsolve the LP (Relax).

Then they introduce the integrality constraints in anintelligent way to receive the optimal integer solution, withbetter performance than the naive way, yet still exponential.

I Under certain conditions, the convex polytope is ofpolynomial size.

Identify these occasions, and solve the usual way.

Dalya Gartzman Tel Aviv University

ILP Solvers

Page 19: ILP solvers

Prologue LP Solvers Adding Intergality Constraints Special Cases of ILP Epilogue

The Simplex Method (Dantzig) - The Basic Principle

I The feasible region, P = {x : Ax ≤ b}is a convex polytope.

I If an optimal solution exists, it isobtained in at least one vertex of P.

I No need to study infinitely manysolutions in P, just the vertices.

I And we don‘t even have to study allthe vertices! Start with any vertex,and trail along the polytope in thedirection that optimizes theobjective function.

I An optimal solution is expected to befound in polynomial time.

Dalya Gartzman Tel Aviv University

ILP Solvers

Page 20: ILP solvers

Prologue LP Solvers Adding Intergality Constraints Special Cases of ILP Epilogue

The Simplex Method (Dantzig) - The Basic Principle

I The feasible region, P = {x : Ax ≤ b}is a convex polytope.

I If an optimal solution exists, it isobtained in at least one vertex of P.

I No need to study infinitely manysolutions in P, just the vertices.

I And we don‘t even have to study allthe vertices! Start with any vertex,and trail along the polytope in thedirection that optimizes theobjective function.

I An optimal solution is expected to befound in polynomial time.

Dalya Gartzman Tel Aviv University

ILP Solvers

Page 21: ILP solvers

Prologue LP Solvers Adding Intergality Constraints Special Cases of ILP Epilogue

The Simplex Method (Dantzig) - The Basic Principle

I The feasible region, P = {x : Ax ≤ b}is a convex polytope.

I If an optimal solution exists, it isobtained in at least one vertex of P.

I No need to study infinitely manysolutions in P, just the vertices.

I And we don‘t even have to study allthe vertices! Start with any vertex,and trail along the polytope in thedirection that optimizes theobjective function.

I An optimal solution is expected to befound in polynomial time.

Dalya Gartzman Tel Aviv University

ILP Solvers

Page 22: ILP solvers

Prologue LP Solvers Adding Intergality Constraints Special Cases of ILP Epilogue

The Simplex Method (Dantzig) - The Basic Principle

I The feasible region, P = {x : Ax ≤ b}is a convex polytope.

I If an optimal solution exists, it isobtained in at least one vertex of P.

I No need to study infinitely manysolutions in P, just the vertices.

I And we don‘t even have to study allthe vertices! Start with any vertex,and trail along the polytope in thedirection that optimizes theobjective function.

I An optimal solution is expected to befound in polynomial time.

Dalya Gartzman Tel Aviv University

ILP Solvers

Page 23: ILP solvers

Prologue LP Solvers Adding Intergality Constraints Special Cases of ILP Epilogue

The Simplex Method (Dantzig) - The Basic Principle

I The feasible region, P = {x : Ax ≤ b}is a convex polytope.

I If an optimal solution exists, it isobtained in at least one vertex of P.

I No need to study infinitely manysolutions in P, just the vertices.

I And we don‘t even have to study allthe vertices! Start with any vertex,and trail along the polytope in thedirection that optimizes theobjective function.

I An optimal solution is expected to befound in polynomial time.

Dalya Gartzman Tel Aviv University

ILP Solvers

Page 24: ILP solvers

Prologue LP Solvers Adding Intergality Constraints Special Cases of ILP Epilogue

The Simplex Method (Dantzig) - The Basic Principle

I The feasible region, P = {x : Ax ≤ b}is a convex polytope.

I If an optimal solution exists, it isobtained in at least one vertex of P.

I No need to study infinitely manysolutions in P, just the vertices.

I And we don‘t even have to study allthe vertices! Start with any vertex,and trail along the polytope in thedirection that optimizes theobjective function.

I An optimal solution is expected to befound in polynomial time.

Dalya Gartzman Tel Aviv University

ILP Solvers

Page 25: ILP solvers

Prologue LP Solvers Adding Intergality Constraints Special Cases of ILP Epilogue

The Simplex Method (Dantzig) - The Basic Principle

I The feasible region, P = {x : Ax ≤ b}is a convex polytope.

I If an optimal solution exists, it isobtained in at least one vertex of P.

I No need to study infinitely manysolutions in P, just the vertices.

I And we don‘t even have to study allthe vertices! Start with any vertex,and trail along the polytope in thedirection that optimizes theobjective function.

I An optimal solution is expected to befound in polynomial time.

Dalya Gartzman Tel Aviv University

ILP Solvers

Page 26: ILP solvers

Prologue LP Solvers Adding Intergality Constraints Special Cases of ILP Epilogue

The Simplex Method (Dantzig) - In Practice

Introducing slack variables

For every row (Ax)i ≤ bi assign a new variable si to getai1x1 + · · ·+ ainxn + si = bi si ≥ 0

So now we have a set of linear equations and some newconstraints

Ax + Is = b s ≥ 0

The new set of equations still defines a convex polytope, and theoptimal solution still lies in its vertices.

Phase I of the algorithm finds a vertex of the new polytope, bysolving a modified LP on the set of variables s.

Phase II finds, for any given vertex, a neighbor with a better valueof the target function.

After a polynomially many expected amount of steps of phase II ,we will reach our destination.

Dalya Gartzman Tel Aviv University

ILP Solvers

Page 27: ILP solvers

Prologue LP Solvers Adding Intergality Constraints Special Cases of ILP Epilogue

The Simplex Method (Dantzig) - In PracticeIntroducing slack variables

For every row (Ax)i ≤ bi assign a new variable si to getai1x1 + · · ·+ ainxn + si = bi si ≥ 0

So now we have a set of linear equations and some newconstraints

Ax + Is = b s ≥ 0

The new set of equations still defines a convex polytope, and theoptimal solution still lies in its vertices.

Phase I of the algorithm finds a vertex of the new polytope, bysolving a modified LP on the set of variables s.

Phase II finds, for any given vertex, a neighbor with a better valueof the target function.

After a polynomially many expected amount of steps of phase II ,we will reach our destination.

Dalya Gartzman Tel Aviv University

ILP Solvers

Page 28: ILP solvers

Prologue LP Solvers Adding Intergality Constraints Special Cases of ILP Epilogue

The Simplex Method (Dantzig) - In PracticeIntroducing slack variables

For every row (Ax)i ≤ bi assign a new variable si to getai1x1 + · · ·+ ainxn + si = bi si ≥ 0

So now we have a set of linear equations and some newconstraints

Ax + Is = b s ≥ 0

The new set of equations still defines a convex polytope, and theoptimal solution still lies in its vertices.

Phase I of the algorithm finds a vertex of the new polytope, bysolving a modified LP on the set of variables s.

Phase II finds, for any given vertex, a neighbor with a better valueof the target function.

After a polynomially many expected amount of steps of phase II ,we will reach our destination.

Dalya Gartzman Tel Aviv University

ILP Solvers

Page 29: ILP solvers

Prologue LP Solvers Adding Intergality Constraints Special Cases of ILP Epilogue

The Simplex Method (Dantzig) - In PracticeIntroducing slack variables

For every row (Ax)i ≤ bi assign a new variable si to getai1x1 + · · ·+ ainxn + si = bi si ≥ 0

So now we have a set of linear equations and some newconstraints

Ax + Is = b s ≥ 0

The new set of equations still defines a convex polytope, and theoptimal solution still lies in its vertices.

Phase I of the algorithm finds a vertex of the new polytope, bysolving a modified LP on the set of variables s.

Phase II finds, for any given vertex, a neighbor with a better valueof the target function.

After a polynomially many expected amount of steps of phase II ,we will reach our destination.

Dalya Gartzman Tel Aviv University

ILP Solvers

Page 30: ILP solvers

Prologue LP Solvers Adding Intergality Constraints Special Cases of ILP Epilogue

The Simplex Method (Dantzig) - In PracticeIntroducing slack variables

For every row (Ax)i ≤ bi assign a new variable si to getai1x1 + · · ·+ ainxn + si = bi si ≥ 0

So now we have a set of linear equations and some newconstraints

Ax + Is = b s ≥ 0

The new set of equations still defines a convex polytope, and theoptimal solution still lies in its vertices.

Phase I of the algorithm finds a vertex of the new polytope, bysolving a modified LP on the set of variables s.

Phase II finds, for any given vertex, a neighbor with a better valueof the target function.

After a polynomially many expected amount of steps of phase II ,we will reach our destination.

Dalya Gartzman Tel Aviv University

ILP Solvers

Page 31: ILP solvers

Prologue LP Solvers Adding Intergality Constraints Special Cases of ILP Epilogue

The Simplex Method (Dantzig) - In PracticeIntroducing slack variables

For every row (Ax)i ≤ bi assign a new variable si to getai1x1 + · · ·+ ainxn + si = bi si ≥ 0

So now we have a set of linear equations and some newconstraints

Ax + Is = b s ≥ 0

The new set of equations still defines a convex polytope, and theoptimal solution still lies in its vertices.

Phase I of the algorithm finds a vertex of the new polytope, bysolving a modified LP on the set of variables s.

Phase II finds, for any given vertex, a neighbor with a better valueof the target function.

After a polynomially many expected amount of steps of phase II ,we will reach our destination.

Dalya Gartzman Tel Aviv University

ILP Solvers

Page 32: ILP solvers

Prologue LP Solvers Adding Intergality Constraints Special Cases of ILP Epilogue

The Simplex Method (Dantzig) - In PracticeIntroducing slack variables

For every row (Ax)i ≤ bi assign a new variable si to getai1x1 + · · ·+ ainxn + si = bi si ≥ 0

So now we have a set of linear equations and some newconstraints

Ax + Is = b s ≥ 0

The new set of equations still defines a convex polytope, and theoptimal solution still lies in its vertices.

Phase I of the algorithm finds a vertex of the new polytope, bysolving a modified LP on the set of variables s.

Phase II finds, for any given vertex, a neighbor with a better valueof the target function.

After a polynomially many expected amount of steps of phase II ,we will reach our destination.

Dalya Gartzman Tel Aviv University

ILP Solvers

Page 33: ILP solvers

Prologue LP Solvers Adding Intergality Constraints Special Cases of ILP Epilogue

The Simplex Method (Dantzig) - In PracticeIntroducing slack variables

For every row (Ax)i ≤ bi assign a new variable si to getai1x1 + · · ·+ ainxn + si = bi si ≥ 0

So now we have a set of linear equations and some newconstraints

Ax + Is = b s ≥ 0

The new set of equations still defines a convex polytope, and theoptimal solution still lies in its vertices.

Phase I of the algorithm finds a vertex of the new polytope, bysolving a modified LP on the set of variables s.

Phase II finds, for any given vertex, a neighbor with a better valueof the target function.

After a polynomially many expected amount of steps of phase II ,we will reach our destination.

Dalya Gartzman Tel Aviv University

ILP Solvers

Page 34: ILP solvers

Prologue LP Solvers Adding Intergality Constraints Special Cases of ILP Epilogue

The Dual Simplex Method (Lemke) - Preliminaries

The Dual Problem

For a primal LP{

max . ctxs.t. Ax≤b

}define the dual to be

{min. btys.t. Aty≥c

}.

This is called the symmetrical version, since the dual to the dual isthe primal.There exists also a non-symmetrical version, of which we will notdiscuss today.

Fenchel’s duality theorem

For every system as described above, andevery x , y feasible solutions of the primal anddual, respectively, ctx ≤ bty .Furthermore, the primal has an optimalsolution x∗ iff the dual had an optimal solutiony∗, and in this case ctx∗ = bty∗.

Dalya Gartzman Tel Aviv University

ILP Solvers

Page 35: ILP solvers

Prologue LP Solvers Adding Intergality Constraints Special Cases of ILP Epilogue

The Dual Simplex Method (Lemke) - Preliminaries

The Dual Problem

For a primal LP{

max . ctxs.t. Ax≤b

}define the dual to be

{min. btys.t. Aty≥c

}.

This is called the symmetrical version, since the dual to the dual isthe primal.There exists also a non-symmetrical version, of which we will notdiscuss today.

Fenchel’s duality theorem

For every system as described above, andevery x , y feasible solutions of the primal anddual, respectively, ctx ≤ bty .Furthermore, the primal has an optimalsolution x∗ iff the dual had an optimal solutiony∗, and in this case ctx∗ = bty∗.

Dalya Gartzman Tel Aviv University

ILP Solvers

Page 36: ILP solvers

Prologue LP Solvers Adding Intergality Constraints Special Cases of ILP Epilogue

The Dual Simplex Method (Lemke) - Preliminaries

The Dual Problem

For a primal LP{

max . ctxs.t. Ax≤b

}define the dual to be

{min. btys.t. Aty≥c

}.

This is called the symmetrical version, since the dual to the dual isthe primal.There exists also a non-symmetrical version, of which we will notdiscuss today.

Fenchel’s duality theorem

For every system as described above, andevery x , y feasible solutions of the primal anddual, respectively, ctx ≤ bty .Furthermore, the primal has an optimalsolution x∗ iff the dual had an optimal solutiony∗, and in this case ctx∗ = bty∗.

Dalya Gartzman Tel Aviv University

ILP Solvers

Page 37: ILP solvers

Prologue LP Solvers Adding Intergality Constraints Special Cases of ILP Epilogue

The Dual Simplex Method (Lemke) - Preliminaries

The Dual Problem

For a primal LP{

max . ctxs.t. Ax≤b

}define the dual to be

{min. btys.t. Aty≥c

}.

This is called the symmetrical version, since the dual to the dual isthe primal.

There exists also a non-symmetrical version, of which we will notdiscuss today.

Fenchel’s duality theorem

For every system as described above, andevery x , y feasible solutions of the primal anddual, respectively, ctx ≤ bty .Furthermore, the primal has an optimalsolution x∗ iff the dual had an optimal solutiony∗, and in this case ctx∗ = bty∗.

Dalya Gartzman Tel Aviv University

ILP Solvers

Page 38: ILP solvers

Prologue LP Solvers Adding Intergality Constraints Special Cases of ILP Epilogue

The Dual Simplex Method (Lemke) - Preliminaries

The Dual Problem

For a primal LP{

max . ctxs.t. Ax≤b

}define the dual to be

{min. btys.t. Aty≥c

}.

This is called the symmetrical version, since the dual to the dual isthe primal.There exists also a non-symmetrical version, of which we will notdiscuss today.

Fenchel’s duality theorem

For every system as described above, andevery x , y feasible solutions of the primal anddual, respectively, ctx ≤ bty .Furthermore, the primal has an optimalsolution x∗ iff the dual had an optimal solutiony∗, and in this case ctx∗ = bty∗.

Dalya Gartzman Tel Aviv University

ILP Solvers

Page 39: ILP solvers

Prologue LP Solvers Adding Intergality Constraints Special Cases of ILP Epilogue

The Dual Simplex Method (Lemke) - Preliminaries

The Dual Problem

For a primal LP{

max . ctxs.t. Ax≤b

}define the dual to be

{min. btys.t. Aty≥c

}.

This is called the symmetrical version, since the dual to the dual isthe primal.There exists also a non-symmetrical version, of which we will notdiscuss today.

Fenchel’s duality theorem

For every system as described above, andevery x , y feasible solutions of the primal anddual, respectively, ctx ≤ bty .Furthermore, the primal has an optimalsolution x∗ iff the dual had an optimal solutiony∗, and in this case ctx∗ = bty∗.

Dalya Gartzman Tel Aviv University

ILP Solvers

Page 40: ILP solvers

Prologue LP Solvers Adding Intergality Constraints Special Cases of ILP Epilogue

The Dual Simplex Method (Lemke) - Preliminaries

The Dual Problem

For a primal LP{

max . ctxs.t. Ax≤b

}define the dual to be

{min. btys.t. Aty≥c

}.

This is called the symmetrical version, since the dual to the dual isthe primal.There exists also a non-symmetrical version, of which we will notdiscuss today.

Fenchel’s duality theorem

For every system as described above, andevery x , y feasible solutions of the primal anddual, respectively, ctx ≤ bty .

Furthermore, the primal has an optimalsolution x∗ iff the dual had an optimal solutiony∗, and in this case ctx∗ = bty∗.

Dalya Gartzman Tel Aviv University

ILP Solvers

Page 41: ILP solvers

Prologue LP Solvers Adding Intergality Constraints Special Cases of ILP Epilogue

The Dual Simplex Method (Lemke) - Preliminaries

The Dual Problem

For a primal LP{

max . ctxs.t. Ax≤b

}define the dual to be

{min. btys.t. Aty≥c

}.

This is called the symmetrical version, since the dual to the dual isthe primal.There exists also a non-symmetrical version, of which we will notdiscuss today.

Fenchel’s duality theorem

For every system as described above, andevery x , y feasible solutions of the primal anddual, respectively, ctx ≤ bty .Furthermore, the primal has an optimalsolution x∗ iff the dual had an optimal solutiony∗, and in this case ctx∗ = bty∗.

Dalya Gartzman Tel Aviv University

ILP Solvers

Page 42: ILP solvers

Prologue LP Solvers Adding Intergality Constraints Special Cases of ILP Epilogue

The Dual Simplex Method (Lemke) - Preliminaries

The Dual Problem

For a primal LP{

max . ctxs.t. Ax≤b

}define the dual to be

{min. btys.t. Aty≥c

}.

This is called the symmetrical version, since the dual to the dual isthe primal.There exists also a non-symmetrical version, of which we will notdiscuss today.

Fenchel’s duality theorem

For every system as described above, andevery x , y feasible solutions of the primal anddual, respectively, ctx ≤ bty .Furthermore, the primal has an optimalsolution x∗ iff the dual had an optimal solutiony∗, and in this case ctx∗ = bty∗.

Dalya Gartzman Tel Aviv University

ILP Solvers

Page 43: ILP solvers

Prologue LP Solvers Adding Intergality Constraints Special Cases of ILP Epilogue

The Dual Simplex Method (Lemke) - Some Details

I According to the thm. stated above, if our LP is feasible, andhas an optimal solution - solve the dual with the simplexmethod.

I Conversion takes time, but we don‘t have to go all the way.

I Begin with a dual-feasible solution, and continue with linearalgebra techniques to get y∗ = x∗.

Benefits

I Sometimes the dual is simpler, esp. when the number ofconstraints is much larger than the number of variables.

I Easier to introduce new constraints.

Dalya Gartzman Tel Aviv University

ILP Solvers

Page 44: ILP solvers

Prologue LP Solvers Adding Intergality Constraints Special Cases of ILP Epilogue

The Dual Simplex Method (Lemke) - Some Details

I According to the thm. stated above, if our LP is feasible, andhas an optimal solution - solve the dual with the simplexmethod.

I Conversion takes time, but we don‘t have to go all the way.

I Begin with a dual-feasible solution, and continue with linearalgebra techniques to get y∗ = x∗.

Benefits

I Sometimes the dual is simpler, esp. when the number ofconstraints is much larger than the number of variables.

I Easier to introduce new constraints.

Dalya Gartzman Tel Aviv University

ILP Solvers

Page 45: ILP solvers

Prologue LP Solvers Adding Intergality Constraints Special Cases of ILP Epilogue

The Dual Simplex Method (Lemke) - Some Details

I According to the thm. stated above, if our LP is feasible, andhas an optimal solution - solve the dual with the simplexmethod.

I Conversion takes time, but we don‘t have to go all the way.

I Begin with a dual-feasible solution, and continue with linearalgebra techniques to get y∗ = x∗.

Benefits

I Sometimes the dual is simpler, esp. when the number ofconstraints is much larger than the number of variables.

I Easier to introduce new constraints.

Dalya Gartzman Tel Aviv University

ILP Solvers

Page 46: ILP solvers

Prologue LP Solvers Adding Intergality Constraints Special Cases of ILP Epilogue

The Dual Simplex Method (Lemke) - Some Details

I According to the thm. stated above, if our LP is feasible, andhas an optimal solution - solve the dual with the simplexmethod.

I Conversion takes time, but we don‘t have to go all the way.

I Begin with a dual-feasible solution, and continue with linearalgebra techniques to get y∗ = x∗.

Benefits

I Sometimes the dual is simpler, esp. when the number ofconstraints is much larger than the number of variables.

I Easier to introduce new constraints.

Dalya Gartzman Tel Aviv University

ILP Solvers

Page 47: ILP solvers

Prologue LP Solvers Adding Intergality Constraints Special Cases of ILP Epilogue

The Dual Simplex Method (Lemke) - Some Details

I According to the thm. stated above, if our LP is feasible, andhas an optimal solution - solve the dual with the simplexmethod.

I Conversion takes time, but we don‘t have to go all the way.

I Begin with a dual-feasible solution, and continue with linearalgebra techniques to get y∗ = x∗.

Benefits

I Sometimes the dual is simpler, esp. when the number ofconstraints is much larger than the number of variables.

I Easier to introduce new constraints.

Dalya Gartzman Tel Aviv University

ILP Solvers

Page 48: ILP solvers

Prologue LP Solvers Adding Intergality Constraints Special Cases of ILP Epilogue

The Dual Simplex Method (Lemke) - Some Details

I According to the thm. stated above, if our LP is feasible, andhas an optimal solution - solve the dual with the simplexmethod.

I Conversion takes time, but we don‘t have to go all the way.

I Begin with a dual-feasible solution, and continue with linearalgebra techniques to get y∗ = x∗.

Benefits

I Sometimes the dual is simpler, esp. when the number ofconstraints is much larger than the number of variables.

I Easier to introduce new constraints.

Dalya Gartzman Tel Aviv University

ILP Solvers

Page 49: ILP solvers

Prologue LP Solvers Adding Intergality Constraints Special Cases of ILP Epilogue

The Dual Simplex Method (Lemke) - Some Details

I According to the thm. stated above, if our LP is feasible, andhas an optimal solution - solve the dual with the simplexmethod.

I Conversion takes time, but we don‘t have to go all the way.

I Begin with a dual-feasible solution, and continue with linearalgebra techniques to get y∗ = x∗.

Benefits

I Sometimes the dual is simpler, esp. when the number ofconstraints is much larger than the number of variables.

I Easier to introduce new constraints.

Dalya Gartzman Tel Aviv University

ILP Solvers

Page 50: ILP solvers

Prologue LP Solvers Adding Intergality Constraints Special Cases of ILP Epilogue

Additional Interesting LP Solvers

Prune and Search (Meggido 1983)Pairing the set of constraints to find theinferior and removing it.Performance: O

(n22

d )where n is the number

of constraints and d the number of variables.

Interior Point (Karmarkar 1984)

Find a point inside the feasible region, andreach the optimal solution from within.Performance: O

(n2d

)Random Incremental Algorithm (Sharir, Welzl 1992)

Randomly add constraints and maintain the optimal solution.Performance: O

(n3.5

)where n is the number of variables.

Dalya Gartzman Tel Aviv University

ILP Solvers

Page 51: ILP solvers

Prologue LP Solvers Adding Intergality Constraints Special Cases of ILP Epilogue

Additional Interesting LP Solvers

Prune and Search (Meggido 1983)

Pairing the set of constraints to find theinferior and removing it.Performance: O

(n22

d )where n is the number

of constraints and d the number of variables.

Interior Point (Karmarkar 1984)

Find a point inside the feasible region, andreach the optimal solution from within.Performance: O

(n2d

)Random Incremental Algorithm (Sharir, Welzl 1992)

Randomly add constraints and maintain the optimal solution.Performance: O

(n3.5

)where n is the number of variables.

Dalya Gartzman Tel Aviv University

ILP Solvers

Page 52: ILP solvers

Prologue LP Solvers Adding Intergality Constraints Special Cases of ILP Epilogue

Additional Interesting LP Solvers

Prune and Search (Meggido 1983)Pairing the set of constraints to find theinferior and removing it.

Performance: O(n22

d )where n is the number

of constraints and d the number of variables.

Interior Point (Karmarkar 1984)

Find a point inside the feasible region, andreach the optimal solution from within.Performance: O

(n2d

)Random Incremental Algorithm (Sharir, Welzl 1992)

Randomly add constraints and maintain the optimal solution.Performance: O

(n3.5

)where n is the number of variables.

Dalya Gartzman Tel Aviv University

ILP Solvers

Page 53: ILP solvers

Prologue LP Solvers Adding Intergality Constraints Special Cases of ILP Epilogue

Additional Interesting LP Solvers

Prune and Search (Meggido 1983)Pairing the set of constraints to find theinferior and removing it.Performance: O

(n22

d )where n is the number

of constraints and d the number of variables.

Interior Point (Karmarkar 1984)

Find a point inside the feasible region, andreach the optimal solution from within.Performance: O

(n2d

)Random Incremental Algorithm (Sharir, Welzl 1992)

Randomly add constraints and maintain the optimal solution.Performance: O

(n3.5

)where n is the number of variables.

Dalya Gartzman Tel Aviv University

ILP Solvers

Page 54: ILP solvers

Prologue LP Solvers Adding Intergality Constraints Special Cases of ILP Epilogue

Additional Interesting LP Solvers

Prune and Search (Meggido 1983)Pairing the set of constraints to find theinferior and removing it.Performance: O

(n22

d )where n is the number

of constraints and d the number of variables.

Interior Point (Karmarkar 1984)

Find a point inside the feasible region, andreach the optimal solution from within.Performance: O

(n2d

)Random Incremental Algorithm (Sharir, Welzl 1992)

Randomly add constraints and maintain the optimal solution.Performance: O

(n3.5

)where n is the number of variables.

Dalya Gartzman Tel Aviv University

ILP Solvers

Page 55: ILP solvers

Prologue LP Solvers Adding Intergality Constraints Special Cases of ILP Epilogue

Additional Interesting LP Solvers

Prune and Search (Meggido 1983)Pairing the set of constraints to find theinferior and removing it.Performance: O

(n22

d )where n is the number

of constraints and d the number of variables.

Interior Point (Karmarkar 1984)

Find a point inside the feasible region, andreach the optimal solution from within.Performance: O

(n2d

)Random Incremental Algorithm (Sharir, Welzl 1992)

Randomly add constraints and maintain the optimal solution.Performance: O

(n3.5

)where n is the number of variables.

Dalya Gartzman Tel Aviv University

ILP Solvers

Page 56: ILP solvers

Prologue LP Solvers Adding Intergality Constraints Special Cases of ILP Epilogue

Additional Interesting LP Solvers

Prune and Search (Meggido 1983)Pairing the set of constraints to find theinferior and removing it.Performance: O

(n22

d )where n is the number

of constraints and d the number of variables.

Interior Point (Karmarkar 1984)

Find a point inside the feasible region, andreach the optimal solution from within.

Performance: O(n2d

)Random Incremental Algorithm (Sharir, Welzl 1992)

Randomly add constraints and maintain the optimal solution.Performance: O

(n3.5

)where n is the number of variables.

Dalya Gartzman Tel Aviv University

ILP Solvers

Page 57: ILP solvers

Prologue LP Solvers Adding Intergality Constraints Special Cases of ILP Epilogue

Additional Interesting LP Solvers

Prune and Search (Meggido 1983)Pairing the set of constraints to find theinferior and removing it.Performance: O

(n22

d )where n is the number

of constraints and d the number of variables.

Interior Point (Karmarkar 1984)

Find a point inside the feasible region, andreach the optimal solution from within.Performance: O

(n2d

)

Random Incremental Algorithm (Sharir, Welzl 1992)

Randomly add constraints and maintain the optimal solution.Performance: O

(n3.5

)where n is the number of variables.

Dalya Gartzman Tel Aviv University

ILP Solvers

Page 58: ILP solvers

Prologue LP Solvers Adding Intergality Constraints Special Cases of ILP Epilogue

Additional Interesting LP Solvers

Prune and Search (Meggido 1983)Pairing the set of constraints to find theinferior and removing it.Performance: O

(n22

d )where n is the number

of constraints and d the number of variables.

Interior Point (Karmarkar 1984)

Find a point inside the feasible region, andreach the optimal solution from within.Performance: O

(n2d

)

Random Incremental Algorithm (Sharir, Welzl 1992)

Randomly add constraints and maintain the optimal solution.Performance: O

(n3.5

)where n is the number of variables.

Dalya Gartzman Tel Aviv University

ILP Solvers

Page 59: ILP solvers

Prologue LP Solvers Adding Intergality Constraints Special Cases of ILP Epilogue

Additional Interesting LP Solvers

Prune and Search (Meggido 1983)Pairing the set of constraints to find theinferior and removing it.Performance: O

(n22

d )where n is the number

of constraints and d the number of variables.

Interior Point (Karmarkar 1984)

Find a point inside the feasible region, andreach the optimal solution from within.Performance: O

(n2d

)Random Incremental Algorithm (Sharir, Welzl 1992)

Randomly add constraints and maintain the optimal solution.Performance: O

(n3.5

)where n is the number of variables.

Dalya Gartzman Tel Aviv University

ILP Solvers

Page 60: ILP solvers

Prologue LP Solvers Adding Intergality Constraints Special Cases of ILP Epilogue

Additional Interesting LP Solvers

Prune and Search (Meggido 1983)Pairing the set of constraints to find theinferior and removing it.Performance: O

(n22

d )where n is the number

of constraints and d the number of variables.

Interior Point (Karmarkar 1984)

Find a point inside the feasible region, andreach the optimal solution from within.Performance: O

(n2d

)Random Incremental Algorithm (Sharir, Welzl 1992)

Randomly add constraints and maintain the optimal solution.

Performance: O(n3.5

)where n is the number of variables.

Dalya Gartzman Tel Aviv University

ILP Solvers

Page 61: ILP solvers

Prologue LP Solvers Adding Intergality Constraints Special Cases of ILP Epilogue

Additional Interesting LP Solvers

Prune and Search (Meggido 1983)Pairing the set of constraints to find theinferior and removing it.Performance: O

(n22

d )where n is the number

of constraints and d the number of variables.

Interior Point (Karmarkar 1984)

Find a point inside the feasible region, andreach the optimal solution from within.Performance: O

(n2d

)Random Incremental Algorithm (Sharir, Welzl 1992)

Randomly add constraints and maintain the optimal solution.Performance: O

(n3.5

)where n is the number of variables.

Dalya Gartzman Tel Aviv University

ILP Solvers

Page 62: ILP solvers

Prologue LP Solvers Adding Intergality Constraints Special Cases of ILP Epilogue

Solving ILP by Relaxing

I Given the ILP{

max . ctxs.t. Ax≤b, x is integral

}, define its

LP relaxedion{

max . ctxs.t. Ax≤b

}.

I The feasible region of the ILP is included in that of the LP

P = {x : Ax ≤ b, x is integral}⊂ K = {x : Ax ≤ b}

I Once we have found K , how do we find P?I And what can K tell us about P?

Dalya Gartzman Tel Aviv University

ILP Solvers

Page 63: ILP solvers

Prologue LP Solvers Adding Intergality Constraints Special Cases of ILP Epilogue

Solving ILP by Relaxing

I Given the ILP{

max . ctxs.t. Ax≤b, x is integral

}, define its

LP relaxedion{

max . ctxs.t. Ax≤b

}.

I The feasible region of the ILP is included in that of the LP

P = {x : Ax ≤ b, x is integral}⊂ K = {x : Ax ≤ b}

I Once we have found K , how do we find P?

I And what can K tell us about P?

Dalya Gartzman Tel Aviv University

ILP Solvers

Page 64: ILP solvers

Prologue LP Solvers Adding Intergality Constraints Special Cases of ILP Epilogue

Solving ILP by Relaxing

I Given the ILP{

max . ctxs.t. Ax≤b, x is integral

}, define its

LP relaxedion{

max . ctxs.t. Ax≤b

}.

I The feasible region of the ILP is included in that of the LP

P = {x : Ax ≤ b, x is integral}⊂ K = {x : Ax ≤ b}

I Once we have found K , how do we find P?

I And what can K tell us about P?

Dalya Gartzman Tel Aviv University

ILP Solvers

Page 65: ILP solvers

Prologue LP Solvers Adding Intergality Constraints Special Cases of ILP Epilogue

Solving ILP by Relaxing

I Given the ILP{

max . ctxs.t. Ax≤b, x is integral

}, define its

LP relaxedion{

max . ctxs.t. Ax≤b

}.

I The feasible region of the ILP is included in that of the LP

P = {x : Ax ≤ b, x is integral}⊂ K = {x : Ax ≤ b}

I Once we have found K , how do we find P?

I And what can K tell us about P?

Dalya Gartzman Tel Aviv University

ILP Solvers

Page 66: ILP solvers

Prologue LP Solvers Adding Intergality Constraints Special Cases of ILP Epilogue

Solving ILP by Relaxing

I Given the ILP{

max . ctxs.t. Ax≤b, x is integral

}, define its

LP relaxedion{

max . ctxs.t. Ax≤b

}.

I The feasible region of the ILP is included in that of the LP

P = {x : Ax ≤ b, x is integral}⊂ K = {x : Ax ≤ b}

I Once we have found K , how do we find P?

I And what can K tell us about P?

Dalya Gartzman Tel Aviv University

ILP Solvers

Page 67: ILP solvers

Prologue LP Solvers Adding Intergality Constraints Special Cases of ILP Epilogue

Solving ILP by Relaxing

I Given the ILP{

max . ctxs.t. Ax≤b, x is integral

}, define its

LP relaxedion{

max . ctxs.t. Ax≤b

}.

I The feasible region of the ILP is included in that of the LP

P = {x : Ax ≤ b, x is integral}⊂ K = {x : Ax ≤ b}

I Once we have found K , how do we find P?

I And what can K tell us about P?

Dalya Gartzman Tel Aviv University

ILP Solvers

Page 68: ILP solvers

Prologue LP Solvers Adding Intergality Constraints Special Cases of ILP Epilogue

Solving ILP by Relaxing

I Given the ILP{

max . ctxs.t. Ax≤b, x is integral

}, define its

LP relaxedion{

max . ctxs.t. Ax≤b

}.

I The feasible region of the ILP is included in that of the LP

P = {x : Ax ≤ b, x is integral}⊂ K = {x : Ax ≤ b}

I Once we have found K , how do we find P?

I And what can K tell us about P?

Dalya Gartzman Tel Aviv University

ILP Solvers

Page 69: ILP solvers

Prologue LP Solvers Adding Intergality Constraints Special Cases of ILP Epilogue

The Integrality Gap

Since P ⊂ K , the optimal solution ctx∗ is an upper bound for theintegral one.

But simply taking bctx∗c is not enough!

Lovasz proved that the integrality gap IG = MintMfrac

for the Set Coverproblem on n vertices is Hn, the n-th harmonic number.

But here is a fun example where it does work:

Dalya Gartzman Tel Aviv University

ILP Solvers

Page 70: ILP solvers

Prologue LP Solvers Adding Intergality Constraints Special Cases of ILP Epilogue

The Integrality Gap

Since P ⊂ K , the optimal solution ctx∗ is an upper bound for theintegral one.

But simply taking bctx∗c is not enough!

Lovasz proved that the integrality gap IG = MintMfrac

for the Set Coverproblem on n vertices is Hn, the n-th harmonic number.

But here is a fun example where it does work:

Dalya Gartzman Tel Aviv University

ILP Solvers

Page 71: ILP solvers

Prologue LP Solvers Adding Intergality Constraints Special Cases of ILP Epilogue

The Integrality Gap

Since P ⊂ K , the optimal solution ctx∗ is an upper bound for theintegral one.

But simply taking bctx∗c is not enough!

Lovasz proved that the integrality gap IG = MintMfrac

for the Set Coverproblem on n vertices is Hn, the n-th harmonic number.

But here is a fun example where it does work:

Dalya Gartzman Tel Aviv University

ILP Solvers

Page 72: ILP solvers

Prologue LP Solvers Adding Intergality Constraints Special Cases of ILP Epilogue

The Integrality Gap

Since P ⊂ K , the optimal solution ctx∗ is an upper bound for theintegral one.

But simply taking bctx∗c is not enough!

Lovasz proved that the integrality gap IG = MintMfrac

for the Set Coverproblem on n vertices is Hn, the n-th harmonic number.

But here is a fun example where it does work:

Dalya Gartzman Tel Aviv University

ILP Solvers

Page 73: ILP solvers

Prologue LP Solvers Adding Intergality Constraints Special Cases of ILP Epilogue

The Integrality Gap

Since P ⊂ K , the optimal solution ctx∗ is an upper bound for theintegral one.

But simply taking bctx∗c is not enough!

Lovasz proved that the integrality gap IG = MintMfrac

for the Set Coverproblem on n vertices is Hn, the n-th harmonic number.

But here is a fun example where it does work:

Dalya Gartzman Tel Aviv University

ILP Solvers

Page 74: ILP solvers

Prologue LP Solvers Adding Intergality Constraints Special Cases of ILP Epilogue

Branch-and-Bound Algorithm (Land, Doig)

1. Solve the LP relaxation to get bounds onthe solution and maintain those boundsduring the procedure.

2. If the lower and upper bounds are equal orclose enough, stop.

3. else

3.1 Say x∗i is a fraction.

Branch the problem to two sub-problems. one sub-problemwith xi ≤ bx∗

i c and the other with xi ≥ dx∗i e.

3.2 Bound the solution of the subproblem. If the new boundsimprove the current bounds, update and go back to (2).If a subproblem is infeasible, unbounded, or produces a uselessbound, prune this branch.

Dalya Gartzman Tel Aviv University

ILP Solvers

Page 75: ILP solvers

Prologue LP Solvers Adding Intergality Constraints Special Cases of ILP Epilogue

Branch-and-Bound Algorithm (Land, Doig)

1. Solve the LP relaxation to get bounds onthe solution and maintain those boundsduring the procedure.

2. If the lower and upper bounds are equal orclose enough, stop.

3. else

3.1 Say x∗i is a fraction.

Branch the problem to two sub-problems. one sub-problemwith xi ≤ bx∗

i c and the other with xi ≥ dx∗i e.

3.2 Bound the solution of the subproblem. If the new boundsimprove the current bounds, update and go back to (2).If a subproblem is infeasible, unbounded, or produces a uselessbound, prune this branch.

Dalya Gartzman Tel Aviv University

ILP Solvers

Page 76: ILP solvers

Prologue LP Solvers Adding Intergality Constraints Special Cases of ILP Epilogue

Branch-and-Bound Algorithm (Land, Doig)

1. Solve the LP relaxation to get bounds onthe solution and maintain those boundsduring the procedure.

2. If the lower and upper bounds are equal orclose enough, stop.

3. else

3.1 Say x∗i is a fraction.

Branch the problem to two sub-problems. one sub-problemwith xi ≤ bx∗

i c and the other with xi ≥ dx∗i e.

3.2 Bound the solution of the subproblem. If the new boundsimprove the current bounds, update and go back to (2).If a subproblem is infeasible, unbounded, or produces a uselessbound, prune this branch.

Dalya Gartzman Tel Aviv University

ILP Solvers

Page 77: ILP solvers

Prologue LP Solvers Adding Intergality Constraints Special Cases of ILP Epilogue

Branch-and-Bound Algorithm (Land, Doig)

1. Solve the LP relaxation to get bounds onthe solution and maintain those boundsduring the procedure.

2. If the lower and upper bounds are equal orclose enough, stop.

3. else

3.1 Say x∗i is a fraction.

Branch the problem to two sub-problems. one sub-problemwith xi ≤ bx∗

i c and the other with xi ≥ dx∗i e.

3.2 Bound the solution of the subproblem. If the new boundsimprove the current bounds, update and go back to (2).If a subproblem is infeasible, unbounded, or produces a uselessbound, prune this branch.

Dalya Gartzman Tel Aviv University

ILP Solvers

Page 78: ILP solvers

Prologue LP Solvers Adding Intergality Constraints Special Cases of ILP Epilogue

Branch-and-Bound Algorithm (Land, Doig)

1. Solve the LP relaxation to get bounds onthe solution and maintain those boundsduring the procedure.

2. If the lower and upper bounds are equal orclose enough, stop.

3. else

3.1 Say x∗i is a fraction.

Branch the problem to two sub-problems.

one sub-problemwith xi ≤ bx∗

i c and the other with xi ≥ dx∗i e.

3.2 Bound the solution of the subproblem. If the new boundsimprove the current bounds, update and go back to (2).If a subproblem is infeasible, unbounded, or produces a uselessbound, prune this branch.

Dalya Gartzman Tel Aviv University

ILP Solvers

Page 79: ILP solvers

Prologue LP Solvers Adding Intergality Constraints Special Cases of ILP Epilogue

Branch-and-Bound Algorithm (Land, Doig)

1. Solve the LP relaxation to get bounds onthe solution and maintain those boundsduring the procedure.

2. If the lower and upper bounds are equal orclose enough, stop.

3. else

3.1 Say x∗i is a fraction.

Branch the problem to two sub-problems. one sub-problemwith xi ≤ bx∗

i c and the other with xi ≥ dx∗i e.

3.2 Bound the solution of the subproblem. If the new boundsimprove the current bounds, update and go back to (2).If a subproblem is infeasible, unbounded, or produces a uselessbound, prune this branch.

Dalya Gartzman Tel Aviv University

ILP Solvers

Page 80: ILP solvers

Prologue LP Solvers Adding Intergality Constraints Special Cases of ILP Epilogue

Branch-and-Bound Algorithm (Land, Doig)

1. Solve the LP relaxation to get bounds onthe solution and maintain those boundsduring the procedure.

2. If the lower and upper bounds are equal orclose enough, stop.

3. else

3.1 Say x∗i is a fraction.

Branch the problem to two sub-problems. one sub-problemwith xi ≤ bx∗

i c and the other with xi ≥ dx∗i e.

3.2 Bound the solution of the subproblem. If the new boundsimprove the current bounds, update and go back to (2).If a subproblem is infeasible, unbounded, or produces a uselessbound, prune this branch.

Dalya Gartzman Tel Aviv University

ILP Solvers

Page 81: ILP solvers

Prologue LP Solvers Adding Intergality Constraints Special Cases of ILP Epilogue

Branching and Bounding

Branching Techniques

I Most Infeasible Branching

I Pseudo Cost Branching

I Strong Branching

Bounding Techniques

I Naıve - Solve relaxation to get upper bound, and find feasiblesolution for lower bound.

I Computationaly hard - Solve with Lagrangian Relaxation toget both upper and lower bounds.

I In Practice - A combination

Dalya Gartzman Tel Aviv University

ILP Solvers

Page 82: ILP solvers

Prologue LP Solvers Adding Intergality Constraints Special Cases of ILP Epilogue

Branching and Bounding

Branching Techniques

I Most Infeasible Branching

I Pseudo Cost Branching

I Strong Branching

Bounding Techniques

I Naıve - Solve relaxation to get upper bound, and find feasiblesolution for lower bound.

I Computationaly hard - Solve with Lagrangian Relaxation toget both upper and lower bounds.

I In Practice - A combination

Dalya Gartzman Tel Aviv University

ILP Solvers

Page 83: ILP solvers

Prologue LP Solvers Adding Intergality Constraints Special Cases of ILP Epilogue

Branching and Bounding

Branching Techniques

I Most Infeasible Branching

I Pseudo Cost Branching

I Strong Branching

Bounding Techniques

I Naıve - Solve relaxation to get upper bound, and find feasiblesolution for lower bound.

I Computationaly hard - Solve with Lagrangian Relaxation toget both upper and lower bounds.

I In Practice - A combination

Dalya Gartzman Tel Aviv University

ILP Solvers

Page 84: ILP solvers

Prologue LP Solvers Adding Intergality Constraints Special Cases of ILP Epilogue

Branching and Bounding

Branching Techniques

I Most Infeasible Branching

I Pseudo Cost Branching

I Strong Branching

Bounding Techniques

I Naıve - Solve relaxation to get upper bound, and find feasiblesolution for lower bound.

I Computationaly hard - Solve with Lagrangian Relaxation toget both upper and lower bounds.

I In Practice - A combination

Dalya Gartzman Tel Aviv University

ILP Solvers

Page 85: ILP solvers

Prologue LP Solvers Adding Intergality Constraints Special Cases of ILP Epilogue

Branching and Bounding

Branching Techniques

I Most Infeasible Branching

I Pseudo Cost Branching

I Strong Branching

Bounding Techniques

I Naıve - Solve relaxation to get upper bound, and find feasiblesolution for lower bound.

I Computationaly hard - Solve with Lagrangian Relaxation toget both upper and lower bounds.

I In Practice - A combination

Dalya Gartzman Tel Aviv University

ILP Solvers

Page 86: ILP solvers

Prologue LP Solvers Adding Intergality Constraints Special Cases of ILP Epilogue

Branching and Bounding

Branching Techniques

I Most Infeasible Branching

I Pseudo Cost Branching

I Strong Branching

Bounding Techniques

I Naıve - Solve relaxation to get upper bound, and find feasiblesolution for lower bound.

I Computationaly hard - Solve with Lagrangian Relaxation toget both upper and lower bounds.

I In Practice - A combination

Dalya Gartzman Tel Aviv University

ILP Solvers

Page 87: ILP solvers

Prologue LP Solvers Adding Intergality Constraints Special Cases of ILP Epilogue

Branching and Bounding

Branching Techniques

I Most Infeasible Branching

I Pseudo Cost Branching

I Strong Branching

Bounding Techniques

I Naıve - Solve relaxation to get upper bound, and find feasiblesolution for lower bound.

I Computationaly hard - Solve with Lagrangian Relaxation toget both upper and lower bounds.

I In Practice - A combination

Dalya Gartzman Tel Aviv University

ILP Solvers

Page 88: ILP solvers

Prologue LP Solvers Adding Intergality Constraints Special Cases of ILP Epilogue

Branching and Bounding

Branching Techniques

I Most Infeasible Branching

I Pseudo Cost Branching

I Strong Branching

Bounding Techniques

I Naıve - Solve relaxation to get upper bound, and find feasiblesolution for lower bound.

I Computationaly hard - Solve with Lagrangian Relaxation toget both upper and lower bounds.

I In Practice - A combination

Dalya Gartzman Tel Aviv University

ILP Solvers

Page 89: ILP solvers

Prologue LP Solvers Adding Intergality Constraints Special Cases of ILP Epilogue

Branching and Bounding

Branching Techniques

I Most Infeasible Branching

I Pseudo Cost Branching

I Strong Branching

Bounding Techniques

I Naıve - Solve relaxation to get upper bound, and find feasiblesolution for lower bound.

I Computationaly hard - Solve with Lagrangian Relaxation toget both upper and lower bounds.

I In Practice - A combination

Dalya Gartzman Tel Aviv University

ILP Solvers

Page 90: ILP solvers

Prologue LP Solvers Adding Intergality Constraints Special Cases of ILP Epilogue

Branch-and-Cut Algorithm (Padberg, Rinaldy) -Perliminaries

Gomory’s cut

When solving the LP relaxations, we getctx∗ = z where x∗ has fractional coordinates.We wish to separate x∗ from K .We can do this by adding bctx∗c ≤ bzc to thelist of constraints.

Cutting Planes (Gomory, Chvatal)

· Solve the LP relaxation to get K· Use Gomory‘s cuts to get K ′, P ⊂ K ′ ⊂ K· Repeat the process until you get P

Dalya Gartzman Tel Aviv University

ILP Solvers

Page 91: ILP solvers

Prologue LP Solvers Adding Intergality Constraints Special Cases of ILP Epilogue

Branch-and-Cut Algorithm (Padberg, Rinaldy) -Perliminaries

Gomory’s cut

When solving the LP relaxations, we getctx∗ = z where x∗ has fractional coordinates.We wish to separate x∗ from K .We can do this by adding bctx∗c ≤ bzc to thelist of constraints.

Cutting Planes (Gomory, Chvatal)

· Solve the LP relaxation to get K· Use Gomory‘s cuts to get K ′, P ⊂ K ′ ⊂ K· Repeat the process until you get P

Dalya Gartzman Tel Aviv University

ILP Solvers

Page 92: ILP solvers

Prologue LP Solvers Adding Intergality Constraints Special Cases of ILP Epilogue

Branch-and-Cut Algorithm (Padberg, Rinaldy) -Perliminaries

Gomory’s cut

When solving the LP relaxations, we getctx∗ = z where x∗ has fractional coordinates.

We wish to separate x∗ from K .We can do this by adding bctx∗c ≤ bzc to thelist of constraints.

Cutting Planes (Gomory, Chvatal)

· Solve the LP relaxation to get K· Use Gomory‘s cuts to get K ′, P ⊂ K ′ ⊂ K· Repeat the process until you get P

Dalya Gartzman Tel Aviv University

ILP Solvers

Page 93: ILP solvers

Prologue LP Solvers Adding Intergality Constraints Special Cases of ILP Epilogue

Branch-and-Cut Algorithm (Padberg, Rinaldy) -Perliminaries

Gomory’s cut

When solving the LP relaxations, we getctx∗ = z where x∗ has fractional coordinates.We wish to separate x∗ from K .

We can do this by adding bctx∗c ≤ bzc to thelist of constraints.

Cutting Planes (Gomory, Chvatal)

· Solve the LP relaxation to get K· Use Gomory‘s cuts to get K ′, P ⊂ K ′ ⊂ K· Repeat the process until you get P

Dalya Gartzman Tel Aviv University

ILP Solvers

Page 94: ILP solvers

Prologue LP Solvers Adding Intergality Constraints Special Cases of ILP Epilogue

Branch-and-Cut Algorithm (Padberg, Rinaldy) -Perliminaries

Gomory’s cut

When solving the LP relaxations, we getctx∗ = z where x∗ has fractional coordinates.We wish to separate x∗ from K .We can do this by adding bctx∗c ≤ bzc to thelist of constraints.

Cutting Planes (Gomory, Chvatal)

· Solve the LP relaxation to get K· Use Gomory‘s cuts to get K ′, P ⊂ K ′ ⊂ K· Repeat the process until you get P

Dalya Gartzman Tel Aviv University

ILP Solvers

Page 95: ILP solvers

Prologue LP Solvers Adding Intergality Constraints Special Cases of ILP Epilogue

Branch-and-Cut Algorithm (Padberg, Rinaldy) -Perliminaries

Gomory’s cut

When solving the LP relaxations, we getctx∗ = z where x∗ has fractional coordinates.We wish to separate x∗ from K .We can do this by adding bctx∗c ≤ bzc to thelist of constraints.

Cutting Planes (Gomory, Chvatal)

· Solve the LP relaxation to get K· Use Gomory‘s cuts to get K ′, P ⊂ K ′ ⊂ K· Repeat the process until you get P

Dalya Gartzman Tel Aviv University

ILP Solvers

Page 96: ILP solvers

Prologue LP Solvers Adding Intergality Constraints Special Cases of ILP Epilogue

Branch-and-Cut Algorithm (Padberg, Rinaldy) -Perliminaries

Gomory’s cut

When solving the LP relaxations, we getctx∗ = z where x∗ has fractional coordinates.We wish to separate x∗ from K .We can do this by adding bctx∗c ≤ bzc to thelist of constraints.

Cutting Planes (Gomory, Chvatal)

· Solve the LP relaxation to get K

· Use Gomory‘s cuts to get K ′, P ⊂ K ′ ⊂ K· Repeat the process until you get P

Dalya Gartzman Tel Aviv University

ILP Solvers

Page 97: ILP solvers

Prologue LP Solvers Adding Intergality Constraints Special Cases of ILP Epilogue

Branch-and-Cut Algorithm (Padberg, Rinaldy) -Perliminaries

Gomory’s cut

When solving the LP relaxations, we getctx∗ = z where x∗ has fractional coordinates.We wish to separate x∗ from K .We can do this by adding bctx∗c ≤ bzc to thelist of constraints.

Cutting Planes (Gomory, Chvatal)

· Solve the LP relaxation to get K· Use Gomory‘s cuts to get K ′, P ⊂ K ′ ⊂ K

· Repeat the process until you get P

Dalya Gartzman Tel Aviv University

ILP Solvers

Page 98: ILP solvers

Prologue LP Solvers Adding Intergality Constraints Special Cases of ILP Epilogue

Branch-and-Cut Algorithm (Padberg, Rinaldy) -Perliminaries

Gomory’s cut

When solving the LP relaxations, we getctx∗ = z where x∗ has fractional coordinates.We wish to separate x∗ from K .We can do this by adding bctx∗c ≤ bzc to thelist of constraints.

Cutting Planes (Gomory, Chvatal)

· Solve the LP relaxation to get K· Use Gomory‘s cuts to get K ′, P ⊂ K ′ ⊂ K· Repeat the process until you get P

Dalya Gartzman Tel Aviv University

ILP Solvers

Page 99: ILP solvers

Prologue LP Solvers Adding Intergality Constraints Special Cases of ILP Epilogue

Branch-and-Cut Algorithm (Padberg, Rinaldy)

Problem Each application of thecuts is polynomial, but thenumber of iterations isexponential.

Solution Integrate withbranch-and-bound to getbranch-and-cut.

In addition, apply all cutspossible at every iteration.

Performance Tenfolds faster than anyother MILP algorithmsavailable at the time.

Dalya Gartzman Tel Aviv University

ILP Solvers

Page 100: ILP solvers

Prologue LP Solvers Adding Intergality Constraints Special Cases of ILP Epilogue

Branch-and-Cut Algorithm (Padberg, Rinaldy)

Problem Each application of thecuts is polynomial, but thenumber of iterations isexponential.

Solution Integrate withbranch-and-bound to getbranch-and-cut.

In addition, apply all cutspossible at every iteration.

Performance Tenfolds faster than anyother MILP algorithmsavailable at the time.

Dalya Gartzman Tel Aviv University

ILP Solvers

Page 101: ILP solvers

Prologue LP Solvers Adding Intergality Constraints Special Cases of ILP Epilogue

Branch-and-Cut Algorithm (Padberg, Rinaldy)

Problem Each application of thecuts is polynomial, but thenumber of iterations isexponential.

Solution Integrate withbranch-and-bound to getbranch-and-cut.

In addition, apply all cutspossible at every iteration.

Performance Tenfolds faster than anyother MILP algorithmsavailable at the time.

Dalya Gartzman Tel Aviv University

ILP Solvers

Page 102: ILP solvers

Prologue LP Solvers Adding Intergality Constraints Special Cases of ILP Epilogue

Branch-and-Cut Algorithm (Padberg, Rinaldy)

Problem Each application of thecuts is polynomial, but thenumber of iterations isexponential.

Solution Integrate withbranch-and-bound to getbranch-and-cut.In addition, apply all cutspossible at every iteration.

Performance Tenfolds faster than anyother MILP algorithmsavailable at the time.

Dalya Gartzman Tel Aviv University

ILP Solvers

Page 103: ILP solvers

Prologue LP Solvers Adding Intergality Constraints Special Cases of ILP Epilogue

Branch-and-Cut Algorithm (Padberg, Rinaldy)

Problem Each application of thecuts is polynomial, but thenumber of iterations isexponential.

Solution Integrate withbranch-and-bound to getbranch-and-cut.In addition, apply all cutspossible at every iteration.

Performance Tenfolds faster than anyother MILP algorithmsavailable at the time.

Dalya Gartzman Tel Aviv University

ILP Solvers

Page 104: ILP solvers

Prologue LP Solvers Adding Intergality Constraints Special Cases of ILP Epilogue

Additional Interesting ILP Solvers

Lift and Project (Balas, Ceria, Cornuejols 1993)

Lifting the problem to a higher dimension,solve a simpler problem there, and project toget a better optimal solution.

Column generation (Appelgren 1969)Solve the LP relaxation for only some of thevariables/inequalities.Find a variable/inequality that violates thesolution, and resolve.

Dalya Gartzman Tel Aviv University

ILP Solvers

Page 105: ILP solvers

Prologue LP Solvers Adding Intergality Constraints Special Cases of ILP Epilogue

Additional Interesting ILP Solvers

Lift and Project (Balas, Ceria, Cornuejols 1993)

Lifting the problem to a higher dimension,solve a simpler problem there, and project toget a better optimal solution.

Column generation (Appelgren 1969)Solve the LP relaxation for only some of thevariables/inequalities.Find a variable/inequality that violates thesolution, and resolve.

Dalya Gartzman Tel Aviv University

ILP Solvers

Page 106: ILP solvers

Prologue LP Solvers Adding Intergality Constraints Special Cases of ILP Epilogue

Additional Interesting ILP Solvers

Lift and Project (Balas, Ceria, Cornuejols 1993)

Lifting the problem to a higher dimension,solve a simpler problem there, and project toget a better optimal solution.

Column generation (Appelgren 1969)Solve the LP relaxation for only some of thevariables/inequalities.Find a variable/inequality that violates thesolution, and resolve.

Dalya Gartzman Tel Aviv University

ILP Solvers

Page 107: ILP solvers

Prologue LP Solvers Adding Intergality Constraints Special Cases of ILP Epilogue

Additional Interesting ILP Solvers

Lift and Project (Balas, Ceria, Cornuejols 1993)

Lifting the problem to a higher dimension,solve a simpler problem there, and project toget a better optimal solution.

Column generation (Appelgren 1969)

Solve the LP relaxation for only some of thevariables/inequalities.Find a variable/inequality that violates thesolution, and resolve.

Dalya Gartzman Tel Aviv University

ILP Solvers

Page 108: ILP solvers

Prologue LP Solvers Adding Intergality Constraints Special Cases of ILP Epilogue

Additional Interesting ILP Solvers

Lift and Project (Balas, Ceria, Cornuejols 1993)

Lifting the problem to a higher dimension,solve a simpler problem there, and project toget a better optimal solution.

Column generation (Appelgren 1969)Solve the LP relaxation for only some of thevariables/inequalities.

Find a variable/inequality that violates thesolution, and resolve.

Dalya Gartzman Tel Aviv University

ILP Solvers

Page 109: ILP solvers

Prologue LP Solvers Adding Intergality Constraints Special Cases of ILP Epilogue

Additional Interesting ILP Solvers

Lift and Project (Balas, Ceria, Cornuejols 1993)

Lifting the problem to a higher dimension,solve a simpler problem there, and project toget a better optimal solution.

Column generation (Appelgren 1969)Solve the LP relaxation for only some of thevariables/inequalities.Find a variable/inequality that violates thesolution, and resolve.

Dalya Gartzman Tel Aviv University

ILP Solvers

Page 110: ILP solvers

Prologue LP Solvers Adding Intergality Constraints Special Cases of ILP Epilogue

A Sales Man Traveling to 318 Cities

In 1980, Crowder and Padberg showed a cleverway of using the branch-and-cut algorithm forsolving relatively large TSP.

How large?the size of their largest optimized net was 318.Its exponent is ∼ 1googol = 10100!And how was this done?The algorithms starts with a simple Hamiltonian cycle, and everycut corresponds to a ”shortcut”.

I Analogous logic can be applied for some special cases of Max.Indep. Set Problem.Here every cut will correspond to an odd cycle.

I Having ILP ∈ NP tells us that we can‘t do this for everyproblem, and every n, but for some we can try...

Dalya Gartzman Tel Aviv University

ILP Solvers

Page 111: ILP solvers

Prologue LP Solvers Adding Intergality Constraints Special Cases of ILP Epilogue

A Sales Man Traveling to 318 Cities

In 1980, Crowder and Padberg showed a cleverway of using the branch-and-cut algorithm forsolving relatively large TSP.

How large?the size of their largest optimized net was 318.Its exponent is ∼ 1googol = 10100!And how was this done?The algorithms starts with a simple Hamiltonian cycle, and everycut corresponds to a ”shortcut”.

I Analogous logic can be applied for some special cases of Max.Indep. Set Problem.Here every cut will correspond to an odd cycle.

I Having ILP ∈ NP tells us that we can‘t do this for everyproblem, and every n, but for some we can try...

Dalya Gartzman Tel Aviv University

ILP Solvers

Page 112: ILP solvers

Prologue LP Solvers Adding Intergality Constraints Special Cases of ILP Epilogue

A Sales Man Traveling to 318 Cities

In 1980, Crowder and Padberg showed a cleverway of using the branch-and-cut algorithm forsolving relatively large TSP.

How large?the size of their largest optimized net was 318.Its exponent is ∼ 1googol = 10100!

And how was this done?The algorithms starts with a simple Hamiltonian cycle, and everycut corresponds to a ”shortcut”.

I Analogous logic can be applied for some special cases of Max.Indep. Set Problem.Here every cut will correspond to an odd cycle.

I Having ILP ∈ NP tells us that we can‘t do this for everyproblem, and every n, but for some we can try...

Dalya Gartzman Tel Aviv University

ILP Solvers

Page 113: ILP solvers

Prologue LP Solvers Adding Intergality Constraints Special Cases of ILP Epilogue

A Sales Man Traveling to 318 Cities

In 1980, Crowder and Padberg showed a cleverway of using the branch-and-cut algorithm forsolving relatively large TSP.

How large?the size of their largest optimized net was 318.Its exponent is ∼ 1googol = 10100!And how was this done?

The algorithms starts with a simple Hamiltonian cycle, and everycut corresponds to a ”shortcut”.

I Analogous logic can be applied for some special cases of Max.Indep. Set Problem.Here every cut will correspond to an odd cycle.

I Having ILP ∈ NP tells us that we can‘t do this for everyproblem, and every n, but for some we can try...

Dalya Gartzman Tel Aviv University

ILP Solvers

Page 114: ILP solvers

Prologue LP Solvers Adding Intergality Constraints Special Cases of ILP Epilogue

A Sales Man Traveling to 318 Cities

In 1980, Crowder and Padberg showed a cleverway of using the branch-and-cut algorithm forsolving relatively large TSP.

How large?the size of their largest optimized net was 318.Its exponent is ∼ 1googol = 10100!And how was this done?The algorithms starts with a simple Hamiltonian cycle, and everycut corresponds to a ”shortcut”.

I Analogous logic can be applied for some special cases of Max.Indep. Set Problem.Here every cut will correspond to an odd cycle.

I Having ILP ∈ NP tells us that we can‘t do this for everyproblem, and every n, but for some we can try...

Dalya Gartzman Tel Aviv University

ILP Solvers

Page 115: ILP solvers

Prologue LP Solvers Adding Intergality Constraints Special Cases of ILP Epilogue

A Sales Man Traveling to 318 Cities

In 1980, Crowder and Padberg showed a cleverway of using the branch-and-cut algorithm forsolving relatively large TSP.

How large?the size of their largest optimized net was 318.Its exponent is ∼ 1googol = 10100!And how was this done?The algorithms starts with a simple Hamiltonian cycle, and everycut corresponds to a ”shortcut”.

I Analogous logic can be applied for some special cases of Max.Indep. Set Problem.Here every cut will correspond to an odd cycle.

I Having ILP ∈ NP tells us that we can‘t do this for everyproblem, and every n, but for some we can try...

Dalya Gartzman Tel Aviv University

ILP Solvers

Page 116: ILP solvers

Prologue LP Solvers Adding Intergality Constraints Special Cases of ILP Epilogue

A Sales Man Traveling to 318 Cities

In 1980, Crowder and Padberg showed a cleverway of using the branch-and-cut algorithm forsolving relatively large TSP.

How large?the size of their largest optimized net was 318.Its exponent is ∼ 1googol = 10100!And how was this done?The algorithms starts with a simple Hamiltonian cycle, and everycut corresponds to a ”shortcut”.

I Analogous logic can be applied for some special cases of Max.Indep. Set Problem.Here every cut will correspond to an odd cycle.

I Having ILP ∈ NP tells us that we can‘t do this for everyproblem, and every n, but for some we can try...

Dalya Gartzman Tel Aviv University

ILP Solvers

Page 117: ILP solvers

Prologue LP Solvers Adding Intergality Constraints Special Cases of ILP Epilogue

Totally Unimodular Matrices

I A matrix A is totally unimodular if each sub-determinant of Ais 0,1 or -1.

I Necessary condition - every entry of A is 0,1 or -1.I Sufficient condition - some property that is easier to check

(Hofman, Kruskal 1956).I Important property - if b is an integral vector, then{x : Ax ≤ b}, {x : Ax ≤ b, x ≥ 0} are integral polygones.

I Useful corollary - if some form of the ILP yields a totallyunimodular matrix, solve LP in polynomial time to get theinteger optimal solution.

Dalya Gartzman Tel Aviv University

ILP Solvers

Page 118: ILP solvers

Prologue LP Solvers Adding Intergality Constraints Special Cases of ILP Epilogue

Totally Unimodular Matrices

I A matrix A is totally unimodular if each sub-determinant of Ais 0,1 or -1.

I Necessary condition - every entry of A is 0,1 or -1.I Sufficient condition - some property that is easier to check

(Hofman, Kruskal 1956).I Important property - if b is an integral vector, then{x : Ax ≤ b}, {x : Ax ≤ b, x ≥ 0} are integral polygones.

I Useful corollary - if some form of the ILP yields a totallyunimodular matrix, solve LP in polynomial time to get theinteger optimal solution.

Dalya Gartzman Tel Aviv University

ILP Solvers

Page 119: ILP solvers

Prologue LP Solvers Adding Intergality Constraints Special Cases of ILP Epilogue

Totally Unimodular Matrices

I A matrix A is totally unimodular if each sub-determinant of Ais 0,1 or -1.

I Necessary condition - every entry of A is 0,1 or -1.

I Sufficient condition - some property that is easier to check(Hofman, Kruskal 1956).

I Important property - if b is an integral vector, then{x : Ax ≤ b}, {x : Ax ≤ b, x ≥ 0} are integral polygones.

I Useful corollary - if some form of the ILP yields a totallyunimodular matrix, solve LP in polynomial time to get theinteger optimal solution.

Dalya Gartzman Tel Aviv University

ILP Solvers

Page 120: ILP solvers

Prologue LP Solvers Adding Intergality Constraints Special Cases of ILP Epilogue

Totally Unimodular Matrices

I A matrix A is totally unimodular if each sub-determinant of Ais 0,1 or -1.

I Necessary condition - every entry of A is 0,1 or -1.I Sufficient condition - some property that is easier to check

(Hofman, Kruskal 1956).

I Important property - if b is an integral vector, then{x : Ax ≤ b}, {x : Ax ≤ b, x ≥ 0} are integral polygones.

I Useful corollary - if some form of the ILP yields a totallyunimodular matrix, solve LP in polynomial time to get theinteger optimal solution.

Dalya Gartzman Tel Aviv University

ILP Solvers

Page 121: ILP solvers

Prologue LP Solvers Adding Intergality Constraints Special Cases of ILP Epilogue

Totally Unimodular Matrices

I A matrix A is totally unimodular if each sub-determinant of Ais 0,1 or -1.

I Necessary condition - every entry of A is 0,1 or -1.I Sufficient condition - some property that is easier to check

(Hofman, Kruskal 1956).I Important property - if b is an integral vector, then{x : Ax ≤ b}, {x : Ax ≤ b, x ≥ 0} are integral polygones.

I Useful corollary - if some form of the ILP yields a totallyunimodular matrix, solve LP in polynomial time to get theinteger optimal solution.

Dalya Gartzman Tel Aviv University

ILP Solvers

Page 122: ILP solvers

Prologue LP Solvers Adding Intergality Constraints Special Cases of ILP Epilogue

Totally Unimodular Matrices

I A matrix A is totally unimodular if each sub-determinant of Ais 0,1 or -1.

I Necessary condition - every entry of A is 0,1 or -1.I Sufficient condition - some property that is easier to check

(Hofman, Kruskal 1956).I Important property - if b is an integral vector, then{x : Ax ≤ b}, {x : Ax ≤ b, x ≥ 0} are integral polygones.

I Useful corollary - if some form of the ILP yields a totallyunimodular matrix, solve LP in polynomial time to get theinteger optimal solution.

Dalya Gartzman Tel Aviv University

ILP Solvers

Page 123: ILP solvers

Prologue LP Solvers Adding Intergality Constraints Special Cases of ILP Epilogue

Totally Unimodular Matrices

I A matrix A is totally unimodular if each sub-determinant of Ais 0,1 or -1.

I Necessary condition - every entry of A is 0,1 or -1.I Sufficient condition - some property that is easier to check

(Hofman, Kruskal 1956).I Important property - if b is an integral vector, then{x : Ax ≤ b}, {x : Ax ≤ b, x ≥ 0} are integral polygones.

I Useful corollary - if some form of the ILP yields a totallyunimodular matrix, solve LP in polynomial time to get theinteger optimal solution.

Dalya Gartzman Tel Aviv University

ILP Solvers

Page 124: ILP solvers

Prologue LP Solvers Adding Intergality Constraints Special Cases of ILP Epilogue

Totally Unimodular Matrices - A Nice Application

I The incidence matrix of a directed graph is TU.

I Max − Flow can be stated as{

max . stxs.t. c≤x≤d ,Ax=0

}where c , d

indicate the capacities, s is the indicator for edges leaving thesource, and A is the incidence matrix.

⇒ If c, d are integral, then max − flow is integral, and can befound in polynomial time.

I Since the dual is the Min − Cutproblem, we also get themax − flow −min − cut theorem.

Dalya Gartzman Tel Aviv University

ILP Solvers

Page 125: ILP solvers

Prologue LP Solvers Adding Intergality Constraints Special Cases of ILP Epilogue

Totally Unimodular Matrices - A Nice Application

I The incidence matrix of a directed graph is TU.

I Max − Flow can be stated as{

max . stxs.t. c≤x≤d ,Ax=0

}where c , d

indicate the capacities, s is the indicator for edges leaving thesource, and A is the incidence matrix.

⇒ If c, d are integral, then max − flow is integral, and can befound in polynomial time.

I Since the dual is the Min − Cutproblem, we also get themax − flow −min − cut theorem.

Dalya Gartzman Tel Aviv University

ILP Solvers

Page 126: ILP solvers

Prologue LP Solvers Adding Intergality Constraints Special Cases of ILP Epilogue

Totally Unimodular Matrices - A Nice Application

I The incidence matrix of a directed graph is TU.

I Max − Flow can be stated as{

max . stxs.t. c≤x≤d ,Ax=0

}where c , d

indicate the capacities, s is the indicator for edges leaving thesource, and A is the incidence matrix.

⇒ If c, d are integral, then max − flow is integral, and can befound in polynomial time.

I Since the dual is the Min − Cutproblem, we also get themax − flow −min − cut theorem.

Dalya Gartzman Tel Aviv University

ILP Solvers

Page 127: ILP solvers

Prologue LP Solvers Adding Intergality Constraints Special Cases of ILP Epilogue

Totally Unimodular Matrices - A Nice Application

I The incidence matrix of a directed graph is TU.

I Max − Flow can be stated as{

max . stxs.t. c≤x≤d ,Ax=0

}where c , d

indicate the capacities, s is the indicator for edges leaving thesource, and A is the incidence matrix.

⇒ If c, d are integral, then max − flow is integral, and can befound in polynomial time.

I Since the dual is the Min − Cutproblem, we also get themax − flow −min − cut theorem.

Dalya Gartzman Tel Aviv University

ILP Solvers

Page 128: ILP solvers

Prologue LP Solvers Adding Intergality Constraints Special Cases of ILP Epilogue

Totally Unimodular Matrices - A Nice Application

I The incidence matrix of a directed graph is TU.

I Max − Flow can be stated as{

max . stxs.t. c≤x≤d ,Ax=0

}where c , d

indicate the capacities, s is the indicator for edges leaving thesource, and A is the incidence matrix.

⇒ If c, d are integral, then max − flow is integral, and can befound in polynomial time.

I Since the dual is the Min − Cutproblem, we also get themax − flow −min − cut theorem.

Dalya Gartzman Tel Aviv University

ILP Solvers

Page 129: ILP solvers

Prologue LP Solvers Adding Intergality Constraints Special Cases of ILP Epilogue

ILP Solver for a Fixed Number of Variables

Lenstra showed, in 1983, that given any fixed number of variables,an ILP can be solved in polynomial time (in the number ofconstraints).

If the variebles are restricted to the set{0, 1}, this algorithm is equivalent tocomplete enumeration. For other uses,it suggests a new approach forexamining the feasible region, usingmethods from geometry of numbers.

In any case, this algorithm‘s running time is exponential in thenumber of variables, thus not recommended for big problems.

Dalya Gartzman Tel Aviv University

ILP Solvers

Page 130: ILP solvers

Prologue LP Solvers Adding Intergality Constraints Special Cases of ILP Epilogue

ILP Solver for a Fixed Number of Variables

Lenstra showed, in 1983, that given any fixed number of variables,an ILP can be solved in polynomial time (in the number ofconstraints).

If the variebles are restricted to the set{0, 1}, this algorithm is equivalent tocomplete enumeration. For other uses,it suggests a new approach forexamining the feasible region, usingmethods from geometry of numbers.

In any case, this algorithm‘s running time is exponential in thenumber of variables, thus not recommended for big problems.

Dalya Gartzman Tel Aviv University

ILP Solvers

Page 131: ILP solvers

Prologue LP Solvers Adding Intergality Constraints Special Cases of ILP Epilogue

ILP Solver for a Fixed Number of Variables

Lenstra showed, in 1983, that given any fixed number of variables,an ILP can be solved in polynomial time (in the number ofconstraints).

If the variebles are restricted to the set{0, 1}, this algorithm is equivalent tocomplete enumeration. For other uses,it suggests a new approach forexamining the feasible region, usingmethods from geometry of numbers.

In any case, this algorithm‘s running time is exponential in thenumber of variables, thus not recommended for big problems.

Dalya Gartzman Tel Aviv University

ILP Solvers

Page 132: ILP solvers

Prologue LP Solvers Adding Intergality Constraints Special Cases of ILP Epilogue

ILP Solver for a Fixed Number of Variables

Lenstra showed, in 1983, that given any fixed number of variables,an ILP can be solved in polynomial time (in the number ofconstraints).

If the variebles are restricted to the set{0, 1}, this algorithm is equivalent tocomplete enumeration. For other uses,it suggests a new approach forexamining the feasible region, usingmethods from geometry of numbers.

In any case, this algorithm‘s running time is exponential in thenumber of variables, thus not recommended for big problems.

Dalya Gartzman Tel Aviv University

ILP Solvers

Page 133: ILP solvers

Prologue LP Solvers Adding Intergality Constraints Special Cases of ILP Epilogue

Where do we go from here?

I Estimating large problems

I Find more ”special cases” for which the ILP can be solvedwith better performance

I Use multi-disciplinary methods to improve existing algorithms

I Solving LP with NLP methods

Dalya Gartzman Tel Aviv University

ILP Solvers

Page 134: ILP solvers

Prologue LP Solvers Adding Intergality Constraints Special Cases of ILP Epilogue

Where do we go from here?

I Estimating large problems

I Find more ”special cases” for which the ILP can be solvedwith better performance

I Use multi-disciplinary methods to improve existing algorithms

I Solving LP with NLP methods

Dalya Gartzman Tel Aviv University

ILP Solvers

Page 135: ILP solvers

Prologue LP Solvers Adding Intergality Constraints Special Cases of ILP Epilogue

Where do we go from here?

I Estimating large problems

I Find more ”special cases” for which the ILP can be solvedwith better performance

I Use multi-disciplinary methods to improve existing algorithms

I Solving LP with NLP methods

Dalya Gartzman Tel Aviv University

ILP Solvers

Page 136: ILP solvers

Prologue LP Solvers Adding Intergality Constraints Special Cases of ILP Epilogue

Where do we go from here?

I Estimating large problems

I Find more ”special cases” for which the ILP can be solvedwith better performance

I Use multi-disciplinary methods to improve existing algorithms

I Solving LP with NLP methods

Dalya Gartzman Tel Aviv University

ILP Solvers

Page 137: ILP solvers

Prologue LP Solvers Adding Intergality Constraints Special Cases of ILP Epilogue

Where do we go from here?

I Estimating large problems

I Find more ”special cases” for which the ILP can be solvedwith better performance

I Use multi-disciplinary methods to improve existing algorithms

I Solving LP with NLP methods

Dalya Gartzman Tel Aviv University

ILP Solvers

Page 138: ILP solvers

Prologue LP Solvers Adding Intergality Constraints Special Cases of ILP Epilogue

Where do we go from here?

I Estimating large problems

I Find more ”special cases” for which the ILP can be solvedwith better performance

I Use multi-disciplinary methods to improve existing algorithms

I Solving LP with NLP methods

Dalya Gartzman Tel Aviv University

ILP Solvers

Page 139: ILP solvers

Prologue LP Solvers Adding Intergality Constraints Special Cases of ILP Epilogue

Bibliography

I Theory of linear and integer programing - A. Schrijver

I Advances in linear and integer programing - J. E. Beasley

I Integer and linear Programing - S. Zionts

I Integer programing with a Fixed Number of Variables - H. W.Lenstra Jr.

I A lift-and-project cutting plane algorithm for mixed 0-1programs - E. Balas et al.

I Maximum stable set problem: a branch-and-cut solver - S.Rebennack

I Linear Programming (undergrad. course, UCLA) - L.Vandenberghe

WikipediaDalya Gartzman Tel Aviv University

ILP Solvers

Page 140: ILP solvers

Prologue LP Solvers Adding Intergality Constraints Special Cases of ILP Epilogue

Thank you for listening!

[email protected]

These slides are available athttp://www.slideshare.net/DalyaGartzman

Dalya Gartzman Tel Aviv University

ILP Solvers