CIVIL-557
Decision Aid Methodologies
In Transportation
Transport and Mobility Laboratory TRANSP-OR
Γcole Polytechnique FΓ©dΓ©rale de Lausanne EPFL
Virginie Lurkin
Lecture 4:
Integer programming I: Branch and Bound
On the previous lectureβ¦
Container Storage Inside a Container Terminal
o Two-phase simplex method
o Duality theory
Suppose 2 new containers are expected to arrive for storage in the next planning
period of a terminal. Suppose there are only 2 blocks in the terminal, each with 20
storage spaces. For the moment, there are 6 containers in block 1 and 12 containers
in block 2.
π΅ = π
B = π
A = ππ
ππ = ππ
ππ = π
π =ππ+π+π
πΓππ= π. πππ
πππππππ§π π’1+ + π’1
β + π’2+ + π’2
β
π π’πππππ‘ π‘ππ₯1βπ’1
+ + π’1β = 0.5
π₯2βπ’2+ + π’2
β = 1.5π₯1+π₯2 = 2π₯1, π₯2, π’1
+, π’1β, π’2
+, π’2β β₯ 0
What if the situation is differentβ¦
Linear programming
ππ π
Optimal solution: (π. π, π. π, 0, 0, 0, 0)
π₯1 and π₯2 represent the number of containers to be assigned, respectively, to
block 1 and 2
π₯1 and π₯2, in reality, should be integer
Linear (mixed) integer programming
Linear integer programming
π΄π₯ β€ π
π₯ β₯ 0,
subject to
where π΄ β βπΓπ, b β βπ, c β βπ
πππ πππ₯π₯ β
π
Pure integer programming (IP):
A mixed-integer programming (MIP) problem is one where only some of
the decision variables are constrained to be integer values
Linear (mixed) integer programming
How to solve integer programming problem?
Integer solutions
Objective function
Optimum of LP
relaxation
Nearest integer
(not feasible)
IP optimum
Intuitive idea: remove the integer constraints from the IP formulation (LP relaxation)
o Solve the LP relaxation and round to the nearest integer
Not the best idea!
Linear (mixed) integer programming
How to solve integer programming problem?
There is no optimality conditions for integer programming problem
Explicit enumeration is normally impossible due to the exponentially increasing
number of potential solutions
Exact methods can be used to find the optimal solution:
o Two main methods:
o Computing a true optimal solution might be very costly
Heuristic methods can be used to find a good solution:
o Does not guarantee optimality
Branch and Bound (B&B) and Cutting planes
Heuristics
Heuristics
Algorithms and complexity measures
An algorithm is a finite set of instructions that when executed on an input can
produce an output after finitely many steps.
The input encodes an instance of the problem that the algorithm is supposed to
solve, and the output encodes a solution for the given problem instance.
The computation of an algorithm on an input is the sequence of steps it performs.
Algorithms do not always terminate; they may perform an infinite computation for
some, or any, input.
Heuristics
Algorithms and complexity measures
There are problems that can be easily and precisely defined in mathematical terms,
yet that provably cannot be solved by any algorithm (undecidable/ uncomputable
problems.
In practice, we cannot expect to be able to solve real world problem instances of
arbitrary size to optimality.
Depending on the size of an instance or depending on the available CPU time we will
often have to be satisfied with computing approximate solutions.
Heuristics
Algorithms and complexity measures
The term heuristic is used for algorithms which find solutions among all possible
ones ,but they do not guarantee that the best will be found.
A good heuristic algorithm should fulfil the following properties:
o A solution can be obtained with reasonable computational effort.
o The solution should be near optimal (with high probability).
o The likelihood for obtaining a bad solution (far from optimal) should be low.
How can we evaluate the performance of an heuristic algorithm ?
1. Rearrange blocks so that ππ increases with π
2. Determine π₯π in increasing order of π using:
π₯1 = min{π,max 0, π΄ Γ F β π1 }
π₯π = mππ{max 0, π΄ Γ F β ππ , π βπ=1
πβ1
π₯π} βπ β₯ 2
Heuristics
A concrete example for our container storage problem
Branch and Bound algorithm
Branch and Bound algorithm
General idea
Implicit enumeration of the feasible set, by ruling out large sets of feasible
points
How ?
o By partitioning the feasible set into smaller subsets
o By using bounds to identify subsets that are guaranteed not to contain an
optimal solution
πΊπΊπ
πΊπ
πΊππ πΊππ
πΊππ πΊππ
Theorem 1:
ππis an optimization problem (minimization)
π is the feasible set of the optimization problem π
π is partitioned into πΎ subsets: π1 βͺ β¦ βͺ Sm βͺ Sm+1 βͺ β¦ βͺ SK
For each π = 1,2, β¦ , πΎ, let πππ‘ π₯πβ be an optimal solution of the
optimization problem ππ(minimization). Let π be such that:
π π₯πβ β€ π π₯π
β , π = 1,β¦ , πΎ
Then π₯πβ is an optimal solution of the optimization problem π
πΊππ πΊππ
πΊππ πΊππ
Branch and Bound algorithm
Branch and Bound algorithm
Partitioning the original problem into smaller subproblems
For each feasible subproblem, we will calculate a lower bound (best possible value)
Lower bounds are obtained by solving relaxations of the original subproblems
Calculate lower bounds is much easier (simplex algorithm can be used)
Partitioning:
Partitioning the feasible set into smaller subsets
Using bounds:
Any feasible solution provides an upper bound on the optimal solution (minimization)
A feasible solution is a solution that satisfies all the constraints
πΊππΊπ
π
π1 π2
π11 π12
Branch and Bound algorithm
πΏπ΅π
πΏπ΅π1 πΏπ΅π2
Some candidates are excluded based on the value of the lower bound
Using bounds:
ππ΅
πΊ If πΏπ΅π2 β₯ ππ΅If πΏπ΅π1 < ππ΅
Branching refers to exploring the subtree of an active node
Bounding refers to estimating a lower bound at an active node
For each π = 1,2, β¦ ,π, let πππ‘ π₯πβ be an optimal solution of ππ.
For each π = π + 1,β¦ , πΎ, let π(ππ) be a lower bound on ππ:
π ππ β€ π π₯π βπ β ππ
Let π be such that:
π π₯πβ β€ π π₯π
β , π = 1,β¦ ,π
and
π π₯πβ β€ π ππ , π = π + 1,β¦ , πΎ
Then π₯πβ is an optimal solution of the optimization problem π
Branch and Bound algorithm
Theorem 2:
Branch and Bound algorithm
Branching:
π
π1 π2
πΏπ΅π
ππ΅
Select a variable π₯π that has a fractional value:
π < π₯π < π + 1, with π integer
Create two new LP-subproblems:
1. Previous LP-subproblem + constraint π₯π β€ π (π = π₯π )
2. Previous LP-subproblem + constraint π₯π β₯ π + 1 (π + 1 = π₯π )
How to branch?
Algorithm
Solve the relaxation π ππ and denote π₯π β the optimal value
π ππ β π(π₯π β )
π’ π > π ππ ?
π₯π β integer?
no
π β π0π’ π β +β
π’ π , Upper Bound for π
no
Discard ππ from πyes
π ππ ,Lower Bound for ππ
no
yesπ π₯π
β < π’(π)?
π empty?
no
π₯β = (π₯π β )
π’ π β π π₯π β
yes
end
πβ, πβ= π π·
yes
Select π such that (π₯π β )π is not integer
Create πππ by adding the constraint π₯π β€ (π₯π
β )π to ππCreate ππ
π by adding the constraint π₯π β₯ (π₯π β )π to ππ
π β π βͺ πππ , ππ
π \Pπ
Select a problem ππ in π. ππ infeasible?
yes
no
Branch and Bound algorithm
Exploration of the search tree:
π
π1 π2
π11 π12
πΏπ΅π
πΏπ΅π1 πΏπ΅π2
ππ΅
If πΏπ΅π2 β₯ ππ΅If πΏπ΅π1 < ππ΅
π122 π123
π1232π1231
π121
Depth-first search (DFS): explores as far as possible along each branch before backtracking
Branch and Bound algorithm
Exploration of the search tree:
π
π1 π2
π11 π12
πΏπ΅π
πΏπ΅π1 πΏπ΅π2
ππ΅
If πΏπ΅π2 β₯ ππ΅If πΏπ΅π1 < ππ΅
π122 π123
Breadth-first search (BFS): explores nodes level by level
π121
Depending on the problem at hand, either DFS or BFS could be advantageous
Branch and Bound algorithm
Back to our case:
Branch and Bound algorithm
πππππππ§π π’1+ + π’1
β + π’2+ + π’2
β
π π’πππππ‘ π‘ππ₯1βπ’1
+ + π’1β = 0.5
π₯2βπ’2+ + π’2
β = 1.5π₯1+π₯2 = 2π₯1, π₯2, π’1
+, π’1β, π’2
+, π’2β β₯ 0
π₯π β = 0.5, 1.5, 0, 0, 0, 0 and π π₯π
β = 0
π0
π’ π = +β
π0ππ0
π
π₯1 β€ 0 π₯1 β₯ 1 π’ π > π π0
π π0 = 0
π₯1, π₯2 are not integers
Select π₯1 for branching
π(π0) = 0
πΉ(π·π)
Back to our case:
Branch and Bound algorithm
π₯π β = 0, 2, 0, 0.5, 0.5, 0 and π π₯π
β = 1
π0
π’ π = +β
π0ππ0
π
π₯1 β€ 0 π₯1 β₯ 1 π’ π > π π0
π π0π = 1
π₯1, π₯2 are integers π’ π = 1
Discard π0π
π(π0) = 0
πππππππ§π π’1+ + π’1
β + π’2+ + π’2
β
π π’πππππ‘ π‘ππ₯1βπ’1
+ + π’1β = 0.5
π₯2βπ’2+ + π’2
β = 1.5π₯1+π₯2 = 2π₯1β€ 0π₯1, π₯2, π’1
+, π’1β, π’2
+, π’2β β₯ 0
πΉ(π·ππ )
π(π0π) = 1
π’ π = 1
Back to our case:
Branch and Bound algorithm
π₯π β = 1, 1, 0.5, 0, 0, 0.5 and π π₯π
β = 1
π0
π’ π = +β
π0ππ0
π
π₯1 β€ 0 π₯1 β₯ 1 π’ π > π π0
π π0π = 1
π₯1, π₯2 are integers
Discard π0π
π(π0) = 0
πππππππ§π π’1+ + π’1
β + π’2+ + π’2
β
π π’πππππ‘ π‘ππ₯1βπ’1
+ + π’1β = 0.5
π₯2βπ’2+ + π’2
β = 1.5π₯1+π₯2 = 2π₯1β₯ 1π₯1, π₯2, π’1
+, π’1β, π’2
+, π’2β β₯ 0
π(π0π) = 1
π’ π = 1
π π₯π β = 1
end
π₯π β = 0, 2, 0, 0.5, 0.5, 0 or π₯π
β = 1, 1, 0.5, 0, 0, 0.5
πΉ(π·ππ)
Linear (mixed) integer programming
Some integer programming tricks
Integer programming tricks
Example 1
o The value of variable π₯ must be either zero or between particular positive bounds
Suppose that a supplier of some item requires that if an item is ordered, then its batch
size must be between a particular minimum and maximum value
Discontinuous values
o π₯ is the number of ordered units
o Algebraic notation: π₯ = 0 β£ π β€ π₯ β€ π’
Integer programming tricks
We introduce an indicator variable
An indicator variable is a binary variable (0/1) that indicates a certain state in a model
A set of constraints is used to create the desired properties:
π₯ β€ π’π¦π₯ β₯ ππ¦π¦ β {0,1}
Discontinuous values
π¦ = α01
for π₯ = 0
for π β€ π₯ β€ π’
Integer programming tricks
Example 2
Fixed costs
Suppose that a manager must decide which of π warehouses to use for meeting the
demands of π customers for a good. The decisions to be made are which warehouses
to operate and how much to ship from any warehouse to any customer
o There is a fixed operating cost associated to each warehouse
o Decisions must be made about tradeoffs between transportation costs and costs
for operating distribution centers
o Very common issue in modeling distribution systems
Integer programming tricks
We introduce an indicator variable
Data:
Fixed costs
π¦π = α10
if warehouse π is opened
if warehouse π is not opened
π₯ππ = amount to be sent from warehouse π to customer π
Other decisions variables:
ππ = fixed operating cost for warehouse π
ππ = per-unit operating cost at warehouse π
πππ = transportation cost for shipping from warehouse π to customer π
Integer programming tricks
Model:
Fixed costs
min
π=1
π
π=1
π
(ππ+πππ)π₯ππ +
π=1
π
πππ¦π
s. t.
π=1
π
π₯ππ = ππ , βπ = 1,2, β¦ , π
π=1
π
π₯ππ β€ π΄π¦π , βπ = 1,2, β¦ ,π π΄ is a big number
π₯ππ β₯ 0, βπ = 1,2, β¦ ,π, βπ = 1,2, β¦ , π
π¦π β 0,1 , βπ = 1,2, β¦ ,π
Integer programming tricks
Example 3
Suppose that a company has to choose among two modes of operation for the
manufacturing process of its products on a single machine
o Very common issue in modeling manufacturing process
o Each mode has its own performance (number of units produced by minute)
Eitherβ¦ or β¦
o A mode of operation might better suit to some products
Integer programming tricks
We introduce an indicator variable
Data:
Eitherβ¦ or β¦
π¦π = α10
if process 1 is chosen for product π
if process 2 is not chosen for product π
π₯π = number of units of product π produced by the company
Other decisions variables:
πππ = per-unit time needed (in minutes) to produce product π using process π
ππ = maximum amount of time the machine can be used if process π is used
Integer programming tricks
Constraints:
Eitherβ¦ or β¦
s. t.
π=1
π
π1ππ₯π β€ π1 if process 1 is chosen for product π, i.e. if π¦π = 1
or
s. t.
π=1
π
π2ππ₯π β€ π2 if process 2 is chosen for product π, i.e. if π¦π = 0
π₯π β₯ 0, βπ = 1,2, β¦ , π
π¦π β {0,1}
s. t.
π=1
π
π1ππ₯π β€ π1 +π(1 β π¦π)
π=1
π
π2ππ₯π β€ π2 +ππ¦π
Integer programming tricks
Constraints:
Ifβ¦ then β¦
π₯π β₯ 0, βπ = 1,2, β¦ , π
π¦π β {0,1}
s. t.
π=1
π
π1ππ₯π β€ π1 +π(1 β π¦π)
π=1
π
π2ππ₯π β€ π2 +ππ¦π
o If process 1 is applied to product 1, it has to be applied to product 2
if π¦1 = 1, then π¦2 = 1
π¦2 β₯ π¦1
Integer programming tricks
Data:
πππ = per-unit cost to produce product π using process π
Elimination of products of variables
Objective function:
min
π=1
π
(π1ππ₯π π¦π + c2jπ₯π(1 β π¦π))
nonlinear
π=1
π
((π1πβπ2π)π₯π π¦π + c2jπ₯π)
min
π=1
π
((π1πβπ2π)π₯π π¦π + c2jπ₯π)
Integer programming tricks
Elimination of products of variables
o In general, a product of two variables can be replaced by one variable,
o A number of new constraints is imposed on the new variable
continuous variables
binary variables
Product of a binary variable and a continuous variable
π§π β€ π’π¦π
π§π β€ π₯π
π§π β₯ π₯π β u(1 β yj)
π§π β₯ 0
0 β€ π₯π β€ π’Upper bound on π₯π
New variable π§π π§π= π₯π Γ π¦π
Integer programming tricks
Product of two binary variables
π₯ππ¦π¦
binary variables binary variables
Product of two continuous variables
New variable π§π π§π= π₯π Γ π¦π
π§π β€ π₯π
π§π β€ π¦π
π§π β₯ π₯π + π¦π β 1
π§π β {0,1}
π₯ππ¦π¦
continuous variables continuous variables
π§π =1
2(π₯π + π¦π)
π€π =1
2(π₯π β π¦π)
New variables π§π and π€π π§π2βπ€π
2 = π₯π Γ π¦π
Quadratic function
Linear (mixed) integer programming
IP and MIP have extremely wide applications in practice
Knapsack problem
Which containers should be loaded in the vessel?
Knapsack problem
π€π: weight if container π
ππ: price of container π
o Data
If container π is selected
Otherwiseπ₯π = α
10
o Decision variables
π¦ππ±
π
πππ₯π
o Objective function
π
π€ππ₯π β€ πΆ
o Constraints
πΆ: capacity of the vessel
Which boxes should be loaded in the container in order to maximize the revenue?
Related problems deal with the loading of containers in trains or aircrafts
Knapsack problems
Facility location problem
Where to locate inland container depots?
Inland container depots: storage area for empty shipping containers situated at
inland points away from sea ports
Facility location problem
Where to locate inland container depots?
Inland container depots: storage area for empty shipping containers situated at
inland points away from sea ports
πΌ: set of π alternative facility locations
J: set of π customer zones
o Data
πππ: per-unit cost for supplying customer zone π by using the facility at π
Facility location problem
ππ: fixed cost of establishing a facility location π
ππ : capacity of facility location π
If a facility is established at location π
Otherwiseπ¦π = α
10
o Decision variables
π₯ππ = Units of customer zone πβs demand satisfied by facility at location π
ππ: total demand for customer zone π
Facility location problem
π¦ππ±
π=1
π
π=1
π
ππππ₯ππ +
π=1
π
πππ¦π
o Objective function
π=1
π
π₯ππ = ππ , βπ = 1,β¦ , π
o Constraints
π=1
π
π₯ππ β€ ππ , βπ = 1,β¦ ,π
π=1
π
π₯ππ β€ (
π=1
π
ππ) π¦π , βπ = 1, β¦ ,π
π₯ππ β₯ 0, βπ = 1,2, β¦ ,π; βπ = 1,2, β¦ , π
π¦π β 0,1 , βπ = 1,2, β¦ ,π
Assignment problem
Berth allocation problem
The berth allocation problem consists of assigning incoming ships to berthing
position
π΅: set of π berths
π: set of π vessels
o Data
βππ: handling time of vessel π at berth π
If a vessel π is allocated to berth π
Otherwiseπ¦ππ = α
10
o Decision variables
Assignment problem
π¦ππ
π=1
π
π=1
π
βπππ¦ππ
o Objective function
o Constraints
π=1
π
π¦ππ = 1, βπ = 1,β¦ , π
π=1
π
π¦ππ = 1, βπ = 1,β¦ ,π
π¦ππ β 0,1 , βπ = 1,2, β¦ ,π
Assignment problem
Summary
Making Branch and Bound work well in practice requires lots of good ideas
Heuristics are often used to find good initial solutions
An idea for speeding up Branch and Bound is to add valid inequalities (next
lecture)
Β« Divide ut imperes Β»
Main references
β’ Murty, K. G. (2015). Intelligent Modeling Essential to Get Good Results: Container Storage Inside a
ContainerTerminal. In Case Studies in Operations Research (pp. 1-15). Springer NewYork.
β’ Bierlaire, M. (2015).Optimization: principles and algorithms. EPFL Press, Lausanne, Switzerland.
Top Related