Linear Algebra
Slide -25: Linear Programming
Fall 2020
Sharif University of Technology
Fall 2020Dr. Hamid Reza Rabiee
Linear Programming
Fall 2020Hamid Reza Rabiee
Titles
• What is Linear Programming?
• Dual Problem
• Simplex Method
• Other Methods
Fall 2020Hamid Reza Rabiee
Titles
• What is Linear Programming?
• Dual Problem
• Simplex Method
• Other Methods
Fall 2020Hamid Reza Rabiee
Linear Programming Definition
• Linear algebra plus 2 new ideas: inequalities and
minimization.
• We have matrix 𝐴, and two vectors: 𝑏 and 𝑐.
• In 𝐴 we have 𝑛 > 𝑚: meaning that number of unknowns are more than equations.
• E.g: 𝐴 = [1 2 31 0 1
], 𝑏 = 1, 2 , c = [1, 2, 3]
• You may notice that A is 𝑛 × 𝑚, b is 1 × 𝑚, c is 1 × 𝑛.
Hamid Reza Rabiee Fall 2020
Linear Programming Definition
• Having A, b and c, our objective in linear programming
is:
• To minimize 𝑐 · 𝑥 subject to the requirements 𝐴𝑥 = 𝑏 and 𝑥 ≥ 0. 𝑥 being the variable 𝑛 × 1 vector.
• E.g: 𝐴 = [1 1 2], 𝑏 = [4], cT =[5 3 8]
• 𝑥 = [
𝑥1𝑥2𝑥3]
• Minimize 5𝑥1 + 3𝑥2 + 8𝑥3 𝑤ℎ𝑖𝑙𝑒 𝐴𝑥 = 𝑏, 𝑥 ≥ 0
Hamid Reza Rabiee Fall 2020
Linear Programming Definition
• In another perspective we first restrict our answer space by
conditions of 𝐴𝑥 = 𝑏 and 𝑥 ≥ 0, then we try to find a point in the restricted space that has the lowest cost.
Hamid Reza Rabiee Fall 2020
Recap: Linear Programming Definition
• So our problem is to look for the lowest cost in our
polygon (triangle in the previous example) which is
created by the constraints.
• The lowest 𝑐. 𝑥 (our goal) can only happen on the vertices of the polygon (the restricted area).
• E.g. in our example, we should only check 𝑐. 𝑥 in three tips of the triangle and pick its minimum and we can be sure
that it is the lowest possible 𝑐. 𝑥 subject to the constraints.
Hamid Reza Rabiee Fall 2020
Titles
• What is Linear Programming?
• Dual Problem
• Simplex Method
• Other Methods
Fall 2020Hamid Reza Rabiee
Dual Problem
• Every linear programming problem, referred to as
a primal problem, can be converted into a dual problem.
• The dual problem provides an upper bound to the optimal
value of the primal problem.
• With the same matrix and vector definitions we have:
• Dual problem: Maximize 𝒃. 𝒚 subject to 𝑨𝑻𝒚 ≤ 𝒄](Primal: minimize 𝑐 · 𝑥 subject to the requirements 𝐴𝑥 = 𝑏and 𝑥 ≥ 0)
Hamid Reza Rabiee Fall 2020
Recap: Dual Problem
• Each linear programming problem has a dual problem in
which we –instead of minimizing the cost- try to maximize
our income 𝑏. 𝑦. • Notice that y and x are do not necessarily have the same
dimensions.
• (Reading page 485 of Strang Textbook for getting a better
intuition is highly recommended).
Hamid Reza Rabiee Fall 2020
Titles
• What is Linear Programming?
• Dual Problem
• Simplex Method
• Other Methods
Fall 2020Hamid Reza Rabiee
Simplex Method
• Bear in mind that we want to see which of the polygon
edges can produce the least cost.
• There are many solutions to this problem, one of the most
used and oldest ones is the Simplex method.
• It aims to solve the problem by constructing a feasible
solution at one edge of the polygon and then walk along a
path on the edges of the polytope to vertices with non-
decreasing values of the objective function until an
optimum is reached for sure.
Hamid Reza Rabiee Fall 2020
Simplex Method
• With having the vertices, you start with one of the vertices
and calculate how much choosing other vertices will
decrease your cost.
• At each corner we have n non-zero and m zero variables.
• The simplex method must decide which component
"enters“ the vector by becoming positive, and which
component "leaves" by becoming zero.
• That exchange is chosen so as to lower the total cost.
Hamid Reza Rabiee Fall 2020
Simplex Method Example
• 𝑀𝑖𝑛𝑖𝑚𝑖𝑧𝑒 𝑡ℎ𝑒 𝑐𝑜𝑠𝑡: 𝑐. 𝑥 = 3𝑥1+ 𝑥2+ 9𝑥3+ 𝑥4 𝑠. 𝑡. 𝑥≥ 0
• 𝑥1+ 2𝑥3+ 𝑥4 = 4• 𝑥2+ 𝑥3− 𝑥4 = 2
• Starting corner = (4, 2, 0, 0), c.x = 14
Hamid Reza Rabiee Fall 2020
Solution: Strang page 487
Titles
• What is Linear Programming?
• Dual Problem
• Simplex Method
• Other Methods
Fall 2020Hamid Reza Rabiee
Other Methods
• There are several other methods that help us find the
optimal values of x. Interior point methods are one of
them.
• Instead of moving along the edges of the polygon, they try
to move inside the polygon and find the value.
• Each of the methods come in handy in various problems in
their efficiency depends heavily on the structure of the
problem.
Hamid Reza Rabiee Fall 2020
Summary
• Linear programming is an optimization problem which is
dealing with linear functions and constraints. It is of a high
importance in management problems.
• Dual Problem: Each LP problem has a Dual form that can
be used to understand and solve the primal problem instead
• Simplex Method: A method to find the lowest-costing
vertex in the space where the conditions apply.
Fall 2020Hamid Reza Rabiee
Special Thanks to Armin Moradi for preparing this presentation.
Hamid Reza Rabiee Fall 2020
Top Related