Linear Algebrace.sharif.edu/.../Lectures/27_Inclass_Linear-Programming.pdfLinear Programming...

Linear Algebra

Slide -25: Linear Programming

Fall 2020

Sharif University of Technology

Fall 2020Dr. Hamid Reza Rabiee

Linear Programming

Fall 2020Hamid Reza Rabiee

Titles

• What is Linear Programming?

• Dual Problem

• Simplex Method

• Other Methods


Linear Programming Definition

• Linear algebra plus 2 new ideas: inequalities and

minimization.

• We have matrix 𝐴, and two vectors: 𝑏 and 𝑐.

• In 𝐴 we have 𝑛 > 𝑚: meaning that number of unknowns are more than equations.

• E.g: 𝐴 = [1 2 31 0 1

], 𝑏 = 1, 2 , c = [1, 2, 3]

• You may notice that A is 𝑛 × 𝑚, b is 1 × 𝑚, c is 1 × 𝑛.

Hamid Reza Rabiee Fall 2020


• Having A, b and c, our objective in linear programming

is:

• To minimize 𝑐 · 𝑥 subject to the requirements 𝐴𝑥 = 𝑏 and 𝑥 ≥ 0. 𝑥 being the variable 𝑛 × 1 vector.

• E.g: 𝐴 = [1 1 2], 𝑏 = [4], cT =[5 3 8]

• 𝑥 = [

𝑥1𝑥2𝑥3]

• Minimize 5𝑥1 + 3𝑥2 + 8𝑥3 𝑤ℎ𝑖𝑙𝑒 𝐴𝑥 = 𝑏, 𝑥 ≥ 0



• In another perspective we first restrict our answer space by

conditions of 𝐴𝑥 = 𝑏 and 𝑥 ≥ 0, then we try to find a point in the restricted space that has the lowest cost.


Recap: Linear Programming Definition

• So our problem is to look for the lowest cost in our

polygon (triangle in the previous example) which is

created by the constraints.

• The lowest 𝑐. 𝑥 (our goal) can only happen on the vertices of the polygon (the restricted area).

• E.g. in our example, we should only check 𝑐. 𝑥 in three tips of the triangle and pick its minimum and we can be sure

that it is the lowest possible 𝑐. 𝑥 subject to the constraints.


Titles


• Dual Problem

• Simplex Method

• Other Methods


Dual Problem

• Every linear programming problem, referred to as

a primal problem, can be converted into a dual problem.

• The dual problem provides an upper bound to the optimal

value of the primal problem.

• With the same matrix and vector definitions we have:

• Dual problem: Maximize 𝒃. 𝒚 subject to 𝑨𝑻𝒚 ≤ 𝒄](Primal: minimize 𝑐 · 𝑥 subject to the requirements 𝐴𝑥 = 𝑏and 𝑥 ≥ 0)


Recap: Dual Problem

• Each linear programming problem has a dual problem in

which we –instead of minimizing the cost- try to maximize

our income 𝑏. 𝑦. • Notice that y and x are do not necessarily have the same

dimensions.

• (Reading page 485 of Strang Textbook for getting a better

intuition is highly recommended).


Titles


• Dual Problem

• Simplex Method

• Other Methods


Simplex Method

• Bear in mind that we want to see which of the polygon

edges can produce the least cost.

• There are many solutions to this problem, one of the most

used and oldest ones is the Simplex method.

• It aims to solve the problem by constructing a feasible

solution at one edge of the polygon and then walk along a

path on the edges of the polytope to vertices with non-

decreasing values of the objective function until an

optimum is reached for sure.


Simplex Method

• With having the vertices, you start with one of the vertices

and calculate how much choosing other vertices will

decrease your cost.

• At each corner we have n non-zero and m zero variables.

• The simplex method must decide which component

"enters“ the vector by becoming positive, and which

component "leaves" by becoming zero.

• That exchange is chosen so as to lower the total cost.


Simplex Method Example

• 𝑀𝑖𝑛𝑖𝑚𝑖𝑧𝑒 𝑡ℎ𝑒 𝑐𝑜𝑠𝑡: 𝑐. 𝑥 = 3𝑥1+ 𝑥2+ 9𝑥3+ 𝑥4 𝑠. 𝑡. 𝑥≥ 0

• 𝑥1+ 2𝑥3+ 𝑥4 = 4• 𝑥2+ 𝑥3− 𝑥4 = 2

• Starting corner = (4, 2, 0, 0), c.x = 14


Solution: Strang page 487

Titles


• Dual Problem

• Simplex Method

• Other Methods


Other Methods

• There are several other methods that help us find the

optimal values of x. Interior point methods are one of

them.

• Instead of moving along the edges of the polygon, they try

to move inside the polygon and find the value.

• Each of the methods come in handy in various problems in

their efficiency depends heavily on the structure of the

problem.


Summary

• Linear programming is an optimization problem which is

dealing with linear functions and constraints. It is of a high

importance in management problems.

• Dual Problem: Each LP problem has a Dual form that can

be used to understand and solve the primal problem instead

• Simplex Method: A method to find the lowest-costing

vertex in the space where the conditions apply.


Special Thanks to Armin Moradi for preparing this presentation.


Linear Algebrace.sharif.edu/.../Lectures/27_Inclass_Linear-Programming.pdfLinear Programming...

Documents

Transcript of Linear Algebrace.sharif.edu/.../Lectures/27_Inclass_Linear-Programming.pdfLinear Programming...