Linear Programmingweb.cse.ohio-state.edu/~lai.1/6331/7.LP.pdf · 2012. 11. 19. · Linear...
Transcript of Linear Programmingweb.cse.ohio-state.edu/~lai.1/6331/7.LP.pdf · 2012. 11. 19. · Linear...
-
Linear Programming Reading: CLRS, Ch. 29 or references
CSE 6331 Algorithms Steve Lai
-
1 2
1 2
1 2 1 1 2 2
Let , , , , be real numbers and let , , , be variables.
Linear function: ( , , , )
Linear equality
Linear functions and linear constraints
n
n
n n n
a a a bx x x
f x x x a x a x a x= + + +
•
•
•
1 2
1 2
1 2
: ( , , , ) (hyperplane)
Linear inequalities: ( , , , ) (half-space) ( , , , ) (half-space)
n
n
n
f x x x b
f x x x bf x x x b
=
≤≥
•
2
-
1 2
1 1 1 2
Given real numbers , , , where 1 , 1 ,
find real numbers , , , to
optimize a linear objective function
General Linear Programming Problems
i j ij
n
n n
b c a i m j nx x x
c x c x c x
≤ ≤ ≤ ≤
+ + +
•
1
subject to linear constrainlts
for 1 . n
ij j ij
a x b i m=
≤ = ≤ ≤ ≥
∑
3
-
1 2
1 1 2 2
1
find real rnu
1 , 111
, ,
Given r
, mbers
eal numbers
that
for 1
0 f
m
aximize
subject to
o
Standard Maximum Form
ij
i
j
n
n n
n
ij j ij
j
a i m j nb i mc j n
x x x
c x c x c x
a x b i m
x=
≤ ≤ ≤ ≤≤ ≤≤ ≤
+ + +
≤ ≤ ≤
≥
∑
r 1 j n≤ ≤4
-
11 12 1 1 1
21 22 2 2 2
1 1
and
maximiz
e
subject t
o
In Matrix NotationT
n
n
m m mn n n
a a a x ba a a x b
a a a x b
≤ ≥
≤
c x
Ax b x 0
5
-
( ) ( )
( ) ( )
1 2 1 2
1 2 1 2
11 12 1
21 22 2
1 1
-vectors:
-ve
, , , , , , ,
, , , , , , ,
cto
rs:
-matrix:
NotationT T
m m
T Tn n
n
n
m m mn
b b b y y y
x x x c c c
a a aa a a
a a
m
n
m n
a
= =
= =
=
×
b y
x c
A
6
-
( ) ( )
1 1 2 2
11 12 1
21 22 21 2 1 2
1 1
minimize
subject to
and
Standard Minimum Form (in Matrix Notation)T
m m
T T
n
nm m
m m mn
y b y b y b
a a aa a a
y y y c c c
a a a
= + + +
≥ ≥
≥
y b
y A c y 0
7
-
1 2
1 2
Variables: , , , Feasible solution: a vector ( , , , ) that
satisfies all the constraints. Feasible region : the set of all feasible solutions. Optimal solut
Terminology
n
n
x x xx x x
••
Ω••
ion: a solution that optimizes the objective function. Optimal objective value *: A linear program may be
or ( is nonempty or empty)
feasible infeasible bound or ed unbounded
f•
Ω•
( * is finite or infinite)f
8
-
1 2
1 2
1 2
1 2
1 2
maximize
subject to
2 4 4 2 12
1 , 0
objective function
main constraint
s
Examplex x
x xx xx x
x x
+
+ ≤ + ≤ − + ≤
≥ nonnegativity const ra ints
9
-
10 Source: Linear programming, Thomas S. Ferguson, UCLA
-
persons are available for jobs. If person works on job for one day, the value is . Choose an
assignment of persons to jobs to maximize th
Probl
e to l
em:
ta
Example: Optimal Assignment Problem
ij
I J ij a
1 1
1
1
Let
maximize
subjec
value. be the portion of time of person to work on job .
1 for 1,2, ,
1 for 1,2, ,
0
t to
ij
I J
ij iji j
I
ijiJ
ijj
ij
x i j
a x
x j J
x i I
x
= =
=
=
≤ =
≤ =
≥
∑∑
∑
∑
for 1,2, , and 1,2, ,i I j J= = 11
-
Two maximization LPs, and , are if for each feasible solution to with objective value there is a
equivalent
correspond feasible solution ting o with
Equivalence of Linear ProgramsL L
zLL
′
′ ′
•x
x
equivalent
objective value , and .
A maximization LP, , and a minimization LP, , are if for each feasible solution to with objective value there is a cor
vi
re
ce
sponding f
ve
ea
rsa
sibl
z
z
L LL′•
xe
solution to with objective value , and vice versa.
L z′ −′x
12
-
We can always convert an LP into an equivalent LP in standard (maximum or minimum) form.
A linear program may not be in standard maximum form for any of thes
Converting to Standard Maximum Form•
•e reasons:
1. The objective funtion may be a minimization.
2. There may be " " (instaed of " ") inequality constraints.
3. There may be equality constraints.
4. There may be some without nonnegix
≥ ≤
ativity constraints.
13
-
1 1
1 1
1. Convert minimization to maximixation: Minimize ( , , ) maximize ( , , )
2. Convert a " " constraint to a " " constraint:
( )
3. Convert an equality c
n n
n n
ij j i ij j ij j
f x x f x x
a x b a x b= =
−
≥ ≤
≥ − ≤ −
⇒
⇒∑ ∑
1 1
1
1 1
onstraint into two inequalities:
( )
n n
ij j i ij j inj j
ij j i n nj
ij j i ij j ij j
a x b a x ba x b
a x b a x b
= =
=
= =
⇒ ⇒
≤ ≤
= ≥ − ≤ −
∑ ∑∑
∑ ∑
14
-
1
4. If is without nonnegativity constraint,
we replace with and add 0, 0.
Thus, becomes , and
becomes .
A feasible solution ,
j
j j j j j
j j j j j j
j j j j j j
xx x x x x
c x c x c xa x a x a x
x
′ ′′ ′ ′′− ≥ ≥
′ ′′−
′ ′′−
( )( )1
, , , , to the new LP
corresponds to a feasible solution , , , ,
to the original LP.
j j n
j n
x x x
x x x
′ ′′
15
-
1 2 1 2
1 21 2
1 21 2
1 21
1
2 3 2 3
subject to
min
subject
im
to
77
7 2 4
ize maximiz
2 4
0
e
0
Examplex x x x
x xx x
x xx x
x xx
x
− + −
+ ≤+ =
− − ≤ −− ≤
− ≤≥
≥
16
-
1 1
2 2 2
2
1 2 1 2 2
1 2 1 2
1
2
1 2
2maximize 2 maximize 2
subject to
Replace wit
subject t
h :
3 3 3
o
7 77
2 4
0
x x
x x x x xx x x xx x
x x
x x x
x
x
′ ′′+ ≤ + − ≤−
′ ′′−
′ ′′−
− ≤ −
−
−
≥
+
− +− ≤
2
1 2 2
1 2 2
72 2 4
, , 0
xx x xx x x
′′ ≤ −′ ′′− + ≤
′ ′′ ≥
17
-
1 2 3
1 2 3
1 2 3
1 2 3
2 3
3
2 2
1 2
maximize 2 3 3
subject to
77
Renaming ,
2 2
as , yie
4, , 0
lds:x
x x x
x x xx x xx x xx x x
x x x
− +
+
′
− ≤− − + ≤ −
− + ≤≥
′′
18
-
1 1 2 2
Each LP has an intimately related LP, called its dual. The original LP is called the pri
maximi
mal.
Primal (standar
ze
subjec
d maximum )
t
L :
P
Duality
Tn nc x c x c x
•
•
= + + +c x
1 1 2 2
to
minimiz
and
e
Dua
subject
l (stan
andto
dard minimum LP
):
Tm m
T T
y b y b y b
≤ ≥
= + + +
≥ ≥
•
Ax b x 0
y b
y A c y 0
19
-
1 2
1 11 12 1 1
2 21 22 2
The dual of the Standard Minimum LP is the Standard Maximum LP. The primal max LP and the dual min LP can be
simultaneously displayed as:
n
n
n
x x xy a a a by a a a
•
•
≤≤
2
1 2
1
m m m mn m
m n
b
y a a a bc c c
≤≥ ≥ ≥
20
-
1 2 1 2 3
1 2
1 2
1 2
maximize minimize 4 12
subject to
Primal problem: Dual
subject to
2 4
problem
4 2 12
:
Example
x x y y y
x xx xx x
+ + +
+ ≤+ ≤
− +
1 2 3
1 2 3
1 2 1 2 3
4 1 2 2 1
1 , 0 , , 0
y y yy y y
x x y y y
+ + ≥+ − ≥
≤≥ ≥
21
-
1 2
1
2
3
1 2 44 2 121 1 11 1
Examplex x
yyy
≤≤
− ≤≥ ≥
22
-
If is a feasible solution to the standard maximumLP ( , , ) and a feasible solution to its dual, thenTheorem.
Proof.
.
,
,
Weak Duality Theorem
T T
T T
T T T T
≤
≤ ≥ ⇒ ≤
≥ ≥ ⇒ ≥
xA b c y c x y b
Ax b x 0 y Ax y by A c y 0 y Ax c x
.
If a standard LP and its dual are both feasible,then they are bounded feasible.
If and are feasible solutions to the primal anddual L
Corollary
Ps, respe
.
Corctiv
ollary.ely, and
T T
T T
⇒ ≤
=
c x y b
x yc x y b, then
and are optimal solutions to their respective problems.x y
23
-
A feasible solution to the primal linear program( , , ) is optimal iff there is a feasible solution to its duallinear program such
Theore
that . In this ca
m.
se, is
Strong Duality Theorem
T T=
xA b c y
c x y b y also optimal.
24
-
Primal
feasible bounded feasible unbounded infeasiblef.b. yes no no
Dual f.u. no no yes
i no yes yes
25
-
( ) ( )( ) ( )
1 2 1 2
1 2 1 2
11 1
to and
, , , , , , ,
Findmaximiz
,
where
esubject t
, , , , , ,
o
LP in equality form
T
T Tm m
T Tn m n m
b b b y y y
x x x c c c
a x a
+ +
= ≥
= =
= =
+
xc xAx b x 0
b y
x c
12 2 1, 1
21 1 22 2 2,
1 1 1 2 ,
n m n m
n m n m n
m m m n m n m m
x a x ba x a x a x b
a x a x a x b
+ +
+ +
+ +
+ + =+ + + =
+ + + =
26
-
Assume rank( ) . Let be a matrix formed by choosing columns out ofthe columns of . If B is a non-singular matrix and if all the variables not associated with the columns i
Basic SolutionsA m
B mm n A
n
=
+
basic solutio
n are set to zero, then thesolution to the resulting system of equations is called a
. The variables associated with thecolumns in
nbasic variables; are called the other va b ria
i
B
m xB
non-basic variablesbasic feasible soluti
les are . If a basic solution is feasible, it iscalld a . There are at most
( )! basic solutions
n
.! !
o
m nm
m nCm n
+ +=27
-
1 2
1 2
A set is said to be if for all pointsconvex
extreme poin
, we have (1 ) for all 0 1. A point in a convex set is called an
t
Basic feasible solutions are extreme pointsC x x C
x x Cx Cλ λ λ
∈+ − ∈
•≤ ≤
•
1 2 1 2
1 2
if there exist no , , where , and 0 1 such that (1 ) . Geometrically, extreme points are . The feasible region of an LP is a conv
corner points (verti
B
cesex set.
as i f
)
c
x x C x xx x x
λλ λ
∈
••
≠+
•
≤ ≤= −
of an LP (in the equality form) are of the feasible regions. If the LP is feasible bounded, t
easible solutionsextreme points
optimum soluthen at least one
occurs at an extreme poion int!•
28
-
Simplex Method
A system of linear inequalities define a polytope (or simplex) as a feasible region. The simplex algorithm begins at a starting vertex and moves along the edges of the polytope until it reaches the vertex of the optimum solution.
29 Source: Linear programming, Thomas S. Ferguson, UCLA
v̂
http://en.wikipedia.org/wiki/Vertex_(geometry)
-
1 2
Find maximize su
to bject to
Find
ma
Stand maximum form:
Equivalent sl
and
and ( ,
, , ) to
ack f r o m:
Converting standard form to slack form
T
Tms s s
•
•
≤ ≥
=
xc xAx b x 0
x s (or minimize ) and ,
may be written
ximize subje as
ct tr .
oo
T T−+ = ≥
• + = − − = −≥
=
c x c xAx s b x 0 s 0
Ax s b s b Ax s Ax b30
-
1 2
1 2
1 11 12 1 1
2 21 22 2 2
1
, , , are called .
The equality and the objective function can be displayed in a tableau:
1 1
slack variables
m
T
n
n
n
m m
s s s
x x xs a a a bs a a a
z
b
s a a
− = −
=
•
− −
• s Ax bc x
2
1 0m mn m
m n
a bz c c c− − −
31
-
{ }
{ }1 2
1 2
and , :
Basic variables: , , ,
Non-basic variab
minimiz
les: , , ,
The solution i
esubje
s a, b
ct
as
to
Basic and non-basic variablesT
m
n
B s s s
N x x x
•
•
−+ = ≥ ≥
=
=
=• =
c xs Ax b x 0 s
x 0 s b
0
ic solution.
basic feasible solut If , the solution is a .
If and , the solution is an
ion
optimum solution.
≥
≥ −
•
≥•
b 0
b 0 c 0
32
-
33
1
1 11 1 1 1
1
1
1
s.t. and , :
1 1
where 0 initially If 0, we can arp
Maximize
ounv i ot d
Pivot OperationT
n
j n
i in i
m m mj mn m
j n
i i
j
j
i
j
v
x xs a a a b
a a b
s a a a bz c c c v v
x
s
a
a
+ + = ≥ ≥
− −
− − − =
• ≠
c x s Ax b x 0 s 0
, switching the roles
of and ij
i j
as x
-
34
( )1 1 , 1 1 , 1 1
, 1 , 111 1 1
Dividing by the equation
,
we have an equivalent equation:
.
Substituting
i i j j i j j in n i
i j
ij
ij
ij ij i
i ji in ij j
j ij ijn
i j
j i
s a x a x a
a
a
a a a
x a x b
a aa a bx
x
x xsa
xa
x
− − + +
− +− +
•
•
= − + + + + + + +
= − + + + + + + +
11 1
this into every other equation yields:
k kj i kj ink k kn n
ij ij
kj ik i
ji
ij
j
j
a a a as a x a x
a a
a bb
a
xa
b
a −
+ − + + + + −
= −
-
35
1
1 11 11 1 1
1
1
1
1
After pivoting around , we have the equivalent LP:
1 1
ˆ
ˆ
ˆ ˆ ˆ ˆ
1
n
j in j in
ij ij
mj in mj im m m
i
j
ij
i in ij
i
n mi
j
ij
ij ij
mj
ij
j ij
j n
ij
x xa a a
saa
a a bx
bs a a b
a a
a a a bs a a b
a az c c c v
a
a
a
a a
a
a
− −
− −−
− −
−
−
− −
-
36
( )
( )
1
1. Pivot element: 1
2. Other elements in the pivot row:
3. Other elements in the pivot column:
4. All other elements
E xample:
:
3
Summary of Pivot Operation
r p r pc q c p q rc p
p p
r r p
c c p
q q rc
p
p
⇒− −
→
→
→−
−
•
→
0 5 3 2 9 2 7 23 1 1 2 3 2 1 2
0 4 3 0 4 3
−− ⇒ −2
-
1 1
Find minimize
to an
Stand min
d
and ( , , , ) to
imu
subject to
m form:
Equivalent slack
form:
Find
Converting minimum form to slack form
T
T
Tnu u u
≥ ≥
=
•
•
yy by A c y 0
y s minimize subject t
and , o
T
T= − ≥ ≥
y bs y A c y 0 s 0
37
-
{ }{ }
1 2
1 11 12 1 1
2 21 22 2 2 1 2
1 2
1 2
1
The equality and objective function can be displayed in a
tableau:
, , ,
, , ,
0
The s
n
n
n m
n
m m m mn m
n
T T
m
s s sy a a a by a a a b B s s s
N y y yy a a a b
c c c
==
− −
• = −
•
−
s y A c y b
, basic solution.
basic feasible solution
olution is a
If , the solution is a . If and , the solution is optimum soluan tion
=
− ≥≥ ≥•
=
•−
−
c 0b 0 c
y 0 s c
038
-
39
( )1
So we display both the primal and the dual in thesame tableau.
Pivoting for the dual follows the same rule:
r p r pc q c p q rc pp
⇒− −
-
1 2
1 11 12 1 1
2 21 22 2 2
1 2
1
0
n
n
n
m m m mn m
m n
x x xy a a a by a a a b
y a a a bz c c c− − −
40
-
1 2
1 11 12 1 1
2 21 22 2 2
1
Pivot madly until we suddently find that all entries in the lastrow and last column (exclusive of the corner) are nonnegative.
ˆˆ ˆ ˆˆˆ ˆ ˆ
ˆ ˆ
Pivot Madly Method
n
n
n
m m
r r r
t a a a b
t a a a b
t a
{ }{ }
{ }{ }
1
1
12
11
0 for , ,ˆ for ,
Solution for primal:
Soluti,
ˆ ˆ for , ,ˆ
on for dual:
ˆO
ˆ0 for , , 0
ˆ ˆ ˆ ˆptimal value
j j n
i i i m
j j n jm mn m
i i mm n
x x r r
x b x t t
y c r r ca a b
y y t tc c c v
v
= ∈
= ∈
= − = −
= ∈ =− −
=−
41
-
Suppose after pivoting for a while we have the tableau:
If , then is a feasible solution for the maximization problem. If , then is a f
,
, easible s
v
= =
−
≥
−
•
≥• = = −
r 0 t b
t 0
rt A b
c
r0 c
b 0
c olution for the minimization problem. If and , then we have an optimal solution
for both problems.≥ − ≥• b 0 c 0
42
-
The same as the pivot madly method except that we now choose the points to pivot on more systematically.
The goal is to make all entries in the last row and last column (ex
The Simplex Method•
•clusive of the corner) nonnegative.
v−
rt A b
c
43
-
0
0
0 0 0 0
0
, 0
, ,
Case 1: . Take any column with 0. Among
those with 0, choose the for which the ratio
is smallest. Pivot around . (I
Pivot rules for the simplex method (1)
j
i j
i i j i j
j c
i a i
b a a
≥ −
b 0
0
1 2 3 4 5 1 2 3
1 1
2 2
3 3
4 4
f no such ,
the max problem is feasible unbounded.)
1 1 1 6 6 1 3 10 4 4 4 0 2 4
0 3 1 7 7 2 3 15 1 4 2
33 2
23
31212
0 5 4 25 2 4 2 1 0 5 4 0
i
r r r r r r r rt tt tt tt t
−
− − − −− − − − −
−−−−−
44
-
45
( )
0
0
0
0
0
0
0
,
,
After the pivot, stays nonnegative, the entry
becomes positive, and the value never gets less ( gets greater if 0).
1
Properties of rule 1:
j
i
i
i j i
j
i j
c
vv b
r p r pc q c p q rc p
b
a b
c
a
v
p
−
≠
⇒− −
−
• b
( )( )
0 0 0 0 0
0 0 0 0 0 0 0
0 0 0 0 0 0 0
, ,
, , , ,
, ,
1
i j i i j
i j i j i i j i i j
j i j j i i j
a b a
a a b a b a
c a v c b a
⇒ − −
+
-
0, 0
Case 2: some are negative. Take the first 0. Find any negative entry in row , say 0. (If there is no such ,
the max problem is infeasible.)
Pivot rules for the simplex method (2)
i k
k j
b bk a j
<<
•
[ ] [ ][ ]
[ ]
0 0
0 0 0 0 0
0 0
, ,
, 0 ,
0 ,
1 2 3 4
1
2
3
4
Compare with
for which 0, 0 and choose such that
is smallest ( may be .) Pivot around .
1 3 11 2 4 4
2 1 3 1 21 4 2 0
5 2 4 2 0
1 24
5
k k j i i j
i i j i i j
i j
b a b a
b a i b a
i k a
r r r rtttt
≥ >
− − −−
− − −− −
−
−
1 2 3
1
2
3
4
0 1 2 41 3 3 1
2 2 3 15 1 4 0
r r rtttt
− −− −
−−
46
-
47
( )
0
The objective in Case 2 is to get to Case 1. With the pivot, the nonnegative stay nonnegative,
and becomes no smaller (it gets larger if 0).
1
Properties of rule 2:
i
k i
bb b
r p r pc q c cp
p q r p
≠
⇒− −
••
( )( )
0 0 0 0 00
0 0 0 0 0 0 0 0
00 0 0 0 0 0 0
0 0, ,
, , , ,
, ,
,
,
1
i j i i ji
i j i i j i j i i j i i j
k j k k j i j
i j
k k j i i j
a b ab
a b a a b a b a
a
a
b a a b a b a
⇒ − −
− −
-
0
0 0
0
, 0 ,
Case 3: . Treat it as the dual problem. Take any column such that 0. Among those with
0, choose the for which the ratio is
Pivot rules for the simplex method (3)
i
i j j i j
i b j
a j c a
≥<
< −
• − c 0
0 0, 0
1 2 3 4
1
2
3
closest to zero. Pivot around . (If there is no such ,
the dual problem is feasible unbounded.)
1 1 3 1 60 3 2 4 4 2 0 3 1 25 1 4 2 3
i ja j
r r r rttt
− −
−
48
-
1 2 3
2 3
1 3
1 2 3
1 2 3
2 3
1
Primal: maximize 2 subject to all 0 and
2 3 3 2
2 1
Dual: minimize 3 2 subject to all 0 and
2 1
Example
i
i
x x x x
x xx xx x x
y y y y
y yy y
+ + ≥
+ ≤− + + ≤
+ + ≤
+ + ≥
−
•
≥
•
++ + 3
1 2 3
12 3 2y y y
≥+ + ≥
49
-
[ ][ ]
1 2 3 1 3 3
1 1
2 2
3 2
1 3 2
1 1
3
2
0 1 2 3 2 1 1 21 0 3 2 1 0 2
2 1 1 2 1 1 11 1 2 0 1 1 1 1
5 3 1 1 3 4 3 1 3 0 1 3 2 3 7 3 1 1 3 1 3
Maximum solution for
2 3 1 1 3 5 3
primal:
31
x x x x y xy yy yy x
x y yy xxx
→
− −− −→
− − − −
− − −−
−
2 3
1 2 3
0, 1 3, 2 3l
0, Minimum solution for dua
Optimal value for bot1 3
h:, 1
5 3
x x
y y y
= = =
= = =
50
-
The simplex rules as stated may lead to cycling, even though it is rare in practical problems. There are strategies (modified simplex methods) for
avoiding cycling. Smallest-subscript r
Cycling•
•
• ule: If there is a choice of pivot rows or a choice of pivot columns, select the row (or column) with the variable having the lowest subscript, or if there are no variables, with the varia
xx y ble having the lowest
subscript.
51
-
References
• T.S. Ferguson, “Linear Programming: A Concise Introduction,” http://www.math.ucla.edu/~tom/LP.pdf .
• Lecture notes: http://www.math.cuhk.edu.hk/~wei/LP11.html
52
http://www.math.ucla.edu/~tom/LP.pdfhttp://www.math.cuhk.edu.hk/~wei/LP11.html
Linear Programming�Reading: CLRS, Ch. 29 or references�Slide Number 2Slide Number 3Slide Number 4Slide Number 5Slide Number 6Slide Number 7Slide Number 8Slide Number 9Slide Number 10Slide Number 11Slide Number 12Slide Number 13Slide Number 14Slide Number 15Slide Number 16Slide Number 17Slide Number 18Slide Number 19Slide Number 20Slide Number 21Slide Number 22Slide Number 23Slide Number 24Slide Number 25Slide Number 26Slide Number 27Slide Number 28Simplex MethodSlide Number 30Slide Number 31Slide Number 32Slide Number 33Slide Number 34Slide Number 35Slide Number 36Slide Number 37Slide Number 38Slide Number 39Slide Number 40Slide Number 41Slide Number 42Slide Number 43Slide Number 44Slide Number 45Slide Number 46Slide Number 47Slide Number 48Slide Number 49Slide Number 50Slide Number 51References