Submodular Function Minimization
-
Upload
judah-william -
Category
Documents
-
view
71 -
download
5
description
Transcript of Submodular Function Minimization
Submodular Function Minimization
Collette CoullardIMA and Northwestern University
November 25, 2002
Outline
• Introduction to Submodular Function Minimization (SFM)– Definitions– Applications- Why is SFM interesting?– History
• Separation-optimization equivalence• Developments• Recent developments
– Iwata, Fleischer, Fugishige algorithm– Lex Schrijver’s algorithm
– Schrijver’s Algorithm
• Conclusions
Submodular Function Definitions
f 2E ℝ is submodular if
Equivalently:
ES R e
S R E, e E
f(S {e}) – f(S) f(R {e}) – f(R)
EA B
A,B E
f(A) + f(B) f(AB) + f(AB)
subsets of E
marginal value of e wrt S (cost)
marginal value of e wrt R (cost)
Example: Number of Different Colors
h
g
j
ik
e
E
a
b
c
d
f(E)=5
f({a,b,d,e,k})= 5
f({a,b,d,e,k,g})= 5
f({e,k,g})= 3
f({e,k})= 2
marginal value of g
1
0
<
f’({a,b,d,e,k})= 3
f’({a,b,d,e,k,g})= 6
f’({e,k,g})= 2
f’({e,k})= 02
3
<
marginal value of g
Example: Number of Connections (Edges))
h
g
j
ik
e
E
a
b
c
d
f’(E)= 11
Example: - Number of Connections (Edges))
h
g
j
ik
e
E
a
b
c
d
f’({a,b,d,e,k})= -3
f’({a,b,d,e,k,g})= -6
f’({e,k,g})= -2
f’({e,k})= 0
f’(E)= -11
-2
-3
<
marginal value of g
Example: Number of Colors - Number of Edges: f’’ = f + f’
h
g
j
ik
e
E
a
b
c
d
f’’({a,b,d,e,k})= 5-3=2
f’’({a,b,d,e,k,g})= 5-6=-1
f’’({e,k,g})= 3-2=1
f’’({e,k})= 2-0=2
f’’(E)= 5 -11= -6
-1
-3
<
marginal value of g
Example: Directed Cuts
G = digraph with node set V
A V\{s,t}s t
A
f(A) := number of arcs leaving node set A {s}
f(A) + f(B) f(AB) + f(AB)
A
B
AA
BB
Example: Benefit and Risk Tradeoff
DDCC
DDCC
DDCCi 2
i
EE: customers i : customers i
S S
( { }) ( ) ( { }) ( )f i f f iS S R Rf
marginal risk-marginal risk-marginal marginal benefit wrt benefit wrt SS
marginal risk-marginal marginal risk-marginal benefit wrt benefit wrt RR
2( )
Ri
i i Rif a bR
Benefit termBenefit term Risk termRisk term
RR
Net cost
Submodular Function Minimization: Find S E with minimum f(S)
Example: Find minimum (cardinality or capacity) (s,t)-cut
s t(Note: ƒ not monotone)
Example: Find minimum net value (colors-edges) node set
Submodular Function Minimization: Find S E with minimum f(S)
Example: Find minimum (cardinality or capacity) (s,t)-cut
s t(Note: ƒ not monotone)
Example: Find minimum net cost (max net profit) customer set
DDCC
Optimization and Separation
Theorem (Grötschel,Lovasz, Schrijver 1981) The optimization problem for P can be solvedin polynomial time if and only if the separation problem can be solved for P in polynomial time.
Given x0 and P, either conclude x0 P or return valid inequalityaxb with ax0 > b.
P
x0
Given c and P, find x* P such that cx* cx, for every x P.
P
x*
Optimization and Separation
Greedy Algorithm (Edmonds): ce1
ce2
x*e1 = f({e1})x*e2= f({e1,e2})-f({e1})
P
x*
P: x(A) f(A), A E x(E) = f(E)
xe1
xe2
Given x0 and P, either conclude x0 P or return valid inequalityx(A) f(A), with x0(A) > f(A) ormin { f(A) - x0(A) : A E }
P
x0
submodular
submodular function minimization
Challenge (Grötschel,Lovasz, Schrijver 1981)
Since Optimization and Separation are polynomially equivalent, and the optimization problem for P has a strongly polynomial combinatorial algorithm (GA), there should be a strongly polynomial combinatorial algorithm for the separation problem for P.
There should be a strongly polynomial algorithm forSubmodular Function Minimization (SFM).
Developments• Submodular function and matroids
– Rado (1942)– Ingleton (1959)
• Submodular functions and optimization: Greedy algorithm maximizes linear functions over polymatroids– Edmonds (1970)
• Polynomial solvability of submodular function minimization (SFM) via ellipsoid algorithm (separation-optimization equivalence)– Grötschel, Lovász, Schrijver (1981,1988)
• Partial order of polymatroid extreme points +finite combinatorial algorithm for SFM– Bixby, Cunningham, Topkis (1985)
• Pseudopolynomial combinatorial algorithm for SFM– Cunningham (1985)
• Strongly polynomial algorithm for symmetric SFM– Queyranne (1998)
• Strongly polynomial combinatorial algorithms for SFM– Iwata, Fleischer, Fugishige (1999), Iwata, Fleischer (2000), Fugishige,
Iwata(2001)– Schrijver (1999)
Submodular function and matroids
A matroid M=(E,I)a finite set
a collection of subsets of E (called independent sets) satisfying
I 1: empty set is independent
I 2: subsets of independent sets are independent I 3: All maximal independent subsets of A E have same size, called rank of A, r(A)
r is a submodular function on E
Matroid Examples
edge sets with no cycles
G=(V,E)
M=(E,I)
I is the edge se: | | t of a fores: t( ) maxI A
Ir A
Forest Matroid of a Graph
Rank function on 2E:
submodular
Matroid Examples
edge sets with no cutsets
G=(V,E)
M=(E,I)
Cutset Matroid of a Graph
Rank function on 2E:
I conta: ins no cuts| e|: t( ) maxI A
Ir A
submodular
Can E be partitioned into at most k forests?(k independent sets)
This is a special case of SFM.
Matroid Partition
G=(V,E)
1
1
| |: ... | \ | ( )max mink
k iiA EJ E
I E r AIJ J A
a submodular function on E
Matroid Intersection
G=(UV,E)
Given matroids M1=(E,I1) and M2=(E,I2),
max {|I|: I I1 I2} = min {f1(A) + f2(E\A) : A E}
a submodular function on E
Special case of SFM.
Maximum matching in bipartite graph is a special case.
Mathematical Programming Formulationof SFM
Maximize
Subject to: ,( )
,
0,
e
e
e
e E
e SS E
e E
f S
x
x
x
Minimize
Subject to::
( )
,1,
y 0,
S
S
S
S E
S e S
f S y
e E
S E
y
P: D:
Theorem. (Edmonds) This system is TDI, which implies D always hasIntegral optimal solutions.
•D models the submodular function minimization problem
•Feasible solutions to P provide lower bounds on f(S*)
•Optimal solutions to P provide certificate of optimality for S*
Schrijver’s Algorithm for SFM
•Maintains a feasible solution x to P’:
:
0:
,( ),
( )
e
e
ee S
e
e E
E xMaximize
Subject to
e
S Ef S
f Ex
x
x
•Maintains feasibility of x by keeping:
1 1
0, 1, ik k
i i ii i
x b
•Iteratively improves:
0: e
ee E x
x
b is an extreme point of P’
1
1 1 1
There is some ordering , , of such that
j( , , ) ( , , ), for each =1, , .j
n
jje
e e E
f e eef e nb
1
1 1 1
For some ordering , , of ,
( , , ) ( , , ), for ,j=1 , .j
n
jje
e e E
f e f eb e e n
Extreme Points
,( ) ( ),
( ) ( )
S ES f S
x E f E
xb is an extreme point of P’:
Feasibility proof for b :
smallest last j
Suppose ( ) ( )Ab f A
Then
some A E
( ) ( \ ) ( ) ( \ )je j jb b b A ffA e A A e
1 1 1( , , ) ( , , )jjf e f e ee
1 1 1( , , ) ( , , )jjf e f e ee
choice of A
submodularity of f
52 31 4 6
, , , , , , , , ,nje e ee e ee eb b b b b b b b
1 1
1 2 1 2
1 2 3 1 2 3
1 2 1 2
({ }) ({ })
({ , }) ({ , })
({ , , }) ({ , , })
({ , , , }) ({ , , , })n n
e f e
e e f e e
e e e f e e e
e e e f e e e
bbb
b
Lower ideals of the ordering are tight sets for b.
Show examples in Excel.
Optimality Condition: We have an xRE and an R E with
( ) ( ), for every S S Ex f S x is feasible for P’
1 2 1 2
1
1 1
1
1 1
2
2
1 1 1 1 1
2 2 2 2 2
1
1
, , , , , , , , , , , ,
, , , , , ,
, , , , , ,
r r
r
r
r n r n
r n
r n
e e e
k k k k
e e e e e e
e e e
e e
e
e e
k e ek
e
b b b b b
b b b b
x x x
b
x
b b b
x
b
b
R has all the negative components and no positive components of x
( )x R
1 1
( ) ( ) ( ) ( )i ii
k k
i i
fR RX fR Rb
R is the set of initial elements in the order, for each bi
( )f R
Optimality condition examples
0 18 38( , 13 1, , , , )1x
( ) 71 ( ) ( ) ( ),x xf S f SE SE E
2
17 0 5( , , ,8,0 15,14)xBalaji,Lisa,Peh Balaji,Lisa,{ } { }Peh( ) 67 ( ) ( ) ( ), S Ex xf S f S
4
{Toshio,Peh,Scott} {Toshio,Peh,Scott}( ) 15 ( ) ( ) ( ),f S f S S Ex x
1 1
2
1
2
2 3 1 4 5 6
1 2 3 4 5 6
= 0.7( , , 15, 24, 2, 1)
=0.3( 2
- 20 - 10
-5, 1, , 24, 2, 1)_ _
41_ ___
b
b
___________________________ ( , , , 24, 2, 1)- 6.5 - 6.7 - 1.8x
Auxiliary Network G=(E,A)
case 1: no path from + to -.
R
is a lower ideal in each order .
( ) (
R
) ( ) ( ) .x xf S f S S ER R
an arc ( , ) if in some order e f e f
1 2( , ,..., ,..., ,..., )
n
iuu u e fb b b b b b
a node for each element e E
0ex +
+
+
+ 0ex -
-
-
Auxiliary Network G=(E,A) an arc ( , ) if in some order e f e f
a node for each element e E
+ +
+
+
-
-
-
0ex 0ex
case 2: There is a path from + to -. Use last arc (s,t) to “improve x”.
st
Improving x: An Iteration, algebraically
x = 1 b1 + 2 b2 + ... + k bk ;
1 , ... , k 0; 1 + ... + k = 1
s t
= ’1 b’1 + ’2 b’2 + ... + ’k b’l ; b1
+ -1:.
+1
s
t
= 1 ( ’1 b’1 + ... + ’k b’l) + 2 b1 + ... + k bk
x’ = x + 1
-1:..
+1
s
t
= ’1 b’1 + ’2 b’2 + ... + ’k b’l ; b1
+ -1:.
+1
s
t
Schrijver’s clever idea
, , ,
({ }) ({ , }) ({ })
({ , }) ({ }) ({ })
({ , , }) ({ , }) ({ , , })
({ , , , }) ({ , , })
u s v s t s
f s f u s f u
f s u f s f u
f s u v f s u f u s v
f s u v t f s u v
s
u
v
t
b b bb
({ , }) ({ }) ({ , }) ({})
({ , , }) ({ , }) ({ , , }) ({ , })
({ , }) ({ }) ({, , , }) ({ , , })
({ , , , }) ({ , , }) ({ , , , }) ({ , , }) ({})
f v s f v f t s f t
f v s u f v s f t s u f t s
f u s f v f t s u v f t s u
f u s v t f u s v f v s u t f v s u f t
, , ,
({ }) ({ , }) ({ }) ({ })
({ , }) ({ })
({ , , }) ({ , })
({ , , , }) ({ , , })
u s v s t s
f s f u s f u f s
f s u f s
f s u v f s u
f s u v t f s u v
s
u
v
t
b bb b b b b
({ , }) ({ }) ({ }) ({ , }) ({}) ({ })
({ }) ({ , }) ({ }) ({ , , }) ({ , }) ({ , }) ({ }) ({ , , }) ({ , }) ({ , }) ({ })
0 ({ }) ({ , , }) ({ , }) ({ , , , }
0 0
f v s f v f s f t s f t f s
f u f s u f s f v s u f v s f s u f s f t s u f t s f s u f s
f v f s u v f s u f t s u v
, , ,
0
0 0
) ({ , , }) ({ , , }) ({ , })
({})
1
0
0
1
1
0
0
1
u s v s t s
f t s u f s u v f s u
f t
, , , , , ,
, , , , , ,
1
0
0
1
u s u s v s v s t s t s
u s u s v s v s t s t s
b bb b
b b
b
bb
b
An Iteration, geometrically
x = 1 b1 + 2 b2 + ... + k bk ;
1 , ... , k 0; 1 + ... + k = 1
= ’1 b’1 + ’2 b’2 + ... + ’k b’l ; b1
+ -1:.
+1
s
t
= 1 ( ’1 b’1 + ... + ’k b’l) + 2 b1 + ... + k bk
x + 1 -1:.
+1
s
t
x’ = ...
Complexity of O(n6)
2
'
' , for every
' , for some , this can happen at most times.
else, '
if t' t, then t'<t, because t was chosen with largest index.
else t'=t. Same t in consecutive iterati
.
u u
u u
t t
d d
if d d u
d d
n
u E
ons.
if s' s, then s'<s, because s was chosen with largest index.
else s'=s. Same s and t in consecutive iterations.
The must be different from , since is gone.
b' b b
t' t
Either ( ', , ) ( , , ),
or : ( , , ) ( , , ) has decreased.
Either ' , for some ,
or ((d' ,t'),s', ', ' ) is lexicographically smaller than ((d ,t),s, , )
u u
dist s t distb b
b b
s t
dist s t dis s
d u
bt t
d
.
n n n n
t(d ,t)
s
Conclusions and future work
• Submodular Function Minimization (SFM) generalizes several combinatorial optimization problems.
• Schrijver’s elegant algorithm – known basic ideas from ‘80s– new idea on step direction– clever tie-breaking for O(n6) complexity bound
• C++ implementations in the works– Schrijver Algorithm– IFF Algorithms– possibly Queyranne’s column-generation idea– others?