Lecture 4
C25 Optimization Hilary 2013 A. Zisserman
Optimization on graphs
• Max-flow & min-cut
• The augmented path algorithm
• Optimization for binary image graphs
• Applications
Max FlowGiven: a weighted directed graph with two distinguished nodes:
• source s, • sink (destination) t
Interpret edge weights (all positive) as capacities
Goal: Find maximum flow from s to t
• Flow does not exceed capacity in any edge
• Flow at every vertex satisfies equilibrium[ flow in equals flow out ]
e.g. oil flowing through pipes, internet routing
B1 reminder
Example 3 cont.
Slide: Robert Sedgewick and Kevin Wayne
B1 reminder
Example 2 cont.B1 reminder
TS
1
2
1
2 1
3
2
Max flow example
• max flow is unique = 2
• but there may be multiple paths (solutions) that achieve it
Matlab LP function linprog for max-flow
>> f = [ -1; -1; 0; 0; 0 ];
>> A = [1 0 0 0 00 1 0 0 00 0 1 0 00 0 0 1 00 0 0 0 1 ];
>> b = [1; 2; 2; 3; 1];
>> Aeq = [1 0 -1 -1 0 0 1 1 0 -1];
>> beq = [ 0; 0 ];
>> lb = zeros(5,1);
>> x = linprog( f, A, b, Aeq, beq, lb );
TS
1
2
1
2 1
3
2
>> Optimization terminated.
>> x
x = 1.0 1.0 0.0 1.0 1.0
>> flow = - x’ * f
flow = 2.0
e1
e2
e3e4
e5
e1 e2 e3 e4 e5
Another example
Source
Sink
v1 v2
2/2
3/5
5/9
4/43
1/1
Max Flow = 7
Source
Sink
v1 v2
2
5
9
42
1
The st-Mincut Problem
Source
Sink
v1 v2
2
5
9
42
1
Slides from Pushmeet Kohli
• Each node is either assigned to the source S or sink T
• The cost of the edge (i, j) is taken if (i∈S) and (j∈T)
•An st-cut (S,T) divides the nodes between source and sink
• The cost of the cut is the sum of costs of all edges going from S to T
• The st-min-cut is the cut with lowest cost
The st-Mincut Problem
Source
Sink
v1 v2
2
5
9
42
1
5 + 2 + 9 = 16
• Each node is either assigned to the source S or sink T
• The cost of the edge (i, j) is taken if (i∈S) and (j∈T)
•An st-cut (S,T) divides the nodes between source and sink
• The cost of the cut is the sum of costs of all edges going from S to T
• The st-min-cut is the cut with lowest cost
The st-Mincut Problem
Source
Sink
v1 v2
2
5
9
42
1
2 + 1 + 4 = 7
• Each node is either assigned to the source S or sink T
• The cost of the edge (i, j) is taken if (i∈S) and (j∈T)
•An st-cut (S,T) divides the nodes between source and sink
• The cost of the cut is the sum of costs of all edges going from S to T
• The st-min-cut is the cut with lowest cost
Min-cut\Max-flow Theorem
Source
Sink
v1 v2
2
5
9
42
1
In every network, the maximum flow equals the cost of the st-mincut
Max flow = min cut = 7
Next: the augmented path algorithm for computing the max-flow/min-cut
Maxflow Algorithms
Augmenting Path Based Algorithms
1. Find path from source to sink with positive capacity
2. Push maximum possible flow through this path
3. Repeat until no path can be found
Source
Sink
v1 v2
2
5
9
42
1
Algorithms assume non-negative capacity
Flow = 0
Maxflow Algorithms
Augmenting Path Based Algorithms
1. Find path from source to sink with positive capacity
2. Push maximum possible flow through this path
3. Repeat until no path can be found
Source
Sink
v1 v2
2
5
9
42
1
Algorithms assume non-negative capacity
Flow = 0
Maxflow Algorithms
Augmenting Path Based Algorithms
1. Find path from source to sink with positive capacity
2. Push maximum possible flow through this path
3. Repeat until no path can be found
Source
Sink
v1 v2
2-2
5-2
9
42
1
Algorithms assume non-negative capacity
Flow = 0 + 2
Maxflow Algorithms
Source
Sink
v1 v2
0
3
9
42
1
Augmenting Path Based Algorithms
1. Find path from source to sink with positive capacity
2. Push maximum possible flow through this path
3. Repeat until no path can be found
Algorithms assume non-negative capacity
Flow = 2
Maxflow Algorithms
Source
Sink
v1 v2
0
3
9
42
1
Augmenting Path Based Algorithms
1. Find path from source to sink with positive capacity
2. Push maximum possible flow through this path
3. Repeat until no path can be found
Algorithms assume non-negative capacity
Flow = 2
Maxflow Algorithms
Source
Sink
v1 v2
0
3
9
42
1
Augmenting Path Based Algorithms
1. Find path from source to sink with positive capacity
2. Push maximum possible flow through this path
3. Repeat until no path can be found
Algorithms assume non-negative capacity
Flow = 2
Maxflow Algorithms
Source
Sink
v1 v2
0
3
5
02
1
Augmenting Path Based Algorithms
1. Find path from source to sink with positive capacity
2. Push maximum possible flow through this path
3. Repeat until no path can be found
Algorithms assume non-negative capacity
Flow = 2 + 4
Maxflow Algorithms
Source
Sink
v1 v2
0
3
5
02
1
Augmenting Path Based Algorithms
1. Find path from source to sink with positive capacity
2. Push maximum possible flow through this path
3. Repeat until no path can be found
Algorithms assume non-negative capacity
Flow = 6
Maxflow Algorithms
Source
Sink
v1 v2
0
3
5
02
1
Augmenting Path Based Algorithms
1. Find path from source to sink with positive capacity
2. Push maximum possible flow through this path
3. Repeat until no path can be found
Algorithms assume non-negative capacity
Flow = 6
Maxflow Algorithms
Source
Sink
v1 v2
0
2
4
02+1
1-1
Augmenting Path Based Algorithms
1. Find path from source to sink with positive capacity
2. Push maximum possible flow through this path
3. Repeat until no path can be found
Algorithms assume non-negative capacity
Flow = 6 + 1
Maxflow Algorithms
Source
Sink
v1 v2
0
2
4
03
0
Augmenting Path Based Algorithms
1. Find path from source to sink with positive capacity
2. Push maximum possible flow through this path
3. Repeat until no path can be found
Algorithms assume non-negative capacity
Flow = 7
Maxflow Algorithms
Source
Sink
v1 v2
0
2
4
03
0
Flow = 7
Source
Sink
v1 v2
2
5
9
42
1
2 + 1 + 4 = 7
Min‐cut = 7
Image Graphs
pixels
• many loops
• very large number of nodes (variables) – millions
• dynamic programming can’t be used
Consider binary graphs (h = 2)
ji
Example: noise removal in an image
• n is the number of pixels in the image, e.g. n = 1 M
• Graph structure: each vertex is connected to four neighbours
1
0
i
N (i)
f(x) =Xi
(zi−xi)2+dφ(xi, xi−1)
noisy data
zi = xi+ wi
where
xi ∈ {0,1} wi ∼ N(0,σ2)and
φ(xi, xj) =
(0 ifxi = xj1 ifxi 6= xj.
Graph cut algorithmsBinary optimization: each variable x has one of two possible values
f(x) =nXi=1
{mi(xi) +X
j∈N(i)φi(xi, xj)}
If f(x) is sub-modular, then it can be optimized by the Min-Cut algorithm
xi ∈ {0,1}
The cost function f(x) is sub-modular if
φ(0,0) + φ(1,1) <= φ(0,1) + φ(1,0)
N (i) is the neighbourhood of node i i
Complexity of minimization:
• exhaustive search O(2n)
• min cut O(n3)
original
original plus noise
Min x
20 40 60 80 100 120 140 160 180 200 220
20
40
60
20 40 60 80 100 120 140 160 180 200 220
20
40
60
20 40 60 80 100 120 140 160 180 200 220
20
40
60
d = 10
20 40 60 80 100 120 140 160 180 200 220
20
40
60
20 40 60 80 100 120 140 160 180 200 220
20
40
60
20 40 60 80 100 120 140 160 180 200 220
20
40
60
original
original plus noise
Min x
d = 60
n ∼ 20 K
Optimization using graph cuts
Stage 1: map the cost function f(x) onto a flow network so that a cut of the network corresponds to the cost f(x)
Stage 2: compute the min-cut of the network using an augmented path algorithm
pqw
n-links a cuts
t
Map f(x) onto network flow
Construct a network so that a cut corresponds to an assignment of xi
m1(1) D
C B
A
x1 x2
Label ‘0’
Label ‘1’ m2(1)
m1(0) m2(0)
= 0
= 1
f(x) =nXi=1
{mi(xi) +X
j∈N(i)φi(xi, xj)}
m1(1) D
C B
A
x1 x2
m2(1)
m1(0) m2(0)
Sink (1)
x1
Source (0)
m1(1)
m1(0)
m2(1)
m2(0)
For unary terms only:
Sink (1)
x1 x2
Source (0)
m1(1)
m1(0)
m2(1)
m2(0)
x2
x1 = 1, x2 = 1
→ f(x) = m1(1) +m2(1)
cut
Sink (1)
x1 x2
Source (0)
m1(1) + C - A
m1(0)
m2(1) + D - C
m2(0)
DC
BA
m1(1) D
C B
A
x1 x2
m2(1)
m1(0) m2(0)
0 1
0
1
x1
x2
= A +C-AC-A
00
0 1
0
1 D-C0
D-C0
0 1
0
1 00
B+C-A-D0
0 1
0
1
+ +
B+C-A-D
Now, include pair wise term …
add C-A if x1 = 1
add D-C if x2 = 1
Sub-modular constraint: flows must be positive. So, B+C-A-D >= 0
Sink (1)
x1x2
Source (0)
m1(1) + C - A
m1(0)
m2(1) + D - C
m2(0)
B+C-A-Dx1 = 1, x2 = 1
→ f(x) = m1(1) +m2(1) +D − A
m1(1) D
C B
A
x1 x2
m1(0)
cut
Sink (1)
x1x2
Source (0)
m1(1) + C - A
m1(0)
m2(1) + D - C
m2(0)
B+C-A-Dx1 = 0, x2 = 1
→ f(x) = m1(0) +m2(1)
+D − C +B+ C − A−D= m1(0) +m2(1) +B −A
m1(1) D
C B
A
x1 x2
m1(0)
cut
Summary: optimization using graph cuts
Stage 1: map the cost function f(x) onto a flow network so that a cut of the network corresponds to the cost f(x)
Stage 2: compute the min-cut of the network using an augmented path algorithm
Applications
Optimization of binary image graph using graph-cuts:
1. Image cut-out and editing
2. Image quilting
3. Interactive Digital Photo-montage
1. Image cut-out by binary segmentation
Object - white, Background - green/grey Graph G = (V,E)
Each vertex corresponds to a pixel
Edges define a 4-neighbourhood grid graph
Assign a label to each vertex from L = {obj,bkg}
Graph G = (V,E)
Cost of a labelling f : V L Per Vertex Cost
Cost of label ‘obj’ low Cost of label ‘bkg’ high
Object - white, Background - green/grey
Graph G = (V,E)
Cost of a labelling f : V L
Cost of label ‘obj’ high Cost of label ‘bkg’ low
Per Vertex Cost
UNARY COST
Object - white, Background - green/grey
Graph G = (V,E)
Cost of a labelling f : V L Per Edge Cost
Cost of same label low
Cost of different labels high
Object - white, Background - green/grey
Graph G = (V,E)
Cost of a labelling f : V L
Cost of different labels low
Per Edge Cost
PAIRWISECOST
Object - white, Background - green/grey
Graph G = (V,E)
Problem: Find the labelling with minimum cost f*
Object - white, Background - green/grey
f(x) =nXi=1
{mi(xi) +X
j∈N(i)φi(xi, xj)}jiN (i)
• xi = 1 for foreground pixels, xi = 0 for background
• mi(xi) is likelihood that pixel at i is foreground (if xi = 1), or back-
ground (if xi = 0 ), e.g. using colour histogram of seed regions
• φ(xi, xj) penalizes a change of state:
φi(xi, xj) =
(0 ifxi = xj
γe−β(Ii−Ij)2ifxi 6= xj.
Application: foreground/background image segmentation
foreground Seed Pixels
Background Seed Pixels
use seed pixels to learn colour distribution
Image editing …
Available in Microsoft Office …
Image Quilting
Example: Texture Synthesis
Goal of Texture Synthesis: create new samples of a given texture
Many applications: virtual environments, hole-filling, texturing surfaces
Input texture
B1 B2
Random placement of blocks
block
B1 B2
Neighboring blocksconstrained by overlap
B1 B2
Minimal errorboundary cut
Algorithm
• Pick size of block and size of overlap
• Synthesize blocks in raster order
• Search input texture for block that satisfies overlap constraints (above and left)
• Paste new block into resulting texture> use graph cuts to compute minimal error boundary cut
Efros & Freeman 2001, Kwatra et al. 2003
min. error boundary
Minimal error boundary
overlapping blocks vertical boundary
__ ==22
overlap error
Interactive Digital Photomontage
Agarwala et al. 2004
Use graph-cuts to quilt images
Top Related