Connecting Graph Cuts and Level Sets
Transcript of Connecting Graph Cuts and Level Sets
![Page 1: Connecting Graph Cuts and Level Sets](https://reader033.fdocuments.us/reader033/viewer/2022043002/626b08c0662a4440e6283fc5/html5/thumbnails/1.jpg)
European Conference on Computer Vision 2006 : “Graph Cuts vs. Level Sets”, Y. Boykov (UWO), D. Cremers (U. of Bonn), V. Kolmogorov (UCL)
ECCV 2006 tutorial on
Graph Cuts vs. Level Sets
part III
Connecting Graph Cuts and Level Sets
Vladimir Kolmogorov
University College London
Yuri Boykov
University of Western Ontario
Daniel Cremers
University of Bonn
![Page 2: Connecting Graph Cuts and Level Sets](https://reader033.fdocuments.us/reader033/viewer/2022043002/626b08c0662a4440e6283fc5/html5/thumbnails/2.jpg)
European Conference on Computer Vision 2006 : “Graph Cuts vs. Level Sets”, Y. Boykov (UWO), D. Cremers (U. of Bonn), V. Kolmogorov (UCL)
Graph Cuts versus Level Sets
Part I: Basics of graph cuts
Part II: Basics of level-sets
Part III: Connecting graph cuts and level-sets
Part IV: Global vs. local optimization algorithms
![Page 3: Connecting Graph Cuts and Level Sets](https://reader033.fdocuments.us/reader033/viewer/2022043002/626b08c0662a4440e6283fc5/html5/thumbnails/3.jpg)
European Conference on Computer Vision 2006 : “Graph Cuts vs. Level Sets”, Y. Boykov (UWO), D. Cremers (U. of Bonn), V. Kolmogorov (UCL)
Graph Cuts versus Level Sets
Part III: Connecting graph cuts and level sets
• Minimal surfaces, global and local optima (VK)
• Integral and differential approaches (YB)
• Metrics on the space of contours, learning and shape
prior in graph cuts and level-sets (DC)
![Page 4: Connecting Graph Cuts and Level Sets](https://reader033.fdocuments.us/reader033/viewer/2022043002/626b08c0662a4440e6283fc5/html5/thumbnails/4.jpg)
European Conference on Computer Vision 2006 : “Graph Cuts vs. Level Sets”, Y. Boykov (UWO), D. Cremers (U. of Bonn), V. Kolmogorov (UCL)
Discrete vs. continuous functionals
Geodesic active contours
∑∑ +=qp
qppq
p
pp xxExEE,
),()()(x ∫=
C
dsNsCgCE )),(()(r
Graph cuts
Both can incorporate basic segmentation cues
• Image contrast
• Regional bias
• Alignment (flux)
![Page 5: Connecting Graph Cuts and Level Sets](https://reader033.fdocuments.us/reader033/viewer/2022043002/626b08c0662a4440e6283fc5/html5/thumbnails/5.jpg)
European Conference on Computer Vision 2006 : “Graph Cuts vs. Level Sets”, Y. Boykov (UWO), D. Cremers (U. of Bonn), V. Kolmogorov (UCL)
Incorporating image contrast:
graph cuts [Boykov&Jolly’01]
n-links
s
t a cuthard constraint
hard constraint
∆−=
22exp
σ
pq
pq
Iw
pqI∆
σ
![Page 6: Connecting Graph Cuts and Level Sets](https://reader033.fdocuments.us/reader033/viewer/2022043002/626b08c0662a4440e6283fc5/html5/thumbnails/6.jpg)
European Conference on Computer Vision 2006 : “Graph Cuts vs. Level Sets”, Y. Boykov (UWO), D. Cremers (U. of Bonn), V. Kolmogorov (UCL)
Riemannian metric
(space varying, tensor D(·))
Euclidian metric
(constant)
distance map distance
map
Incorporating image contrast:
geodesic active contours [Caselles,Kimmel,Sapiro’97]
∫ ⋅=
C
dsgCE )()(
Define Riemannian metric from image gradient
Compute geodesics
• shortest curve between two points
A
B
A
B
![Page 7: Connecting Graph Cuts and Level Sets](https://reader033.fdocuments.us/reader033/viewer/2022043002/626b08c0662a4440e6283fc5/html5/thumbnails/7.jpg)
European Conference on Computer Vision 2006 : “Graph Cuts vs. Level Sets”, Y. Boykov (UWO), D. Cremers (U. of Bonn), V. Kolmogorov (UCL)
Metrication errors on graphs
discrete metric ???
Minimum cost cut
(standard 4-neighborhoods)
Continuous metric space
(no geometric artifacts!)
Minimum length geodesic contour
(image-based Riemannian metric)
![Page 8: Connecting Graph Cuts and Level Sets](https://reader033.fdocuments.us/reader033/viewer/2022043002/626b08c0662a4440e6283fc5/html5/thumbnails/8.jpg)
European Conference on Computer Vision 2006 : “Graph Cuts vs. Level Sets”, Y. Boykov (UWO), D. Cremers (U. of Bonn), V. Kolmogorov (UCL)
C
“Cut metrics” and “Riemannian metrics” allow to compute
contour “length” in 2D (or “area” in 3D)
Cut Metrics :
cuts impose metric properties on graphs
∑∈
=Ce
ewC ||||
contour’s “length”
![Page 9: Connecting Graph Cuts and Level Sets](https://reader033.fdocuments.us/reader033/viewer/2022043002/626b08c0662a4440e6283fc5/html5/thumbnails/9.jpg)
European Conference on Computer Vision 2006 : “Graph Cuts vs. Level Sets”, Y. Boykov (UWO), D. Cremers (U. of Bonn), V. Kolmogorov (UCL)
C ∑∈
=Ce
ewC ||||
Geo-cuts [Boykov,Kolmogorov’03]:
Combining graph cuts and geodesic active contours
Given geometric functional, e.g. ∫=
C
dsNsCgCE )),(()(r
construct graph such that ∑∈
≡≈Ce
ewCCE ||||)(
![Page 10: Connecting Graph Cuts and Level Sets](https://reader033.fdocuments.us/reader033/viewer/2022043002/626b08c0662a4440e6283fc5/html5/thumbnails/10.jpg)
European Conference on Computer Vision 2006 : “Graph Cuts vs. Level Sets”, Y. Boykov (UWO), D. Cremers (U. of Bonn), V. Kolmogorov (UCL)
“Distance maps” for cut metric
“standard”
4-neighborhoods
(Manhattan metric)
256-neighborhoods8-neighborhoods
Euclidean metric
Riemannianmetric
D(p)=const
![Page 11: Connecting Graph Cuts and Level Sets](https://reader033.fdocuments.us/reader033/viewer/2022043002/626b08c0662a4440e6283fc5/html5/thumbnails/11.jpg)
European Conference on Computer Vision 2006 : “Graph Cuts vs. Level Sets”, Y. Boykov (UWO), D. Cremers (U. of Bonn), V. Kolmogorov (UCL)
What metrics can be approximated?
Question: What continuous functionals can be approximated with geo-cuts?
[Kolmogorov,Boykov’05]:
• Geometric length (e.g. Riemannian)– Distance map for g() is convex & symmetric
• Regional bias
• Flux of a given vector field
∫ ⋅=
C
dsgCE )()(
∫∫Ω
+ daf
∫ ⋅+
C
dsN )(rr
v
Ω
![Page 12: Connecting Graph Cuts and Level Sets](https://reader033.fdocuments.us/reader033/viewer/2022043002/626b08c0662a4440e6283fc5/html5/thumbnails/12.jpg)
European Conference on Computer Vision 2006 : “Graph Cuts vs. Level Sets”, Y. Boykov (UWO), D. Cremers (U. of Bonn), V. Kolmogorov (UCL)
Geometric measures used in level set segmentation[Acknowledgement: Ron Kimmel’s presentation]
weighted arc-length ∫ ⋅=
C
dsgCE )()(
functional evolution equation
( )NNggCt
rr ⋅∇−=
weighted area ∫∫Ω
= dafCE )( NfCt
r =
alignment(flux) ∫ ⋅=
C
dsNCE )()(rr
v NCt
rr )(div v−=
robust alignment ∫ ⋅−=
C
dsNCE ||)(rr
v NNCt
rrrr ))(divsign( vv ⋅=
Ω
![Page 13: Connecting Graph Cuts and Level Sets](https://reader033.fdocuments.us/reader033/viewer/2022043002/626b08c0662a4440e6283fc5/html5/thumbnails/13.jpg)
European Conference on Computer Vision 2006 : “Graph Cuts vs. Level Sets”, Y. Boykov (UWO), D. Cremers (U. of Bonn), V. Kolmogorov (UCL)
Flux
vector field: some vector defined at each point p
• “stream of water” with a given speed at each location
pvr
pvr
C1 ∫ ⋅=
C
dsNCflux )()(rr
v
flux: “amount of water” passing through a given contour
flux(C1) > flux(C2)
C2
n
n
• Changes sign with orientation
![Page 14: Connecting Graph Cuts and Level Sets](https://reader033.fdocuments.us/reader033/viewer/2022043002/626b08c0662a4440e6283fc5/html5/thumbnails/14.jpg)
European Conference on Computer Vision 2006 : “Graph Cuts vs. Level Sets”, Y. Boykov (UWO), D. Cremers (U. of Bonn), V. Kolmogorov (UCL)
Segmentation of thin objects [Vasilevskiy,Siddiqi’02]
• Vector field: I∇=vr
pvr
![Page 15: Connecting Graph Cuts and Level Sets](https://reader033.fdocuments.us/reader033/viewer/2022043002/626b08c0662a4440e6283fc5/html5/thumbnails/15.jpg)
European Conference on Computer Vision 2006 : “Graph Cuts vs. Level Sets”, Y. Boykov (UWO), D. Cremers (U. of Bonn), V. Kolmogorov (UCL)
Riemannian length + Flux [Kimmel,Bruckstein’03]
Riemannianlength
Flux of
Riemannianlength+Flux
I∇
![Page 16: Connecting Graph Cuts and Level Sets](https://reader033.fdocuments.us/reader033/viewer/2022043002/626b08c0662a4440e6283fc5/html5/thumbnails/16.jpg)
European Conference on Computer Vision 2006 : “Graph Cuts vs. Level Sets”, Y. Boykov (UWO), D. Cremers (U. of Bonn), V. Kolmogorov (UCL)
Robust alignment
∫ ⋅∇=
C
dsNICE )()(r
assumes bright object, dark background
∫ ⋅∇−=
C
dsNICE ||)(r
no such assumption
“Robust alignment”
[Kimmel,Bruckstein’03]
![Page 17: Connecting Graph Cuts and Level Sets](https://reader033.fdocuments.us/reader033/viewer/2022043002/626b08c0662a4440e6283fc5/html5/thumbnails/17.jpg)
European Conference on Computer Vision 2006 : “Graph Cuts vs. Level Sets”, Y. Boykov (UWO), D. Cremers (U. of Bonn), V. Kolmogorov (UCL)
Geometric measures used in level set segmentation[Acknowledgement: Ron Kimmel’s presentation]
weighted arc-length ∫ ⋅=
C
dsgCE )()(
functional evolution equation
( )NNggCt
rr ⋅∇−=
weighted area ∫∫Ω
= dafCE )( NfCt
r =
alignment(flux) ∫ ⋅=
C
dsNCE )()(rr
v NCt
rr )(div v−=
robust alignment ∫ ⋅−=
C
dsNCE ||)(rr
v NNCt
rrrr ))(divsign( vv ⋅=
Ωsame as in geo-cuts
non-submodular
![Page 18: Connecting Graph Cuts and Level Sets](https://reader033.fdocuments.us/reader033/viewer/2022043002/626b08c0662a4440e6283fc5/html5/thumbnails/18.jpg)
European Conference on Computer Vision 2006 : “Graph Cuts vs. Level Sets”, Y. Boykov (UWO), D. Cremers (U. of Bonn), V. Kolmogorov (UCL)
Graph cuts vs. level sets
for geodesic active contours
Graph Cuts Level Sets
Minimize the same functional(Geo-cuts)
global minimum local minimum
for the class of functionalsdiscussed previously
• Connection only approximate: E(C) ≈ ||C||
• Even stronger connection: continuous maxflow
![Page 19: Connecting Graph Cuts and Level Sets](https://reader033.fdocuments.us/reader033/viewer/2022043002/626b08c0662a4440e6283fc5/html5/thumbnails/19.jpg)
European Conference on Computer Vision 2006 : “Graph Cuts vs. Level Sets”, Y. Boykov (UWO), D. Cremers (U. of Bonn), V. Kolmogorov (UCL)
Continuous maxflow
[Iri’79],[Strang’83],[Appleton,Talbot’03]
• Analogue of discrete maxflow
• Solves continuous problem in subset of Rn
• Flow = vector field
![Page 20: Connecting Graph Cuts and Level Sets](https://reader033.fdocuments.us/reader033/viewer/2022043002/626b08c0662a4440e6283fc5/html5/thumbnails/20.jpg)
European Conference on Computer Vision 2006 : “Graph Cuts vs. Level Sets”, Y. Boykov (UWO), D. Cremers (U. of Bonn), V. Kolmogorov (UCL)
pqpq wf ≤
0=∑q
pqf
qppq ff −=
Discrete maxflow
Flow defined on graph edges
•
Capacity constraint:
Flow conservation:
(for p ≠ s,t)
Maximize flow out of the source(s):max→∑
q
sqf
4-neighbourhood system
![Page 21: Connecting Graph Cuts and Level Sets](https://reader033.fdocuments.us/reader033/viewer/2022043002/626b08c0662a4440e6283fc5/html5/thumbnails/21.jpg)
European Conference on Computer Vision 2006 : “Graph Cuts vs. Level Sets”, Y. Boykov (UWO), D. Cremers (U. of Bonn), V. Kolmogorov (UCL)
pqpq wf ≤
0=∑q
pqf
qppq ff −=
Discrete maxflow
Flow defined on graph edges
•
Capacity constraint:
Flow conservation:
(for p ≠ s,t)
[Ford&Fulkerson theorem]:
Maximum flow saturates minimum cut
![Page 22: Connecting Graph Cuts and Level Sets](https://reader033.fdocuments.us/reader033/viewer/2022043002/626b08c0662a4440e6283fc5/html5/thumbnails/22.jpg)
European Conference on Computer Vision 2006 : “Graph Cuts vs. Level Sets”, Y. Boykov (UWO), D. Cremers (U. of Bonn), V. Kolmogorov (UCL)
Continuous maxflow
Flow = vector field
Capacity constraint:
Flow conservation:
(for p ∉ s,t)
Maximize flow out of the source:
gf p ≤||r
0div =pfr
max )(div →∫s
p dafr
![Page 23: Connecting Graph Cuts and Level Sets](https://reader033.fdocuments.us/reader033/viewer/2022043002/626b08c0662a4440e6283fc5/html5/thumbnails/23.jpg)
European Conference on Computer Vision 2006 : “Graph Cuts vs. Level Sets”, Y. Boykov (UWO), D. Cremers (U. of Bonn), V. Kolmogorov (UCL)
Continuous maxflow
gf p ≤||r
Flow = vector field
Capacity constraint:
Flow conservation:
(for p ∉ s,t)
Maximum flow saturates minimum cut
Ngf pp
rr=
0div =pfr
( for p ∈ C* )
![Page 24: Connecting Graph Cuts and Level Sets](https://reader033.fdocuments.us/reader033/viewer/2022043002/626b08c0662a4440e6283fc5/html5/thumbnails/24.jpg)
European Conference on Computer Vision 2006 : “Graph Cuts vs. Level Sets”, Y. Boykov (UWO), D. Cremers (U. of Bonn), V. Kolmogorov (UCL)
Solving continuous maxflow
[Appleton,Talbot’03,06]: numerical algorithm
• Vector field stored on edges
– Horizontal edges => x-component
– Vertical edges => y-component
• Flow conservation similar to the discrete case
• But - capacity constraint:
• Report 0.1 pixels accuracy (on average)
xf1
yf2
xf4
yf3
222gff yx ≤+