Improved Moves for Truncated Convex Models
description
Transcript of Improved Moves for Truncated Convex Models
Improved Moves for Truncated Convex Models
M. Pawan Kumar
Philip Torr
AimEfficient, accurate MAP for truncated convex models
V1 V2 … … …
… … … … …
… … … … …
… … … … Vn
Random Variables V = { V1, V2, …, Vn}
Edges E define neighbourhood
Aim
Va Vb
li
lkab;ik
Accurate, efficient MAP for truncated convex models
ab;ik = wab min{ d(i-k), M }
ab;ik
i-k
wab is non-negative
Truncated Linear
i-k
ab;ik
Truncated Quadratic
d(.) is convexa;i b;k
MotivationLow-level Vision
• Smoothly varying regions
• Sharp edges between regions
min{ |i-k|, M}
Boykov, Veksler & Zabih 1998
Well-researched !!
Things We Know• NP-hard problem - Can only get approximation
• Best possible integrality gap - LP relaxation
Manokaran et al., 2008
• Solve using TRW-S, DD, PP
Slower than graph-cuts
• Use Range Move - Veksler, 2007
None of the guarantees of LP
Real MotivationGaps in Move-Making Literature
LPMove-Making
Potts
Truncated Linear
Truncated Quadratic
2
Multiplicative Bounds
2 + √2
O(√M)
Chekuri et al., 2001
Real MotivationGaps in Move-Making Literature
LPMove-Making
Potts
Truncated Linear
Truncated Quadratic
2
Multiplicative Bounds
2
2 + √2 2M
O(√M) -
Boykov, Veksler and Zabih, 1999
Real MotivationGaps in Move-Making Literature
LPMove-Making
Potts
Truncated Linear
Truncated Quadratic
2
Multiplicative Bounds
2
2 + √2 4
O(√M) -
Gupta and Tardos, 2000
Real MotivationGaps in Move-Making Literature
LPMove-Making
Potts
Truncated Linear
Truncated Quadratic
2
Multiplicative Bounds
2
2 + √2 4
O(√M) 2M
Komodakis and Tziritas, 2005
Real MotivationGaps in Move-Making Literature
LPMove-Making
Potts
Truncated Linear
Truncated Quadratic
2
Multiplicative Bounds
2
2 + √2
O(√M)
2 + √2
O(√M)
Outline
• Move Space
• Graph Construction
• Sketch of the Analysis
• Results
Move Space
Va Vb
• Initialize the labelling
• Choose interval I of L’ labels
• Each variable can
• Retain old label
• Choose a label from I
• Choose best labelling
Iterate over intervals
Outline
• Move Space
• Graph Construction
• Sketch of the Analysis
• Results
Two Problems
Va Vb
• Choose interval I of L’ labels
• Each variable can
• Retain old label
• Choose a label from I
• Choose best labelling
Large L’ => Non-submodular
Non-submodular
First Problem
Va Vb Submodular problem
Ishikawa, 2003; Veksler, 2007
First Problem
Va Vb Non-submodularProblem
First Problem
Va Vb Submodular problem
Veksler, 2007
First Problem
Va Vb
am+1
am+2
an
t
am+2
bm+1
bm+2
bn
bm+2
First Problem
Va Vb
am+1
am+2
an
t
am+2
bm+1
bm+2
bn
bm+2
First Problem
Va Vb
am+1
am+2
an
t
am+2
bm+1
bm+2
bn
bm+2
First Problem
Va Vb
am+1
am+2
an
t
am+2
bm+1
bm+2
bn
bm+2
First Problem
Va Vb
Model unary potentials exactly
am+1
am+2
an
t
am+2
bm+1
bm+2
bn
bm+2
First Problem
Va Vb
Similarly for Vb
am+1
am+2
an
t
am+2
bm+1
bm+2
bn
bm+2
First Problem
Va Vb
Model convex pairwise costs
am+1
am+2
an
t
am+2
bm+1
bm+2
bn
bm+2
First Problem
Va Vb
Overestimated pairwise potentials
Wanted to model
ab;ik = wab min{ d(i-k), M }
For all li, lk I
Have modelled
ab;ik = wab d(i-k)
For all li, lk I
Second Problem
Va Vb
• Choose interval I of L’ labels
• Each variable can
• Retain old label
• Choose a label from I
• Choose best labelling
Non-submodular problem !!
Second Problem - Case 1
Va Vb
am+1
am+2
an
t
bm+1
bm+2
bn
s∞ ∞
Both previous labels lie in interval
Second Problem - Case 1
Va Vb
am+1
am+2
an
t
bm+1
bm+2
bn
s∞ ∞
wab d(i-k)
Second Problem - Case 2
Va Vb
Only previous label of Va lies in interval
am+1
am+2
an
t
bm+1
bm+2
bn
s∞ ub
Second Problem - Case 2
Va Vb
am+1
am+2
an
t
bm+1
bm+2
bn
ub : unary potential of previous label of Vb
M
s∞ ub
Second Problem - Case 2
Va Vb
am+1
am+2
an
t
bm+1
bm+2
bn
M
wab d(i-k)
s∞ ub
Second Problem - Case 2
Va Vb
am+1
am+2
an
t
bm+1
bm+2
bn
M
wab ( d(i-m-1) + M )
s∞ ub
Second Problem - Case 3
Va Vb
am+1
am+2
an
t
bm+1
bm+2
bn
Only previous label of Vb lies in interval
Second Problem - Case 3
Va Vb
am+1
am+2
an
t
bm+1
bm+2
bn
sua
∞
ua : unary potential of previous label of Va
M
Second Problem - Case 4
Va Vb
am+1
am+2
an
t
bm+1
bm+2
bn
Both previous labels do not lie in interval
Second Problem - Case 4
Va Vb
am+1
am+2
an
t
bm+1
bm+2
bn
sua ub
Pab : pairwise potential for previous labels
ab
Pab
MM
Second Problem - Case 4
Va Vb
am+1
am+2
an
t
bm+1
bm+2
bn
wab d(i-k)
sua ub
ab
Pab
MM
Second Problem - Case 4
Va Vb
am+1
am+2
an
t
bm+1
bm+2
bn
wab ( d(i-m-1) + M )
sua ub
ab
Pab
MM
Second Problem - Case 4
Va Vb
am+1
am+2
an
t
bm+1
bm+2
bn
Pab
sua ub
ab
Pab
MM
Graph Construction
Va Vb
Find st-MINCUT. Retain old labellingif energy increases.
am+1
am+2
an
bm+1
bm+2
bn
t
ITERATE
Outline
• Move Space
• Graph Construction
• Sketch of the Analysis
• Results
Analysis
Va Vb
Current labelling f(.)
QC ≤ Q’C
Va Vb
Global Optimum f*(.)
QP
Previous labelling f’(.)
Va Vb
Analysis
Va Vb
Current labelling f(.)
QC ≤ Q’C
Va Vb
Partially Optimal f’’(.) Previous labelling f’(.)
Va Vb
Q’0≤
Analysis
Va Vb
Current labelling f(.)
QP - Q’C
Va Vb
Partially Optimal f’’(.) Previous labelling f’(.)
Va Vb
QP- Q’0≥
Analysis
Va Vb
Current labelling f(.)
QP - Q’C
Va Vb
Partially Optimal f’’(.) Local Optimal f’(.)
Va Vb
QP- Q’0≤ 0 ≤ 0
Analysis
Va Vb
Current labelling f(.)
Va Vb
Partially Optimal f’’(.) Local Optimal f’(.)
Va Vb
QP- Q’0 ≤ 0Take expectation over all intervals
AnalysisTruncated Linear
QP ≤ 2 + max 2M , L’L’ MQ*
L’ = M 4Gupta and Tardos, 2000
L’ = √2M 2 + √2
Truncated Quadratic
QP ≤ O(√M)Q*
L’ = √M
Outline
• Move Space
• Graph Construction
• Sketch of the Analysis
• Results
Synthetic Data - Truncated Linear
Faster than TRW-S Comparable to Range Moves
With LP Relaxation guarantees
Time (sec)
Energy
Synthetic Data - Truncated Quadratic
Faster than TRW-S Comparable to Range Moves
With LP Relaxation guarantees
Time (sec)
Energy
Stereo Correspondence
Disparity Map
Unary Potential: Similarity of pixel colour
Pairwise Potential: Truncated convex
Stereo Correspondence
Algo Energy1 Time1 Energy2 Time2
Swap 3678200 18.48 3707268 20.25
Exp 3677950 11.73 3687874 8.79
TRW-S 3677578 131.65 3679563 332.94
BP 3789486 272.06 5180705 331.36
Range 3686844 97.23 3679552 141.78
Our 3613003 120.14 3679552 191.20
Teddy
Stereo Correspondence
Algo Energy1 Time1 Energy2 Time2
Swap 3678200 18.48 3707268 20.25
Exp 3677950 11.73 3687874 8.79
TRW-S 3677578 131.65 3679563 332.94
BP 3789486 272.06 5180705 331.36
Range 3686844 97.23 3679552 141.78
Our 3613003 120.14 3679552 191.20
Teddy
Stereo Correspondence
Algo Energy1 Time1 Energy2 Time2
Swap 645227 28.86 709120 20.04
Exp 634931 9.52 723360 9.78
TRW-S 634720 94.86 651696 226.07
BP 662108 170.67 2155759 244.71
Range 634720 39.75 651696 80.40
Our 634720 66.13 651696 80.70
Tsukuba
Summary
• Moves that give LP guarantees
• Similar results to TRW-S
• Faster than TRW-S because of graph cuts
Questions Not Yet Answered
• Move-making gives LP guarantees– True for all MAP estimation problems?
• Huber function? Parallel Imaging Problem?
• Primal-dual method?
• Solving more complex relaxations?
Questions?
Improved Moves for Truncated Convex Models
Kumar and Torr, NIPS 2008
http://www.robots.ox.ac.uk/~pawan/