Solving Markov Random Fields using Second Order Cone Programming Relaxations M. Pawan Kumar Philip...
-
Upload
mary-price -
Category
Documents
-
view
215 -
download
0
Transcript of Solving Markov Random Fields using Second Order Cone Programming Relaxations M. Pawan Kumar Philip...
![Page 1: Solving Markov Random Fields using Second Order Cone Programming Relaxations M. Pawan Kumar Philip Torr Andrew Zisserman.](https://reader035.fdocuments.us/reader035/viewer/2022062619/5515eb3f550346dd6f8b5179/html5/thumbnails/1.jpg)
Solving Markov Random Fields using
Second Order Cone Programming Relaxations
M. Pawan Kumar
Philip Torr
Andrew Zisserman
![Page 2: Solving Markov Random Fields using Second Order Cone Programming Relaxations M. Pawan Kumar Philip Torr Andrew Zisserman.](https://reader035.fdocuments.us/reader035/viewer/2022062619/5515eb3f550346dd6f8b5179/html5/thumbnails/2.jpg)
Aim• Accurate MAP estimation of pairwise Markov random fields
2
5
4
2
6
3
3
7
0
1 1
0
0
2 3
1
1
4 1
0
V1 V2 V3 V4
Label ‘0’
Label ‘1’
Labelling m = {1, 0, 0, 1}
Random Variables V = {V1,..,V4}
Label Set L = {0,1}
![Page 3: Solving Markov Random Fields using Second Order Cone Programming Relaxations M. Pawan Kumar Philip Torr Andrew Zisserman.](https://reader035.fdocuments.us/reader035/viewer/2022062619/5515eb3f550346dd6f8b5179/html5/thumbnails/3.jpg)
Aim• Accurate MAP estimation of pairwise Markov random fields
2
5
4
2
6
3
3
7
0
1 1
0
0
2 3
1
1
4 1
0
V1 V2 V3 V4
Label ‘0’
Label ‘1’
Cost(m) = 2
![Page 4: Solving Markov Random Fields using Second Order Cone Programming Relaxations M. Pawan Kumar Philip Torr Andrew Zisserman.](https://reader035.fdocuments.us/reader035/viewer/2022062619/5515eb3f550346dd6f8b5179/html5/thumbnails/4.jpg)
Aim• Accurate MAP estimation of pairwise Markov random fields
2
5
4
2
6
3
3
7
0
1 1
0
0
2 3
1
1
4 1
0
V1 V2 V3 V4
Label ‘0’
Label ‘1’
Cost(m) = 2 + 1
![Page 5: Solving Markov Random Fields using Second Order Cone Programming Relaxations M. Pawan Kumar Philip Torr Andrew Zisserman.](https://reader035.fdocuments.us/reader035/viewer/2022062619/5515eb3f550346dd6f8b5179/html5/thumbnails/5.jpg)
Aim• Accurate MAP estimation of pairwise Markov random fields
2
5
4
2
6
3
3
7
0
1 1
0
0
2 3
1
1
4 1
0
V1 V2 V3 V4
Label ‘0’
Label ‘1’
Cost(m) = 2 + 1 + 2
![Page 6: Solving Markov Random Fields using Second Order Cone Programming Relaxations M. Pawan Kumar Philip Torr Andrew Zisserman.](https://reader035.fdocuments.us/reader035/viewer/2022062619/5515eb3f550346dd6f8b5179/html5/thumbnails/6.jpg)
Aim• Accurate MAP estimation of pairwise Markov random fields
2
5
4
2
6
3
3
7
0
1 1
0
0
2 3
1
1
4 1
0
V1 V2 V3 V4
Label ‘0’
Label ‘1’
Cost(m) = 2 + 1 + 2 + 1
![Page 7: Solving Markov Random Fields using Second Order Cone Programming Relaxations M. Pawan Kumar Philip Torr Andrew Zisserman.](https://reader035.fdocuments.us/reader035/viewer/2022062619/5515eb3f550346dd6f8b5179/html5/thumbnails/7.jpg)
Aim• Accurate MAP estimation of pairwise Markov random fields
2
5
4
2
6
3
3
7
0
1 1
0
0
2 3
1
1
4 1
0
V1 V2 V3 V4
Label ‘0’
Label ‘1’
Cost(m) = 2 + 1 + 2 + 1 + 3
![Page 8: Solving Markov Random Fields using Second Order Cone Programming Relaxations M. Pawan Kumar Philip Torr Andrew Zisserman.](https://reader035.fdocuments.us/reader035/viewer/2022062619/5515eb3f550346dd6f8b5179/html5/thumbnails/8.jpg)
Aim• Accurate MAP estimation of pairwise Markov random fields
2
5
4
2
6
3
3
7
0
1 1
0
0
2 3
1
1
4 1
0
V1 V2 V3 V4
Label ‘0’
Label ‘1’
Cost(m) = 2 + 1 + 2 + 1 + 3 + 1
![Page 9: Solving Markov Random Fields using Second Order Cone Programming Relaxations M. Pawan Kumar Philip Torr Andrew Zisserman.](https://reader035.fdocuments.us/reader035/viewer/2022062619/5515eb3f550346dd6f8b5179/html5/thumbnails/9.jpg)
Aim• Accurate MAP estimation of pairwise Markov random fields
2
5
4
2
6
3
3
7
0
1 1
0
0
2 3
1
1
4 1
0
V1 V2 V3 V4
Label ‘0’
Label ‘1’
Cost(m) = 2 + 1 + 2 + 1 + 3 + 1 + 3
![Page 10: Solving Markov Random Fields using Second Order Cone Programming Relaxations M. Pawan Kumar Philip Torr Andrew Zisserman.](https://reader035.fdocuments.us/reader035/viewer/2022062619/5515eb3f550346dd6f8b5179/html5/thumbnails/10.jpg)
Aim• Accurate MAP estimation of pairwise Markov random fields
2
5
4
2
6
3
3
7
0
1 1
0
0
2 3
1
1
4 1
0
V1 V2 V3 V4
Label ‘0’
Label ‘1’
Cost(m) = 2 + 1 + 2 + 1 + 3 + 1 + 3 = 13
Minimum Cost Labelling = MAP estimate
Pr(m) exp(-Cost(m))
![Page 11: Solving Markov Random Fields using Second Order Cone Programming Relaxations M. Pawan Kumar Philip Torr Andrew Zisserman.](https://reader035.fdocuments.us/reader035/viewer/2022062619/5515eb3f550346dd6f8b5179/html5/thumbnails/11.jpg)
Aim• Accurate MAP estimation of pairwise Markov random fields
2
5
4
2
6
3
3
7
0
1 1
0
0
2 3
1
1
4 1
0
V1 V2 V3 V4
Label ‘0’
Label ‘1’
Objectives• Applicable for all neighbourhood relationships• Applicable for all forms of pairwise costs• Guaranteed to converge
![Page 12: Solving Markov Random Fields using Second Order Cone Programming Relaxations M. Pawan Kumar Philip Torr Andrew Zisserman.](https://reader035.fdocuments.us/reader035/viewer/2022062619/5515eb3f550346dd6f8b5179/html5/thumbnails/12.jpg)
MotivationSubgraph Matching - Torr - 2003, Schellewald et al - 2005
G1
G2
Unary costs are uniform
V2 V3
V1
MRF
ABCD A
BCD
ABCD
![Page 13: Solving Markov Random Fields using Second Order Cone Programming Relaxations M. Pawan Kumar Philip Torr Andrew Zisserman.](https://reader035.fdocuments.us/reader035/viewer/2022062619/5515eb3f550346dd6f8b5179/html5/thumbnails/13.jpg)
MotivationSubgraph Matching - Torr - 2003, Schellewald et al - 2005
G1
G2
| d(mi,mj) - d(Vi,Vj) | <
12
YES NO
Potts Model
Pairwise Costs
![Page 14: Solving Markov Random Fields using Second Order Cone Programming Relaxations M. Pawan Kumar Philip Torr Andrew Zisserman.](https://reader035.fdocuments.us/reader035/viewer/2022062619/5515eb3f550346dd6f8b5179/html5/thumbnails/14.jpg)
Motivation
V2 V3
V1
MRF
ABCD A
BCD
ABCD
Subgraph Matching - Torr - 2003, Schellewald et al - 2005
![Page 15: Solving Markov Random Fields using Second Order Cone Programming Relaxations M. Pawan Kumar Philip Torr Andrew Zisserman.](https://reader035.fdocuments.us/reader035/viewer/2022062619/5515eb3f550346dd6f8b5179/html5/thumbnails/15.jpg)
Motivation
V2 V3
V1
MRF
ABCD A
BCD
ABCD
Subgraph Matching - Torr - 2003, Schellewald et al - 2005
![Page 16: Solving Markov Random Fields using Second Order Cone Programming Relaxations M. Pawan Kumar Philip Torr Andrew Zisserman.](https://reader035.fdocuments.us/reader035/viewer/2022062619/5515eb3f550346dd6f8b5179/html5/thumbnails/16.jpg)
MotivationMatching Pictorial Structures - Felzenszwalb et al - 2001
Part likelihood Spatial Prior
Outline
Texture
Image
P1 P3
P2
(x,y,,)
MRF
![Page 17: Solving Markov Random Fields using Second Order Cone Programming Relaxations M. Pawan Kumar Philip Torr Andrew Zisserman.](https://reader035.fdocuments.us/reader035/viewer/2022062619/5515eb3f550346dd6f8b5179/html5/thumbnails/17.jpg)
Motivation
Image
P1 P3
P2
(x,y,,)
MRF
• Unary potentials are negative log likelihoods
Valid pairwise configuration
Potts Model
Matching Pictorial Structures - Felzenszwalb et al - 2001
12
YES NO
![Page 18: Solving Markov Random Fields using Second Order Cone Programming Relaxations M. Pawan Kumar Philip Torr Andrew Zisserman.](https://reader035.fdocuments.us/reader035/viewer/2022062619/5515eb3f550346dd6f8b5179/html5/thumbnails/18.jpg)
Motivation
P1 P3
P2
(x,y,,)
Pr(Cow)Image
• Unary potentials are negative log likelihoodsMatching Pictorial Structures - Felzenszwalb et al - 2001
Valid pairwise configuration
Potts Model
12
YES NO
![Page 19: Solving Markov Random Fields using Second Order Cone Programming Relaxations M. Pawan Kumar Philip Torr Andrew Zisserman.](https://reader035.fdocuments.us/reader035/viewer/2022062619/5515eb3f550346dd6f8b5179/html5/thumbnails/19.jpg)
Outline• Integer Programming Formulation
• Previous Work
• Our Approach– Second Order Cone Programming (SOCP)– SOCP Relaxation– Robust Truncated Model
• Applications– Subgraph Matching– Pictorial Structures
![Page 20: Solving Markov Random Fields using Second Order Cone Programming Relaxations M. Pawan Kumar Philip Torr Andrew Zisserman.](https://reader035.fdocuments.us/reader035/viewer/2022062619/5515eb3f550346dd6f8b5179/html5/thumbnails/20.jpg)
Integer Programming Formulation2
5
4
2
0
1 3
0
V1 V2
Label ‘0’
Label ‘1’Unary Cost
Unary Cost Vector u = [ 5
Cost of V1 = 0
2
Cost of V1 = 1
; 2 4 ]
Labelling m = {1 , 0}
![Page 21: Solving Markov Random Fields using Second Order Cone Programming Relaxations M. Pawan Kumar Philip Torr Andrew Zisserman.](https://reader035.fdocuments.us/reader035/viewer/2022062619/5515eb3f550346dd6f8b5179/html5/thumbnails/21.jpg)
Integer Programming Formulation2
5
4
2
0
1 3
0
V1 V2
Label ‘0’
Label ‘1’Unary Cost
Unary Cost Vector u = [ 5 2 ; 2 4 ]T
Labelling m = {1 , 0}
Label vector x = [ -1
V1 0
1
V1 = 1
; 1 -1 ]T
Recall that the aim is to find the optimal x
![Page 22: Solving Markov Random Fields using Second Order Cone Programming Relaxations M. Pawan Kumar Philip Torr Andrew Zisserman.](https://reader035.fdocuments.us/reader035/viewer/2022062619/5515eb3f550346dd6f8b5179/html5/thumbnails/22.jpg)
Integer Programming Formulation2
5
4
2
0
1 3
0
V1 V2
Label ‘0’
Label ‘1’Unary Cost
Unary Cost Vector u = [ 5 2 ; 2 4 ]T
Labelling m = {1 , 0}
Label vector x = [ -1 1 ; 1 -1 ]T
Sum of Unary Costs = 12
∑i ui (1 + xi)
![Page 23: Solving Markov Random Fields using Second Order Cone Programming Relaxations M. Pawan Kumar Philip Torr Andrew Zisserman.](https://reader035.fdocuments.us/reader035/viewer/2022062619/5515eb3f550346dd6f8b5179/html5/thumbnails/23.jpg)
Integer Programming Formulation2
5
4
2
0
1 3
0
V1 V2
Label ‘0’
Label ‘1’Pairwise Cost
Labelling m = {1 , 0}
0Cost of V1 = 0 and V1 = 0
0
00
0Cost of V1 = 0 and V2 = 0
3
Cost of V1 = 0 and V2 = 11 0
00
0 0
10
3 0
Pairwise Cost Matrix P
![Page 24: Solving Markov Random Fields using Second Order Cone Programming Relaxations M. Pawan Kumar Philip Torr Andrew Zisserman.](https://reader035.fdocuments.us/reader035/viewer/2022062619/5515eb3f550346dd6f8b5179/html5/thumbnails/24.jpg)
Integer Programming Formulation2
5
4
2
0
1 3
0
V1 V2
Label ‘0’
Label ‘1’Pairwise Cost
Labelling m = {1 , 0}
Pairwise Cost Matrix P
0 0
00
0 3
1 0
00
0 0
10
3 0
Sum of Pairwise Costs14
∑ij Pij (1 + xi)(1+xj)
![Page 25: Solving Markov Random Fields using Second Order Cone Programming Relaxations M. Pawan Kumar Philip Torr Andrew Zisserman.](https://reader035.fdocuments.us/reader035/viewer/2022062619/5515eb3f550346dd6f8b5179/html5/thumbnails/25.jpg)
Integer Programming Formulation2
5
4
2
0
1 3
0
V1 V2
Label ‘0’
Label ‘1’Pairwise Cost
Labelling m = {1 , 0}
Pairwise Cost Matrix P
0 0
00
0 3
1 0
00
0 0
10
3 0
Sum of Pairwise Costs14
∑ij Pij (1 + xi +xj + xixj)
14
∑ij Pij (1 + xi + xj + Xij)=
X = x xT Xij = xi xj
![Page 26: Solving Markov Random Fields using Second Order Cone Programming Relaxations M. Pawan Kumar Philip Torr Andrew Zisserman.](https://reader035.fdocuments.us/reader035/viewer/2022062619/5515eb3f550346dd6f8b5179/html5/thumbnails/26.jpg)
Integer Programming FormulationConstraints
• Each variable should be assigned a unique label
∑ xi = 2 - |L|i Va
• Marginalization constraint
∑ Xij = (2 - |L|) xij Vb
![Page 27: Solving Markov Random Fields using Second Order Cone Programming Relaxations M. Pawan Kumar Philip Torr Andrew Zisserman.](https://reader035.fdocuments.us/reader035/viewer/2022062619/5515eb3f550346dd6f8b5179/html5/thumbnails/27.jpg)
Integer Programming FormulationChekuri et al. , SODA 2001
x* = argmin 12
∑ ui (1 + xi) + 14
∑ Pij (1 + xi + xj + Xij)
∑ xi = 2 - |L|i Va
∑ Xij = (2 - |L|) xij Vb
xi {-1,1}
X = x xT
ConvexNon-Convex
![Page 28: Solving Markov Random Fields using Second Order Cone Programming Relaxations M. Pawan Kumar Philip Torr Andrew Zisserman.](https://reader035.fdocuments.us/reader035/viewer/2022062619/5515eb3f550346dd6f8b5179/html5/thumbnails/28.jpg)
Outline• Integer Programming Formulation
• Previous Work
• Our Approach– Second Order Cone Programming (SOCP)– SOCP Relaxation– Robust Truncated Model
• Applications– Subgraph Matching– Pictorial Structures
![Page 29: Solving Markov Random Fields using Second Order Cone Programming Relaxations M. Pawan Kumar Philip Torr Andrew Zisserman.](https://reader035.fdocuments.us/reader035/viewer/2022062619/5515eb3f550346dd6f8b5179/html5/thumbnails/29.jpg)
Linear Programming Formulation
x* = argmin 12
∑ ui (1 + xi) + 14
∑ Pij (1 + xi + xj + Xij)
∑ xi = 2 - |L|i Va
∑ Xij = (2 - |L|) xij Vb
xi {-1,1}
X = x xT
Chekuri et al. , SODA 2001Retain Convex Part
Relax Non-convex Constraint
![Page 30: Solving Markov Random Fields using Second Order Cone Programming Relaxations M. Pawan Kumar Philip Torr Andrew Zisserman.](https://reader035.fdocuments.us/reader035/viewer/2022062619/5515eb3f550346dd6f8b5179/html5/thumbnails/30.jpg)
Linear Programming Formulation
x* = argmin 12
∑ ui (1 + xi) + 14
∑ Pij (1 + xi + xj + Xij)
∑ xi = 2 - |L|i Va
∑ Xij = (2 - |L|) xij Vb
xi [-1,1]
X = x xT
Chekuri et al. , SODA 2001Retain Convex Part
Relax Non-convex Constraint
![Page 31: Solving Markov Random Fields using Second Order Cone Programming Relaxations M. Pawan Kumar Philip Torr Andrew Zisserman.](https://reader035.fdocuments.us/reader035/viewer/2022062619/5515eb3f550346dd6f8b5179/html5/thumbnails/31.jpg)
Linear Programming Formulation
x* = argmin 12
∑ ui (1 + xi) + 14
∑ Pij (1 + xi + xj + Xij)
∑ xi = 2 - |L|i Va
∑ Xij = (2 - |L|) xij Vb
xi [-1,1]
Chekuri et al. , SODA 2001Retain Convex Part
![Page 32: Solving Markov Random Fields using Second Order Cone Programming Relaxations M. Pawan Kumar Philip Torr Andrew Zisserman.](https://reader035.fdocuments.us/reader035/viewer/2022062619/5515eb3f550346dd6f8b5179/html5/thumbnails/32.jpg)
Feasible Region (IP) x {-1,1}, X = x2
Linear Programming Formulation
![Page 33: Solving Markov Random Fields using Second Order Cone Programming Relaxations M. Pawan Kumar Philip Torr Andrew Zisserman.](https://reader035.fdocuments.us/reader035/viewer/2022062619/5515eb3f550346dd6f8b5179/html5/thumbnails/33.jpg)
Feasible Region (IP)Feasible Region (Relaxation 1)
x {-1,1}, X = x2
x [-1,1], X = x2
Linear Programming Formulation
![Page 34: Solving Markov Random Fields using Second Order Cone Programming Relaxations M. Pawan Kumar Philip Torr Andrew Zisserman.](https://reader035.fdocuments.us/reader035/viewer/2022062619/5515eb3f550346dd6f8b5179/html5/thumbnails/34.jpg)
Feasible Region (IP)Feasible Region (Relaxation 1)Feasible Region (Relaxation 2)
x {-1,1}, X = x2
x [-1,1], X = x2
x [-1,1]
Linear Programming Formulation
![Page 35: Solving Markov Random Fields using Second Order Cone Programming Relaxations M. Pawan Kumar Philip Torr Andrew Zisserman.](https://reader035.fdocuments.us/reader035/viewer/2022062619/5515eb3f550346dd6f8b5179/html5/thumbnails/35.jpg)
Linear Programming Formulation
• Bounded algorithms proposed by Chekuri et al, SODA 2001
• -expansion - Komodakis and Tziritas, ICCV 2005
• TRW - Wainwright et al., NIPS 2002
• TRW-S - Kolmogorov, AISTATS 2005
• Efficient because it uses Linear Programming
• Not accurate
![Page 36: Solving Markov Random Fields using Second Order Cone Programming Relaxations M. Pawan Kumar Philip Torr Andrew Zisserman.](https://reader035.fdocuments.us/reader035/viewer/2022062619/5515eb3f550346dd6f8b5179/html5/thumbnails/36.jpg)
Semidefinite Programming Formulation
x* = argmin 12
∑ ui (1 + xi) + 14
∑ Pij (1 + xi + xj + Xij)
∑ xi = 2 - |L|i Va
∑ Xij = (2 - |L|) xij Vb
xi {-1,1}
X = x xT
Lovasz and Schrijver, SIAM Optimization, 1990Retain Convex Part
Relax Non-convex Constraint
![Page 37: Solving Markov Random Fields using Second Order Cone Programming Relaxations M. Pawan Kumar Philip Torr Andrew Zisserman.](https://reader035.fdocuments.us/reader035/viewer/2022062619/5515eb3f550346dd6f8b5179/html5/thumbnails/37.jpg)
x* = argmin 12
∑ ui (1 + xi) + 14
∑ Pij (1 + xi + xj + Xij)
∑ xi = 2 - |L|i Va
∑ Xij = (2 - |L|) xij Vb
xi [-1,1]
X = x xT
Semidefinite Programming Formulation
Retain Convex Part
Relax Non-convex Constraint
Lovasz and Schrijver, SIAM Optimization, 1990
![Page 38: Solving Markov Random Fields using Second Order Cone Programming Relaxations M. Pawan Kumar Philip Torr Andrew Zisserman.](https://reader035.fdocuments.us/reader035/viewer/2022062619/5515eb3f550346dd6f8b5179/html5/thumbnails/38.jpg)
Semidefinite Programming Formulation
x1
x2
xn
1
...
1 x1 x2... xn
1 xT
x X
=
Rank = 1
Xii = 1
Positive SemidefiniteConvex
Non-Convex
![Page 39: Solving Markov Random Fields using Second Order Cone Programming Relaxations M. Pawan Kumar Philip Torr Andrew Zisserman.](https://reader035.fdocuments.us/reader035/viewer/2022062619/5515eb3f550346dd6f8b5179/html5/thumbnails/39.jpg)
Semidefinite Programming Formulation
x1
x2
xn
1
...
1 x1 x2... xn
1 xT
x X
=
Xii = 1
Positive SemidefiniteConvex
![Page 40: Solving Markov Random Fields using Second Order Cone Programming Relaxations M. Pawan Kumar Philip Torr Andrew Zisserman.](https://reader035.fdocuments.us/reader035/viewer/2022062619/5515eb3f550346dd6f8b5179/html5/thumbnails/40.jpg)
Schur’s Complement
A B
BT C
=I 0
BTA-1 I
A 0
0 C - BTA-1B
I A-1B
0 I
0
A 0 C -BTA-1B 0
![Page 41: Solving Markov Random Fields using Second Order Cone Programming Relaxations M. Pawan Kumar Philip Torr Andrew Zisserman.](https://reader035.fdocuments.us/reader035/viewer/2022062619/5515eb3f550346dd6f8b5179/html5/thumbnails/41.jpg)
Semidefinite Programming Formulation
X - xxT 0
1 xT
x X
=1 0
x I
1 0
0 X - xxT
I xT
0 1
Schur’s Complement
![Page 42: Solving Markov Random Fields using Second Order Cone Programming Relaxations M. Pawan Kumar Philip Torr Andrew Zisserman.](https://reader035.fdocuments.us/reader035/viewer/2022062619/5515eb3f550346dd6f8b5179/html5/thumbnails/42.jpg)
x* = argmin 12
∑ ui (1 + xi) + 14
∑ Pij (1 + xi + xj + Xij)
∑ xi = 2 - |L|i Va
∑ Xij = (2 - |L|) xij Vb
xi [-1,1]
X = x xT
Semidefinite Programming Formulation
Relax Non-convex Constraint
Retain Convex PartLovasz and Schrijver, SIAM Optimization, 1990
![Page 43: Solving Markov Random Fields using Second Order Cone Programming Relaxations M. Pawan Kumar Philip Torr Andrew Zisserman.](https://reader035.fdocuments.us/reader035/viewer/2022062619/5515eb3f550346dd6f8b5179/html5/thumbnails/43.jpg)
x* = argmin 12
∑ ui (1 + xi) + 14
∑ Pij (1 + xi + xj + Xij)
∑ xi = 2 - |L|i Va
∑ Xij = (2 - |L|) xij Vb
xi [-1,1]
Semidefinite Programming Formulation
Xii = 1 X - xxT 0
Retain Convex PartLovasz and Schrijver, SIAM Optimization, 1990
![Page 44: Solving Markov Random Fields using Second Order Cone Programming Relaxations M. Pawan Kumar Philip Torr Andrew Zisserman.](https://reader035.fdocuments.us/reader035/viewer/2022062619/5515eb3f550346dd6f8b5179/html5/thumbnails/44.jpg)
Feasible Region (IP) x {-1,1}, X = x2
Semidefinite Programming Formulation
![Page 45: Solving Markov Random Fields using Second Order Cone Programming Relaxations M. Pawan Kumar Philip Torr Andrew Zisserman.](https://reader035.fdocuments.us/reader035/viewer/2022062619/5515eb3f550346dd6f8b5179/html5/thumbnails/45.jpg)
Feasible Region (IP)Feasible Region (Relaxation 1)
x {-1,1}, X = x2
x [-1,1], X = x2
Semidefinite Programming Formulation
![Page 46: Solving Markov Random Fields using Second Order Cone Programming Relaxations M. Pawan Kumar Philip Torr Andrew Zisserman.](https://reader035.fdocuments.us/reader035/viewer/2022062619/5515eb3f550346dd6f8b5179/html5/thumbnails/46.jpg)
Feasible Region (IP)Feasible Region (Relaxation 1)Feasible Region (Relaxation 2)
x {-1,1}, X = x2
x [-1,1], X = x2
x [-1,1], X x2
Semidefinite Programming Formulation
![Page 47: Solving Markov Random Fields using Second Order Cone Programming Relaxations M. Pawan Kumar Philip Torr Andrew Zisserman.](https://reader035.fdocuments.us/reader035/viewer/2022062619/5515eb3f550346dd6f8b5179/html5/thumbnails/47.jpg)
Semidefinite Programming Formulation
• Formulated by Lovasz and Schrijver, 1990
• Finds a full X matrix
• Max-cut - Goemans and Williamson, JACM 1995
• Max-k-cut - de Klerk et al, 2000
• Accurate
• Not efficient because of Semidefinite Programming
![Page 48: Solving Markov Random Fields using Second Order Cone Programming Relaxations M. Pawan Kumar Philip Torr Andrew Zisserman.](https://reader035.fdocuments.us/reader035/viewer/2022062619/5515eb3f550346dd6f8b5179/html5/thumbnails/48.jpg)
Previous Work - Overview
LP SDP
ExamplesTRW-S,
-expansion
Max-k-Cut
Accuracy Low High
Efficiency High Low
Is there a Middle Path ???
![Page 49: Solving Markov Random Fields using Second Order Cone Programming Relaxations M. Pawan Kumar Philip Torr Andrew Zisserman.](https://reader035.fdocuments.us/reader035/viewer/2022062619/5515eb3f550346dd6f8b5179/html5/thumbnails/49.jpg)
Outline• Integer Programming Formulation
• Previous Work
• Our Approach– Second Order Cone Programming (SOCP)– SOCP Relaxation– Robust Truncated Model
• Applications– Subgraph Matching– Pictorial Structures
![Page 50: Solving Markov Random Fields using Second Order Cone Programming Relaxations M. Pawan Kumar Philip Torr Andrew Zisserman.](https://reader035.fdocuments.us/reader035/viewer/2022062619/5515eb3f550346dd6f8b5179/html5/thumbnails/50.jpg)
Second Order Cone Programming
Second Order Cone || v || t OR || v ||2 st
x2 + y2 z2
![Page 51: Solving Markov Random Fields using Second Order Cone Programming Relaxations M. Pawan Kumar Philip Torr Andrew Zisserman.](https://reader035.fdocuments.us/reader035/viewer/2022062619/5515eb3f550346dd6f8b5179/html5/thumbnails/51.jpg)
Minimize fTx
Subject to || Ai x+ bi || <= ciT x + di
i = 1, … , L
Linear Objective Function
Affine mapping of Second Order Cone (SOC)
Constraints are SOC of ni dimensions
Feasible regions are intersections of conic regions
Second Order Cone Programming
![Page 52: Solving Markov Random Fields using Second Order Cone Programming Relaxations M. Pawan Kumar Philip Torr Andrew Zisserman.](https://reader035.fdocuments.us/reader035/viewer/2022062619/5515eb3f550346dd6f8b5179/html5/thumbnails/52.jpg)
Second Order Cone Programming
|| v || t tI v
vT t0
LP SOCP SDP
=1 0
vT I
tI 0
0 t2 - vTv
I v
0 1
Schur’s Complement
![Page 53: Solving Markov Random Fields using Second Order Cone Programming Relaxations M. Pawan Kumar Philip Torr Andrew Zisserman.](https://reader035.fdocuments.us/reader035/viewer/2022062619/5515eb3f550346dd6f8b5179/html5/thumbnails/53.jpg)
Outline• Integer Programming Formulation
• Previous Work
• Our Approach– Second Order Cone Programming (SOCP)– SOCP Relaxation– Robust Truncated Model
• Applications– Subgraph Matching– Pictorial Structures
![Page 54: Solving Markov Random Fields using Second Order Cone Programming Relaxations M. Pawan Kumar Philip Torr Andrew Zisserman.](https://reader035.fdocuments.us/reader035/viewer/2022062619/5515eb3f550346dd6f8b5179/html5/thumbnails/54.jpg)
Matrix Dot Product
A B = ∑ij Aij Bij
A11 A12
A21 A22
B11 B12
B21 B22
= A11 B11 + A12 B12 + A21 B21 + A22 B22
![Page 55: Solving Markov Random Fields using Second Order Cone Programming Relaxations M. Pawan Kumar Philip Torr Andrew Zisserman.](https://reader035.fdocuments.us/reader035/viewer/2022062619/5515eb3f550346dd6f8b5179/html5/thumbnails/55.jpg)
SDP Relaxation
x* = argmin 12
∑ ui (1 + xi) + 14
∑ Pij (1 + xi + xj + Xij)
∑ xi = 2 - |L|i Va
∑ Xij = (2 - |L|) xij Vb
xi [-1,1]
Xii = 1 X - xxT 0
Derive SOCP relaxation from the SDP relaxation
FurtherRelaxation
![Page 56: Solving Markov Random Fields using Second Order Cone Programming Relaxations M. Pawan Kumar Philip Torr Andrew Zisserman.](https://reader035.fdocuments.us/reader035/viewer/2022062619/5515eb3f550346dd6f8b5179/html5/thumbnails/56.jpg)
1-D ExampleX - xxT 0
X - x2 ≥ 0
For two semidefinite matrices, the dot product is non-negative
A A 0
x2 X
SOC of the form || v ||2 st
![Page 57: Solving Markov Random Fields using Second Order Cone Programming Relaxations M. Pawan Kumar Philip Torr Andrew Zisserman.](https://reader035.fdocuments.us/reader035/viewer/2022062619/5515eb3f550346dd6f8b5179/html5/thumbnails/57.jpg)
Feasible Region (IP)Feasible Region (Relaxation 1)Feasible Region (Relaxation 2)
x {-1,1}, X = x2
x [-1,1], X = x2
x [-1,1], X x2
SOCP Relaxation
Same as the SDP formulation
![Page 58: Solving Markov Random Fields using Second Order Cone Programming Relaxations M. Pawan Kumar Philip Torr Andrew Zisserman.](https://reader035.fdocuments.us/reader035/viewer/2022062619/5515eb3f550346dd6f8b5179/html5/thumbnails/58.jpg)
2-D Example
X11 X12
X21 X22
1 X12
X12 1
=X =
x1x1 x1x2
x2x1 x2x2
xxT =x1
2 x1x2
x1x2
=x2
2
![Page 59: Solving Markov Random Fields using Second Order Cone Programming Relaxations M. Pawan Kumar Philip Torr Andrew Zisserman.](https://reader035.fdocuments.us/reader035/viewer/2022062619/5515eb3f550346dd6f8b5179/html5/thumbnails/59.jpg)
2-D Example(X - xxT)
1 - x12 X12-x1x2. 0
1 0
0 0 X12-x1x2 1 - x22
x12 1
-1 x1 1
C1. 0 C1 0
![Page 60: Solving Markov Random Fields using Second Order Cone Programming Relaxations M. Pawan Kumar Philip Torr Andrew Zisserman.](https://reader035.fdocuments.us/reader035/viewer/2022062619/5515eb3f550346dd6f8b5179/html5/thumbnails/60.jpg)
2-D Example(X - xxT)
1 - x12 X12-x1x2
C2. 0
. 00 0
0 1 X12-x1x2 1 - x22
x22 1
LP Relaxation-1 x2 1
C2 0
![Page 61: Solving Markov Random Fields using Second Order Cone Programming Relaxations M. Pawan Kumar Philip Torr Andrew Zisserman.](https://reader035.fdocuments.us/reader035/viewer/2022062619/5515eb3f550346dd6f8b5179/html5/thumbnails/61.jpg)
2-D Example(X - xxT)
1 - x12 X12-x1x2
C3. 0
. 01 1
1 1 X12-x1x2 1 - x22
(x1 + x2)2 2 + 2X12
SOC of the form || v ||2 st
C3 0
![Page 62: Solving Markov Random Fields using Second Order Cone Programming Relaxations M. Pawan Kumar Philip Torr Andrew Zisserman.](https://reader035.fdocuments.us/reader035/viewer/2022062619/5515eb3f550346dd6f8b5179/html5/thumbnails/62.jpg)
2-D Example(X - xxT)
1 - x12 X12-x1x2
C4. 0
. 01 -1
-1 1 X12-x1x2 1 - x22
(x1 - x2)2 2 - 2X12
SOC of the form || v ||2 st
C4 0
![Page 63: Solving Markov Random Fields using Second Order Cone Programming Relaxations M. Pawan Kumar Philip Torr Andrew Zisserman.](https://reader035.fdocuments.us/reader035/viewer/2022062619/5515eb3f550346dd6f8b5179/html5/thumbnails/63.jpg)
SOCP Relaxation
Consider a matrix C1 = UUT 0
(X - xxT)
||UTx ||2 X . C1
C1 . 0
Continue for C2, C3, … , Cn
SOC of the form || v ||2 st
Kim and Kojima, 2000
![Page 64: Solving Markov Random Fields using Second Order Cone Programming Relaxations M. Pawan Kumar Philip Torr Andrew Zisserman.](https://reader035.fdocuments.us/reader035/viewer/2022062619/5515eb3f550346dd6f8b5179/html5/thumbnails/64.jpg)
SOCP Relaxation
How many constraints for SOCP = SDP ?
Infinite. For all C 0
We specify constraints similar to the 2-D example
![Page 65: Solving Markov Random Fields using Second Order Cone Programming Relaxations M. Pawan Kumar Philip Torr Andrew Zisserman.](https://reader035.fdocuments.us/reader035/viewer/2022062619/5515eb3f550346dd6f8b5179/html5/thumbnails/65.jpg)
SOCP RelaxationMuramatsu and Suzuki, 2001
1 0
0 0
0 0
0 1
1 1
1 1
1 -1
-1 1
Constraints hold for the above semidefinite matrices
![Page 66: Solving Markov Random Fields using Second Order Cone Programming Relaxations M. Pawan Kumar Philip Torr Andrew Zisserman.](https://reader035.fdocuments.us/reader035/viewer/2022062619/5515eb3f550346dd6f8b5179/html5/thumbnails/66.jpg)
SOCP RelaxationMuramatsu and Suzuki, 2001
1 0
0 0
0 0
0 1
1 1
1 1
1 -1
-1 1
a + b
+ c + d
a 0
b 0
c 0
d 0
Constraints hold for the linear combination
![Page 67: Solving Markov Random Fields using Second Order Cone Programming Relaxations M. Pawan Kumar Philip Torr Andrew Zisserman.](https://reader035.fdocuments.us/reader035/viewer/2022062619/5515eb3f550346dd6f8b5179/html5/thumbnails/67.jpg)
SOCP RelaxationMuramatsu and Suzuki, 2001
a+c+d c-d
c-d b+c+d
a 0
b 0
c 0
d 0Includes all semidefinite matrices where
Diagonal elements Off-diagonal elements
![Page 68: Solving Markov Random Fields using Second Order Cone Programming Relaxations M. Pawan Kumar Philip Torr Andrew Zisserman.](https://reader035.fdocuments.us/reader035/viewer/2022062619/5515eb3f550346dd6f8b5179/html5/thumbnails/68.jpg)
SOCP Relaxation - A
x* = argmin 12
∑ ui (1 + xi) + 14
∑ Pij (1 + xi + xj + Xij)
∑ xi = 2 - |L|i Va
∑ Xij = (2 - |L|) xij Vb
xi [-1,1]
Xii = 1 X - xxT 0
![Page 69: Solving Markov Random Fields using Second Order Cone Programming Relaxations M. Pawan Kumar Philip Torr Andrew Zisserman.](https://reader035.fdocuments.us/reader035/viewer/2022062619/5515eb3f550346dd6f8b5179/html5/thumbnails/69.jpg)
SOCP Relaxation - A
x* = argmin 12
∑ ui (1 + xi) + 14
∑ Pij (1 + xi + xj + Xij)
∑ xi = 2 - |L|i Va
∑ Xij = (2 - |L|) xij Vb
xi [-1,1]
(xi + xj)2 2 + 2Xij (xi - xj)2 2 - 2Xij
Specified only when Pij 0
![Page 70: Solving Markov Random Fields using Second Order Cone Programming Relaxations M. Pawan Kumar Philip Torr Andrew Zisserman.](https://reader035.fdocuments.us/reader035/viewer/2022062619/5515eb3f550346dd6f8b5179/html5/thumbnails/70.jpg)
Triangular Inequality
• At least two of xi, xj and xk have the same sign
• At least one of Xij, Xjk, Xik is equal to one
Xij + Xjk + Xik -1Xij - Xjk - Xik -1-Xij - Xjk + Xik -1-Xij + Xjk - Xik -1
• SOCP-B = SOCP-A + Triangular Inequalities
![Page 71: Solving Markov Random Fields using Second Order Cone Programming Relaxations M. Pawan Kumar Philip Torr Andrew Zisserman.](https://reader035.fdocuments.us/reader035/viewer/2022062619/5515eb3f550346dd6f8b5179/html5/thumbnails/71.jpg)
Outline• Integer Programming Formulation
• Previous Work
• Our Approach– Second Order Cone Programming (SOCP)– SOCP Relaxation– Robust Truncated Model
• Applications– Subgraph Matching– Pictorial Structures
![Page 72: Solving Markov Random Fields using Second Order Cone Programming Relaxations M. Pawan Kumar Philip Torr Andrew Zisserman.](https://reader035.fdocuments.us/reader035/viewer/2022062619/5515eb3f550346dd6f8b5179/html5/thumbnails/72.jpg)
Robust Truncated ModelPairwise cost of incompatible labels is truncated
Potts ModelTruncated Linear Model
Truncated Quadratic Model
• Robust to noise
• Widely used in Computer Vision - Segmentation, Stereo
![Page 73: Solving Markov Random Fields using Second Order Cone Programming Relaxations M. Pawan Kumar Philip Torr Andrew Zisserman.](https://reader035.fdocuments.us/reader035/viewer/2022062619/5515eb3f550346dd6f8b5179/html5/thumbnails/73.jpg)
Robust Truncated ModelPairwise Cost Matrix can be made sparse
P = [0.5 0.5 0.3 0.3 0.5]
Q = [0 0 -0.2 -0.2 0]
Reparameterization
Sparse Q matrix Fewer constraints
![Page 74: Solving Markov Random Fields using Second Order Cone Programming Relaxations M. Pawan Kumar Philip Torr Andrew Zisserman.](https://reader035.fdocuments.us/reader035/viewer/2022062619/5515eb3f550346dd6f8b5179/html5/thumbnails/74.jpg)
Compatibility Constraint
Q(ma, mb) < 0 for variables Va and Vb
Relaxation ∑ Qij (1 + xi + xj + Xij) < 0
SOCP-C = SOCP-B + Compatibility Constraints
![Page 75: Solving Markov Random Fields using Second Order Cone Programming Relaxations M. Pawan Kumar Philip Torr Andrew Zisserman.](https://reader035.fdocuments.us/reader035/viewer/2022062619/5515eb3f550346dd6f8b5179/html5/thumbnails/75.jpg)
SOCP Relaxation
• More accurate than LP
• More efficient than SDP
• Time complexity - O( |V|3 |L|3)
• Same as LP
• Approximate algorithms exist for LP relaxation
• We use |V| 10 and |L| 200
![Page 76: Solving Markov Random Fields using Second Order Cone Programming Relaxations M. Pawan Kumar Philip Torr Andrew Zisserman.](https://reader035.fdocuments.us/reader035/viewer/2022062619/5515eb3f550346dd6f8b5179/html5/thumbnails/76.jpg)
Outline• Integer Programming Formulation
• Previous Work
• Our Approach– Second Order Cone Programming (SOCP)– SOCP Relaxation– Robust Truncated Model
• Applications– Subgraph Matching– Pictorial Structures
![Page 77: Solving Markov Random Fields using Second Order Cone Programming Relaxations M. Pawan Kumar Philip Torr Andrew Zisserman.](https://reader035.fdocuments.us/reader035/viewer/2022062619/5515eb3f550346dd6f8b5179/html5/thumbnails/77.jpg)
Subgraph MatchingSubgraph Matching - Torr - 2003, Schellewald et al - 2005
G1
G2
Unary costs are uniform
V2 V3
V1
MRF
ABCD A
BCD
ABCD
Pairwise costs form a Potts model
![Page 78: Solving Markov Random Fields using Second Order Cone Programming Relaxations M. Pawan Kumar Philip Torr Andrew Zisserman.](https://reader035.fdocuments.us/reader035/viewer/2022062619/5515eb3f550346dd6f8b5179/html5/thumbnails/78.jpg)
Subgraph Matching
• 1000 pairs of graphs G1 and G2
• #vertices in G2 - between 20 and 30
• #vertices in G1 - 0.25 * #vertices in G2
• 5% noise to the position of vertices
• NP-hard problem
![Page 79: Solving Markov Random Fields using Second Order Cone Programming Relaxations M. Pawan Kumar Philip Torr Andrew Zisserman.](https://reader035.fdocuments.us/reader035/viewer/2022062619/5515eb3f550346dd6f8b5179/html5/thumbnails/79.jpg)
Subgraph Matching
Method Time (sec) Accuracy (%)
LP 0.85 6.64
SDP-A 35.0 93.11
SOCP-A 3.0 92.01
SOCP-B 4.5 94.79
SOCP-C 4.8 96.18
![Page 80: Solving Markov Random Fields using Second Order Cone Programming Relaxations M. Pawan Kumar Philip Torr Andrew Zisserman.](https://reader035.fdocuments.us/reader035/viewer/2022062619/5515eb3f550346dd6f8b5179/html5/thumbnails/80.jpg)
Outline• Integer Programming Formulation
• Previous Work
• Our Approach– Second Order Cone Programming (SOCP)– SOCP Relaxation– Robust Truncated Model
• Applications– Subgraph Matching– Pictorial Structures
![Page 81: Solving Markov Random Fields using Second Order Cone Programming Relaxations M. Pawan Kumar Philip Torr Andrew Zisserman.](https://reader035.fdocuments.us/reader035/viewer/2022062619/5515eb3f550346dd6f8b5179/html5/thumbnails/81.jpg)
Pictorial Structures
Image
P1 P3
P2
(x,y,,)
MRF
Matching Pictorial Structures - Felzenszwalb et al - 2001
Outline
Texture
![Page 82: Solving Markov Random Fields using Second Order Cone Programming Relaxations M. Pawan Kumar Philip Torr Andrew Zisserman.](https://reader035.fdocuments.us/reader035/viewer/2022062619/5515eb3f550346dd6f8b5179/html5/thumbnails/82.jpg)
Pictorial Structures
Image
P1 P3
P2
(x,y,,)
MRF
Unary costs are negative log likelihoods
Pairwise costs form a Potts model
| V | = 10 | L | = 200
![Page 83: Solving Markov Random Fields using Second Order Cone Programming Relaxations M. Pawan Kumar Philip Torr Andrew Zisserman.](https://reader035.fdocuments.us/reader035/viewer/2022062619/5515eb3f550346dd6f8b5179/html5/thumbnails/83.jpg)
Pictorial Structures
LBP
GBP
SOCP
![Page 84: Solving Markov Random Fields using Second Order Cone Programming Relaxations M. Pawan Kumar Philip Torr Andrew Zisserman.](https://reader035.fdocuments.us/reader035/viewer/2022062619/5515eb3f550346dd6f8b5179/html5/thumbnails/84.jpg)
Pictorial Structures
LBP
GBP
SOCP
![Page 85: Solving Markov Random Fields using Second Order Cone Programming Relaxations M. Pawan Kumar Philip Torr Andrew Zisserman.](https://reader035.fdocuments.us/reader035/viewer/2022062619/5515eb3f550346dd6f8b5179/html5/thumbnails/85.jpg)
Pictorial Structures
LBP
GBP
SOCP
![Page 86: Solving Markov Random Fields using Second Order Cone Programming Relaxations M. Pawan Kumar Philip Torr Andrew Zisserman.](https://reader035.fdocuments.us/reader035/viewer/2022062619/5515eb3f550346dd6f8b5179/html5/thumbnails/86.jpg)
Pictorial Structures
LBP
GBP
SOCP
LBP and GBP do not converge
![Page 87: Solving Markov Random Fields using Second Order Cone Programming Relaxations M. Pawan Kumar Philip Torr Andrew Zisserman.](https://reader035.fdocuments.us/reader035/viewer/2022062619/5515eb3f550346dd6f8b5179/html5/thumbnails/87.jpg)
Pictorial StructuresROC Curves for 450 +ve and 2400 -ve images
![Page 88: Solving Markov Random Fields using Second Order Cone Programming Relaxations M. Pawan Kumar Philip Torr Andrew Zisserman.](https://reader035.fdocuments.us/reader035/viewer/2022062619/5515eb3f550346dd6f8b5179/html5/thumbnails/88.jpg)
Pictorial StructuresROC Curves for 450 +ve and 2400 -ve images
![Page 89: Solving Markov Random Fields using Second Order Cone Programming Relaxations M. Pawan Kumar Philip Torr Andrew Zisserman.](https://reader035.fdocuments.us/reader035/viewer/2022062619/5515eb3f550346dd6f8b5179/html5/thumbnails/89.jpg)
Conclusions• We presented an SOCP relaxation to solve MRF
• More efficient than SDP
• More accurate than LP, LBP, GBP
• #variables can be reduced for Robust Truncated Model
• Provides excellent results for subgraph matching and pictorial structures
![Page 90: Solving Markov Random Fields using Second Order Cone Programming Relaxations M. Pawan Kumar Philip Torr Andrew Zisserman.](https://reader035.fdocuments.us/reader035/viewer/2022062619/5515eb3f550346dd6f8b5179/html5/thumbnails/90.jpg)
Future Work
• Quality of solution– Additive bounds exist– Multiplicative bounds for special cases ??
• Message passing algorithm ??– Similar to TRW-S or -expansion– To handle image sized MRF