ASCENT courseware mapping reference - ASCENT- Center for Technical
A Flexible Tensor Block Coordinate Ascent Scheme for ... · A Flexible Tensor Block Coordinate...
1
A Flexible Tensor Block Coordinate Ascent Scheme for Hypergraph Matching Quynh Nguyen Ngoc 1,2 , Antoine Gautier 2 , Matthias Hein 2 1 Max Planck Institute for Informatics, Saarbrücken, Germany. 2 Saarland University, Saarbrücken, Germany. Baseline 20 40 60 80 100 Accuracy 0 0.2 0.4 0.6 0.8 1 (a) 10 pts vs 30 pts Baseline 20 40 60 80 100 Accuracy 0 0.2 0.4 0.6 0.8 1 SM IPFP RRWM MPM HGM TM RRWHM BCAGM BCAGM+IPFP BCAGM+MP (b) 30 pts vs 30 pts Figure 1: CMU house dataset: Our algorithms are denoted by BCAGM, BCAGM+IPFP, BCAGM+MP. (Best viewed in color.) Graph resp. hypergraph matching has been used in a variety of problems in computer vision, especially in feature correspondence problems. While graph matching [1, 4] is limited to pairwise geometric relations, hypergraph matching [2, 3] can integrate better geometric information by taking into account the higher order relation between groups of points. Thus, they cope better with geometric transformation such as scaling and other forms of noise. In this paper, we present a new algorithmic framework for solving a third order hypergraph matching problem, where higher order geometric features are used. Given two sets of feature points V and V 0 with n 1 = | V |≤ n 2 = | V 0 |,a third order hypergraph matching problem is formulated as max x∈M S 3 (x) := n ∑ i, j,k=1 F 3 ijk x i x j x k . (1) where x ∈ R n 1 n 2 is a vectorized version of an assignment matrix X which belongs to the set M = X ∈{0, 1} n 1 ×n 2 ∑ n 1 i=1 X ij ≤ 1, ∑ n 2 j=1 X ij = 1 . By abuse of notation, we write x ∈ M to denote X ∈ M. Each entry F 3 ijk of the third order symmetric tensor F 3 roughly represents the geometric re- lation between three correspondences encoded in {i, j, k}. While previous work tackle this NP-hard polynomial optimization problem by relaxing the discrete constraint set M to continuous domain, we propose in this paper to directly handle the original constraint set. This is motivated by the result of previous work [1, 3, 4] which have shown the importance of one-to-one constraints in graph/hypergraph matching performance. Our main idea is to optimize instead of the original score function the multilinear form associ- ated to it. Multilinear Form. The multilinear form F m : R n × ... × R n → R associ- ated to an m-th order tensor F m is given by: F m (x 1 ,..., x m )= n ∑ i 1 ,...,i m F m i 1 ...i m x 1 i 1 ... x m i m , and the score function S m : R n → R is defined by S m (x) := F m (x,..., x). We show below the main steps and contributions of our paper: 1. We lift the original third order tensor F 3 to a fourth order tensor F 4 and show that the third and fourth order problems are equivalent, F 4 i jkl = F 3 ijk + F 3 ijl + F 3 ikl + F 3 jkl . (2) This lift is important as it allows us to connect the optimization of a score function with the optimization of its associated multilinear form. In particular, if S 4 is convex then it holds for all x, y, z, t ∈ R n , This is an extended abstract. The full paper is available at the Computer Vision Foundation webpage. (a) Input: 10 pts vs 30 pts (b) MPM 4/10 (15.63) (c) TM 5/10 (18.37) (d) RRWHM 7/10 (26.36) (e) BCAGM 10/10 (43.09) (f) BCAGM+MP 10/10 (43.09) Figure 2: CMU house dataset: Blue dots indicate outlier points. Red lines indicate incorrect matches. The matching accuracy (matching score) is re- ported for each method. (Best viewed in color.) (a) 34 pts vs 44 pts, 10 outliers (b) TM 10/34 (1714.99) (c) HGM 9/34 (614.73) (d) RRWHM 28/34 (5230.52) (e) BCAGM 28/34 (5298.77) (f) BCAGM+MP 34/34 (5377.27) Figure 3: Car dataset: The number of correct matches and the objective score are reported. (Best viewed in color.) (a) F 4 (x, x, y, y) ≤ max u∈{x,y} F 4 (u, u, u, u). (b) F 4 (x, y, z, t) ≤ max u∈{x,y,z,t} F 4 (u, u, u, u). 2. Given S 4 is convex, we prove that the optimization of the multilinear form F 4 (x, y, z, t) is always equivalent to the optimization of the score function S 4 (x). In particular, it holds for any compact set D ⊂ R n that max x∈D F 4 (x, x, x, x)= max x,y∈D F 4 (x, x, y, y)= max x,y,z,t∈D F 4 (x, y, z, t) (3) 3. If S 4 is not convex, we provide a way to make it convex where the modification is constant on the set of assignment matrices M and extend this modification to a fourth order multilinear form. 4. In the algorithms, we optimize the multilinear form F 4 in a block coordinate ascent style leading to monotonic ascent directly on the set of assignment matrices M. 5. Our algorithms beat all state-of-the-art methods for third order (but also second order) graph matching in extensive experiments. Figure 1 and Figure 2 show some matching results on the CMU house dataset. 6. Our algorithms have competitive running time to all other methods. [1] M. Cho, J. Lee, and K. M. Lee. Reweighted random walks for graph matching. In ECCV, 2010. [2] O. Duchenne, F. Bach, I. Kweon, and J. Ponce. A tensor-based algorithm for high-order graph matching. PAMI, 33:2383–2395, 2011. [3] J. Lee, M. Cho, and K. M. Lee. Hyper-graph matching via reweighted random walks. In CVPR, 2011. [4] M. Leordeanu, M. Hebert, and R. Sukthankar. An integer projected fixed point method for graph matching and map inference. In NIPS, 2009.
Transcript of A Flexible Tensor Block Coordinate Ascent Scheme for ... · A Flexible Tensor Block Coordinate...
A Flexible Tensor Block Coordinate Ascent Scheme for Hypergraph Matching
Quynh Nguyen Ngoc1,2, Antoine Gautier2, Matthias Hein2
1Max Planck Institute for Informatics, Saarbrücken, Germany. 2Saarland University, Saarbrücken, Germany.
Baseline20 40 60 80 100
Accura
cy
0
0.2
0.4
0.6
0.8
1
(a) 10 pts vs 30 pts
Baseline20 40 60 80 100
Accura
cy
0
0.2
0.4
0.6
0.8
1
SMIPFPRRWMMPMHGMTMRRWHMBCAGMBCAGM+IPFPBCAGM+MP
(b) 30 pts vs 30 pts
Figure 1: CMU house dataset: Our algorithms are denoted by BCAGM,BCAGM+IPFP, BCAGM+MP. (Best viewed in color.)
Graph resp. hypergraph matching has been used in a variety of problemsin computer vision, especially in feature correspondence problems. Whilegraph matching [1, 4] is limited to pairwise geometric relations, hypergraphmatching [2, 3] can integrate better geometric information by taking intoaccount the higher order relation between groups of points. Thus, they copebetter with geometric transformation such as scaling and other forms ofnoise. In this paper, we present a new algorithmic framework for solvinga third order hypergraph matching problem, where higher order geometricfeatures are used.
Given two sets of feature points V and V ′ with n1 = |V | ≤ n2 = |V ′|, athird order hypergraph matching problem is formulated as
maxx∈M
S3(x) :=n
∑i, j,k=1
F3i jk xi x j xk. (1)
where x ∈ Rn1n2 is a vectorized version of an assignment matrix X whichbelongs to the set M =
{X ∈ {0,1}n1×n2
∣∣ ∑n1i=1 Xi j ≤ 1, ∑
n2j=1 Xi j = 1
}.
By abuse of notation, we write x ∈ M to denote X ∈ M. Each entry F3i jk
of the third order symmetric tensor F3 roughly represents the geometric re-lation between three correspondences encoded in {i, j,k}. While previouswork tackle this NP-hard polynomial optimization problem by relaxing thediscrete constraint set M to continuous domain, we propose in this paper todirectly handle the original constraint set. This is motivated by the resultof previous work [1, 3, 4] which have shown the importance of one-to-oneconstraints in graph/hypergraph matching performance. Our main idea is tooptimize instead of the original score function the multilinear form associ-ated to it.
Multilinear Form. The multilinear form Fm : Rn× . . .×Rn→ R associ-ated to an m-th order tensor Fm is given by:
Fm(x1, . . . ,xm) =n
∑i1,...,im
Fmi1...im x1
i1 . . .xmim ,
and the score function Sm : Rn→ R is defined by Sm(x) := Fm(x, . . . ,x).We show below the main steps and contributions of our paper:
1. We lift the original third order tensor F3 to a fourth order tensor F4
and show that the third and fourth order problems are equivalent,
F4i jkl = F3
i jk +F3i jl +F3
ikl +F3jkl . (2)
This lift is important as it allows us to connect the optimization ofa score function with the optimization of its associated multilinearform. In particular, if S4 is convex then it holds for all x,y,z, t ∈ Rn,
This is an extended abstract. The full paper is available at the Computer Vision Foundationwebpage.
(a) Input: 10 pts vs 30 pts (b) MPM 4/10 (15.63) (c) TM 5/10 (18.37)
(d) RRWHM 7/10 (26.36) (e) BCAGM 10/10 (43.09) (f) BCAGM+MP 10/10 (43.09)
Figure 2: CMU house dataset: Blue dots indicate outlier points. Red linesindicate incorrect matches. The matching accuracy (matching score) is re-ported for each method. (Best viewed in color.)
(a) 34 pts vs 44 pts, 10 outliers (b) TM 10/34 (1714.99)
(c) HGM 9/34 (614.73) (d) RRWHM 28/34 (5230.52)
(e) BCAGM 28/34 (5298.77) (f) BCAGM+MP 34/34 (5377.27)
Figure 3: Car dataset: The number of correct matches and the objectivescore are reported. (Best viewed in color.)
(a) F4(x,x,y,y)≤ maxu∈{x,y}
F4(u,u,u,u).
(b) F4(x,y,z, t)≤ maxu∈{x,y,z,t}
F4(u,u,u,u).
2. Given S4 is convex, we prove that the optimization of the multilinearform F4(x,y,z, t) is always equivalent to the optimization of the scorefunction S4(x). In particular, it holds for any compact set D⊂Rn that
maxx∈D
F4(x,x,x,x) = maxx,y∈D
F4(x,x,y,y) = maxx,y,z,t∈D
F4(x,y,z, t) (3)
3. If S4 is not convex, we provide a way to make it convex where themodification is constant on the set of assignment matrices M andextend this modification to a fourth order multilinear form.
4. In the algorithms, we optimize the multilinear form F4 in a blockcoordinate ascent style leading to monotonic ascent directly on theset of assignment matrices M.
5. Our algorithms beat all state-of-the-art methods for third order (butalso second order) graph matching in extensive experiments. Figure 1and Figure 2 show some matching results on the CMU house dataset.
6. Our algorithms have competitive running time to all other methods.
[1] M. Cho, J. Lee, and K. M. Lee. Reweighted random walks for graph matching.In ECCV, 2010.
[2] O. Duchenne, F. Bach, I. Kweon, and J. Ponce. A tensor-based algorithm forhigh-order graph matching. PAMI, 33:2383–2395, 2011.
[3] J. Lee, M. Cho, and K. M. Lee. Hyper-graph matching via reweighted randomwalks. In CVPR, 2011.
[4] M. Leordeanu, M. Hebert, and R. Sukthankar. An integer projected fixed pointmethod for graph matching and map inference. In NIPS, 2009.
