arXiv:1506.05274v1 [cs.CV] 17 Jun 2015 · Figure 2: Action of the mixed norm kCA B(h(v))k2;1. Left:...

13
Eurographics Symposium on Geometry Processing 2015 Mirela Ben-Chen and Ligang Liu (Guest Editors) Volume 34 (2015), Number 5 Partial Functional Correspondence E. Rodolà 1 , L. Cosmo 2 , M. M. Bronstein 3 , A. Torsello 2 , D. Cremers 1 1 TU Munich, 2 University of Venice, 3 USI Lugano Figure 1: Partial functional correspondence between two pairs of shapes with large missing parts. For each pair we show the matrix C representing the functional map in the spectral domain, and the action of the map by transferring colors from one shape to the other. The special slanted-diagonal structure of C induced by the partiality transformation is first estimated from spectral properties of the two shapes, and then exploited to drive the matching process. Abstract In this paper, we propose a method for computing partial functional correspondence between non-rigid shapes. We use perturbation analysis to show how removal of shape parts changes the Laplace-Beltrami eigenfunctions, and exploit it as a prior on the spectral representation of the correspondence. Corresponding parts are optimization variables in our problem and are used to weight the functional correspondence; we are looking for the largest and most regular (in the Mumford-Shah sense) parts that minimize correspondence distortion. We show that our approach can cope with very challenging correspondence settings. Categories and Subject Descriptors (according to ACM CCS): I.3.5 [Computer Graphics]: Computational Geometry and Object Modeling—Shape Analysis 1. Introduction The problem of shape correspondence is one of the most fun- damental problems in computer graphics and geometry pro- cessing, with a plethora of applications ranging from texture mapping to animation. A particularly challenging setting is that of non-rigid correspondence, where the shapes in ques- tion are allowed to undergo deformations, which are typi- cally assumed to be approximately isometric (such a model appears to be good for, e.g., human body poses). Even more challenging is partial correspondence, where one is shown only a subset of the shape and has to match it to a deformed full version thereof. Partial correspondence problems arise in numerous applications that involve real data acquisition by 3D sensors, which inevitable lead to missing parts due to occlusions or partial view. Related work. Shape correspondence is one of the most re- searched topics in geometry processing, with many popular methods such as [BBK06, KLCF10, KLF11]. Here, we limit our attention to the partial correspondence setting; for a com- prehensive overview of correspondence problems, we refer the reader to the recent survey [VKZHCO11]. In the domain of partial rigid shape alignment and match- ing, e.g., for 3D scan completion applications, many ver- sions of regularized ICP approaches exist, see for example [AMCO08, ART15]. Deformation-driven shape matching [ZSCO * 08] can be used in the non-rigid case. Our present work draws inspiration from [BB08, BBBK09], which com- bines metric distortion minimization with part optimization. More recent works have explored alignment of tangent spaces [BWW * 14], weighted matching of bags of local de- submitted to Eurographics Symposium on Geometry Processing (2015) arXiv:1506.05274v1 [cs.CV] 17 Jun 2015

Transcript of arXiv:1506.05274v1 [cs.CV] 17 Jun 2015 · Figure 2: Action of the mixed norm kCA B(h(v))k2;1. Left:...

Page 1: arXiv:1506.05274v1 [cs.CV] 17 Jun 2015 · Figure 2: Action of the mixed norm kCA B(h(v))k2;1. Left: Noisy landmark matches between complete and partial shape, represented as Gaussian

Eurographics Symposium on Geometry Processing 2015Mirela Ben-Chen and Ligang Liu(Guest Editors)

Volume 34 (2015), Number 5

Partial Functional Correspondence

E. Rodolà1, L. Cosmo2, M. M. Bronstein3, A. Torsello2, D. Cremers1

1TU Munich, 2University of Venice, 3USI Lugano

Figure 1: Partial functional correspondence between two pairs of shapes with large missing parts. For each pair we show thematrix C representing the functional map in the spectral domain, and the action of the map by transferring colors from oneshape to the other. The special slanted-diagonal structure of C induced by the partiality transformation is first estimated fromspectral properties of the two shapes, and then exploited to drive the matching process.

AbstractIn this paper, we propose a method for computing partial functional correspondence between non-rigid shapes. Weuse perturbation analysis to show how removal of shape parts changes the Laplace-Beltrami eigenfunctions, andexploit it as a prior on the spectral representation of the correspondence. Corresponding parts are optimizationvariables in our problem and are used to weight the functional correspondence; we are looking for the largestand most regular (in the Mumford-Shah sense) parts that minimize correspondence distortion. We show that ourapproach can cope with very challenging correspondence settings.

Categories and Subject Descriptors (according to ACM CCS): I.3.5 [Computer Graphics]: Computational Geometryand Object Modeling—Shape Analysis

1. Introduction

The problem of shape correspondence is one of the most fun-damental problems in computer graphics and geometry pro-cessing, with a plethora of applications ranging from texturemapping to animation. A particularly challenging setting isthat of non-rigid correspondence, where the shapes in ques-tion are allowed to undergo deformations, which are typi-cally assumed to be approximately isometric (such a modelappears to be good for, e.g., human body poses). Even morechallenging is partial correspondence, where one is shownonly a subset of the shape and has to match it to a deformedfull version thereof. Partial correspondence problems arisein numerous applications that involve real data acquisitionby 3D sensors, which inevitable lead to missing parts due toocclusions or partial view.

Related work. Shape correspondence is one of the most re-searched topics in geometry processing, with many popularmethods such as [BBK06, KLCF10, KLF11]. Here, we limitour attention to the partial correspondence setting; for a com-prehensive overview of correspondence problems, we referthe reader to the recent survey [VKZHCO11].

In the domain of partial rigid shape alignment and match-ing, e.g., for 3D scan completion applications, many ver-sions of regularized ICP approaches exist, see for example[AMCO08, ART15]. Deformation-driven shape matching[ZSCO∗08] can be used in the non-rigid case. Our presentwork draws inspiration from [BB08, BBBK09], which com-bines metric distortion minimization with part optimization.

More recent works have explored alignment of tangentspaces [BWW∗14], weighted matching of bags of local de-

submitted to Eurographics Symposium on Geometry Processing (2015)

arX

iv:1

506.

0527

4v1

[cs

.CV

] 1

7 Ju

n 20

15

Page 2: arXiv:1506.05274v1 [cs.CV] 17 Jun 2015 · Figure 2: Action of the mixed norm kCA B(h(v))k2;1. Left: Noisy landmark matches between complete and partial shape, represented as Gaussian

2 E. Rodolà et al. / Partial Functional Correspondence

scriptors [PBB13], sparse minimum distortion correspon-dence [RBA∗12], bilateral maps [vKZH13], shape extrem-ities [SY14], and other constructions for the purpose ofnon-rigid partial matching. In the context of collectionsof shapes, partial correspondence has been considered in[VKTS∗11, HG13].

Our work is based on the functional maps framework[OBCS∗12], in which shape correspondence is modeledas a linear operator between spaces of functions on theshapes. Such operators can be efficiently represented inthe Laplace-Beltrami eigenbasis [OBCS∗12]. Huang andGuibas [HWG14] used functional maps to analyze large col-lections of shapes in which some shapes may be only par-tially similar, constructing graphs of functional maps.

Contribution. In this paper, we propose an extension to thefunctional correspondence framework to allow dealing withpartial correspondence. Specifically, we consider a scenarioof matching a part of a deformed shape to some full model.Such scenarios are very common for instance in robotics ap-plications, where one has to match an object acquired bymeans of a 3D scanner (and thus partially occluded) witha reference object known in advance. We use an explicitpart model over which optimization is performed, as wellas a regularization on the spectral representation of the func-tional correspondence accounting for a special structure ofthe Laplacian eigenfunctions as a result of part removal.Theoretical study of this behavior based on perturbationanalysis of Laplacian matrices is another contribution of ourwork. We show experimentally that the proposed approachallows dealing with very challenging partial correspondencesettings.

The rest of the paper is organized as follows. In Sec-tion 2, we review the basic concepts in the spectral geom-etry of shapes and describe the functional correspondenceapproach. Section 3 introduces our partial correspondencemodel, and Section 4 describes its implementation details.Section 5 studies the behavior of Laplacian eigenfunctionsin the case of missing parts, motivating the regularizationsused in the previous sections. Section 6 presents experimen-tal results, and finally, Section 7 concludes the paper.

2. Background

In this paper, we model a shape as a two-dimensional Rie-mannian manifold M (possibly with boundary). At inte-rior points, the manifold is locally homeomorphic to a two-dimensional Euclidean space known as the tangent planeTxM. The Riemannian metric is given in the form of aninner product 〈·, ·〉TxM : TxM× TxM→ R on the tangentplane, varying smoothly with x. The exponential map is amapping expx : TxM→M from the tangent plane to themanifold.

Given a smooth real function f :M→R, let us define the

function on the tangent plane by the composition f ◦ expx :TxM→ R. Then, the intrinsic gradient is defined as

∇M f = ∇( f ◦ expx)(0), (1)

where∇ denotes the standard Euclidean gradient in the tan-gent plane. The intrinsic gradient is thus a vector in thetangent plane. Similarly, the positive semi-definite Laplace-Beltrami operator ∆M : L2(M)→ L2(M) is defined as

∆M f = −∆( f ◦ expx)(0), (2)

where ∆ is the standard Laplacian. The Laplace-Beltrami op-erator is related to the intrinsic gradient through the Stokesformula,∫M

∆M f (x)g(x)dx = −∫M〈∇M f (x),∇Mg(x)〉TxMdx,

for any smooth f ,g ∈ L2(M).

The Laplace-Beltrami operator admits an eigen-decomposition ∆Mφi(x) =−λiφi(x) for i≥ 1, witheigenvalues 0 = λ1 < λ2 ≤ . . . and eigenfunctions{φi}i≥1 forming an orthonormal basis on L2(M), i.e.,〈φi,φ j〉L2(M) = δi j.

Fourier analysis. Let L2(M) = { f : M → R :∫M f (x)2dx < ∞} denote the space of functions on M,

and let 〈 f ,g〉L2(M) =∫M f (x)g(x)dx be the standard inner

product. Given an orthonormal basis {φi}i≥1 ⊆ L2(M),a function f ∈ L2(M) can be expanded into the Fourierseries as

f (x) = ∑i≥1〈 f ,φi〉L2(M)φi(x) . (3)

Functional correspondence. Let us be now given two man-ifolds, N and M. Ovsjanikov et al. [OBCS∗12] proposedmodeling functional correspondence between shapes as alinear operator T : L2(N ) → L2(M) between spaces offunctions on N and M. One can easily see that classicalvertex-wise correspondence is a particular setting where Tmaps delta-functions to delta-functions.

Assuming to be given two orthonormal bases {φi}i≥1 and{ψi}i≥1 on L2(N ) and L2(M) respectively, the functionalcorrespondence can be expressed w.r.t. to these bases as fol-lows:

T f = T ∑i≥1〈 f ,φi〉L2(N )φi = ∑

i≥1〈 f ,φi〉L2(N )T φi

= ∑i j≥1〈 f ,φi〉L2(N ) 〈T φi,ψ j〉L2(M)︸ ︷︷ ︸

ci j

ψ j , (4)

where f ∈ L2(N ) is some function on N . Thus, T amountsto a linear transformation of the Fourier coefficients of ffrom basis {φi}i≥1 to basis {ψi}i≥1, which is captured bythe coefficients ci j. Truncating the Fourier series at the firstk coefficients, one obtains a rank-k approximation of T as ak× k matrix C = (ci j).

submitted to Eurographics Symposium on Geometry Processing (2015)

Page 3: arXiv:1506.05274v1 [cs.CV] 17 Jun 2015 · Figure 2: Action of the mixed norm kCA B(h(v))k2;1. Left: Noisy landmark matches between complete and partial shape, represented as Gaussian

E. Rodolà et al. / Partial Functional Correspondence 3

Descriptor dimension0 50 100 150 200 250 300 350

Err

or

0

5

10

15Match

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15

Err

or

0

2

4

6

8

10

12

Figure 2: Action of the mixed norm ‖CA− B(η(v))‖2,1.Left: Noisy landmark matches between complete and partialshape, represented as Gaussian functions centered at corre-sponding points. Correct matches are shown in green, mis-matches in red, and correct matches along the boundary inblue. Top right: Bar plot showing ‖CAi−B(η(v))i‖2, wherei ranges over the matches and C is an optimal map found byour method. Even under an outlier ratio of ∼ 30%, all mis-matches are correctly detected and accounted for. Bottomright: Same plot when using dense descriptors.

In order to compute C, Ovsjanikov et al. [OBCS∗12]assume to be given a set of q corresponding functions{ f1, . . . , fq} ⊆ L2(N ) and {g1, . . . ,gq} ⊆ L2(M). Denotingby ai j = 〈 f j,φi〉L2(N ) and bi j = 〈g j,ψi〉L2(M) the k×q ma-trices of the respective Fourier coefficients, functional corre-spondence boils down to the linear system

CA = B . (5)

If q ≥ k, this system (5) is determined and is solved in theleast squares sense to determine C.

Structure of C. We note that the coefficients C depend onthe choice of the bases. In particular, it is convenient to usethe Laplace-Beltrami eigenbasis as a natural Fourier basis;truncating the series at the first k coefficients has the effect of‘low-pass’ filtering thus producing smooth correspondences.In the following, we will tacitly assume that {φi}i≥1 and{ψi}i≥1 are the eigenfunctions of the respective Laplacians.

Furthermore, note that the system (5) has qk equations andk2 variables. However, in many situations the actual numberof variables is significantly smaller, as C manifests a cer-tain structure which can be taken advantage of. In particular,if N and M are isometric and have simple spectrum (i.e.,the Laplace-Beltrami eigenvalues have no multiplicity), thenT φi = ±ψi, or in other words, ci j = ±δi j. In more realis-tic scenarios (approximately isometric shapes), the matrix Cwould manifest a funnel-shaped structure, with the majorityof elements distant from the diagonal close to zero.

3. Partial functional maps

For simplicity, throughout the paper we consider the settingwhere we are given a full model shapeM and another queryshapeN that corresponds to an approximately isometrically

deformed partM′ ⊂M. A more general setting whereMand N have some common parts M′ and N ′ is a ratherstraightforward extension of the presented approach.

Following [BB08], we model the part M′ by means ofan indicator function v :M→ {0,1} such that v(x) = 1 ifx ∈M′ and zero otherwise. Assuming that v is known, thepartial functional correspondence betweenN andM can beexpressed as T f = vg, where v can be regarded as a kind ofmask, and anything outside the region where v = 1 shouldbe ignored. Expressed w.r.t. bases {φi}i≥1 and {ψi}i≥1,the partial functional correspondence takes the form CA =B(v), where B(v) denotes a matrix of weighted inner prod-ucts with elements given by bi j(v) =

∫M v(x)ψi(x)g j(x)dx

(when v(x) ≡ 1, B is simply the matrix of Fourier coeffi-cients defined in (5)).

This brings us to the problem we are considering through-out this paper, involving optimization w.r.t. correspondence(encoded by the coefficients C) and the part v,

minC,v‖CA−B(η(v))‖2,1 +ρcorr(C)+ρpart(v) , (6)

where η(t) = 12 (tanh(2t−1)+1) saturates the part indi-

cator function between zero and one. Here ρcorr and ρpartdenote regularization terms for the correspondence and thepart, respectively; these terms are explained below. We usethe L2,1 matrix norm (equal to the sum of L2-norms of ma-trix columns) to handle possible outliers in the correspond-ing data, as such a norm promotes column-sparse matrices(see Fig. 2 for an example).

Note that in order to avoid a combinatorial optimizationover binary-valued v, we use a continuous v with values inthe range (−∞,+∞), saturated by the non-linearity η. Thisway, η(v) becomes a soft membership function with valuesin the range [0,1].

Part regularization. Similarly to [BB08], we try to find thepart with area closest to that of the query and with shortestboundary. This can be expressed as

ρpart(v) = µ1

(area(N )−

∫M

η(v)dx)2

(7)

+ µ2

∫M

ξ(v)‖∇Mv‖dx ,

where ξ(t) ≈ δ

(η(t)− 1

2

)and the norm is on the tan-

gent space. The µ2-term in (7) is an intrinsic version of theMumford-Shah functional [MS89], measuring the length ofthe boundary of a part represented by a (soft) membershipfunction. This functional was used previously in image seg-mentation applications [VC02].

submitted to Eurographics Symposium on Geometry Processing (2015)

Page 4: arXiv:1506.05274v1 [cs.CV] 17 Jun 2015 · Figure 2: Action of the mixed norm kCA B(h(v))k2;1. Left: Noisy landmark matches between complete and partial shape, represented as Gaussian

4 E. Rodolà et al. / Partial Functional Correspondence

Correspondence regularization. For the correspondence,we use the penalty

ρcorr(C) = µ3‖C◦W‖2F +µ4 ∑

i 6= j(C>C)2

i j

+ µ5 ∑i((C>C)ii−di)

2 , (8)

where ◦ denotes Hadamard (element-wise) matrix product.The µ3-term models a special slanted-diagonal structure ofC that we observe in partial matching problems (see Fig. 4);the theoretical motivation for this behavior is presented inSection 5. Here, W is a weight matrix with zeros along theslanted diagonal and large values outside (see Figure 3; de-tails on the computation of W are provided in Appendix A).The µ4-term promotes orthogonality of C by penalizing theoff-diagonal elements of C>C. The reason is that for iso-metric shapes, the functional map is volume-preserving, andthis is manifested in orthogonal C [OBCS∗12]. Note thatdifferently from the baseline case dealing with full shapes,in our setting we can only require orthogonality (hence areapreservation) going in the direction from partial to completemodel; for this reason, we do not impose any restrictions onCC> and we say that the matrix is semi-orthogonal. Finally,the µ5-term models the rank of the matrix C, where vectord = (d1, . . . ,dk) determines how many singular values of Care non-zero (the estimation of d is described in AppendixA).

Figure 3: Weight matrix W obtained with k = 100, r = 60,and σ = 0.03 (see Appendix A for details)

Remark. The fact that matrix C is of low rank is a directconsequence of partiality. This can be understood by re-calling that the (non-truncated) functional map representa-tion amounts to an orthogonal change of basis; since in thestandard basis the correspondence matrix is low-rank (as itcontains zero-sum rows), this property is preserved by thechange of basis.

In Fig. 4 we show an example of a valid partial functionalmap C, illustrating its main properties.

0 20 40 60 80 100

Sin

gula

r val

ue

0

0.5

1

1.5

2

2.5

3

3.5

4

4.5

Figure 4: Left: A partial functional map C from N toM islow-rank and manifests a slanted-diagonal structure due toeigenvector interactions along the boundary of N . Middle:If the map is volume-preserving then its full-rank sub-matrixis semi-orthogonal. Observe the trail of small values alongthe diagonal. Right: Singular values of C.

Alternating scheme. To solve the optimization prob-lem (6), we perform an alternating optimization w.r.t. to Cand v, repeating the following steps until convergence:

C-step: Fix v∗, solve for correspondence C

minC‖CA−B(η(v∗))‖2,1 +ρcorr(C) . (9)

V-step: Fix C∗, solve for part v

minv‖C∗A−B(η(v))‖2,1 +ρpart(v) . (10)

An example of this alternating process applied to a pair ofshapes is shown in Fig. 5.

4. Implementation

Discretization. In the discrete setting, the manifold M issampled at n points x1, . . . ,xn. On these points, we con-struct a manifold triangular mesh (V,E,F) with verticesV = {1, . . . ,n}, in which each interior edge i j ∈ E is sharedby exactly two triangular faces ik j and jhi ∈ F , and bound-ary edges belong to exactly one triangular face. We denoteby X the n× 3 matrix of the embedding coordinates of themesh.

A function on the manifold is represented by an n-dimensional vector f = ( f (x1), . . . , f (xn))

>. The discretiza-tion of the Laplacian takes the form L = S−1H us-ing the classical cotangent formula [Mac49, Duf59, PP93,MDSB03],

hi j =

(cotαi j + cotβi j)/2 i j ∈ E;−∑k 6=i hik i = j;0 else;

(11)

where S = diag(s1, . . . ,sn) and si =13 ∑ jk:i jk∈F si jk denotes

the local area element at vertex i and si jk denoting the area

submitted to Eurographics Symposium on Geometry Processing (2015)

Page 5: arXiv:1506.05274v1 [cs.CV] 17 Jun 2015 · Figure 2: Action of the mixed norm kCA B(h(v))k2;1. Left: Noisy landmark matches between complete and partial shape, represented as Gaussian

E. Rodolà et al. / Partial Functional Correspondence 5

Iteration 1 2 3 4

Figure 5: An example of the matching process operating on two shapes from TOSCA. The algorithm alternatingly optimizesover corresponding part (top row) and functional correspondence (bottom row). Corresponding points between full and partialshape are shown with the same color. This solution was obtained by using 30 eigenfunctions on both manifolds.

of triangle i jk, and αi j,βi j denote the angles ∠ik j,∠ jhi ofthe triangles sharing the edge i j.

The first k eigenfunctions and eigenvalues of the Lapla-cian are computed by performing the generalized eigen-decomposition HΦΦΦ = SΦΦΦΛΛΛ, where ΦΦΦ = (φφφ1, . . . ,φφφk) is ann× k matrix containing as columns the discretized eigen-functions and ΛΛΛ = diag(λ1, . . . ,λk) is the diagonal matrix ofthe corresponding eigenvalues.

Discretization of regularization terms. Our goal is to dis-cretize the µ1- and µ2-terms in (7). The part membershipfunction is discretized as an n-dimensional vector v. The µ1term is simply discretized as

∫Mη(v)dx ≈ s>η(v), where

the non-linearity η is applied element-wise.

We first express the discrete intrinsic gradient per faceof the mesh. Let i jk ∈ F be a triangle, and denote by P =(xk − xi,x j − xi) the 3× 2 matrix of triangle edge vectors.Let u = (vk−vi,v j−vi)

> be the 2-dimensional vector of themembership function gradient along the edges of the trian-gle. Then, the norm of the intrinsic gradient ‖∇Mv‖ for the

triangle i jk is discretized as gi jk(v) =(

u>(P>P)−1u)1/2

;

we denote by g(v) the |F|-dimensional vector containing thenorms of the gradient in all the triangles. In order to ob-tain the vertex-wise values of the intrinsic gradient norm,we construct an n×|F| matrix Q whose element qi j =

13 if

the jth triangle includes vertex i and zero otherwise. Then,∫M ξ(v)‖∇Mv‖dx≈ ξ(v)>SQg(v) [BB08].

Numerical optimization. We implemented our matchingframework in Matlab/C++ using the manifold optimizationtoolbox [BMAS14]. Each optimization step was performedby the method of conjugate gradients. In all our experimentswe observed convergence in 3-5 iterations (around 5 mins.for a pair of shapes).

We start the alternating scheme by optimizing over C andfixing v∗ = 1, where 1 is a vector of n ones. The initial valuefor C at the first iteration of the alternating process is set

to the weight matrix C = W. In order to construct W weneed to have an estimate of the expected rank of the soughtfunctional map, which in turn determines its diagonal slope.The same value is then used to construct d. A simple methodfor computing such an estimate is given in the next section;see also Fig. 6 for an illustration of this approach.

Finally, in order to account for noisy data we run a re-finement step after each C-step. Specifically, assume C∗ isa local optimum of problem (9). Then, we solve again forC by setting µ3 = 0 and replacing the data term in (9) with‖CΦΦΦ

>−ΨΨΨ>

Π‖2,1, where Π is a permutation matrix align-ing columns of ΨΨΨ

> with columns of C∗ΦΦΦ>. Together withthe semi-orthogonality requirement on C, this refinementstep can be seen as a generalization to partial maps of theICP-like technique found in [OBCS∗12].

5. Perturbation analysis of the Laplacian spectrum

By a slight abuse of notation, in this section we denote byM the set of vertices on the full shape, and let N ⊆M bethe set of vertices on the partial shape. With N we denotethe restriction to the remaining vertices M\N . Our aimis to characterize the eigenvalues and eigenvectors of LM,Laplacian ofM, in terms of perturbations of the eigenvaluesand eigenvectors of the Laplacians LN and LN ofN andNrespectively [MH88].

In this analysis we assume that the discrete Laplacian inuse has a support of limited range. This means that the non-zero entries for the i-th row will correspond to vertices topo-logically close to the i-th vertex in the mesh, and that thediscretization is of limited topological diameter (the maxi-mum topological distance between non-zero entries). Notethat this assumption holds for classical FEM schemes suchas the cotangent rule [MDSB03], since in this case the i-th row will have non-zero entries in correspondence withi’s immediate neighbors, resulting in a range of the supportof topological diameter 2. In case of the Graph Laplacian,the range has topological diameter equal to 1. As a result

submitted to Eurographics Symposium on Geometry Processing (2015)

Page 6: arXiv:1506.05274v1 [cs.CV] 17 Jun 2015 · Figure 2: Action of the mixed norm kCA B(h(v))k2;1. Left: Noisy landmark matches between complete and partial shape, represented as Gaussian

6 E. Rodolà et al. / Partial Functional Correspondence

Figure 6: Neumann spectra of a full and partial shape, thelatter being a connected subset of the former. The two spec-tra align only partially, a fact that allows to easily derivean estimate for the rank of the functional map relating thetwo shapes. In this example with 50 eigenfunctions, the es-timated rank is 23 (the estimation procedure is shown asdashed lines).

of this property, there is a boundary band B = ∂N ∪ ∂Nwith ∂N ⊆ N and ∂N ⊆ N , such that only the entries ofthe Laplacians LN and LN between nodes in B are affectedby the cut. In particular, for the Graph Laplacian, the twoboundary layers will be 1 vertex-thick, while for the cotan-gent Laplacian they will be 2 vertices-thick.

Without loss of generality we can assume that the verticesin M are ordered such that the vertices in N come beforethose in N . Further, within N the boundary vertices ∂Nare at the end, while within N the boundary vertices ∂Nare at the beginning. With this ordering we can define theparametric matrix

L(t) =

(LN 0

0 LN

)+ t

0 0 0 00 DN E 0

0 E> DN 00 0 0 0

,

(12)where the second and third rows/columns of the last matri-ces are over the vertices in ∂N and ∂N respectively. HereDN and DN represent the variations, with respect to the fullLaplacian LM, of the Laplacians LN and LN within nodesin ∂N and ∂N respectively, while E represents the variationsacross the boundary.

These matrices are such that L(1) = LM, while for t = 0the eigenvalues and eigenvectors of L(0) correspond to theeigenvalues and (extended) eigenvectors of LN and LN .Given a correct order of the nodes, due to the limit in therange of the support of the Laplacian, the matrices DN andDN will have a band-diagonal structure, with the exceptionof a few wrap-around points due to cyclic ordering along thecut. In particular, in the case of the Graph Laplacian, DNand DN will be diagonal with almost constant diagonal el-ements if the mesh is sufficiently dense and regular, whilefor the cotangent Laplacian they will be (almost) tridiagonaland diagonally dominant.

Eigenfunction 2 3 4 5 6

Eigenfunction 2 5 6 8 10

Figure 7: Correspondence between the first ten LB eigen-functions of two nearly-isometric shapes in presence of par-tiality. Note the different sequences in the two cases: not alleigenfunctions on the complete shape (in this case, the 3rd,4th, 7th, and 9th) have a corresponding function on the par-tial shape. In turn, the last 4 eigenfunctions of the partialshape do not appear among the first 10 of the full shape.In this example, the correspondence between eigenfunctionswas obtained by comparing the respective eigenvalues.

Eigenvalue perturbation (rank of C). Here we take thesimplifying assumptions that LM and LN do not have re-peated eigenvalues. Let LN = ΦΦΦ

>ΛΛΛΦΦΦ and LM = ΦΦΦ

>ΛΛΛΦΦΦ

be the spectral decompositions of LN and LN respectively.Following [MH88] we can write the derivative of λi eigen-value of LN and, thus, of L(0), as:

λ′i = φφφ

>i DN φφφi = ∑

v,w∈∂N(DN )vwφivφiw = 〈φφφi,φφφi〉DN . (13)

With the expression φφφ>i DN φφφi we make a slight abuse of no-

tation, as the product should be interpreted over the restric-tion of φφφi to ∂N or, equivalently, DN should be interpretedas being zero-padded to the size of the full mesh.

Note that the first derivative of the eigenvalues only de-pends on the squared DN norm of the corresponding eigen-vectors along the boundary. The continuity of the eigenval-ues in L(t) helps in explaining the interleave of the eigen-basis seen in Fig. 7, as the eigenvalues and correspondingeigenvectors of LN and LN overlap with little modificationsgoing towards the full Laplacian L(1) = LM. We make useof this fact in order to estimate the rank of C as the index ofthe largest eigenvalue of LN smaller or equal to the largesteigenvalue of LM (see Fig. 6).

submitted to Eurographics Symposium on Geometry Processing (2015)

Page 7: arXiv:1506.05274v1 [cs.CV] 17 Jun 2015 · Figure 2: Action of the mixed norm kCA B(h(v))k2;1. Left: Noisy landmark matches between complete and partial shape, represented as Gaussian

E. Rodolà et al. / Partial Functional Correspondence 7

Eigenvector0 20 40 60 80 1000

2

4

6

8

10

Eigenvector0 20 40 60 80 1000

2

4

6

8

10

Figure 8: Left: A model is cut in two different ways (redand green curves), each producing a partial shape with dif-ferent boundary. Function f (16) is plotted over the model.Middle: Ground-truth functional map between the completemodel and the partial shape produced by the red cut (top),and values of f along the cut (bottom). Right: Plots associ-ated to the green cut.

Eigenvector perturbation (shape of C). The eigenvectorderivatives for eigenvectors of LN are given by

φφφ′i =

n

∑j=1j 6=i

φφφ>i DN φφφ j

λi−λ jφφφ j +

n

∑j=1

φφφ>i E φφφ j

λi−λ jφφφ j . (14)

Note that the second summation has support overN and thusprovides the completion of the eigenvector on the missingpart. The formula for the second summation is correct only ifthe eigenvalues of LN are all distinct from λi. If LN sharessome eigenvalues with LN the formula would be slightlydifferent [MH88], but it would still only have support overN , hence providing eigenvector completion.

On the other hand, the first summation is responsible forthe modifications of the eigenvectors over the nodesN . Herethe numerator has a term φφφ

>i DN φφφ j which, since DN is

band-diagonal and diagonally dominant, acts as a dot prod-uct of the eigenvectors over the boundary layer, thus point-ing to large mixing of eigenvectors with a strong co-presencenear the boundary. In turn, the term λi− λ j at the denomi-nator forces a strong mixing of eigenvectors correspondingto similar eigenvalues. This results in an amplification ofthe variation for higher eigenvalues, as eigenvalues tend todensify on the higher end of the spectrum. This results in aspread of the functional map for higher frequencies, as canbe seen from the examples in Fig. 4 and 8.

Remark. The first-order modification in the eigenvectorswithin N , which determines the shape of C, is due only toeigenvector interactions along the boundary. It is not a func-tion of the missing area, as intuition may suggest.

As a consequence, the variation of the eigenvectors due tothe mixing within the partial shape can be reduced eitherby shortening the boundary, or by reducing the strength ofthe boundary interaction. This can be done by selecting aboundary along which eigenvectors with similar eigenvaluesare either orthogonal, or both small.

We can measure the variation of the eigenbasis as a func-tion of the boundary B splittingM intoN andN as

∂ΦΦΦ(B) =n

∑i=1‖φφφ′i‖

2N =

n

∑i=1

n

∑j=1j 6=i

φφφ>i DN φφφ j

λi−λ j

2

. (15)

In Fig. 8 we see an example of two different cuts withdifferent interaction strengths. The function plotted on thecat model is

f (v) =n

∑i, j=1

j 6=i

(φivφ jv

λi−λ j

)2

. (16)

Assuming DN diagonal and with constant diagonal ele-ments k, then we have

k∫B

f (v)dv≥ ∂ΦΦΦ(B) , (17)

in fact:

∂ΦΦΦ(B) ≈ kn

∑i=1

n

∑j=1j 6=i

∑v∈∂M φiv φ jv

λi−λ j

2

≤ k ∑v∈∂M

n

∑i, j=1

j 6=i

(φiv φ jv

λi−λ j

)2

= k ∑v∈∂M

f (v).(18)

The cuts plotted in Fig. 8 have same length, but one cut goesalong a symmetry axis of the shape and through low valuesof f , while the other goes through rather high values of f .As a result we see that in the first case a lot of the eigenvec-tors are preserved and the functional map is tight along theslanted diagonal, while in the second case all the eigenvec-tors vary by a large amount and the corresponding functionalmap is more dispersed.

6. Experimental results

Datasets. In the literature there has been a general lack ofbenchmarks aimed at evaluating the performance of match-ing methods under partiality transformations. Here we in-troduce two datasets to tackle this more challenging sce-nario. As base models, we use shapes from the TOSCAdataset [BBK08], consisting of 76 nearly-isometric shapessubdivided into 8 classes. Each class comes with a “null”shape in a standard pose, and ground-truth correspondencesare provided for all shapes within the same class. In orderto make the datasets more challenging and avoid compatibletriangulations, all shapes were remeshed in advance to 10kvertices by iterative pair contractions [GH97]. We processedthe shapes in two different ways:

Regular cuts. The null shape of each class is cut with aplane using 6 different orientations, including an exact cutalong the symmetry plane (note that each null shape has

submitted to Eurographics Symposium on Geometry Processing (2015)

Page 8: arXiv:1506.05274v1 [cs.CV] 17 Jun 2015 · Figure 2: Action of the mixed norm kCA B(h(v))k2;1. Left: Noisy landmark matches between complete and partial shape, represented as Gaussian

8 E. Rodolà et al. / Partial Functional Correspondence

an extrinsic bilateral symmetry). The 6 cuts are then trans-ferred to the remaining poses using the ground-truth corre-spondence, resulting in 456 partial shapes in total. Some ex-amples are shown in Fig. 2, 6, and 7.

Irregular holes. Given a shape and an “area budget”telling the amount of surface area to keep, we derive addi-tional shapes by an erosion process applied to the surface.Specifically, seed holes are placed at a few farthest samplesover the shape; the holes are then enlarged until the specifiedarea budget is attained. We do so for all shapes in TOSCAand for area budgets equal to 40%, 70%, and 90% of the to-tal surface area, resulting in a total amount of 684 shapes.Examples of this dataset are shown in Fig. 10, 13.

Where not specified otherwise, we use 120 random partialshapes for the first dataset and 80 for the second, equallydistributed among the different classes. Each partial shape isthen matched to the null shape of the corresponding class.

Error measure. As quantitative measure for correspon-dence quality we use the error criterion originally introducedin [KLF11] to evaluate the quality of point-wise maps. Theinput quantity is a functional map C, which is converted toa point-wise counterpart by using the nearest-neighbor ap-proach described in [OBCS∗12]. We plot cumulative curvesshowing the percent of matches which have error smallerthan a variable threshold. Finally, we write pe to denote the(normalized) area below these curves up to error level e.

6.1. Data term

Due to the particular nature of the problem, in all our ex-periments we only make use of dense, local descriptors as

0

1

2 3 4 5 6 7 8 9 1020

25

30

35

Support size (% diameter)

p 0.02

2 3 4 5 6 7 8 9 1060

65

70

75

Support size (% diameter)

p 0.16

Figure 9: Left: Normalized L2 distance between local pointdescriptors on a partial shape (body top) and the corre-sponding descriptors on the complete shape (in transpar-ent white). Points close to the boundary have larger distancedue to missing parts. Right: Bar plots showing the local andglobal accuracy of our matching method, as we increase thesupport of local descriptors. Small neighborhoods inducesensitivity to surface noise, whereas descriptors with largesupport are affected by boundary effects. The optimal valuesare found in-between (red frames).

SHOT SHOT+MS SHOT+IH Allcat (p0.16) 79.21 78.87 76.54 77.67victoria (p0.16) 79.64 80.02 76.90 77.82cat (p0.02) 33.85 33.99 32.58 32.94victoria (p0.02) 32.55 32.50 31.50 31.95

Table 1: Performance comparison on a representative sub-set of TOSCA (regular cuts) under different local descriptorsas the data term. We show results in terms of both global(p0.16) and local (p0.02) accuracy.

a data term. This is in contrast with the more common sce-nario in which full shapes are being matched – thus allow-ing to employ more robust, globally-aware features such aslandmark matches, repeatable surface regions, and variousspectral quantities [OBCS∗12].

Local descriptors. We experimented with several choicesof descriptors, driven both by efficiency and accuracyconsiderations. In particular, we used combinations ofSHOT [TSDS10], Mesh Saliency (MS) [LVJ05], and Inte-gral Hashes (IH) [ART15]. Other robust descriptors suchas curvature and Integral Invariants [PWHY09] were alsotested, but ultimately excluded from the analysis due to theirsensitivity to surface noise or large amounts of partiality (re-sulting, for instance, in inconsistent voxelizations in the caseof [PWHY09]). We remark that all the considered descrip-tors are based on extrinsic quantities. SHOT was computedusing 10 normal bins (352 dimensions), while both IH andMS were evaluated over 5 scales (5 dimensions each).

In Table 1 we report a performance comparison of ourmethod on the TOSCA dataset (regular cuts), with differentdescriptor combinations as the data term.

Support size. Since the descriptors are defined on a localneighborhood at each point, it is natural to ask to what ex-tent the size of the support plays a role in the matching accu-racy. For this purpose, we analyzed the performance of ourmethod across increasing support sizes. The results of thisexperiment, together with an illustration of the boundary ef-fects due to partiality, are given in Fig. 9. We emphasize that,differently from other methods [ART15, PBB13] which ig-nore points close to the boundary in order to avoid boundaryeffects, in our formulation we retain all shape points. This ismotivated by our choice of a robust norm for the data term,as also depicted in Fig. 2.

6.2. Sensitivity analysis

We conducted a set of experiments aimed at evaluating thesensitivity of our approach to different parametrizations. Asin the previous section, in order to reduce overfitting we onlyused a subset of TOSCA (regular cuts), namely composed ofthe cat and victoria shape classes (20 pairs).

Rank. In the first experiment we study the change in ac-curacy as the rank of the functional map is increased; this

submitted to Eurographics Symposium on Geometry Processing (2015)

Page 9: arXiv:1506.05274v1 [cs.CV] 17 Jun 2015 · Figure 2: Action of the mixed norm kCA B(h(v))k2;1. Left: Noisy landmark matches between complete and partial shape, represented as Gaussian

E. Rodolà et al. / Partial Functional Correspondence 9

Partiality 20% 40% 80%60%

Partiality (%)10 20 30 40 50 60 70 80 900

20

40

60

80

100e=0.16 Ourse=0.16 Ovsjanikov et al.

e=0.02 Ourse=0.02 Ovsjanikov et al.

p e

Figure 10: Comparisons with the baseline functional mapspipeline in terms of p0.02 and p0.16 at increasing levels ofpartiality. The two methods perform comparably well at lowpartiality; as the amount of missing parts increases, ourmethod maintains high accuracy (example solutions on top)while the quality of [OBCS∗12] drops significantly.

Geodesic error0 0.05 0.1 0.15 0.2 0.25

% C

orre

spon

denc

es

0

20

40

60

80

100

OursOvsjanikov et al.

10050

100150

150

50

Geodesic error0 0.05 0.1 0.15 0.2 0.25

0

20

40

60

80

100

Cuts - OursCuts - Ovsjanikov et al.Holes - OursHoles - Ovsjanikov et al.

Figure 11: Left: Correspondence quality obtained on a sub-set of TOSCA at increasing rank (reported as labels on top ofthe curves). Right: Full comparisons on the TOSCA datasets“regular cuts” and “irregular holes”.

corresponds to using an increasing number of basis func-tions for the two shapes being matched. For this experi-ment we compare with the baseline method of Ovsjanikov etal. [OBCS∗12] by using the same dense descriptors as ours.For fair comparisons, we did not impose map orthogonal-ity or operator commutativity constraints [OBCS∗12], whichcannot obviously be satisfied due to partiality. The results ofthis experiment are reported in Fig. 11 (left). As we can seefrom the plots, our method allows to obtain more accuratesolutions as the rank increases, while an opposite behavioris observed for the other method.

Representation. Our method is general enough to be ap-plied to different shape representations, as long as a properdiscretization of the Laplace operator is available. In Fig. 12we show some qualitative examples of correspondences pro-duced by our algorithm on point clouds, where we used themethod described in [BSW09] to construct a discrete Lapla-cian. Due to the non-rigid deformations the point cloud is

Figure 12: Partial functional correspondence with pointclouds. Similar colors encode corresponding points.

allowed to undergo, this is traditionally considered a partic-ularly challenging problem, with few methods currently ca-pable of giving satisfactory solutions without exploiting con-trolled conditions or domain-specific information (e.g., theknowledge that the shape being matched is that of a human).These are, to the best of our knowledge, the best results tobe published so far for this particular class of problems.

6.3. Comparisons

We compared our method with the baseline approach ofOvsjanikov et al. [OBCS∗12] on both datasets (200 shapepairs in total); the results are shown in Fig. 11 (right). As anadditional experiment, we compared the two methods acrossincreasing amounts of partiality. The rationale behind thisexperiment is to show that, at little or no partiality, our ap-proach converges to the one described in [OBCS∗12], cur-rently among the state of the art in non-rigid shape matching.However, as partiality increases so does the sensitivity of thelatter method. Fig. 10 shows the results of this experiment.

Parameters for our method were chosen on the basis ofthe previous analysis. Specifically, we used dense SHOT de-scriptors as data, k = 100 eigenfunctions per shape, and setµ1 = µ3 = 1, µ4 = µ5 = 103, and µ2 = 102. Additional ex-amples of partial matchings obtained with our method areshown in Fig. 13 and in the supplementary material.

7. Discussion and conclusions

In this paper we tackled the problem of dense matching ofdeformable shapes under partiality transformations. We castour formulation within the framework of functional maps,which we adapted and extended to deal with this more chal-lenging scenario. Our approach is fully automatic and makes

submitted to Eurographics Symposium on Geometry Processing (2015)

Page 10: arXiv:1506.05274v1 [cs.CV] 17 Jun 2015 · Figure 2: Action of the mixed norm kCA B(h(v))k2;1. Left: Noisy landmark matches between complete and partial shape, represented as Gaussian

10 E. Rodolà et al. / Partial Functional Correspondence

Figure 13: Examples of partial functional correspondence obtained with our method on the proposed datasets. Notice howregions close to the boundary are still accurately matched despite the noisy descriptors.

exclusive use of dense local features as a measure of simi-larity. Coupled with a robust prior on the functional corre-spondence derived from a perturbation analysis of the shapeLaplacians, this allowed us to devise an effective optimiza-tion process with remarkable results on very challengingcases. In addition to our framework for partial functionalcorrespondence we also introduced two new datasets com-prising hundreds of shapes, which we hope will foster fur-ther research on this challenging problem.

One of the main issues of our method concerns the exis-tence of multiple optima, which is in turn related to the pres-ence of non-trivial self-isometries on the considered man-ifolds. Since most natural shapes are endowed with intrin-sic symmetries, one may leverage this knowledge in or-der to avoid inconsistent matchings. For example, includ-ing a smoothness prior on the correspondence might allevi-ate such imperfections and thus provide better-behaved solu-tions. Secondly, since the main focus of this paper is on tack-ling partiality rather than general deformations, our currentformulation does not explicitly address the cases of topo-logical changes and inter-class similarity (e.g., matching aman to a gorilla). However, the method can be easily ex-tended to employ more robust descriptors such as pairwisefeatures [RABT13,vKZH13], or to simultaneously optimizeover ad-hoc functional bases on top of the correspondence.Finally, extending our approach to tackle entire shape col-lections, as opposed to individual pairs of shapes, representsa further exciting direction of research.

Appendix A - Discretization

We describe more in depth the discretization steps and givethe implementation details of our algorithm (we skip trivialderivations for the sake of compactness). Detailed gradientsfor each term are given in Appendix B.

Mesh parametrization. We denote by S a triangle mesh ofn points, composed of triangles S j for j = 1, . . . ,m. In the

following derivations, we will consider the classical triangle-based parametrization described by the charts x j : R2→ R3

x j(α,β) = x j,1 +α(x j,2− x j,1)+β(x j,3− x j,1) , (19)

with α ∈ [0,1] and β ∈ [0,1−α]. With x j,k ∈ R3 we denotethe 3D coordinates of vertex k ∈ {1,2,3} in triangle S j .

Each triangle S j is equipped with a discrete metric tensorwith coefficients

g j =

(E j FjFj G j

), (20)

where E j = ‖x j,2−x j,1‖2, Fj = 〈x j,2−x j,1,x j,3−x j,1〉, andG j = ‖x j,3−x j,1‖2. The volume element for the j-th triangle

is then given by√

detg j =√

E jG j−F2j .

Integral of a scalar function. Scalar functions f : S→ Rare assumed to behave linearly within each triangle. Hence,f (x(α,β)) is a linear function of (α,β) and it is uniquelydetermined by its values at the vertices of the triangle. Theintegral of f over S j is then simply given by:∫ 1

0

∫ 1−α

0f (α,β)

√detg jdβdα (21)

=∫ ∫

f (0,0)(1−α−β)+ f (1,0)α+ f (0,1)β√

detg jdβdα

=16( f (0,0)+ f (1,0)+ f (0,1))

√E jG j−F2

j

=13( f (0,0)+ f (1,0)+ f (0,1))area(S j) , (22)

where f (0,0) = f (x j,1), f (1,0) = f (x j,2), and f (0,1) =f (x j,3).

Gradient of a scalar function. For the intrinsic gradient off we get the classical expression in local coordinates:

∇ f =(

∂x j∂α

∂x j∂β

)( E j FjFj G j

)−1( fαfβ

), (23)

submitted to Eurographics Symposium on Geometry Processing (2015)

Page 11: arXiv:1506.05274v1 [cs.CV] 17 Jun 2015 · Figure 2: Action of the mixed norm kCA B(h(v))k2;1. Left: Noisy landmark matches between complete and partial shape, represented as Gaussian

E. Rodolà et al. / Partial Functional Correspondence 11

where we write fα to denote the partial derivative ∂ f∂α

=f j,2 − f j,1 and similarly for fβ. The norm of the intrinsicgradient over triangle S j is then given by:

‖∇ f‖ (24)

=√〈∇ f ,∇ f 〉

=

√(fα fβ

)( E j FjFj G j

)−1( fαfβ

)

=

√(fα fβ

)( G j −Fj−Fj E j

)(fαfβ

)1√

detg j

=

√√√√ f 2αG j−2 fα fβFj + f 2

βE j

detg j. (25)

Note that, since we take f to be linear, the gradient ∇ f isconstant within each triangle. We can then integrate∇ f overS j as follows:∫

S j

‖∇ f (x)‖√

detg jdαdβ (26)

=∫

S j

√√√√ f 2αG j−2 fα fβFj + f 2

βE j

detg j

√detg jdαdβ

=∫

S j

√f 2αG j−2 fα fβFj + f 2

βE j dαdβ

=12

√f 2αG j−2 fα fβFj + f 2

βE j . (27)

In the following, we writeN andM to denote the partialand full shape respectively. Further, let {λNi }i=1,...,k be thefirst k eigenvalues of the Laplacian on N , and similarly for{λMi }i=1,...,k. The functional map C has size k× k.

Mumford-Shah functional (µ2-term). Following Equa-tions (22) and (27), we immediately obtain:∫

Sξ(v)‖∇v‖dx

=m

∑j=1

∫S j

ξ(α,β)‖∇v‖√

detg jdβdα

=m

∑j=1

√v2

αG j−2vαvβFj + v2βE j

∫S j

ξ(α,β)dβdα

≈ 16

m

∑j=1

√v2

αG j−2vαvβFj + v2βE j(ξ(0,0)+ξ(1,0)+ξ(0,1)) ,

where ξ(0,0) = ξ(v(x j,1)), ξ(1,0) = ξ(v(x j,2)), andξ(0,1) = ξ(v(x j,3)).

Weight matrix (µ3-term). Recall from Section 5 and Fig-ure 6 that an estimate for the rank of C can be easily com-puted as

r = max{i | λNi < maxj

λMj } . (28)

We use this information in order to construct the weight ma-trix W, whose diagonal slope directly depends on r.

To this end, we model W as a regular k× k grid in R2.The slanted diagonal of W is a line segment d(t) = p+ t n

‖n‖with t ∈ R, where p = (1,1)> is the matrix origin, andn = (1,r/k)> is the line direction with slope r/k. The high-frequency spread in C is further accounted for by funnel-shaping W along the slanted diagonal. We arrive at the fol-lowing expression for W:

wi j = e−σ

√i2+ j2‖ n‖n‖ × ((i, j)>−p)‖ , (29)

where the second factor is the distance from the slanted di-agonal d, and σ ∈ R+ regulates the spread around d. In ourexperiments we set σ = 0.03.

Orthogonality (µ4,µ5-terms). For practical reasons, we in-corporate the off-diagonal and diagonal terms within oneterm with the single coefficient µ4,5. In addition, we rewritethe off-diagonal penalty using the following equivalent ex-pression:

∑i 6= j

(CTC)2i j = ‖C>C‖2

F−∑i(C>C)2

ii . (30)

Vector d ∈ Rk is constructed by setting the first r elements(according to (28)) equal to 1, and the remaining k− r ele-ments equal to 0.

Appendix B - Gradients

We find local solutions to each optimization problem by the(nonlinear) conjugate gradient method. In this Section wegive the detailed gradient derivations of all terms involved inthe optimization.

In order to keep the derivations practical, we will modelfunction v by its corresponding n-dimensional vector v. Notethat, depending on the optimization step, the gradients arecomputed with respect to either v or C.

Data term (w.r.t. v). Let q denote the number of corre-sponding functions between the two shapes. Matrices F andG contain the column-stacked functions defined overN andM and have size n× q. The respective projections onto thecorresponding functional spaces ΦΦΦ and ΨΨΨ are stored in thek×q matrices A and B respectively. Let us write Hi j to iden-tify the elements of the matrix CA−B(η(v)), we then have

∂vp‖CA−B(η(v))‖2,1

=∂

∂vp

q

∑j=1

(n

∑i=1

H2i j

) 12

=q

∑j=1

(n

∑i=1

H2i j

)− 12 n

∑i=1

Hi j∂

∂vpHi j . (31)

submitted to Eurographics Symposium on Geometry Processing (2015)

Page 12: arXiv:1506.05274v1 [cs.CV] 17 Jun 2015 · Figure 2: Action of the mixed norm kCA B(h(v))k2;1. Left: Noisy landmark matches between complete and partial shape, represented as Gaussian

12 E. Rodolà et al. / Partial Functional Correspondence

Since B(η(v))i j = ∑nk ΨΨΨ

Tikη(vk)Fk j, we have:

∂vpHi j =

∂vp[CAi j−Bi j] = ΨΨΨ

TipFp j

∂vpη(vp) . (32)

Finally:

∂vpη(vp) = 1− tanh2(2vp−1). (33)

Area term (µ1-term w.r.t. v). The derivative of the dis-cretized area term is:

∂vp

19

(n

∑i=1

(SN )i−n

∑i=1

(SM)iη(vi)

)2

=29

(n

∑i=1

(SN )i−n

∑i=1

(SM)iη(vi)

)SM

∂vpη(vp)

where (SM)i and (SN )i are the local area elements associ-ated with the i-th vertex of meshesM and N respectively.For the derivative of η(vp) see equation (33).

Mumford-Shah functional (µ2-term w.r.t. v). Computingthe gradient ∇v

∫S ξ(v)‖∇v‖dx involves computing partial

derivatives of ξ(v) with respect to v. These are simply givenby:

∂vkξ(vk) =

∂vke−

tanh(2vk−1)4σ2

=−1− tanh2(2vk−1)2σ2 e−

tanh(2vk−1)4σ2 .

In the following derivations we set D j ≡√v2

αG j−2vαvβFj + v2βE j, and D j = 0 whenever ∇v = 0.

The gradient of the Mumford-Shah functional is thencomposed of the partial derivatives:

∂vk

∫S

ξ(v)‖∇v‖dx

=m

∑j=1

∂vk

∫S j

ξ(v)‖∇v‖

=16 ∑

j∈N(k)

∂vkD j(ξ(vk)+ξ(v j,2)+ξ(v j,3))

=16 ∑

j∈N(k)(ξ(vk)+ξ(v j,2)+ξ(v j,3))

∂vkD j +D j

∂vkξ(vk)

=16 ∑

j∈N(k)(ξ(vk)+ξ(v j,2)+ξ(v j,3))

12

1D j

∂vkD2

j +D j∂

∂vkξ(vk)

=16 ∑

j∈N(k)(ξ(vk)+ξ(v j,2)+ξ(v j,3))K j +D j

∂vkξ(vk)

where we write K j ≡ 1D j

((vk − v j,2)(G j − Fj) + (vk −v j,3)(E j − Fj)), and j ∈ N(k) are the indices of the trian-gles containing the k-th vertex. Note that we slightly abusenotation by writing vk, v j,2, and v j,3 to denote the three ver-tices of the j-th triangle, even though in general the orderingmight be different depending on the triangle.

Data term (w.r.t. C). The derivative of the data term withrespect to C is similar to (31). The only difference is in thepartial derivative:

∂CpqHi j =

∂Cpq

k

∑i=1

CikAk j =

{CpqAq j if i = p0 otherwise.

(34)

Weight matrix (µ3-term w.r.t. C). This is simply given by:

∂Cpq‖C◦W‖2

F = Cpq(Wpq)2 . (35)

Orthogonality (µ4,5-term w.r.t. C). The gradient of the lastterm can be finally obtained as:

∂Cpq

[‖C>C‖2

F−∑i(C>C)2

ii +∑i((C>C)ii−di)

2

]

= 4(CCT C)pq +2∑i

[−∑

kC2

ki∂

∂Cpq∑k

C2ki +(∑

kC2

ki−di)∂

∂Cpq∑k

C2ki

]= 4[(CCT C)pq−dqCpq] .

References[AMCO08] AIGER D., MITRA N. J., COHEN-OR D.: 4-points

congruent sets for robust pairwise surface registration. TOG 27,3 (2008), 85. 1

[ART15] ALBARELLI A., RODOLÀ E., TORSELLO A.: Fast andaccurate surface alignment through an isometry-enforcing game.Pattern Recognition 48 (2015), 2209–2226. 1, 8

[BB08] BRONSTEIN A. M., BRONSTEIN M. M.: Not only sizematters: regularized partial matching of nonrigid shapes. In Proc.NORDIA (2008). 1, 3, 5

[BBBK09] BRONSTEIN A., BRONSTEIN M., BRUCKSTEIN A.,KIMMEL R.: Partial similarity of objects, or how to compare acentaur to a horse. IJCV 84, 2 (2009), 163–183. 1

[BBK06] BRONSTEIN A. M., BRONSTEIN M. M., KIMMELR.: Generalized multidimensional scaling: a framework forisometry-invariant partial surface matching. PNAS 103, 5 (2006),1168–1172. 1

[BBK08] BRONSTEIN A., BRONSTEIN M., KIMMEL R.: Nu-merical Geometry of Non-Rigid Shapes. Springer, 2008. 7

[BMAS14] BOUMAL N., MISHRA B., ABSIL P.-A., SEPUL-CHRE R.: Manopt, a Matlab toolbox for optimization on man-ifolds. Journal of Machine Learning Research 15 (2014), 1455–1459. URL: http://www.manopt.org. 5

[BSW09] BELKIN M., SUN J., WANG Y.: Constructing laplaceoperator from point clouds in rd. In Proc. SODA (2009), Societyfor Industrial and Applied Mathematics, pp. 1031–1040. 9

[BWW∗14] BRUNTON A., WAND M., WUHRER S., SEIDEL H.-P., WEINKAUF T.: A low-dimensional representation for robustpartial isometric correspondences computation. Graphical Mod-els 76, 2 (2014), 70 – 85. 1

[Duf59] DUFFIN R. J.: Distributed and lumped networks. Journalof Mathematics and Mechanics 8, 5 (1959), 793–826. 4

[GH97] GARLAND M., HECKBERT P. S.: Surface simplifica-tion using quadric error metrics. In Proc. SIGGRAPH (1997),pp. 209–216. 7

submitted to Eurographics Symposium on Geometry Processing (2015)

Page 13: arXiv:1506.05274v1 [cs.CV] 17 Jun 2015 · Figure 2: Action of the mixed norm kCA B(h(v))k2;1. Left: Noisy landmark matches between complete and partial shape, represented as Gaussian

E. Rodolà et al. / Partial Functional Correspondence 13

[HG13] HUANG Q.-X., GUIBAS L.: Consistent shape maps viasemidefinite programming. Computer Graphics Forum 32, 5(2013), 177–186. 2

[HWG14] HUANG Q., WANG F., GUIBAS L. J.: Functional mapnetworks for analyzing and exploring large shape collections.TOG 33, 4 (2014), 36. 2

[KLCF10] KIM V. G., LIPMAN Y., CHEN X., FUNKHOUSERT. A.: Möbius transformations for global intrinsic symmetryanalysis. Comput. Graph. Forum 29, 5 (2010), 1689–1700. 1

[KLF11] KIM V. G., LIPMAN Y., FUNKHOUSER T. A.: Blendedintrinsic maps. TOG 30, 4 (2011), 79. 1, 8

[LVJ05] LEE C. H., VARSHNEY A., JACOBS D. W.: Meshsaliency. ACM Trans. Graph. 24, 3 (2005), 659–666. 8

[Mac49] MACNEAL R. H.: The solution of partial differentialequations by means of electrical networks. PhD thesis, CaliforniaInstitute of Technology, 1949. 4

[MDSB03] MEYER M., DESBRUN M., SCHRÖDER P., BARRA. H.: Discrete differential-geometry operators for triangulated2-manifolds. Visualization&Mathematics (2003), 35–57. 4, 5

[MH88] MURTHY D. V., HAFTKA R. T.: Derivatives of eigenval-ues and eigenvectors of a general complex matrix. Intl. J. Numer.Met. Eng. 26, 2 (1988), 293–311. 5, 6, 7

[MS89] MUMFORD D., SHAH J.: Optimal approximations bypiecewise smooth functions and associated variational problems.Comm. Pure and Applied Math. 42, 5 (1989), 577–685. 3

[OBCS∗12] OVSJANIKOV M., BEN-CHEN M., SOLOMON J.,BUTSCHER A., GUIBAS L.: Functional maps: a flexible rep-resentation of maps between shapes. ACM Trans. Graph. 31, 4(July 2012), 30:1–30:11. 2, 3, 4, 5, 8, 9

[PBB13] POKRASS J., BRONSTEIN A. M., BRONSTEIN M. M.:Partial shape matching without point-wise correspondence. Nu-mer. Math. Theor. Meth. Appl. 6 (2013), 223–244. 2, 8

[PP93] PINKALL U., POLTHIER K.: Computing discrete minimalsurfaces and their conjugates. Experimental mathematics 2, 1(1993), 15–36. 4

[PWHY09] POTTMANN H., WALLNER J., HUANG Q.-X.,YANG Y.-L.: Integral invariants for robust geometry processing.Comput. Aided Geom. Des. 26, 1 (2009), 37–60. 8

[RABT13] RODOLÀ E., ALBARELLI A., BERGAMASCO F.,TORSELLO A.: A scale independent selection process for 3dobject recognition in cluttered scenes. International Journal ofComputer Vision 102, 1-3 (2013), 129–145. 10

[RBA∗12] RODOLÀ E., BRONSTEIN A., ALBARELLI A.,BERGAMASCO F., TORSELLO A.: A game-theoretic approachto deformable shape matching. In Proc. CVPR (June 2012),pp. 182–189. 2

[SY14] SAHILLIOGLU Y., YEMEZ Y.: Partial 3-d correspon-dence from shape extremities. Computer Graphics Forum 33,6 (2014), 63–76. 2

[TSDS10] TOMBARI F., SALTI S., DI STEFANO L.: Uniquesignatures of histograms for local surface description. In Proc.ECCV (2010), pp. 356–369. 8

[VC02] VESE L. A., CHAN T. F.: A multiphase level set frame-work for image segmentation using the Mumford and Shahmodel. IJCV 50, 3 (2002), 271–293. 3

[VKTS∗11] VAN KAICK O., TAGLIASACCHI A., SIDI O.,ZHANG H., COHEN-OR D., WOLF L., HAMARNEH G.: Priorknowledge for part correspondence. Computer Graphics Forum30, 2 (2011), 553–562. 2

[vKZH13] VAN KAICK O., ZHANG H., HAMARNEH G.: Bilat-eral maps for partial matching. Computer Graphics Forum 32, 6(2013), 189–200. 2, 10

[VKZHCO11] VAN KAICK O., ZHANG H., HAMARNEH G.,COHEN-OR D.: A survey on shape correspondence. ComputerGraphics Forum 30, 6 (2011), 1681–1707. 1

[ZSCO∗08] ZHANG H., SHEFFER A., COHEN-OR D., ZHOUQ., VAN KAICK O., TAGLIASACCHI A.: Deformation-drivenshape correspondence. Computer Graphics Forum 27, 5 (2008),1431–1439. 1

submitted to Eurographics Symposium on Geometry Processing (2015)