Ilya Eckstein Vitaly Surazhsky Craig Gotsman · Figure 1: Texture mapping with hard constraints....

10
EUROGRAPHICS 2001 / A.Chalmers and T.-M.Rhyne Volume 20 (2001), Number 3 (Guest Editors) © The Eurographics Association and Blackwell Publishers 2001. Published by Blackwell Publishers, 108 Cowley Road, Oxford OX4 IJF, UK and 350 Main Street, Malden, MA 02148, USA Texture Mapping with Hard Constraints Ilya Eckstein Vitaly Surazhsky Craig Gotsman Computer Science Department, Technion – Israel Institute of Technology {ekshteyn|vitus|gotsman}@cs.technion.ac.il Abstract We show how to continuously map a texture onto a 3D triangle mesh when some of the mesh vertices are con- strained to have given (u,v) coordinates. This problem arises frequently in interactive texture mapping appli- cations and, to the best of our knowledge, a complete and efficient solution is not available. Our techniques always guarantee a solution by introducing extra (Steiner) vertices in the triangulation if needed. We show how to apply our methods to texture mapping in multi-resolution scenarios and image warping and morphing. 1. Introduction Interactive 3D authoring applications frequently map 2D textures onto a 3D triangle mesh. This involves assigning (u,v) coordinates to the vertices of the 3D mesh, which defines a piecewise affine mapping of the texture image to the 3D mesh triangles. While many applications do not care about the precise values of the (u,v) coordinates, as long as the result looks reasonable, there are applications in which these values are critical. For example, when texture map- ping a photograph of a face to a 3D mesh of a head, it is important that the eyes and the nose in the photograph are aligned precisely with the eyes and the nose in the mesh. Figure 1 illustrates what might happen if these constraints are not satisfied. Precise alignment means that the (u,v) coordinates associated with feature points in the 3D mesh are constrained to very specific values, known as hard con- straints. The (u,v) coordinates of the rest of the mesh are not as constrained, as long as the resulting mapping is visu- ally appealing. By “visually appealing” we mean that the texture is mapped in a continuous manner onto the 3D mesh, namely, does not contain foldovers or tears, and is not too distorted, despite the constraints. While many pa- pers [4,6,11,12,13,14,16,19] deal with the problem of mini- mal-distortion texture-mapping, to the best of our knowledge, none of them permit hard constraints in the mapping. The technique of Levy and Mallet [13] permits soft constraints, i.e. where the constraints are allowed to be only approximately satisfied. This paper shows how to obtain continuous piecewise-affine mappings of a texture map to a 3D mesh with hard constraints. Since, in general, arbitrary hard constraints might prevent any solution to the problem, we present a general technique for satisfying constraints, which involves the introduction of extra vertices into the mesh. These vertices are known as Steiner vertices. Our technique solves the problem of tex- ture mapping with hard constraints and also the following problem that arises when texture mapping a multiresolution mesh, namely, maintaining a legal parametrization of the mesh at all resolution levels: Assume that a full-resolution mesh has a continuous parametrization (i.e. texture coordi- nates), and a simplification algorithm has generated a sequence of vertex removals and hole retriangulations for the mesh (for an example of such an algorithm – see [9]). The texture coordinates of the vertices remaining in the lower resolution meshes are just inherited from the highest resolution mesh, and the new connectivity defines a new parametrization. In many cases, this parametrization of the low resolution mesh will be legal (i.e. the 2D triangulation will not contain foldovers), but this is not guaranteed. We show how, given a high-resolution 3D mesh and its simpli- fication history, it is possible to parametrize the mesh such that the parametrization indeed remains legal throughout the simplification procedure. This paper is organized as follows: Section 2 elaborates on the connection between parametrization of 3D meshes and the embedding of their connectivity graph in the plane. Section 3 describes our parametrization technique in detail. Section 4 provides experimental results related to various applications, and we discuss extensions and conclude in Section 5. 2. Texture Mapping as Parametrization or Embed- ding Assigning texture coordinates to a 3D mesh can be re- garded as a parametrization of the mesh surface, where each 3D vertex is assigned a 2D parameter value, and each point on the mesh surface is parametrized by the appropri- ate convex combination of the parameter values of the ver- tices of the surface triangle in which it resides. Care must be taken that the parametrization is legal, i.e. that no two points on the mesh surface are mapped to the same point in the 2D parameter domain. This may be achieved if the

Transcript of Ilya Eckstein Vitaly Surazhsky Craig Gotsman · Figure 1: Texture mapping with hard constraints....

Page 1: Ilya Eckstein Vitaly Surazhsky Craig Gotsman · Figure 1: Texture mapping with hard constraints. (a) Texture map with feature points marked. These should be mapped to the appropriate

EUROGRAPHICS 2001 / A.Chalmers and T.-M.Rhyne Volume 20 (2001), Number 3 (Guest Editors)

© The Eurographics Association and Blackwell Publishers 2001. Published by Blackwell Publishers, 108 Cowley Road, Oxford OX4 IJF, UK and 350 Main Street, Malden, MA 02148, USA

Texture Mapping with Hard Constraints

Ilya Eckstein Vitaly Surazhsky Craig Gotsman

Computer Science Department, Technion – Israel Institute of Technology

{ekshteyn|vitus|gotsman}@cs.technion.ac.il

Abstract We show how to continuously map a texture onto a 3D triangle mesh when some of the mesh vertices are con-strained to have given (u,v) coordinates. This problem arises frequently in interactive texture mapping appli-cations and, to the best of our knowledge, a complete and efficient solution is not available. Our techniques always guarantee a solution by introducing extra (Steiner) vertices in the triangulation if needed. We show how to apply our methods to texture mapping in multi-resolution scenarios and image warping and morphing.

1. Introduction Interactive 3D authoring applications frequently map 2D textures onto a 3D triangle mesh. This involves assigning (u,v) coordinates to the vertices of the 3D mesh, which defines a piecewise affine mapping of the texture image to the 3D mesh triangles. While many applications do not care about the precise values of the (u,v) coordinates, as long as the result looks reasonable, there are applications in which these values are critical. For example, when texture map-ping a photograph of a face to a 3D mesh of a head, it is important that the eyes and the nose in the photograph are aligned precisely with the eyes and the nose in the mesh. Figure 1 illustrates what might happen if these constraints are not satisfied. Precise alignment means that the (u,v) coordinates associated with feature points in the 3D mesh are constrained to very specific values, known as hard con-straints. The (u,v) coordinates of the rest of the mesh are not as constrained, as long as the resulting mapping is visu-ally appealing. By “visually appealing” we mean that the texture is mapped in a continuous manner onto the 3D mesh, namely, does not contain foldovers or tears, and is not too distorted, despite the constraints. While many pa-pers [4,6,11,12,13,14,16,19] deal with the problem of mini-mal-distortion texture-mapping, to the best of our knowledge, none of them permit hard constraints in the mapping. The technique of Levy and Mallet [13] permits soft constraints, i.e. where the constraints are allowed to be only approximately satisfied. This paper shows how to obtain continuous piecewise-affine mappings of a texture map to a 3D mesh with hard constraints. Since, in general, arbitrary hard constraints might prevent any solution to the problem, we present a general technique for satisfying constraints, which involves the introduction of extra vertices into the mesh. These vertices are known as Steiner vertices. Our technique solves the problem of tex-ture mapping with hard constraints and also the following problem that arises when texture mapping a multiresolution

mesh, namely, maintaining a legal parametrization of the mesh at all resolution levels: Assume that a full-resolution mesh has a continuous parametrization (i.e. texture coordi-nates), and a simplification algorithm has generated a sequence of vertex removals and hole retriangulations for the mesh (for an example of such an algorithm – see [9]). The texture coordinates of the vertices remaining in the lower resolution meshes are just inherited from the highest resolution mesh, and the new connectivity defines a new parametrization. In many cases, this parametrization of the low resolution mesh will be legal (i.e. the 2D triangulation will not contain foldovers), but this is not guaranteed. We show how, given a high-resolution 3D mesh and its simpli-fication history, it is possible to parametrize the mesh such that the parametrization indeed remains legal throughout the simplification procedure. This paper is organized as follows: Section 2 elaborates on the connection between parametrization of 3D meshes and the embedding of their connectivity graph in the plane. Section 3 describes our parametrization technique in detail. Section 4 provides experimental results related to various applications, and we discuss extensions and conclude in Section 5. 2. Texture Mapping as Parametrization or Embed-

ding Assigning texture coordinates to a 3D mesh can be re-garded as a parametrization of the mesh surface, where each 3D vertex is assigned a 2D parameter value, and each point on the mesh surface is parametrized by the appropri-ate convex combination of the parameter values of the ver-tices of the surface triangle in which it resides. Care must be taken that the parametrization is legal, i.e. that no two points on the mesh surface are mapped to the same point in the 2D parameter domain. This may be achieved if the

Page 2: Ilya Eckstein Vitaly Surazhsky Craig Gotsman · Figure 1: Texture mapping with hard constraints. (a) Texture map with feature points marked. These should be mapped to the appropriate

Eckstein, Surazhsky and Gotsman / Texture Mapping with Hard Constraints

© The Eurographics Association and Blackwell Publishers 2001.

straight-line graph in the plane induced by the 3D vertex parameter values and its connectivity graph is a legal trian-gulation of the 2D parameter points, i.e. that the edges do not intersect and all the triangles have the same orientation. In this case we may also say that the 3D mesh has been legally embedded in the plane. Note that actually only the connectivity of the 3D mesh plays a role in the problem. The geometry of the mesh, i.e. the 3D coordinates of the vertices, does not have to be used in order to compute an embedding. Since it is impossible to legally embed a mesh topologi-cally equivalent to a closed sphere in the plane, we restrict our discussion to the case of a mesh with a boundary, which is topologically compatible with a rectangular tex-ture map. Closed meshes, as also those of higher genus, may be reduced to the case we treat by cutting the mesh into pieces, but then the issue of continuity across bounda-ries must be addressed. Embedding a 3D mesh with a single boundary (i.e. topo-logically equivalent to the disk) in the plane has been ad-dressed by many authors. Since usually the embedding is not unique, a “good” embedding, in a variety of senses, is sought. One of the main concerns is that the embedding preserve the geometric properties of the 3D mesh, e.g. edge lengths, areas, angles, etc. Tutte [18] first provided the most basic way of computing embeddings using barycen-tric coordinates, in which the boundary vertices are mapped to a convex position and each interior vertex is positioned at the centroid of the locations of its neighbors. Computing this embedding involves solving a set of linear equations derived from the connectivity graph adjacency matrix, and results in an embedding with the minimal sum of squares of the edge lengths. Floater [7] later generalized this to arbi-trary convex combinations and used this mechanism to generate “shape-preserving” parametrizations. Eck et al [6] used harmonic mappings to achieve a similar effect and minimize distortions in a sense. Other embedding methods,

some which do not require mapping the boundary vertices to a convex shape, are described by Zigelman et al [19] and Sheffer and de Sturler [16]. None of these methods are easily extendable to parametrizations with hard constraints. A legal triangulation on n points may be formulated as a solution to a set of t quadratic inequalities, where t is the number of triangles, as follows: If {(xi,yi), i=1,..,n} are the locations of the n vertices, and (x1,y1), (x2,y2) (x3,y3) are the coordinates of some triangle as defined by the connectivity graph, the following inequality must hold:

0111

33

22

11

>yxyxyx

where |.| denotes determinant. Similar inequalities must hold for all t triangles. This means that the signed area of all the triangles must be positive, implying that the triangles be non-degenerate and consistently oriented. Note that if the boundary vertices are fixed, boundary triangles impose linear (or constant) inequalities. Sheffer and de Sturler [16] have formulated a different set of (highly non-linear) inequalities on the angles of a trian-gulation to guarantee legality. Both formulations can easily incorporate hard constraints as extra equalities, but solving either of them involve heavy numerics, which become slow, unwieldy and unstable, hence unpractical for large meshes. Adding a cost function to minimize distortion fur-ther complicates matters, as the optimization process might get stuck in local minima. Additionally, adding hard con-straints may eliminate all solutions, such as we will see in the next section. Standard interpolation techniques to map R3 to R2, such as radial basis functions (RBF), based on Gaussian [1] or B-Spline basis functions [5] may satisfy hard constraints, but cannot guarantee that the induced 2D triangulation will be valid, i.e. not contain foldovers, as Figure 2 demonstrates.

(a) (b) (c) (d)

Figure 1: Texture mapping with hard constraints. (a) Texture map with feature points marked. These should be mapped to the appropriate geometric features marked on (b). (b) 3D face with features corresponding to those of (a). (c) Render-ing of result when constraints are ignored. Texture coordinates were generated using the “shape preservation” method of Floater [7]. Note the texture nose and the geometric nose. (d) Rendering of result when constraints are satisfied using our techniques.

Page 3: Ilya Eckstein Vitaly Surazhsky Craig Gotsman · Figure 1: Texture mapping with hard constraints. (a) Texture map with feature points marked. These should be mapped to the appropriate

Eckstein, Surazhsky and Gotsman / Texture Mapping with Hard Constraints

© The Eurographics Association and Blackwell Publishers 2001.

Alexa [2] addressed the problem of embedding a closed genus-0 3D mesh on the sphere with constraints, in order to align the vertices of two such meshes in a 3D morphing application. This is quite similar to our problem (which occurs in the plane). However, the heuristic (iterative) solu-tion offered by Alexa is quite slow, and not guaranteed to find a solution, even if one exists. It also does not introduce Steiner vertices to create a solution when one is not possi-ble without them. Thus an algorithmic solution is sought, which is both effi-cient and will produce some solution (by introducing Steiner vertices) even if the constraints are difficult (or impossible) to satisfy. The ability to solve the problem efficiently for all inputs is what distinguishes our solution from all previous work. 3. Embedding with Hard Constraints Embedding a 3D mesh in the plane with hard constraints seems to be a very difficult problem, even if such an em-bedding exists, which might not be the case. Hence, we make the problem easier by allowing the 2D embedding to contain extra vertices, called Steiner vertices. This means that a 3D edge will be mapped to a 2D path of edges. Steiner vertices are frequently used in graph-drawing ap-plications, which are also a variant of the embedding prob-lem. See e.g. Pach and Wenger [15]. Figure 3 shows an example of a constrained connectivity for which a straight-line 2D embedding does not exist, but the introduction of Steiner vertices admits a solution. 3.1 The embedding procedure Our embedding procedure is as described in Algorithm 1. It first removes all the unconstrained vertices from the mesh (the simplification phase), and then restores them in reverse order (the reconstruction phase). The reconstruction phase maintains the invariant that the embedding remains legal after each vertex insertion. Steps 2, 3c and 3d of Algorithm 1 deserve further explanation. Let us begin with the latter two. The “kernel fix” procedure tries to move vertices of the polygon in which the vertex in question lies in order that the polygon be star-shaped (have a non-empty kernel). Note that if a vertex is not located inside the (possible empty) kernel of its neighborhood polygon, at least one of its neighbors must be in a similar predicament. Hence we (serially) examine locations of all vertices and identify all those which do not reside inside the kernel of their neighborhood polygons. For each such vertex, we position it inside the corresponding kernel if its neighbors form a star-shaped polygon. Were there no hard constraints, this procedure could ultimately result in a legal triangulation of the point set. Figure 4 illustrates this procedure. If the con-straints do not allow this procedure to converge, we choose the location of the vertex to be the point in the plane which minimizes the variance of the areas of the triangles formed by the vertex and the polygon vertices.

The “kernel fix” procedure increases the chance of a legal embedding, but can not guarantee it, since there exist inputs for which a solution does not exist. When the “kernel fix” procedure fails, we resort to Steiner vertices. In this case, we discard all the changes performed by the “kernel fix” procedure and apply the “Steiner embed” procedure. Note that the neighborhood polygon is always simple (since the embedding generated until now is legal due to the main invariant). Hence, all we have to do is to legally position the new vertex inside the polygon. The “Steiner embed” procedure positions that vertex and additional Steiner verti-ces in the polygon such that the new vertex is connected to all polygon vertices by polygonal paths, instead of straight-line edges, and then triangulates the resulting non-triangular faces inside the polygon. The method uses a variant of Gotsman and Surazhsky [10] on the so-called “universal” method of Aronov et al [3] for compatibly tri-angulating pairs of polygons. See Figure 5 for an example.

1. (The embedding phase) Ignore the hard constraints.

Compute barycentric coordinates for the vertices (using the "shape preservation" method of Floater [7]), and use them to generate a planar embedding of the connectivity graph.

2. (The simplification phase) Simplify the mesh con-

nectivity using “constrained simplification” (Algo-rithm 2), until only the subset of constrained vertices remain embedded at their prescribed locations with some legal triangulation. Maintain a record of the entire simplification history, together with the 2D barycentric coordinates of each removed vertex rela-tive to its neighbors at the moment of removal.

3. (The reconstruction phase) Restore the uncon-

strained vertices of the mesh (and their connectivity) in the reverse order of simplification, by doing the following:

a. Embed the vertex at a location in the plane de-

termined by the barycentric coordinates of its neighbors at that level, as computed in step 2.

b. If the computed location of the vertex is outside the kernel of the polygon formed by its neighbors (meaning the embedding is now ille-gal), place the vertex at at arbitrary location in-side the kernel.

c. If the kernel is empty – run the “kernel fix” procedure on the triangulation.

d. If the kernel is still empty - run the “Steiner embed” procedure on the polygon.

Algorithm 1: Constrained embedding. 3.2 Constrained simplification The main invariant of our algorithm is that each iteration of the reconstruction phase starts with a legal embedding and

Page 4: Ilya Eckstein Vitaly Surazhsky Craig Gotsman · Figure 1: Texture mapping with hard constraints. (a) Texture map with feature points marked. These should be mapped to the appropriate

Eckstein, Surazhsky and Gotsman / Texture Mapping with Hard Constraints

© The Eurographics Association and Blackwell Publishers 2001.

maintains its legality. Naturally, we have to ensure that the last step of the procedure, i.e. the embedding of only the hard constrained vertices, is legal too. The problem is that if we employ an arbitrary simplification scheme in the sim-plification phase, the resulting triangulation at the last step may be illegal when we apply the geometric constraints. A better alternative is to work towards some legal triangula-tion of the constrained vertices, e.g. the Delaunay triangula-tion. We call this procedure “constrained simplification”, and need the following definitions to describe it: • A fixed vertex is a vertex constrained in the input. The

set of fixed vertices of a triangulation will be denoted by VF.

• A fixed edge is an edge connecting two fixed vertices. The set of fixed edges of a triangulation will be denoted by EF.

• A fixed triangle is a triangle with three fixed vertices (thus, all its edges are fixed as well).

• T(V,E) is the initial (non-simplified) planar triangula-tion and TF(VF,EF) is the desired output of the simplifi-cation procedure. VF⊆V.

• A cut-border (CB) in a triangulation T(V,E) is a simple closed cycle of edges in EF ∩E.

The simplification algorithm proceeds as described in Al-gorithm 2 and illustrated in Figure 6. It maintains a dy-namic cut-border which separates the region of the mesh that has been simplified from that not yet simplified. Algorithm 2 maintains the invariant that the cut-border is always simple. Note that the intermediate triangulations are not required to be legal during the simplification process. However, we still try to keep them as legal as possible so that the barycentric coordinates stored for the removed vertices are legal, which is desirable for the subsequent reconstruction step. This scheme is not guaranteed to suc-ceed. However, experiments show that in practical scenar-ios, failures are extremely rare. 3.3 Minimizing distortion Although Algorithm 1 yields a legal planar triangulation, the result may be significantly distorted, in the sense that the triangles may be long and “skinny”, the constraints having had a global effect on the result. This is undesirable, especially in interactive texture mapping applications, where dragging a control point should have only a local effect. The number of Steiner vertices might also be unnec-cesarily large. To overcome this, we employ an on-the-fly relaxation procedure, aimed at keeping the distortion mini-mal during the reconstruction phase of the algorithm. The idea behind this procedure is as follows: at the simplifica-tion phase we choose several levels of the multiresolution hierarchy to be fully recorded levels: for each such level we record barycentric coordinates of every unconstrained inte-rior vertex (relative to its neighbors). At the reconstruction phase, each time we reach a fully recorded level we attempt to re-embed every unconstrained vertex inside its neighborhood polygon according to its original barycentric

coordinates, applying steps 3(a), 3(b) of Algorithm 1. If the attempt fails for some vertex, it remains in its existing loca-tion, which is always legal. In this manner the relaxation procedure reproduces the original (unconstrained) barycen-tric coordinates for many unconstrained vertices, minimiz-ing distortion.

Constrained-simplification(T,TF) 1. Initialize CB to be the boundary of T (since, in par-

ticular, the boundary is fixed). 2. While T contains non-fixed vertices (V∩VF≠φ):

a. Let e be in CB and let v be the vertex within CB forming a triangle with e.

b. If v∈VF, remove e from CB and add to CB the edges that connect the endpoints of e to v and goto (a). See Figure 6(b),(c).

c. If v∉VF, remove v from T (Figure 6(d)) and re-triangulate the resulting polygonal hole by add-ing all the triangles of TF incident on CB possi-ble. We might remain with holes that still have to be retriangulated. See Figure 6(e).

d. Apply Delaunay triangulation to the holes (Fig-ure 6(f)). If any fixed edges were added in this triangulation, try to eliminate them by flipping edges (Figure 6(g)). We want to avoid fixed edges, since they might cause the cut-border to be non-simple in the future.

e. If the previous step failed, find any (not necessarily geometrically legal) retriangulation which does not add fixed edges. If nothing found, return to step (c) and attempt to remove a different vertex v∉VF. If all such v fail - report failure.

3. Report success.

Algorithm 2: Constrained simplification.

(a) (b)

Figure 2: Constrained embedding using RBF. (a) A planar triangulation with a fixed boundary and two constrained interior vertices: a and b. (b) Changing the position of b and using RBF to generate new positions for the other un-constrained interior vertices. The result is not a legal em-bedding

Page 5: Ilya Eckstein Vitaly Surazhsky Craig Gotsman · Figure 1: Texture mapping with hard constraints. (a) Texture map with feature points marked. These should be mapped to the appropriate

Eckstein, Surazhsky and Gotsman / Texture Mapping with Hard Constraints

© The Eurographics Association and Blackwell Publishers 2001.

(a) (b) Figure 3: Straight-line embedding using Steiner vertices. (a) A connectivity graph with all vertices constrained which can-not be embedded as a triangulation in the plane. (b) Embedding for the input of (a) using (green) Steiner vertices. The dashed edges are added to keep the graph a legal triangulation.

v u

0

vu

(a) (b) (c) Figure 4: The “kernel fix” procedure. (a) The red polygon is not star-shaped, hence no location of the central vertex v will result in a legal triangulation. (b) The vertex u neighboring on v is outside its neighborhood polygon’s kernel. (c) Moving u into the kernel of its neighborhood polygon solves both problems.

(a) (b)

Figure 5: Inserting a vertex into a non-star-shaped polygon using Steiner vertices. (a) Polygon and inserted central vertex. Note the crossing edges. (b) Spiderweb triangulation of polygon using (green) Steiner vertices. The brown edges partition the polygon into convex parts, the blue paths correspond to the original blue edges and the dashed ones were added to obtain a legal triangulation.

v

Page 6: Ilya Eckstein Vitaly Surazhsky Craig Gotsman · Figure 1: Texture mapping with hard constraints. (a) Texture map with feature points marked. These should be mapped to the appropriate

Eckstein, Surazhsky and Gotsman / Texture Mapping with Hard Constraints

© The Eurographics Association and Blackwell Publishers 2001.

exterior(simplified)

cut border

interior(non-sim p

lified)

(a) (b) (c)

forbiddenedge

(d) (e) (f)

(g)

Figure 6: Constrained simplification of a triangulation with (marked) fixed vertices. (a) A (red) cut-border is maintained. All fixed (filled) triangles are outside the cut-border (b),(c) Updating the cut-border after a new fixed vertex (the large one) is encountered. (d) Removing a non-fixed vertex. (e) Identifying the one fixed triangle that may be added at the current stage. (f) Triangulating the remaining holes. Note the fixed edge added by the retriangulation procedure, which is forbidden at this stage. (g) Several edge flips performed to eliminate the forbidden edge.

The relaxation also reduces the number of Steiner vertices, as we are able to remove some of those introduced during the reconstruction phase. These vertices were critical in the coarse triangulations, but as more original vertices are re-stored, the triangulation becomes more flexible and many Steiner vertices are no longer needed. Hence, when in the relaxation routine we discover that the neighborhood poly-gon of a vertex previously embedded using Steiner vertices is now star-shaped, we may eliminate all these Steiner ver-tices. See Figure 7 for some results. 4. Experimental Results In this section we demonstrate the results achieved by our algorithm and compare them with results from regular tex-ture-mapping algorithms and commercial packages. 4.1 3D texture mapping Figure 8(a) shows a 2D texture and 8(b) a 3D mesh of a female torso, where a correspondence between various

features on the 2D texture and 3D meshes are marked. The objective is to texture map the image onto the mesh such that the corresponding features coincide, and it looks like a well-fitted vest. Figure 8(c) shows what a interactive au-thoring system not supporting constrained texture mapping generates and Figure 8(d) shows our results. The 3D mesh in this example contained 928 vertices and we introduced 32 constraints. Solving for the constrained parametrization took less than one second on a state-of-the-art PC, which was typical for all the examples we experimented with. 4.2 Multiresolution texture mapping In most multiresolution scenarios, texture coordinates are defined for the highest resolution, and inherited in lower resolution versions of the model, which contain a retriangu-lated subset of the high resolution vertex set. However, nothing guarantees that the inherited parametrization, i.e. embedding in the plane of the subset, triangulated using the low-resolution connectivity, will be legal. If the 3D mesh is

Page 7: Ilya Eckstein Vitaly Surazhsky Craig Gotsman · Figure 1: Texture mapping with hard constraints. (a) Texture map with feature points marked. These should be mapped to the appropriate

Eckstein, Surazhsky and Gotsman / Texture Mapping with Hard Constraints

© The Eurographics Association and Blackwell Publishers 2001.

well-behaved, illegal triangulations at lower resolutions will rarely occur, but meshes with complex geometry can-not avoid this. Hence, an important practical problem is, given a mesh and its simplification history – a sequence of vertex removals and hole retriangulations – generate a pa-rametrization which will be legal at all resolution levels. Our algorithm to achieve this is as described in Algorithm 3, possibly also requiring introduction of Steiner vertices. Note that we do not use the “kernel fix” procedure at Step 3 of Algorithm 3 because we are not allowed to change the locations of the vertices already embedded. In this manner we preserve the legality of the triangulation for all levels of resolution already reconstructed. Figure 9 shows parametrizations of a mesh at three levels of detail, when the original “arbitrary” parametrization is used, and when the one generated by Algorithm 3 is used. The former loses its validity very quickly as resolution is reduced, while the latter remains valid throughout the sim-plification procedure. 4.3 2D warping and morphing Another application of our constrained texture mapping procedure is to 2D image warping. This is actually no more than a special case of the problem described above, where the 3D mesh is just a 2D triangulation. The exact problem is: Given a 2D triangulation of an image with feature verti-ces, and new locations for a subset of the vertices, is it pos-sible to generate new locations for the rest of the vertices such that the resulting induced triangulation (using the original connectivity) is still legal? In this case we say that the two triangulations are compatible. This will facilitate a piecewise-affine warp of the image such that the constraints (moved vertices) are satisfied. Once we have generated two compatible triangulations, we can use the method of Floater and Gotsman [8] to morph between them. The objective is to generate vertex trajecto-ries interpolating the two sets of vertex locations, such that

they induce legal intermediate triangulations. Naïve morph-ing methods, e.g. those obtained by using straight line tra-jectories, fail on most inputs, especially those involving rotations of the points. The method of Floater and Gotsman interpolates the barycentric coordinate representation of the triangulation, hence maintains legality of the triangulation throughout the morph. Figure 10 shows an example where compatible triangulations have been generated and morphed.

1. Execute steps (1) and (2) of Algorithm 1, with the

sole difference that the simplification process used here is the given simplification history (instead of the “constrained simplification”), until the coars-est simplification level is reached.

2. If the parametrization at the coarsest level is ille-gal, run the “kernel fix” procedure to make it le-gal. This is guaranteed to succeed, as there are no hard-constrained vertices.

3. Restore the vertices of the mesh (and their connectivity) in the reverse order of simplification, performing the following steps:

• Embed the vertex at a location in the plane de-

termined by the barycentric coordinates of its neighbors at that level, as computed in step 2.

• If the computed location of the vertex is not in the kernel of the vertex’s neighborhood poly-gon (meaning the embedding is illegal), place it inside the kernel.

• If the kernel is empty - run the “Steiner em-bed” procedure on the polygon.

Algorithm 3: Multiresolution parametrization.

(a) (b) (c)

Figure 7: Minimizing distortion. (a) Original mesh in which marked vertices will be moved to new locations. (b) New pa-rametrization with (blue) Steiner vertices generated by our algorithm, satisfying hard constraints after vertices have been moved (essentially rotated). No relaxation was used. (c) Result after relaxation.

Page 8: Ilya Eckstein Vitaly Surazhsky Craig Gotsman · Figure 1: Texture mapping with hard constraints. (a) Texture map with feature points marked. These should be mapped to the appropriate

EUROGRAPHICS 2001 / A.Chalmers and T.-M.Rhyne Volume 20 (2001), Number 3 (Guest Editors)

© The EUROGRAPHICS Association and Blackwell Publishers 2001. Published by Blackwell Publishers, 108 Cowley Road, Oxford OX4 IJF, UK and 350 Main street, Malden, MA 02148, USA

(a) (b) (c) (d)

Figure 8: Texture mapping with hard constraints. (a) 2D texture with some feature points. (b) 3D mesh with features corre-sponding to those in (a). (c) Results when constraints are ignored. (d) Our result when the constraints are satisfied.

(a) (b) (c)

(d) (e) (f)

Figure 9: Multiresolution parametrization. (a) A given parametrization of a 3D mesh. (b,c) Parametrizations of two lower resolutions of the mesh, where the parameter values are inherited from (a). Note the multiple edge intersections, making the parametrizations illegal. (d-f) Parametrization of the mesh generated using our method, shown at the same levels of detail (and identical connectivities) as (a-c). Note that (e) contains two Steiner vertices, marked by ‘+’s.

Page 9: Ilya Eckstein Vitaly Surazhsky Craig Gotsman · Figure 1: Texture mapping with hard constraints. (a) Texture map with feature points marked. These should be mapped to the appropriate

Eckstein, Surazhsky and Gotsman / Texture Mapping with Hard Constraints

© The Eurographics Association and Blackwell Publishers 2001.

(a) (b) (c)

Figure 10: Morphing using constrained parametrization. (a) Source image with triangulation of control points. Purple points are features that will be moved to new locations in the target image. (b) Target image with new triangulation satisfy-ing constraints generated by our method. (c) Result generated by SurfaceSuite package in an attempt to satisfy the con-straints. Note the tear (discontinuity) in the image. Bottom line: Morph between (a) and (b) generated by algorithm of Floater and Gotsman [8] for morphing compatible triangulations. 5. Discussion and Conclusion We have presented an efficient algorithm for the generation of a parametrization of a 3D mesh in the presence of hard constraints. The algorithm may generate Steiner vertices in the case that a simple solution does not exist. This seems to be the first algorithm to deal with hard constraints in a sys-tematic manner, and is applicable to a number of common problems in 2D and 3D modeling, including multiresolu-tion texture mapping and image morphing. Our method generates continuous piecewise-affine pa-rametrizations of 3D meshes containing Steiner vertices. It is easy to see that many Steiner vertices might be needed in the worst case, which is undesirable, as it increases the complexity of the mapping. Relaxation indeed alleviates this, but a method to bring to a minimum the number of Steiner vertices used is still open. A number of heuristic parameters are used in our algorithm, and their effect on the results is not fully clear yet. An ex-ample is the precise location of a vertex in the kernel dur-ing the “kernel fix” procedure. Right now we position it somewhere along the line joining the kernel centroid with the original location, so that, on the one hand, it is in the kernel, and on the other hand, it does not move too much from its original location.

From a theoretical point of view, Algorithm 3 produces a legal parametrization for all resolution levels of a multi-resolution model. However, from a practical point of view, this parametrization leaves much to be desired in terms of its quality. Relaxation in its current form is impossible since it must be done on multiple resolution levels. We are experimenting with this, as well as with ways of guiding the parametrization by incorporating hard constraints in this scenario as well. A computational geometric problem which is closely re-lated to the constrained embedding problem is that of com-patible geometries: Given two n-vertex planar graph con-nectivities, position n vertices in the plane, such that both the straight-line graphs induced by the two connectivities are legal triangulations. This will allow the use of common texture coordinates for two n-vertex 3D models with differ-ent connectivity graphs, or the changing of the connectivity of a given 3D model (as in a remeshing or mesh optimiza-tion scenario) without having to modify the model’s texture coordinates. This is a subject of future research. Acknowledgements This research was partially supported by European project HPRN-CT-99-00117 (MINGLE).

Page 10: Ilya Eckstein Vitaly Surazhsky Craig Gotsman · Figure 1: Texture mapping with hard constraints. (a) Texture map with feature points marked. These should be mapped to the appropriate

Eckstein, Surazhsky and Gotsman / Texture Mapping with Hard Constraints

© The Eurographics Association and Blackwell Publishers 2001.

References [1] N. Arad and D. Reisfeld. Image warping using few

anchor points and radial functions. Computer Graphics Forum, 14(1):35-46, 1995.

[2] M. Alexa. Merging polyhedral shapes with scattered

features. The Visual Computer, 16:26-37, 2000. [3] B. Aronov, R. Seidel and D. Souvaine. On compati-

ble triangulations of simple polygons. Computational Geometry: Theory and Applications, 3:27-35, 1993.

[4] C. Bennis, J. M. Vezien and G. Iglesias. Piecewise

surface flattening for non-distorted texture mapping, ACM SIGGRAPH Conference Proceedings, pp. 237-246, 1991.

[5] P. Borrel and A. Rappoport, Simple constrained de-

formations for geometric modeling and interactive design. ACM Transactions on Graphics, 13(2):137-155, 1994.

[6] M. Eck, T. DeRose, T. Duchamp, H. Hoppe, M.

Lounsbery, and W. Stuetzle. Multiresolution analysis of arbitrary meshes. ACM SIGGRAPH Conference Proceedings, pp. 173-182, 1995.

[7] M. S. Floater. Parameterization and smooth ap-

proximation of surface triangulation. Computer Aided Geometric Design, 14:231-250, 1997.

[8] M. S. Floater and C. Gotsman. How to morph tilings

injectively. Journal of Computational and Applied Mathematics, 101:117-129, 1999.

[9] M. Garland and P. Heckbert. Surface simplification

using quadric error metrics. ACM SIGGRAPH Con-ference Proceedings, pp. 209-216, 1997.

[10] C. Gotsman and V. Surazhsky. Guaranteed intersec-

tion-free polygon morphing. Computers and Graph-ics, 25(1):67-75, 2001.

[11] K. Hormann, G. Greiner. MIPS - An efficient global

parametrization method. Curve and Surface Design Conference Proceedings, pp. 153-162, Saint-Malo, 1999.

[12] W. F. Lee, W. Sweldens, P. Schroder, L. Cowsar and

D. Dobkin. MAPS: Multiresolution adaptive parame-terization of surfaces. ACM SIGGRAPH Conference Proceedings, pp. 95-104, 1998.

[13] Levy and J. L. Mallet. Non-distorted texture mapping

for sheared triangulated meshes. ACM SIGGRAPH Conference Proceedings, pp. 343-352, 1998.

[14] J. Maillot, H. Yahia and A. Verroust. Interactive

texture mapping. ACM SIGGRAPH Conference Pro-ceedings, pp. 27-34, 1993.

[15] J. Pach and R. Wenger. Embedding planar graphs

with fixed vertex locations. Proceedings of Graph Drawing '98. Lecture Notes in Computer Science 1547, Springer-Verlag, 1998, pp. 263-274.

[16] A. Sheffer and E. de Sturler. Surface parametrization

for meshing by triangulation by flattening. Proceed-ings of the 9th International Meshing Roundtable, pp. 161-172, 2000.

[17] Sven Technologies, Inc. Surface Suite Pro. Available

at http://www.avatarmaker.com/products/surfacesuite/

[18] W. T. Tutte. How to draw a graph. Proc. London

Math. Soc., 13:743-768, 1963. [19] G. Zigelman, R. Kimmel and N. Kiryati. Texture

mapping using surface flattening via multi-dimensional scaling", Technical Report CIS-2000-01, Computer Science Department,Technion, Israel.