Deformable Objects Alive!

Stelian Coros1 Sebastian Martin1 Bernhard Thomaszewski1 Christian Schumacher2 Robert Sumner1

Markus Gross1,2

1Disney Research Zurich 2ETH Zurich

Figure 1: Our method for controlling deformable characters allows animators to bring passive elastic objects, such as this tea cup, to life.

Abstract

We present a method for controlling the motions of active de-formable characters. As an underlying principle, we require thatall motions be driven by internal deformations. We achieve this bydynamically adapting rest shapes in order to induce deformationsthat, together with environment interactions, result in purposefuland physically-plausible motions. Rest shape adaptation is a pow-erful concept and we show that by restricting shapes to suitablesubspaces, it is possible to explicitly control the motion styles ofdeformable characters. Our formulation is general and can be com-bined with arbitrary elastic models and locomotion controllers. Wedemonstrate the efficiency of our method by animating curve, shell,and solid-based characters whose motion repertoires range fromsimple hopping to complex walking behaviors.

CR Categories: I.3.7 [Computer Graphics]: Three-DimensionalGraphics and Realism—Animation; I.6.8 [Simulation and Model-ing]: Types of Simulation—Animation

Keywords: physically-based simulation, animation control

Links: DL PDF

1 Introduction

Simulating deformable objects is a well-studied problem, but con-trolling the way they move is a completely different challenge.While most characters are articulated, many are inherently de-formable and have no rigid internal structure. Examples range fromsimple jelly creatures to complex cartoon characters with stylizeddeformation behavior such as walking carpets, singing candelabra,

or diligent broomsticks. Regardless of their nature, these virtual ac-tors have to be active: they must walk, creep, roll or propel in somemanner—and these motions need to be directable.

In order to animate articulated figures such as humanoids or ani-mals, animators can resort to physics-based locomotion controllersthat translate high-level commands (walk left, crouch, or jump) intosequences of joint torques that yield the desired behavior. While theproblem of articulated motion control has been studied extensively,relatively little work has been done to create automatic methods thatsupport the animation of active deformable characters.

Passive deformable objects can be easily animated using physics-based simulation, and, in general, external control forces can beused in order to achieve desired motion objectives. These externalforces, however, typically do not sum up to zero and have non-vanishing rotational components. They can therefore change linearand angular momenta in arbitrary, non-physical ways, which maylead to unrealistic motions.

In this work, we present a new approach for intuitive control ofelastically deformable characters. As an underlying principle, weallow characters to propel using only internal energy, which is gen-erated by dynamically adapting the characters’ rest shapes. Becausethe internal control forces generated by our framework are derivedfrom an elastic potential, they automatically preserve momentum.The problem formulation we propose is aware of the forces arisingfrom interactions with the environment, and therefore rest shapeadaptations lead to desired locomotion behaviors.

Rest state adaptation is a powerful concept that provides animatorswith various levels of control over the resulting motion styles. Inparticular, we investigate two rest-pose adaptation strategies:

• Cage-based adaptation: the deformation of the rest state isdriven by a coarse mesh that embeds the rest pose. This ap-proach results in a very flexible parameterization of the restshape that promotes low-frequency deformations. In addition,this strategy requires very little user input and is quick to setup.

• Example-based adaptation: this approach allows for artisticcontrol over the resulting motions. In this setting, the spaceof rest shape deformations is restricted to an interpolation ofa set of input poses, which results in a parameter space thatis typically much smaller than when using cage-based adap-tation.

To the best of our knowledge, our method is the first to enable sim-ulated deformable characters to autonomously propel themselvesusing only internal energy. We believe that rest shape adapta-tion, paired with appropriate rest pose parameterizations and con-trol laws will allow artists to easily animate a much wider class ofcharacters than previously possible.

2 Related Work

The ways in which objects and characters move play a vital role inthe creation of believable virtual worlds for video games, animatedmovies and training applications. The problem of creating realis-tic motions has been studied extensively, resulting in a wealth ofliterature that is relevant to our work.

Physics-based techniques provide a particularly appealing approachto animating deformable objects and characters. Starting withthe pioneering work of Terzopoulos et al. [1987], many researchprojects have been aimed at increasing the diversity, efficiency, andaccuracy of physical simulations. Since space does not permit us toreview a representative set of related works, we only mention thetwo methods which found direct application in our framework: thediscrete shell model by Grinspun et al. [2003] for deformable sur-faces and the finite element model by Irving and colleagues [2007]for volumetric solids.

Simulating physical objects efficiently is only half the battle how-ever. A great deal of control over the resulting motions is also re-quired in order to provide artists with the power needed to createcompelling visual scenes. At the same time, it is very importantthat the resulting motions remain physically plausible. Despite re-ceiving a great deal of attention from the research community, thisremains to a large extent an open problem [O’Brien 2011]. We aimto address this challenge, and to help position our work with respectto previous methods, we group existing approaches as follows:

Control of passive objects: It is oftentimes desirable to simu-late seemingly passive objects whose behaviors are, in fact, user-controlled. To this end, Twigg and James [2007] present a methodthat exploits the speed of multi-body simulations. By simulatingnumerous variations of the same scene in parallel, the proposed sys-tem lets users interactively chose desirable outcomes and thereforecontrol the motions of the simulated objects. The system proposedby Popovic et al. [2000] automatically manipulates various simula-tion parameters, such as masses, moments of inertia, surface nor-mals and elastic coefficients in order to allow simple dynamically-simulated objects to achieve user-set goals. Twigg and Kacic-Alesic [2011] perform off-line optimizations of the rest lengths ofsprings in a deformable object simulation in order to maintain thelook of artist-modeled objects once gravity is applied to a scene.Martin et al. [2011] describe a framework that allows users to steer,through examples, the behavior of passive deformable objects asthey interact with their environments. In contrast to this line of re-search, we are interested in animating deformable objects as if theywere alive, autonomous characters.

Control of soft objects using external forces: A popular approachto controlling the behavior of deformable objects is through theuse of external forces. Barbic and Popovic [2008] use a time-varying Linear Quadratic Regulator controller to compute forcesthat steer simulated deformable objects or fluids towards desiredtarget configurations, and Jeon and Choi [2007] use a combinatorialdiscrete optimization approach to obtain physically-plausible, user-controlled motions for cloth. Barbic et al. [2009] present a space-time optimization method that operates in a low-dimensional sub-space defined by input keyframes. The optimization aims to elimi-nate errors in tracking the input keyframes while minimizing errorsin the dynamics. These errors can be interpreted as the influence

of applied external forces. This method was applied to FEM andmass-spring simulations of deformable objects. TRACKS [Bergouet al. 2007] enforces the low-frequency motions of thin shells usingconstrained Lagrangian mechanics, while allowing high-frequencymotions to naturally appear due to the object’s dynamics. The ad-joint method, a non-linear gradient-based optimization approach,was successfully applied to control the behavior of fluid simula-tions [McNamara et al. 2004] and particle systems [Wojtan et al.2006]. Our work shares many of the goals of these existing meth-ods. However, instead of using non-conservative external forces,the motions we generate are the result of an internal potential fieldthat is automatically modulated.

Control of articulated rigid figures: The concept of using forwarddynamics simulations in conjunction with control policies for thepurpose of character animation was introduced over two decadesago [Girard and Maciejewski 1985; Raibert and Hodgins 1991].Since then, a large number of control frameworks applicable tocharacters represented by articulated rigid body hierarchies havebeen presented. Examples include, among others, control meth-ods for legged locomotion [Yin et al. 2007; Wu and Popovic 2010;Coros et al. 2010; Lee et al. 2010; de Lasa et al. 2010] (and manyothers), athletic activities [Hodgins et al. 1995], swimming [Tu andTerzopoulos 1994; Tan et al. 2011] and flying [Wu and Popovic2003]. These control policies generate joint torques that, when ap-plied to the joints of the articulated figures, result in purposeful mo-tions. This approach has been shown to generalize well to charac-ters that are simulated using two-way coupling between articulatedrigid bodies and deformable objects [Jain and Liu 2011; Kim andPollard 2011]. The joint torques applied to the articulated figuresare used to actively drive the motions, while the deformable objectsare simulated passively. In contrast, in this paper we investigatethe problem of generating desirable internal forces for continuouslydeforming characters that have no well-defined internal or externalrigid structure. In this setting, concepts such as torques and jointsdo not exist. We therefore use a different, more general approach tomotion control that works by automatically adjusting the objects’rest configurations.

Control of soft objects using internal forces: Real creatures movearound their environments using only internal energy, which is gen-erated through muscle contractions. Simulating muscles and theireffects on the motions of virtual characters is a topic that has re-ceived much attention from the academic community. This lineof research holds the promise of creating virtual characters whosemotions are indistinguishable from the motions of real humans andanimals. While this goal is not yet within reach, significant progresshas been made. For instance, Sueda et al. [2008] introduced a con-trollable strand-based muscle model that was used to create real-istic animations of a human hand. Teran et al. [2005] used finiteelements to simulate muscles of a virtual human torso whose mo-tions were kinematically controlled. Tu et al. [1994] modeled fishas mass spring systems, and their motions were controlled by ad-justing the springs’ rest lengths. Here we extend the concept ofmuscles to arbitrary deformable objects. Our approach is relatedto the work of Ijiri et al. [2009], who introduced a system that letsusers manually specify deformation fields for the rest poses of soliddeformable objects. In contrast to this work, our method automati-cally computes appropriate rest pose configurations that allow sim-ulated rods, shells and solids to walk, crawl and hop autonomously.

3 Method Overview

In the framework we propose, rest state adaptations provide themeans of actuation and propulsion for deformable characters. Thegoal of our method is therefore to automatically compute changes tothe rest configuration of the objects in a way that results in directed

Figure 2: Overview of our method: Our solver takes as input adynamic model which describes an object’s internal energy, a setof objectives, a parameterization of the rest pose and a regularizer.The output consists of an appropriate rest pose and the correspond-ing state of the dynamic model at the next time step.

motion. In this section we describe the essence of our approach andits different components, which are also illustrated in Fig. 2.

Dynamic Models We start directly in the discrete setting and letX and x denote vertex positions of a deformable object (curve,shell or solid) in the undeformed and deformed configurations re-spectively. The deformation behavior of the object is governed byan elastic potential W (X,x), which gives rise to internal forcesfint = −∂W/∂x. Together with external forces fext (gravity,contacts, friction), the dynamic behavior is described by the well-known equations of motion

Mx− fint(X,x)− fext = 0 , (1)

where M is the mass matrix and x are nodal accelerations. For thepurpose of motion control, we dismiss the option of adding externalforces that are not the result of correct physical interactions with theenvironment. Instead, we require that all motions be a direct resultof internal deformations. To this end, we consider X as the solecontrol parameters. In other words, we adapt the rest shape of theobject to yield the required deformations for motion control. Thisapproach is similar in spirit to the way muscles work and, as formuscles, the resulting control forces are always momentum con-serving.

In order to formalize this approach, let g(x, t) denote a vector-valued objective function whose components measure the deviationfrom a given set of (to be defined) time-dependent motion goals. Weseek to find undeformed and deformed configurations that minimizethe distance to the motion objectives while satisfying the equationsof motions (1). This can be cast as a constrained optimization prob-lem of the form

minx,X

1

2|g(x, t)|2 s.t. c(X,x, x) = 0 , (2)

where we use c to encode (1) as a hard constraint. In summary, weseek to find optimal control parameters X that best achieve the mo-tion objectives g while exactly satisfying the physics constraints c.We solve this problem numerically using Lagrangian optimizationbut postpone details to Sec. 6.

The optimization problem above can be combined with any timeintegration scheme. Using the well-known implicit Euler method,Equation (1) is discretized in time as

M(xn+1

h2− xnh2− vn

h

)− fint(X,xn+1)− fext = 0 , (3)

for a given time step n. In this form, the hard constraints can beinterpreted as an implicit map X 7→ xn+1 that parameterizes the

space of possible solutions. From this space, our solver chooses thesolution that minimizes the objective function.

Rest State Parametrization In the most general setting, everyvertex of the rest shape is an independent control variable for theoptimization problem described by Equation (2). For deformableobjects with a large number of vertices, this can lead to optimiza-tion problems with an unwieldy number of parameters. To addressthis, we show that using reduced linear subspaces for rest pose de-formations is a simple, yet very powerful strategy. In particular, wechoose the rest state configuration X(p) as

X(p) = X0 + Lp, (4)

where X0 is the initial rest state, p describes generalized displace-ments, and L is a linear map between changes in rest state coordi-nates X and the reduced control variables p. With this representa-tion, we can restate our optimization problem in terms of the statevariables x and the reduced control parameters p as

minx,p

1

2|g(x, t)|2 + r(p) s.t. c(X(p),x) = 0 , (5)

where it is implied that, as per Equation (3), nodal accelerations xare an explicit function of x. The additional regularizing term r(p)will be described shortly. The linear map L determines the spaceof allowable rest shapes, and implicitly, the capabilities of the de-formable objects we seek to control. Selecting appropriate linearmaps is, therefore, of great importance for our framework. We dis-cuss two practical approaches to choosing flexible and efficient restpose parameterizations in Sections 4 and 5.

Contacts and Friction We implement contact and friction forcesusing implicit springs that are active only when vertices of the sim-ulation mesh are in contact with external objects. For each collid-ing (deformable) vertex i, we first compute the point of contact ciby projecting onto the external triangles. We then apply a contactforce that is proportional to deviations from ci in the triangle’s nor-mal direction and a friction force that is proportional to in-planedeviations from ci. The contact and friction forces are added to thefext term in equation (3), which then becomes an explicit functionof xn+1.

Adapting the rest shape affects the motion of all vertices and thusthe forces generated at the contact points. The implicit nature ofthe friction and contact forces makes them visible to our optimiza-tion method, which automatically exploits them in order to help thecharacters propel.

Controller Having established the basic means for deformableobject actuation, we now describe a simple controller that trans-lates high-level motion goals (walk left, hop, etc.) into sequencesof objectives that can be cast into algebraic functions of the statevariables via g(x, t). This vector-valued objective function is theresult of concatenating an arbitrary number of constraints that arelinear in the positions of the vertices:

g(x, t) = [w1g1(x, t), w2g2(x, t), ..., wngn(x, t)]T , (6)

gi(x, t) =∑j

wxjxj − si(t), (7)

where s(t) indicates a time varying goal target. The weight wi isused to scale the importance of goal gi relative to the other goals.Each goal constraint gi affects a subset of the object’s vertices, as

indicated by the weights wxj . As an example, for the center-of-mass constraints we use throughout the paper, the vertex weightswxj are chosen as mj/

∑imi. For the vertices not affected by a

specific goal, we simply set the weights wxj to zero.

It is often convenient to represent constraints as a function of ve-locity, rather than positions. For instance, to get an object to hop,a controller can simply output a desired upwards velocity for thecenter of mass. The goal targets are therefore exposed as:

gi(x, t) = kp(∑j

wxjxj−sxi(t))+kd(∑j

wxjvj−svi(t)) (8)

where kp, kd, sxi(t), and svi(t) are provided by the controller. Byexpressing the velocity of each vertex as a function of the positionsat the previous time step xn, v = (x − xn)/h, we represent thegoal target from Equation (7) as:

si(t) =h

hkp + kd(kpsxi + kdsvi +

kdh

∑j

wxjxnj) (9)

and the goal weight from Equation (6) as:

wi(t) =hkp + kd

h(10)

We note that solving Equation (5) is equivalent to simultaneouslysolving for the internal control forces that minimize the objectiveg and integrating the system state forward in time. As a result,the effective planning horizon of our method is equal to the simu-lation time step. Nevertheless, this simple control strategy is botheffective and intuitive. As described in more detail in Section 7, weused it to create hopping and rolling motions for deformable ob-jects of various shapes and sizes. We also used it successfully tocontrol more complicated behaviors such as bipedal walking. Forthese motions, control strategies that output high-level motion fea-tures such as targets for the positions of the center of mass and feet[Coros et al. 2010; de Lasa et al. 2010] are a natural fit for ourcontroller.

Regularizer A rest shape which is optimal with respect to Equa-tion (2) is not necessarily desirable in all regards. For example,we generally prefer smoothly deformed rest shapes over ones withlarge isolated distortions. Such preferences can be expressed for-mally in terms of a regularizing potential r(p) that is added to theobjective term as in Equation (5). The concrete form of the regu-larizer depends on the rest pose parametrization and we postponedetails to Sections 4 and 5.

4 Subspaces for Rest Shape Adaptations

The previous section has laid out the basis for controlling de-formable objects by adapting their rest shapes. As we will show,the way in which this adaptation occurs greatly impacts the result-ing motion. Conceptually the simplest case is to consider each ver-tex of the rest shapes as a control parameter in the optimizationproblem (5). For this case, the linear map L from Equation (4) is anidentity matrix of appropriate dimensions. Fig. 3 (a) demonstratesthis approach on a simple example in which an I-shaped deformablecharacter is asked to hop to the left. While this adaptation strategycan reveal unexpected but plausible means of locomotion, it alsohas disadvantages. First, it results in larger problems, which aremore difficult and expensive to solve. Second, it can potentiallylead to large, non-smooth changes of the rest shape.

Regularizer Undesirable distortions can be counteracted by in-troducing an adequate regularizer. Visually, if two solutions aresimilar in terms of Equation (5) we prefer the one that is closer tothe initial rest shape X0. This can be formalized by introducing aregularizer based on the elastic potential between the current andinitial rest poses:

r(p) = Wreg(X0,X(p)) , (11)

where we choose the regularizing potential Wreg to be of the sameform as the elastic potential W but allow for different material pa-rameters. This is convenient if, for instance, changes in volumeshould be penalized more severely than other deformations.

Coarse-Scale Adaptation It is often not necessary or even de-sirable to leverage all degrees of freedom in order to create pur-poseful motions. For example, having the center of mass of a char-acter follow a given trajectory can often be achieved using onlylow-frequency deformations.

A common approach for applying coarse-scale deformations tohigh-resolution geometry are spatial deformation methods. Har-monic coordinates [Joshi et al. 2007] are particularly attractive forour setting: the deformation field is defined by the vertices of an en-closing polygonal cage (no vertices on the inside), it is linear withrespect to the cage vertices, and it is smooth (no isolated distortionsor intersections). More concretely, let Cj ∈ R3 denote the cagevertices and let hj be the associated harmonic coordinate function.The deformed position of a given rest state vertex Xd,i ∈ R3 isthen defined as

Xd,i = X0,i +∑j

hj(X0,i)Cj = X0,i +∑j

hijCj .

The harmonic coordinates hij are directly identified with the entriesof the linear map in (4) as Lij = hij , and the control parameters pbecome the cage vertices Cj . The coordinates have to be computednumerically but since they are defined over the initial rest state X0,these computations have to be done only once.

Fig. 3 (b) shows the result of the cage-based adaptation schemeapplied to the I-character. It can be noted that the deformations ofthe rest state are much smoother than for the case of unrestrictedrest shape adaptation. This effect translates directly into smootherdeformed shapes and, thus, a notably different style of motion.

The cage-based scheme introduced here allows us to restrict restshape adaptations to lower frequency deformations. This signifi-cantly reduces the search space, thus making the optimization prob-lem easier to solve. However, this method still lacks direct controlover the type of deformations the rest shape may undergo. We ad-dress this by resorting to example-based deformation subspaces, asdescribed next.

5 Example-Based Adaptation

Being able to exert control over the rest shape adaptation is an im-portant feature since the ways in which a character moves deter-mine its personality. In order to further structure the space such asto reflect a characteristic style of motion, we turn towards example-based techniques.

The concept of example-based rest shape control can be summa-rized as follows: we let the artist choose a set of example poses thatdescribe desirable character deformations; we then convert theseposes into a representation suitable for interpolation and use con-vex combinations of the examples as the subspace for rest shapeadaptations.

Figure 3: Different rest state adaptation strategies lead to different locomotion styles: unrestricted (a), cage-based (b), example-based witha bent (c) and a compressed (d) input pose (shown as inset figures in the upper left corners).

A central question in this context is how to best interpolate betweenshapes. While the number of existing methods is abundant, mostof them follow a common scheme: the input shapes are first con-verted into an intermediate representation such as Laplacian coor-dinates, deformation gradients or Green strains. The interpolateddata is then obtained as a weighted (convex) combination of the in-put data. Finally, interpolated geometry is reconstructed by solvinga system of (nonlinear) equations. From a practical point of view,having an implicit and possibly nonlinear relation between the inputand interpolated rest shapes can make the computation of rest statederivatives that are required to solve Equation (5) quite complex.Ideally, we would like to avoid the reconstruction step altogetherand directly work on interpolated representations. Fortunately, thiscan be achieved in a simple and efficient manner as described next.

5.1 Incompatible Shape Interpolation

We start by pointing out two important facts: first, discrete elasticenergies are usually computed by summing up elemental contribu-tions. Second, these elemental contributions are typically based onrotation-invariant deformation measures such as the Green strain.As a consequence, cutting the rest state of a given deformed objectinto disjoint elements and applying a random rotation to each ofthem does not change the elastic energy or its derivatives.

Motivated by these observations, we resort to an incompatible dis-cretization and model the rest state as an assembly of disjoint el-ements. Similar approaches have been pursued before, albeit fordifferent purposes such as geometric modeling [Botsch et al. 2006]or remeshing [Wicke et al. 2010]. Compared to the compatible set-ting, it has the striking advantage that shape interpolation reducesto element interpolation which is much simpler: we first registereach element in its different poses to a common reference frame byfactoring out the rotations of the individual poses. In this commonframe, interpolating between different element poses is then simplya matter of interpolating vertex positions. Using this interpolationscheme for example-based rest shape adaptations, we can computeenergies and their rest state derivatives in a simple and efficient way.

In order to translate this concept to the formal setting of Equa-tion (4), we let Xe

0 denote the vector of concatenated vertex po-sitions Xe

0,i ∈ R3, i = {1, . . . , 4}, for a given undeformed tetra-hedral element e and encode all its example poses as correspond-ing vectors Xe

k. We then compute the deformation gradient Fek =SkS

−10 between the undeformed element and each example, where

Sk,S0 ∈ R3×3 are matrices with columns Sk,j = Xk,j − Xk,4,with j = {1, 2, 3}, holding the elements’ edge vectors. Finally, weapply a polar decomposition to Fek to obtain the rotation Re

k thataligns example k with the reference frame as Xe

k,i = (Rek)TXe

k,i.The concatenations of these elemental vectors then defines an inter-polated rest shape as

X(p) = X0 +∑k

(Xk −X0)pk . (12)

This incompatible rest state interpolation is very efficient since theunrotated positions Xe

j stay constant over time and can thus be pre-computed. Equation (12) can be cast into the form of (4) by choos-ing the entries of the linear operator as Lij = (Xj−X0)i. It shouldbe noted that, compared to unrestricted and cage-based adaptation,the dimension of X(p) is higher since vertices are no longer sharedamong the elements. Computation times are, however, largely gov-erned by the size of p, which is now determined by the number ofexample poses. Since the latter is typically much smaller than thenumber of control parameters used for unrestricted or cage-basedadaptation, the example-based approach is more efficient in this re-gard as well.

Using examples to define a subspace for rest shape adaptation pro-vides an animator with explicit control over the styles of the result-ing motions. Fig. 3 (c) and (d) illustrate this technique applied tothe I-character. The example space is defined by the initial unde-formed pose and one additional example, a pose where the I is bent(c), respectively, compressed (d). In both cases, it can be noticedthat the character makes heavy use of the example pose to gatherenough energy for the jump. The different example poses lead tosubstantially different yet plausible jumping styles that respect theartistic intent.

Regularizer Although example-based interpolation behaves ro-bustly for moderate extrapolation, the behavior in this regions istypically less intuitive and we therefore require the rest shape toremain within the convex hull of the examples. In the case ofglobal examples that extend over the entire domain of the object,this is achieved by introducing a regularizing term as r(p) =12(∑i pi − 1)2. Similarly, negative values of the control param-

eters are penalized by the regularizer.

It is sometimes more convenient to define example poses that onlyaffect localized regions of a deformable object. This is the case, forinstance, when specifying poses that control the range of possibledeformations of individual legs. We treat these cases by setting upconvexity constraints on the corresponding subsets of the controlparameters.

5.2 Extension to Curves and Shells

The interpolation described by Equation (12) applies equally forboth 2D and 3D finite elements. For the case of elastic curves andshells, we additionally have to account for bending deformation.We will only describe the approach for shells here since the formu-lation for curves is completely analogous. Following Grinspun et al.[2003], we measure the difference between a discrete undeformedtriangle mesh and its deformed counterpart in terms of differencesin edge lengths and dihedral angles. Recent work [Winkler et al.2010; Frohlich and Botsch 2011] already demonstrated that linearinterpolation in this length-angle strain space leads to appreciableand intuitive behavior.

In analogy to the 3D case, we resort to an incompatible rest stateand represent a shell surface with ne edges by a set of ne disjointhinge elements, each consisting of two edge-adjacent triangles. Wefirst register the different example poses of each hinge element toa common frame by applying affine transformations that align thecenter edges with their reference counterpart. This setting allowsus to linearly interpolate vertex positions between hinge elementsand then extract lengths and angles from the interpolated geometry.Note that for stretch deformation, doing so is exactly equivalent tointerpolating edge lengths between the disjoint hinge elements. Forbending deformation, this approach leads to a different interpola-tion speed compared to angle interpolation but is otherwise equiv-alent. As for the 3D case, the central advantage of this incompat-ible approach is that the element-wise reconstruction is analyticalwhereas the compatible setting requires a (nonlinear) reconstruc-tion step.

5.3 Validation

We have used example-based rest shape adaptations for several ofthe animations presented in Sec. 7 and observed desirable behaviorboth in terms of directability and numerical performance. In or-der to better isolate the behavior of our incompatible interpolationscheme, we additionally compare with two reference solutions: themethod by Martin et al. [2011] for solids and the one by Frohlichand Botsch [2011] for shells. Both use a strain space representationfor interpolating between examples poses: Green strain for solids,edge lengths and angles for shells. Note that, while each geomet-ric configuration has its strain space counterpart, the reverse is nottrue, which is why both methods perform a closest point projectionin a least squares sense. Similarly, our incompatible configurationsdo not generally have exact compatible analogues. For comparison,we reconstruct compatible meshes by solving a static problem withthe interpolated incompatible mesh as rest state.

For solids, we interpolate between two poses of a bar, one straightthe other twisted, using 100 uniformly-sampled weights αi ∈ [0, 1]and reconstruct the corresponding geometries. Our results matchthe reference solution very closely for all α values. Since differ-ences in geometry are hardly perceptible, we only show interpola-tion results obtained with our method in Fig. 4.

Figure 4: Interpolation between two poses of a solid bar (orange)using our incompatible scheme (blue).

To test the incompatible interpolation scheme for shells, we appliedour method to one of the examples used by Frohlich and Botsch[2011], consisting of a straight and a helically-twisted tube. Fig. 5shows the results of our method compared to the reference solutionfor three sample weights. We obtained good overall shape approx-imation without any stretching artifacts, which can be attributed tothe constant-rate interpolation of the edge lengths. Note that, as ex-pected, the geometries do not match exactly since our interpolationspeed for angles is different from the reference solution.

Figure 5: Interpolation between two poses of a shell (orange) usingour method (blue) and a reference implementation (yellow).

6 Lagrangian Optimization

The motion of our controlled deformable characters is governedby the optimization problem (5), which is nonlinear in both ob-jective function and constraints. In order to solve Equation (5)in every time step, we employ Sequential Quadratic Programming(SQP) [Nocedal and Wright 2000] and solve the arising linear sys-tems with a custom-tailored block solver.

SQP Formulation Following the SQP algorithm, we start bydefining a Lagrangian function for the optimization problem as

L(x,p, λ) =1

2|g(x, t)|2 + r(p) + λT c(X(p),x) .

Using a Newton-Raphson method, we iteratively solve the first or-der optimality conditions

0 =∂L

∂x= g(x, t)

∂g

∂x+ λT (

1

h2M− ∂ftot

∂x(X(p),x))

0 =∂L

∂p=∂r∂p

(p)− λT ∂fint∂X

(X(p),x)∂X

∂p

0 =∂L

∂λ= Mx− ftot(X(p),x)

where we used ftot(X(p),x) = fint(X(p),x)+fext(x) to accountfor the fact that fext includes position-dependent contact forces. Ineach iteration, we compute corrections ∆x, ∆p and ∆λ by solvinga linear system of the form A BT DT

B C ET

D E 0

∆x∆p∆λ

=

abc

, (13)

whose matrix blocks A = ∂xxL, B = ∂pxL, C = ∂ppL, D =∂λxL and E = ∂λpL correspond to the partial Hessians of theLagrangian and the vectors a = ∂L/∂x, b = ∂L/∂p and c =∂L/∂λ are the partial gradients of L. Having solved for the searchdirection, we determine an admissible step size using a linesearchmethod as xn+1

pn+1

λn+1

=

xnpnλn

− α ∆x

∆p∆λ

. (14)

Starting with a full step of α = 1.0, we successively half the stepsize if the trial solution increases the value of the merit function

ψ(x,p) =1

2|g(x, t)|2 + r(p) + γ |c(X,x)|L1 .

The parameter γ allows us to adaptively control the importance ofconstraint satisfaction. We start with a small value of γ to allowfor larger steps in the beginning and then progressively increase itsvalue to obtain accurate constraint satisfaction upon convergence.

Block Solver The linear system (13) is symmetric, indefinite, andhas a special block structure: B, C and E are dense, whereas Ais sparse. While there are efficient direct solvers for symmetricindefinite problems, the dense blocks significantly deteriorate theperformance of factorization methods.

Fortunately, we can solve (13) efficiently with a factorizationmethod by using Gaussian block elimination. This step takes ad-vantage of the fact that D = 1/h2M − ∂(ftot)/∂x is invertible.From the last row of (13) we deduce ∆x = D−1(c − E∆p) and,rearranging terms, obtain

(BT −AF)∆p + DT∆λ = a−Ad

(C−BF)∆p + ET∆λ = b−Bd ,

where F = D−1E and d = D−1c. We solve the first equationsymbolically for ∆λ and substitute the resulting expression intothe second equation to obtain a linear system for ∆p as

K∆p = b−Bd− FTa + FTAd , (15)

where K = (C−BF−FTBT + FTAF). Having precomputedF and d, we numerically solve for ∆p and use the result to succes-sively compute the remaining corrections

DT∆λ = a−Ad + (AF−BT )∆p (16)x = d− F∆p . (17)

Note that this block-based formulation can be implemented very ef-ficiently using a sparse factorization solver: prefactoring the (sym-metric) matrix D once, we can compute F and d using backsubsti-tution and solve (16) in the same manner. The linear system (15) isdense, but since it is only of size dim(p) the computation costs areinsignificant compared to the sparse solve for D.

7 Results

We demonstrate the versatility of our approach by creating crawl-ing, rolling, hopping and walking motions for various deformableobjects. Our results, which are best seen in the accompanyingvideo, are summarized in Table 1, and snapshots of some of theobjects we controlled are shown in Fig. 6. We apply our method to2D curves and solids, as well as thin shells and volumetric solids in3D. The elastic potentials that we used for the various deformablemodels are described in Appendix A.

Control The simplest approach to controlling the behavior of de-formable objects is to prescribe desired values for the center ofmass position or velocity. Note, however, that our optimizationframework cannot guarantee that these objectives are always well-satisfied. For instance, if an object is in mid-air, changing the restpose configuration has no effect on the position of the center ofmass or its rate of change. Indeed, our characters can only propelby purposefully interacting with their environment through frictionand contact forces—a strategy that is automatically discovered. TheTea Cups, 3D Cross, Ace and letters S, G, I and P were controlledusing this strategy.

Center-of-mass control is well suited to rolling and hoping motions.For more difficult control problems, however, this strategy starts tobecome ineffective. For instance, ensuring that a character lands onits feet after a hop requires specific initial conditions for the take-off, which may not be sufficiently well represented by the objectivesplaced on the center of mass. This is also the case for the walkingbehaviors we implemented. For these, we add additional soft con-straints that act on vertices on the feet of the characters in order to

control foot placements. Similarly, constraints on vertices on thebody are used to control the character’s orientation. The targets forthe positions of the feet and the center of mass are computed usinga strategy similar to the one described by Coros et al. [2010]. Theuse of foot-placement feedback is particularly important to ensurethat characters do not lose their balance. We applied the walkingcontrollers to the letters A, R and H, the Castle, Pirate and Cheesycharacters.

Rest Pose Parameterization The cage rest pose parameteriza-tion that we describe provides a great deal of flexibility (dependingon the resolution of the cage, of course), making it useful for allthe types of controllers we implemented. However, it does not al-low users to explicitly affect the deformations that the charactersundergo. In the spirit of [Martin et al. 2011], we address this lim-itation by allowing the space of rest-pose deformations to be ex-plicitly specified through examples. A static solver or mesh editingsoftware was used to create the inputs.

For the 3D Cross character shown in Fig. 6 we used one examplefor each leg, which allowed for stretching motions. For Pirate, weprovided groups of example poses that locally deformed the feet,hips and the length of each leg. These poses were designed to allowthe character to control its center of mass and foot placements. Wealso used examples to animate Ace, a 3D shell character.

The Crusty characters (Fig. 6) were created to illustrate the dif-ference between cage-based and example-based rest shape adapta-tion. The dark Crusty character was controlled using the cage-basedstrategy. The side-to-side motion emerges naturally. The secondCrusty character is controlled using the example-based strategy.Through the example rest poses we provided, the side-to-side mo-tion was removed, and instead, a desirable slight twist about thevertical axis was introduced. The kinematic objectives were identi-cal for the two tests.

Timing Information The results we described here run in real-time or at interactive rates. The average time spent computing onesecond of simulated motion for each example is given in Table 1.All simulations were performed with a step size between 0.01s and0.033s, on a single core of an Intel Core i7 960, 3.2 GHz. Forvisualization purposes, we used high-resolution meshes embeddedin relatively low-resolution simulation meshes.

Model #x #p RPP DM time (s)Sphere 246 24 cage 3D solid 2.6Ace 1 231 3 examples 3D shell 9.2Ace 2 231 42 cage 3D shell 14.1Castle 1 142 22 cage 2D solid 1.1Castle 2 64 22 cage 2D rod 0.57Tea Cups 228-375 24 cage 3D solid 5.64-16.5S and G 100-130 8-36 cage 2D solid 0.8-2.1I and P 54-64 3 ex 2D solid 0.8-0.9A, R, H 66-104 20-28 cage 2D solid 0.8-1.93D Cross 1128 6 ex 3D solid 21.7Pirate 188 12 ex 2D solid 3.16Cheesy 243 150 cage 3D solid 8.6Crusty 1 114 12 ex 3D solid 7.8Crusty 2 114 57 cage 3D solid 9.8

Table 1: Summary of results with columns indicating the numberof DOFs, number of control parameters, adaptation scheme, elasticmodel and average computation time for one second of animation.

Figure 6: Deformable objects in action. In row order: Lettersmoving towards their designated locations. 3D Cross uses his ex-tendable legs for fast locomotion. Two Crusty characters race eachother. Cheesy runs away from a bunch of rampaging grapes. Piratewalks around in a 2D world. Ace’s trick.

8 Limitations and Future Work

The control framework that we propose automatically changes therest configuration of deformable objects to allow them to propelthemselves through purposeful physical interactions with the en-vironment. We demonstrate the effectiveness of our approach byanimating a variety of characters that can crawl, roll, hop and walk.

Our method is not without limitations. The evolution of the state ofthe dynamical system is currently coupled with the control method.As such, our controller always uses a planning horizon that is equalin length with the time step of the simulation. Characters that areexceedingly soft cannot be well-controlled by our framework, asthe effect of changing the rest pose may be too weak over one timestep. Adapting a space-time optimization method to work with ourproblem formulation is one way in which we can increase the plan-ning horizon.

Adapting the rest pose is one way of modulating the internal energyof deformable objects. An alternative is to change the material stiff-ness parameters. This is a very interesting extension to our work.Currently, for instance, if a character is too soft for its weight, thenit would be unable to stand or walk. Automatically adjusting stiff-nesses in a non-homogeneous way could address this limitation.

The example-based rest shape adaptation strategy we propose al-lows users to control the range of internal deformations that the ob-jects are allowed to undergo. This is a key ingredient in controllingthe motions of a character, but the motion objectives output by thecontrollers play an equally important role. The timing of a walk cy-cle, the motions of the arms or the facial expressions, for instance,can add to the personality conveyed by a character. In future workwe plan to further investigate ways in which we can increase thelevel of artistic control over the resulting motions.

9 Acknowledgements

We would like to thank Alec Jacobson, Fabian Hahn, Joe Schmid,Olga Sorkine and the anonymous reviewers for their useful com-ments and suggestions. Many thanks to Maurizio Nitti and AlessiaMarra for creating the Cheesy and Pirate characters.

References

BARBIC, J., AND POPOVIC, J. 2008. Real-time control of phys-ically based simulations using gentle forces. In Proc. of ACMSIGGRAPH Asia ’08.

BARBIC, J., DA SILVA, M., AND POPOVIC, J. 2009. Deformableobject animation using reduced optimal control. In Proc. of ACMSIGGRAPH ’09.

BERGOU, M., MATHUR, S., WARDETZKY, M., AND GRINSPUN,E. 2007. TRACKS: Toward Directable Thin Shells. In Proc. ofACM SIGGRAPH ’07.

BOTSCH, M., PAULY, M., GROSS, M., AND KOBBELT, L. 2006.PriMo: Coupled prisms for intuitive surface modeling. In Proc.of Symp. on Geometry Processing (SGP ’06).

COROS, S., BEAUDOIN, P., AND VAN DE PANNE, M. 2010. Gen-eralized biped walking control. In Proc. of ACM SIGGRAPH’10.

DE LASA, M., MORDATCH, I., AND HERTZMANN, A. 2010.Feature-based locomotion controllers. In Proc. of ACM SIG-GRAPH ’10.

FROHLICH, S., AND BOTSCH, M. 2011. Example-driven defor-mations based on discrete shells. Comput. Graph. Forum 30, 8,2246–2257.

GIRARD, M., AND MACIEJEWSKI, A. 1985. Computational mod-eling for the computer animation of legged figures. In Proc. ofACM SIGGRAPH ’85.

GRINSPUN, E., HIRANI, A. N., DESBRUN, M., AND SCHRODER,P. 2003. Discrete shells. In Proc. of ACM SIG-GRAPH/Eurographics Symp. on Computer Animation (SCA’03).

HODGINS, J., WOOTEN, W., BROGAN, D., AND O’BRIEN, J.1995. Animating human athletics. In Proc. of ACM SIGGRAPH’95.

IJIRI, T., TAKAYAMA, K., YOKOTA, H., AND IGARASHI, T.2009. Procdef: Local-to-global deformation for skeleton-freecharacter animation. In Proceedings of Pacific Graphics ’09.

IRVING, G., SCHROEDER, C., AND FEDKIW, R. 2007. Volumeconserving finite element simulations of deformable models. InProc. of ACM SIGGRAPH ’07.

JAIN, S., AND LIU, C. K. 2011. Controlling physics-based char-acters using soft contacts. In Proc. of ACM SIGGRAPH Asia’11.

JEON, H., AND CHOI, M.-H. 2007. Interactive motion controlof deformable objects using localized optimal control. In IEEEInternational Conference on Robotics and Automation, 2582 –2587.

JOSHI, P., MEYER, M., DEROSE, T., GREEN, B., ANDSANOCKI, T. 2007. Harmonic coordinates for character articu-lation. In Proc. of ACM SIGGRAPH ’07.

KIM, J., AND POLLARD, N. S. 2011. Fast simulation of skeleton-driven deformable body characters. ACM Trans. Graph. 30.

LEE, Y., KIM, S., AND LEE, J. 2010. Data-driven biped control.In Proc. of ACM SIGGRAPH ’10.

MARTIN, S., THOMASZEWSKI, B., GRINSPUN, E., AND GROSS,M. 2011. Example-based elastic materials. In Proc. of ACMSIGGRAPH ’11.

MCNAMARA, A., TREUILLE, A., POPOVIC, Z., AND STAM, J.2004. Fluid control using the adjoint method. In Proc. of ACMSIGGRAPH ’04.

NOCEDAL, J., AND WRIGHT, S. J. 2000. Numerical Optimization.Springer.

O’BRIEN, J., 2011. Thoughts on physically based animation.Keynote talk, Symposium on Computer Animation (SCA).

POPOVIC, J., SEITZ, S. M., ERDMANN, M., POPOVIC, Z., ANDWITKIN, A. 2000. Interactive manipulation of rigid body simu-lations. In Proc. of ACM SIGGRAPH ’00.

RAIBERT, M. H., AND HODGINS, J. K. 1991. Animation of dy-namic legged locomotion. In Proc. of ACM SIGGRAPH ’91.

SUEDA, S., KAUFMAN, A., AND PAI, D. K. 2008. Musculoten-don simulation for hand animation. In Proc. of ACM SIGGRAPH’08.

TAN, J., GU, Y., TURK, G., AND LIU, C. K. 2011. Articulatedswimming creatures. In Proc. of ACM SIGGRAPH ’11.

TERAN, J., SIFAKIS, E., BLEMKER, S. S., NG-THOW-HING, V.,LAU, C., AND FEDKIW, R. 2005. Creating and simulatingskeletal muscle from the visible human data set. IEEE Transac-tions on Visualization and Computer Graphics 11, 317–328.

TERZOPOULOS, D., PLATT, J., BARR, A., AND FLEISCHER, K.1987. Elastically deformable models. In Proc. of ACM SIG-GRAPH ’87.

TU, X., AND TERZOPOULOS, D. 1994. Artificial fishes: physics,locomotion, perception, behavior. In Proc. of ACM SIGGRAPH’94.

TWIGG, C. D., AND JAMES, D. L. 2007. Many-worlds browsingfor control of multibody dynamics. In Proc. of ACM SIGGRAPH’07.

TWIGG, C. D., AND KACIC-ALESIC, Z. 2011. Optimization forsag-free simulations. In Proc. of ACM SIGGRAPH/EurographicsSymp. on Computer Animation (SCA ’11).

WICKE, M., RITCHIE, D., KLINGNER, B. M., BURKE, S.,SHEWCHUK, J. R., AND O’BRIEN, J. F. 2010. Dynamic local

remeshing for elastoplastic simulation. In Proc. of ACM SIG-GRAPH ’10.

WINKLER, T., DRIESEBERG, J., ALEXA, M., AND HORMANN,K. 2010. Multi-scale geometry interpolation. In Proc. of Euro-graphics ’10.

WOJTAN, C., MUCHA, P. J., AND TURK, G. 2006. Keyframecontrol of complex particle systems using the adjoint method.In Proc. of ACM SIGGRAPH/Eurographics Symp. on ComputerAnimation (SCA ’06).

WU, J.-C., AND POPOVIC, Z. 2003. Realistic modeling of birdflight animations. In Proc. of ACM SIGGRAPH ’03.

WU, J., AND POPOVIC, Z. 2010. Terrain-adaptive bipedal loco-motion control. In Proc. of ACM SIGGRAPH ’10.

YIN, K., LOKEN, K., AND VAN DE PANNE, M. 2007. SIMBI-CON: Simple biped locomotion control. In Proc. of ACM SIG-GRAPH ’07.

A Elastic Models

The elastic potentials used in this work are standard except for a fewmodifications made to obtain more efficient higher-order deriva-tives. We use a computer algebra software to automatically gen-erated code for all required derivatives and therefore only list theenergy expressions here. In order to compute the energy of a givenelement, we first compute its energy density using its deformed andundeformed positions x and X and then integrate over its initial ge-ometry defined by X. While x and X change during simulation, Xis assumed constant.

For curves in 2D-space, we define the elastic potential as

WCurve2D =∑i

1

2kl(l

2i − L2

i )2 1

L3i

+∑a

1

2kb(θa −Θa)2Ta ,

where li/Li is the deformed/undeformed length of edge i, θa/Θa isthe deformed/undeformed angle enclosed by two adjacent edges iand j, and Ta = 1

2(Li+Lj). Note that the division by L3

i accountsfor the proper scaling of the energy with respect to mesh resolution.We use an analogous formulation for thin shells, replacing θ andΘ with the dihedral angle between two edge-adjacent triangles (see[Grinspun et al. 2003]).

The potential for 2D solids is modeled using Finite Elements anda modified St.Venant-Kirchhoff material similar to [Martin et al.2011],

WSolid2D =∑e

1

2kl||Ee||2Ae +

1

2ka(ae −Ae)2

1

Ae,

with the Green strain E = 12(FtF − I) and a/A denoting the area

of the deformed/undeformed triangle. The deformation gradient Fis computed as described in Sec. 5.

We also use a straightforward extension of this formulation for 3Dvolumetric solids but, though tractable, computations become quiteintense. For this reason, we constructed a simplified energy thatquantifies the distortion of a tetrahedron by measuring the changein edge lengths as

WSolid3D =∑e

6∑j=1

1

12kl(l

2ej −L2

ej)2 Ve

L4ej

+1

2kv(ve− Ve)2

1

Ve.