Curve Analogies

Aaron Hertzmann Nuria OliverBrain Curless Steven M. SeitzUniversity of Washington Microsoft Research

Thirteenth Eurographics Workshop on Rendering(2002)

Goal

Automatically learn how to generate an output curve form an input curve

Contributions

Learning curve styles may be formulated as a texture synthesis problem

Curve analogies may be learned from data, by learning specific statistics within curves and relationships between them

These algorithm can be used to capture hand-drawn styles of curves

Introduce

The most general problem statement A : A’ :: B : B’ (unfiltered vs. filtered)

A 2D curve can be written in parametric form

Control point

Parameter

Linear interpolation between the adjacent control point

Linear Interpolation

tup tlo

t

pup

plo

f(t)

lowerupper

lowerupperupperlower

tt

tttttf

p)(p)(

)(

if (t, p) is a control point

otherwise

p)( tf

Algorithms

Curve synthesis Curve synthesis with constraints Curve analogies Multiresolution curve synthesis Multiresolution curve analogies

Problem statement:given an example curve A’, generate a

new curve B’ of a specified arc length LB’

Curve Synthesis

Cost Function

Use A’ to define a cost function over possible curves, and then attempt to generate a new curve B’ with minimal cost

measures the “difference” between the local shape of B’ around ti andthe local shape of A’ around si

i

jij

sAtBdBE ),',,'(min)'(neighborhood distance metric

Neighborhood Distance Metric

each neighborhood as a set of K samples

tangent features (smoothness properties of the curve)

k

kkkkR

ji

RwkRw

sAtBd

)||'b'a'||||'b't'a'(||min

),',,'(22

't,'

)(''a kk sA )(''b kk sB}...1{ Kk

||'a'a||

'a'a'a

1

1

kk

kkk

||'b'b||

'b'b'b

1

1

kk

kkk

Curve Coherence

Copy coherent segments from A’ to B’Let S(i) be the source index for each cont

rol point in B’

),',,'(minarg)( jij sAtBdiS

Curve Coherence

A control point in B’ is coherent with the preceding control point if and the curve segment

Penalize non-coherent control points by multiplying the distance d(.) by

),( ii pt),( 11 ii pt

)()1( iSiS

)1(')('))1(('))((' ii tBtBiSAiSA

)/1( D

Initialize

Initializes B’(t) to be an empty curve Randomly-chosen segment of three con

secutive control points from A’ to B’ The remainder of the B’ curve is generat

ed by nearest neighbor sampling

Nearest Neighbor Sampling

Generate a new sample Goal:

search for the value p* for that best matches the resulting neighborhood of some neighborhood j* in A’

),',,'(minarg*)*,p()),('(

jnewjtB

sAtBdjnew

ttt maxnew

)( newtB

Cost Function

Optimal rigid transformationaligns the existing neighborhood samples

to Cost function can be written

jjR 't,'

}a,a{ kk }b,b{ kk

}a,a{ kk

}b,b{ kk

CsARwBE jjjjnew 2

)'t)(''(p)'(

Coherence

The optimal choice for is given by transforming the position of to the neighborhood in B’: this gives

Finally, we evaluate the cost for this value , and apply the coherence penaltycoherent of a candidate

jp)(' jsA

jjjj tsAR ')(''p

jpjd

jsiS )1( )1(')('2

3))1(('))( ii tBtBiSAiA'(S,

Pseudocode

Algorithms

Curve synthesis Curve synthesis with constraints Curve analogies Multiresolution curve synthesis Multiresolution curve analogies

Curve Synthesis with Constraints

Problem statement:soft constraint:

• pass near specific position•

hard constraint:• pass through specific position•

represent:

2q)'(' ccc tBw

ccc wtB ,q)'('

ccc wt ,q,

Algorithm

A curve synthesis with constraints is quite similar to that withoutAssume that the first point on B’ is

constrained; we use the constraint to initialize the curve

The more general case can be handled by synthesizing forward from the first constraint, and then synthesizing backwards form the last constraint

Algorithm

Unconstrained control points are synthesized exactly as in the previous algorithm

Control points specified by hard constraints are immediately replaced with the position of the hard constraint

For soft constraints, we must modify the cost function

Cost Function of Soft Constraint

Soft constraint when

The candidates are now computed by optimizing the above quadratic equation

the position of is no longer a direct copy of the example position

ccc wt ,q, newc tt

CwsARwBE cjcjjjjnew 22

qp)'t)(''(p)'(

jp

cnew

ccjjjnewj ww

wsARw

q)'t)(''(

p

jp

Algorithms

Curve synthesis Curve synthesis with constraints Curve analogies Multiresolution curve synthesis Multiresolution curve analogies

Curve Analogies

Problem statement:give an example “unfiltered” curve A and

example “filtered” curve A’, we would like to “learn” the transformation from A to A’, and apply it to a new curve B to generate and output curve B’

':::': BBAA

Representation

Represent the correspondence between A and A’, and between B and B’ require the A, A’ and B, B’ curves to have

the same parameterizatione.g.

Note:• no constraint on the relative arc length

)(')( jj sAsA )(')( ii tBtB

Shape Relationships

GoalsThe shape of B’ should “look like” the

shape of A’The shape of B’ should have the same

relationship to the shape of B as the shape of A’ has to A

The relative positions and orientations of the B’ curve should have the same relationship to B as the position and orientations of A’ to A

Positional Relationship

By measuring the offsets between curvesuse neighborhood center-of-mass as a

sort of “summary” of the local neighborhood position

we want the offset from A to A’ to match the offset from B to B’

k

kk

w

w aa

k

kk

w

w a'a'

k

kk

w

w bb

k

kk

w

w b'b'

2)'()'('),',,,',( bbaaRsAAtBBd jiOFFSET

Cost Function

We have found if works best to use separate translations t and t’ for matching A to B and A’ to B’, respectively, but to use the same rotation from A to B as from A’ to B’

),,,(),',,'(min)'( jiBjii

jsAtBdwsAtBdBE

),',,,',( jiOFFSETOFFSET sAAtBBdw

Algorithm

The A/B curve matching term affects the curve alignment and the computation of example: sharp concave curve the translations t and t’ are computed for

each pair of curves separately, and the a single R’ is computed that aligns the pairs of curves

jd),,,( jiB sAtBdw

Algorithm

The curve offset term effects both matching penalty and the computation of the optimal

We can write the offset term asjp

jd

2

2 )'

)a'a('b(1

p),',,,',(

k kj

k kkjjiOFFSET ww

bwR

mmsAAtBBd

k kj

j

ww

wm

k kj

k kkj ww

bwm

'p'b

2)'()'('),',,,',( bbaaRsAAtBBd jiOFFSET

Pseudocode

Algorithms

Curve synthesis Curve synthesis with constraints Curve analogies Multiresolution curve synthesis Multiresolution curve analogies

Problems of Single Scale Algorithm

We would like to be able to use large neighborhood sized computationally expensive

Step size is not directly related to arc length

Multiresolution Curve Synthesis

Problem statement:give an example A’ curve, and

synthesize an output curve B’ this is done by synthesizing a Gaussian

pyramid of curves from coarse to fine the final output B’ is the finest level of

the output pyramid

Multiresolution Curve Synthesis

the coarsest level of the output pyramid is a single-scale curve synthesis problem based on the coarsest level of the input pyramid

for remaining levels of the pyramid, the relationships can be expressed as:

llll BBAA ':'::':' 11

Constraints

Any constraints placed on the output curve must be applied to the higher levels as well

The texture would have to bend to meet non-colinear constraints

We address this problem by softening the constraints: the weight of each constraint is replaced withccc wt ,q,

111 ))(( lLwc

cw

Pseudocode

Algorithms

Curve synthesis Curve synthesis with constraints Curve analogies Multiresolution curve synthesis Multiresolution curve analogies

Multiresolution curve analogies

Synthesis at the finer levels of the pyramid can be viewed as a “generalized analogy”

to ensure that has consistent relationships with

llllll BBBAAA ':',::':', 11

lB'

1', ll BB

llll BBAA ':'::':' 11

Pseudocode

Application and Experiments

Discussion and Future Work

Learning parameters Alternative representations Drawing systems Additional features Animation 3D signal processing