Direct manipulation of recurrent models

Computers & Graphics 27 (2003) 143–151

Chaos and graphics

Direct manipulation of recurrent models

Paul Shermana, John C. Hartb,*aSeattle, WA, USA

bDepartment of Computer Science, University of Illinois, 1304 W. Springfield Avenue, Urbana, IL 61801, USA

Accepted 18 October 2002

Abstract

This paper describes an interactive modeling method which allows a user to directly manipulate a fractal object

defined by a recurrent iterated function system (RIFS). The method allows the user to nail points on the attractor to

their current position, and drag other points to a desired position. The attractor changes shape to accomodate the new

location of the dragged point. An incremental Newton’s method solves the system of equations generated to find a new

set of RIFS parameters that satisfy the constraints. These solutions can be generated in real time for simple models on

modern computer hardware, resulting in a fully interactive recurrent modeling system for designing models that closely

match specific natural shapes.

r 2002 Elsevier Science Ltd. All rights reserved.

Keywords: Recurrent iterated function systems; Fractals; Interactive modeling

1. Introduction

The infinite detail and self-similarity of fractal models

provides a compact method for approximating the detail

of the natural world. Most commercial modeling and

animation packages provide some sort of fractal

generation techniques in order to provide artists with

methods to mathematically create the rough, random

and detailed surfaces that are common in nature.

A linear fractal is a shape made from sections that are

affine copies of larger sections. Recall that an affine

transformation is a linear transformation plus a

translation. Linear fractals are completely deterministic.

While some rendering techniques (e.g. the ‘‘chaos

game’’) use random number sequences, the resulting

shape is independent of the random number sequence.

This determinism means that we can use linear fractal

surfaces to model specific instances of real-world shapes

[1]. The shape and character of random fractal models

[2], such as those based on the Perlin noise function [3],

depend on the sequence of numbers generated by a

pseudorandom number generator. Hence random frac-

tals are not well suited to model specific natural shapes

whereas linear fractals provide a more precise fractal

representation of specific detail.

Linear fractal specification techniques include the

initiator/generator method [4], iterated function systems

[5], recurrent iterated function systems [6], L-systems [7]

and the object instancing paradigm [8].

The problem of rendering a linear fractal from its

specification is well researched [5,9,10]. The difficult

problem is determining a linear fractal specification

of a particular shape. This is the so-called inverse

problem of linear fractals. The collage theorem [11]

established an approximation error upper bound based

on an object’s self-tiling. Fractal image compression [12]

encoded small chunks of image data. Similarity hashing

[13] applied a computer vision technique called geo-

metric hashing to detect self-similarity in a shape’s

feature points.

Geometric modeling rarely depends on automatic

techniques. More commonly, shape design is employed,

where the user is encouraged to manipulate a shape into

a desired form. Interactive techniques are popular for

shape design as they provide quick user feedback.*Corresponding author.

E-mail address: [email protected] (J.C. Hart).

0097-8493/03/$ - see front matter r 2002 Elsevier Science Ltd. All rights reserved.

PII: S 0 0 9 7 - 8 4 9 3 ( 0 2 ) 0 0 2 5 2 - 2

Interactive modeling of linear fractals is quite difficult.

A common method for modeling linear fractals approx-

imates the goal object with a simpler polygon, and

allows the user to interactively tile the polygon with

smaller copies [5]. Typically, these methods depend on

the intuition of the user to define an approximate self-

tiling of an object. Moreover, the parameters of self-

similarity do not have very geometrically intuitive

interpretations, and their changes affect the global shape

of the object at all levels of detail.

In this paper, we present a method for modeling

fractal objects which is both interactive and direct—the

user can set constraints by nailing a number of points to

their current positions. Then the user can click and drag

any desired point on the linear fractal shape. The

recurrent model then changes shape to accomodate the

new position of the dragged point, as shown in Fig. 1. It

accomplishes this by finding the parameters of a new

RIFS that meets the demands of the constraints.

Section 2 begins with a review of the theory of

recurrent modeling and the RIFS representation. Sec-

tion 3 sets up the constraint system and shows how to

solve it using Newton’s method. Section 4 demonstrates

the system with examples and results. Section 5

concludes with a summary and directions for future

work.

This method is presented in terms of recurrent iterated

function systems, but there is no intrinsic reason why it

could not be extended to other fractal paradigms.

Section 5 also describes how the method could also be

applied to the parameters of an L-system, or the

transformations of a scene graph.

2. Recurrent iterated function systems

An iterated function system (IFS) is a set of maps

fwigNi¼1 from a metric space ðX ; dÞ into itself. In the

interest of simplicity, we will assume the metric space

consisting of Rn and the Euclidean distance metric, and

denote it as Rn: Given a set SCRn; the Hutchinson

operator w of the IFS is defined as

wðSÞ ¼[N

i¼1

wiðSÞ: ð1Þ

When all of the maps of an IFS are contractive, then

there exists a unique, non-empty, compact set ACRn

called its attractor. The attractor is the set that results

from repeated application of the Hutchinson operator.

Given any non-empty bounded set BCRn;

A ¼ limi-N

w1iðBÞ; ð2Þ

where w1i ¼ w3w1ði�1Þ and w11 ¼ w: The attractor of theIFS is also defined as the unique non-empty solution to

the recurrence

A ¼ wðAÞ: ð3Þ

Hence, the IFS describes a specific shape by the

transformations that take the shape to each of its

components.

The recurrent iterated function system (RIFS) consists

of an IFS whose maps are controlled by a directed graph

G: Each edge /i; jSAG indicates that the composition

wj3wi is allowed. As with IFSs, if an RIFS is comprised

of contractive maps, then there exists a unique, non-

empty compact attractor ACRn; which is defined below.To keep the notation consistent with that for IFSs, we

will define the recurrent Hutchinson operator, w on

set-vectors AAðRnÞN as

wðAÞ ¼ ðw1ðAÞ;w2ðAÞ;y;wN ðAÞÞ; ð4Þ

where

wkðAÞ ¼[

/j;kSAG

wkðAjÞ: ð5Þ

The reason for defining the recurrent Hutchinson

operator on a set vector is because each

Aj ¼[

/i;jSAG

wjðAiÞ ð6Þ

is a union of (possibly) overlapping partitions (including

itself if /j; jSAG). Since we define the attractor

Fig. 1. Direct manipulation of various fractal objects. Nailed and dragged points are denoted with blue circles and lines, respectively.

P. Sherman, J.C. Hart / Computers & Graphics 27 (2003) 143–151144

A ¼[N

j¼1

Aj ; ð7Þ

it is possible to keep the individual components of the

partitions separate by defining the attractor defined in

(7) as a vector of sets A ¼ ðA1;A2;y;AN Þ: Then, we candefine the RIFS attractor A ¼

Si Ai consistently with

the definition in (3) as

A ¼ wðAÞ ð8Þ

and consistent with the definition in (2) if, for a given set

vector B ¼ ðB1;B2;y;BN Þ where each BiCRn is a

bounded non-empty set,

A ¼ limi-N

w1iðBÞ: ð9Þ

2.1. The parameter space of 2-D linear fractals

For an RIFS defined on R2; each map wi is defined by

the affine transformation

wiðx; yÞ ¼ ðaix þ biy þ ei; cix þ diy þ fiÞ: ð10Þ

Thus, each map of the RIFS can be denoted by a 6-

vector of parameters vi ¼ ðai ; bi; ci; di; ei; fiÞ and the

entire RIFS can be represented by the graph G and a

6N-vector V ¼ ðv1; v2;y; vN Þ:For example, Sierpinski’s gasket (Fig. 2) is repre-

sented by a complete digraph and the 6-vectors

v1 ¼ ð0:5; 0:0; 0:0; 0:5; 0:0; 0:5Þ;

v2 ¼ ð0:5; 0:0; 0:0; 0:5;�0:433;�0:25Þ;

v3 ¼ ð0:5; 0:0; 0:0; 0:5; 0:433;�0:25Þ; ð11Þ

which forms a point in the 18-dimensional parameter

space of all 3-map IFSs. Note that this point is trivially

not unique, since any parameter vector V 0 which results

from a permutation of the v1; v2 and v3 vectors results in

the same IFS.

Another RIFS is the fractal pound sign (Fig. 2) which

is defined by a four-vertex graph that is complete except

for edges of the form oi; i >; and four affine transfor-

mations which are represented by the 6-vectors

v1 ¼ ð0:5; 0:0; 0:0; 0:5;�0:5;�0:5Þ;

v2 ¼ ð0:5; 0:0; 0:0; 0:5;�0:5; 0:5Þ;

v3 ¼ ð0:5; 0:0; 0:0; 0:5; 0:5; 0:5Þ;

v4 ¼ ð0:5; 0:0; 0:0; 0:5; 0:5;�0:5Þ: ð12Þ

This RIFS is represented by a point in a 24-

dimensional parameter space of all attractors whose

components are made from smaller copies of the other

three components.

2.2. Addresses

An address s of a point xAA is the infinite sequence of

RIFS map indexes ði1; i2;yÞ such that

x ¼ wi13wi23?ðyÞ ð13Þ

for any yAR2 (due to a variation on the contraction

mapping theorem).

Each point in an attractor has one or more unique

addresses associated with it. Hence, we use these

addresses to parameterize the attractor. The function

xðsÞ is single valued. (If one considers the sequence s tobe the N-ary expansion of a number between zero and

one, then the function can be considered as

x : ½0; 1Þ-A:)However, the function sðxÞ is multivalued, and a

single point in an attractor often has multiple addresses.

An attractor whose points have unique addresses is

necessarily disconnected. But not all disconnected

attractors have points with unique addresses.

We can compute the address of a point xAA by

creating a hierarchy of bounding disk for the attractor.

Given a bounding disk B*A; then the disks wðBÞ*A

and w3wðBÞ*A: Each application of the Hutchinson

operator to the bounding disks creates up to N times

Fig. 2. Sierpinski’s gasket and the fractal pound sign.

P. Sherman, J.C. Hart / Computers & Graphics 27 (2003) 143–151 145

more smaller bounding disks that bound the attractor

even tighter [10]. Organizing these bounding disks into a

hierarchy yields the address of point x as the ancestry of

disks that contain it.

3. Constraining linear fractals

We develop a technique for constraining points in the

attractor to locations on the screen. The constraint can

be a ‘‘nail’’ or a ‘‘drag.’’ A nail constrains the point to

remain in its current position whereas a drag constrains

a point in the attractor to follow the mouse pointer to a

new location on the screen.

These constraints are specified using the addresses of

the points. The constraint is of the form

xðsjÞ ¼ xj ; ð14Þ

which means that the point with address sj should

correspond to the location xj :Given the sequence sj ¼ ði1; i2;yÞ; constraint (14)

expands to the system of equations

xj ¼ wi13wi23?3 ð0; 0Þ: ð15Þ

For an infinite sequence s; the RHS of (15) would

expand to an ‘‘infinite degree polynomial’’ in the 6N

unknowns ai;y; fi for i ¼ 1;y;N:We can approximate s with a finite prefix sequence of

length n determined by screen precision. This approx-

imation yields

xj ¼ wi13wi23?3win ð0; 0Þ; ð16Þ

which is a high-degree polynomial in the 6N unknowns

vi ¼ ðai;y; fiÞ for i ¼ 1;y;N: In fact, the number of

terms of this polynomial is exponential in n: We

represent this polynomial with the function fj of the

6N parameter vector V such that

xj ¼ fjðV Þ ¼ fjðv1; v2;y; vN Þ: ð17Þ

In order to solve the constraints, we need as many

constraints as variables in the system. Hence we

constrain the position of 3N 2-D points xj for j ¼1;y; 3N: This yields the system of equations

f1ðV Þ � x1 ¼ ð0; 0Þ

f2ðV Þ � x2 ¼ ð0; 0Þ

^

f3N ðV Þ � x3N ¼ ð0; 0Þ ð18Þ

which we concisely denote as

F ðV Þ ¼ ðf1ðV Þ � x1; f2ðV Þ � x2;y; f3N ðV Þ � x3N Þ

¼ 0: ð19Þ

3.1. Solution

The numerical solving engine needs to work on a

system of 6N equations in 6N unknowns. The system is

polynomial, but generally consists of thousands of

terms. Due to the high degree of the polynomial, it

was prohibitive to store its terms explicitly. We instead

computed them procedurally.

Newton’s method works quite well with the procedur-

al evaluation of the functions, so long as the Jacobian of

the system can be provided. We approximate the

Jacobian using forward differencing based on the

procedural evaluation of the function at 6N nearby

points.

Newton’s method solves system (19) by generating a

sequence of parameter vectors that converges to a

solution under the iteration

Vk ¼ Vk�1 � J�1ðVk�1ÞF ðVk�1Þ; ð20Þ

where J is the Jacobian of F :Rather than inverting the Jacobian, a common

technique for increasing the efficiency and robustness

of multidimensional Newton’s method is to define

DV ¼ �J�1ðV ÞF ðV Þ ð21Þ

such that the iteration becomes Vk ¼ Vk�1 þ DVk�1:The increment DV is then found by solving

JðV ÞDV ¼ �F ðV Þ: ð22Þ

Other common techniques we used for hastening

convergence and avoiding divergence include ‘‘back-

tracking,’’ ‘‘model-trust,’’ ‘‘hookstep’’ and ‘‘double

dogleg’’ [14]. Our implementation typically converged

in five iterations.

Newton’s method converges rapidly and reliably when

given an initial guess sufficiently close to the solution.

When a point on the attractor is dragged to a new

location very far from its original location, Newton’s

method will likely not converge to a solution. However,

large distances can be divided into any number of small

steps, and these steps can be made small enough such

that Newton’s method converges.

In fact, Newton’s method is fast enough that it can

solve simple RIFS systems in real time. When a point on

the attractor is dragged to a new location, intermediate

attractors are generated as a form of interactive feedback

to show the user the effects of the manipulation.

3.2. Underconstrained systems

To solve for an N map RIFS, one needs to solve for

6N unknowns. For a fully determined system, this

requires constraining the position of 3N addresses. A

usable system must however be able to handle situations

where anywhere between 1 and 3N constraints are

specified.


Constraints where zero, one or two points are nailed

simplify to affine transformations, which can be solved

analytically.

* Zero nailed points (Fig. 3): Let Tð Þ be a translationby the vector ðdx; dyÞ

w0iðx; yÞ ¼T3wi3T

�1ðx; yÞ

¼wiðx � dx; y � dyÞ þ ðdx; dyÞ: ð23Þ

* One nailed point (Fig. 4): Let Rð Þ rotate ðx; yÞ aboutthe nailed point onto the line segment from the nailed

point to ðx0; y0Þ. Let Sð Þ be a stretch (scale in one

direction only) centered at the nailed point in the

direction of ðx0; y0Þ: Then

w0i ¼ S3R3wi3R

�13S�1: ð24Þ

* Two nailed points (Fig. 5): Let Shð Þ be the shear thatleaves the two nailed points fixed but moves ðx; yÞ toðx0; y0Þ. Then

w0i ¼ Sh3wi3Sh�1: ð25Þ

When the user nails between 3 and 3N � 1 points, the

deformation is non-affine (no longer a linear transfor-

mation plus translation), but the system is still under-

constrained. In order to solve this case, we take

advantage of the fact that points generally have more

than one address, and that we are approximating

addresses.

The address resolution algorithm terminates when the

bounding disk diameter reaches screen resolution.

However, the attractor is infinitely detailed and each

pixel of an attractor (that is not totally disconnected)

contains an infinite number of uniquely addressed

points.

Hence given a selected pixel and a given address, we

can append any valid index to that address to add an

additional constraint to the system. This property gives

us the ability to create an arbitrarily high number of fixed

points as necessary to fill a system of 6N unknowns.

4. Results

In this section, we will present a description of the

prototype modeling system that we developed and show

how it can be used to create RIFSs which are shaped

according to the vision of the user.

Fig. 3. Manipulation with zero nailed points and one dragged point P0:

Fig. 4. Manipulation with one nailed point Q1 and one dragged point P0:


In the first example, shown in Fig. 6, one can see how

the shaping of an RIFS attractor is accomplished, and

how it is possible to use the fixed points to freeze shape

elements that are desired. From Snapshot 1 to Snapshot

2, the narrow triangular placement of the three fixed

points, combined with the relatively distant drag point,

yield the shape that retains its basic topology, but

becomes turned up at the edges.

This first move illustrates one of the most important

elements of RIFS manipulation under this system,

namely the global effect of a local move. As one can

see in Snapshot 2, while we were performing manipula-

tions on only one corner of the attractor, corresponding

corners in other parts of the attractor were transformed

as well. This property of local changes having global

effects has its parallel in the interactive manipulation

of spline-based objects. As with the interactive

manipulation of splines, the ability to effect large

changes to the whole with small changes to its parts is

a powerful tool which can yield some very interesting

results. However, as discussed in Section 5, it would

also be desirable to allow local changes to have local

effects.

In Fig. 7, the progressive shaping of an object is

shown. First it is scaled down and rotated (Snapshot 2)

and then more fixed points are added and one of the

corners is turned up (Snapshots 3 and 4). Then, all of the

fixed points are cleared and new ones are set (Snapshot

5) and modeling continues (Snapshot 6). Since the

system is constantly solving for a new RIFS, it is easy to

progressively refine an object. This shaping and reshap-

ing can continue indefinitely.

Fig. 8 shows an early attempt to use the system to

model a given shape. Our initial three-map IFS formed a

rough boundary of a leaf (a), but not the maple leaf

symbol we were after (in red). Constraining and

dragging resulting in images (b) and (c), but we ended

up wanting to impose symmetry and straightening

operations to the maps in addition to the direct

manipulation, which yielded (d).

5. Conclusions

We have presented a method for the interactive

modeling of fractal objects and described a prototype

Fig. 5. Manipulation with two nailed points.

Fig. 6. Smiling Sierpinski’s gasket.


implementation. This method is fast, interactive, in-

tuitive and very extensible. With it, we are able to model

a wide variety of fractal objects using familiar ‘‘click-

and-drag’’ operations.

The system is very easy to use, as are most direct

manipulation systems. The reaction time is interactive,

but the response to quick dragging movements is

sluggish, as the incremental Newton-iterated intermedi-

ate attractors are generated.

Direct manipulation works best when small tweaks

are made to the attractor, but does not appear to be the

best interface when major changes are needed. Direct

manipulation is only a single tool in a modeler’s toolkit

for recurrent modeling.

The current implementation remains unstable.

Depending on the configuration of nailed points, the

system can sometimes be unsolvable as some maps

are not represented in the addresses selected. One

simple example is that all nine constraints could be

imposed on Sierpinski’s gasket corresponding to

nine address each of which only contains maps

1 and 2.

Fig. 7. Manipulations of manipulations.

Fig. 8. Attempt to model the maple leaf symbol.


The system works best when the sections of the

attractor do not overlap. Overlapping sections cause

multiple addresses for points, and make address

computation inefficient. When manipulating a leaf, it

was easier to manipulate the three maps that together

form the boundary of the leaf, leaving out the fourth

map that filled in its interior.

The system does not currently require the affine

transformations to be contractive. It is far too easy to

drag a point ‘‘too far,’’ causing the attractor to

disappear. The system should instead enforce contrac-

tivity as an additional constraint.

We have found it frustrating that we cannot currently

constrain connectedness. Manipulating branching struc-

tures is tedious because the RIFS attractor often

becomes disconnected into an infinite array of disjoint

twigs, or an IFS attractor evaporates into a dust.

Constraining the parameters to remain in a connected-

ness locus [15] may provide the topological constraint

needed to solve this problem.

5.1. Applications

This technique was described as a method for

manipulating RIFS attractors, but also applies to L-

systems. The address of a point in an L-system’s

geometry depends on the actions the turtle has taken

prior to reaching the point. The ‘‘address’’ of a point is

then the turtle commands issued to reach that point.

These actions can be traced back to the L-system

productions that generated them. Hence, this technique

could solve for the new L-system that accomodates the

new location of the turtle at that point in the drawing

process.

This technique can also be applied to general scene

description languages, such as VRML. The ‘‘address’’ of

a polygon depends not only on the coordinates of the

polygon, but the transformations that have taken place

above it in the scene graph hierarchy. This becomes a

particularly difficult problem when the scene graph

contains a lot of instancing at multiple levels in the

hierarchy. This technique could provide a method for

controlling the placement of objects in scenes that

contain large numbers of objects, such as fields of grass

or forests of trees.

5.2. Future work

The most obvious next step would be the extension of

the system to 3D. This technique is not limited to two

dimensions, though there are problems associated with

interaction in 3D environments and the real-time display

of 3D fractal objects.

As far as the existing system is concerned, inclusion

of a method for localizing changes to the model needs

to be added. For example, Sierpinski’s gasket could be

constructed out of three smaller independent gaskets,

such that changes to one of the subgaskets does

not affect the other two. This operation could be

described by the user by specifying the subset of the

attractor to manipulate with a prefix of the address. The

ability to subdivide recurrent models further localizes

their control in a manner similar to hierarchical B-

splines.

Acknowledgements

This work was part of the Recurrent Modeling

Project, supported in part by the National Science

Foundation under grant CCR-95209809 and a gift funds

from Intel Corp. This research was performed in the

Image Research Laboratory at Washington State Uni-

versity.

References

[1] Cochran WO, Hart JC, Flynn PJ. On approximating

rough curves with fractal functions. In: Proceedings

of the Graphics Interface ’98, Vancouver, BC, June 1998.

p. 65–72.

[2] Musgrave FK, Kolb CE, Mace RS. The synthesis and

rendering of eroded fractal terrains. Computer Graphics

1989;23(3):41–50.

[3] Perlin K. An image synthesizer. Computer Graphics

1985;19(3):287–96.

[4] Mandelbrot BM. The fractal geometry of nature. San

Francisco: W.H. Freeman, 1982.

[5] Barnsley MF, Jacquin A, Mallassenet F, Rueter L, Sloan

AD. Harnessing chaos for image synthesis. Computer

Graphics 1988;22(4):131–40.

[6] Barnsley MF, Elton JH, Hardin DP. Recurrent iterated

function systems. Constructive Approximation 1989;5:

3–31.

[7] Prusinkiewicz P, Lindenmayer A. The algorithmic beauty

of plants. New York: Springer, 1990.

[8] Hart JC. The object instancing paradigm for linear

fractal modeling. In: Proceedings of the Graphics

Interface, Vancouver, BC. Morgan Kaufmann, 1992.

p. 224–31.

[9] Demko S, Hodges L, Naylor B. Construction of fractal

objects with iterated function systems. Computer Graphics

1985;19(3):271–8.

[10] Hart JC, DeFanti TA. Efficient antialiased rendering

of 3-D linear fractals. Computer Graphics 1991;25(3):

91–100.

[11] Barnsley MF, Ervin V, Hardin D, Lancaster J. Solution

of an inverse problem for fractals and other sets.

Proceedings of the National Academy of Science

1986;83:1975–7.


[12] Jacquin AE. Image coding based on a fractal theory of

iterated contractive image transformations. IEEE Transac-

tions on Image Processing 1992;1(1):18–30.

[13] Hart JC, Cochran WO, Flynn PJ. Similarity hashing: a

model-based vision solution to the inverse problem of

recurrent iterated function systems. Fractals 1997;3:39–50.

[14] Dennis JE, Schnabel RB. Numerical methods for uncon-

strained optimization and nonlinear equations. New

Jersey: Prentice-Hall, 1984.

[15] Burch B, Hart JC. Linear fractal shape interpolation. In:

Proceedings of the Graphics Interface ’97, Kelowna, BC,

May 1997. p. 155–62.


Direct manipulation of recurrent models

Documents

Transcript of Direct manipulation of recurrent models