A COLOR ANIMATION SYSTEM

BASED ON THE MULTIPLANE TECHNIQUE

Marc Levoy

Program of Computer GraphicsCornell University

Abstract

This paper describes an animation package currently under development at the Cornell Program ofComputer Graphics.

The basic algorithm employed is linear or non-linear interpolation between successive pairs of keyframes. These key frames are composed of artwork input by the animator on a graphic tablet and displayedon either a black and white vector scope or a color halftone CRT. The initial working environment is two-dimensional, and the individual images are combined using a multiplane cel animation technique to producedepth and motion illusions. Real-time film previewing, utilizing an on-the-fly interpolation algorithm,provides the artist with instant playback of animated sequences.

KEYWORDS: Key-frame Animation, Real-time Animation, Multiplane Animation, Interactive Computer GraphicsCOMPUTING REVIEWS CLASSIFICATION: 3.41, 8.2

Interaction between the artist and his animationis purely pictorial, and picture generation pro-

ceeds from these key drawings instead of fromprogrammed commands. By subdividing each sketchinto discrete strokes, and from there into a largenumber of closely spaced points, the computer canbe programmed to generate frames intermediate tothe key drawings using a simple linear interpola-tion algorithm.

Several existing animation systems are basedon this approach, including those developed byWein (8) and Catmull (1). There are, however,major problems with key-frame animation. First,there must be a one-to-one correspondence ofstrokes between the two terminal sketches in atransformation. While this scheme provides asimple means for controlling the overall transfor-mation, certain classes of actions become ex-tremely difficult to achieve, such as large rota-tions of objects. Second, linear interpolationbetween two substantially disparate drawings in-variably results in the "collapsing phenomenon",wherein the lines of the drawing appear to dis-integrate and mingle chaotically during the courseof the transformation. Wein has provided severalexcellent suggestions for alleviating these arti-facts of key-frame interpolation (7). A thirdimpediment relates to the organization of timingsin the key-frame method. By adopting an algorithmfor animation wherein each transformation isplanned separately, the problem of maintainingsmooth transitions becomes critical. In most ofthe work produced to date the tempo of the keyframes is highly apparent,and since continuity isessential, the key-frame method presents a defin-ite constraint in this area. Finally, a limita-tion is imposed by forcing the artist to inhabitan essentially two-dimensional world. While it isa simple matter of draftsmanship to produce a sta-tic image which successfully portrays depth, it isentirely a different problem to produce key-frameinterpolations of objects that retain true per-spective throughout the transformation.

I. Introduction

The last five years have seen the developmentof several sophisticated computer animation sys-tems. To review a fairly well established taxono-my, two general categories can be identified:language-driven systems and picture-driven systems.Each type offers certain advantages to the artist,although each contains certain inevitable draw-backs. These may be summarized briefly as fol-lows:

Language-driven systems:

Beginning with Kenneth Knowlton's pioneerpixellation films of the early sixties and con-tinuing up through the current research of theOhio State University Group (2) and others, alllanguage driven animators have in common a de-sire to achieve maximum flexibility in theiranimation environment through the provision ofa wide repertoire of picture manipulation com-mands. By allowing the artist to specify anytwo-dimensional or three-dimensional transfor-mation upon geometries in the working dataspace, these systems provide considerable free-dom for the\development of artistic scenarios.

There are, however, distinct difficultiesin this approach. First the generation of three-dimensional objects with sufficient ease and ofadequate complexity to serve as artistic ele-ments in computer animation is a problem of con-siderable complexity in itself. Second, sincethe working environment of these animation sys-tems is manipulated and distorted according tocommands entered on a computer terminal, theartist must acquire at least some familiaritywith language syntax.

Picture-driven, key-frame animation systems:

In systems of this class the animator pre-pares selected images at key intervals in thesequence and sketches them on a graphic tablet.

This work has been partially sponsored by the National Science Foundation under grant humber DCR74-14694entitled "Development of Computer Graphics Techniques & Applications."

65

The Cornell Program of Computer Graphics iscurrently developing an animation system for even-tual use in a production studio environment. Thesystem is basically designed on the key-framemodel of computer animation, for which excellentdescriptions can be found in (4) and (8). It alsoprovides three-dimensional capabilities,. but, inorder to prevent these additional degrees of free-dom from becoming too unwieldy, they are dispensedonly in a carefully controlled manner, as des-cribed in the next section.

II. THE CONCEPT OF MULTIPLANE ANIMATION

In the animated cartoons of the 1920's, thebackgrounds were designed strictly as static, two-dimensional backdrops. They lent a sense of placeand scale to the animated characters in the fore-ground cel, but rarely dealt with the space be-tween these figures and their horizon.

When Walt Disney began work in 1936 on hisfirst feature-length film, Snow White, it becamenecessary to depict settings wherein the charac-ters were separated by vast distances from theirbackdrops. Carefully constructed perspectivescould provide an impression of depth as long asthe observer's viewpoint was static, but a verydifficult problem arose when the script calledfor a zoom or a panoramic sweep. In order tosimulate a movement of the imaginary observer to-ward a character in the foreground, the figureshould enlarge in the field of view. This can beaccomplished simply by bringing the camera closerto the animation platten. The background scenealso increases in apparent size during the zoom,but at a much slower rate due to its greater dis-tance from the observer. If the background celwere kept flush with the foreground figure as ithad always been in traditional animation, itwould enlarge as rapidly as the foreground andthe illusion of depth would be lost.

This problem was solved by Disney with theaid of an awesome machine called a multiplanecamera, shown in Figure la. Each layer of ani-mation is situated at a different distance fromthe camera and illuminated independently. Thus,to achieve a realistic zoom, each cel layer ismoved toward the camera at a speed inversely pro-

Figure la: The multiplane camera stand used byWalt Disney in his productions of the 1930's.

portional to its distance from the imaginary ob-server. The differential ratios in these ratesdetermine the apparent position of each drawingplane in depth. This was and still is one of thecleverest devices ever to be used in animation.Unfortunately, each cel layer required severaltechnicians to operate it, and for this reasonit has been used sparingly since the first fea-ture films. Nevertheless, it represents an ex-cellent algorithm for generating animated se-quences of correct perspective and realisticthree-dimensional motion from a set of essen-tially two-dimensional drawings (3).

III. THE CORNELL ANIMATION SYSTEM

1. General organization:

Figure 2 illustrates the layout of the anima-tion package. It has been developed as a dual-display system, alternating between a black andwhite vector scope and a color halftone CRT.

Figure 2: Schematic organization of the Cornellanimation system, showing the interface betweenthe vector and color graphic systems. For a com-plete description of the equipment configurationof the Cornell laboratory, refer to (5), also inthis issue.

Figure lb: The multiplane editor of the Cornellanimation system (with its menu).

66

Input of the artwork can be initiated fromeither device, depending upon the graphic mediumselected by the artist. This process is entirelypictorial, and the primary input means is a gra-phic tablet and pen.

The data then passes to the multiplane man-ipulation routines. Here, the sketches and paint-ings are combined into a multiplane animationenvironment analogous in concept to that used byWalt Disney. This segment of the animation pack-age operates exclusively on the vector display,since that device offers the best real-timecapabilities.

At this stage, the data consists of a seriesof key frames specifying the exact state of theanimation at selected intervals in the sequence.The final step is for the computer to perform themathematical interpolation between these keyframes, and to display the resulting sequence onone of the output devices. Two means for preview-ing are available to the artist, real-time black& white playback or stop-frame color playback.The restriction of real-time playback to the vec-tor scope is due entirely to the limitations ofavailable technology. After examining the anima-tion, the artist then has the option to refineeither his artwork or the multiplane setup andthen repeat the procedure. When the previewingfinally yields satisfactory results, productionfilming may begin. For this purpose, each com-posite image is displayed one frame at a time onthe color CRT and filmed using a permanentlymounted 16mm camera. This process, although slow,is performed automatically by the computer andrequires no user intervention.

In the following sections, each of the threebasic steps outlined above will be explained indetail, and examples will be provided for eachstage of the process.

2. Building the two-dimensional planes:

Each plane in the multiplane environment con-tains one animation sequence developed by the ar-

tist. Several different animation techniques areprovided, but all of them generate a continuum oftwo-dimensional images for that plane.

a) Key-frame interpolation of line drawingswith linear output (Figure 3a)

The procedure for developing a key-frame sequencefrom line drawings is well known and has been des-cribed elsewhere (8). The final product of thismethod is a sequence of frames containing vector

approximations of interpolated curves. This maybe displayed on either a vector scope or a raster

CRT.

b) Key-frame interpolation of line drawingswith color output (Figure 3b).

Utilizing a simple area-filling algorithm, conti-guous areas enclosed by the original line drawingcan be flooded with colors from a "palette" anddisplayed on a color raster CRT, yielding an imagesimilar in appearance to a traditionally opaquedanimation cel. If a set of "seeds" to drive the

area-filler is interpolated along with the profilelines, intermediate frames can be automaticallygenerated with full color opaquing. When theseframes are later superimposed over other sequencesin the multiplane environment, multiple-cel opaqu-ing is effected.

c) Static color backdrop (Figure 4a)

The rear planes in Disney's multiplane animationscontained highly elaborate backgrounds, far toocomplicated to be viably animated. Many of themwere oil paintings on glass plates, a nearly in-credible combination of materials to handle, butone which allowed great variation in texture andtonal delicacy. As part of the Cornell animationsystem, a painting program has been developed inwhich the artist works with computer-generatedanalogies of traditional drawing implements, suchas pens, brushes and washes. (This program issimilar in design to the SHAZAM system of Xeroxand its descendants in other graphics laboratories(1) (6). Using this program, full color backdropsof considerable complexity can be achieved.

d) Color-palette interpolation of staticbackdrops (Figures 5a & 5b)

Although the complexity of painted backgroundsprohibits geometrical animation, there existsequally interesting possibilities in the interpo-lation of the color palette. For example, sup-pose that the artist colors a rendering of animaginary temple in its nighttime array of bluesand deep purples. Then, without repainting thebasic figure, a sequence of palettes can be speci-fied that depict the play of light and color inthe hall as the sun rises. These palettes canfinally be used by the computer as key frames fromwhich an entire animated sequence of subtle chro-matic shifts can be produced.

e) Back-projection of three-dimensionalsimulation(Figures 6a & 6b)

Constraining the artist to inhabit an exclusivelythree-dimensional world has definite drawbacks.There are, however, instances in which three-dimensional simulation promises to yield the bestbackgrounds for a particular scenario. Archi-tectural settings exemplify stagesets that wouldnot alter in shape, yet might be viewed from avariety of different angles during the course ofan animation. The Cornell animation system pro-vides these three-dimensional simulation capabil-ities, although only in the following carefullystructured manner. Attaining the correct arrange-ment of two-dimensional animation planes within athree-dimensional stageset is analogous to theproblem of combining animation and live-actionfilming in one sequence. Unless the spatialarrangements are executed flawlessly, one of themedia will "go flat", or seem to lose its perspec-tive depth. Figure 6a illustrates this difficulty.The simultaneous juggling of the two image typeswith respect to perspective alignment would be ex-tremely cumbersome to handle in this setup. Onesolution to this problem, and the one adopted forthe Cornell animation system, is to separate thetwo media, compose one independently, then super-impose the other at a later stage. Specifically,the three-dimensional simulation is worked out

67

Figure 3a: Key-frame interpolation of linedrawing - vector output

Figure 4a: Static color background displayed oncolor screen.

Figure 5a: Combination of two-dimensional andthree-dimensional animation, in night colors.

Figure 3b: Key-frame interpolation of linedrawing - color output

Figure 4b: Vector approximation of 4a for displayon vector scope (allowing real-time manipulation).

Figure 5b: Combination of two-dimensional andthree-dimensional animation, in morning colors.

68

Figure 6a: Direct combinationof 3d and 2d animation

Figure 6b: Back projectedcombination of 3d and 2danimation.

first, complete with camera angles and motion dy-namics. Then the resulting sequence is back-pro-jected as a continuum of two-dimensional images onthe background plane of the multiplane environ-ment, as diagrammed in Figure 6b. Finally, theartist proceeds to develop his two-dimensionalanimation as a superimposition over the simulationsequence. In much the same way that traditionalanimation was executed to match a pre-recordedsound track, the multiplane animation in this ex-ample would be altered and refined to overlay apreviously established background sequence. us-ing this. method, correct perspective and align-ment between the two media are virtually assured.The fountain and temple in Figures 5a & 5b werecombined in this manner, with the temple input asa two-dimensional painting and the fountain as athree-dimensional simulation.

Figure 7a: Manipulating the observer's frustrumof vision in the multiplane editor.

3. Constructing the multiplane animation:

Figure lb illustrates the geometrical struc-ture of the Cornell animator (compare to Figurela). The imaginary observer in this system de-fines a three-dimensional cone or frustrum ofvision which is transpierced at carefully de-fined intervals with data planes. The two-dimensional raw animation sequences of the artistreside on these planes.

By manipulating the object planes and obser-ver's position, the artist can choose the bestviews and sequences to tell his story. As Figures7a & 7b illustrate, the multiplane effects areachieved by translating the observer's positionin any of the three directions or by tilting theline of sight in any of the three degrees of ro-tational freedom. In addition, translations, ro-tations and scaling functions may be specifiedfor each plane in the environment for the possi-bility of interrelational movements among the celsof the composite.

For the previous stages, some of the tech-niques described utilize a vector-drawing displaywhile others depend on color raster graphics.For the next stage in the process however, allwork must be representable on the vector scope.To provide this capability, an edge-finding algor-ithm can be utilized to generate a coarse vectorapproximation of those drawings created originallyfor the color display. This vector simulcrum, anexample of which is shown in Figure 4b, containssubstantially less information about detail andtexture than its color genesis, but it is adequatefor the task of arranging the planes in space andspecifying the sequence dynamics.

Interaction between the artist and the compu-ter is entirely pictorial, requiring no knowledgeof programming or language syntax. Motion feed-back is immediate, in real-time and at full resol-ution, utilizing the fast vector drawing system.Throughout this process, the locations of theplanes and observer are being constantly tracked,so that acceptable sequences can be replayed auto-matically and included in the final animation.

Figure 7b: Manipulating the object planes in themultiplane editor.

69

4. Real-time previewing:

One important feature in any animation systemis the ability to preview animation sequences atcine projection speed (24 frames per second), orat least at standard animation speed (12 framesper second). In the Cornell system, real-timeplayback is achieved using the fast vector draw-ing display. For this purpose, the vector approx-imations of all color drawing is utilized.

A hardware matrix processor is capable ofhandling in real-time whatever translations androtations may be specified for each plane in theenvironment. Unfortunately, however, the linearinterpolations utilized in the key-frame sequencesrequire substantial processing time since the cal-culations must be applied in software to each ofthe thousands of points in each sketch. Thesemassive computations presently make real-timeplayback extremely difficult to achieve.

The possible programming strategies forachieving real-time playback of interpolated se-quences fall into one of two categories:

1) Pre-calculated interpolation, in whichthe geometry of each intermediate frameis computed before the previewing beginsand deposited on a large capacity, highspeed mass storage device for subsequentplayback.

2) On-the-fly interpolation, in which thegeometry of each intermediate frame iscomputed by the host computer just priorto its display on the screen.

The inherent disadvantages of pre-calculatingthe in-between frames and storing them on the diskare fairly evident. First, it incurs waiting timebetween the input and playback phases of the pro-cess. Second, it places an absolute limit on thenumber of frames that may be previewed at once,depending upon the particular storage medium.Third, any subsequent modifications to the anima-tion require that the entire sequence be recompu-ted before it may be previewed again.

By using an intelligent graphics processorthat is substantially independent from its hostcomputer, sufficient computing power remains inthe CPU to implement an on-the-fly interpolationscheme. Furthermore, utilizing algorithm andcode optimization techniques, the computationsnecessary to perform simple linear interpolationcan be diminished significantly from the basicformula.

Three processes are actually involved inproducing the continuous animation on the screen:

1) Loading of each pair of key frames fromthe disk.

2) Interpolation of the sketch data.

3) Matrix transformation and display of theintermediate frames.

70

The key to producing this fast continuous anima-tion lies in the fact that each process isachieved by a separate part of the computer sys-tem, operating in an asynchronous, multi-taskingenvironment. The three independent devices in-volved are the mass storage controller, the CPU,and the display processor.

These three asynchronously operating devicesare slaved to the previewer in the Cornell systemusing a network of pseudo-simultaneous co-routinesexecuting in the central computer, as shown inFigure 8. At the start of key frame (n), thedisk controller is instructed to begin transfer-ring the data for key frame(n+l) to a core buf-fer. At the same time that the disk transfer istriggered, interpolation commences on the firstplane of key frame (n). When this plane is com-plete, the display processor is commanded totransform and display the plane. This is achievedby loading the plane's geometry into the displaypipeline along with the matrices desired fortranslation and rotation. Meanwhile, the CPU isworking on the interpolations for the second plane.When all planes have been computed and processedfor display, work begins again for the next frame.By the time the entire key frame is exhausted, thedisk operation is complete and the next key frameis ready for interpolation.

Since the display processing operates fasterthan the interpolations, the critical path in-cludes all of the interpolations, but no otherprocesses, as the diagram indicates. If by chancethe work required to display a single frame iscompleted ahead of schedule, an internal clockdelays the program for the proper interval oftime. The work will never fall behind schedule,since before the playback process begins, eachsketch has been subdivided only into a number ofpoints that can be handled at 12 frames per second.For our present equipment configuration, thislimit has been measured at about 2800 points,which allows sketches of far greater complexitythan can be previewed using the pre-calculationmethod, due to disk-to-core transfer speed limi-tations. Of course, sketches of considerablecomplexity take on a slightly coarser appearancedue to this filtering/subdivision algorithm, butthe results so far have been very satisfactory.For sketches containing an initial point countof under 5000, the filter-down is almost impos-sible to detect. Very seldom has any sketch pro-duced by the artists working on the Cornell sys-tem exceeded 7000 points, which makes the tech-nique viable for use in a production environment.

In addition to the real-time playback capa-bility, individual frames in the animated se-quence can be transferred to the color displayfor stop-frame examination. The image thus pre-sented to the artist looks exactly as will thefinal film, including multiple-cel opaquing andfull color shading.

IV. CONCLUSION

This paper has presented an overview of thekey-frame computer animation system currentlyunder development at the Cornell Program ofComputer Graphics.

The basic algorithm employed is a linear in-terpolation between successive pairs of key frames.These key frames are composed of artwork input bythe animator on a graphic tablet and displayed oneither a black and white vector scope or a colorhalftone CRT. The attributes of this animationpackage are:

1) The initial working environment is two-dimensional, and any of the followinggraphic media may be used:

a. Free-form line sketches.

b. Line sketches with color infilling.

c. Color paintings.

d. Two-dimensional projections of com-plex three-dimensional scenes.

2) These two-dimensional images are combinedusing a multiplane cel animation tech-nique to produce depth and motion illu-sions.

3) Real-time film previewing, utilizing anon-the-fly interpolation algorithm, pro-vides the artist with instant playbackof animated sequences.

4) All interaction with the program is pic-torial in order to facilitate the use ofthis system by the non-programming artist.

It is evident that numerous approaches havebeen taken to computer animation. This can beascribed,to a large extent, to variations in artis-tic taste. There are, however, problems common toall animation systems which must be solved beforethey can be viably used for production work. Theseinvolve increasing the resolution, allowing forreal time playback at this increased resolution,and providing a more efficient input system.Future research in computer animation must addressthese issues, as described below:

1) Most currently available color frame buf-fers are designed for the 512 x 512 nomi-nal resolution of a video monitor. Inorder to obtain a satisfactory image fortheater projection, a resolution of atleast 2000 x 2000 is required. The enor-mous data storage problems this implies,and the lack of any viable device forpreviewing such images, pose formidableproblems for computer animation systems.

2) The complex pixel-by-pixel computationsrequired to generate a single frame ofcolor animation make real-time color pre-viewing very difficult to achieve. Theonly viable solution yet evolved for thislimitation is to compute each frame before-hand and store it on a video disk or stop-frame video tape-recorder for subsequentplayback in real-time.

3) Most animation systems currently requirethat the artist input his sketches manual-ly, using a graphic tablet and pen. Thistime-consuming extra step can be elimina-ted through the use of a high-precisionoptical scanner. The edge-finding andstroke identification algorithms necessaryfor such a procedure pose complex problemsin themselves, but if solved, promise tofurther ease the labor burden on theanimator.

V. REFERENCES

1. Catmull, Edward, (Personal conference at NewYork Institute of Technology), New York,(1976).

2. Defanti, Thomas, "The Graphics Symbiosis Sys-tem", Real-Time Film Animation, annual reportto the NSF, submitted by the Computer GraphicsResearch Group at Ohio State University,(1973), pp. 5-116.

3. Finch, Christopher, The Art of Walt Disney,Walt Disney Productions, (1975).

4. Gracer and Blasgen, "Karma: A System for Story-board Animation", UAIDE,(1970), pp.210-224.

5. Greenberg, Donald P., "An InterdisciplinaryLaboratory for Graphics Research and Applica-tions", Proceedings of the Fourth AnnualConference on Computer Graphics, InteractiveTechniques, and Image Processing (July, 1977).

6. Shoup, Richard, (Presentation of the SHAZAManimation system at the Third Annual SIGGRAPHConference), Phila. Pa., (1976).

7. Wein, Marceli, "Interactive Skeleton Techniquesfor Enhanced Motion Dynamics in Key-FrameAnimation", Communications of the ACM,Oct. 1976).

8. Wein, Marceli, and Lestor Burtnyk, "ComputerGenerated Key-frame Animation", Journal SMPTE,(March 1971), pp. 149-153.

71