EUROGRAPHICS '92 / A. Kilgour and L. Kjelldahl (Guest Editors), Blackwell Publishers © Eurographics Association, 1992

Volume 11, (1992), number 3

Rendering NURB Regions For 2D Animation

Philip Willis and Robert Oddy

Computing Group, School of Mathematical Sciences, University Of Bath Claverton Down, Avon, BA2 7AY, United Kingdom

Abstract We describe an accurate method of rendering by scan-conversion of closed regions bounded by NURBs, with particular comments on its application to computer-based 2D animation. It is shown that the method is fast, analytically accurate, and can be readily extended to include anti-aliasing and clipping.

Keywords: NURBs, animation, rendering, anti-aliasing

1 Introduction

Traditional cel-based animation has been poorly served by the computing community, which has tended to concentrate on the use of computers for three-dimensional animation. Attempts to use pixel-based systems for producing two-dimensional animation have had a limited success, but have been mainly restricted to simple trace-and-paint applications where colour infill is especially helpful [6].

Our approach is to try to bring together the best of computer modelling and rendering techniques within a cel-based system, thereby adopting an approach which is outwardly familiar to the animator but internally harnesses the computer. We start with the conventional view taken in 3D modelling, namely that an object should be represented by a data structure so that it can be readily manipulated and modified. In doing this we have already implied that paint programs are not an allowed solution because they have no data structure above the pixel array, Then we continue to argue conventionally: in order to view the frame we will need a lighting model and a renderer. What we are doing which is new is bringing this consistently to bear on a 2D system while solving the problems that arise on the way. The advantages of this approach are all of those arising from a correct computer-based representation, namely that picture elements can be modified, scaled, rotated, and translated in a device-independent form with no reference to output spatial resolution, output colour resolution and, significantly for animation, with no regard for the output frame-rate. This approach is thus future- proof in that any scene modelled this way can at a future date be re-rendered at any required resolution or frame rate. It also gives back to the animator something which cels remove, namely a view ofmotion as a continuous action.

In this paper we focus on the foundation layers, namely the modelling and rendering of individual static frames, of what we intend to be a complete 2D animation system. For now, we are assuming that cels are drawn with generalised painted regions and it is these regions which need to be represented and rendered. Thus the main direction of the work here reported is that of scan-converting regions bounded by NURBS (non-uniform rational B-splines, based on the work of [1]). We do not consider the higher level, which is the provision of control and a user-interface which actually allows animation to be designed and tested.

C-36 P. Willis et al. / Rendering NURB Regions

2 The Basic Approach

We adopt a traditional cel animation arrangement; that is, pictures are created on clear sheets which are overlayed, illuminated and then photographed to yield a frame of finished film. Some of the cels are then changed or moved, the next frame is illuminated and exposed, and so on.

Our computer-based system mimics this in the first instance. However we also take advantage of good graphics practice t o give us further advantages not available to the traditional animator. For the most part, these arise from the way the pictures are modelled, the way they are realised and the way they are combined. We will examine these in turn.

3 Modelling A Picture

3.1 Modelling the picture colouring

We consider first an isolated cel. A way is needed to distinguish painted areas from clear areas, and opaque paint from transparent paint, so that multiple cels can subsequently be overlayed to produce the correct colouring. To do this we require a colour model which allows for transparency and which is not specifically pixel-based. The alpha channel model is the one most commonly used for overlaying images, but this is not in fact adequate. The alpha value is primarily a pixel-coverage factor; it takes no account of the geometry within the pixel and so can only give the illusion of transparency in simple cases. Recognizing this, we have developed a general purpose colour representation which we call the beta model [5] and this provides all that we need. Briefly, this works by assuming that paints have two components, a medium and a pigment that is suspended within the medium, each of which is separately coloured. The medium corresponds to how the material affects transmitted light while the particles correspond to how the material affects reflected, scattered light. The beta parameter determines the proportion of particles to medium, allowing fully transparent (though coloured) materials as well as fully opaque ones. A particular advantantage of the beta model is that it supports arbitrary mixing and overlaying of transparent and opaque paints.

Hand in hand with this representation of the colour of the picture components we also adopt a lighting model. Again it is important to note that we are fully modelling the entire system and we are specifically not assuming that the colour of the model component is itself the output colour. Instead, we assume that each cel is illuminated from the front by a light of known brightness and colour, and that there is a similar but independent light behind the cel. Lighting parameters can of course be modified or even animated. Thus we model full control over front and rear lighting of each cel (not just over the complete stack of cels). Lighting can be dynamically controlled and is an important way of achieving certain special effects.

3.2 Modelling the picture geometry

Next, we need to represent the coloured areas of the cel, in a way which can readily be changed interactively. We have adopted the non-uniform rational B-spline (NURBS) for this, being a very general form of curve. Our basic primitive is thus an irregular 2D region bounded by a closed NURB and filled by a single colouring.

Of course, other forms might be just as acceptable but NURBS have a generality which is appealing at the cost of some computing power. In fact the key item here is not the NURBS representation but the fact that all pictures are constructed of closed areas. These areas are not defined with respect to a pixel grid and are thus an abstract, fully transformable form. One consequence of this is that there is

P. Willis et al. / Rendering NURB Regions C-37

no equivalent to a Bresenham line in our system. Lines are simply elongated regions and they therefore scale correctly, looking thicker when magnified for example, just as in our earlier work [7]. Areas of colour are broad regions with a regular or irregular outline. One of the attractions of NURBS is their ability to represent regular shapes like circles as well as free-from shapes.

The interior of a region is assumed to be coloured but not necessarily by a single colour. This part undoubtedly needs a proper study, but for now we are assuming that the interior colouring can be mapped from a general colour space. Even so, many cartoon applications only require a uniform interior colour and this is what we currently support.

Complex shapes or colourings can be built up hierarchically, with overlapping and/or translucent regions where appropriate. The shapes are readily transformable and, again, the transforms can be applied hierarchically in the familiar manner. Note that what we are really saying is that there i s a properly constructed model of the picture, independent of its visual form, just as we would expect to find in engineering CAD and other applications but which has often been lacking from this kind of application.

Hence all items have a fully-defined geometry, allowing all the usual transformations such as rotation, translation and scaling. Furthermore, because there i s really only one underlying primitive, rendering becomes greatly simplified. Of course, all items are represented in an abstract, pixel-independent space throughout the modelling (and indeed through most of the rendering).

4 Rendering A Picture Cel

So far we have described the basis on which we model pictures. In practice, most of our programming effort has centred on rendering. This is because we need to be certain that we can render successfully a combination of NURB regions of beta colouring with a generalised lighting, working with layers of cels. Achieving this means we first need to solve the fundamental problem of efficiently scan-converting a closed NURB. In our case we assume that all NURBS will be of degree three, and will thus produce cubic polynomial NURB segments.

4.1 Rendering the NURB regions

We use scan-conversion of NURBS to yield the anti-aliased geometry of the viewable picture. Here we see one benefit of using NURBS: because they are general purpose curves, our scan-converter need only handle this one type of primitive, making it much easier to cope with difficult cases properly. Conversely, it is much easier to optimise the scan-conversion process when only one primitive is in use.

The scan-converter itself assumes that the NURB regions are presented to it in the proper order, front-most region first (so that regions overlie in the correct sense) and so all sorting is performed by a preparation phase in the main application program (which is where the ordering information is to be found). This frees the converter from any knowledge of how the application models the pictures, easing code development as well as allowing it to be used with other applications such as illustration packages.

Our view of scan-conversion is that we calculate the intersections of a given scanline with a given NURB, producing a sequence of interior and exterior spans. We do not perform the intersections incrementally, in the interest of obtaining robust results and to allow selective change of conversion resolution. The spans across a given scanline can then be combined according to our colour model, though it is also possible to extend this phase to include CSG operations between NURB regions, giving another modelling option.

C-38 P. Willis et al. / Rendering NURB Regions

Figure 1: Basic Features Of A NURB

In detail, scan-conversion of a single closed NURB region proceeds as follows. We start with the control points for the NURB and calculate from them the cubic curve which represents each segment (see Figure 1). With N control points we will produce N segments because the NURB is closed (An open NURB with control points to is closed by making with the same weights and inter-knot spacing). The method we use to calculate the cubic curves is a modification of that of Choi, Yoo and Lee [3].

Their approach shows how to calculate a coefficient matrix, once for each segment, given the parametric inter-knot spacing for the NURB curve as a whole. The resulting matrix contains the majority of the values required for calculation of an (x, y) position on the NURB segment, given a parametric value in the range [0.0, 1.0). The application of the matrix requires two simple multiplications, firstly of the matrix with a vector comprising the x, y or w part of each control point (where w is the weight associated with each point). Secondly, the result of this (a vector) must then be evaluated into a cubic polynomial of the parametric value u by multiplication with another vector:

The matrix can be applied separately to the x, y and w parts of the control points. Application of the matrix to the x or y parts results in a homogenous x or y coordinate, which must then be rationalised (i.e. divide the homogenous coordinate by the result of the application of the matrix to w) before further use.

P. Willis et al. / Rendering NURB Regions C-39

In order to calculate the intersection of a particular scanline (i.e. an infinite line of constant Y) with a segment of the NURB, we need to evaluate the standard rational form of the NURB curve such that the result is Y. The standard form is:

By rearranging the above equation, we produce a cubic equation the roots of which correspond to the parametric positions of the intersections between the segment and the line:

Given that Y has already been chosen all that remains to be done is to evaluate the above directly, i.e. without using iteration. We use the pre-calculated values in the coefficient matrix, to yield parametric values which are in turn used to compute the required x values (the points of intersection), This need only be done for parameters between [0.0, 1.0) on each segment as only this range contributes to the NURB (see Figure 1). We used the REDUCE [4] computer algebra package both to verify the Choi coefficient matrix and to generate the C code that implements it.

Our method has several advantages. Like Choi, Yoo and Lee [3], we pre-calculate the coefficient matrix so that it may be used both for the calculation of the intersection and in the evaluation of the rational form. The amount of computation and the complexity of the required program are further significantly reduced in our scan-conversion approach because there is no Y calculation necessary, thus halving the computation for the intersection. No computational gains would arise from evaluating forward differences for the curve, since the resulting difference matrix is of a similar size as the coefficient matrix, whilst introducing numerical problems.

Additionally, it can be seen that our method provides significant advantages over polygonal approxi- mation methods. By its very nature, our method automatically adapts to the resolution of the output device, rather than requiring the use of a heuristic function (the degree of parameterisation) to es- timate the degree of polygonisation required. This adaptation shows itself at its best when weights are used to change the shape of the curve because no re-parameterisation is necessary. Additionally, our method will always produce correct output even if the NURB becomes highly curved or self- intersecting. Furthermore, should we be presented with conic sections (one of the most commonly used NURB forms in CAD/CAM and computer graphics), then the standard form simplifies to a quadratic and so root-finding is easier.

Finally, it is important to note that the entire calculation is performed with floating point arithmetic; in particular the x and y values are real quantities, not integers, so not only is there no presumption of discrete pixels but also we retain full resolution, well beyond the relatively poor resolution of any physical display. Thus our representation is both inherently portable and well-capable of addressing even the very high resolutions used in, for examples, the printing industry or in originating high-quality movie film.

Furthermore, each Y scanline is computed independently of any other, so there is no incremental updating and therefore no possibility of accumulating errors. Data structure management is also simplified: we build one output list anew for each scanline. This makes clipping against the upper and lower edges of the screen entirely unnecessary: such regions are simply never needed. We thereby avoid one potential problem with incremental methods, namely that when most of the curve is above

C-40 P. Willis et al. / Rendering NURB Regions

Figure 2: Aliased Cel

the top of the screen we do not waste time computing invisible sections in order to calculate the shape of the visible. Horizontal clipping (against the left and right edges of the screen) is also very simple. Each line/cubic-curve pair produces one of four results: none, one, two or three intersections. These floating-point x values, and those for all other active segments, are insertion-sorted into a linked list representing the output at the current y position. Some of these values will, when mapped onto the screen, be outside the left or right picture edges. Those spans beyond the right-hand edge can be rejected immediately they are found. Any which straddle the right-hand edge can be clipped immediately or at the stage when the scan-line is turned into pixels: either is fast to do. It is however important to retain those beyond the left-hand edge so that we can correctly determine which spans are internal to the region. This is done by walking along the linked list, starting with an “internal” flag set to false. In general each new intersection toggles this flag, identifying the interior regions. In practice, some intercept pairs may be identical (because the curve is self-intersecting) in which case both may be discarded as they cancel out. (It is not strictly necessary to keep the entire list for the part to the left of the screen, as long as the parity of the “internal” flag is correctly maintained, but our current implementation does take the simple solution.) This approach i s analytically accurate, incorporating as it does spans which are smaller than a pixel and spans which are not pixel-aligned, and so avoids the usual special-case problems which arise when determining interiors. Figure 2 shows a NURB shape scan-converted to pixel accuracy; in contrast, Figure 3 shows the same shape with the correct horizontal alignment represented through partial pixel intensities. It is an important advantage of retaining accurate intersection information that it allows the correct calculation of horizontal anti- aliasing.

P. Willis et al. / Rendering NURB Regions C-41

Figure 3: Horizontal Anti-Aliasing

Figure 4: Full Anti-Aliasing

C-42 P. Willis et al. / Rendering NURB Regions

We have devised, but not yet implemented, a general geometric solution to anti-aliasing which makes use of the accurate horizontal information together with a good approximation to the geometry in the vertical direction. Currently we are using super-sampling to achieve vertical anti-aliasing, though even here our approach is efficient because we have already computed the horizontal intersections accurately. Put another way, the cost of super-sampling with scan-conversion is linear with resolution, whereas pixel-sampling methods have a cost proportional to the square of the resolution. Figure 4 shows the test shape with 5 to 1 vertical supersampling, as well as horizontal anti-aliasing. Not only does this give very good edges to a general drawing primitive, it also means that slow movement of the shape becomes possible. This is at least as important as edge quality to our animation application area, where pixel-aligned systems would produce a very jerky result and/or “crawling” edges.

4.2 Rendering the colour

Our lighting model ensures that each cel has a front-light and a back-light, each of which can be independently controlled for colour and brightness. The front-light affects the solid paints while the back-light affects the translucent ones. The calculation of the interaction of light with paint is described in our earlier paper [5] and is the basis of our colour rendering, as well as the modelling of overlays and paint mixing.

The beta model permits both hard and soft masks (mattes), which are optionally coloured. So, together with the cel lighting, quite sophisticated lighting and special effects are possible. Lighting is itself a powerful weapon in the animator’s armoury. For example, we can model a scene in the conventional way and then illuminate it for dusk or night time. A judicious choice of lighting will modify the perceived colours of the scene and can even be used to change the relative emphasis of scene components, as when light falling from a window picks out a central character in an otherwise night-time scene. The two colour plates (Figures 5 and 6)*are pixel images arranged on cel layers, matted together with all of the lighting and colour combination performed using the beta model. It is important to note that the original foreground and background images have not been changed between these two views.

5 Rendering Multiple Cels

Cels can be stacked together, just as they would be for conventional animation. We retain the option of a pair of lights for each cel, in order to give us localised control, but it i s important to note that cross-cel lighting effects can now take place. For example, there can be light from a rear cel illuminating one in front. In addition, it is also possible to include pair of lights for every stack of cels.

The basic lighting scheme provides a uniform light evenly illuminating the entire cel. This can easily be modified by providing additional cels to mask parts of the scene. These matte cels are created in the same way as any other; that is, they are NURB drawings with no reference to the pixel structure of the final image, There is thus no need to treat them as special cases within the renderer. It is important to realise that the NURB boundaries are not quantised by the renderer to unit pixel values and so can be blended correctly with any underlying image. This approach, using geometrically accurate boundaries, thus avoids the complexities associated with the A-buffer [2].

In a similar fashion, texture effects can be produced as a modulation of a picture, The modulation is represented in a cel which is then placed over the picture to be modulated. Here we typically rely on locally varying the translucent element of the beta model, the medium, while having no reflective component.

Another useful property of the beta model is that it can be used to implement general filters to

* See page C-465 for Figures 5 and 6.

P. Willis et al. / Rendering NURB Regions C-43

supply a range of image-modification effects. This allows “foggy” cels, or cels which are partly clear and partly translucent, to be used to enhance the basic scene without directly changing its modelled representation. This, together with the independent lighting model, makes for greater re-use of cels. Such filters are often more usefully implemented functionally, rather than as highly elaborate NURB sets, but again the beta model ensures that these are easily incorporated in the renderer.

It is acceptable to combine a stack of cels to produce a single resultant cel, which is then re-used for the creation of a more complex scene. However, the renderer would be typically used with many cels invariant across a sequence of frames, when it would be better to convert each changing cel separately, and then combine this with previously created foreground and background cels. Again, the beta model guarantees this is always possible.

What this means in practice is that, for example, an animator can construct a complex background from a variety of cels and, when satisfied with the result, the cels can be combined to produce a single cel with the same visual effect, with lower storage costs, and reduced computation at later stages. It is also possible to hold cels in scan-converted internal form (i.e. before the final RGB conversion) and to re-use these in conjunction with new cels.

6 Advantages And Possible Developments

Our method for NURB region rendering offers a number of advantages. Firstly, since all stages prior to the final output are held in floating point form (high precision integer could also be used), each element can be freely transformed without regard to pixel alignment problems. A transformed NURB remains a NURB and so the converter produces the correct picture regardless. This not only aids the modelling process, it also ensures that elements can be recycled, without the limitations imposed by other systems. There are neither image degradation nor quantisation artefacts introduced.

In contrast with incremental methods, any NURB can be treated in a uniform manner. In particular, incremental conversion of a cubic will sometimes result in the need for new computations for the same curve, part way through the conversion, as extra intercepts are dealt with. This is unnecessarily demanding to monitor and it is space and time consuming to preprocess the cubics into pieces which are single-valued. Our method computes all of the intercepts explicitly, avoiding this problem entirely.

A bonus of working in floating point is the full accuracy of the computation, avoiding special cases and the loss of otherwise useful data. In particular horizontal anti-aliasing is available at the low additional cost of computing partial intensities from information already present. As a result, supersampling is only needed in the vertical direction, giving an anti-aliasing cost which is linear, rather than quadratic, with resolution.

Clipping against a rectangular region, such as the screen, has zero cost for the upper and lower edges and almost no additional cost for the left and right edges: again all of the information needed is already present and it is largely a matter of clamping extents.

All of the above result from our current implementation. A number of interesting enhancements also exist. We have the option of incorporating in a simple manner the CPG operations to combine 2D shapes, because this can be done at the scanline level. For some applications there would be a substantial gain simply because it will work with any class of shape which can be scan-converted. We lose little of the simplicity in doing this because such operations can be embedded in the input region stream, for example in RPN form, and would drop out easily from a regular walk of the corresponding and conventional tree-structured model.

The potential to include colouring as a function, rather than as a fixed property, ensures that quite subtle visual effects can be achieved easily, for example through additional filter cels. Our colour model

C-44

does not require filters to be scalar operations, again extending the range of visual effects. We have demonstrated these advantages with pixel images, to test the beta colour model, and that model is already part of our NURB system in anticipation of this addition.

Our approach will also work for more general image composition applications. It has the advantage over the A-buffer approach [2] in not requiring pixel-by-pixel data for matting images together. It also means that areas to be matted can be bounded by curves rather than polygons and, where moving mattes are required, these curves can be animated without needing to ensure good polygonisation.

Proper boundary definition is especially pertinent in animation sequences containing a slowly moving NURB. Polygonisation of the NURB is not necessarily constant between frames, which leads to an aliasing artefact known as “creeping”. The major strength of our method is that it does not use any sort of polygonal approximation and so automatically avoids this problem.

P. Willis et al. / Rendering NURB Regions

7 Acknowledgements

The authors of this paper wish to thank the Science and Engineering Research Council for supplying the Research Studentship which supports R.J.Oddy and for the recent grant which allows this research to continue.

We are also grateful to Peter Harding for permission to base the colour pictures showing our lighting technique on his artwork.

References

[1] Carl De Boor. On Calculating with B-splines. Journal of Approximation Theory, (6):50-62, 1972.

[2] Loren Carpenter. The A-Buffer, an Antialiased Bidden Surface Method. Computer Graphics (SIGGRAPH 84), 18(3):103-108, July 1984.

[3] B.K. Choi, W.S. Yoo, and C.S. Lee. Matrix representation for NURB curves and surfaces. Computer-Aided Design, 22(4):235-240, May 1990. Butterworth & Co.

82, pages 263-272, 1982. Springer-Verlag.

10(2):121-127, June 1991. North-Holland.

Graphics (SIGGRAPH 81), 15(3):253-262, August 1981.

puter Graphics Forum, 6(2):133-140, May 1987. North-Holland.

[4] A. Hearn. REDUCE - A Case Study in Algebra System Development. Proceedings of EUROCAM

[5] Robert J . Oddy and Philip J . Willis. A Physically Based Colour Model. Computer Graphics Forum,

[6] Bruce A. Wallace. Merging and Transformation of raster images for cartoon animation. Computer

[7] Philip Willis and Geoff Watters. Scan Converting Extruded Lines at Ultra High Definition. Com-