Paint-on-Glass AnimationThe Fellowship of Digital Paint and Artisanal Control

Tom Van Laerhoven, Fabian Di Fiore, William Van Haevre, Frank Van ReethHasselt University – tUL – IBBT

Expertise Centre for Digital MediaWetenschapspark, 2

BE-3590 Diepenbeek, Belgium{tom.vanlaerhoven, fabian.difiore, william.vanhaevre, frank.vanreeth}@uhasselt.be

AbstractIn this paper we deal with paint-on-glass ani-mation, which is a technique for making ani-mated films by pushing slow-drying paints fromframe to frame directly under the camera. Asartwork is continuously destroyed upon creatingnew frames, the animator is not able to rehearseor refine the animation afterwards. Furthermore,due to impracticable issues like how to stack uplayers containing wet paint or how to overlaymasks on the wet medium, one has to take bothcreative as technical decisions for each shot.

Our approach consists of an interactive paintsetup that physically simulates paint media. To-gether with a set of digital tools the artist is re-lieved from the difficult task of sustaining a con-stant frame-to-frame coherence while animatingand is given the possibility to modify or undoearlier paint modifications. Regarding the inter-action part, the artist stays in full control by em-ploying a tangible interface. This allows artiststo push pigment with their fingers, use special-purpose digital brushes, or even to employ reallife tools like paper tissue.

Feedback from artists confirms the real-lifebehaviour of stretching and smudging the paintas well as interacting with the setup, resultingin a natural and reality-based methodology.

Keywords: animation, 2D animation, painterlyanimation, paint system, paint-on-glass

1 Introduction

In spite of its name, paint-on-glass animationis best defined as a technique for making an-imated films by pushing around some kind of(wet) medium directly under the camera andrecording it frame-by-frame (Figure 1). It iscategorised as an “under-the-camera-technique”as it is nearly always undertaken by an indi-vidual artist or animator rather than by a pro-duction studio using factory-like processes. Asthe medium is pushed around directly under thecamera and recorded frame-by-frame, each im-age seems to merge from the previous one andmelt into the next, resulting in movement thatcan be very fluid and organic – a continual pro-cess of metamorphosis.

A well-known practitioner of the paint-on-glass animation technique is Russian animatorAleksander Petrov. His most famous work is theaward winning short film “The Old Man and theSea” which took two and a half years to com-plete [1] illustrating the complexity of the tech-nique.

Motivation. As the medium is pushed aroundfrom frame to frame, artwork is continuouslydestroyed as new artwork is created. Withoutthe ability to rehearse and refine the animationand without the disposal of layering and mask-ing tools, the animator must plough on regard-

Figure 1: An artist at work using an “under-the-camera-technique” with a combina-tion of sand and ink c⃝ Gerald Conn.

less incorporating any errors into the sequence.On the one hand this stimulates spontaneity

and delivers work that is very fresh and dis-tinctive as it celebrates the method of its mak-ing. For example, characters may move fromplace to place not by walking, but by beingsmudged away to re-form out of the backgroundat the required location. On the other hand,this particular form of step-ahead animation is apainstaking process which demands much self-discipline, endurance and concentration as theresulting frames are never a series of clean im-ages of the kind produced by other animationtechniques, but also contain a record of theirmaking due to the pushing process.

Contribution. In this paper we present our ap-proach to facilitate paint-on-glass animation. Inclose collaboration with artists and animatorswe developed a set of digital tools that do en-able them to achieve the fine quality of a fin-ished painting for each frame, while preservingall artisan control.Primarily, our system features:

∙ a real-time free-form paint tool to phys-ically simulate the complex behaviour ofdifferent paint media (gouache, water-colour, impasto, pastel);

∙ painting/animating by pushing the draw-ing media around resulting in a stretch andsmear effect;

∙ a tangible interface consisting of a multi-touch screen allowing the use of one’s fin-gers as well as physical media includingbrushes and tissues;

∙ several tools that are absent in the tradi-tional method due to the wet nature of thepainting media such as working with layersand masks;

∙ the availability of animation tools likeonion skinning and flipbook animation;

∙ digital tools to control the drying time, cor-rect mistakes, remove pigment and/or wa-ter as desired, and the ability to save, loador reuse (intermediate) results.

Paper Organisation. Section 2 surveys workwe consider related to ours. Section 3 describesour approach, starting from the employed paint-ing setup to an overview of the digital paint-on-glass animation process. Section 4 providesclarifying examples in which we also discussour results. Section 5 is our conclusions sectionwhich also sets the context for future work.

2 Related Work

We start this section by giving a brief overviewof traditional paint-on-glass animation. Then,we elaborate on digital paint solutions that weconsider related to ours.

2.1 Traditional Paint-on-Glass Animation

What we give here is an account as it might befor a practiced artist in a home studio. We sepa-rately discuss the painting process (i.e. paintinga single frame) and the animation process (i.e.making a series of drawings).

Painting. The painting process itself resem-bles painting on a canvas as the initial paintis added with brushes or other tools includingsponges, cotton buds, small sticks, strong tissueor even fingers.

As the paint medium will be reused fromframe to frame, the artist is constrained to workwith slow drying paint media like oil paint inorder to keep working with the medium across

several hours or even days. This causes alsomany impracticable issues like how to stack uplayers of wet paint or how to overlay masks onthe wet medium.

Animating. For classic 2D animation the an-imator draws or sets up objects one frame at atime in sequential order until the sequence iscomplete, also referred to as step-ahead anima-tion. In this way there is one physical drawingor image per frame. In the case of paint-on-glassanimation the artist is working directly under-the-camera: the artist has to record the currentframe before proceeding to the next one as cur-rent artwork is continuously destroyed for creat-ing each new frame. When a sequence has beenfilmed, there is no raw material left besides thefinal film. So, while animating one cannot relyon techniques like onion skinning or flipbookanimation as previous artwork is not availableanymore. Hence, there is also no artwork to turnto when modifying the film afterwards.

2.2 Digital Paint Systems

In this section we will discuss digital paint solu-tions that we consider related to ours.

Painting. Looking at the majority of pub-lished paint systems [2, 3, 4, 5], virtual brushesare by default operated by some kind of digitalstylus in combination with a digital tablet. Al-though popular and basically intuitive, this wayof working is nothing more than an approxima-tion of ‘the real thing’. Issues such as parallax,the lack of tufts and absence of physical toolsneed to be solved first.

Other paint systems explicitly target novel in-terfaces to bridge the gap between physical anddigital painting. Vandoren et al. developed Intu-Paint, which is a tangible interface for a digitalpaint easel, using an interactive surface and elec-tronic brushes with a tuft of bristles [6]. Thisapproach provides natural interaction and en-ables detailed tracking of specific brush strokes.Additional tangible and finger-based input tech-niques allow for specific paint operations or ef-fects. Okaichi et al. propose an interactive oilpaint system using a painting knife [7] that iscontrolled using a haptic feedback device. Mi-crosoft Surface [8] is based on the IR image

capture of an IR illuminated diffuser screen un-derneath a transparent surface allowing naturalfinger-based interaction. Presence of real ob-jects (e.g., brushes) can be detected, but onlya blurred image of the brush is generated. Re-cently, Vandoren et al. presented a novel inter-active canvas FluidPaint allowing the use of realwet brushes [9].

Animating. Regarding animation, only smallinitiatives have been taken so far. Van Laer-hoven et al. introduce the concept of brushesand pigments enhanced with behavioural in-telligence that allow the user to enrich andanimate interactively created images by semi-autonomously embedding procedural animationinto them [10]. Some commercial digital art ap-plications also claim to target at creating ani-mations ([11, 12, 13]), however, the animationpart mainly is limited to the availability of akeyframe editor and the use of animated brushesto draw frames along a path.

3 Approach

In this section we give an overview of the mainparts of our approach. We first describe the em-ployed digital paint setup. Next, similar to thetraditional process (Section 2.1) we will high-light both the painting and animation process.

3.1 Digital Paint Setup

For the simulation part, we make use of an inter-active paint system that adopts physically-basedalgorithms to simulate the complex interactionof a layered virtual canvas with different paintmedia, including watercolour, gouache and im-pasto, but also dry media like pencil or pastel[14]. This paint system encapsulates the dryingprocess, paint diffusion as well as different pig-ment attributes (Figure 2(a)). Furthermore, it re-lies on Navier-Stokes for defining the velocitiesof the fluid body and on the specialised Kubelka-Munk colour mixing scheme for rendering paintlayers. These features are vital elements to ob-tain the much sought after ‘natural look’ in dig-ital artwork. Every step of the simulation makesfull use of programmable graphics hardware toachieve interactive simulation rates.

(a) (b)

Figure 2: a) Physically-based paint simulation. b) The setup includes an interactive surface on whichthe user can manipulate paint with multiple fingers at once, or using special-purpose tools.

Another important component of the systemrelates to its virtual brush design that relieson constrained energy optimisation and on ananisotropic friction model to deform the brushtuft and adequately translate the user’s input(e.g., originating from a tablet interface) intorealistic and predictable strokes. This designalso allows for bidirectional pigment transfer be-tween the canvas and the selected drawing tool.

Regarding the interaction part, we chose touse a tangible interface to enable users to in-tuitively manipulate the drawing media (Figure2(b)). The setup consists of an interactive mul-titouch surface based on the principle of frus-trated total internal reflection (FTIR) [6]. Thecontact surface of each object on the paint ta-ble is tracked by a 0.3 megapixel video camera,equipped with an infrared band-pass filter, afterwhich the footprint of the objects touching thesurface is transferred to the paint simulation forfurther processing and rendering. In contrast tothe original paint table, our setup is able to trackmultiple objects on the surface simultaneously.This way, artists can push pigment with multi-ple fingers at once, use special-purpose digitalbrushes with tufts, or even employ paper tissue.

A major advantage of this setup is that it pro-vides coinciding action and perception spaces,resulting in a natural, reality-based methodol-ogy. In addition, it encourages multiple usersto work collaboratively on the same artwork.

3.2 Painting

Starting from a blank canvas, the artist createsor alters an image by adding some kind of paintmedium (e.g., gouache, impasto, watercolour,

pastel) using fingers or several available tools(brushes, pencils, spatulas, sponges, fingers). Abackground image can be depicted as a refer-ence and parts of the drawing canvas can interac-tively be masked to prevent paint from reachingor settling down on the masked parts. Drawingscan also be separated into layers allowing forrobust manipulations of the frame content suchas depth ordering and hiding/revealing objects.Furthermore, users can interactively control as-pects like drying time (including the possibilityto keep paint permanent wet or pause the dry-ing), undo mistakes, remove pigment and wateras desired and have the ability to save, load orreuse intermediate results.

Watercolour images made with the systemcontain the typical effects that can be recognisedin images produced with real thin paint, like thedark-edge effect, watercolour glazing, wet-on-wet painting and the use of different pigmenttypes.

3.3 Animating

While animating, each subsequent frame in theanimation is created by pushing the drawingmedium of the artwork in the current frame to-wards its desired position for the next frame.During interaction, movement on the multitouchsurface is converted to a vector field based on thecurrent velocity of the brush/finger (see Figure3(a)), which is then applied to the paint (bothpigment and water) by means of an advectionstep −(v⃗ ⋅ ∇)p, where v⃗ denotes the 2D veloc-ity vector field and p the amount of paint at acertain position on the canvas. As in our paintsystem mass is contained in the grid cells of the

(a) (b)

Figure 3: a) Pushing paint results in a velocity field that locally manipulates the exchange of pigmentquantities. b) Moving water to a right neighbouring cell. The dark areas represent thevolume of displaced water.

digital canvas, the advection scheme boils downto individual cells exchanging content with theirneighbours (Figure 3(b)) [14]. For each cell, wemeasure the volume of paint that is exchangedwith all neighbouring cells as follows.

When calculating the amount of mass thatflows from the centre cell to its right neighbourwith velocities v⃗center and v⃗right respectively, asshown in Figure 3(b) (left), any point along thisborder has velocity:

v⃗average =v⃗center + v⃗right

2(1)

Within a given time step Δt, a distance Δx =(v⃗average)xΔt is travelled. In case we are calcu-lating the movement of water, the area coveredby the volume of displaced water is given by thecoloured area in Figure 3(b) (left), and equalsΔx(cellheight). The total volume of displacedwater then is ΔV = Δx(cellheight)wi,j (Fig-ure 3(b) (right)). The change in water quantityis given by the following equation:

Δw =ΔV

(cellwidth× cellheight)(2)

This procedure is repeated to calculate the ex-changed water quantities with the three remain-ing neighbours. We add up the results and divideit by four, because each neighbour contributesexactly one quarter to the total flux. The sameapproach can be followed to work out the move-ment of pigment.

Figure 4 depicts some stills of a user pushingpaint (in this case impasto) with two fingers onan interactive multitouch surface.

In order to assure more control over the waythe artist is obliged to make things move, he/shecan rely on digital animation tools to make ma-nipulative operations more predictable includinga keyframe editor, onion skinning and a flipbooktool (Figure 5). The onion skinning tool aids theartist to achieve good frame-to-frame coherence,resulting in smoother frame transitions by show-ing trace layers of previously painted frameswhereas the digital flipbook tool provides im-mediate feedback on the status of the animationbeing created.

4 Results

Figure 7 depicts some snapshots of a paint-on-glass animation of a bird created with impastopaint illustrating the stretch and smear effects.Even for the start frame, the artist chose togradually refine a basic image (painted using abrush) using his fingers only (see Figure 6). Theentire animation consists of 31 frames of whicheach is a modified version of its predecessor byonly pushing paint from frame to frame.

Discussion. The target audience of our setupis skilled artists as the paint-on-glass techniqueitself requires some experience in the field ofpainting. The platform itself is targeted at bothtraditional and digital artists as the interface issimilar to the traditional environment.

Therefore traditional as well as digitalpainters were involved in an informal user testevaluating the setup as well as creating the ex-amples. Only basic instructions were given (i.e.

Figure 4: Some stills depicting a user pushing paint (impasto) with two fingers on an interactive mul-titouch surface.

Figure 5: When drawing a new frame the animator can check temporal coherence by flipping throughall previous frames.

explaining the user interface). After that, the testparticipants were allowed to work on their sub-jects of choice without any training.

All participants worked between two andthree hours to complete their animations (43frames on average) which is significantly fasterthan the traditional way of working (i.e. around20 frames a day, on average). Although the testsession was rather short we can say that all par-ticipants were very positive. They found theuser interface familiar and praised the real-lifebehaviour of manipulating the paint as well asinteracting with the setup, resulting in a naturaland reality-based methodology.

5 Conclusions

In this paper we dealt with making animatedfilms by pushing around wet media directly un-der the camera, known as paint-on-glass ani-mation. Without the ability to rehearse and re-fine the animation as opposed to classic 2D an-imation, this particular form of animation is apainstaking process for which animators need topersevere regardless incorporating any errors orunwanted smudging into the sequence.

Our approach tackles the issues inherent topaint-on-glass animation by offering the ani-mator an interactive paint system that adoptsphysically-based algorithms to simulate thecomplex interaction of a virtual canvas with dif-ferent paint media, and a set of digital tools thatdo enable him/her to achieve the fine quality of afinished painting for each frame while preserv-ing all artisan control. Regarding the interactionpart, a tangible interface was chosen to enableusers to intuitively manipulate the drawing me-dia using their own hands.

Future Work. The emphasis in this paper ison paint-on-glass animation. However, the sameapproach could be applied to any “under-the-camera-technique” such as animating using finesand, sugar or coffee grind. For the simulationpart this requires a bidirectional transfer of par-ticles; particles need to be deposited or scrapedoff again. Regarding the interaction process wehave to consider how to sprinkle dry media.

Furthermore, we strongly believe in investi-gating new interaction techniques which are notpresent at all in traditional animation such as us-ing simple hand gestures and finger taps to eas-ily navigate between frames or assets. Inspired

by ShapeTouch [15] another option is to exploitthe shape of hands to work on the interactivesurface just like when dealing with real physi-cal objects (e.g., pinning, peeling or flicking).

Acknowledgements

We gratefully express our gratitude to the Eu-ropean Fund for Regional Development and theFlemish Government, and to Xemi Morales andKarel Robert for their artistic contribution.

References

[1] Aleksandr Petrov. World WideWeb, http://en.wikipedia.org/wiki/Aleksandr_Petrov_(animator).

[2] Suguru Saito and Masayuki Nakajima. 3Dphysics-based brush model for painting.In Proceedings of SIGGRAPH Confer-ence abstracts and applications, page 226.ACM, 1999.

[3] W. V. Baxter, V. Scheib, M. C. Lin, andD. Manocha. DAB: Interactive hapticpainting with 3D virtual brushes. In Pro-ceedings of SIGGRAPH, pages 461–468.ACM, 2001.

[4] Nelson Chu and Chiew-Lan Tai. Real-timepainting with an expressive virtual Chinesebrush. IEEE Computer Graphics and Ap-plications, 24(5):76–85, 2004.

[5] Tom Van Laerhoven and Frank Van Reeth.Brush up your painting skills: Realisticbrush design for interactive painting ap-plications. The Visual Computer. Com-puter Graphics International (CGI2007),23:763–771, September 2007.

[6] Peter Vandoren, Tom Van Laerhoven, LucClaesen, Johannes Taelman, Chris Ray-maekers, and Frank Van Reeth. Intu-Paint: Bridging the gap between physi-cal and digital painting. In Proceedingsof TABLETOP 2008, IEEE InternationalWorkshop on Horizontal Interactive Hu-man Computer System, pages 71–78, Oc-tober 2008.

[7] N. Okaichi, H. Johan, T. Imagire, andT. Nishita. A virtual painting knife. TheVisual Computer. Computer Graphics In-ternational (CGI2008), pages 753–763,2008.

[8] Microsoft. Microsoft surface. World WideWeb, http://www.microsoft.com/surface/.

[9] Peter Vandoren, Luc Claesen, TomVan Laerhoven, Johannes Taelman, ChrisRaymaekers, Eddy Flerackers, and FrankVan Reeth. FluidPaint: an interactivedigital painting system using real wetbrushes. In Proceedings of ITS 2009,ACM ACM International Conference onInteractive Tabletops and Surfaces, pages53–56, November 2009.

[10] Tom Van Laerhoven, Fabian Di Fiore, andFrank Van Reeth. Hand-painted animationwith intelligent brushes. Computer Anima-tion and Virtual Worlds. Computer Anima-tion and Social Agents (CASA2008), pages365–374, 2008.

[11] Corel Painter X. World Wide Web, http://www.corel.com/.

[12] Project Dogwaffle. World Wide Web,http://www.thebest3d.com/dogwaffle/.

[13] Adobe PhotoShop. World Wide Web,http://www.photoshop.com/.

[14] Tom Van Laerhoven and Frank Van Reeth.Real-time simulation of watery paint. InJournal of Computer Animation and Vir-tual Worlds (Special Issue CASA 2005),volume 16:3–4, pages 429–439. J. Wiley& Sons, Ltd., October 2005.

[15] Xiang Cao, Andrew Wilson, Ravin Bal-akrishnan, Ken Hinckley, and Scott Hud-son. ShapeTouch: Leveraging contactshape on interactive surfaces. In Proceed-ings of TABLETOP 2008, IEEE Interna-tional Workshop on Horizontal InteractiveHuman Computer System, pages 139–146,October 2008.

Figure 6: The making of an entire new frame (last image) from scratch by gradually refining a basicimage (first image) using fingers only.

Figure 7: Some snapshots of a paint-on-glass animation of a bird created with impasto paint. Eachimage is a modified version of its predecessor by only pushing paint from frame to frame.