QUARTZ COMPOSERin the art and motion design of futurismo zugakousaku

ian grant

november 2005

e l l i n g t o n . t v u . a c . u k / m a /

TABLE OF CONTENTS

INTRODUCTION 4

Quartz Composer 4

PixelShox: Brief History of Quartz Composer 6

Comparison with MaxMSP and Jitter 7

Comparison with Director / Shockwave 3D? 8

THE POETICS OF QUARTZ COMPOSER 10

Time-Based Imaging 11

The Quartz Composer Audience 13

Futurismo Zugakousaku’s “Who Are You?” 16

(a) Description 16

(b) Hybridity 17

(c) Dynamics 17

(d) Process 18

Summary 26

BIBLIOGRAPHY 28

Texts 28

Web 28

Images and Sources 28

quartz composer in the art and motion design of futurismo zugakousaku 3

QUARTZ COMPOSERin the art and design of futurismo zugakousaku

Ian GrantNovember 2005

INTRODUCTION

Screen-shots from “Hotel Margate” by futurismo zugakousaku 1

This essay will examine the role that an emerging technology, Apple’s Quartz

Composer 2 , is playing in the development of interactive computer arts. I will offer

interpretations of select examples of work by Japanese web motion/graphics art-

ist Zugakousaku (found at www.zugakousaku.com).

Q U A R T Z C O M P O S E R

Quartz Composer (Quartz Composer) is a technology that exploits the graphics

processing unit (GPU) of the host Apple Mac and integrates:

★ graphic standards as defined by Open-GL and Open-GL Shader Language, ena-

bling real-time video, 3D and image processing,

★ with Apple’s Quicktime technology for image handling and dissemination

quartz composer in the art and motion design of futurismo zugakousaku 4

1 www.zugakousaku.com/index.cgi?hotel&&en

2 http://developer.apple.com/graphicsimaging/quartz/quartzcomposer.html

★ data aggregation standards defined by RSS and a logic / control language de-

fined by javascript,

★ midi and audio input processing allowing sophisticated image and composition

control by external sources and sensors,

★ with xCode, Apples Integrated Developing Environment (IDE) - allowing Quartz

Composer compositions to be embedded in a wider coding environment for more

complex control and interoperability.

Quartz Composer is an opportunity for interactive motion graphics artists to ex-

plore an interesting early foray into ‘resolution independent’ imaging.

‘Quartz’ is Apple’s brand name for a suite of technologies internally known as

‘core graphics’. It is based on a PDF graphics/imaging model and extends from

still image and text into 3D graphics with Open-GL and real-time video processing

with a collection of techniques and APIs called ‘core video’.

In addition to real-time interactions and integration with the image processing li-

braries Apple calls ‘core image’ 3 , Quartz Composer, according to Apple, can

“make data-driven visual effects, and even perform live performance anima-

tions.” (Apple Developer Connection, 2006).

In an earlier iteration, Quartz Composer, as PixelShox had a vibrant life fa-

voured by V-Jays and live-motion graphics artists.

5 quartz composer in the art and motion design of futurismo zugakousaku

3 resolution-independent 2d graphics based on PDF technologies and 3D graphics based on hardware accelerated open-GL

P I X E L S H O X : B R I E F H I S T O R Y O F Q U A R T Z C O M P O S E R

Quartz Composer is a software that elegantly combines many core apple tech-

nologies and is based on “PixelShox Studio” written by Pierre-Olivier Latour, circa

2003. Pierre-Olivier Latour now works for Apple and Quartz Composer is a small,

but significant, part of a wider coding architecture 4.

An early vision for PixelShox was for artistic use “professionally in any project

where interactive visuals are required: V-Jaying, multimedia installations, presen-

tations, [or] concerts.” (Latour, 2003). Quartz Composer is significantly pruned of

functionality when compared with PixelShox. Although it was never completed, it

is likely that the past functionality of PixelShox may be a rough roadmap of where

Quartz Composer could head. One repeated feature request is support for arbi-

trary 3D objects.

6 quartz composer in the art and motion design of futurismo zugakousaku

4 Quartz Composer compositions integrate nicely into the Cocoa programming environment, utilising ‘bindings’ to make it easy to write applications that connect GUIs to visual compositions. Also Quartz Composer compositions can be turned into “Image Units’ and made available as video processing plug-ins for compatible applications, like

Avid’s Soft-image, Apple’s Final Cut Pro and Motion.

C O M P A R I S O N W I T H M A X M S P A N D J I T T E R

Quartz Composer benefits from an interface and approach that is more contem-

porary and has significant ease of use compared with the closest comparable

software. Playing with real-time open-GL space is elusive for artists as the pro-

gramming expertise necessary is relatively advanced. Although Quartz Composer

and MaxMSP/Jitter share a visual programming environment, MaxMSP/Jitter is

more complex and has rich feature laden environment. Midi and audio synthesis

and output and not possible in Quartz Composer. The following chart outlines

simple comparisons between the packages 5.

Q U A R T Z C O M P O S E R M A X / M S P J I T T E R

environment visual programming inter-face - patch based pro-gramming

same

output native quicktime mov

qtz - plays in quicktime or as native mac screen-saver

plays full screen in authoring environment

can be used and linked to a GUI in Apples IDE

no audio out

significant midi and audio synthesis

GUI building integrated into package.

input quicktime native image

video,

audio,

midi

file-system

internet

rss

images

video,

midi,

audio,

serial data,

3D files etc.

7 quartz composer in the art and motion design of futurismo zugakousaku

5 Source of speed / performance info: http://eskatonia.net/qcblog/quartz-composer-for-vjs/

Q U A R T Z C O M P O S E R M A X / M S P J I T T E R

video processing speed faster (direct to GPU) slower (CPU)coding javascript (subset)

Open-GL Shader Lan-guage (subset) - allows kernel programming

modular logic patches

javascript (subset)

modular logic patches

GUI logic patches

cost free with Mac OS 10.4 $850.00 funct ional i ty needs to be added with 3rd party add-ons.

platform platform specific (Mac OS 10.4)

cross platform

authoring total open authoring - share solutions

closed authoring - com-piled stand-alone

strengths cost, ease of use

speed / performance

emerging online commu-nity

audio / midi processing

synthesis

mature online community

weaknesses single platform,

limited 3d model support (possible with 3rd party hack)

no audio out

cost, complexity

speed / performance

C O M P A R I S O N W I T H D I R E C T O R / S H O C K W A V E 3 D

As an example to indicate the revolutionary shift between generations of soft-

ware, try creating an open-GL space with a video textured quad, spinning and re-

sizing in response to the amplitude of an incoming audio signal in Director MX

2004. Two extras later (with a possible expense) and two or so pages of 3D lingo

you may have a result that is running after compilation and possibly without

8 quartz composer in the art and motion design of futurismo zugakousaku

hardware acceleration. Quartz Composer could do the preceding task with built in

functionality by dragging four patches into the editor space, zero code and take

approximately 30 seconds to author.

Director and Shockwave 3D has more sophisticated 3D model import capabilities.

You can only import 3DS (3D Studio Max) models into Quartz Composer with a

third party hack 6.

9 quartz composer in the art and motion design of futurismo zugakousaku

6 Reverse engineering Quartz Composer - see http://www.clockskew.com/blog/Import 3DS files and textures into QC - see http://blog.livedoor.jp/quartzcomposer/

THE POETICS OF QUARTZ COMPOSER

This essay will now focus on an aesthetic and technical dissection of Quartz

Composer as used by Futurismo Zugakousaku. The work could be categorised as

‘sketches’ in motion graphics, time-based explorations inspired by a strong illus-

tration led graphic art, sometimes in the form of screen-savers or in the form

short cyclical, repeating vignettes.

“Approach to motion graphics of automatism.

I made all these samples at the time for less than one hour.

Quartz Composer can make a impromptu. An image is lightly

created such as poetry and music. Designer can make a mo-

tion graphics by this tool in real time.

I want to make a work like a new surrealist of the new genera-

tion.”

(Zugakousaku, 2005; translated by google)

It is worthwhile to look at the aesthetic nature of the Quartz Composer technology

and investigate in what ways the logical structure inherent in the software may

effect the final resonance of art work made with it. First, a quick discussion of

Time in Quartz Composer, then a consideration of the context of reception - the

Audience and User base for artefacts made with Quartz Composer.

quartz composer in the art and motion design of futurismo zugakousaku 10

T I M E - B A S E D I M A G I N G

Quartz Composer stands in an interesting, complex relation to Time. The anima-

tion only appears to loop be-

cause the X position is trans-

lated but ends in a position that

overlaps exactly with the start

position. The X position value

then loops back to it’s original

value not the time of the com-

position. Composition time is

linear and infinite - starting from

0 to infinity. “Local” or “Patch

Time” similarly starts when the

patch is activated and contin-

ues, but it can be controlled:

started, stopped and reset.

Some patches have an option

to change their timebase:

Illustrated here are the “3D Transformation” and “Image With Movie” patches

An External timebase also exists. This can synchronise patch playback to, for ex-

ample, the “Interpolation” patch, counts and the Midi Clock patch. According to

Apple’s documentation Quartz Composer compositions “don't have defined dura-

tion or frame rates” (Apple Computers, 2005): logical for a media that is designed

quartz composer in the art and motion design of futurismo zugakousaku 11

to operate in real time and perform to the best capabilities of the video hardware

in the host machine.

12 quartz composer in the art and motion design of futurismo zugakousaku

"Parent" (default): the time is simply the same as the one passed to the par-

ent macro patch - if all its parent patches have this setting, then the time will

be the same as the global composition rendering time,

"Local": the time passes at the same speed as the parent macro patch

time, but is reset to 0 whenever the patch starts being used by the renderer

patches it is connected to (i.e. when the renderer patches Enable inputs go

from False to True),

"External": this adds a new "Patch Time" input to the patch through which

the time can be manually specified.

Timebase Controls in Quartz Composer (Apple Computers 2005b)

T H E Q U A R T Z C O M P O S E R A U D I E N C E

Futurismo Zugakousaku is primarily known through a web presence. There fol-

lows a brief outline of the current audience / user base for Quartz Composer in-

spired work.

★ Currently, the primary audience is developers interested in real-time performance

and the real-time processing of computer graphics.

★ A ‘popular’ context is emerging in the form of screen-savers and dissemination of

compositions / movies through art and design web sites. The screen saver as po-

tential art space has been explored before and will not be discussed in the pre-

sent essay.

★ Site specific - an emerging prominent use of Quartz Composer in the V-Jaying

and clubbing communities. Quartz Composer plays a role in the real-time visuali-

sation and performance systems. The full screen context of Quartz Composer -

playing within the authoring context provides an ideal source for projected images

and motion graphics.

★ Software - Quartz Composer compositions are mediated by other software.

Quartz Composer plays a role in the development of ‘image units’ (image and

video processing plug-ins). Users/audience witness the image / video processing

executed by Quartz Composer vicariously through ‘host applications’ like Avid’s

suite, Apple’s Motion or Final Cut Pro. Here the composition or image processing

is dominated by an external creative workflow. In one sense, the work of Futur-

ismo Zugakousaku is evidence of an exploratory practice that emerges during the

quartz composer in the art and motion design of futurismo zugakousaku 13

development of commercial video processing plug-ins for the Avid suite of video

editing applications 7.

14 quartz composer in the art and motion design of futurismo zugakousaku

7 See the Avid section of http://www.zugakousaku.com/index.cgi?quartz&avid&en

In analysing examples of work disseminated by Futurismo Zugakousaku, I want

to evaluate how Quartz and Quartz Composer technology has facilitated the crea-

tion of work which would probably not be possible in other media. I will use the

following broad headings to guide a visual and technical critique:

★ Description of the Motion / Visual Design: Communication of Concept and Con-

text

★ Hybridity: How has the possibility of combining different media created new kinds

of art?

★ Dynamics: Time and event management in digital artworks - the model of time in

Quartz Composer is interesting and complex.

★ Process: How are the effects achieved?

In the analysis below of “Who Are You?” I will go into a little more detail about the

general architecture of a Quartz Composer composition. In future examples, I will

focus on notable and remarkable elements.

quartz composer in the art and motion design of futurismo zugakousaku 15

F U T U R I S M O Z U G A K O U S A K U ’ S “ W H O A R E Y O U ? ” 8

( A ) D E S C R I P T I O N

A simple animation parades a

cloned figure from left to right in an

never-ending loop. The figure, re-

sembling an Edwardian swimmer,

recedes row upon row to a blur in

the distance. On every passing

loop, the movement smoothly stops

and a slogan appears in English and Japanese. Each slogan describes a role or

occupation that is selected at random from a large array - it tends not to repeat.

quartz composer in the art and motion design of futurismo zugakousaku 16

8 Direct Link to Online Demo Movie: http://www.zugakousaku.com/quicktime.cgi?web_data/quartz/samples/

( B ) H Y B R I D I T Y

The juxtaposition of random text and image is a technique Zugakousaku uses

frequently - and harkens to the Surrealist love of random collisions of words cre-

ating new and surprising associations and meaning. The image doesn’t change

so the implication is that the language of role masks homogeneity - that below the

surface of language we are all the same With just the use of a single image, rep-

licated and scaled into the distance, Zugakousaku achieves a minimalism and

clarity of design that I find engaging. Other work, e.g. “The 20th Century Voyage”

uses RSS 9 and blends together motion, data, text and image in real time - such

hybridity is characteristic of work made with Quartz Composer.

( C ) D Y N A M I C S

The quality of movement, impossible to capture with a sequence of still images, is

hypnotic: smooth, rhythmic, organic.

Screen-shots from “Who Am I?” by futurismo zugakousaku

Each row of men moves at a slightly different pace, achieve a ‘parallax effect’ that

gives an excellent illusion of depth and receding distance. In this example the vir-

tual 3D space of Open-GL is not used, rather an illusion of depth in 2D - some-

times know as two-and-a-half D. The depth of field blur adds to the illusion of

depth.

17 quartz composer in the art and motion design of futurismo zugakousaku

9 RSS - Really Simple Syndication or Rich Site Summary: An XML file contain basic data about stories on web-sites

( D ) P R O C E S S

Quartz Composer has a single

frame editing window. Patches are

colour coded (pink means content

will be rendered ) and if they have

square corners they are hierarchi-

cal. If you double click on this

patch, you can edit any sub-

patches ‘inside’ it. For more information on basic editing in Quartz Composer

please see the documentation at Apple’s developers web site 10.

quartz composer in the art and motion design of futurismo zugakousaku 18

10 Working with Quartz Composer http://developer.apple.com/graphicsimaging/quartz/quartzcomposer.html QC Programming Guide http://developer.apple.com/documentation/GraphicsImaging/Conceptual/QuartzComposer/

A M A C R O P A T C H

this is text in the box

Each patch in Quartz Composer

has an information panel providing

contextual help and areas to set

input values. Here the 3D Trans-

form patch simply translates

(moves) its whole rendered con-

tents down a bit. (-0.45 on the Y

axis)

Double clicking the 3D Transforma-

tion patch (more accurately it is

called an environment) reveals the

following sub-patches:

In a sense the 3D Transformation environment ‘renders’ or displays the results of

it’s sub-patches. In this case: the green patches are ‘controllers’. The interpola-

tion and the range patch combine to create a set of values that animate the -X

shift. In the inspector of the interpolation patch is the key to how the animation

appears so ‘organic’.

quartz composer in the art and motion design of futurismo zugakousaku 19

The interpolation patch uses a

custom curve to control the out-

put of a range of values - in this

case set to change between -3

and -3.5 over the duration of 5

seconds.

The range patch output is plugged into the input “Translation X” with a yellow pipe

or “noodle”. These values move the image in the -X direction (from right to left)

over a duration of 5 seconds The values do not simply increase in a linear way

over time but according to the weightings visualised in the customised curve.

Creating a smooth, flowing movement.

The parent “3D Transformation” patch contains yet another “3D Transformation” -

a child or sub-patch.

So: a Quartz Composer composition can be a set of nested patches and sub-

patches. Zugakousaku’s “Who Am I?” contains the following pattern of hierarchi-

cal nesting:

20 quartz composer in the art and motion design of futurismo zugakousaku

(0) ▼ Root Macro Patch

(1) ▼ 3D Transformation

(2) ▼ 3D Transformation

(3) ▼ man (Iterator)

(4) ▼ 3D Transformation

(5) ▶ Replicate in Space

(3) ▶ text (Macro)

(1) ▶ Credit

The outline above gives an indication of how the patches interrelate. Change the

X position of the “3D Transform” at level 1 and you move the entire nested con-

tents. The Quartz Composer editing environment gives you an outline of this list

in what is called the “Hierarchy Browser”. As we shall see, each area/node,

though, can contain a complex collection of patches.

Next I’ll dig down into the composition and find the relevant parts that create the

cloned man image and create the parallax / depth effect.

21 quartz composer in the art and motion design of futurismo zugakousaku

The man patch has a

custom title “man”. The

inputs and the inspector

reveal it is an ‘iterator’

patch. An iterator be-

haves like a simple “for”

loop, repeating its con-

tained patches until the

number of iterations (repeats) is equal, in this case, to five. The iteration is zero-

indexed (0-4). This iteration refers to the number of the rows of men. The large

diagram (below) represents the patches inside the iterator patch:

quartz composer in the art and motion design of futurismo zugakousaku 22

I T E R A T O R

A

B

C

D

(A) The single man image. Note that dwelling over a connected output brings up a

tool-tip preview of the output. The same source image is passed four times

through “Gaussian Blur” patches with different settings.

(B) The very clever “Multiplexer” patch switches between source inputs using an

index. In this case the index is provided by the iterator “index”. As the iterator in-

crements from 0-4, the multiplex result is switched between the 4 blurred images

and the original non blurred source and provided to the “image” input on the 3D

Transformation patch. This “Image” input is what is known as a published input

from the patch hierarchically below the current patch. More of the feature of pub-

lishing inputs and outputs in Apple’s documentation.

(C) At the same time, a “Math” patch performs a calculation, using the increment-

ing “Current Position” as an initial value and multiplying it by 5. The initial value is

greyed out because the value is supplied via a ‘noodle’ from the “Iterator Vari-

ables” patch. This value is passed into the “3D Transformation” patch and moves

the produced image forward an

increasing distance on the Z

axis every iteration.

(D) Drill into the “3D Transfor-

mation” and you find the follow-

ing single “Replicate in Space”

patch:

23 quartz composer in the art and motion design of futurismo zugakousaku

The “Replicate in Space” patch and it’s

inspector (illustrated below) control

making copies of the patches contents

in 3D space. In the case of “Who Are

You?”, 16 copies of the man image (in

different ‘blur’ states) are made 1:1

scale each translated on the X axis by

7.5 units. (See Apple’s documentation

for more information on the resolution

independent unit and co-ordinate sys-

tem used in Quartz Composer). The X

quartz composer in the art and motion design of futurismo zugakousaku 24

R E P L I C A T E I N S P A C E

contents of the replicate in space patch

translation value is so chosen to match the position of the man image at the be-

ginning and the end of the scrolling animation. Note that the “Image” input on the

patch is a dark filled circle. This signifies that the input is published and therefore

available as an input of the containing patch. Likewise, inside the “Replicate in

Space” patch an “Input Splitter” patch has it’s input published and given a custom

name “Image”. Using an “Input Splitter” is a technique that allows reuse of a

common resource or value. With careful publishing, “Input Splitters” (and most

other patches) can be used to pass values, images and data between levels of a

composition.

The “Sprite” patch is a “renderer” - i.e.. it displays a quad 11 on the screen. The

next paragraph will sound complicated as it attempts an over-arching description

of the entire graphical process:

the “Sprite” displays a man image, duplicated 16 times and each copy offset by

the “Replicate in Space” patch. The image is passed in from the “Multiplexer” that

selects blurred versions of the images via a “3D Transformation” patch that cre-

ates the rows of men going backwards. As this patch is contained in the “Iterator”

patch we get 5 rows of men. The parent “3D Transformation” patch is animated

moving right to left on the X axis - moving the whole scene smoothly.

An interesting construction principle emerges: the smallest element, the single

image, is at the deepest level of the hierarchy. It is operated on (translated, dupli-

cated, blurred, colorised) by operations higher up the chain. A complete discus-

sion of the execution order of patches and the ‘data flow’ within a quartz com-

poser composition is beyond the scope of this essay.

25 quartz composer in the art and motion design of futurismo zugakousaku

11 A rectangular polygon that can be textured with video or images and position in Cartesian 3D space

S U M M A R Y

Quartz Composer stands in an exciting position as a new technology facilitating

artists who wish to make high-performance, hybrid interactive artworks with soft-

ware that ships free with Mac OS 10.4. It is simple to use and learn but has hid-

den, complex potential. This present essay sought to

★ introduce the technology, with brief reference to similar technologies, it’s history

and future role in Apple’s graphics architecture

★ deconstruct an artefact by one artist, Futurismo Zugakousaku, as a means for ex-

ploring the form and potential of Quartz Composer as an artistic medium. I dis-

cussed audience, time and dynamics, and hybridity. I described the motion graph-

ics and visual design and then talked about the technical process.

The description of the technical process sounds involved and complex but it

doesn’t do justice to the flow and ease of development in the drag and drop, vis-

ual programming environment.

In conclusion, the method of aesthetic enquiry is rich and needs to be applied to

more examples, including:

★ Real Time Video Processing in Quartz Composer. Programmer / artist Sam Kass

explores advanced real time video processing including optical flow, motion and

colour detection.

Screen-shots from works by sam kass (2005)

quartz composer in the art and motion design of futurismo zugakousaku 26

and

★ Other offerings from Futurismo Zugakousaku. I selected a simple, visually rich

example. More complexity, technique and surprises await in the open sources of

Futurism's other work.

Screen-shots from “falling text”, “jelly worm”, “squids”, “light passage” and “Black

White” by futurismo zugakousaku (2005)

Ian Grant

January 2006

27 quartz composer in the art and motion design of futurismo zugakousaku

BIBLIOGRAPHY

T E X T S

Gelphman, D. and Laden, B. (2005) Program-ming with Quartz: 2D and PDF Graphics in Mac OS X, Morgan Kaufmann, NY.

Greene, R. (2004) Internet art, Thames & Hud-son, London ; New York.

Scholder, A. (Ed.) (2001) Interaction: Artistic Practice in the Network, Distributed Art Publish-ers.

Stallabrass, J. (2003) Internet Art: The Online Clash of Culture and Commerce, Tate Publish-ing, London.

W E B

Apple Computers, (2005a). Technical note tn2145: efficiently using quartz composer com-positions with quicktime. Retrieved Jan. 07, 2006, from Apple Developer Connection Web site: http://developer.apple.com/technotes/tn2005/tn2145.html#TNTAG9 .

Apple Computers, (2005b). Technical q&a qa1427: what is the timebase submenu avail-able in the contextual menu of some patches in quartz composer?. Retrieved Jan. 07, 2006, from Apple Developer Connection Web site: http://developer.apple.com/qa/qa2005/qa1427.html.

Apple Computers, (2006). Working with quartz composer. Retrieved Jan. 6, 2006, from Apple Developer Connection Web site: http://developer.apple.com/graphicsimaging/quartz/quartzcomposer.html.

Kass, Sam, (2005). Sam Kass's Blog. Retrieved Jan. 07, 2006, from Sam Kass's Blog Web site: http://www.samkass.com/blog/ .

Latour, P. (2003). Pixelshox: visual creation en-gine. Retrieved Feb. 26, 2006, from PixelShox Technology Web site: http://www.pol-online.net/pixelshox_technology/index.php.

Zugakousaku. Futurismo Zugakousaku. 26 Dec. 2005 http://www.zugakousaku.com/index.cgi?quartz&samples&en&.

Zugakousaku. Futurismo Zugakousaku. 06 Nov. 2005 http://www.zugakousaku.com/index.cgi?quartz&concept&en.

I M A G E S A N D S O U R C E S

Where not covered by creative commons li-censes, All trademarks, logos, images, source code and screen-shots are the copyright of their respective owners: Apple Computers, P. Latour, Futurismo Zugakousaku and Sam Kass.

quartz composer in the art and motion design of futurismo zugakousaku 28