In CADE2004 Web Proceedings of Computers in Art and Design

Education Conference, M. Agger Eriksen., L. Malmborg, J. Nielsen

(eds.). Copenhagen Business School, Denmark and Malmö University,

Sweden, 29 June – 1 July 2004. asp.cbs.dk/cade2004/proceedings/

Alt-Space: Audio-visual interactive software for

developing narrative environments

Peter Waters

Surrey Institute of Art & Design,

University College

E-mail: pwaters@surrart.ac.uk

Anthony Rowe

Surrey Institute of Art & Design,

University College

E-mail: ant@squidsoup.org

ABSTRACT: Communicating the ethos of a subject (e.g. abrand or cultural or historical narrative) within anenvironment, such as a museum or visitor centre, is oftenundermined by the poor integration of digital technologiesin the process of interpretation.

This often comes about through a fragmented approach tothese environments which fails to take advantage of thepossibilities of new media to engage our senses andconnect us to a subject.

Our approach to this problem is to create a tool for thecreative sketching of space that provides the ability toexplore the more sensory aspects of spatial design to testthe sequencing of events relative to a narrative.

Through the development of this resource we hope to

enable students (and practitioners) to experiment with a

wide range of media and test their proposals before

committing resources and (inevitably) limited funds1 to

ideas that fail to engage the visitor effectively.

KEYWORDS: Interdisciplinary, narrative environments,multi-sensory, interactivity, accessibility

INTRODUCTION

An interdisciplinary project between the new mediacompany squidsoup and the interior design programme atthe Surrey Institute of Art & Design. University College,Farnham, England developed out of the programme’sresearch into reality based virtual spaces and thepossibilities offered by blurring the boundaries of actualand virtual spatial design.

A visit by Land Design Studio (creators of UK basedenvironments such as the National Maritime Museum(Cornwall), the Play Zone (Millennium Dome, London)and BAFTA winners for the Famous Grouse Experience(Crieff, Scotland)) to the institute to present the work of

the practice encouraged the programme to explore theseareas further.

Land’s multi-disciplinary approach to narrativeenvironments ensures cohesion and a more effectivesolution2 than the traditional museum display.

The practice has also pioneered the use of digitaltechnology within their designs in a way that challengesthe ubiquitous deployment of touch screens and plasmadisplays and explores the potential of the new media tointerpret a subject in a more engaging way3. The PlayZone was the inspiration for the practice’s work. Heredigital artists from all over the world were effectively‘showcased’ and new opportunities for user interactionexplored. This, in turn, informed their own designsolutions and led to a diverse series of outcomes such astheir work at the Famous Grouse Experience and theNational Maritime Museum.

At the Famous Grouse the whisky company’s humorousadvertising campaign is translated into an interactiveenvironment where seismic detectors pick up thevibrations of the visitors as they move around. This altersboth floor and screen projected images engaging thevisitor in a light-hearted and amusing way andmaintaining the company’s brand image of whisky as amore ‘youthful’ product4.

The synchronisation of all of these elements is critical andaccess is limited to eight people at a time to ensure thesystem is not overloaded. The installation was designedsimply for fun rather than to explain the processesinvolved in whisky production. It offers a pre-determined,prescribed experience with no opportunity to delvedeeper.

In a different situation it might have been possible toexplore other aspects of the culture surrounding theproduct such as customs and traditions or the differingattributes of whiskies enabling the visitor to interrogate orexplore the subject to greater depth through other forms ofdigital media. For example current low-techscratch’n’sniff cards (used to identify the complex ‘nose’of the spirit) are effective but could be expanded in futurethrough the use of pioneering digital scent technology.5

In the Maritime Museum the fusion of artefact, audio-visual mechanisms and interactive technologies combineto enhance the understanding of the museum’s collectionof sea-going craft, meteorology, navigation, boatconstruction and the landscape/seascape and tidesimmediately outside the building6. The technology isembedded into the interpretation process enabling thevisitor to interrogate and explore the exhibits more freelyand to a greater depth than previously.

(see www.landdesignstudio.co.uk) 7

A funded knowledge transfer project and the engagement

of Land’s Creative Director, Peter Higgins, as theprogramme’s Course Advisor led to a shift in the cultureof the Interior Design programme. We perceived thatthrough the inclusion of digital technologies withtraditional concerns of construction technology, spatialcomposition, materials, colour, texture, light andatmosphere exciting new possibilities for the study andpractice of interior design (particularly withinenvironments that feature a strong narrative context)would be possible.

NARRATIVE ENVIRONMENTS

The design consultancy Casson Mann demonstrates thepower of these new possibilities to inform, engage andencourage discussion through their work for BNFL(British Nuclear Fuels Ltd) at their Sellafield VisitorCentre in Cumbria, England.

In ‘Sparking Reaction’, 2002 the visitor is literally‘immersed’ in the debate surrounding the use of nuclearpower i.e. the pros and cons: the risks involved and thebenefits gained over traditional fuel sources. It alsotouches on matters of major public concern, includingnuclear waste disposal, leukaemia and even the misuse ofnuclear capability.

Entry is through a stark tunnel that leads to the reception.From here the main exhibition space is accessed and thesubdued light and unsettling soundtrack (recorded withinthe nearby nuclear power station) sets the tone for theexhibition.

The debate, realised in collaboration with graphicdesigner Nick Bell of UNA, evolves in a darkenedenvironment (named the Core) where six 8 x 6m angledprojection screens enable the projection of animated textthat metamorphoses into a style that is in keeping with theissues or sentiments expressed as the visitor engages withit.

Getting people to read was the key challenge for CassonMann, as Roger Mann explains:

'People at exhibitions simply read as little as possible, butas this is such an emotive subject, we felt that justicecould be done to it only by including a lot of textualinformation.'

An interactive immersion cinema, the first of its kind inthe UK is featured beyond the Core zone anddemonstrates how visitors can be actively engagedthrough the use of digital mechanisms:

‘Illuminated by red neon lighting, the interior of thisspace contains twenty touch-screen terminals that givevisitors the illusion of participating in a TV quiz show.Each contestant is invited to run the country, a job thatinvolves weighing the merits of different viewpointsbefore taking decisions. At the end, participants areconfronted with the repercussions of their choices.

Here, again, visitors have to think, form opinions and facethe consequences.’ 8

Their overall solution was well received:

‘Casson Mann's design is exemplary in its exploration ofhow to deal with a large volume of information presentedin an exhibition setting. Atmospheric zones andcomfortably paced projections engage, provoke andinspire the visitor to think first and decide later.’8

Inspired by examples such as Land and Casson Mann webegan researching and experimenting with digitaltechnologies that would enable students to engage withthese processes and develop ideas within their ownprojects. The objective was to develop software thatwould provide them with a facility to explore, test and(ultimately) communicate their proposals through a fullyinteractive, navigable 3D environment in a way that is notpossible through conventional CAD software.

POST-SYMBOLIC COMMUNICATION

Initially the task was to re-evaluate existing procedures.The process of spatial design involves three key stages:experimentation with the possibilities for a space,communication of a design proposal to a client andimplementation through to the full realisation of theconcept.

Like many other creative processes the full implicationsof a design are not realised until the design is completed.Prior to this any mechanism for testing or communicatingan idea is purely symbolic and often one dimensional:• Models communicate space in a relative sense but to

a scale significantly removed from human scale andcan not convey atmosphere.

• A coloured perspective (CAD visual) cancommunicate atmosphere (lighting, ambience etc) butfail to deal with transitions from one space to anotheror a sequence of spaces.

• A fully rendered CAD walk-through can conveyprogression through a series of spaces and theiratmospheric qualities but fails to communicate thetechnical issues.

• Technical drawings can explain how a design fitstogether but not the tactile qualities of the materialsreferred to.

• A material sample board enables the viewer to

touch and appreciate the qualities (or otherwise) of a

range of materials but not the full implications of the

material: noise underfoot, durability etc. For this yet

more ‘devices’ are required.

In Riding the Giant Worm to Saturn: Post-SymbolicCommunication in Virtual Reality9 Jaron Lanier talks of atime when, through the application of virtual reality, weare, like Superman, able to ‘fly around and pick up abuilding and turn it upside-down’.

In this way he envisages a time when architects (andinterior designers) can dispense with these symbolic,metaphorical representations that require an orchestratedpresentation to gain appreciation and acceptance of theirdesigns and use virtual reality to introduce a client to theirbuilding. Within this environment a client might say:

‘“Well, let’s move this window over,” and simply

moves it. That is post-symbolic communication. They

are actually in the building, and no longer relying on

models or animations or specifications or blue prints.

They are dealing with a direct experience not a

representation of an experience.’

Ultimately, the experience of a museum visitor could beenhanced in a similar way through this ability to probeand explore objects and data to a desired level.

SCENARIO: INTERPRETING THE RAINFOREST

Standing in the middle of the Borneo rainforest amidsttrees that soar to 150 feet or more you get a sense that thisis to the natural world what Manhattan is to urbanenvironments. The scale, sound and apparent chaos ofdaily routine envelopes your every sense and yet, thoughthe whole panoply of life and death is played out allaround us, we are able to access only a small part of it.

In Times Square we are surrounded by a microcosm ofhuman concerns: lifestyles, gadgets, fantasies, adventuresand ground breaking news of tragedies and terrorist acts.This 3D multi-media experience is every channel-hoppersdream: no remote, just turn your head and feast your eyes.The dramas of the immediate environment however arehidden from view as life evolves or disintegrates inboardrooms, apartments or some back street gutter.

The Borneo rainforest is just as effective at concealing therealities of existence from our eyes. We can detect a highlevel of activity through the cacophony of sound thatsurrounds us, and changes, almost imperceptibly, with thetime of day as each shift clocks off but we struggle to findthe source of the noise or witness the dramas that unfoldendlessly.The actual experience of many jungle adventurers is wellrecorded:

‘Many of Borneo's forest trees rise straight up from thejungle floor before spreading out into the leafy crownsthat support the rolling green surface of the canopy. Therain forest canopy has been described as an entire plantcommunity above ground. Numerous animals found inthis habitat are born and die without ever touching theground.

As usual in the rain forest, no mammals are visible exceptfor an occasional squirrel scurrying along a low branchand there are only quick flashes of birds in the canopy.Yet the sounds of unseen creatures surround us. TheBornean rain forest is rarely silent. The wash of forest

sounds seem random at first, however, the backgrounddrone is an incessant almost irritating whine of cicadas setagainst the endless "ttok-took-tarook-took-took" ofbarbets up in the canopy. Punctuating the bird songs aremysterious guttural grunts, roars and barks: monkeys,orangutan, barking deer? These sounds always comefrom just around the bend. It is never easy to tell exactlyfrom where or what is calling.’

Or memories of:

“…early morning walks in the forest, being greeted by the

rich whooping calls of the Bornean Gibbons as well as the

symphony of sound from the millions of insects and

birds.”10

‘Sounds are an important part of the jungle experience.We awake in the morning and fall asleep at night to thesounds of animals in the trees above us. Frogs sing out"what, what, what, what, what" from ponds along the roadto camp. Large tropical locusts make a variety of veryloud screeching sounds…. the songs of the locustsreminded us of Skillsaws, imagined nuclear power plantwarning sirens, coffee grinders, and chainsaws…overheadis the constant "tuk, took, tuk, tuk, tuk" of barbets. Theseare bright green birds with many different head coloringsthat hide in the trees overhead and are rarely seen, butalways heard.

Hornbills are frequent companions at Buda…they arevery loud flyers, and we are often aware of their presenceeven though we cannot see the birds through the thickcanopy of the trees overhead.’11

To actually see any of these creatures you have to go tothe nearby Natural History Section of the Museum inKota Batu about 5km from Brunei’s capital, Bandar SeriBegawan. There, in traditionalglass case format with scenic dioramas, is a vision of yourworst fears. All of the bugs your search failed to turn upin the jungle contained in one square metre of suspendedanimation: fearsome, intriguing but lifeless.

Alternatively, if you are unable to travel quite so far, a tripto the Dynamic Earth in Edinburgh might not only giveyou a better sense of what exists in the primal rainforestbut help to communicate aspects of the place that youmay never discover even if you remained in the heart ofthe jungle for a considerable period of time.

Here, the designers employ the usual array ofmechanisms; images, video, diagrams and soundscombined with a cyclical experience of changing weather(including simulated rainfall) enabling the viewer to attainsome degree of awareness and understanding.

Access to the full subject, however, is filtered by theconcerns of the interpreter and the creative skill andinventiveness of the educational team, the designer andcurrent technology. Interrogation of the full story is

inevitably, as in any exhibition, not an option. The abilityto fly through the canopy, focus on a particular species orgain a fuller understanding of the issues is limited by themechanisms and skills available.

Ideally a fusion of all three scenarios is required withsuper-human attributes included: x-ray vision and acutesenses of hearing, smell, taste and touch.

The main concerns of those involved in the world ofinterpretation is essentially that of how best to encapsulatean experience within an environment and bring it to life ina way that communicates a sense of that experience,informing a variety of users with differing levels ofengagement.

Lanier’s vision of ‘Post-Symbolic Communication’ wouldtransform this experience and reveal so much more thateven the most passive visitor might be engaged moreeffectively and depart better informed if not inspired.

Example: The Churchill Museum

Such a solution is, of course, a long way off for exploring3D space, natural or man made, but already someinstallations are beginning to provide a sense of what thisapproach could provide. Casson Mann have recentlydeveloped a solution for the Churchill Museum in Londonalong with the designers Small Design Firm (software)and UNA (London) Designers (graphics).

Through an integrated flatbed combination of digitalmechanisms: timeline, lightbox, filing cabinet and movingimages (archive footage) and innovative and engaginggraphical devices that evoke aspects of the narrative (e.g.undulating, rippling text that introduces the sinking of theLusitania) Casson Mann enable visitors to moveeffortlessly vertically or laterally through 6,000 items thathelp to define Churchill’s life12.

THE PARTNERSHIP

Part of the programme team’s research led Peter Watersand Michael Thomas to visit CADE 2003 where they metAnthony Rowe of squidsoup a Keynote Speaker that year.

After seeing and discussing the work of squidsoup in thearea of sensory interaction within navigable, spatial(though abstract) environments a working relationshipwas formed to explore the possibilities of applying thesedevelopments to spatial design.

squidsoup have significant expertise in creating bothabstract and reality-based virtual spaces that haveexplored such intangible notions as atmosphere andsensory perception. Projects of theirs such as ‘altzero’(www.altzero.com) 13 and ‘Ghosts’(www.squidsoup.com/ghosts) 14 both explore similarissues, but in more abstract terms; altzero comprises a tool(altzerocompose) that allows users to import sounds,position them and control playback, and publish the

resulting compositions for online or offline use. They arealso currently building an educational project with seed-funding from Nesta Futurelab that builds on the virtualpossibilities of the puppet theatre – the children createpuppets from virtual plasticine and enact stories in ashared virtual theatre.

The interior design team have many years’ experience ofarchitecture and design in all of its manifestations: retail,hospitality, museum design etc. They provide the projectwith an understanding of the more theoretical aspects ofspatial articulation, structural considerations and thesequencing of narrative environments as well as extensiveexperience of the processes involved in thecommunication of 3D space.

The programme is also involved in research into a diverserange of issues surrounding the process of interpretationfor example: accessibility for differing disabilities, multi-lingual translation and environmental issues. Membershipof an interpretation network and participation in theformation of a new MSc Interpretation programme hasenabled the team to extend their knowledge ofinterpretative environments to inform the project.

PROTOTYPE SOFTWARE DEVELOPMENT

The project team developed the prototype application(figures 1 & 2) for the creative sketching of 3D internalspaces providing the ability to explore the more sensoryaspects of spatial, narrative design.

Figure 1: A simple grid enables the user to

map out the space

The main focus was on creating a tool for rapidprototyping / sketching of ideas, to get a feel foratmosphere and other relatively intangible aspects ofspace design.

The emphasis was on ease of access through an intuitive

interface and appropriateness to the experimental stage of

a project rather than the processes involved in

conventional CAD packages which relate to the other end

of the process i.e. communicating a design to a client.

This form of communication requires expensive,

specialist software and skills and is too far along the

design path to effectively influence the exploratory stage

of a project.

Figure 2: Scale can be manipulated in a relative

sense by increasing or decreasing the figure in

the foreground

This initial prototype includes:- physical space design- basic object design within a space- navigation through the space, and interface design- sound representation within the space- control over texture, scale, light and shadow- human representations within the space- asset importing (sound, image and video)

The project focussed on the following key areas in orderto be able to evolve a realistic and relevant solution. Theend result was a usable piece of prototype software with aview to enabling staff and students at the Surrey Institute(and other interested parties) to test the prototype andprovide feedback.

The key issues / developments were:

1. Navigation through the space.• Speed of movement through a space

providing an impression of scale.• As, in this program, walkthroughs are freely

navigable spaces, mechanisms to limitmovement (i.e no-go areas, one-way doors)have been incorporated.

2. Sound representation within the space.Spatially positioned sounds can be importedinto a space and located.• Control over range (and echo/reverb and

other real-time effects). Background noises,people noise.

3. Control over texture, scale and light.• Walls can be coloured/tinted and painted.

• Textures and video clips can be importedand positioned on a wall.

• Lights can be positioned and coloured – thiscan be used to simulate daylight. Althoughshade is dealt with effectively, shadows arenot currently a feature of many real-time 3Dengines. We envisage simulating shadowsautomatically through phase 3 of the project.

4. Human representations within the space –human presence within a space is a vital part ofcreating a sense of perspective, scale and life(figure 3).

• A strong visual method for creating theimpression of others’ presence within aspace has been achieved by using alpha-channelled layered textures on planes. Thishas a considerable performance benefit oversolid 3D dummy models, as well as beingfar more textural and visually appealing.This will be combined with appropriateaudio feedback.

Figure 3: Representation of people

within a space

5. File saving and exporting projects. A self-contained player is anticipated for phase 3,together with a mechanism for recordingspecific walkthroughs.

6. Basic object design within a space. Twomethods were used here.• Importing textures with an alpha channel,

which can then be placed on dynamic planes(same method as for people, above)

• A simple object designer, allowing for thecreation of shaped planes that can becombined to create furniture and otherfeatures.

• Fine-tuning of interaction, modelling toolsand the interface design (figure 4 & 5).

Figure 4: Objects, graphics, lighting, texture

and movie files imported

Figure 5: Part simulation of the space

with sound and video

FUTURE DEVELOPMENTS

These developments are the early stages of a project tocreate an intuitive design tool that will raise the level ofawareness amongst our students and enable a moreinformed approach to the design of narrativeenvironments or space generally. Initial feedback from thestudents is very positive particularly from those studentswell versed in the use of conventional CAD programs.

Our eventual aim is not only to change our student’sperceptions but transform the way space is perceived.

As Alberto Lacovoni states:

‘If the main objective….is that of reconciling theindividual with his environment, overcoming that form ofspatial alienation which is caused by the impossibility oftransforming it according to one’s desires, operatingcreatively within the “zone of uncertainty betweensubjectivity and objectivity, between the imagined andreality” (Morin 74), then technology can become a verypowerful instrument of modulation.’15

As a result the programme team are now sourcing fundsand partners to continue the work with squidsoup andextend the project further.

Through subsequent stages we will be seeking toincorporate the use of VR Caves to enhance theexperience of movement and interactivenavigation.16

This phase of the project will also refer to conceptssurrounding subjects such as game design and storytellingas a means of exploring, evolving and communicating aproposed environment. 17

We will also focus further on future possibilities based onthe process of interpretation through subjects as diverse asthe natural environment, cultural and heritage sites andbrand communication.

Ultimately, we hope that we can develop the softwaresufficiently to test mechanisms relating to accessibility fora wide range of disabilities through the use of VR caves,data gloves and other modes of interaction.

As the project is still in the early stages of developmentwe are seeking feedback from visitors to CADE and thepossible involvement of interested parties to realise theprogram’s full capabilities.

REFERENCES1. Beck, L. and Capel, T. Interpretation for the 21st

Century, Sagamore Publishing, 19982. Collaborative processes, Architects Journal, Jan 20033. The Exhibitionist, Blueprint, June 20024. Marketing Magazine, April 20025. Schmitt, B. Experiential Marketing: How to get

Customers to Sense, Feel, Think, Act and Relate toYour Company and Brands. New York: Free Press,1999

6. Locum Destination Review, Summer 20027. www.landdesignstudio.co.uk8. Mecozzi, D. Fuel for Debate, (Nov/Dec 2002), Frame

299. Russell, M. Riding the Giant Worm to Saturn: Post-

Symbolic Communication in Virtual Reality. In:Druckery, T ed. ARS Electronica: Facing the Future,pp. 242-243, MIT Press, 1999

10. www.eglinpix.com/borneo11. www.northcoast.com/~rchilds12. www.cassonman.co.uk13. www.altzero.com14. www.squidsoup.com/ghosts15. Iacavoni, A. Game Zone: Playgrounds between

Virtual Scenarios and Reality, Birkhauser, 200416. Reiser, M. and Zapp, A. New Screen Media: Cinema

/ Art / Narrative, BFI Publishing, 200217. Engeli, M. Digital Stories: The Poetics of

Communication, Birkhauser, 2001