Locative Media

Digital media with a sense of place, embedded into the real physical world

Locative Media

Can You See Me Know?

grafedia.net

• Pervasive gaming: world as a game-board• Space annotation: media with a specific

position in space• Location awareness & GPS-enabled

locative media• Mobile music & locative audio• Social spaces• etc

Locative Media

Can You See Me Know?

grafedia.net

Mobility as creative act

Creative use of public space

Aesthetic Urban PracticesOrigins

Aesthetic Urban Practices

Graffiti• 3D, ephemeral, transient,

layers...

Origins

Reclaim the streets

Walking• situationist dérive, psycho-geography• aboriginal walkabouts

Aesthetic Urban PracticesOrigins

Aesthetic Urban Practices

Urban sports:• skateboarding• parkour>> urban space as resource

for aesthetic movements

Origins

Mark Weiser’s vision (1991)– disappearing computer– everyday world literally used as interface away from

desktop settings, available at hand in the real world: where needed, “where the action is”

“The most profound technologies are those that disappear. They weave themselves into the fabric of everyday life until they are indistinguishable from it.” (Weiser)

Ubiquitous ComputingOrigins

Implementing the vision– Mobile devices combined with computers

embedded in the environment – Awareness of physical & social context + each other

>> Mapping the digital world onto the physical one>> User interface: tangible and embedded in the real world

Ubiquitous ComputingOrigins

Technologies– context awareness– mobile computing– tangible interfaces – social navigation– embedded sensor networks – global positioning– wearable computing– augmented & mixed-reality– ad hoc and p2p user networks– etc

Ubiquitous ComputingOrigins

Examples– “walk-up-pop-up”– wearables– ambient displays– intelligent work environments– augmented, interconnected everyday

objects– etc

Media cup, TecO

Ubiquitous ComputingOrigins

Everyday physical world: • not designed for the purpose of these new activities • offers a rich and heterogeneous variety of engaging

interaction • situates them in cultural and social context, with existing

web of meaning• more than a setting, a resource for computer-mediated

aesthetic interaction

>> Everyday activities as basis for interaction + everyday physical real world as interface

Ubiquitous ComputingOrigins

• Audio, positioning, mobile-telephony• Mobile phone, Bluetooth, iPod, Zune,

cameraphones, GPS-mobiles, RFID tags, 2D Barcodes, etc

• Smart phones SDK (>> programmable)

– devices always at hand– user always “on-line”– all of your music with you

at anytime

Consumer ElectronicsOrigines technologiques

Can You See Me Know? Blast Theory + Equator

Pervasive Computing: The world as game-board

• Botfighters and Pirates! • Backseat Gaming • Can You See Me Now?• iPerG• ...

Pervasive GamingProjects

SPACE ANNOTATION: media with specific position in space

Examples: – Virtual:

Geonotes, Urban Tapestries

– Physical: Yellow Arrows, Grafedia

grafedia.net

Yellow Arrow, Count Media

Space AnnotationProjects

• GPS-drawing• Non-linear narratives• Tracking and mapping paths

Hundekopf, knifeandfork

Drift, Teri Rueb

Biomapping, Christian Nold

GPS & PositioningProjects

SOCIAL SPACES: connecting people in public space

• Hummingbirds

• Jabberwocky

• MobiTip

Familiar strangers, Intel Research

Social ComputingProjects

• Audio space annotation

• Mobile music sharing/listening:

- distributed

- ad hoc

- sound walks

• Mobile music making:

- situated

- collaborative

• Wearable audio

• Mobile phone as platform for sound-art...

Locative AudioProjects

Hear&There(Rozier, MIT Medialab, 1999)

Locative Audio

Audio space annotation

Tacticle Sound Garden [TSG] (Mark Shepard, Buffalo Univ. 2004-06)

Locative Audio

Audio space annotation

Tejp / Audio tags(PLAY & FAL, 2003-04)

Locative Audio

Audio space annotation

Tejp / Audio tags(PLAY & FAL, 2003-04)

Exploring embodied interaction with digital space annotation

F.ex: audio tags whispering to by-passers as they approach them, creating a short space of intimacy in public space

Locative Audio

Audio space annotation

Tejp / Audio tags(PLAY & FAL, 2003-04)

Locative Audio

Audio space annotation

Tejp / Audio tags(PLAY & FAL, 2003-04)

Locative Audio

Audio space annotation

Audio Bombing(Fleming et al., 2007)

Sonic Graffiti(C-Y Lee, 2007)

Locative Audio

Audio space annotation

[Murmur] (murmur.ca)

Locative Audio

Audio space annotation

Location 33 (Carter & Liu, USC, 2005)

Distributed and located music

Locative Audio

SoundPryer (Mattias Östergren, Interactive Institute, 2001)

TunA (Arianna Bassoli et al.,

Medialab Europe, 2002)

Locative Audio

Mobile music sharing

Locative Audio

Bass Station (Mark Argo & Ahmi Wolf, 2003)

Push!Music(Håkansson et al., 2005)

Mobile music sharing

• Drift (Rueb)

• 34n118w (Knowlton, Spellman, 2002)

• Craving (Garnicnig, Haider, 2007)

• Seven Mile Boots (Beloff et al., 2003-04)

• The Case at Kulturhuset (Knifeandfork, 2004)

• Riot! (Mobile Bristol, Hewlett Packard)

Sound walkLocative Audio

Seven Mile Boots (Beloff et al., 2003-04)

Sound walkLocative Audio

Bit Radio(Bureau of Inverse Technology)

Radio piratesLocative Audio

7/11 (New Beginnings, Göteborg)

Radio piratesLocative Audio

Key Chain Radio Station (Rikako Sakai, Ivrea, 2004)

Radio piratesLocative Audio

Locative Audio

Sonic City (Gaye et al., FAL & PLAY, 2002-04)

Sound Lens(Toshio Iwai, Tokyo Univ.)

Solarcoustics: CONNECT (Barnard, ITP/NYU, 2005)

Situated music making

Locative Audio

Sonic City (Gaye et al.FAL & PLAY, 2002-04)

Mobile music making with the city as interface: Creating a real-time personal soundscape of electronic music by walking through and interacting with urban environments

Situated music

making

Locative Audio

Sound Mapping (video)(Mott et al., Reverberant, 1997)

Sonic Interface (Akitsugu Maebayashi, 1999)

Warbike(McCallum, 2005-06)

Skatesonic (video) (van Toder, 2006)

Situated music

making

Locative Audio

ImprovE (video)(Wideberg & Hasan, 2006)

CosTune (Nishimoto et al., ATR, 2001)

Malleable Mobile Music (Atau Tanaka, Sony CSL, 2004)

Collaborative mobile music making

Locative Audio

China Gates (Clay, Majoe, 2006)

Sequencer404 (Hatcher, Jimison et al., 2006)

Cellphonia (Bull et al, 2006)

Collaborative mobile music making

Locative Audio

Nomadic Radio (Shawney, MIT Medialab, 1998)

Sonic Fabric (Alice Santaro, 2002)

Wearable audio

Locative Audio

”Personal instruments”(Krzysztof Wodiczko, 1969)

(Chelle Hugues, RCA/CRD, 2000)

Wearable audio

Locative Audio

Robotcowboy (Wilcox, 2007)

Hearing Sirens (Cathy van Eck, 2007)

Wearable audio

• Kadoum (Waagenaar, 2000)

• Dialtones. A Telesymphony (Levin, 2001)

• Pocket Gamelan / Mandala (Schiemer, Havryliv, 2006)

• Egotone (I. Lee, 2007)

Locative Audio

Mobile phones as platforms

Locative Audio

• CaMus (Rohs, Essel, Roth, 2006)

• TRATTI (Beloff, Pichlmair, 2006-07)

• Intelligent streets (Sonic Studio & Univ. of Westminster, 2004)

Mobile phones as platforms

Locative Audio

Output

• Output: Headphones vs boombox vs using everyday objects

SoundbugTM speakers & piezos

Flower Speakers (LET’S corporation, Japan, 2004)

Interactions happening anywhere, on the move :• taking advantage of the mobile setting: playing with

social and geographic dynamics implied by mobility >> outdoors everyday space, location and social context

becoming resources for interaction as you move through space

>> spontaneous & situated collaboration with people around or distributed across the city

Interaction Properties

Locative media

Interactions happening anywhere, on the move • becoming embedded in the physical and social context

of everyday life >> people managing interaction in heterogeneous

context>> and in simultaneity

with other activities(crossing a street...waiting for the bus...)

Interaction Properties

Locative media

tunA, Bassoli et al, Medialab Europe, 2002

• User-authored content spread across public space: raises questions about – property of information– privacy & surveillance– spamming?

• Augmenting environments and supporting activities with embedded computation: what if it changes what makes things what they are?

• If ubicomp spreads into public space, according to whose will? Top-down corporations, government vs bottom-up citizens, communities? Conflicts of interests?

QuestionsLocative media

• User control (Greenfield): How do you know you are interacting with a computer if invisible? How do you protect your privacy? avoid false commands? How do you know where to look for interaction?

• How to query/notify presence, access, place, manipulate media?

• How is the place? Who is there? What activities are going on there? How mobile is/are the user(s)? What meaning do the place, activities, and things around have and for whom?

QuestionsLocative media

• Ubicomp vs pervasive computing: at hand when needed vs always on everywhere

• Connect physical and virtual world: technical and HCI issue but also sociological, aesthetic, even political and environmental. F.ex. Yellow Arrow vs Geonotes: – physical vs virtual markers– Graffiti style interaction vs screen-based

QuestionsLocative media

Enabling technologiesAvailable to General Public

• Mobile peer-to-peer• Tracking, positioning and placement• Sensing and data-processing• Content creation and manipulation

Enabling TechnologiesAvailable to the General Public

* Server-Client

* Mobile peer-to-peer:– Bluetooth– WiFi– Infrared

Enabling TechnologiesAvailable to the General Public

* Bluetooth• Standard communication protocol for wireless personal

area network (PANs) • Connect and exchange information (commands, files)

between devices • Microwave radio frequency -> non-directional• Short range (power-class-dependent: 1 -10 - 100 m) • Use: BluetunA, bluejacking, Nokia’s Digidress

Enabling TechnologiesMobile Peer-to-Peer

* WiFi• Wireless local area network• Radio, non-directional• Internet and VoIP phone access, network connectivity for

for consumer electronics, etc• Connect to local access points• Server-client vs ad hoc networks

Enabling TechnologiesMobile Peer-to-Peer

* Phones vs Wifi-enabled PDAs• Connectivity: closed/open network vs operators• Cost• Range• Distributed vs ad hoc vs server-client• Compatibility• Programmability: SDK, OS• Memory, speed

Enabling TechnologiesMobile Peer-to-Peer

* Platform: Opentrek

• http://www.develant.com/opentrek.php• Peer-to-peer networking platform specifically designed

for Wireless Ad Hoc Networks • Cross-platform! • Ad hoc networking -> collaborate

Enabling TechnologiesMobile Peer-to-Peer

* Tracking, positioning and placement– Phone cells– WiFi hotspots– GPS– Virtual media– Physical markers: 2D barcodes, RFID, user ID to

phone

Enabling TechnologiesAvailable to the General Public

* Global Positioning System (GPS) • 30 geo-stationary satellites -> location, speed, direction,

path• Shadows, accuracy• Use: CYSMN?, GPS drawing, Drift• GPS-enabled phones, PDAs• Platform: Geotracing

http://www.geotracing.com

Enabling TechnologiesTracking, Positioning and Placement

* Geotracing

http://www.geotracing.com

Enabling TechnologiesTracking, Positioning and Placement

* Placing media: socialight.net• In-place and remote annotation

with smart-phone /PDA• social network community• sound, text, images, video• google maps + GPS

Enabling TechnologiesTracking, Positioning and Placement

* RFID• Radio-frequency identification• Storing and remotely retrieving data • Storage & processing + antenna• Physical markers• Tagging objects• Range: 5-20cm• Passive (powered by inductivity

when used) vs active RFID

Enabling TechnologiesTracking, Positioning and Placement

* RFID– Uses:– Passports– Transport payments– Product tracking– Automotive– Animal identification– RFID in inventory systems– Human implants– RFID in libraries

• Controversy: privacy issues. Shielding?

Enabling TechnologiesTracking, Positioning and Placement

* 2D barcodes

• QR (Quick Response) code, Datamatrix code, etc• Physical markers• Can store between one and 500 characters • Tag objects, places• Scan with cameraphones

-> hyperlink (physical mobile interaction)• How to: Kaywa reader http://reader.kaywa.com/ +

generator: http://qrcode.kaywa.com/

Enabling TechnologiesTracking, Positioning and Placement

* Unique ID to phone

• Physical markers with unique IDs• Tag objects, places• Send number to server

-> store & retrieve media• Arrows available, but not

ID generator

Enabling TechnologiesTracking, Positioning and Placement

* Sensing:– sensors– data processing: microcontrollers

Enabling TechnologiesAvailable to the General Public

* Micro-controllers• Basic Stamp II, Basic X – 24 http://www.basicx.com

Tutorial: http://www.tigoe.net/pcomp/index.shtml• Arduino

– open source hardware physical computing I/O platform

– cheap (20 Euro)– easy (Processing)– assemble yourself– stand-alone or connect to

computer (MAX/MSP, etc)– www.arduino.cc

Enabling TechnologiesSensor Data Processing

* Creating and manipulating content:– Mobile Processing– Python– J2ME– miniMIXA– PdA (Pd on PDAs, linux)– Keyworx

Enabling TechnologiesAvailable to the General Public

* Mobile Processing• http://mobile.processing.org• Open source programming environment for design and

prototyping software for mobile phones. • Similar to Processing environment. • Runs on Java powered mobile devices.• Bluetooth -> communication• Control example: attach light sensor on screen so

sending info from phone to laptop

Enabling TechnologiesCreating and Manipulating Content

* MiniMIXACommercial DJ software for mobile phones, PDAshttp://www.youtube.com/watch?v=R6BSGy8mMsU

* KeyworxMultimedia platform (base for GeoTracing f.ex.)http://www.keyworx.org/

* PDa (Puredata anywhere): Pd for Linux on PDAshttp://gige.xdv.org/pda/

Enabling TechnologiesCreating and Manipulating Content

* Python PyS60• Interactive object-oriented language• Nokia S60 phones and more• Record, playback, play MIDI notes, control MAX/MSP

patches...• http://www.python.org/• PyS60: http://www.forum.nokia.com/python and

http://www.mobilenin.com/pys60/menu.htm

• Tutorial (Jürgen Scheible - Mobilenin)

Enabling TechnologiesCreating and Manipulating Content

• 3rd party software (Java, etc)• Hacking hardware: use camera, microphone, speakers,

audio out...

Enabling TechnologiesHacking mobile phones