Four Tests in Three Summers: Assessing visitor preferences in handhelds at SFMOMA

Peter SamisAssociate Curator, InterpretationSan Francisco Museum of Modern Art

Tate Handheld Symposium • 5 Sept. 2008

Case Study: Matthew Barney.

Matthew Barney sculptures in the gallery

Our problem: subtract the man/film from the gallery, and…

How to make the mute plastic speak?

Traditionally, this would have been cause for a multimedia feature.

…online, by the galleries, in the Koret Center.

To this, we added multiple format audio tours:

• Podcast: download from home

• Gallery X-plorer: rent in Atrium

• Cell phone: turn on in show

Collaboration with Antenna Audio to develop content for all three.

Downloadable podcast map

Cell phone promotional card - collab w/ Guide by Cell

At first, we thought we were setting out to measure visitor preferences for three different types of audio tours.

And we did that.

What did we find out?

• There is no clear winner• Traditional devices

skew older• iPods and cell phones

seem to skew younger*• Visitors appreciate

multiple experience options

* statistics not fully conclusive

Different devices support different touring styles:

Cell phones averaged 4.6 stops out of 10, indicating à la carte, on demand use

iPods and traditional audio tour devices averaged 70-80% stop use: a more immersive, custom experienceIn fact, iPod and traditional headset tour users had more in common than these users might suspect…

TABLE 35AWARENESS OF THE AUDIO GUI DE HEADSET DEVICE (IN PERCENT)

DEVICE USEDPODCAST(n =18)

CELLPHONE

(n =48)

AUDIOGUIDE(n =53)

NONE(n =132)

AUDIO GUIDE AWARENESS % % % %

Used the audio guide headset 0 0 100 0Aware of audio guide headset but didnot use it

83 52 0 73

Unaware of audio guide headset 17 48 0 27

Those who chose the iPod and cell phone formats rated them more highly than traditional headset tour users rated theirs.

(although the content was identical!)

TABLE 43RATINGS OF INTERPRETIVE OFFERINGS

RATING7-POINT RATING SCALE:DID NOT HELP ME APPRECIATE BARNEY’S ART (1) /HELPED ME APPRECIATE BARNEY’S ART (7) n MEAN ±

Cell phone audio tour 46 6.2 1.10Podcast audio tour 18 6.2 0.81Antenna audio guide headset tour 50 5.6 1.44Learning Lounge 95 5.5 1.45Exhibition brochure 131 5.2 1.53Exhibition Web site 31 5.2 1.37Drawing Restraint 9 film 40 5.1 1.92Exhibition introduction wall text 182 4.7 1.65

Phone logs enabled us to see patterns in where visitors wanted information most:

(or perhaps where they or could find the labels!)

In fact, GuideBy Cell now mashes up area codes and Google Maps to reveal where visitors are from:

Stats: Cell-phone Tours

Good News: Guide by Cell records

• 1st exhibition with over 10,000 calls

• Average message selection in other museums is only 33%

Bad News:• Still only averaging

45 visitors/day• Only 3.7% of

exhibition visitors

So somewhere along the way,

the target

shifted.

The elephant in the room is that:

The vast majority of our visitors do not use technology during their museum visit.

So how do we reach them?

If our goal is to use technology, we may just nod and move on. But if our goal is to enhance visitor experience, we have to think larger.

“Variable on-demand mediation.”–Brad Johnson, Second Story Interactive

Interpretive Menu: analog + digital

• One curatorial intro panel• Exhibition brochure• Docent tours

Audio tour delivered via: • Antenna MP3 gallery guide• Downloadable podcast• Cell phone tour courtesy

GuideByCell

Learning Lounge in show:• Plasma screen with Barney

video interview clips• Barney FAQ wall graphics• Interactive feature/website• Books & catalogs

Here’s what that looks like:

INTERPRETIVE OFFERINGS (COLLAPSED) USED BYRESPONDENTS (I N PERCENT)

USED

INTERPRETIVE OFFERINGS (n = 251) %

Exhibition introduction wall text 78Exhibition brochure 55Learning Lounge (one or more offerings) 51Audio tour (audio guide headset, cellphone, or podcast)

47

Drawing Restraint 9 film 17Exhibition Web site 15SFMOMA docent- led public tour 2

Use of offerings by respondents:

But on the other hand:

TABLE 43RATINGS OF INTERPRETIVE OFFERINGS

RATING7-POINT RATING SCALE:DID NOT HELP ME APPRECIATE BARNEY’S ART (1) /HELPED ME APPRECIATE BARNEY’S ART (7) n MEAN ±

Cell phone audio tour 46 6.2 1.10Podcast audio tour 18 6.2 0.81Antenna audio guide headset tour 50 5.6 1.44Learning Lounge 95 5.5 1.45Exhibition brochure 131 5.2 1.53Exhibition Web site 31 5.2 1.37Drawing Restraint 9 film 40 5.1 1.92Exhibition introduction wall text 182 4.7 1.65

What helped make meaning?

So what did we learn? What did visitors prefer?

Audio: Different people opt for different devices

Video: Visitors want to see and hear the voice of the artist above all else

and perhaps most surprisingly:

Visitors opt for the analog first.

Or put another way:

Use Pattern / Value Rating of Barney Interpretive Resources

0102030405060708090

Introductory wall textExhibition brochureLearning Lounge

Audio tour (3 options)Exhibition Web site

% of visitors using resource

01234567

Visitor rating of resource value

USED (%)

RATING (1–7 scale)

This is ourOpportunity Space!

Is it time to re-train our visitors? Change their expectations?Change our approach? All of the above?

How do we activate that opportunity space?

Next Tests: 2007Traditional audio tour vs. iPod

Picasso & American Art & Brice Marden

Matisse: The Painter as Sculptor

Take your time: Olafur Eliasson/ Joseph Cornell/Jeff

Wall

When visitors were given the choice…

Exhibitions in 2007 (in sequence)

Antenna XP - “Classic”(mp

3)

iPod

Picasso & Marden

59% 41%

Matisse 55% 45%

Eliasson /Cornell / Wall

51% 49%

Do we detect a trend?

0%

10%

20%

30%

40%

50%

60%

70%

1. Picasso & Marden 2. Matisse 3. Eliasson /Cornell /Wall

XP-Classic (mp4)iPod

Visitors liked the fact the iPod showed an image of the work.

But the menu function was poor.

Fast forward to summer 2008 (right now):

Enter

Frida Kahlo

MENU: analog + digital mix• Brochure• Wall texts• Multimedia tour with low

price barrier to entry• Learning Lounge• Supplementary galleries

to add context • Kiosk / Website• Video • Podcasts

Antenna XP-Vision multimedia player

Visitor Feedback

NEXT STEPS

Purpose-built or Visitor-supplied ?• Controlled platform• Hi-fidelity audio,

video & interactivity• Free of associations

& hassles of home & office

• Only about the art• Focused, optimized

for tour experience

• Rewrites the economic model: No need for museum to lease fleets of players

• No need to staff and sell players to visitors

• Free to visitors• More casualbut• Quality of experience

cannot be assured

Questions we have now:

Unless we rewrite the spec for what a quality tour experience is

to a low enough threshold that it can bedelivered with confidence to a wide varietyof visitors in a non-proprietary manner

Does that mean we bypass the bells and whistles of Flash and the iPhone?

Or do we opt for that expressive power and continue to supply the hardware on the old lease contract model?

I, for one, no longer believe it will all sort itself out…

“in two years.”

Soon to come… an SFMOMA iPhone title:

In the meantime…

Don’t worry, folks—

This is only a test!

Thank you.