1yahooresearchberkeley.comRahul Nair

What’s in a place?Adventures with Location-Aware Media

Rahul Nair

Yahoo! Research Berkeley

2yahooresearchberkeley.comRahul Nair

Flickr “geotagged”

20+ million images

Can we do better?

3yahooresearchberkeley.comRahul Nair

Talk Outline

• Extracting information from geo-tagged photos

• World Explorer: Visualization

• ZoneTag: Creation

• Zurfer: Consumption

4yahooresearchberkeley.comRahul Nair

Attraction Map of Paris

Stanley Milgram, 1976. Psychological Maps of Paris

5yahooresearchberkeley.comRahul Nair

Attraction Map of Paris

Y!RB, 2007.

6yahooresearchberkeley.comRahul Nair

Location-driven Modeling

• Derive meaningful data about map regions

• E.g., representative tags, photos

7yahooresearchberkeley.comRahul Nair

Data Description

8yahooresearchberkeley.comRahul Nair

Issues

• Sparse data set

• Photographer bias

– In location

– In tags

• Incorrect data

9yahooresearchberkeley.comRahul Nair

Heuristics

• Number of photographs denotes the “importance” of a location

• Users will use a common subset of tags to describe objects/locations

• Concentrated tag usage indicates descriptiveness

10yahooresearchberkeley.comRahul Nair

Algorithm

• Clustering: k-Means, get set of k clusters

• “Document” C is bag of all tags in cluster

• For each tag in C calculate:

– TF = |P(C,t)|

– IDF = |P(R)| / |P(R, t)|

– UF = |U(C,t)|/|U(C)|

11yahooresearchberkeley.comRahul Nair

Scoring

• Score (t) = TF * IDF * UF

• Threshold values

– 30+ photographs

– Minimum 3 users

– Score > 1

• Final dataset: (tag, score, latitude, longitude)

12yahooresearchberkeley.comRahul Nair

Talk Outline

• Mining information from geo-tagged photos

• World Explorer: Visualization

• ZoneTag: Creation

• Zurfer: Consumption

13

DEMO

14yahooresearchberkeley.comRahul Nair

Precomputation

• Divide the world into equal sized non-overlapping tiles

• Compute and store the tags for each tile

• Repeat for different zoom

levels

15yahooresearchberkeley.comRahul Nair

Retrieval

• Find the tile level closest in size to the request area

• Select the tiles that fully cover the request area

• Return the tags that fall within the request area

16yahooresearchberkeley.comRahul Nair

User Study

10 subjects

• 6 female, 4 male

• Ages 20-60

• Varying technical knowledge

• No geotagged photos of their own

17yahooresearchberkeley.comRahul Nair

Experiment tasks

• Vacation recap

• San Francisco tour

• Explore a new city

18yahooresearchberkeley.comRahul Nair

Recall

Reminded the subject about locations

• “It brings out memories”

• “Oh my God! This place has the best restaurants”

• “We wanted to see the Polynesian Cultural Center"

19yahooresearchberkeley.comRahul Nair

Discovery

Participants discovered previously unknown locations and events

– “I’ve never heard of this festival”

– “There is car racing which I'd probably go see”

20yahooresearchberkeley.comRahul Nair

Needle & Haystack

• Excellent visualization of the Haystack

• Hard to find specific information

– “Where was Culver City again?”

• No way to search

– “I guess what I’m looking for are bull fighting pictures”

21yahooresearchberkeley.comRahul Nair

Other Responses

• Gets the “vibe” of a place

• Share with other people

• Tags did not always match the mental model of a location

• Wanted more tags

• Want more info about tags

22yahooresearchberkeley.comRahul Nair

Conclusions

• Extract meaningful aggregate information from georeferenced data

• Allows users to explore locations in a new way

• Users like using the overview but also want the ability to search

23yahooresearchberkeley.comRahul Nair

Future work

• Adding search capability

• Show photos in places with no tags

• Differentiate locations and events

• Apply to other types of georeferenced data

24yahooresearchberkeley.comRahul Nair

Talk Outline

• Mining information from geo-tagged photos

• World Explorer: Visualization

• ZoneTag: Creation

• Zurfer: Consumption

25yahooresearchberkeley.comRahul Nair

Current Mobile Experience

• Difficult to share (or even save!)

• Hard to find

– No context

– No semantic information

Current mobile experience?

26yahooresearchberkeley.comRahul Nair

Current Mobile Experience

• Difficult to share (or even save!)

• Hard to find

– No context

– No semantic information

27yahooresearchberkeley.comRahul Nair

ZoneTag Experience

• 2-click upload (same key!)

• Photo uploaded with location and time metadata

28yahooresearchberkeley.comRahul Nair

Where does location come from?

• Bluetooth GPS (when available)

• User-contributed cell tower mapping

29yahooresearchberkeley.comRahul Nair

ZoneTag Experience

• Tagging made easy

– Tag/annotate your photos from the phone

30yahooresearchberkeley.comRahul Nair

Where do tags come from?

• Tags I used in this context (`home’)

• Tags my friends used in this context

• Tags other people used in this context (‘Ricoh’, ‘California Research Center’)

– E.g., TagMaps data

31yahooresearchberkeley.comRahul Nair

Where do tags come from?

• Stuff around you:

– Yahoo! Local

– Upcoming.org

• Stuff from you (any RSS 2.0 feed):

– Calendar

– Favorite hangouts (Wayfaring, Plazes, Socialight)

32yahooresearchberkeley.comRahul Nair

Example of Tagging

34yahooresearchberkeley.comRahul Nair

Talk Outline

• Mining information from geo-tagged photos

• World Explorer: Visualization

• ZoneTag: Creation

• Zurfer: Consumption

35yahooresearchberkeley.comRahul Nair

Flickr “geotagged”

20+ million images

Can we do mobile?

36yahooresearchberkeley.comRahul Nair

Strengths of Mobile

• Personal

• Easy to access

• Networked

• Context aware

The ultimate photo wallet

37yahooresearchberkeley.comRahul Nair

Design Goals

• Engagement and Discovery

– Spatial

– Social

• Allow customization

• Complete access to your photos

• Search and Filtering capability

38yahooresearchberkeley.comRahul Nair

Channel Metaphor

• Each row is a single channel

• Navigate using 4 way

– Left & right to browse channel (infinite scroll)

– Up & down to change channels

39yahooresearchberkeley.comRahul Nair

Detail view

• Enlarge photo

– Scroll through channel

• Photo details

• Add comments

• Add to favourites

40yahooresearchberkeley.comRahul Nair

Nearby photos channel

• Photos from the users current location

• Local Highlights

• My Nearby

41yahooresearchberkeley.comRahul Nair

Social Channels

• Contacts photos

– Expanded view

• Recent Activity

42yahooresearchberkeley.comRahul Nair

My Stuff Channel

• My Photos

– By Location

– By Tag

• My Photo Wallet

• My Favourites

43yahooresearchberkeley.comRahul Nair

Custom Channels

• Users can create channels to match their interests

– Tags

– Groups

– Location aware (dogs near me)

44yahooresearchberkeley.comRahul Nair

Search and Filter

45yahooresearchberkeley.comRahul Nair

Try it out

http://zurfer.research.yahoo.com

46yahooresearchberkeley.comRahul Nair

Conclusions

• It is possible to extract information from georeferenced media

• Users like browsing the extracted data

• It can help users tag new media

• We hope it helps them browse on a mobile device

47

Questions?

Rahul Nairrnair@yahoo-inc.com

http://tagmaps.research.yahoo.comhttp://zonetag.research.yahoo.com http://zurfer.research.yahoo.com