July 2, 2009 1

Connecting Content to Community in

Social Media

Munmun De Choudhury1, Hari Sundaram1, Yu-Ru Lin1

Ajita John2 and Dorée Duncan Seligmann2

1Arts, Media and Engineering, Arizona State University2Avaya Labs Research, NJ

@ ICME 2009

Via image content, user tags and user communication

Photos Communities

July 2, 2009 2

The Problem

July 2, 2009 2@ ICME 2009

Photos Communities

Given an image, how do we recommend the relevant communities that

would enable social interaction and enhance its “reachability” to other

users?

July 2, 2009 3

Community contributed social media…

@ ICME 2009

Media-sharing

Concept-sharing

(tagging)

Communication

(commenting)

July 2, 2009 4@ ICME 2009

TravelPleasure

New York City

Long Exposure

Portraiture Photography

Canon 350DNature and Colors

July 2, 2009 5

Why is this problem challenging?

An empirical observation…

Photography Critique

New York City Landmarks & Icons

b&w new york city (& the people in it)

?

?

?

@ ICME 2009

July 2, 2009 6@ ICME 2009

Simple keyword search returns

too many groups!

July 2, 2009 7

visit the world - the travel guide

10 Million Photos

Photography Critique

Flickr Central

New York City Landmarks & Icons

b&w new york city (& the people in it)“Most relevant”

“Most recent activity”

“Group size”

@ ICME 2009

July 2, 2009 8

Quality of content?

Community practices?

Richness of communication?

@ ICME 2009

July 2, 2009 9

I want to find the right set of

communities for my photos… help!

July 2, 2009 9@ ICME 2009

Photos Communities

July 2, 2009 10

Approach

July 2, 2009 10@ ICME 2009

July 2, 2009 11

System Overview

@ ICME 2009

July 2, 2009 12

1 2 3, ,

i i i iα α α =

f C T O� � ��

Feature Extraction

Image content features –

color histogram, color moment, GLCM, phase

symmetry, radial symmetry, phase congruency,

SIFT, Ci∈ℜ+1×m

User tagging history features –

frequency distribution of user tags in the past,

Ti∈ℜ+1×n

User communication features –

frequency of comments by the user on various

groups, Oi∈ℜ+1×p

@ ICME 2009

July 2, 2009 13

1 2 3, ,

i i i iα α α =

f C T O� � ��

@ ICME 2009

latent state, zi group, Gj,1

j

j i

i GjG ∈

= ∑Y f

( ) ( )

( ) ( ) ( ) ( )

( ) ( )

| ,

, . | . |

| . |

j j

j j

z

j

z

P G i P G i

P G i P z P i z P G z

P z i P G z

∝

=

∝

∑

∑

Probability of recommending group Gj to image i:

Model learning

Folding-in technique

Image-group association often depends upon latent intent of the user – based on

content, her prior tagging activity or communication…

July 2, 2009 14@ ICME 2009

Model Learning –

maximize the expectation of the feature space Fm of the groups Gj

Folding-in Technique –

for a new image i, maximize the expectation of the feature space Fm given the

latent state z

( )( ) ( ) ( )( ) ( ) ( )

( ) ( )

( ) ( )

( ) ( )

'

,

,

,

. | . |-step: | , .

' . | ' . | '

-step: | . | , ,

| . | , ,

. | , .

m j

m j

m j

z

j m j m j

m

m m j m j

j

m j m j

m j

P z P F z P G zE P z F G

P z P F z P G z

M P G z P z F G

P F z P z F G

P z P z F G

=

∝

∝

∝

∑

∑

∑

∑∑

Y

Y

Y

( ) ( ) ( )

( ) ( ),

-step: | , | . | .

-step: | . | , ,

m m

m i m

m

E P z i F P F z P z i

M P z i P z i F

∝

∝∑Z

July 2, 2009 15

Experimental Results

July 2, 2009 15@ ICME 2009

July 2, 2009 16

Flickr Dataset

� Seed crawling from images ranked by Flickr’s proprietary “interestingness”

criterion.

� 15,689 images

� 925 groups

� Upload time period from March 21, 2008 to August 20, 2008

14Mean number of comments per photo

3Mean number of groups per photo

6Mean number of tags per photo

@ ICME 2009

July 2, 2009 17

Evaluation of Recommendations

Experimental Setup:

• 80% training (12,551 images) , 20% testing (3,138 images)

• Metrics are computed at k=1, 3, 10

• Three recommendation analysis scenarios:

• Image-based

• User-based

• Group-based

@ ICME 2009

1 3 10 1 3 10 1 3 10

July 2, 2009 18

Evaluation of Recommendations

@ ICME 2009

1 3 10 1 3 10 1 3 10

1 3 10 1 3 10 1 3 10

July 2, 2009 19

Evaluation of Feature Categories

@ ICME 2009

Significant improvement in recall over tag

only featuresSignificant improvement in precision over

communication only features

July 2, 2009 20

Conclusions

July 2, 2009 20@ ICME 2009

Photos Communities

July 2, 2009 21

Problem:

How do we connect shared media to relevant

communities?

Approach:

Characterize relationship of

images to communities via

content, tags and communication

Results:

Meaningful recommendations of

communities @ ICME 2009

Photos Communities

July 2, 2009 22July 2, 2009 22

Munmun.Dechoudhury@asu.edu

Thanks!

Questions? Suggestions?

July 2, 2009 22@ ICME 2009

July 2, 2009 23

Related Work

� Collective learning

• Recommendation of books, movies; Schein, Popescul, Ungar et al 2002.

• Dynamic evolution of the properties of the items not considered.

� Social media group and tag recommendation

• Sigurbjornsson & Zwol 2008, Garg & Weber 2008, Chen, Chang, Chang et al

2008.

• No consideration of social interaction.

� User behavior in social media groups

• Negoescu & Gatica-Perez 2008.

• Connection between media content and group behavior not explicit.

@ ICME 2009