Inferring Topical Attributes of Users in the Twitter …...Inferring Topical Attributes of Users in...
Transcript of Inferring Topical Attributes of Users in the Twitter …...Inferring Topical Attributes of Users in...
![Page 1: Inferring Topical Attributes of Users in the Twitter …...Inferring Topical Attributes of Users in the Twitter Social Network Saptarshi Ghosh Post-doctoral researcher Max Planck Institute](https://reader031.fdocuments.us/reader031/viewer/2022040519/5e7798d568c761423b1b7495/html5/thumbnails/1.jpg)
Inferring Topical Attributes of Users in the Twitter Social Network
Saptarshi Ghosh Post-doctoral researcher Max Planck Institute for Software Systems, Germany socialnetworks.mpi-sws.org Dynamics On and Of Complex Networks 2014
![Page 2: Inferring Topical Attributes of Users in the Twitter …...Inferring Topical Attributes of Users in the Twitter Social Network Saptarshi Ghosh Post-doctoral researcher Max Planck Institute](https://reader031.fdocuments.us/reader031/viewer/2022040519/5e7798d568c761423b1b7495/html5/thumbnails/2.jpg)
Joint work with … MPI-SWS, Germany
n Md. Bilal Zafar
n Juhi Kulshrestha
n Mainack Mondal
n Krishna Gummadi
IIT Kharagpur, India
n Parantapa Bhattacharya
n Naveen Sharma (currently at Univ. Washington)
n Niloy Ganguly
![Page 3: Inferring Topical Attributes of Users in the Twitter …...Inferring Topical Attributes of Users in the Twitter Social Network Saptarshi Ghosh Post-doctoral researcher Max Planck Institute](https://reader031.fdocuments.us/reader031/viewer/2022040519/5e7798d568c761423b1b7495/html5/thumbnails/3.jpg)
Profile of a Twitter user
![Page 4: Inferring Topical Attributes of Users in the Twitter …...Inferring Topical Attributes of Users in the Twitter Social Network Saptarshi Ghosh Post-doctoral researcher Max Planck Institute](https://reader031.fdocuments.us/reader031/viewer/2022040519/5e7798d568c761423b1b7495/html5/thumbnails/4.jpg)
Twitter social network n u à v: user u follows user v
n u subscribes to the tweets posted by v
n u is a follower of v
n v is a following of u
![Page 5: Inferring Topical Attributes of Users in the Twitter …...Inferring Topical Attributes of Users in the Twitter Social Network Saptarshi Ghosh Post-doctoral researcher Max Planck Institute](https://reader031.fdocuments.us/reader031/viewer/2022040519/5e7798d568c761423b1b7495/html5/thumbnails/5.jpg)
Twitter – an information network
n “We’re not a social network, we’re an information network” – Michael Abbott, VP Engineering, Twitter
n Twitter slogan q What are you doing? à changed to à q Discover what’s happening right now, anywhere in the world
n Millions of users rely on Twitter to discover real-time content on various topics
![Page 6: Inferring Topical Attributes of Users in the Twitter …...Inferring Topical Attributes of Users in the Twitter Social Network Saptarshi Ghosh Post-doctoral researcher Max Planck Institute](https://reader031.fdocuments.us/reader031/viewer/2022040519/5e7798d568c761423b1b7495/html5/thumbnails/6.jpg)
Discovering information in Twitter n Plethora of information
q Over 600 million users q Over 400 million tweets posted daily
n Infeasible for an individual user to discover interesting information on her own
![Page 7: Inferring Topical Attributes of Users in the Twitter …...Inferring Topical Attributes of Users in the Twitter Social Network Saptarshi Ghosh Post-doctoral researcher Max Planck Institute](https://reader031.fdocuments.us/reader031/viewer/2022040519/5e7798d568c761423b1b7495/html5/thumbnails/7.jpg)
Our research
n Objective: enable Twitter users find information relevant to their interests
n How to locate interesting information on a topic?
q Identify topical experts
n Who should be provided with information on a topic? q Identify users who are interested in the topic
![Page 8: Inferring Topical Attributes of Users in the Twitter …...Inferring Topical Attributes of Users in the Twitter Social Network Saptarshi Ghosh Post-doctoral researcher Max Planck Institute](https://reader031.fdocuments.us/reader031/viewer/2022040519/5e7798d568c761423b1b7495/html5/thumbnails/8.jpg)
Identifying topical experts Inferring topical interests of users Application: Interaction among experts and interested users
![Page 9: Inferring Topical Attributes of Users in the Twitter …...Inferring Topical Attributes of Users in the Twitter Social Network Saptarshi Ghosh Post-doctoral researcher Max Planck Institute](https://reader031.fdocuments.us/reader031/viewer/2022040519/5e7798d568c761423b1b7495/html5/thumbnails/9.jpg)
Features used by prior approaches
![Page 10: Inferring Topical Attributes of Users in the Twitter …...Inferring Topical Attributes of Users in the Twitter Social Network Saptarshi Ghosh Post-doctoral researcher Max Planck Institute](https://reader031.fdocuments.us/reader031/viewer/2022040519/5e7798d568c761423b1b7495/html5/thumbnails/10.jpg)
Our proposal …
n Utilize social annotations to infer topics of expertise of popular Twitter users q How does the Twitter crowd describe a user?
n Social annotations obtained through Twitter Lists q A feature by which one can create a named group
containing some group related users
![Page 11: Inferring Topical Attributes of Users in the Twitter …...Inferring Topical Attributes of Users in the Twitter Social Network Saptarshi Ghosh Post-doctoral researcher Max Planck Institute](https://reader031.fdocuments.us/reader031/viewer/2022040519/5e7798d568c761423b1b7495/html5/thumbnails/11.jpg)
How Lists work ?
![Page 12: Inferring Topical Attributes of Users in the Twitter …...Inferring Topical Attributes of Users in the Twitter Social Network Saptarshi Ghosh Post-doctoral researcher Max Planck Institute](https://reader031.fdocuments.us/reader031/viewer/2022040519/5e7798d568c761423b1b7495/html5/thumbnails/12.jpg)
Using Lists to infer topics for users
n If user u is an expert / authority in a certain topic q u is likely to be included in several Lists q List names / descriptions provide valuable semantic cues
to the topics of expertise of u
![Page 13: Inferring Topical Attributes of Users in the Twitter …...Inferring Topical Attributes of Users in the Twitter Social Network Saptarshi Ghosh Post-doctoral researcher Max Planck Institute](https://reader031.fdocuments.us/reader031/viewer/2022040519/5e7798d568c761423b1b7495/html5/thumbnails/13.jpg)
Topics inferred from Lists
linux, tech, open, software, libre, gnu, computer, developer, ubuntu, unix
politics, senator, congress, government, republicans, Iowa, gop, conservative
celebs, actors, famous, movies, comedy, funny, music, hollywood, pop culture
![Page 14: Inferring Topical Attributes of Users in the Twitter …...Inferring Topical Attributes of Users in the Twitter Social Network Saptarshi Ghosh Post-doctoral researcher Max Planck Institute](https://reader031.fdocuments.us/reader031/viewer/2022040519/5e7798d568c761423b1b7495/html5/thumbnails/14.jpg)
Who-is-who service for Twitter http://twitter-app.mpi-sws.org/who-is-who/
Inferring Who-is-who in the Twitter Social Network ACM Workshop on Online Social Networks 2012, ACM Computer Communication Review 2012
![Page 15: Inferring Topical Attributes of Users in the Twitter …...Inferring Topical Attributes of Users in the Twitter Social Network Saptarshi Ghosh Post-doctoral researcher Max Planck Institute](https://reader031.fdocuments.us/reader031/viewer/2022040519/5e7798d568c761423b1b7495/html5/thumbnails/15.jpg)
Identify topical experts: methodology
n Given a query (topic)
n Identify experts on the topic using Lists
n Rank identified experts w.r.t. given topic q Relevance of expert to topic q Popularity of expert
![Page 16: Inferring Topical Attributes of Users in the Twitter …...Inferring Topical Attributes of Users in the Twitter Social Network Saptarshi Ghosh Post-doctoral researcher Max Planck Institute](https://reader031.fdocuments.us/reader031/viewer/2022040519/5e7798d568c761423b1b7495/html5/thumbnails/16.jpg)
Cognos n Search system for topical experts in Twitter
http://twitter-app.mpi-sws.org/whom-to-follow/
Cognos: Crowdsourcing Search for Topic Experts in Microblogs, ACM SIGIR Conference 2012
![Page 17: Inferring Topical Attributes of Users in the Twitter …...Inferring Topical Attributes of Users in the Twitter Social Network Saptarshi Ghosh Post-doctoral researcher Max Planck Institute](https://reader031.fdocuments.us/reader031/viewer/2022040519/5e7798d568c761423b1b7495/html5/thumbnails/17.jpg)
Cognos results for “politics”
![Page 18: Inferring Topical Attributes of Users in the Twitter …...Inferring Topical Attributes of Users in the Twitter Social Network Saptarshi Ghosh Post-doctoral researcher Max Planck Institute](https://reader031.fdocuments.us/reader031/viewer/2022040519/5e7798d568c761423b1b7495/html5/thumbnails/18.jpg)
Cognos results for “stem cell”
![Page 19: Inferring Topical Attributes of Users in the Twitter …...Inferring Topical Attributes of Users in the Twitter Social Network Saptarshi Ghosh Post-doctoral researcher Max Planck Institute](https://reader031.fdocuments.us/reader031/viewer/2022040519/5e7798d568c761423b1b7495/html5/thumbnails/19.jpg)
Evaluation of Cognos q Cognos compared with Twitter Who To Follow
q Evaluator shown top 10 results (anonymized) by both systems q Evaluator judges which is better / both good / both bad q Judgment by majority voting
q Cognos results judged equally good or better for 60% topics
![Page 20: Inferring Topical Attributes of Users in the Twitter …...Inferring Topical Attributes of Users in the Twitter Social Network Saptarshi Ghosh Post-doctoral researcher Max Planck Institute](https://reader031.fdocuments.us/reader031/viewer/2022040519/5e7798d568c761423b1b7495/html5/thumbnails/20.jpg)
Identifying topical experts Inferring topical interests of users Application: Interaction among experts and interested users
![Page 21: Inferring Topical Attributes of Users in the Twitter …...Inferring Topical Attributes of Users in the Twitter Social Network Saptarshi Ghosh Post-doctoral researcher Max Planck Institute](https://reader031.fdocuments.us/reader031/viewer/2022040519/5e7798d568c761423b1b7495/html5/thumbnails/21.jpg)
Inferring Topics of Interest of a User n Prior attempts
q Use NLP on tweets posted by a user or received by a user q Limitation: tweets often contain day-to-day conversation
n Our approach
q If a user is subscribing to several experts on a topic, she is very likely to be interested in that topic
q Identify which topical experts a user follows, and infer the topics of expertise of those experts
![Page 22: Inferring Topical Attributes of Users in the Twitter …...Inferring Topical Attributes of Users in the Twitter Social Network Saptarshi Ghosh Post-doctoral researcher Max Planck Institute](https://reader031.fdocuments.us/reader031/viewer/2022040519/5e7798d568c761423b1b7495/html5/thumbnails/22.jpg)
Who likes what n System to infer topical interests of users in Twitter
http://twitter-app.mpi-sws.org/who-likes-what/
Inferring User Interests in the Twitter Social Network, ACM Conference on Recommender Systems (RecSys) 2014
![Page 23: Inferring Topical Attributes of Users in the Twitter …...Inferring Topical Attributes of Users in the Twitter Social Network Saptarshi Ghosh Post-doctoral researcher Max Planck Institute](https://reader031.fdocuments.us/reader031/viewer/2022040519/5e7798d568c761423b1b7495/html5/thumbnails/23.jpg)
Topical interests of Lada Adamic
![Page 24: Inferring Topical Attributes of Users in the Twitter …...Inferring Topical Attributes of Users in the Twitter Social Network Saptarshi Ghosh Post-doctoral researcher Max Planck Institute](https://reader031.fdocuments.us/reader031/viewer/2022040519/5e7798d568c761423b1b7495/html5/thumbnails/24.jpg)
Topical interests of Duncan Watts
![Page 25: Inferring Topical Attributes of Users in the Twitter …...Inferring Topical Attributes of Users in the Twitter Social Network Saptarshi Ghosh Post-doctoral researcher Max Planck Institute](https://reader031.fdocuments.us/reader031/viewer/2022040519/5e7798d568c761423b1b7495/html5/thumbnails/25.jpg)
Evaluation of the approach n Accuracy
q Compared with topic modeling (Labeled LDA) on tweets posted / received by a user
q Topics inferred by proposed approach judged significantly better by human volunteers
n Scalability
q Enables inference of interests of more than 77% of all users in Twitter
![Page 26: Inferring Topical Attributes of Users in the Twitter …...Inferring Topical Attributes of Users in the Twitter Social Network Saptarshi Ghosh Post-doctoral researcher Max Planck Institute](https://reader031.fdocuments.us/reader031/viewer/2022040519/5e7798d568c761423b1b7495/html5/thumbnails/26.jpg)
Topical interests of Duncan Watts
Interests not same as expertise
Topical expertise of Duncan Watts
![Page 27: Inferring Topical Attributes of Users in the Twitter …...Inferring Topical Attributes of Users in the Twitter Social Network Saptarshi Ghosh Post-doctoral researcher Max Planck Institute](https://reader031.fdocuments.us/reader031/viewer/2022040519/5e7798d568c761423b1b7495/html5/thumbnails/27.jpg)
Identifying topical experts Inferring topical interests of users Application: Interaction among experts and interested users
![Page 28: Inferring Topical Attributes of Users in the Twitter …...Inferring Topical Attributes of Users in the Twitter Social Network Saptarshi Ghosh Post-doctoral researcher Max Planck Institute](https://reader031.fdocuments.us/reader031/viewer/2022040519/5e7798d568c761423b1b7495/html5/thumbnails/28.jpg)
Topical groups in Twitter Topical Groups = Experts + Seekers
Experts: Users who have expertise on the topic Seekers: Users who are interested in the topic
@BarackObama Expert on Politics
@BarackObama
Seeker on Basketball
![Page 29: Inferring Topical Attributes of Users in the Twitter …...Inferring Topical Attributes of Users in the Twitter Social Network Saptarshi Ghosh Post-doctoral researcher Max Planck Institute](https://reader031.fdocuments.us/reader031/viewer/2022040519/5e7798d568c761423b1b7495/html5/thumbnails/29.jpg)
Identifying topical groups at scale n Crawled data for first 38 million users in Twitter
n 88 million Lists, 1.5 billion social links
n Identified 36 thousand topical groups
Deep Twitter Diving: Exploring Topical Groups in Microblogs at Scale, ACM Conference on Computer Supported Cooperative Work and Social Computing (CSCW) 2014
![Page 30: Inferring Topical Attributes of Users in the Twitter …...Inferring Topical Attributes of Users in the Twitter Social Network Saptarshi Ghosh Post-doctoral researcher Max Planck Institute](https://reader031.fdocuments.us/reader031/viewer/2022040519/5e7798d568c761423b1b7495/html5/thumbnails/30.jpg)
Diversity: Topics and Group Size
![Page 31: Inferring Topical Attributes of Users in the Twitter …...Inferring Topical Attributes of Users in the Twitter Social Network Saptarshi Ghosh Post-doctoral researcher Max Planck Institute](https://reader031.fdocuments.us/reader031/viewer/2022040519/5e7798d568c761423b1b7495/html5/thumbnails/31.jpg)
A Small Number of Very Popular Groups
![Page 32: Inferring Topical Attributes of Users in the Twitter …...Inferring Topical Attributes of Users in the Twitter Social Network Saptarshi Ghosh Post-doctoral researcher Max Planck Institute](https://reader031.fdocuments.us/reader031/viewer/2022040519/5e7798d568c761423b1b7495/html5/thumbnails/32.jpg)
Thousands of Specialized Niche Groups
![Page 33: Inferring Topical Attributes of Users in the Twitter …...Inferring Topical Attributes of Users in the Twitter Social Network Saptarshi Ghosh Post-doctoral researcher Max Planck Institute](https://reader031.fdocuments.us/reader031/viewer/2022040519/5e7798d568c761423b1b7495/html5/thumbnails/33.jpg)
Detecting topical groups n We followed a semantic approach to identify topical
groups
n Could community detection algorithms be used on the social network to detect them?
n Applied BGLL / Louvain algorithm on the Twitter social network to identify communities
![Page 34: Inferring Topical Attributes of Users in the Twitter …...Inferring Topical Attributes of Users in the Twitter Social Network Saptarshi Ghosh Post-doctoral researcher Max Planck Institute](https://reader031.fdocuments.us/reader031/viewer/2022040519/5e7798d568c761423b1b7495/html5/thumbnails/34.jpg)
Detecting topical groups
n BGLL largely unable to detect topical groups, especially the smaller ones (on niche topics)
n Topical groups do not have good structural quality
on which most community detection algorithms rely
n Difference explained by a sociological theory on how groups form in a social network
![Page 35: Inferring Topical Attributes of Users in the Twitter …...Inferring Topical Attributes of Users in the Twitter Social Network Saptarshi Ghosh Post-doctoral researcher Max Planck Institute](https://reader031.fdocuments.us/reader031/viewer/2022040519/5e7798d568c761423b1b7495/html5/thumbnails/35.jpg)
Why do groups form?
Identity Based Groups
Low Reciprocity
Low Personal Interactions High Topicality of discussions
Examples: audience at a conference, topical groups
Bond Based Groups
High Reciprocity
High Personal Interactions Low Topicality of discussions
Examples: family, personal friends
Common Identity and Bond Theory - Prentice, Personality and Social Psychology Bulletin, 1994
![Page 36: Inferring Topical Attributes of Users in the Twitter …...Inferring Topical Attributes of Users in the Twitter Social Network Saptarshi Ghosh Post-doctoral researcher Max Planck Institute](https://reader031.fdocuments.us/reader031/viewer/2022040519/5e7798d568c761423b1b7495/html5/thumbnails/36.jpg)
Summary n How to identify topical expertise / interests of users?
q Proposed crowdsourcing based methods
n Identified topical groups in Twitter at scale
n Ongoing work q Developing search / recommender systems to help users
find important content relevant to their interests
Thank You !