A Computational Analysis of Agenda Setting Theory
-
Upload
alice-oh -
Category
News & Politics
-
view
745 -
download
3
Transcript of A Computational Analysis of Agenda Setting Theory
![Page 2: A Computational Analysis of Agenda Setting Theory](https://reader034.fdocuments.us/reader034/viewer/2022042607/55515f74b4c9059f768b4d43/html5/thumbnails/2.jpg)
Our Research Overview
• Topic Modeling (Machine Learning)
• CIKM 2011: Distance-dependent Chinese restaurant franchise (ddCRF)
• ICML 2012: Dirichlet process with random mixed measures (DP-MRM)
• CIKM 2012: Recursive chinese restaurant process for modeling topic hierarchies (rCRP)
• NIPS Big Learning Workshop 2012: Distributed Online Learning for Latent Dirichlet Allocation (DoLDA)
• IJCAI 2013: Context Dependent Conceptualization (at MSRA)
• Computational Social Science
• WSDM 2011: Aspect sentiment unification model for online review analysis
• ICWSM 2012: Social aspects of emotions in Twitter conversations
• ACL 2012: Self-disclosure and relationship strength in Twitter conversations
• AAAI 2013: Hierarchical Aspect Sentiment Model for Online Reviews (at MSRA)
2
![Page 3: A Computational Analysis of Agenda Setting Theory](https://reader034.fdocuments.us/reader034/viewer/2022042607/55515f74b4c9059f768b4d43/html5/thumbnails/3.jpg)
Agenda Setting Theory How does media affect the thoughts of the audience?
![Page 4: A Computational Analysis of Agenda Setting Theory](https://reader034.fdocuments.us/reader034/viewer/2022042607/55515f74b4c9059f768b4d43/html5/thumbnails/4.jpg)
Agenda Setting Theory (McCombs & Shaw, 1972)
• Media affects audiences by having an influence on
• What to think about
• How to think about it
• Examples of traditional media studies
• Media affects the outcome of presidential elections (Perloff and Krauss, 1985)
• Media coverage influences the control of infectious diseases (Cui et al., 2008)
• Tone of news articles affects the number of visitors to museums (Zyglidopoulos et al., 2012)
![Page 5: A Computational Analysis of Agenda Setting Theory](https://reader034.fdocuments.us/reader034/viewer/2022042607/55515f74b4c9059f768b4d43/html5/thumbnails/5.jpg)
1.Use of traditional off-line newspapers and TV as target media
• Analysis is limited to a small volume over a short duration
• Issues are arbitrarily chosen
2.Use of off-line MIP (Most Important Problems) surveys
• Self-reports are not reliable
• Only a small subset of the population can be surveyed
3.Use of manual coding for content analysis
• You need experts
• It is difficult to replicate and generalize to other domains
Limitation of Traditional Media Studies
![Page 6: A Computational Analysis of Agenda Setting Theory](https://reader034.fdocuments.us/reader034/viewer/2022042607/55515f74b4c9059f768b4d43/html5/thumbnails/6.jpg)
Computational Analysis of Agenda Setting Theory
1.Use of traditional off-line newspapers and TV as target media
• Crawl online news to get several years’ data
• Use machine learning to automatically discover the important issues
2.Use of off-line MIP (Most Important Problems) surveys
• Look at counts of social media shares
• Look at counts of user comments
3.Use of manual coding for content analysis
• Use unsupervised machine learning to analyze content for tone (polarity) of articles and comments
• Try it for different issues to see whether ML approach can generalize over many domains
![Page 7: A Computational Analysis of Agenda Setting Theory](https://reader034.fdocuments.us/reader034/viewer/2022042607/55515f74b4c9059f768b4d43/html5/thumbnails/7.jpg)
7
Gay marriage COMMENT
SHARE
AUDIENCE’S BEHAVIOR
![Page 8: A Computational Analysis of Agenda Setting Theory](https://reader034.fdocuments.us/reader034/viewer/2022042607/55515f74b4c9059f768b4d43/html5/thumbnails/8.jpg)
7
Gay marriage COMMENT
SHARE
AUDIENCE’S BEHAVIOR
![Page 9: A Computational Analysis of Agenda Setting Theory](https://reader034.fdocuments.us/reader034/viewer/2022042607/55515f74b4c9059f768b4d43/html5/thumbnails/9.jpg)
8
Section #Articles #Comments #Commenters #Shares
Politics 1,863 174,680 14,106 2,080,889
Business 2,043 130,921 17,791 3,657,544
Opinion 4,820 149,618 30,556 6,620,489
Sports 814 17,282 5,484 712,507
Technology 456 13,571 4,993 570,732
Science 945 50,113 11,114 4,709,041
World 3,673 134,572 14,882 3,534,637
Health 3,060 92,964 18,185 6,001,082
Total 17,674 763,721 117,111 27,886,921
From http://www.npr.org/
2011.01 – 2013.04
DATA STATISTICS
![Page 10: A Computational Analysis of Agenda Setting Theory](https://reader034.fdocuments.us/reader034/viewer/2022042607/55515f74b4c9059f768b4d43/html5/thumbnails/10.jpg)
9
Section Issue (Labeled by using Mturk) #Articles
Politics presidential electioninfringement of human rightsrace for Washingtongovernment economics presidential campaigns and money candidate-marriage & immigration political viewpoints
575195167274163261157
Business economic decline under Obamaemployment and paid slavery agriculturebanks and loan stock market and business housing markettax and businessenergy and finance new business and running
514218131198166170180222138
Health health care reform laws vaccinationHIV and treatment medication healthcare and costs food and obesitysleep study and children food and safety health tech and new treatment mental health in families
349189496197224245210223125117
Issue Detection using HDP
Detected Issue list and the number of articles of each issue for three sections out of eight sections.
![Page 11: A Computational Analysis of Agenda Setting Theory](https://reader034.fdocuments.us/reader034/viewer/2022042607/55515f74b4c9059f768b4d43/html5/thumbnails/11.jpg)
Correlation between the volume of articles (per week) and audience’s interests (following comments). Correlation value for (a) is 0.786 and it shows strong agenda setting effects., also correlation value for (b) is 0.418 and it shows weak agenda setting effects.
10
▶ Effects from media exposure CORRELATION IN ISSUE
Jan 01 2011
![Page 12: A Computational Analysis of Agenda Setting Theory](https://reader034.fdocuments.us/reader034/viewer/2022042607/55515f74b4c9059f768b4d43/html5/thumbnails/12.jpg)
11
Section Keywords Issue (Labeled by using Mturk) commentcomment shareshareSection Keywords Issue (Labeled by using Mturk)
corr. effect size corr. effect size
Politics romney gingrich republican santorum president obama house people political state republican election party walker president obama tax house congress romney campaign obama money million court law state ice supreme marriage romney obama president republican voters
presidential electioninfringement of human rightsrace for Washingtongovernment economics presidential campaigns and money candidate-marriage & immigration political viewpoints
0.873**** 0.845**** 0.855**** 0.903**** 0.836**** 0.895**** 0.878****
LargeLargeLargeLargeLargeLargeLarge
0.161 0.562**** 0.511**** 0.347*** 0.367** 0.417** 0.372*
noneLargeLargeMediumMediumMediumMedium
Business percent economy year jobs debt rate tax people job can work time jobs year yearsfood farmers year beer corn prices new bank banks financial money new company news new company people can stock nowpeople city new can home like now housing tax can people state new like year get new oil gas company american car industry like can people new company get year
economic decline under Obamaemployment and paid slavery agriculturebanks and loan stock market and business housing markettax and businessenergy and finance new business and running
0.870**** 0.732**** 0.634**** 0.786**** 0.736**** 0.670**** 0.767**** 0.702**** 0.750****
LargeLargeLargeLargeLargeLargeLargeLargeLarge
0.304*** 0.346*** 0.230* 0.268** 0.441** 0.360* 0.278** 0.423** 0.278**
MediumMediumSmallSmallMediumMediumSmallMediumSmall
Health health law care insurance people federalhealth people vaccine virus new flu cancer women people percent risk hiv drug drugs fda people can new patients care health patients hospital hospitals food people can health new like weight can people study sleep kids children food can people health like may new get can patients people cancer brain new people can like life health get know
health care reform laws vaccinationHIV and treatment medication healthcare and costs food and obesitysleep study and children food and safety health tech and new treatment mental health in families
0.564**** 0.640**** 0.399* 0.447** 0.706**** 0.702**** 0.541**** 0.428** 0.544**** 0.418**
LargeLargeMediumMediumLargeLargeLargeMediumLargeMediu,m
0.241** 0.341*** 0.279** 0.149 0.615**** 0.162* 0.456** 0.330*** 0.172 0.360*
SmallMediumSmallnoneLargeSmallMediumMediumnoneMedium
CORRELATION & EFFECT SIZE
![Page 13: A Computational Analysis of Agenda Setting Theory](https://reader034.fdocuments.us/reader034/viewer/2022042607/55515f74b4c9059f768b4d43/html5/thumbnails/13.jpg)
Q: Why do we see larger agenda setting effects for some issues?H: Previous studies argue for a relationship between agenda setting and relevance and/or uncertainty.
Previous research in media studies argue that larger agenda setting affects can be seen for issues with
• Low relevance and High uncertainty (Schonbach & Weaver, 1985)
• High relevance and High uncertainty (McCombs, 2004)
12
RELEVANCE & UNCERTAINTY
Relevance: The relevance of an issue to the audienceUncertainty: The degree of uncertainty by the audience about the issue
![Page 14: A Computational Analysis of Agenda Setting Theory](https://reader034.fdocuments.us/reader034/viewer/2022042607/55515f74b4c9059f768b4d43/html5/thumbnails/14.jpg)
GOALMeasure the average relevance and uncertainty of each issue to analyze the correlation to the level of agenda setting effect
EXPERIMENT SETTINGS2 sets of randomly distributed issues(each set contains 27 issues)
PARTICIPANTS- 26 American MTurkers, 13 for each set - Various ages from 20s to 50s
13
Measuring Relevance & Uncertainty with MTurk
![Page 15: A Computational Analysis of Agenda Setting Theory](https://reader034.fdocuments.us/reader034/viewer/2022042607/55515f74b4c9059f768b4d43/html5/thumbnails/15.jpg)
14
DEFINITION
ABOUT SECTION
PERSONAL INFO.
MTurk Questionnaires -1
![Page 16: A Computational Analysis of Agenda Setting Theory](https://reader034.fdocuments.us/reader034/viewer/2022042607/55515f74b4c9059f768b4d43/html5/thumbnails/16.jpg)
15
ABOUT ISSUE
Mturk Questionnaires -2
![Page 17: A Computational Analysis of Agenda Setting Theory](https://reader034.fdocuments.us/reader034/viewer/2022042607/55515f74b4c9059f768b4d43/html5/thumbnails/17.jpg)
15
ABOUT ISSUE
Mturk Questionnaires -2
![Page 18: A Computational Analysis of Agenda Setting Theory](https://reader034.fdocuments.us/reader034/viewer/2022042607/55515f74b4c9059f768b4d43/html5/thumbnails/18.jpg)
16
Issues
Relevance
Uncertainty
![Page 19: A Computational Analysis of Agenda Setting Theory](https://reader034.fdocuments.us/reader034/viewer/2022042607/55515f74b4c9059f768b4d43/html5/thumbnails/19.jpg)
17
Results of MTurk
1
2
3
4
5
1 2 3 4 5
Unc
erta
inty
Relevance
20s
Internet and privacy 1
2
3
4
5
1 2 3 4 5
30s-50s Issue
world and football
Internet and privacy
1
2
3
4
5
1 2 3 4 5
60s
world and football world and football
Internet and privacy
![Page 20: A Computational Analysis of Agenda Setting Theory](https://reader034.fdocuments.us/reader034/viewer/2022042607/55515f74b4c9059f768b4d43/html5/thumbnails/20.jpg)
18
Correlations with Agenda Setting Effect
share_corr = 0.480* Relevance
1
2
3
4
5
1 2 3 4 5
Unc
erta
inty
Relevance
Set A Issue
![Page 21: A Computational Analysis of Agenda Setting Theory](https://reader034.fdocuments.us/reader034/viewer/2022042607/55515f74b4c9059f768b4d43/html5/thumbnails/21.jpg)
19
Correlations with Agenda Setting Effect
share_corr = 0.524* Relevance
1
2
3
4
5
1 2 3 4 5
30s-50s Issue
![Page 22: A Computational Analysis of Agenda Setting Theory](https://reader034.fdocuments.us/reader034/viewer/2022042607/55515f74b4c9059f768b4d43/html5/thumbnails/22.jpg)
INFLUENTIAL FACTOR Tone (Polarity) of article
GOALIdentify the effects of article tone, positive and negative, on the commenting and sharing behaviors of the audience
20
Content Polarity & Audience Behavior
![Page 23: A Computational Analysis of Agenda Setting Theory](https://reader034.fdocuments.us/reader034/viewer/2022042607/55515f74b4c9059f768b4d43/html5/thumbnails/23.jpg)
21
Positive and Negative Articles
Proportion of sharing behavior to commenting behavior. Audience tends to leave more comments on negative article set, on the other hand, audience shares more articles in positive article set.
![Page 24: A Computational Analysis of Agenda Setting Theory](https://reader034.fdocuments.us/reader034/viewer/2022042607/55515f74b4c9059f768b4d43/html5/thumbnails/24.jpg)
22
DETECTED POS./NEG. WORDS
The sets of positive and negative words obtained from model analysis for news articles. Words depending on sections differentiate positive and negative traits of each section.
BUSINESS HEALTH OPINION POLITICS Positive joined viral smoothly better balance respect forward empower fair moderate
Negative cutthroat axed lawsuit beating lose opposite battle unjust fuming sequester
Positive care respect admit clarify essential healthy repair benign hope repaired
Negative tough severe emergency affected risk dying war spitting tricks abnormal
Positive spectacular useful created prize confirm love sublime win confident mellow
Negative weird fog distressing slam doubted fail wrong fears slippery peril
Positive expert forward proud consent carol rights great worth integrity truth
Negative ironic heinous arguing dick undo grinding outlaw meaningless theft lost
SCIENCE SPORTS TECHNOLOGY WORLD Positive fortunate cleanup essential credit safety comforting milestone learn gang dim
Negative spill crude busted upset concern problems dark smash prize creating
Positive victory won grace fun champion passion ace belief luck balance
Negative chase shock busted beating defeat thwart lost alleged assault cockeyed
Positive best fancy easy help intelligence strong improve fit trust fame
Negative blocks shabby shy wicked rash shaky mortal grave pity unfinished
Positive free respected support moderate consistent prompt afford gratitude joined affluent
Negative tension protest heavy raging slam war crime oppress poverty poor
![Page 25: A Computational Analysis of Agenda Setting Theory](https://reader034.fdocuments.us/reader034/viewer/2022042607/55515f74b4c9059f768b4d43/html5/thumbnails/25.jpg)
• Presented preliminary research in using computational methods for media studies
• Crawled a corpus of articles including user comments and social sharing counts from NPR over a period of three years
• Showed that sharing patterns and commenting patterns are quite different
• Showed the effects of agenda setting for 57 issues over 8 sections of NPR
• Looked at relevance and uncertainty as two dimensions to explain the various degree of agenda setting for different issues
• Looked at the tone of article (pos, neg) to see whether people react differently
• Identified lots of loose ends
• Please contact me if you are interested in collaborating
Contributions & Future Work
![Page 26: A Computational Analysis of Agenda Setting Theory](https://reader034.fdocuments.us/reader034/viewer/2022042607/55515f74b4c9059f768b4d43/html5/thumbnails/26.jpg)
Set ASet A Set BSet B
#Person Kappa #Person Kappa
Entire set 13 0.0436** 13 0.0192
20s 2 -0.225. 1 One person
30s 3 0.0228 6 0.00883
40s 2 0.0189 2 0.074
50s 4 -0.0382 3 -0.0588
60s 2 0.211. 1 One person
30s-50s 9 0.0448* 11 0.0162
Male 4 0.0158 7 0.0239
Female 9 0.0616** 6 -0.0048
24
DATA: AGREEMENT & SIGNIFICANCE
Calculate Fleiss’ Kappa value for each data set. .p<0.1, *p<0.05,**p<0.01, ***p<0.001,****p<0.0001
![Page 27: A Computational Analysis of Agenda Setting Theory](https://reader034.fdocuments.us/reader034/viewer/2022042607/55515f74b4c9059f768b4d43/html5/thumbnails/27.jpg)
25