TwitInfo: Aggregating and Visualizing Microblogs for Event ... · Journalist Interview...
Transcript of TwitInfo: Aggregating and Visualizing Microblogs for Event ... · Journalist Interview...
![Page 1: TwitInfo: Aggregating and Visualizing Microblogs for Event ... · Journalist Interview backgrounding eyewitnesses. 55 ui walkthrough algorithm design study/evaluation ... Sentiment](https://reader034.fdocuments.us/reader034/viewer/2022042106/5e85a0b5163cb23e1f3101d6/html5/thumbnails/1.jpg)
1
TwitInfo: Aggregating and Visualizing Microblogsfor Event Exploration
Adam Marcus, Michael Bernstein, Osama Badar,David Karger, Sam Madden, Rob Miller
MIT CSAIL
![Page 2: TwitInfo: Aggregating and Visualizing Microblogs for Event ... · Journalist Interview backgrounding eyewitnesses. 55 ui walkthrough algorithm design study/evaluation ... Sentiment](https://reader034.fdocuments.us/reader034/viewer/2022042106/5e85a0b5163cb23e1f3101d6/html5/thumbnails/2.jpg)
2
Tell stories using Twitter
![Page 3: TwitInfo: Aggregating and Visualizing Microblogs for Event ... · Journalist Interview backgrounding eyewitnesses. 55 ui walkthrough algorithm design study/evaluation ... Sentiment](https://reader034.fdocuments.us/reader034/viewer/2022042106/5e85a0b5163cb23e1f3101d6/html5/thumbnails/3.jpg)
3
![Page 4: TwitInfo: Aggregating and Visualizing Microblogs for Event ... · Journalist Interview backgrounding eyewitnesses. 55 ui walkthrough algorithm design study/evaluation ... Sentiment](https://reader034.fdocuments.us/reader034/viewer/2022042106/5e85a0b5163cb23e1f3101d6/html5/thumbnails/4.jpg)
4
![Page 5: TwitInfo: Aggregating and Visualizing Microblogs for Event ... · Journalist Interview backgrounding eyewitnesses. 55 ui walkthrough algorithm design study/evaluation ... Sentiment](https://reader034.fdocuments.us/reader034/viewer/2022042106/5e85a0b5163cb23e1f3101d6/html5/thumbnails/5.jpg)
5
too much
too little
![Page 6: TwitInfo: Aggregating and Visualizing Microblogs for Event ... · Journalist Interview backgrounding eyewitnesses. 55 ui walkthrough algorithm design study/evaluation ... Sentiment](https://reader034.fdocuments.us/reader034/viewer/2022042106/5e85a0b5163cb23e1f3101d6/html5/thumbnails/6.jpg)
6
Identify events of interest
![Page 7: TwitInfo: Aggregating and Visualizing Microblogs for Event ... · Journalist Interview backgrounding eyewitnesses. 55 ui walkthrough algorithm design study/evaluation ... Sentiment](https://reader034.fdocuments.us/reader034/viewer/2022042106/5e85a0b5163cb23e1f3101d6/html5/thumbnails/7.jpg)
7
Identify events of interestLabel events
![Page 8: TwitInfo: Aggregating and Visualizing Microblogs for Event ... · Journalist Interview backgrounding eyewitnesses. 55 ui walkthrough algorithm design study/evaluation ... Sentiment](https://reader034.fdocuments.us/reader034/viewer/2022042106/5e85a0b5163cb23e1f3101d6/html5/thumbnails/8.jpg)
8
Identify events of interestLabel eventsAdd context: sentiment? location?
![Page 9: TwitInfo: Aggregating and Visualizing Microblogs for Event ... · Journalist Interview backgrounding eyewitnesses. 55 ui walkthrough algorithm design study/evaluation ... Sentiment](https://reader034.fdocuments.us/reader034/viewer/2022042106/5e85a0b5163cb23e1f3101d6/html5/thumbnails/9.jpg)
9
Identify events of interestLabel eventsAdd context: sentiment? location?Streaming
![Page 10: TwitInfo: Aggregating and Visualizing Microblogs for Event ... · Journalist Interview backgrounding eyewitnesses. 55 ui walkthrough algorithm design study/evaluation ... Sentiment](https://reader034.fdocuments.us/reader034/viewer/2022042106/5e85a0b5163cb23e1f3101d6/html5/thumbnails/10.jpg)
10
Identify events of interestLabel eventsAdd context: sentiment? location?Streaming
TwitInfo automates the process of telling stories using Twitter
![Page 11: TwitInfo: Aggregating and Visualizing Microblogs for Event ... · Journalist Interview backgrounding eyewitnesses. 55 ui walkthrough algorithm design study/evaluation ... Sentiment](https://reader034.fdocuments.us/reader034/viewer/2022042106/5e85a0b5163cb23e1f3101d6/html5/thumbnails/11.jpg)
11
![Page 12: TwitInfo: Aggregating and Visualizing Microblogs for Event ... · Journalist Interview backgrounding eyewitnesses. 55 ui walkthrough algorithm design study/evaluation ... Sentiment](https://reader034.fdocuments.us/reader034/viewer/2022042106/5e85a0b5163cb23e1f3101d6/html5/thumbnails/12.jpg)
12
![Page 13: TwitInfo: Aggregating and Visualizing Microblogs for Event ... · Journalist Interview backgrounding eyewitnesses. 55 ui walkthrough algorithm design study/evaluation ... Sentiment](https://reader034.fdocuments.us/reader034/viewer/2022042106/5e85a0b5163cb23e1f3101d6/html5/thumbnails/13.jpg)
13
![Page 14: TwitInfo: Aggregating and Visualizing Microblogs for Event ... · Journalist Interview backgrounding eyewitnesses. 55 ui walkthrough algorithm design study/evaluation ... Sentiment](https://reader034.fdocuments.us/reader034/viewer/2022042106/5e85a0b5163cb23e1f3101d6/html5/thumbnails/14.jpg)
14
TwitInfo
event detectionevent labeling
contextstreaming algorithms
![Page 15: TwitInfo: Aggregating and Visualizing Microblogs for Event ... · Journalist Interview backgrounding eyewitnesses. 55 ui walkthrough algorithm design study/evaluation ... Sentiment](https://reader034.fdocuments.us/reader034/viewer/2022042106/5e85a0b5163cb23e1f3101d6/html5/thumbnails/15.jpg)
15
TwitInfo
event detectionevent labeling
contextstreaming algorithms
sentiment correction algorithmevaluation: news consumers
evaluation: pulitzer prize journalist
![Page 16: TwitInfo: Aggregating and Visualizing Microblogs for Event ... · Journalist Interview backgrounding eyewitnesses. 55 ui walkthrough algorithm design study/evaluation ... Sentiment](https://reader034.fdocuments.us/reader034/viewer/2022042106/5e85a0b5163cb23e1f3101d6/html5/thumbnails/16.jpg)
16
[Andre et al. UIST2007] [Diakopoulos et al. VAST 2010][Diakopoulos+Shamma CHI2010] [Dork et al. InfoVis2010][Go et al. 2010] [Leskovec et al. KDD2009][Shamma et al. CSCW2010+2011]
![Page 17: TwitInfo: Aggregating and Visualizing Microblogs for Event ... · Journalist Interview backgrounding eyewitnesses. 55 ui walkthrough algorithm design study/evaluation ... Sentiment](https://reader034.fdocuments.us/reader034/viewer/2022042106/5e85a0b5163cb23e1f3101d6/html5/thumbnails/17.jpg)
17
[Andre et al. UIST2007] [Diakopoulos et al. VAST 2010][Diakopoulos+Shamma CHI2010] [Dork et al. InfoVis2010][Go et al. 2010] [Leskovec et al. KDD2009][Shamma et al. CSCW2010+2011]
![Page 18: TwitInfo: Aggregating and Visualizing Microblogs for Event ... · Journalist Interview backgrounding eyewitnesses. 55 ui walkthrough algorithm design study/evaluation ... Sentiment](https://reader034.fdocuments.us/reader034/viewer/2022042106/5e85a0b5163cb23e1f3101d6/html5/thumbnails/18.jpg)
18
[Andre et al. UIST2007] [Diakopoulos et al. VAST 2010][Diakopoulos+Shamma CHI2010] [Dork et al. InfoVis2010][Go et al. 2010] [Leskovec et al. KDD2009][Shamma et al. CSCW2010+2011]
![Page 19: TwitInfo: Aggregating and Visualizing Microblogs for Event ... · Journalist Interview backgrounding eyewitnesses. 55 ui walkthrough algorithm design study/evaluation ... Sentiment](https://reader034.fdocuments.us/reader034/viewer/2022042106/5e85a0b5163cb23e1f3101d6/html5/thumbnails/19.jpg)
19
[Andre et al. UIST2007] [Diakopoulos et al. VAST 2010][Diakopoulos+Shamma CHI2010] [Dork et al. InfoVis2010][Go et al. 2010] [Leskovec et al. KDD2009][Shamma et al. CSCW2010+2011]
TwitInfo is a streaming storytelling layer that complements these visualizations
![Page 20: TwitInfo: Aggregating and Visualizing Microblogs for Event ... · Journalist Interview backgrounding eyewitnesses. 55 ui walkthrough algorithm design study/evaluation ... Sentiment](https://reader034.fdocuments.us/reader034/viewer/2022042106/5e85a0b5163cb23e1f3101d6/html5/thumbnails/20.jpg)
20
ui walkthroughalgorithm designstudy/evaluation
discussion
![Page 21: TwitInfo: Aggregating and Visualizing Microblogs for Event ... · Journalist Interview backgrounding eyewitnesses. 55 ui walkthrough algorithm design study/evaluation ... Sentiment](https://reader034.fdocuments.us/reader034/viewer/2022042106/5e85a0b5163cb23e1f3101d6/html5/thumbnails/21.jpg)
21
![Page 22: TwitInfo: Aggregating and Visualizing Microblogs for Event ... · Journalist Interview backgrounding eyewitnesses. 55 ui walkthrough algorithm design study/evaluation ... Sentiment](https://reader034.fdocuments.us/reader034/viewer/2022042106/5e85a0b5163cb23e1f3101d6/html5/thumbnails/22.jpg)
22
![Page 23: TwitInfo: Aggregating and Visualizing Microblogs for Event ... · Journalist Interview backgrounding eyewitnesses. 55 ui walkthrough algorithm design study/evaluation ... Sentiment](https://reader034.fdocuments.us/reader034/viewer/2022042106/5e85a0b5163cb23e1f3101d6/html5/thumbnails/23.jpg)
23
![Page 24: TwitInfo: Aggregating and Visualizing Microblogs for Event ... · Journalist Interview backgrounding eyewitnesses. 55 ui walkthrough algorithm design study/evaluation ... Sentiment](https://reader034.fdocuments.us/reader034/viewer/2022042106/5e85a0b5163cb23e1f3101d6/html5/thumbnails/24.jpg)
24
![Page 25: TwitInfo: Aggregating and Visualizing Microblogs for Event ... · Journalist Interview backgrounding eyewitnesses. 55 ui walkthrough algorithm design study/evaluation ... Sentiment](https://reader034.fdocuments.us/reader034/viewer/2022042106/5e85a0b5163cb23e1f3101d6/html5/thumbnails/25.jpg)
25
![Page 26: TwitInfo: Aggregating and Visualizing Microblogs for Event ... · Journalist Interview backgrounding eyewitnesses. 55 ui walkthrough algorithm design study/evaluation ... Sentiment](https://reader034.fdocuments.us/reader034/viewer/2022042106/5e85a0b5163cb23e1f3101d6/html5/thumbnails/26.jpg)
26
![Page 27: TwitInfo: Aggregating and Visualizing Microblogs for Event ... · Journalist Interview backgrounding eyewitnesses. 55 ui walkthrough algorithm design study/evaluation ... Sentiment](https://reader034.fdocuments.us/reader034/viewer/2022042106/5e85a0b5163cb23e1f3101d6/html5/thumbnails/27.jpg)
27
ui walkthroughalgorithm designstudy/evaluation
discussion
![Page 28: TwitInfo: Aggregating and Visualizing Microblogs for Event ... · Journalist Interview backgrounding eyewitnesses. 55 ui walkthrough algorithm design study/evaluation ... Sentiment](https://reader034.fdocuments.us/reader034/viewer/2022042106/5e85a0b5163cb23e1f3101d6/html5/thumbnails/28.jpg)
28
![Page 29: TwitInfo: Aggregating and Visualizing Microblogs for Event ... · Journalist Interview backgrounding eyewitnesses. 55 ui walkthrough algorithm design study/evaluation ... Sentiment](https://reader034.fdocuments.us/reader034/viewer/2022042106/5e85a0b5163cb23e1f3101d6/html5/thumbnails/29.jpg)
29
Large
![Page 30: TwitInfo: Aggregating and Visualizing Microblogs for Event ... · Journalist Interview backgrounding eyewitnesses. 55 ui walkthrough algorithm design study/evaluation ... Sentiment](https://reader034.fdocuments.us/reader034/viewer/2022042106/5e85a0b5163cb23e1f3101d6/html5/thumbnails/30.jpg)
30
LargeRelatively
![Page 31: TwitInfo: Aggregating and Visualizing Microblogs for Event ... · Journalist Interview backgrounding eyewitnesses. 55 ui walkthrough algorithm design study/evaluation ... Sentiment](https://reader034.fdocuments.us/reader034/viewer/2022042106/5e85a0b5163cb23e1f3101d6/html5/thumbnails/31.jpg)
31
LargeRelatively
Streaming
![Page 32: TwitInfo: Aggregating and Visualizing Microblogs for Event ... · Journalist Interview backgrounding eyewitnesses. 55 ui walkthrough algorithm design study/evaluation ... Sentiment](https://reader034.fdocuments.us/reader034/viewer/2022042106/5e85a0b5163cb23e1f3101d6/html5/thumbnails/32.jpg)
32
LargeRelatively
Streamingmean deviation
![Page 33: TwitInfo: Aggregating and Visualizing Microblogs for Event ... · Journalist Interview backgrounding eyewitnesses. 55 ui walkthrough algorithm design study/evaluation ... Sentiment](https://reader034.fdocuments.us/reader034/viewer/2022042106/5e85a0b5163cb23e1f3101d6/html5/thumbnails/33.jpg)
33
LargeRelatively
Adaptive Streamingmean deviation
![Page 34: TwitInfo: Aggregating and Visualizing Microblogs for Event ... · Journalist Interview backgrounding eyewitnesses. 55 ui walkthrough algorithm design study/evaluation ... Sentiment](https://reader034.fdocuments.us/reader034/viewer/2022042106/5e85a0b5163cb23e1f3101d6/html5/thumbnails/34.jpg)
34
LargeRelatively
Adaptive StreamingExponentially weighted moving mean deviation
![Page 35: TwitInfo: Aggregating and Visualizing Microblogs for Event ... · Journalist Interview backgrounding eyewitnesses. 55 ui walkthrough algorithm design study/evaluation ... Sentiment](https://reader034.fdocuments.us/reader034/viewer/2022042106/5e85a0b5163cb23e1f3101d6/html5/thumbnails/35.jpg)
35
LargeRelatively
Adaptive Streaming
RFC 2988: ComputingTCP’s retransmission timer
Exponentially weighted moving mean deviation
![Page 36: TwitInfo: Aggregating and Visualizing Microblogs for Event ... · Journalist Interview backgrounding eyewitnesses. 55 ui walkthrough algorithm design study/evaluation ... Sentiment](https://reader034.fdocuments.us/reader034/viewer/2022042106/5e85a0b5163cb23e1f3101d6/html5/thumbnails/36.jpg)
36
LargeRelatively
Adaptive Streaming
RFC 2988: ComputingTCP’s retransmission timer
Exponentially weighted moving mean deviation
TF-IDF
![Page 37: TwitInfo: Aggregating and Visualizing Microblogs for Event ... · Journalist Interview backgrounding eyewitnesses. 55 ui walkthrough algorithm design study/evaluation ... Sentiment](https://reader034.fdocuments.us/reader034/viewer/2022042106/5e85a0b5163cb23e1f3101d6/html5/thumbnails/37.jpg)
37
LargeRelatively
Adaptive Streaming
RFC 2988: ComputingTCP’s retransmission timer
Exponentially weighted moving mean deviation
TF-IDF
2 datasets: earthquakes, soccer games
Precision: 80-100%Recall: 80-100%
![Page 38: TwitInfo: Aggregating and Visualizing Microblogs for Event ... · Journalist Interview backgrounding eyewitnesses. 55 ui walkthrough algorithm design study/evaluation ... Sentiment](https://reader034.fdocuments.us/reader034/viewer/2022042106/5e85a0b5163cb23e1f3101d6/html5/thumbnails/38.jpg)
38
![Page 39: TwitInfo: Aggregating and Visualizing Microblogs for Event ... · Journalist Interview backgrounding eyewitnesses. 55 ui walkthrough algorithm design study/evaluation ... Sentiment](https://reader034.fdocuments.us/reader034/viewer/2022042106/5e85a0b5163cb23e1f3101d6/html5/thumbnails/39.jpg)
39
![Page 40: TwitInfo: Aggregating and Visualizing Microblogs for Event ... · Journalist Interview backgrounding eyewitnesses. 55 ui walkthrough algorithm design study/evaluation ... Sentiment](https://reader034.fdocuments.us/reader034/viewer/2022042106/5e85a0b5163cb23e1f3101d6/html5/thumbnails/40.jpg)
40
![Page 41: TwitInfo: Aggregating and Visualizing Microblogs for Event ... · Journalist Interview backgrounding eyewitnesses. 55 ui walkthrough algorithm design study/evaluation ... Sentiment](https://reader034.fdocuments.us/reader034/viewer/2022042106/5e85a0b5163cb23e1f3101d6/html5/thumbnails/41.jpg)
41
![Page 42: TwitInfo: Aggregating and Visualizing Microblogs for Event ... · Journalist Interview backgrounding eyewitnesses. 55 ui walkthrough algorithm design study/evaluation ... Sentiment](https://reader034.fdocuments.us/reader034/viewer/2022042106/5e85a0b5163cb23e1f3101d6/html5/thumbnails/42.jpg)
42MAYBE CUT!
![Page 43: TwitInfo: Aggregating and Visualizing Microblogs for Event ... · Journalist Interview backgrounding eyewitnesses. 55 ui walkthrough algorithm design study/evaluation ... Sentiment](https://reader034.fdocuments.us/reader034/viewer/2022042106/5e85a0b5163cb23e1f3101d6/html5/thumbnails/43.jpg)
43
ui walkthroughalgorithm designstudy/evaluation
discussion
![Page 44: TwitInfo: Aggregating and Visualizing Microblogs for Event ... · Journalist Interview backgrounding eyewitnesses. 55 ui walkthrough algorithm design study/evaluation ... Sentiment](https://reader034.fdocuments.us/reader034/viewer/2022042106/5e85a0b5163cb23e1f3101d6/html5/thumbnails/44.jpg)
44
12 participants: four female/8 malefirst half: directed tasks
second half: 5 min write article (soccer, Obama)
how do they use the interface?what can they learn from TwitInfo?
![Page 45: TwitInfo: Aggregating and Visualizing Microblogs for Event ... · Journalist Interview backgrounding eyewitnesses. 55 ui walkthrough algorithm design study/evaluation ... Sentiment](https://reader034.fdocuments.us/reader034/viewer/2022042106/5e85a0b5163cb23e1f3101d6/html5/thumbnails/45.jpg)
45
“After the peace talks, Obama traveled to the ASEAN conference and to NATO to work on issues in those parts of the world. He then spent the week dedicated to domestic economic issues. First he proposed a research tax break, then a $50 billion investment in infrastructure, then the issue came up about whether he should keep some tax breaks that Bush had implemented, and he’s asking for some tax breaks from business, and these are generating some controversy because [. . . ]”
Participant 6:
![Page 46: TwitInfo: Aggregating and Visualizing Microblogs for Event ... · Journalist Interview backgrounding eyewitnesses. 55 ui walkthrough algorithm design study/evaluation ... Sentiment](https://reader034.fdocuments.us/reader034/viewer/2022042106/5e85a0b5163cb23e1f3101d6/html5/thumbnails/46.jpg)
46
![Page 47: TwitInfo: Aggregating and Visualizing Microblogs for Event ... · Journalist Interview backgrounding eyewitnesses. 55 ui walkthrough algorithm design study/evaluation ... Sentiment](https://reader034.fdocuments.us/reader034/viewer/2022042106/5e85a0b5163cb23e1f3101d6/html5/thumbnails/47.jpg)
47
Everyone
![Page 48: TwitInfo: Aggregating and Visualizing Microblogs for Event ... · Journalist Interview backgrounding eyewitnesses. 55 ui walkthrough algorithm design study/evaluation ... Sentiment](https://reader034.fdocuments.us/reader034/viewer/2022042106/5e85a0b5163cb23e1f3101d6/html5/thumbnails/48.jpg)
48
EveryoneSplit
![Page 49: TwitInfo: Aggregating and Visualizing Microblogs for Event ... · Journalist Interview backgrounding eyewitnesses. 55 ui walkthrough algorithm design study/evaluation ... Sentiment](https://reader034.fdocuments.us/reader034/viewer/2022042106/5e85a0b5163cb23e1f3101d6/html5/thumbnails/49.jpg)
49
EveryoneSplit
NewsJunkies
![Page 50: TwitInfo: Aggregating and Visualizing Microblogs for Event ... · Journalist Interview backgrounding eyewitnesses. 55 ui walkthrough algorithm design study/evaluation ... Sentiment](https://reader034.fdocuments.us/reader034/viewer/2022042106/5e85a0b5163cb23e1f3101d6/html5/thumbnails/50.jpg)
50
earthquakes
![Page 51: TwitInfo: Aggregating and Visualizing Microblogs for Event ... · Journalist Interview backgrounding eyewitnesses. 55 ui walkthrough algorithm design study/evaluation ... Sentiment](https://reader034.fdocuments.us/reader034/viewer/2022042106/5e85a0b5163cb23e1f3101d6/html5/thumbnails/51.jpg)
51
“my thoughts and prayers go out to everyone in that country. I hope...”
earthquakes
![Page 52: TwitInfo: Aggregating and Visualizing Microblogs for Event ... · Journalist Interview backgrounding eyewitnesses. 55 ui walkthrough algorithm design study/evaluation ... Sentiment](https://reader034.fdocuments.us/reader034/viewer/2022042106/5e85a0b5163cb23e1f3101d6/html5/thumbnails/52.jpg)
52
Journalist Interview
![Page 53: TwitInfo: Aggregating and Visualizing Microblogs for Event ... · Journalist Interview backgrounding eyewitnesses. 55 ui walkthrough algorithm design study/evaluation ... Sentiment](https://reader034.fdocuments.us/reader034/viewer/2022042106/5e85a0b5163cb23e1f3101d6/html5/thumbnails/53.jpg)
53
Journalist Interview
backgrounding
![Page 54: TwitInfo: Aggregating and Visualizing Microblogs for Event ... · Journalist Interview backgrounding eyewitnesses. 55 ui walkthrough algorithm design study/evaluation ... Sentiment](https://reader034.fdocuments.us/reader034/viewer/2022042106/5e85a0b5163cb23e1f3101d6/html5/thumbnails/54.jpg)
54
Journalist Interview
backgrounding eyewitnesses
![Page 55: TwitInfo: Aggregating and Visualizing Microblogs for Event ... · Journalist Interview backgrounding eyewitnesses. 55 ui walkthrough algorithm design study/evaluation ... Sentiment](https://reader034.fdocuments.us/reader034/viewer/2022042106/5e85a0b5163cb23e1f3101d6/html5/thumbnails/55.jpg)
55
ui walkthroughalgorithm designstudy/evaluation
discussion
![Page 56: TwitInfo: Aggregating and Visualizing Microblogs for Event ... · Journalist Interview backgrounding eyewitnesses. 55 ui walkthrough algorithm design study/evaluation ... Sentiment](https://reader034.fdocuments.us/reader034/viewer/2022042106/5e85a0b5163cb23e1f3101d6/html5/thumbnails/56.jpg)
56
appropriate algorithms for the medium
![Page 57: TwitInfo: Aggregating and Visualizing Microblogs for Event ... · Journalist Interview backgrounding eyewitnesses. 55 ui walkthrough algorithm design study/evaluation ... Sentiment](https://reader034.fdocuments.us/reader034/viewer/2022042106/5e85a0b5163cb23e1f3101d6/html5/thumbnails/57.jpg)
57
appropriate algorithms for the medium
other uses for streaming algorithms/signal processing
![Page 58: TwitInfo: Aggregating and Visualizing Microblogs for Event ... · Journalist Interview backgrounding eyewitnesses. 55 ui walkthrough algorithm design study/evaluation ... Sentiment](https://reader034.fdocuments.us/reader034/viewer/2022042106/5e85a0b5163cb23e1f3101d6/html5/thumbnails/58.jpg)
58
appropriate algorithms for the medium
other uses for streaming algorithms/signal processing
sentiment: caution!
![Page 59: TwitInfo: Aggregating and Visualizing Microblogs for Event ... · Journalist Interview backgrounding eyewitnesses. 55 ui walkthrough algorithm design study/evaluation ... Sentiment](https://reader034.fdocuments.us/reader034/viewer/2022042106/5e85a0b5163cb23e1f3101d6/html5/thumbnails/59.jpg)
59
appropriate algorithms for the medium
other uses for streaming algorithms/signal processing
sentiment: caution!
mixed-initiative storytelling
![Page 60: TwitInfo: Aggregating and Visualizing Microblogs for Event ... · Journalist Interview backgrounding eyewitnesses. 55 ui walkthrough algorithm design study/evaluation ... Sentiment](https://reader034.fdocuments.us/reader034/viewer/2022042106/5e85a0b5163cb23e1f3101d6/html5/thumbnails/60.jpg)
60
TwitInfo
http://twitinfo.csail.mit.edu
[email protected] / @marcua
Adam Marcus, Michael Bernstein, Osama Badar,David Karger, Sam Madden, Rob Miller
![Page 61: TwitInfo: Aggregating and Visualizing Microblogs for Event ... · Journalist Interview backgrounding eyewitnesses. 55 ui walkthrough algorithm design study/evaluation ... Sentiment](https://reader034.fdocuments.us/reader034/viewer/2022042106/5e85a0b5163cb23e1f3101d6/html5/thumbnails/61.jpg)
61
![Page 62: TwitInfo: Aggregating and Visualizing Microblogs for Event ... · Journalist Interview backgrounding eyewitnesses. 55 ui walkthrough algorithm design study/evaluation ... Sentiment](https://reader034.fdocuments.us/reader034/viewer/2022042106/5e85a0b5163cb23e1f3101d6/html5/thumbnails/62.jpg)
62
Twitter data for fun and stories
![Page 63: TwitInfo: Aggregating and Visualizing Microblogs for Event ... · Journalist Interview backgrounding eyewitnesses. 55 ui walkthrough algorithm design study/evaluation ... Sentiment](https://reader034.fdocuments.us/reader034/viewer/2022042106/5e85a0b5163cb23e1f3101d6/html5/thumbnails/63.jpg)
63
Twitter data for fun and stories
How can we tell stories with the tweet stream?
![Page 64: TwitInfo: Aggregating and Visualizing Microblogs for Event ... · Journalist Interview backgrounding eyewitnesses. 55 ui walkthrough algorithm design study/evaluation ... Sentiment](https://reader034.fdocuments.us/reader034/viewer/2022042106/5e85a0b5163cb23e1f3101d6/html5/thumbnails/64.jpg)
64
Twitter data for fun and stories
How can we tell stories with the tweet stream?
How can we extract data from the tweet stream?
![Page 65: TwitInfo: Aggregating and Visualizing Microblogs for Event ... · Journalist Interview backgrounding eyewitnesses. 55 ui walkthrough algorithm design study/evaluation ... Sentiment](https://reader034.fdocuments.us/reader034/viewer/2022042106/5e85a0b5163cb23e1f3101d6/html5/thumbnails/65.jpg)
65
Twitter data for fun and stories
How can we extract data from the tweet stream?
TwitInfo
How can we tell stories with the tweet stream?
![Page 66: TwitInfo: Aggregating and Visualizing Microblogs for Event ... · Journalist Interview backgrounding eyewitnesses. 55 ui walkthrough algorithm design study/evaluation ... Sentiment](https://reader034.fdocuments.us/reader034/viewer/2022042106/5e85a0b5163cb23e1f3101d6/html5/thumbnails/66.jpg)
66
Twitter data for fun and stories
How can we extract data from the tweet stream?
TwitInfo
TweeQL
How can we tell stories with the tweet stream?
![Page 67: TwitInfo: Aggregating and Visualizing Microblogs for Event ... · Journalist Interview backgrounding eyewitnesses. 55 ui walkthrough algorithm design study/evaluation ... Sentiment](https://reader034.fdocuments.us/reader034/viewer/2022042106/5e85a0b5163cb23e1f3101d6/html5/thumbnails/67.jpg)
67
TwitInfo
How can we tell stories with the tweet stream?
![Page 68: TwitInfo: Aggregating and Visualizing Microblogs for Event ... · Journalist Interview backgrounding eyewitnesses. 55 ui walkthrough algorithm design study/evaluation ... Sentiment](https://reader034.fdocuments.us/reader034/viewer/2022042106/5e85a0b5163cb23e1f3101d6/html5/thumbnails/68.jpg)
68
TwitInfo
How can we tell stories with the tweet stream?
![Page 69: TwitInfo: Aggregating and Visualizing Microblogs for Event ... · Journalist Interview backgrounding eyewitnesses. 55 ui walkthrough algorithm design study/evaluation ... Sentiment](https://reader034.fdocuments.us/reader034/viewer/2022042106/5e85a0b5163cb23e1f3101d6/html5/thumbnails/69.jpg)
69
TwitInfo Uses
Automatically identify eventse.g., goals, earthquakes
![Page 70: TwitInfo: Aggregating and Visualizing Microblogs for Event ... · Journalist Interview backgrounding eyewitnesses. 55 ui walkthrough algorithm design study/evaluation ... Sentiment](https://reader034.fdocuments.us/reader034/viewer/2022042106/5e85a0b5163cb23e1f3101d6/html5/thumbnails/70.jpg)
70
TwitInfo Uses
Automatically identify eventse.g., goals, earthquakes
Backgrounding e.g., Obama's last two weeks
![Page 71: TwitInfo: Aggregating and Visualizing Microblogs for Event ... · Journalist Interview backgrounding eyewitnesses. 55 ui walkthrough algorithm design study/evaluation ... Sentiment](https://reader034.fdocuments.us/reader034/viewer/2022042106/5e85a0b5163cb23e1f3101d6/html5/thumbnails/71.jpg)
71
TwitInfo Uses
Automatically identify eventse.g., goals, earthquakes
Backgrounding e.g., Obama's last two weeks
Identifying sources on the grounde.g., interviewing earthquake survivors
![Page 72: TwitInfo: Aggregating and Visualizing Microblogs for Event ... · Journalist Interview backgrounding eyewitnesses. 55 ui walkthrough algorithm design study/evaluation ... Sentiment](https://reader034.fdocuments.us/reader034/viewer/2022042106/5e85a0b5163cb23e1f3101d6/html5/thumbnails/72.jpg)
72
Cautionary Tweet TalesSentiment is tricky e.g., “My warmest prayers go out to the people of Christchurch.”
![Page 73: TwitInfo: Aggregating and Visualizing Microblogs for Event ... · Journalist Interview backgrounding eyewitnesses. 55 ui walkthrough algorithm design study/evaluation ... Sentiment](https://reader034.fdocuments.us/reader034/viewer/2022042106/5e85a0b5163cb23e1f3101d6/html5/thumbnails/73.jpg)
73
Cautionary Tweet Tales
Location is evasive
Sentiment is tricky e.g., “My warmest prayers go out to the people of Christchurch.”
![Page 74: TwitInfo: Aggregating and Visualizing Microblogs for Event ... · Journalist Interview backgrounding eyewitnesses. 55 ui walkthrough algorithm design study/evaluation ... Sentiment](https://reader034.fdocuments.us/reader034/viewer/2022042106/5e85a0b5163cb23e1f3101d6/html5/thumbnails/74.jpg)
74
Cautionary Tweet Tales
Location is evasive
Sentiment is tricky e.g., “My warmest prayers go out to the people of Christchurch.”
Spam is everywhere
![Page 75: TwitInfo: Aggregating and Visualizing Microblogs for Event ... · Journalist Interview backgrounding eyewitnesses. 55 ui walkthrough algorithm design study/evaluation ... Sentiment](https://reader034.fdocuments.us/reader034/viewer/2022042106/5e85a0b5163cb23e1f3101d6/html5/thumbnails/75.jpg)
75
TwitInfo
Automated event detection makes tweet-basedstory-telling possible
![Page 76: TwitInfo: Aggregating and Visualizing Microblogs for Event ... · Journalist Interview backgrounding eyewitnesses. 55 ui walkthrough algorithm design study/evaluation ... Sentiment](https://reader034.fdocuments.us/reader034/viewer/2022042106/5e85a0b5163cb23e1f3101d6/html5/thumbnails/76.jpg)
76
TwitInfo
Automated event detection makes tweet-basedstory-telling possible
Interfaces tell a story, people add context
![Page 77: TwitInfo: Aggregating and Visualizing Microblogs for Event ... · Journalist Interview backgrounding eyewitnesses. 55 ui walkthrough algorithm design study/evaluation ... Sentiment](https://reader034.fdocuments.us/reader034/viewer/2022042106/5e85a0b5163cb23e1f3101d6/html5/thumbnails/77.jpg)
77
TwitInfo
Automated event detection makes tweet-basedstory-telling possible
Interfaces tell a story, people add context
![Page 78: TwitInfo: Aggregating and Visualizing Microblogs for Event ... · Journalist Interview backgrounding eyewitnesses. 55 ui walkthrough algorithm design study/evaluation ... Sentiment](https://reader034.fdocuments.us/reader034/viewer/2022042106/5e85a0b5163cb23e1f3101d6/html5/thumbnails/78.jpg)
78
TweeQL
How can we extract data from the tweet stream?
![Page 79: TwitInfo: Aggregating and Visualizing Microblogs for Event ... · Journalist Interview backgrounding eyewitnesses. 55 ui walkthrough algorithm design study/evaluation ... Sentiment](https://reader034.fdocuments.us/reader034/viewer/2022042106/5e85a0b5163cb23e1f3101d6/html5/thumbnails/79.jpg)
79
TweeQL
“It's a balmy 89°C in Phoenix”
location=Phoenix, temperatureC=89
How can we extract data from the tweet stream?
![Page 80: TwitInfo: Aggregating and Visualizing Microblogs for Event ... · Journalist Interview backgrounding eyewitnesses. 55 ui walkthrough algorithm design study/evaluation ... Sentiment](https://reader034.fdocuments.us/reader034/viewer/2022042106/5e85a0b5163cb23e1f3101d6/html5/thumbnails/80.jpg)
80
TweeQL
“It's a balmy 89°C in Phoenix”
location=Phoenix, temperatureC=89
“I'm starting to dig Obamacare!”
topic=Obama, sentiment=positive
How can we extract data from the tweet stream?
![Page 81: TwitInfo: Aggregating and Visualizing Microblogs for Event ... · Journalist Interview backgrounding eyewitnesses. 55 ui walkthrough algorithm design study/evaluation ... Sentiment](https://reader034.fdocuments.us/reader034/viewer/2022042106/5e85a0b5163cb23e1f3101d6/html5/thumbnails/81.jpg)
81
Twitter data hacking is hard
● Learn the API● Transform data● Stuff it into a database
![Page 82: TwitInfo: Aggregating and Visualizing Microblogs for Event ... · Journalist Interview backgrounding eyewitnesses. 55 ui walkthrough algorithm design study/evaluation ... Sentiment](https://reader034.fdocuments.us/reader034/viewer/2022042106/5e85a0b5163cb23e1f3101d6/html5/thumbnails/82.jpg)
82
Twitter data hacking is hard
● Learn the API● Transform data● Stuff it into a database
Ad-hoc data processing
![Page 83: TwitInfo: Aggregating and Visualizing Microblogs for Event ... · Journalist Interview backgrounding eyewitnesses. 55 ui walkthrough algorithm design study/evaluation ... Sentiment](https://reader034.fdocuments.us/reader034/viewer/2022042106/5e85a0b5163cb23e1f3101d6/html5/thumbnails/83.jpg)
83
Twitter data hacking is hard
● Learn the API● Transform data● Stuff it into a database
Ad-hoc data processing
![Page 84: TwitInfo: Aggregating and Visualizing Microblogs for Event ... · Journalist Interview backgrounding eyewitnesses. 55 ui walkthrough algorithm design study/evaluation ... Sentiment](https://reader034.fdocuments.us/reader034/viewer/2022042106/5e85a0b5163cb23e1f3101d6/html5/thumbnails/84.jpg)
84
TweeQL extracts data from tweets as they pass through the stream
![Page 85: TwitInfo: Aggregating and Visualizing Microblogs for Event ... · Journalist Interview backgrounding eyewitnesses. 55 ui walkthrough algorithm design study/evaluation ... Sentiment](https://reader034.fdocuments.us/reader034/viewer/2022042106/5e85a0b5163cb23e1f3101d6/html5/thumbnails/85.jpg)
85
TweeQL extracts data from tweets as they pass through the stream
Data extractione.g., location, sentiment, temperature, opencalais
![Page 86: TwitInfo: Aggregating and Visualizing Microblogs for Event ... · Journalist Interview backgrounding eyewitnesses. 55 ui walkthrough algorithm design study/evaluation ... Sentiment](https://reader034.fdocuments.us/reader034/viewer/2022042106/5e85a0b5163cb23e1f3101d6/html5/thumbnails/86.jpg)
86
TweeQL extracts data from tweets as they pass through the stream
Data extractione.g., location, sentiment, temperature, opencalais
SQL-like queriese.g., SELECT location, text FROM twitter
![Page 87: TwitInfo: Aggregating and Visualizing Microblogs for Event ... · Journalist Interview backgrounding eyewitnesses. 55 ui walkthrough algorithm design study/evaluation ... Sentiment](https://reader034.fdocuments.us/reader034/viewer/2022042106/5e85a0b5163cb23e1f3101d6/html5/thumbnails/87.jpg)
87
TweeQL extracts data from tweets as they pass through the stream
Data extractione.g., location, sentiment, temperature, opencalais
SQL-like queriese.g., SELECT location, text FROM twitter
stream, not table
![Page 88: TwitInfo: Aggregating and Visualizing Microblogs for Event ... · Journalist Interview backgrounding eyewitnesses. 55 ui walkthrough algorithm design study/evaluation ... Sentiment](https://reader034.fdocuments.us/reader034/viewer/2022042106/5e85a0b5163cb23e1f3101d6/html5/thumbnails/88.jpg)
88
TweeQL demo*nerd alert*
![Page 89: TwitInfo: Aggregating and Visualizing Microblogs for Event ... · Journalist Interview backgrounding eyewitnesses. 55 ui walkthrough algorithm design study/evaluation ... Sentiment](https://reader034.fdocuments.us/reader034/viewer/2022042106/5e85a0b5163cb23e1f3101d6/html5/thumbnails/89.jpg)
89
TweeQL's other features
● Aggregation● Outlier detection● Joins/mashups with other sources
![Page 90: TwitInfo: Aggregating and Visualizing Microblogs for Event ... · Journalist Interview backgrounding eyewitnesses. 55 ui walkthrough algorithm design study/evaluation ... Sentiment](https://reader034.fdocuments.us/reader034/viewer/2022042106/5e85a0b5163cb23e1f3101d6/html5/thumbnails/90.jpg)
90
![Page 91: TwitInfo: Aggregating and Visualizing Microblogs for Event ... · Journalist Interview backgrounding eyewitnesses. 55 ui walkthrough algorithm design study/evaluation ... Sentiment](https://reader034.fdocuments.us/reader034/viewer/2022042106/5e85a0b5163cb23e1f3101d6/html5/thumbnails/91.jpg)
91
Unstructureddata
![Page 92: TwitInfo: Aggregating and Visualizing Microblogs for Event ... · Journalist Interview backgrounding eyewitnesses. 55 ui walkthrough algorithm design study/evaluation ... Sentiment](https://reader034.fdocuments.us/reader034/viewer/2022042106/5e85a0b5163cb23e1f3101d6/html5/thumbnails/92.jpg)
92
+
Structuredqueries
Unstructureddata
![Page 93: TwitInfo: Aggregating and Visualizing Microblogs for Event ... · Journalist Interview backgrounding eyewitnesses. 55 ui walkthrough algorithm design study/evaluation ... Sentiment](https://reader034.fdocuments.us/reader034/viewer/2022042106/5e85a0b5163cb23e1f3101d6/html5/thumbnails/93.jpg)
93
+
Structuredqueries
=> Structureddata
Unstructureddata
![Page 94: TwitInfo: Aggregating and Visualizing Microblogs for Event ... · Journalist Interview backgrounding eyewitnesses. 55 ui walkthrough algorithm design study/evaluation ... Sentiment](https://reader034.fdocuments.us/reader034/viewer/2022042106/5e85a0b5163cb23e1f3101d6/html5/thumbnails/94.jpg)
94
+
Structuredqueries
=> Structureddata
Meaningfulvisualizations
=>
Unstructureddata
![Page 95: TwitInfo: Aggregating and Visualizing Microblogs for Event ... · Journalist Interview backgrounding eyewitnesses. 55 ui walkthrough algorithm design study/evaluation ... Sentiment](https://reader034.fdocuments.us/reader034/viewer/2022042106/5e85a0b5163cb23e1f3101d6/html5/thumbnails/95.jpg)
95
Thanks!
● TweeQL: http://github.com/marcua/tweeql● TwitInfo: http://twitinfo.csail.mit.edu/● Ask me about Mechanical Turk + Journalism!
Adam Marcus@[email protected]://people.csail.mit.edu/marcua
![Page 96: TwitInfo: Aggregating and Visualizing Microblogs for Event ... · Journalist Interview backgrounding eyewitnesses. 55 ui walkthrough algorithm design study/evaluation ... Sentiment](https://reader034.fdocuments.us/reader034/viewer/2022042106/5e85a0b5163cb23e1f3101d6/html5/thumbnails/96.jpg)
96
![Page 97: TwitInfo: Aggregating and Visualizing Microblogs for Event ... · Journalist Interview backgrounding eyewitnesses. 55 ui walkthrough algorithm design study/evaluation ... Sentiment](https://reader034.fdocuments.us/reader034/viewer/2022042106/5e85a0b5163cb23e1f3101d6/html5/thumbnails/97.jpg)
97
![Page 98: TwitInfo: Aggregating and Visualizing Microblogs for Event ... · Journalist Interview backgrounding eyewitnesses. 55 ui walkthrough algorithm design study/evaluation ... Sentiment](https://reader034.fdocuments.us/reader034/viewer/2022042106/5e85a0b5163cb23e1f3101d6/html5/thumbnails/98.jpg)
98
![Page 99: TwitInfo: Aggregating and Visualizing Microblogs for Event ... · Journalist Interview backgrounding eyewitnesses. 55 ui walkthrough algorithm design study/evaluation ... Sentiment](https://reader034.fdocuments.us/reader034/viewer/2022042106/5e85a0b5163cb23e1f3101d6/html5/thumbnails/99.jpg)
99
TweeQL Lessons Learned
● Geographic limitations● Requires grungy regular expressions● Data is not necessarily relational
![Page 100: TwitInfo: Aggregating and Visualizing Microblogs for Event ... · Journalist Interview backgrounding eyewitnesses. 55 ui walkthrough algorithm design study/evaluation ... Sentiment](https://reader034.fdocuments.us/reader034/viewer/2022042106/5e85a0b5163cb23e1f3101d6/html5/thumbnails/100.jpg)
100
![Page 101: TwitInfo: Aggregating and Visualizing Microblogs for Event ... · Journalist Interview backgrounding eyewitnesses. 55 ui walkthrough algorithm design study/evaluation ... Sentiment](https://reader034.fdocuments.us/reader034/viewer/2022042106/5e85a0b5163cb23e1f3101d6/html5/thumbnails/101.jpg)
101
![Page 102: TwitInfo: Aggregating and Visualizing Microblogs for Event ... · Journalist Interview backgrounding eyewitnesses. 55 ui walkthrough algorithm design study/evaluation ... Sentiment](https://reader034.fdocuments.us/reader034/viewer/2022042106/5e85a0b5163cb23e1f3101d6/html5/thumbnails/102.jpg)
102
![Page 103: TwitInfo: Aggregating and Visualizing Microblogs for Event ... · Journalist Interview backgrounding eyewitnesses. 55 ui walkthrough algorithm design study/evaluation ... Sentiment](https://reader034.fdocuments.us/reader034/viewer/2022042106/5e85a0b5163cb23e1f3101d6/html5/thumbnails/103.jpg)
103
![Page 104: TwitInfo: Aggregating and Visualizing Microblogs for Event ... · Journalist Interview backgrounding eyewitnesses. 55 ui walkthrough algorithm design study/evaluation ... Sentiment](https://reader034.fdocuments.us/reader034/viewer/2022042106/5e85a0b5163cb23e1f3101d6/html5/thumbnails/104.jpg)
104
![Page 105: TwitInfo: Aggregating and Visualizing Microblogs for Event ... · Journalist Interview backgrounding eyewitnesses. 55 ui walkthrough algorithm design study/evaluation ... Sentiment](https://reader034.fdocuments.us/reader034/viewer/2022042106/5e85a0b5163cb23e1f3101d6/html5/thumbnails/105.jpg)
105
Breadth
Depth
![Page 106: TwitInfo: Aggregating and Visualizing Microblogs for Event ... · Journalist Interview backgrounding eyewitnesses. 55 ui walkthrough algorithm design study/evaluation ... Sentiment](https://reader034.fdocuments.us/reader034/viewer/2022042106/5e85a0b5163cb23e1f3101d6/html5/thumbnails/106.jpg)
106
![Page 107: TwitInfo: Aggregating and Visualizing Microblogs for Event ... · Journalist Interview backgrounding eyewitnesses. 55 ui walkthrough algorithm design study/evaluation ... Sentiment](https://reader034.fdocuments.us/reader034/viewer/2022042106/5e85a0b5163cb23e1f3101d6/html5/thumbnails/107.jpg)
107
![Page 108: TwitInfo: Aggregating and Visualizing Microblogs for Event ... · Journalist Interview backgrounding eyewitnesses. 55 ui walkthrough algorithm design study/evaluation ... Sentiment](https://reader034.fdocuments.us/reader034/viewer/2022042106/5e85a0b5163cb23e1f3101d6/html5/thumbnails/108.jpg)
108
Challenges
![Page 109: TwitInfo: Aggregating and Visualizing Microblogs for Event ... · Journalist Interview backgrounding eyewitnesses. 55 ui walkthrough algorithm design study/evaluation ... Sentiment](https://reader034.fdocuments.us/reader034/viewer/2022042106/5e85a0b5163cb23e1f3101d6/html5/thumbnails/109.jpg)
109
Challenges
Event annotation
![Page 110: TwitInfo: Aggregating and Visualizing Microblogs for Event ... · Journalist Interview backgrounding eyewitnesses. 55 ui walkthrough algorithm design study/evaluation ... Sentiment](https://reader034.fdocuments.us/reader034/viewer/2022042106/5e85a0b5163cb23e1f3101d6/html5/thumbnails/110.jpg)
110
Challenges
Event annotation
Streaming
![Page 111: TwitInfo: Aggregating and Visualizing Microblogs for Event ... · Journalist Interview backgrounding eyewitnesses. 55 ui walkthrough algorithm design study/evaluation ... Sentiment](https://reader034.fdocuments.us/reader034/viewer/2022042106/5e85a0b5163cb23e1f3101d6/html5/thumbnails/111.jpg)
111
Challenges
Event annotation
StreamingSentiment
![Page 112: TwitInfo: Aggregating and Visualizing Microblogs for Event ... · Journalist Interview backgrounding eyewitnesses. 55 ui walkthrough algorithm design study/evaluation ... Sentiment](https://reader034.fdocuments.us/reader034/viewer/2022042106/5e85a0b5163cb23e1f3101d6/html5/thumbnails/112.jpg)
112
Challenges
Event annotation
StreamingLocationSentiment