This is your Twitter on drugs. Any Questions?

Workshop on Modeling Social Media

Florence, Italy, 19 May 2015

1

This Is Your Twitter on Drugs.Any Questions?

Cody Buntain and Jennifer Golbeck{cbuntain,golbeck}@cs.umd.eduHuman-Computer Interaction LabUniversity of Maryland

What is the point of this talk?

2

Can Twitter yield rapid insights about trends in drug use?

3


4


5

Drug use patterns are changing in the United

States

6


States

Source: National Survey on Drug Use and Health: Summary of National Findings, 2012

Past Month and Past Year Heroin Use Among Persons Aged 12 or Older: 2002-2012

The New Face of HeroinThe explosion of drugs like OxyContin has given way to a heroin epidemic ravaging the least likely corners of America - like bucolic Vermont, which has just woken up to a full-blown crisis

7


States

8

Designer Drugs


States

9

Designer Drugs

Current methods for tracking these trends are slow

11

Current methods have no support for partial results or

regression

14

Researchers have used

Twitter to track a variety of phenomena

16



17

Earthquake Epicenter Estimation

Typhoon Track Estimation

T. Sakaki, M. Okazaki, and Y. Matsuo, “Earthquake shakes Twitter users: real-time event detection by social sensors,” in Proceedings of the 19th

international conference on World wide web, 2010, pp. 851–860.



18D. Hauger and M. Schedl, “Exploring geospatial music listening patterns in

microblog data,” Proc. 10th Int. Work. Adapt. Multimed. Retr. (AMR 2012), 2012.

Artist Popularity by Region

Artist’s Song Popularity Over Time



19

V. Lampos, T. De Bie, and N. Cristianini, “Flu detector - Tracking epidemics on Twitter,” in Machine Learning and Knowledge Discovery in Databases, J. Balcázar, F. Bonchi, A.

Gionis, and M. Sebag, Eds. Springer Berlin Heidelberg, 2010, pp. 599–602.

V. Lampos and N. Cristianini, “Tracking the flu pandemic by monitoring the social web,” 2010 2nd Int. Work. Cogn. Inf. Process. CIP2010, pp. 411–416, 2010.

Twitter-based Flu Scores versus UK’s HPA Data

20 *M. J. Paul, M. Dredze, J. P. Michael, and D. Mark, “You are what you Tweet: Analyzing Twitter for public health,” Icwsm, pp. 265–272, 2011.

You Are What You Tweet: Analyzing Twitter for Public Health*

Rates of Allergy Sufferers in 2010

21 *M. J. Paul, M. Dredze, J. P. Michael, and D. Mark, “You are what you Tweet: Analyzing Twitter for public health,” Icwsm, pp. 265–272, 2011.

You Are What You Tweet: Analyzing Twitter for Public Health*

Rates of Allergy Sufferers in 2010

Drug use should also affect public health

Research Questions

22

Research Questions

• RQ 1 - Can Twitter provide insight in temporal trends in drug use?

22

Research Questions

• RQ 2 - Can Twitter illustrate state-level geographic trends in drug use?

• RQ 1 - Can Twitter provide insight in temporal trends in drug use?

22

Data Collection

23

Data Collection

24

Sample Stream

Data Collection

24

Sample Stream

Filter Stream

Data Collection

24

Sample Stream

Filter Stream

Data Collection

25

Sample Stream

Filter Stream

Data Collection

Distributed Data Store

(HDFS)

25

Sample Stream

Filter Stream

Data Collection


(HDFS)

Drug Keywords

26

Sample Stream

Filter Stream

Data Collection


(HDFS)

Drug Keywords

27

Sample Stream

Filter Stream

Data Collection


(HDFS)

Drug Keywords

Sampled from April-July 2014: 645,761,955 tweets (247GB)

27

Sample Stream

Filter Stream

Data Collection


(HDFS)

Drug Keywords

Sampled from April-July 2014: 645,761,955 tweets (247GB)

Filtered from Oct. 30 - Nov. 26: 176,742,962 tweets (81GB)

27

Sample Stream

Filter Stream

Data Collection


(HDFS)

Drug Keywords

28

Hadoop +

Spark

Case Study #1 Temporal Trends

29

Twee

t Fre

quen

cy

0

150

300

450

600

Date4/3

0/145/5

/14

5/10/1

4

5/15/1

4

5/20/1

4

5/25/1

4

5/30/1

46/4

/146/9

/14

6/14/1

4

6/19/1

4

6/24/1

4

6/29/1

47/4

/147/9

/14

7/14/1

4

7/19/1

4

7/24/1

4

7/29/1

4

CocaineHeroinMethYaba

Sample StreamTemporal Trends

30

Twee

t Fre

quen

cy

0

150

300

450

600

Date

5/3/14

5/8/14

5/13/1

4

5/18/1

4

5/23/1

4

5/28/1

46/2

/146/7

/14

6/12/1

4

6/17/1

4

6/22/1

4

6/27/1

47/2

/147/7

/14

7/12/1

4

7/17/1

4

7/22/1

4

7/27/1

4



(Smoothed)

31

Twee

t Fre

quen

cy

0

150

300

450

600

Date

5/3/14

5/8/14

5/13/1

4

5/18/1

4

5/23/1

4

5/28/1

46/2

/146/7

/14

6/12/1

4

6/17/1

4

6/22/1

4

6/27/1

47/2

/147/7

/14

7/12/1

4

7/17/1

4

7/22/1

4

7/27/1

4



(Smoothed)?

31

Twee

t Fre

quen

cy

0

75

150

225

300

Date

5/3/14

5/8/14

5/13/1

4

5/18/1

4

5/23/1

4

5/28/1

46/2

/146/7

/14

6/12/1

4

6/17/1

4

6/22/1

4

6/27/1

47/2

/147/7

/14

7/12/1

4

7/17/1

4

7/22/1

4

7/27/1

4

Cocaine†HeroinMethYaba


(Smoothed + No RT)

34

Twee

t Fre

quen

cy

0

75

150

225

300

Date

5/3/14

5/8/14

5/13/1

4

5/18/1

4

5/23/1

4

5/28/1

46/2

/146/7

/14

6/12/1

4

6/17/1

4

6/22/1

4

6/27/1

47/2

/147/7

/14

7/12/1

4

7/17/1

4

7/22/1

4

7/27/1

4



(Smoothed + No RT)Popularity by Volume:

Cocaine Meth

Heroin Yaba

35

Twee

t Fre

quen

cy

0

75

150

225

300

Date

5/3/14

5/8/14

5/13/1

4

5/18/1

4

5/23/1

4

5/28/1

46/2

/146/7

/14

6/12/1

4

6/17/1

4

6/22/1

4

6/27/1

47/2

/147/7

/14

7/12/1

4

7/17/1

4

7/22/1

4

7/27/1

4



(Smoothed + No RT)Popularity by Volume:

Cocaine Meth

Heroin Yaba

Consistent with the Global Drug Survey

36

Twee

t Fre

quen

cy

0

75

150

225

300

Date

5/3/14

5/8/14

5/13/1

4

5/18/1

4

5/23/1

4

5/28/1

46/2

/146/7

/14

6/12/1

4

6/17/1

4

6/22/1

4

6/27/1

47/2

/147/7

/14

7/12/1

4

7/17/1

4

7/22/1

4

7/27/1

4



(Smoothed + No RT)

37

Twee

t Fre

quen

cy

0

75

150

225

300

Date

5/3/14

5/8/14

5/13/1

4

5/18/1

4

5/23/1

4

5/28/1

46/2

/146/7

/14

6/12/1

4

6/17/1

4

6/22/1

4

6/27/1

47/2

/147/7

/14

7/12/1

4

7/17/1

4

7/22/1

4

7/27/1

4

Cocaine†Trend 1HeroinTrend 2MethTrend 3YabaTrend 4


(Smoothed + No RT + Trendlines)

38

Twee

t Fre

quen

cy

0

75

150

225

300

Date

5/3/14

5/8/14

5/13/1

4

5/18/1

4

5/23/1

4

5/28/1

46/2

/146/7

/14

6/12/1

4

6/17/1

4

6/22/1

4

6/27/1

47/2

/147/7

/14

7/12/1

4

7/17/1

4

7/22/1

4

7/27/1

4



(Smoothed + No RT + Trendlines)y = 0.5269x + 149.02

R² = 0.30184

39

Case Study #2 Geographic Trends

40

Geographic Trends

41

Geographic TrendsSample +

Filter Stream

HDFS Store of Drug Tweets

Heroin

Cocaine

Meth

Oxy

Filter for

USA

42

Geographic Trends

43

Geographic Trends

!

43

Geographic Trends

!

!

43

Geographic Trends

!

!

Only about 2% of tweets have GPS data

44


Filter Stream

Drug Geocoded Tweet Count

Cocaine 281

Heroin 1,490

Meth 3,462

Oxy 223

45

Cocaine Heroin

Meth Oxy46


Filter Stream

47


Filter Stream

CocaineMin Max

48


Filter Stream

CocaineMin Max

Heroin49


Filter Stream

Min Max

Heroin50


Filter Stream

Min Max

The New Face of HeroinThe explosion of drugs like OxyContin has given way to a heroin epidemic ravaging the least likely corners of America - like bucolic Vermont, which has just woken up to a full-blown crisis

Meth51


Filter Stream

Min Max

Meth52


Filter Stream

Min Max

Region 12 or Older Estimate

Northeast 3.07%

Midwest 3.12%

South 3.35%

West 3.81%

Oxy53


Filter Stream

Min Max

Oxy54


Filter Stream

Min Max

Additional Research: Tweet Disambiguation

55

Tweet Relevancy

56

Tweet Relevancy

!

56

Tweet Relevancy

57

Tweet Relevancy

!

57

Tweet Relevancy

!

!

57

Rele

vant

Per

cent

age

0%

25%

50%

75%

100%

Drug-Related Keywordssyn

thetic

mari

juana

hydroc

odon

elor

tab

methylo

ne

bath sa

ltsthi

zzox

iesdop

e pottwea

kha

shsta

cks ox k2

subbies flu

ff

pink la

dysm

iles

pandas

orang

es

demon

spee

d

malcolm

xne

gra

Tweet Relevancy

58

Rele

vant

Per

cent

age

0%

25%

50%

75%

100%

Drug-Related Keywordssyn

thetic

mari

juana

hydroc

odon

elor

tab

methylo

ne

bath sa

ltsthi

zzox

iesdop

e pottwea

kha

shsta

cks ox k2

subbies flu

ff

pink la

dysm

iles

pandas

orang

es

demon

spee

d

malcolm

xne

gra

Tweet Relevancy

59

Disambiguation with Topic Models

Mallet Topic Modeling Package

60



Oxy Topics

61



Oxy Topics

adderal, amp, bars, buy, day, drugs, ecstasy, it's, morphine, muscle, oxy, oxycodone, percocets,

percs, prescription, purplethizzle, relaxers,

speed, state, xanax

61



Oxy Topics

amino, credits, feelin, feeling, i'm, kinda, man, morons, oxycontin, pain, people, pills, real, reel,

strange, supply, system, treatment, ubos



speed, state, xanax

61



Oxy Topics

gabapertina, gotas, health, ice, lumina, mac, make,

months, oxy, oxygen, oxymoron, prince, redman, sweet, tah, tonight, trailer,

video, youtube

amino, credits, feelin, feeling, i'm, kinda, man, morons, oxycontin, pain, people, pills, real, reel,

strange, supply, system, treatment, ubos



speed, state, xanax

61



Molly Topics back, chest, china, coke, crack, don't, dont, drug, flogging, gas, gotta, green, i'm, jay,

lean, literally

62



Molly Topics back, chest, china, coke, crack, don't, dont, drug, flogging, gas, gotta, green, i'm, jay,

lean, literally

balloon, birthday, body, books, bookslapped, date, enter, happy, harpercollins,

kiss, letting, love, molly, mollysmcadams, nov, turn

weasley, win

ally, anthony, can't, club, emilio, estevez, finally, find, fresh, hall, kicks, michael,

molly, music, official, ringwald, sheedy, video,

youtube

ain't, ass, bill, champagne, cosby, drink, een, enjoyed, ford, girl, gramps, home,

i'm, lewinsky, lol, love, make, molly, monica, pop,

pop, popped, pudding, rock, slip, that's, tom,

wanna, weed, whiskey, wit, world

62

Limitations and Future Work

63

Relevancy Classification

64

Ground Truth

Ground Truth

• Limited concurrent ground truth for comparison

• Tweets about news stories may introduce bias

Ground Truth



• New NSDUH should be released in the fall

Ground Truth



• New NSDUH should be released in the fall

• Comparison with Google search trends

Twitter Data

Limited Location Data

Twitter Data


Drug-Related Tweets

Twitter Data


Geocoded Drug-Related

Tweets

Sampled Tweets

Drug-Related Tweets



Tweets

Sampled Drug-Related

Tweets



Tweets


TweetsInfer User Locations



Tweets


Tweets

Conclusions

71

Twee

t Fre

quen

cy

0

75

150

225

300

Date

5/3/14

5/8/14

5/13/1

4

5/18/1

4

5/23/1

4

5/28/1

46/2

/146/7

/14

6/12/1

4

6/17/1

4

6/22/1

4

6/27/1

47/2

/147/7

/14

7/12/1

4

7/17/1

4

7/22/1

4

7/27/1

4



72

Twee

t Fre

quen

cy

0

75

150

225

300

Date

5/3/14

5/8/14

5/13/1

4

5/18/1

4

5/23/1

4

5/28/1

46/2

/146/7

/14

6/12/1

4

6/17/1

4

6/22/1

4

6/27/1

47/2

/147/7

/14

7/12/1

4

7/17/1

4

7/22/1

4

7/27/1

4



73

Popularity by Volume: Cocaine

Meth Heroin Yaba

Consistent with the Global Drug Survey

Twee

t Fre

quen

cy

0

75

150

225

300

Date

5/3/14

5/8/14

5/13/1

4

5/18/1

4

5/23/1

4

5/28/1

46/2

/146/7

/14

6/12/1

4

6/17/1

4

6/22/1

4

6/27/1

47/2

/147/7

/14

7/12/1

4

7/17/1

4

7/22/1

4

7/27/1

4


Sample StreamTemporal Trendlines

y = 0.5269x + 149.02 R² = 0.30184

74

Twee

t Fre

quen

cy

0

75

150

225

300

Date

5/3/14

5/8/14

5/13/1

4

5/18/1

4

5/23/1

4

5/28/1

46/2

/146/7

/14

6/12/1

4

6/17/1

4

6/22/1

4

6/27/1

47/2

/147/7

/14

7/12/1

4

7/17/1

4

7/22/1

4

7/27/1

4


Sample StreamTemporal Trendlines

y = 0.5269x + 149.02 R² = 0.30184

75

Cocaine increased in popularity on Twitter

Cocaine Heroin

Meth Oxy76


Filter Stream

Cocaine Heroin

Meth Oxy77


Filter Stream

Can extract geographic trends from Twitter

Cocaine Heroin

Meth Oxy78


Filter Stream

Can extract geographic trends from Twitter

(for some drugs at least)

Thanks!

Questions?

79

Contact: @codybuntain

[email protected]

mailto:[email protected]

This is your Twitter on drugs. Any Questions?

Science

Transcript of This is your Twitter on drugs. Any Questions?