Social Media Monitoring to Identify Risk
47
May 25, 2016 Nabarun Dasgupta, Chief Science Officer Epidemico, Inc. felming • 3 rd Pharmacovigilance & Risk Management Strategies Philadelphia Social Media Monitoring to Identify Risk
-
Upload
epidemico -
Category
Data & Analytics
-
view
301 -
download
0
Transcript of Social Media Monitoring to Identify Risk
May 25, 2016
Nabarun Dasgupta, Chief Science Officer
Epidemico, Inc.
felming • 3rd Pharmacovigilance & Risk Management Strategies Philadelphia
Social Media Monitoring
to Identify Risk
Disclosures
Epidemico is a wholly-owned subsidiary of Booz Allen Hamilton
Epidemico is commercializing aspects of this research.
FDA has been funding research technology for 3.5 years.
Office of the Chief Scientist (OCS)
Center for Tobacco Products (CTP)
Office of International Programs (OIP)
GSK has been a development partner to
extend technology and obtain GxP software
validation over the last 3 years.
A public-private partnership entering Year 3.
The WEB-RADR project is supported by the Innovative Medicines
Initiative Joint Undertaking (IMI JU) under grant agreement n°
115632, resources of which are composed of financial contributions
from the European Union's Seventh Framework Programme
(FP7/2007-2013) and EFPIA companies’ in kind contribution.
Introduction
Infographic elementsSlide sub title
The Drug I’m TakingThe Drug I’m Responsible for Monitoring
Proto-AEPosts with resemblance to Adverse Events
AstheniaMedDRA: 10003549
DizzinessMedDRA: 10013573
Data Processing
Collect unstructured data from
social media APIs, third-party
authorized resellers, and
automated scraping.
Acquire
detectsentiment
languagetranslation
statistics consolidatemultiples
detectadverseevents
detectbenefits
de-identifygeo-tag
Data are passed through a series
of apps, emerging as meaningful
bits of information.
Process
detectsentiment
languagetranslation
statistics consolidatemultiples
detectadverseevents
detectbenefits
de-identifygeo-tag API mobile RSS tables reportsCSV
,visuals
Relevant data are passed to another
series of apps in preparation for
human interpretation and analysis.
Export
API mobile RSS tables reportsCSV
,visuals
Relevant data are passed to another
series of apps in preparation for
human interpretation and analysis.
Export
Internet Vernacular
Translation
lost their eyesight
seeing weird color
seeing weird colour
doublevision
couldn’t see
visión doble
blind
googley eyed
blurry vision
changes in visioncross eyed
seeing weird
vision change
blindness
cross visionvisual snow
googly eyed
seeing double
making me eat like a mouse
lost appetite
#notevenhungry
appetite is nonexistent
apetite surpressed
didn’t get hungry
dont want to eat
killed my apetite
miss feeling hungry
killed my appetite
can’t eat
sin hambre
lost my appetite
no appetitey
lack of apetite
stomach small
lost teh appetite
never hungrynever want to eat
cant eat
couldntcrosseyed
blurry
anorexic
apetite surpressed
lack of apetite
killed my apetite lost teh appetite
making me eat like a mouse
never want to eat
Implied
#notevenhungry
no appetitey
Invented words and hashtags
googley eyed
googly eyed
seeing weird colour
seeing weird color
Varied Spelling
sin hambre
visión doble
Other Languages
blurry vision
Emoticons
Typos
making me eat like a mouse
anorexic
lost appetite
#notevenhungry
appetite is nonexistent
apetite surpressed
didn’t get hungry
dont want to eat
miss feeling hungry
killed my appetite
can’t eat
sin hambre
lost my appetite
no appetitey
lack of apetite
stomach small
lost teh appetite
never hungrynever want to eat
cant eat
lost their eyesight
seeing weird color
seeing weird colour
doublevision
couldn’t see
visión doble
blind
googley eyed
changes in visioncross eyed
seeing weird
vision change
blindness
cross visionvisual snow
googly eyed
seeing double
couldntcrosseyed
blurry
killed my apetite
blurry vision
Decreased appetite
MedDRA 10061428
Loss of appetite
SNOMED 79890006
Visual impairment
MedDRA 10047571
Visual impairment
SNOMED 397540003
Machine Learning
vs.
Natural Language Processing
Machine LearningFind reports of importance.
Natural Language ProcessingExtract the most relevant elements.
You are all MedDRA Certified Coders, right?
How to build a machine
learning classifier
Proto-AE Trash
Medicine Prozak® (Fluoxetine) for the treatment of bulimia
nervosa . Buy now Prozac-20mg "Fluoxetine" ...
Benefit
81%27%74%4%2%75%92%
Instant Feedback:
% Confidence that this is a Proto-AE
You just trained the classifier.
Congratulations!
System Performance
82,000,000+posts collected
Public posts from Facebook, Twitter, and patient forums back to 2012.
2,500+regulated products
Drugs, medical devices, vaccines, biologics,
tobacco products, supply chain
400,000+manually curated
Certified coders train machine learningclassifiers and build dictionary.
10,000+Vernacular English phrases
Updated daily by human curators.
Mandarin, French, Spanish, and Dutch next.
Positive Predictive Value or Precision
7 OF 10 posts contain adverse event information (0.68).
Can increase to 100% with manual curation (may vary by product).
If the algorithm says it is a Proto-AE, then 7 out of 10 times is actually is.
Sensitivity or Recall
Automated tools identify 9 OF 10 adverse events
across all products, all time, all data sources (0.88).
The algorithm correctly identifies 9 out of 10 Proto-AEs from the pool of everything.
Albuterol/Salbutamol
Jan 1 to May 24, 2016
In their own words. Indicatorscore
5.25% Twitter posts are Proto-AEs.
There were 10,185 mentions from Jan 1 to May 24, 2016.
Of these, 535 posts had adverse event information.
System organ class (n=535)
Nervous system
General Medical Psychiatric Injury &
Poisoning
Infections
Respiratory
Tremor
Altered state of
consciousness
Drug
ineffective
Insomnia
Bronchitis
Feeling
jittery
Incorrect
dose
Malaise
Hyper-
sensitivity
Non-
specific
rxn
Cough
Restlessness
Pain
Psycho-
motor
hyper-
activity
Dizziness OverdoseHeart
rate
increased
Lung
disorderMemory
impairmentSomnolence Fatigue Dependence
Top preferred terms for Proto-AEs (n=535)
Disproportionately reported Proto-AEs match clinical
trials - with additional AEs not seen in RCTs.
Social media data since 2013; RCT data from NIH DailyMed
RCT PrevelanceNervousness 20%
Tremor 20%
Headache 7%
Tachycardia 5%
Palpitations 5%
Muscle cramps 3%
Sleeplessness 2%
Weakness 2%
Dizziness 2%
Nausea 2%
Top Social Media Proto-AEs by PRRN PRR
Arrhythmia 2 123
Tachycardia 14 45
Tremor 1084 37
Feeling jittery 132 35
Palpitations 56 32
Mouth ulceration 5 31
Candida infection 2 14.7
Heart rate increased 60 11.0
Epistaxis 17 8.8
Pallor 3 8.8
What are good use cases
for social data in safety
and risk management?
Social media can fill knowledge gaps.
In addition to adverse events, the pharmaceutical companies using our social media data have identified
several use cases well-suited for social listening. Social media have already led to product defect recalls,
revision of markings for dosing on a syringe, AE labeling revision requests, and formal safety study
commitments.
http://www.gsk-clinicalstudyregister.com/study/202115
qualitycomplaints
clinicaltrials
benefits alcoholinteractions
drugabuse
patientquestions
counterfeit
What about bias?
All data have bias.Can we interpret the output within context and derive insight?
When do we actually listen to patients?
Each source of safety information provides different perspectives on patient experience.
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Clinical Trials Sponsor AE Practice Data Social Media
PRO Seriousness Completeness
Social media can elucidate what
is important to patients.
1. Patient-Reported Outcomes
Traditional PV focus on most serious of events
may be less central in social media.
2. Seriousness
Social media posts may by less complete,
but may deliver novel insight in aggregate.
3. Completeness
Three Dimensions of Safety Information
Hypothetical data for illustrative purposes only.
What do regulations say
about reporting of
adverse events in social
media?
Our observations.
It is clear that if the AE is on a company-owned social marketing platform, it has to be reported. However, anonymized data from general social media sites may be handled as observational data. The EMA GVP VI module has additional expectations. The IMI WEB-RADR project will provide research guidance for regulatory agencies in 2017. In the US, the focus is on multi-source causality, which can include social media.
But, in practice, the legal/compliance departments within each pharmaceutical company have their unique interpretations.
This work is licensed under the Creative Commons
Attribution-ShareAlike 4.0 International License.
To view a copy of this license, visit
http://creativecommons.org/licenses/by-sa/4.0/
Slides available at @epidemico
