Social Media Measurement 101 + Day of aTONEment
-
Upload
alan-chumley -
Category
Documents
-
view
1.503 -
download
0
description
Transcript of Social Media Measurement 101 + Day of aTONEment
SOCIAL MEDIASOCIAL MEDIA MEASUREMENT 101A Day of aTONEmentA Day of aTONEment
MRA ANNUAL CONFERENCE
JUNE 6, 2011
Alan Chumley
Senior Vice President
@ l h l / @CARMA T t@alanchumley / @CARMA_Tweets
1615 M Street, NW; Suite 750 Washington, DC 20036 USA Tel 1.202.842.1818carma.com
ALAN CHUMLEY, SVP
• 14 years in the industry
R i PR titi ING B ll C d• Recovering PR practitioner: ING, Bell Canada
• Former Director of Measurement @ Hill & Knowlton
• Masters in Communications & Culture:
• Research foci:
• media effects & uses
• reception analysisreception analysis
• audience studies
• University‐level adjunct instructor of PR measurement
Li k dI l 2006• LinkedIn early 2006
• Blogging / Tweeting since early 2007
1615 M Street, NW; Suite 750 Washington, DC 20036 USA Tel 1.202.842.1818carma.com 2
@alanchumley / #mra_ac
ABOUT CARMA
• Traditional & social
M di it i & l i• Media monitoring & analysis
• Largely human‐based….on an automation journey
• In biz since 1984
• Head office here in DC
• 14 offices in 11 countries14 offices in 11 countries
• Clients:
PR f lk i bi i ti• PR folk in big organizations
• Market research folks in big organizations are starting to emerge
1615 M Street, NW; Suite 750 Washington, DC 20036 USA Tel 1.202.842.1818carma.com 3
@alanchumley / #mra_ac
PART 1 (1.5 hrs): SOCIAL MEDIA MEASUREMENT
IN-ROOM STRETCH (5 mins)
PART 2 (30 i ) TONE ALONEPART 2 (30 mins): TONE ALONE
PART 3: (35 mins): REAL DATA WITH MALCOM DE LEO NETBASE
1615 M Street, NW; Suite 750 Washington, DC 20036 USA Tel 1.202.842.1818carma.com
PART 1 (1.5 hrs): SOCIAL MEDIA MEASUREMENT
1615 M Street, NW; Suite 750 Washington, DC 20036 USA Tel 1.202.842.1818carma.com
ISN’T
• Not a Case Study
• Not a mention of Old Spice in sight
• Not a specific ‘how to’ / no one-size-fits all
• Doesn’t focus on a singular or few social media platforms—looks at all holistically
1615 M Street, NW; Suite 750 Washington, DC 20036 USA Tel 1.202.842.1818carma.com
@alanchumley / #mra_ac
ISN’T IS
• An emerging / experimental philosophy / approach / filter / lens through which to look at measuring social media
• Not a Case Study
• Not a mention of Old Spice in sight
• A high level connected / systems way of approaching the problem
b d
• Not a specific ‘how to’ / no one-size-fits all
• About communities not an audience
• Beyond simple counts and sentiment
• Doesn’t focus on a singular or few social media platforms—looks at all holistically
• More sociologist than publicist
Influence potential
1615 M Street, NW; Suite 750 Washington, DC 20036 USA Tel 1.202.842.1818carma.com
• Influence potential
@alanchumley / #mra_ac
Every hour more than 500,000 new blog + micro blog posts status updatesblog + micro-blog posts, status updates and comments are posted
1615 M Street, NW; Suite 750 Washington, DC 20036 USA Tel 1.202.842.1818carma.com
MEASUREMENT ACCOUNTS FOR 5 / 7 TOP SOCIAL MEDIA CHALLENGES
Why is this the case?Gap between leap ofGap between leap of faith true believers and C-suite’s need for metrics to validate the
$
Source: Harvard Business Review: the New Conversation: Taking Social Media from Talk to Action
Surveyed 2100 Harvard Biz Review subscribers
time/$?
1615 M Street, NW; Suite 750 Washington, DC 20036 USA Tel 1.202.842.1818carma.com 9
@alanchumley / #mra_ac
SOCIAL MEDIA IS A GAME CHANGER… …& A SOURCE OF ANXIETY
Top Down Command & Control
of the Message is Gone
• Transparency demanded
• Dialogue vs. Monologue
• Many-to-many not one-to-many
• Conversation & community NOT (strictly) coverageNOT (strictly) coverage
• About relationships
• Relationship trust is co-created by the organization and its communities of interest via conversation
1615 M Street, NW; Suite 750 Washington, DC 20036 USA Tel 1.202.842.1818carma.com
@alanchumley / #mra_ac
SOCIAL MEDIA IS A GAME CHANGER… …& A SOURCE OF ANXIETY
Top Down Command & Control
of the Message is Gone
There is no Passive Ready-to-Consume Audience’
• Transparency demanded
• Dialogue vs. Monologue
• Prosumers not just consumers
• Actively engaged in creating and re-creating conversations and
• Many-to-many not one-to-many
• Conversation & community NOT (strictly) coverage
gcommunities
• Challenge in tracking & measuring something as nebulous as a communityNOT (strictly) coverage
• About relationships
• Relationship trust is co-
nebulous as a community
created by the organization and its communities of interest via conversation
1615 M Street, NW; Suite 750 Washington, DC 20036 USA Tel 1.202.842.1818carma.com
@alanchumley / #mra_ac
SOCIAL MEDIA IS A GAME CHANGER… …& A SOURCE OF ANXIETY
Top Down Command & Control
of the Message is Gone
There is no Passive Ready-to-Consumer ‘Audience’
Disciplinary Lines are Eroding
• Transparency demanded
• Dialogue vs. Monologue
• Prosumers not just consumers
• Actively engaged in creating and re-creating conversations and
• PA, PR, Marketing, CRM, Customer Service etc.
• Customer touch points
• Many-to-many not one-to-many
• Conversation & community NOT (strictly) coverage
gcommunities
• Challenge in tracking & measuring something as nebulous as a community
• Integrated cross-discipline re: strategy
• Integrate cross-discipline NOT (strictly) coverage
• About relationships
• Relationship trust is co-
nebulous as a community re: measurement
created by the organization and its communities of interest via conversation
1615 M Street, NW; Suite 750 Washington, DC 20036 USA Tel 1.202.842.1818carma.com
@alanchumley / #mra_ac
SOCIAL MEDIA IS A GAME CHANGER… …& A SOURCE OF ANXIETY
Top Down Command & Control
of the Message is Gone
There is no Passive Ready-to-Consumer ‘Audience’
Disciplinary Lines are Eroding
• Transparency demanded
• Dialogue vs. Monologue
• Prosumers not just consumers
• Actively engaged in creating and re-creating conversations and
• PA, PR, Marketing, CRM, Customer Service etc.
• Customer touch points
• Many-to-many not one-to-many
• Conversation & community NOT (strictly) coverage
gcommunities
• Challenge in tracking & measuring something as nebulous as a community
• Integrated cross-discipline re: strategy
• Integrate cross-discipline NOT (strictly) coverage
• About relationships
• Relationship trust is co-
nebulous as a community re: measurement
C Suite is Intrigued but Anxious / Want Data to:created by the organization and its communities of interest via conversation
C-Suite is Intrigued but Anxious / Want Data to:• Show why and how they should…• Demonstrate the value of having done so…
1615 M Street, NW; Suite 750 Washington, DC 20036 USA Tel 1.202.842.1818carma.com
@alanchumley / #mra_ac
SIMPLE STATUS QUO
Counts:
Quick Quantity counts:•Fans Likes Follows Etc•Fans, Likes, Follows, Etc.•Tantamount to reach in the offline or 1.0 world•Reach does not equal Influence
Quality:•Quick automated read on sentiment: but concerns re: accuracy
Rankings:
•Visibility, popularity, traffic, link loveVisibility, popularity, traffic, link love•Technorati for blogs•NOT RECOMMENDED: Twitrratr, Klout, •BETTER: Twendz, Traackr, Twittalyzer, Twinfluence
The rarel q alif people or content (the message)•They rarely qualify people or content (the message)
•The message AND the medium are the measure
1615 M Street, NW; Suite 750 Washington, DC 20036 USA Tel 1.202.842.1818carma.com 14
@alanchumley / #mra_ac
LOT’S OF TOOLS OUT THERE
BEWARE OF THE SILOs AVOID USING (ONLY) THE NEW SHINEY FREEBIES
•Isolated / silo’d approach•No one tool on it’s own is sufficient not even one paid tool•No one tool on it s own is sufficient—not even one paid tool•Have to connect the dots with a more relational, systems approach
Would you like your job performance (read: raise/bonus) to be based only on a
1615 M Street, NW; Suite 750 Washington, DC 20036 USA Tel 1.202.842.1818carma.com
(read: raise/bonus) to be based only on a Klout Score, for example?
@alanchumley / #mra_ac
REACH SHOULD BE A RELATIVE NOT AN ABSOLUTE& NOT ALL CONVERSATIONS ARE RELEVANT
Off topic / Irrelevant
R l t “ ”R l tRelevant “+” Relevant “-”Relevant Neutral
1615 M Street, NW; Suite 750 Washington, DC 20036 USA Tel 1.202.842.1818carma.com 16
@alanchumley / #mra_ac
REMEMBER: IT’S SOCIAL MEDIA
•It’s not coverage
•It’s not staticIt s not static
•There isn’t an ‘audience’ of consumers
•Prosumers not just consumers:•Prosumers not just consumers: •simultaneously receiving, interpreting, co-opting, re-appropriating and creating content
It’ t di l th t l•It’s more many-to-many dialogue than one-to-many monologue
•So, traditional off line ways of measuring ‘coverage’ don’t cut it
•They don’t adequately account for the multi-dimensionality and dynamic nature of a conversation and a community of interest in a network
•We need to think bigger
1615 M Street, NW; Suite 750 Washington, DC 20036 USA Tel 1.202.842.1818carma.com 17
gg
@alanchumley / #mra_ac
THINK BIGGER / INTEGRATE / CORRELATE
•Think Bigger / Broader Beyond PR’s Historical Silo –social Media’s Pushing us all that Way •Think about all customer‐facing touch points•Understand PR / Marketing / Business Metrics •Integrate Cross‐Disciplineg p•Report on the Critical Few•Focus on /correlate with Outcomes•Close the loop / connect the dots
S W b Sh d i k M t & St t P ti S Oli Bl h d S i l M di ROI
1615 M Street, NW; Suite 750 Washington, DC 20036 USA Tel 1.202.842.1818carma.com
Source: Weber-Shandwick Measurement & Strategy Practice Source: Oliver Blanchard, Social Media ROI
18
@alanchumley / #mra_ac
THE MEASUREMENT MASHUP
1615 M Street, NW; Suite 750 Washington, DC 20036 USA Tel 1.202.842.1818carma.com 19
@alanchumley / #mra_ac
THINK BIGGER & BROADER MOVE BEYOND ‘MOMENT-IN-TIME’ or ‘VANITY’ METRICS
S t N t k dSystems or Networked Thinking:
Interdependent relationships between pthose in a network
Social Capital:
Connections within and between those in a social network as a source of civic engagement among
1615 M Street, NW; Suite 750 Washington, DC 20036 USA Tel 1.202.842.1818carma.com 20
a community leading to cohesion; that has ‘value’
@alanchumley / #mra_ac
FOCUSING ON CONTENT ANALYSIS AND NETWORK ANALYSISAS A WAY OF MEASURING INFLUENCE POTENTIAL
S t N t k dSystems or Networked Thinking:
Interdependent relationships between pthose in a network
Social Capital:
Connections within and between those in a social network as a source of civic engagement among
1615 M Street, NW; Suite 750 Washington, DC 20036 USA Tel 1.202.842.1818carma.com 21
a community leading to cohesion; that has ‘value’
@alanchumley / #mra_ac
The 7Cs THINKING MORE LIKE A SOCIOLOGIST THAN PUBLICIST
Counting What some call search, syndication and site; stuff we can count like traffic, popularity, fans, followers, retweets, etc.
Content Analysis that is. Using content analysis, ideally a hybrid of some automation for volume and speed and Content y g y , y y phuman analysts for context, nuance and depth to look for how your brand, messaging, topics, issues, stakeholder voices are portrayed; the inclusiveness, accuracy and tone of the conversations.
Conversations And conversationships recognizing that social media is not strictly another communications channel through which to get word out or reach and influence. It is more about genuinely engaged dialogue than a simple corporate monologue; not only a vertical, one‐to‐many conversation between anthan a simple corporate monologue; not only a vertical, one to many conversation between an organization and stakeholders but a lateral, many‐to‐many conversation among stakeholders.
Cohesion The extent to which folks are agreeing with you, your position, your brand essences. The extent to which they are agreeing with each other and coalescing around and advocating a core theme, idea, or some call to action either in support or opposition to your organization’s positionsome call to action either in support or opposition to your organization s position.
Community The extent to which a core group of people with common interests are gathering and growing. Measuring the change in community size, and volume and nature of chatter over time
Connectedness A look at how inter‐connected the highly engaged/key influencers/advocates are and how centrally Connectednesslocated they are in both the network and dialogue. (See Appendix for the 5Ps of influence)
Conversion Stay tuned. More to follow.
1615 M Street, NW; Suite 750 Washington, DC 20036 USA Tel 1.202.842.1818carma.com 22
@alanchumley / #mra_ac
THE 5Ps OF INFLUENCE
Popular Visible, vocal, has a substantial following, reach. In‐bound links, trackbacks, subscribers, bookmarks, followers, friends, views, listens, saves, downloads, etc.
Polarized in tone Neutrality does little to drive influence way or the other. A clearly positive or negative view will polarize readers/followers and is more likely to drive cohesion and mobilize advocates and have those advocates coalesce around a core theme, idea, or call to action.
Prolific / Relevant / Frequent
Raw author contribution and # of on‐topic, related posts
Prominent / Are they an idea starter or spreader; source or spider? They may be prolific but are they prominent? Are they highly inter related inter connected and centrally located in the
Authoritativeprominent? Are they highly inter‐related, inter‐connected, and centrally located in the network? How engaged is this person’s following in a dialogue? How much dialogue is there and what is its nature? Here we need to recognize, though, that authority is contextual and topical. One might be an authority on PR measurement but not on 18th century Russian literature.
/ l H f th f ll / t t ti t ib t d ti d iPromoter / Loyal Advocate
How many of the followers/commentators active contributors advocating, endorsing, advancing (or the opposite) your position? Are they adding links, tags. Is the nature of the language they are using inter‐connective, expanded, clarifying, reinterpreting? RTs, digs, fans, votes, buzz‐ups, up/downloads, shares, likes, invites, favorites, embeds. (More active than the metrics in popularity)
1615 M Street, NW; Suite 750 Washington, DC 20036 USA Tel 1.202.842.1818carma.com 23
@alanchumley / #mra_ac
VARIETY OF OBJECTIVES & CAMPAIGN TYPES (NON-EXHAUSTIVE)
Objectives
• Visibility
Campaign Types
• Visibility• Visits• Volume• Velocity
• Push (get bloggers to review / write)• Pull (drive visitors on‐line)• CSR• Product Launch
• Awareness• Message delivery• Link love• Dialogue vs Monologue
• Product Launch• Product Engagement• Marketing• Blogger Outreach
• Dialogue vs. Monologue• Generate dialogue / comments• Feedback• Using, sharing, pass‐a‐long
• Advocacy• Issues Management• Community building• Customer relations
• Embedding of interactive content• Diggs, Delicious tags, stumbles, etc.• Votes• Etc., etc., etc.
Customer relations• Thought leadership • Executive profiling• Etc., etc., etc.
1615 M Street, NW; Suite 750 Washington, DC 20036 USA Tel 1.202.842.1818carma.com
Etc., etc., etc.
24
@alanchumley / #mra_ac
SOME THINKING ON AN APPROACHAN INDEX IS ONLY PART OF THE ANSWER
1615 M Street, NW; Suite 750 Washington, DC 20036 USA Tel 1.202.842.1818carma.com 25
@alanchumley / #mra_ac
COMPONENTS OF AN INDEX ?
Popularity + Relevance + Authority + Presence + Engagement + Advocacy/Loyalty + Cascade
x (+/‐) Sentiment
=
Influence Potential
• Based on an assumption that the components above are drivers of on-line influence, potential action
• They can be measured individually, but it’s important to measure them collectively as they are inter-related
• The index is intended as a measure of potential to influence not actual real influence that is another• The index is intended as a measure of potential to influence not actual real influence—that is another research project all together.
• All components are important (but not equal) drivers of potential influence on-line
1615 M Street, NW; Suite 750 Washington, DC 20036 USA Tel 1.202.842.1818carma.com
p p ( q ) p
26
@alanchumley / #mra_ac
INDICATORS DEFINED
• Cascade• Citation analysis / ripple index• Network analysis
V l & l i f d h
•Authority•Of both poster and commentators•Idea starters or spreaders?•Degree of inter connectedness / inter relatedness
High authority
low relevance
High authority AND
relevance• Volume & velocity of message spread across the network
• Inter‐connectedness / inter‐relationships among:• all• Only the influentials
• Advocacy (active)
•Degree of inter‐connectedness / inter‐relatedness•Connected but respected?•How central is the influencer to the network?•Following: follower•RTs/1000 followers
• Rele ance
relevance relevance
Low authority AND
relevance
High relevance
low authority
y ( )• Are social media users advancing our cause?• % containing links• % active contributors• Tags added
• Engagement (more coming)
• Relevance• On topic?• Author contribution: how often / prolific?• Topical / textual / temporary
S ti t ( i )Engagement (more coming)• Dialogue? 2‐way? How much? How good? • How many commentators? Repeats? Nature of
the language• Network analysis: inter‐connected/related
• Entity Presence
• Sentiment (more coming)• Multi‐level, multi‐author tone• Post‐level, theme level, sentence level• Gaps in tone in originating content and resulting
discussion
Entity Presence• Present? Prominent? Frequent?• # of + mentions• Competitive share of original post• Competitive share of resulting discussion• Attribution recognition
• Popularity (passive) • ‘Reach’ ?• Voice of the consumer stuff we can count• Links, votes, shares, recommends, subscribers,
registrations, tweets, RTs, followers, fans, friends, diggs shares likes fav’s upvotes buzzups
1615 M Street, NW; Suite 750 Washington, DC 20036 USA Tel 1.202.842.1818carma.com
diggs, shares, likes, fav s, upvotes, buzzups, embeds, tags, wish list saves, forwards, invites, etc.
27
@alanchumley / @CARMA_Tweets
NETWORK ANALYSIS
How are these changing over time?
Source: K. Ognyanova, UCLA, 2010
Might be part of many systems / networks?
Linking networks to networks
1615 M Street, NW; Suite 750 Washington, DC 20036 USA Tel 1.202.842.1818carma.com
@alanchumley / @CARMA_Tweets
(A QUICK ASIDE ON) SENTIMENT AS MARKET RESEARCH PROXY / PREDICTOR
Think here about social media monitoring and analysis as:
•Real time situational awareness•Real time situational awareness•Listening at the point of need•Listening close to the moment of influence
A real-time digital form of or proxy for market research and CRM:
Mining conversations to tease out how people ‘feel’ about products/services/issues/causes and perhaps even:and perhaps even:
1) their intended (reported) behavior and what motivates them to buy, (as cited in the conversations), and
2) customer satisfaction
3)their actual behavior tracked via CRM 2.0 and Web 2.0 analytics
1615 M Street, NW; Suite 750 Washington, DC 20036 USA Tel 1.202.842.1818carma.com
y
29
Source: Crimson Hexagon
@alanchumley / #mra_ac
BUT REMEMBER THAT OBJECTIVES DRIVE METRICS
Objectives
• Visibility
Campaign Types
• Visibility• Visits• Volume• Velocity
• Push (get bloggers to review / write)• Pull (drive visitors on‐line)• CSR• Product Launch
• Awareness• Message delivery• Link love• Dialogue vs Monologue
• Product Launch• Product Engagement• Marketing• Blogger Outreach
• Dialogue vs. Monologue• Generate dialogue / comments• Feedback• Using, sharing, pass‐a‐long
• Advocacy• Issues Management• Community building• Customer relations
• Embedding of interactive content• Diggs, Delicious tags, stumbles, etc.• Votes• Etc., etc., etc.
Customer relations• Thought leadership • Executive profiling• Etc., etc., etc.
1615 M Street, NW; Suite 750 Washington, DC 20036 USA Tel 1.202.842.1818carma.com
Etc., etc., etc.
30
@alanchumley / #mra_ac
IMAGINE A DIFF’T. PYRAMID FOR DIFF’T. CAMPAIGN TYPES / OBJECTIVES
1615 M Street, NW; Suite 750 Washington, DC 20036 USA Tel 1.202.842.1818carma.com 31
@alanchumley / #mra_ac
AN INDEX WITH MOVABLE INDICATORSFLEXIBLE ENOUGH TO ACCOUNT FOR DIFFERENT OBJECTIVES
Indicator Popularity Relevance Authority Brand Engagement Advocacy Cascade SentimentPresence
Score 10 10 10 10 10 10 10 10Weight 1 2 3 4 5 6 7
(@ d f lt)(@ default)
Popularity + Relevance + Authority + Presence + Engagement + Advocacy + Cascade X Sentiment = Influence Potential
• Each of the above indicators can be adjusted to account for different types of campaigns & contexts
• The weights would change. The scores don’t.
• For an organization, new to social media, perhaps they might only want to establish an on‐line presence
• So we’d dial or slide the weights of all but the popularity indicator down to zero
Has a significantly scaled back / rudimentary version of this. This is inspired by but significantly expands on R6 thinking.
1615 M Street, NW; Suite 750 Washington, DC 20036 USA Tel 1.202.842.1818carma.com 32
p y g y p g
@alanchumley / #mra_ac
AN INDEX WITH MOVABLE INDICATORSFLEXIBLE ENOUGH TO ACCOUNT FOR DIFFERENT OBJECTIVES
Indicator Popularity Relevance Authority Brand Engagement Advocacy Cascade SentimentPresence
Score 10 10 10 10 10 10 10 10Weight 1 2 3 4 5 6 7
(@ d f lt)(@ default)
Popularity + Relevance + Authority + Presence + Engagement + Advocacy + Cascade X Sentiment = Influence Potential
• Each of the above indicators can be adjusted to account for different types of campaigns & contexts
• The weights would change. The scores don’t.
• For an organization, new to social media, perhaps they might only want to establish an on‐line presence
• So we’d dial or slide the weights of all but the popularity indicator down to zero
Has a significantly scaled back / rudimentary version of this. This is inspired by but significantly expands on R6 thinking.
1615 M Street, NW; Suite 750 Washington, DC 20036 USA Tel 1.202.842.1818carma.com 33
p y g y p g
@alanchumley / #mra_ac
AN INDEX WITH MOVABLE INDICATORSFLEXIBLE ENOUGH TO ACCOUNT FOR DIFFERENT OBJECTIVES
Indicator Popularity Relevance Authority Brand Engagement Advocacy Cascade SentimentPresence
Score 10 10 10 10 10 10 10 10Weight 1 2 3 4 5 6 7
(@ d f lt)(@ default)
Popularity + Relevance + Authority + Presence + Engagement + Advocacy + Cascade X Sentiment = Influence Potential
• Each of the above indicators can be adjusted to account for different types of campaigns & contexts
• The weights would change. The scores don’t.
• For an organization, new to social media, perhaps they might only want to establish an on‐line presence
• So we’d dial or slide the weights of all but the popularity indicator down to zero
Has a significantly scaled back / rudimentary version of this. This is inspired by but significantly expands on R6 thinking.
1615 M Street, NW; Suite 750 Washington, DC 20036 USA Tel 1.202.842.1818carma.com 34
p y g y p g
@alanchumley / #mra_ac
AN INDEX WITH MOVABLE INDICATORSFLEXIBLE ENOUGH TO ACCOUNT FOR DIFFERENT OBJECTIVES
Indicator Popularity Relevance Authority Brand Engagement Advocacy Cascade SentimentPresence
Score 10 10 10 10 10 10 10 10Weight 1 2 3 4 5 6 7
(@ d f lt)(@ default)
Popularity + Relevance + Authority + Presence + Engagement + Advocacy + Cascade X Sentiment = Influence Potential
• Each of the above indicators can be adjusted to account for different types of campaigns & contexts
• The weights would change. The scores don’t.
• For an organization, new to social media, perhaps they might only want to establish an on‐line presence
• So we’d dial or slide the weights of all but the popularity indicator down to zero
Has a significantly scaled back / rudimentary version of this. This is inspired by but significantly expands on R6 thinking.
1615 M Street, NW; Suite 750 Washington, DC 20036 USA Tel 1.202.842.1818carma.com 35
p y g y p g
@alanchumley / #mra_ac
AN INDEX WITH MOVABLE INDICATORSFLEXIBLE ENOUGH TO ACCOUNT FOR DIFFERENT OBJECTIVES
Indicator Popularity Relevance Authority Brand Engagement Advocacy Cascade SentimentPresence
Score 10 10 10 10 10 10 10 10Weight 1 2 3 4 5 6 7
(@ d f lt)(@ default)
Popularity + Relevance + Authority + Presence + Engagement + Advocacy + Cascade X Sentiment = Influence Potential
• Each of the above indicators can be adjusted to account for different types of campaigns & contexts
• The weights would change. The scores don’t.
• For an organization, new to social media, perhaps they might only want to establish an on‐line presence
• So we’d dial or slide the weights of all but the popularity indicator down to zero
Has a significantly scaled back / rudimentary version of this. This is inspired by but significantly expands on R6 thinking.
1615 M Street, NW; Suite 750 Washington, DC 20036 USA Tel 1.202.842.1818carma.com 36
p y g y p g
@alanchumley / #mra_ac
AN INDEX WITH MOVABLE INDICATORSFLEXIBLE ENOUGH TO ACCOUNT FOR DIFFERENT OBJECTIVES
Indicator Popularity Relevance Authority Brand Engagement Advocacy Cascade SentimentPresence
Score 10 10 10 10 10 10 10 10Weight 1 2 3 4 5 6 7
(@ d f lt)(@ default)
Popularity + Relevance + Authority + Presence + Engagement + Advocacy + Cascade X Sentiment = Influence Potential
• Each of the above indicators can be adjusted to account for different types of campaigns & contexts
• The weights would change. The scores don’t.
• For an organization, new to social media, perhaps they might only want to establish an on‐line presence
• So we’d dial or slide the weights of all but the popularity indicator down to zero
Has a significantly scaled back / rudimentary version of this. This is inspired by but significantly expands on R6 thinking.
1615 M Street, NW; Suite 750 Washington, DC 20036 USA Tel 1.202.842.1818carma.com 37
p y g y p g
@alanchumley / #mra_ac
AN INDEX WITH MOVABLE INDICATORSFLEXIBLE ENOUGH TO ACCOUNT FOR DIFFERENT OBJECTIVES
Indicator Popularity Relevance Authority Brand Engagement Advocacy Cascade SentimentPresence
Score 10 10 10 10 10 10 10 10Weight 1 2 3 4 5 6 7
(@ d f lt)(@ default)
Popularity + Relevance + Authority + Presence + Engagement + Advocacy + Cascade X Sentiment = Influence Potential
• Each of the above indicators can be adjusted to account for different types of campaigns & contexts
• The weights would change. The scores don’t.
• For an organization, new to social media, perhaps they might only want to establish an on‐line presence
• So we’d dial or slide the weights of all but the popularity indicator down to zero
Has a significantly scaled back / rudimentary version of this. This is inspired by but significantly expands on R6 thinking.
1615 M Street, NW; Suite 750 Washington, DC 20036 USA Tel 1.202.842.1818carma.com 38
p y g y p g
@alanchumley / #mra_ac
AN INDEX WITH MOVABLE INDICATORSFLEXIBLE ENOUGH TO ACCOUNT FOR DIFFERENT OBJECTIVES
Indicator Popularity Relevance Authority Brand Engagement Advocacy Cascade SentimentPresence
Score 10 10 10 10 10 10 10 10Weight 1 2 3 4 5 6 7
(@ d f lt)(@ default)
Popularity + Relevance + Authority + Presence + Engagement + Advocacy + Cascade X Sentiment = Influence Potential
• Each of the above indicators can be adjusted to account for different types of campaigns & contexts
• The weights would change. The scores don’t.
• For an organization, new to social media, perhaps they might only want to establish an on‐line presence
• So we’d dial or slide the weights of all but the popularity indicator down to zero
Has a significantly scaled back / rudimentary version of this. This is inspired by but significantly expands on R6 thinking.
1615 M Street, NW; Suite 750 Washington, DC 20036 USA Tel 1.202.842.1818carma.com 39
p y g y p g
@alanchumley / #mra_ac
AN INDEX WITH MOVABLE INDICATORSFLEXIBLE ENOUGH TO ACCOUNT FOR DIFFERENT OBJECTIVES
Indicator Popularity Relevance Authority Brand Engagement Advocacy Cascade SentimentPresence
Score 10 10 10 10 10 10 10 10Weight 1 2 3 4 5 6 7
(@ d f lt)(@ default)
Popularity + Relevance + Authority + Presence + Engagement + Advocacy + Cascade X Sentiment = Influence Potential
• Each of the above indicators can be adjusted to account for different types of campaigns & contexts
• The weights would change. The scores don’t.
• For an organization, new to social media, perhaps they might only want to establish an on‐line presence
• So we’d dial or slide the weights of all but the popularity indicator down to zero
Has a significantly scaled back / rudimentary version of this. This is inspired by but significantly expands on R6 thinking.
1615 M Street, NW; Suite 750 Washington, DC 20036 USA Tel 1.202.842.1818carma.com 40
p y g y p g
@alanchumley / #mra_ac
AN INDEX WITH MOVABLE INDICATORSFLEXIBLE ENOUGH TO ACCOUNT FOR DIFFERENT OBJECTIVES
Indicator Popularity Relevance Authority Brand Engagement Advocacy Cascade SentimentPresence
Score 10 10 10 10 10 10 10 10Weight 1 2 3 4 5 6 7
(@ d f lt)(@ default)
Popularity + Relevance + Authority + Presence + Engagement + Advocacy + Cascade X Sentiment = Influence Potential
• Each of the above indicators can be adjusted to account for different types of campaigns & contexts
• The weights would change. The scores don’t.
• For an organization, new to social media, perhaps they might only want to establish an on‐line presence
• So we’d dial or slide the weights of all but the popularity indicator down to zero
Has a significantly scaled back / rudimentary version of this. This is inspired by but significantly expands on R6 thinking.
1615 M Street, NW; Suite 750 Washington, DC 20036 USA Tel 1.202.842.1818carma.com 41
p y g y p g
@alanchumley / #mra_ac
WHAT DOES THE INDEX INDICATE?
• It’s a relative measure of the quality of social media coverage:– One post relative to another and over time – Many posts relative to a number of other similar clusters of postings
• It is a measure of both output and outtake, but not outcome– Output: how much and how good?– Outtake: how engaged, inter‐connected, inter‐related, volume & velocity of network /
community spread– Outtake: advocacy
d l b f– it does not claim to be a measure of outcome or impact
• It’s a proxy for on‐line influence; it’s about potential to influence– to measure actual influence, we’ve have to survey
• In a diagnostic benchmarking sense the index can help inform an organization’s social media• In a diagnostic, benchmarking sense the index can help inform an organization s social media engagement strategy if tracked over time
• It’s tactical on the fly for mid‐campaign course correction and looking back over the long term
• It’s strategic looking forward: helps shape communications recommendations
• This data can be correlated with tangible outcomes—CARMA Connect.
• This data can be plugged into a market mix model to show causality…isolate for PR’s unique contribution to the marketing mix.
1615 M Street, NW; Suite 750 Washington, DC 20036 USA Tel 1.202.842.1818carma.com 42
@alanchumley / #mra_ac
SO WHAT?WHAT ABOUTWHAT ABOUT CONVERSION & ROI?
1615 M Street, NW; Suite 750 Washington, DC 20036 USA Tel 1.202.842.1818carma.com
WE’RE BACK TO THE PYRAMID THE TOP THIS TIME
1615 M Street, NW; Suite 750 Washington, DC 20036 USA Tel 1.202.842.1818carma.com 44
@alanchumley / #mra_ac
OUTCOMES: KINDS OF CONVERSIONSETTING THE STAGE FOR CORRELATION
• Conversion to a tangible like web traffic, sales, donations, awareness, opinion, usage, brand preference, likelihood to try/buy/recommend.
Micro (steps toward destination) Macro (Destination)
Usually an intermediate means to an end The end
• Could also mean conversion toward any measurable marketing communication or public affairs, issues, advocacy-based objective
Non‐transactional (Engagement) Transactional
Non‐financial Financial
Get to x clicks in a site, spend x time on Sale, donation, objective.
• Here we could / should also be looking at the extent to which the quantity and quality
, psite, download, lead generated, watch a video, created account, contacted us etc.
, ,(paid) subscription etc.
q y q yof social media activity correlates with these outputs.
1615 M Street, NW; Suite 750 Washington, DC 20036 USA Tel 1.202.842.1818carma.com 45
@alanchumley / #mra_ac
2 WEEKS IN THE SOCIAL MEDIA ECOSYSTEM…THEN…YOUR WEBSITE
Where were they? What drove them to your site? What did they do once on your site? What are they ‘worth’ to you?
1615 M Street, NW; Suite 750 Washington, DC 20036 USA Tel 1.202.842.1818carma.com 46
@alanchumley / #mra_ac
SOCIAL MEDIA PERFORMANCE DATA + WEB ANALYTICS
Export Data to Your Web Analytics•#of posts by date•# of posts with key message included•# of posts with key message included
•# of post with brand mentioned X or more times
•# posts X average favorability by date•# posts X average favorability X reach by datep g y y
Or
Integrate web analytics into your social media monitoring platform
1615 M Street, NW; Suite 750 Washington, DC 20036 USA Tel 1.202.842.1818carma.com 47
@alanchumley / #mra_ac
LEVELS OF LINKING OUTPUT TO OUTCOMES
Level of Linkage Notes
Just graph ‘er up Plot # of articles vs. web traffic on a graph
•No indication of relationship other than lines that might trend together
•Could be a fluke
Source: Olivier Blanchard, Social Media ROI
1615 M Street, NW; Suite 750 Washington, DC 20036 USA Tel 1.202.842.1818carma.com 48
@alanchumley / #mra_ac
LEVELS OF LINKING OUTPUT TO OUTCOMES
Level of Linkage Notes
Just graph ‘er up Plot # of articles vs. web traffic on a graph
•No indication of relationship other than lines that might trend together
•Could be a fluke
Correlation •“R score”S t t
•Showing there is l ti hi•Some stats
•You can do this in excel•Pearson’s Product Moment
some relationship beyond coincidence
•Not ‘proving’ media exclusively responsible for sales
Source: Olivier Blanchard, Social Media ROI
1615 M Street, NW; Suite 750 Washington, DC 20036 USA Tel 1.202.842.1818carma.com 49
@alanchumley / #mra_ac
LEVELS OF LINKING OUTPUT TO OUTCOMES
Level of Linkage Notes
Just graph ‘er up Plot # of articles vs. web traffic on a graph
•No indication of relationship other than lines that might trend together
•Could be a fluke
Correlation •“R score”S t t
•Showing there is l ti hi•Some stats
•You can do this in excel•Pearson’s Product Moment
some relationship beyond coincidence
•Not ‘proving’ media exclusively responsible for sales
Source: Olivier Blanchard, Social Media ROI
Causality •Heavy duty stats•You so can’t do this in
•Isolates (statistically
excel•Sophisticated modeling•Market Mix Modeling•Multivariate Regression Analysis
( y‘proves’) for PR’s unique contributionto the communications mix
1615 M Street, NW; Suite 750 Washington, DC 20036 USA Tel 1.202.842.1818carma.com
Regression Analysis•Ton ‘o data, time, $
50
@alanchumley / #mra_ac
TECH INDUSTRY EXAMPLE
Correlating opinion with CARMA IQ (favorability + reach)
1615 M Street, NW; Suite 750 Washington, DC 20036 USA Tel 1.202.842.1818carma.com 51
@alanchumley / #mra_ac
AUTO INDUSTRY EXAMPLE
Monthly Sales Results vs. Four Month CARMA Favorability: Monthly Sales to Four Previous Months’ Favorability
1615 M Street, NW; Suite 750 Washington, DC 20036 USA Tel 1.202.842.1818carma.com 52
@alanchumley / #mra_ac
Toyota’s Heat Map: CARMA Media IQ Metrics by Various Time Frames Correlated to Monthly Sales Resultsy
1615 M Street, NW; Suite 750 Washington, DC 20036 USA Tel 1.202.842.1818carma.com 53
@alanchumley / #mra_ac
UPPERCASE ‘ROI’ vs. lowercase ‘roi’ & Other Return-Offs
ROI roi
MBA PR
Financial Non financialFinancial Non‐financial
Macro Micro (not all conversion events are ROI)
A mathematical equation that works
Usually conceptual only; an equation in words only; can’t actually be calculated; something the industry tosses around too liberallythat works be calculated; something the industry tosses around too liberally
A business metric All‐too‐often a media or audience metric
Direct, clear A loose proxy or an intermediate metric (return on expectation, return on objective return on engagement)return on objective, return on engagement)
1615 M Street, NW; Suite 750 Washington, DC 20036 USA Tel 1.202.842.1818carma.com 54
Source: Olivier Blanchard, Social Media ROI
@alanchumley / #mra_ac
THOSE USING TXT ANALYTICS ARE MEASURING ‘ROI’ NOT ‘roi’
S T t A l ti 2009 U P ti S l ti d P idSource: Text Analytics 2009: User Perspectives on Solutions and Providers
1615 M Street, NW; Suite 750 Washington, DC 20036 USA Tel 1.202.842.1818carma.com
@alanchumley / #mra_ac
SOCIAL MEDIA ROI CALCULATOR(S) THE GOOD & THE BAD
1615 M Street, NW; Suite 750 Washington, DC 20036 USA Tel 1.202.842.1818carma.com
@alanchumley / #mra_ac
SOCIAL MEDIA BOOK BAG
• Social Media Metrics: • Jim Sterne, (@jimsterne)
• Social Media ROI: • Olivier Blanchard (@thebrandbuilder)
• Social Media Analytics: Effective Tools for Building, Interpreting & Using Metrics: • Marshal Sponder (@webmetricsguru)
M Wh M• Measure What Matters: • Katie Paine (@kdpaine)
• Using Web Analytics to Measure Impact of Earned Online Media…:g y p• Seth Duncan (www.instituteforpr.org)
1615 M Street, NW; Suite 750 Washington, DC 20036 USA Tel 1.202.842.1818carma.com 57
@alanchumley / #mra_ac
PART 1 (1.5 hrs): SOCIAL MEDIA MEASUREMENT
IN-ROOM STRETCH (5 mins)
PART 2 (30 mins): TONE ALONE
PART 3: (35 mins): REAL DATA WITH MALCOM DE LEO NETBASE
1615 M Street, NW; Suite 750 Washington, DC 20036 USA Tel 1.202.842.1818carma.com
AGENDA FOR PART 2 DEEP DIVE ON SENTIMENT
•Sentiment Analysis
•Why is it important/ why should we care?
•Sentiment as a Proxy / Predictor
•Applications
•Humans or Machines or Both?
1615 M Street, NW; Suite 750 Washington, DC 20036 USA Tel 1.202.842.1818carma.com 59
@alanchumley / #mra_ac
AGENDA FOR PART 2 DEEP DIVE ON SENTIMENT
•Sentiment Analysis
•Why is it important/ why should we care?
•Sentiment as a Proxy / Predictor
•Applications
•Humans or Machines or Both?•Why Automation?
S S•Situating Automated Sentiment Analysis
•Defining Automated Sentiment Analysis
•Output
•Taxonomies etc.
•Automated Vendors
•Complicationsp
1615 M Street, NW; Suite 750 Washington, DC 20036 USA Tel 1.202.842.1818carma.com 60
@alanchumley / #mra_ac
AGENDA FOR PART 2 DEEP DIVE ON SENTIMENT
•Sentiment Analysis
•Why is it important/ why should we care?
•Sentiment as a Proxy / Predictor
•Applications
•Humans or Machines or Both?•Why Automation?
S S
•Human vs Automated
•Situating Automated Sentiment Analysis
•Defining Automated Sentiment Analysis
•Output•Human vs. Automated
•Human-based vendors
•Human-Based Sentiment Analysis
•Taxonomies etc.
•Automated Vendors
•Complications•Favorability not (just) Sentiment
•Humans supplemented by automation
p
1615 M Street, NW; Suite 750 Washington, DC 20036 USA Tel 1.202.842.1818carma.com 61
@alanchumley / #mra_ac
Linking outputs to outcomes revisited
I AM NOT IAM
A COMPUTATIONAL LINGUISTA SEMANTICIANA SEMANTICIANCOMPUTER ENGINEERCOMPUTER SCIENTISTLOGICIANPSYCHOLINGUIST
BUT CHECK OUT SETH GRIMES’:BUT…CHECK OUT SETH GRIMES :Text Analytics 2009: User Perspectives on Solutions and ProvidersA number of excellent presentations on LinkedIn / Slide Share
I’ve borrowed liberally from both for this Part 2
1615 M Street, NW; Suite 750 Washington, DC 20036 USA Tel 1.202.842.1818carma.com 62
@alanchumley / #mra_ac
I AM NOT IAM
A COMPUTATIONAL LINGUISTA SEMANTICIAN A RECOVERING PR FLACKA SEMANTICIANCOMPUTER ENGINEERCOMPUTER SCIENTISTLOGICIAN
CO G CPR MEASUREMENT MOUTHPIECEWITH AN INTEREST IN MONITORING + MINING ONLINE CONVERSATIONS
PSYCHOLINGUIST
BUT CHECK OUT SETH GRIMES’:BUT…CHECK OUT SETH GRIMES : Sentiment through the eyes of a simpletonText Analytics 2009: User Perspectives on Solutions and Providers
A number of excellent presentations on LinkedIn / Slide Share
I’ve borrowed liberally from both for this Part 2
1615 M Street, NW; Suite 750 Washington, DC 20036 USA Tel 1.202.842.1818carma.com 63
@alanchumley / #mra_ac
IMAGINE A DIFF’T. PYRAMID FOR DIFF’T. CAMPAIGN TYPES / OBJECTIVES
1615 M Street, NW; Suite 750 Washington, DC 20036 USA Tel 1.202.842.1818carma.com 64
@alanchumley / #mra_ac
WHY IS IT IMPORTANT? WHY SHOULD WE CARE?
Questions for biz & gov’t.:
Wh t l i ?•What are people saying?•What’s trending?•How has opinion about x changed?•Does it differ channel to channeloes d e c a e o c a e•How has sentiment correlated with news / marketing / events etc.?•Can we link opinions to transactions?
Computers are trading stock based on automated sentiment analysis of online content
1615 M Street, NW; Suite 750 Washington, DC 20036 USA Tel 1.202.842.1818carma.com 65
@alanchumley / #mra_ac
SENTIMENT ANALYSIS AS MARKET RESEARCH PROXY / PREDICTOR
Think here about social media monitoring and analysis as:
•Real time situational awareness•Listening at the point of need•Listening close to the moment of influence
A real-time digital form of or proxy for market research and CRM:
Mining conversations to tease out how people ‘feel’ about products/services/issues/causesfeel about products/services/issues/causes and perhaps even:
1) their intended (reported) behavior and what motivates them to buy, (as cited in the
ti ) dconversations), and
2) their actual behavior tracked via CRM 2.0 and Web 2.0 analytics
1615 M Street, NW; Suite 750 Washington, DC 20036 USA Tel 1.202.842.1818carma.com 66
Source: Crimson Hexagon
@alanchumley / #mra_ac
APPLICATIONS
1615 M Street, NW; Suite 750 Washington, DC 20036 USA Tel 1.202.842.1818carma.com 67
Call Centre Notes?
@alanchumley / #mra_ac
SITUATING SENTIMENT ANALYSIS AUTOMATED
Text analytics:
•Applies linguistic +/or statistical
Artificial Intelligence
T t A l ti
techniques to extract concepts and
patterns
•Transforms unstructured information intoText Analytics •Transforms unstructured information into
data for analysis
•Unlocks meaning and relationships in
Sentiment Analysis large volumes of information
•Adds semantic understanding of:•SentimentOpinions•Opinions
•Attitudes •Entities•ConceptsFacts
1615 M Street, NW; Suite 750 Washington, DC 20036 USA Tel 1.202.842.1818carma.com 68
@alanchumley / #mra_ac
•Facts•Abstract attributes: “expensive,” “comfortable”
AUTOMATED SENTIMENT ANALYSIS DEFINED
Sentiment analysis or opinion mining refers to the application of:•Natural language processing•Computation linguistics Comp ScienceLinguisticsp g•Text analytics…
…to identify and extract subjective information in content. Statistics
Natural Language Processing• Grounded in statistical machine learning• Much overlap with…
Computational LinguisticsComputational Linguistics• Statistical or rule‐based modeling of language
Text Analytics• Turning text into data• Lexical analysis: word frequencies• Entity recognition• Pattern recognition• Tagging/ annotation
1615 M Street, NW; Suite 750 Washington, DC 20036 USA Tel 1.202.842.1818carma.com
Tagging/ annotation
@alanchumley / #mra_ac
EXTRACTING ATTITUDES TURNING ATTITUDES INTO DATA
Generally speaking, sentiment analysis aims to determine the attitude of a speaker or a writer with respect to some topic or the overall tonality of a document.
The attitude may be his or her:
• judgment or evaluation
• affective state (emotional state of the author when writing), or
• the intended emotional communication (emotional effect the author wishes to have on the reader).
1615 M Street, NW; Suite 750 Washington, DC 20036 USA Tel 1.202.842.1818carma.com 70
@alanchumley / #mra_ac
HUMANS OR MACHINES OR BOTH?ALGORITHM AL GORITHM
Can it be over ridden?Can it be over ridden?Does it learn?
1615 M Street, NW; Suite 750 Washington, DC 20036 USA Tel 1.202.842.1818carma.com 71
@alanchumley / #mra_ac
THE ALGORITHMICS
WHY AUTOMATION?
Time Spent Analyzing Contenty g
w/out automation
Ti STime Spent on Insights &
Implications
w/ automation
1615 M Street, NW; Suite 750 Washington, DC 20036 USA Tel 1.202.842.1818carma.com 72
@alanchumley / #mra_ac
WHY AUTOMATION?
Time Spent Analyzing Contenty g
w/out automation
Ti STime Spent on Insights &
Implications
w/ automation
1615 M Street, NW; Suite 750 Washington, DC 20036 USA Tel 1.202.842.1818carma.com 73
@alanchumley / #mra_ac
THE OUTPUT
E tit E t tiEntity Extraction
Issues / Themes / MessagesMessages
1615 M Street, NW; Suite 750 Washington, DC 20036 USA Tel 1.202.842.1818carma.com 74
@alanchumley / #mra_ac
TAXONOMIES…LEXICONS…DICTIONARIES…BAGS-O-WORDS…CARTRIDGES
PRE‐SET • Words/phrases that are generally good or
GENERALare generally good or bad (Hate. Love)
• Caution!
1615 M Street, NW; Suite 750 Washington, DC 20036 USA Tel 1.202.842.1818carma.com 75
@alanchumley / #mra_ac
TAXONOMIES…LEXICONS…DICTIONARIES…BAGS-O-WORDS…CARTRIDGES
PRE‐SET•Words/phrases that are generally good or badPRE SET
GENERAL
PRE‐SET
generally good or bad (Hate. Love)
•Caution!
•Financial / market newsPRE SET
SPECIFIC•Mergers & Acquisitions•Competition•Better
1615 M Street, NW; Suite 750 Washington, DC 20036 USA Tel 1.202.842.1818carma.com 76
@alanchumley / #mra_ac
TAXONOMIES…LEXICONS…DICTIONARIES…BAGS-O-WORDS…CARTRIDGES
•Words/phrases that are generally good or bad
PRE‐SET GENERAL
PRE SET
generally good or bad (Hate. Love)
•Caution!
•Financial / market newsPRE‐SET
SPECIFIC
CUSTOM
•Mergers & Acquisitions•Competition•Better
CUSTOM
CLIENT/PROJECT
SPECIFIC
•Client‐specific context•Best
1615 M Street, NW; Suite 750 Washington, DC 20036 USA Tel 1.202.842.1818carma.com 77
@alanchumley / #mra_ac
TAXONOMIES…LEXICONS…DICTIONARIES…BAGS-O-WORDS…CARTRIDGES
•Words/phrases that are generally good or bad
PRE‐SET GENERAL
PRE SET
generally good or bad (Hate. Love)
•Caution!
•Financial / market newsPRE‐SET
SPECIFIC
CUSTOM
•Mergers & Acquisitions•Competition•Better
CUSTOM
CLIENT/PROJECT
SPECIFIC
•Client‐specific context•Best•Multiple cartridges
But! Still needs some human intervention / validation /
1615 M Street, NW; Suite 750 Washington, DC 20036 USA Tel 1.202.842.1818carma.com 78
feedback so it can ‘learn’ and improve
@alanchumley / #mra_ac
AUTOMATED TEXT ANALYTICS VENDORS
1615 M Street, NW; Suite 750 Washington, DC 20036 USA Tel 1.202.842.1818carma.com 79
@alanchumley / #mra_ac
1st GENERATION vs. 2nd GENERATION AUTOMATED SENTIMENT
Key Word (only) Conceptual Search-Based
Deeper linguistic level Based
Explicit References
p g
Operates on a semantic level
Understands sentence grammar:References
Counting up the “+” & “-”
•Subject•Predicate•Object•Who?•Action•Idea
•…and the relationships between these•…must contain these terms but in a particular relationship to one another
Most studies show that key word matching is wrong relationship to one another
Opinions, claims, arguments, facts, actions, concepts, messages
N i li it th / i it / i t t
word matching is wrong more than 50% of the time
1615 M Street, NW; Suite 750 Washington, DC 20036 USA Tel 1.202.842.1818carma.com 80
Non-implicit: theme / spirit / intent
@alanchumley / #mra_ac
COMPLICATIONS & PROBLEMS w/ 1ST GEN AUTOMATED APPROACHES
•Sentiment may be of interest at multiple levels:•Document•Several DocumentsStatement / sentence•Statement / sentence
•Entity•Topic or Theme•Concept
•Language is chaotic:•Language is chaotic:•Jargon, slang, (slanguage) •Colloquialism•Forum-specific acronyms•ambiguity, •Anaphora (it, them)Anaphora (it, them)•Irony, sarcasm•Misspellings•Poor grammar•Variations in expression•Synonyms•Polysemy
•Context is key:•Meaning can change depending on context. •Need to look not only at the word but the connective tissues surrounding the word.
1615 M Street, NW; Suite 750 Washington, DC 20036 USA Tel 1.202.842.1818carma.com 81
@alanchumley / #mra_ac
CONTEXT IS KEY
1615 M Street, NW; Suite 750 Washington, DC 20036 USA Tel 1.202.842.1818carma.com 82
@alanchumley / #mra_ac
ANALYSING THE “CONNECTIVE TISSUE” OF A SENTENCE
1615 M Street, NW; Suite 750 Washington, DC 20036 USA Tel 1.202.842.1818carma.com 83
@alanchumley / #mra_ac
HUMAN VS. AUTOMATION BOTH IS BEST
•Most sentiment analysis algorithms rely on
humans using simple terms to express our sentiment AUTOMATION
HUMANS
FOR
about a product or service. We don’t.
C lt l f t li i ti d diff i
FOR VOLUME & SPEED
RELEVANCE, DEPTH, CONTEXT, NUANCE
•Cultural factors, linguistic nuances and differing
contexts make it extremely difficult to automate turning
a string of written text into a simple pro or con Bridge the Gap with Human analysisg p p
sentiment. Human analysis of a sample
•The fact that humans often disagree on the sentiment
of text illustrates how big a task it is for computers to
get this right1615 M Street, NW; Suite 750 Washington, DC 20036 USA Tel 1.202.842.1818carma.com 84
get this right.
@alanchumley / #mra_ac
HUMAN (or hybrid)-BASED VENDORS
1615 M Street, NW; Suite 750 Washington, DC 20036 USA Tel 1.202.842.1818carma.com 85
@alanchumley / #mra_ac
HUMAN-BASED ANALYSIS
Variety of scales out there both in terms of function and form:
Some sound, some silly:
•Just + or -
•Positive = 5; Negative = 0 (should a negative be negative?)
•-5 to +5
•Likert scale-ish: completely / somewhat / neither…
•Green red•Green red
•Sun, clouds, rain
BUT!: TONE ALONE AIN’T ALL THAT
1615 M Street, NW; Suite 750 Washington, DC 20036 USA Tel 1.202.842.1818carma.com 86
BUT!: TONE ALONE AIN T ALL THAT
@alanchumley / #mra_ac
DOES SENTIMENT = FAVORABILITY = INFLUENCE POTENTIAL?
Do you think the sentiment and
how favorably your brand is portrayedhow favorably your brand is portrayed
or how
potentially influential a blog / blogger is
are the same thing?
1615 M Street, NW; Suite 750 Washington, DC 20036 USA Tel 1.202.842.1818carma.com 87
@alanchumley / #mra_ac
FAVORABILITY; NOT (just) SENTIMENT OR TONE
Examining & Scoring factors that speak to the quality of the story + how favorable
the story was in its entirety & context.
NOT:
•Just tone / sentiment
•Accuracy, slant, fairness
•How it might or would likely affect the reader
•Researcher’s personal opinionResearcher s personal opinion
1615 M Street, NW; Suite 750 Washington, DC 20036 USA Tel 1.202.842.1818carma.com 88
@alanchumley / #mra_ac
FAVORABILITY; NOT (just) SENTIMENT OR TONEWHO (else) IS TALKING ABOUT YOU? WHAT ARE THEY SAYING? IMPACT?
We quantify and compare the quality of how favorably eBay and its competitors are portrayed
Entities to be tracked•eBay•PayPalBillM L t•BillMeLater
•Gitti gidiyo•eBay foundation•Skype•Stubhub•CraigslistCraigslist•Etc.
Article Focus•Corporate•Auction•Customer•Product / service•Management•Financial general•Stock price (exclude given
l ?)
1615 M Street, NW; Suite 750 Washington, DC 20036 USA Tel 1.202.842.1818carma.com 89
volume?) •Industry
@alanchumley / #mra_ac
FAVORABILITY; NOT (just) SENTIMENT OR TONEWHO (else) IS TALKING ABOUT YOU? WHAT ARE THEY SAYING? IMPACT?
What stakeholder commentary is appearing in eBay’s coverage? What impact is it having on eBay’s and their competitors’
We quantify and compare the quality of how favorably eBay and its competitors are portrayed
Entities to be tracked•eBay•PayPalBillM L t
Sources•Company official•Customer•Customer group
and their competitors portrayals? Is it the usual list of stakeholder suspects or are others emerging over time?
•BillMeLater•Gitti gidiyo•eBay foundation•Skype•Stubhub•Craigslist
Customer group•Analyst•Partners•Government•Competitor•Etc.Craigslist
•Etc.
Article Focus•Corporate•Auction
Key Topics•Privacy security•Company future•Industry future•Partnerships / alliances•Customer
•Product / service•Management•Financial general•Stock price (exclude given
l ?)
•Partnerships / alliances•Mergers / acquisitions•Corporate strategy / vision•Management•Financial performance general•Stock performance
1615 M Street, NW; Suite 750 Washington, DC 20036 USA Tel 1.202.842.1818carma.com 90
volume?) •Industry
Stock performance•Market share
@alanchumley / #mra_ac
FAVORABILITY; NOT (just) SENTIMENT OR TONEWHO (else) IS TALKING ABOUT YOU? WHAT ARE THEY SAYING? IMPACT?
What stakeholder commentary is appearing in eBay’s coverage? What impact is it having on eBay’s and their competitors’
We quantify and compare the quality of how favorably eBay and its competitors are portrayed
Entities to be tracked•eBay•PayPalBillM L t
Sources•Company official•Customer•Customer group
What messages are appearing? Are there other themes/ideas not on this list that are appearing
and their competitors portrayals? Is it the usual list of stakeholder suspects or are others emerging over time?
•BillMeLater•Gitti gidiyo•eBay foundation•Skype•Stubhub•Craigslist
Customer group•Analyst•Partners•Government•Competitor•Etc.
+/- Messages•Industry leader•Financially healthy•A good investment•Has good strategy / strong vision for the future
in the coverage? What impact are they having on ADP’s and your competitor’s portrayals?
Craigslist•Etc.
Article Focus•Corporate•Auction
Key Topics•Privacy security•Company future•Industry future•Partnerships / alliances
•Has good strategy / strong vision for the future•Well-managed•Growing•Well positioned for competition / outperforming competitors•Excellent customer service (customer endorsement)
•Customer•Product / service•Management•Financial general•Stock price (exclude given
l ?)
•Partnerships / alliances•Mergers / acquisitions•Corporate strategy / vision•Management•Financial performance general•Stock performance
Excellent customer service (customer endorsement)•Customer-focused•Delivers solutions that solve customer needs•Delivers high quality products, services•Delivers high value products, services•Delivers technically sound products, services
1615 M Street, NW; Suite 750 Washington, DC 20036 USA Tel 1.202.842.1818carma.com 91
volume?) •Industry
Stock performance•Market share •Stands behind its products, services
•Products/services are priced well or fairly•Etc.
@alanchumley / #mra_ac
BUT EVEN FAVORABILITY IS ONLY PART OF THE INFLUENCE PICTURE
Examining & Scoring factors that speak to the
quality of the story + how favorable the story was in its entirety & context.
NOT:
•Just tone / sentiment
•Accuracy, slant, fairness
•How it might or would likely affect the reader
•Researcher’s personal opinion
1615 M Street, NW; Suite 750 Washington, DC 20036 USA Tel 1.202.842.1818carma.com 92
@alanchumley / #mra_ac
AN INDEX WITH MOVABLE INDICATORSFLEXIBLE ENOUGH TO ACCOUNT FOR DIFFERENT OBJECTIVES
Indicator Popularity Relevance Authority Brand Engagement Advocacy Cascade SentimentPresence
Score 10 10 10 10 10 10 10 10Weight 1 2 3 4 5 6 7
(@ d f lt)(@ default)
Popularity + Relevance + Authority + Presence + Engagement + Advocacy + Cascade X Sentiment = Influence Potential
• Each of the above indicators can be adjusted to account for different types of campaigns & contexts
• The weights would change. The scores don’t.
• For an organization, new to social media, perhaps they might only want to establish an on‐line presence
• So we’d dial or slide the weights of all but the popularity indicator down to zero
Has a significantly scaled back / rudimentary version of this. This is inspired by but significantly expands on R6 thinking.
1615 M Street, NW; Suite 750 Washington, DC 20036 USA Tel 1.202.842.1818carma.com 93
p y g y p g
@alanchumley / #mra_ac
LEVELS OF LINKING OUTPUT TO OUTCOMES
Level of Linkage Notes
Just graph ‘er up Plot # of articles vs. web traffic on a graph
•No indication of relationship other than lines that might trend together
•Could be a fluke
Correlation •“R score”S t t
•Showing there is l ti hi•Some stats
•You can do this in excel•Pearson’s Product Moment
some relationship beyond coincidence
•Not ‘proving’ media exclusively responsible for sales
Source: Olivier Blanchard, Social Media ROI
Causality •Heavy duty stats•You so can’t do this in
•Isolates (statistically
excel•Sophisticated modeling•Market Mix Modeling•Multivariate Regression Analysis
( y‘proves’) for PR’s unique contributionto the communications mix
1615 M Street, NW; Suite 750 Washington, DC 20036 USA Tel 1.202.842.1818carma.com
Regression Analysis•Ton ‘o data, time, $
94
@alanchumley / #mra_ac
UPPERCASE ‘ROI’ vs. lowercase ‘roi’ & Other Return-Offs
ROI roi
MBA PR
Financial Non financialFinancial Non‐financial
Macro Micro (not all conversion events are ROI)
A mathematical equation that works
Usually conceptual only; an equation in words only; can’t actually be calculated; something the industry tosses around too liberallythat works be calculated; something the industry tosses around too liberally
A business metric All‐too‐often a media or audience metric
Direct, clear A loose proxy or an intermediate metric (return on expectation, return on objective return on engagement)return on objective, return on engagement)
1615 M Street, NW; Suite 750 Washington, DC 20036 USA Tel 1.202.842.1818carma.com 95
Source: Olivier Blanchard, Social Media ROI
@alanchumley / #mra_ac
PART 1 (1.5 hrs): SOCIAL MEDIA MEASUREMENT
IN-ROOM STRETCH (5 mins)
PART 2 (30 mins): TONE ALONE
PART 3: (35 mins): REAL DATA WITH MALCOM DE LEO
1615 M Street, NW; Suite 750 Washington, DC 20036 USA Tel 1.202.842.1818carma.com
Malcolm De Leo Chief EvangelistChief EvangelistNetBase
June 6, 2011
| Confidential | © 2011 NetBase Solutions. All Rights Reserved Worldwide.979
Consumer Insight Methods and Instruments
High In‐store observations
NetnographyIn‐depth interviews
Ethnography
of In
sight
g p y
Test Marketing
Dep
th o Test Marketing
Surveys (online)
Low
Focus Groups
Surveys (intercept)Surveys (phone)
| Confidential | © 2011 NetBase Solutions. All Rights Reserved Worldwide.98
Broad SamplingLow High
ow
| Confidential | © 2011 NetBase Solutions. All Rights Reserved Worldwide.99
A Love Story…
++ =Randy LoveProduct Marketing ManagerComcast Cable Communications
| Confidential | © 2011 NetBase Solutions. All Rights Reserved Worldwide.
100
Communications
The NetBase Difference – Language Understanding
Natural Language Processing
PositiveAttribute
BrandReason
I know Listerine kills germs because it hurts.
I know Listerine kills germs because it hurts.
Text Analytics
Keyword Search
Brand NegativeWord
NegativeWord
NegativeWord
I know Listerine kills germs because it hurts.
y
Keyword Keyword Keyword Keyword Keyword Keyword
| Confidential | © 2011 NetBase Solutions. All Rights Reserved Worldwide.
101
A Short Netnography Case StudyCase Study
| Confidential | © 2011 NetBase Solutions. All Rights Reserved Worldwide.
1021
Burn Burn
| Confidential | © 2011 NetBase Solutions. All Rights Reserved Worldwide.
103
Verbatim
h i h I love Listerine coz it burns.
I hate it when Listerine burns your freaking mouth off!g
I love the burn of Listerine at 530 in
I really hate listerine*it [listerine*] makes Listerine at 530 in
the morning.it [listerine*] makes my tounge burn!
Man I love listerinebut boy does it [listerine] burn!!!
i hate when the listerine starts to burn my cheeks.
| Confidential | © 2011 NetBase Solutions. All Rights Reserved Worldwide.
104
[ ]y
| Confidential | © 2011 NetBase Solutions. All Rights Reserved Worldwide.
105
Overview
This study is an analysis of consumers’ perception of Listerine.
• Source data: Social Media ‐ content on the Internet generated by consumers, e.g. blogs.
• Quantitative analysis: Amount of discussion about Listerine relative to other topics.
Consumer sentiment & passion about Listerine relative to other topics.
Drivers of consumer behavior in the mouthwash category.
• Qualitative analysis: Overall reasons consumers like or dislike Listerine.
K i i ht b t Li t i
| Confidential | © 2011 NetBase Solutions. All Rights Reserved Worldwide.
106
Key insights about Listerine consumers.
Summary of Findings
• Consumers most like Listerine’s germ‐killing action.
• The most prevalent complaint from consumers is that Listerine is too harsh.The most prevalent complaint from consumers is that Listerine is too harsh.
• Overall reasons given for liking and disliking Listerine are as follows (in rank order):
Likes Dislikes
Kills germs Too harsh
Treats toenail fungus Toxicity
Good mosquito repellent Efficacy
Treats bad breath Taste
Treats acne Link to alcoholism
• Listerine has a tremendous number of off‐label uses for personal and home care based on its efficacy.
| Confidential | © 2011 NetBase Solutions. All Rights Reserved Worldwide.
107
Research Approach
• Use freely available consumer generated discussions (“Social Media”) to understand:( Social Media ) to understand:
Overall positive/negative preference for Listerine.
Opportunities for leveraging likes and compensating for orOpportunities for leveraging likes and compensating for or overcoming dislikes.
I ti ti t li d Li t i b t l i l d d• Investigations centralized on Listerine but also included some related terms:
Specific flavors of ListerineSpecific flavors of Listerine
Related Listerine products, e.g. Listerine Strip
| Confidential | © 2011 NetBase Solutions. All Rights Reserved Worldwide.
108
Social Media Sources for Consumer Insights
Internet Forums Online communities where consumers post questions answers
Blogs Commentary or news on a particular subject; other blogs function as
Microblogs Communication platforms where users post short status messagesconsumers post questions, answers,
comments, and general discussions.
Forums are often associated with particular topics such as hobbies, geographies, sports, lifestyles, illnesses and more
subject; other blogs function as personal online diaries.
Some blogs allow readers to leave comments, thus fostering discussions like those found in online forums.
users post short status messages.
The most popular microblog currently is Twitter.
illnesses, and more.
| Confidential | © 2011 NetBase Solutions. All Rights Reserved Worldwide.
109
How many “hits” are there for Listerine?
The term Listerine is mentioned infrequently on social media.
• Netflix is a good comparison since it is mentioned so much on the web.
• Listerine is mentioned about as much as the major airlines.
| Confidential | © 2011 NetBase Solutions. All Rights Reserved Worldwide.
110
Mouthwashes Buzz
| Confidential | © 2011 NetBase Solutions. All Rights Reserved Worldwide.
111
What is the overall sentiment for Listerine?
• As compared to other brands, consumer sentiment for Listerine is moderate.
• Photoshop and Comcast are reliable control variables for positive and p pnegative extremes.
| Confidential | © 2011 NetBase Solutions. All Rights Reserved Worldwide.
112
Sentiment by Brand
| Confidential | © 2011 NetBase Solutions. All Rights Reserved Worldwide.
113
Passion Index
| Confidential | © 2011 NetBase Solutions. All Rights Reserved Worldwide.
114
Positive Themes
| Confidential | © 2011 NetBase Solutions. All Rights Reserved Worldwide.
115
Negative Themes
| Confidential | © 2011 NetBase Solutions. All Rights Reserved Worldwide.
116
Recommend Behavior
| Confidential | © 2011 NetBase Solutions. All Rights Reserved Worldwide.
117
Attributes – Net Scores
| Confidential | © 2011 NetBase Solutions. All Rights Reserved Worldwide.
118
Main Opportunities & Threats
The main positives and negatives were analyzed for opportunities and threatsopportunities and threats.
Potential actions are proposed to address these opportunities and threatsthreats.
| Confidential | © 2011 NetBase Solutions. All Rights Reserved Worldwide.
119
Opportunity: Toenail Fungus Treatment
Potential InnovationInsight
Determine effectiveness of Listerine for toe nail fungus treatment.
Introduce a toe nail fungus treatment “from the makers of Listerine”
The second most mentioned benefit of Listerine is that it can be used to treat toenail fungus.
This theme was 9% of positive comments the makers of Listerine”. This theme was 9% of positive comments.
| Confidential | © 2011 NetBase Solutions. All Rights Reserved Worldwide.
120
Opportunity: Mosquito Repellent
Potential InnovationInsight
Determine effectiveness of Listerine as a mosquito repellent.
Publish a survival kit with Listerine listed as not only a mouth wash but also a
A commonly mentioned benefit of Listerine is that it can be used to repel mosquitoes.
This theme was 6% of positive comments.as not only a mouth wash but also a mosquito repellent.
| Confidential | © 2011 NetBase Solutions. All Rights Reserved Worldwide.
121
Opportunity: Acne Treatment
Potential InnovationInsight
Determine effectiveness of Listerine for acne treatment.
Introduce an acne treatment “from the makers of Listerine”
Many consumers talk about using Listerine to treat their acne.
A few disputed the efficacy of this practice.makers of Listerine”.
This theme was 4% of positive comments.
| Confidential | © 2011 NetBase Solutions. All Rights Reserved Worldwide.
122
Opportunity: Listerine Reminds Me of Grandpa
Potential InnovationInsight
Include a grandfather figure in advertising creative.
Test whether emphasizing the masculinity of Listerine (e g its strong flavor) makes it
A few consumers talked about their emotional connection to Listerine.
Listerine tends to remind some people of their grandfathers of Listerine (e.g. its strong flavor) makes it
more appealing to some consumers.grandfathers.
| Confidential | © 2011 NetBase Solutions. All Rights Reserved Worldwide.
123
Opportunity: Effectiveness of Twice a Day campaign
Insight Listerine and Reach created the “October Oral Care Challenge” for kids 6 and older To complete thekids 6 and older. To complete the challenge, our family had to brush, floss and rinse every day, twice a dayfor two weeks. We currently use, and have used both Listerine and Reach products for years, so this was an easy challenge for our family.
Listerine’s recent ad campaign encourages people to use Listerine twice per day.
Many consumers have not only
Potential Actions
begun to do so, but they also have begun to talk about it.
Lately, some others have been added: gargle twice a day with warm salt water, or Listerine; drink hot liquids,
h t d hi k t d
I brush my teeth 2+ times a day every day, and I use Listerine.
Begin to measure effectiveness of campaigns by looking for the messaging among consumers talking to each other.
such as tea and chicken soup, to ward off or diminish virus proliferation in throat.
I would start using Listerine 2 to 3 times a day.
g
Chuck Norris doesn't brush his teeth; he simply pours Listerine into his vodka twice a day. Also brushing at least twice a day and
using Antiseptic mouthwash like Listerine twice a day AM & PM.
| Confidential | © 2011 NetBase Solutions. All Rights Reserved Worldwide.
124
y
What are the key takeaways/actions
•Why is sentiment important? It’s the elephant in the room
The suggestion box is public The suggestion box is public
Your market research doesn’t wear clothes anymoreanymore…
| Confidential | © 2011 NetBase Solutions. All Rights Reserved Worldwide.
125