Measuring Social Media Prsa Digital Impact

7 Cs of MEASURING SOCIAL MEDIA

PRSA DIGITAL IMPACT CONFERENCE

MAY 6, 2011

1615 M Street, NW; Suite 750 Washington, DC 20036 USA Tel 1.202.842.1818carma.com

Alan Chumley

Senior Vice President

@alanchumley / @CARMA_Tweets


ALAN CHUMLEY, SVP

• 14 years in the industry• Recovering PR practitioner: ING, Bell Canada • Former Director of Measurement @ Hill & Knowlton• Masters in Communications & Culture:

• Research foci: media effects & uses, reception analysis, audience studies

• University-level adjunct instructor of PR measurement• LinkedIn early 2006• Blogging / Tweeting since early 2007

2


ISN’T IS

• An emerging / experimental philosophy / approach / filter / lens through which to look at measuring social media

• A high level connected / systems way of approaching the problem

• About communities not an audience

• More sociologist than publicist

• Not a Case Study

• Not a mention of Old Spice in sight

• Not a specific ‘how to’ / no one-size-fits all

•Not focused on widgets or mobile

• Doesn’t focus on a singular or few social media platforms—looks at all holistically



MEASUREMENT ACCOUNTS FOR 5 / 7 TOP SOCIAL MEDIA CHALLENGES

Source: Harvard Business Review: the New Conversation: Taking Social Media from Talk to Action

Surveyed 2100 Harvard Biz Review subscribers

@alanchumley / @CARMA_Tweets4

Why is this the case?Gap between leap of faith true believers and C-suite’s need for metrics to validate the time/$?


LOT’S OF TOOLS OUT THERE

BEWARE OF THE SILOs AVOID USING (ONLY) THE NEW SHINEY FREEBIES


Would you like your job performance (read: raise/bonus) to be based only on a Klout Score, for example?

•Isolated / silo’d approach•No one tool on it’s own is sufficient—not even one paid tool•Have to connect the dots with a more relational, systems approach

Don’t tout or shout out about Klout:A blunt counting instrumentIgnores context / topical relevance

1615 M Street, NW; Suite 750 Washington, DC 20036 USA Tel 1.202.842.1818carma.com 6

•It’s not coverage

•It’s not static

•There isn’t an ‘audience’ of consumers

•Prosumers not just consumers: •simultaneously receiving, interpreting, co-opting, re-appropriating and creating content

•It’s more many-to-many dialogue than one-to-many monologue

•So, traditional off line ways of measuring ‘coverage’ don’t cut it

•They don’t adequately account for the multi-dimensionality and dynamic nature of a conversation and a community of interest in a network

•We need to think bigger

REMEMBER: IT’S SOCIAL MEDIA



THINK BIGGER & BROADER MOVE BEYOND ‘MOMENT-IN-TIME’ or ‘VANITY’ METRICS

7


Systems or Networked Thinking:

Interdependent relationships between those in a network

Social Capital:

Connections within and between those in a social network as a source of civic engagement among a community leading to cohesion; that has ‘value’

Michael Kelly


THINK BIGGER / INTEGRATE / CORRELATE

•Think Bigger / Broader Beyond PR’s Historical Silo –social Media’s Pushing us all that Way •Think about all customer-facing touch points•Understand PR / Marketing / Business Metrics •Integrate Cross-Discipline•Report on the Critical Few•Focus on /correlate with Outcomes•Close the loop / connect the dots

Source: Weber-Shandwick Measurement & Strategy Practice

•It’s not a ‘keep-your-job’ or ‘buy-a-pool-with-your-bonus’ report card•A data-driven approach / evidence-based communications

Source: Oliver Blanchard, Social Media ROI

8


Michael Kelly


The 7Cs of SOCIAL MEDIA MEASUREMENT THINKING MORE LIKE A SOCIOLOGIST THAN A PUBLICISTCONVERSATIONS & COMMUNITIES VS. COVERAGE

Counting What some call search, syndication and site; stuff we can count like traffic, popularity, fans, followers, retweets, etc.

Content Analysis that is. Using content analysis, ideally a hybrid of some automation for volume and speed and human analysts for context, nuance and depth to look for how your brand, messaging, topics, issues, stakeholder voices are portrayed; the inclusiveness, accuracy and tone of the conversations.

Conversations And conversationships recognizing that social media is not strictly another communications channel through which to get word out or reach and influence. It is more about genuinely engaged dialogue than a simple corporate monologue; not only a vertical, one-to-many conversation between an organization and stakeholders but a lateral, many-to-many conversation among stakeholders.

Cohesion / Capital

The extent to which folks are agreeing with you, your position, your brand essences. The extent to which they are agreeing with each other and coalescing around and advocating a core theme, idea, or some call to action either in support or opposition to your organization’s position.

Community The extent to which a core group of people with common interests are gathering and growing. Measuring the change in community size, and volume and nature of chatter over time

Connectedness A look at how inter-connected the highly engaged/key influencers/advocates are and how centrally located they are in both the network and dialogue.

Conversion Stay tuned. More to follow.

9



THE 6Ps OF INFLUENCEMORE THAN JUST REACH / FREQUENCY RELEVANCY

Popular Visible, vocal, has a substantial following, reach. In-bound links, trackbacks, subscribers, bookmarks, followers, friends, views, listens, saves, downloads, etc.

Polarized in tone Neutrality does little to drive influence way or the other. A clearly positive or negative view will polarize readers/followers and is more likely to drive cohesion and mobilize advocates and have those advocates coalesce around a core theme, idea, or call to action.

Prolific / Relevant / Frequent

Raw author contribution and # of on-topic, related posts

Prominent / Authoritative

Are they an idea starter or spreader; source or spider? They may be prolific but are they prominent? Are they highly inter-related, inter-connected, and centrally located in the network? How engaged is this person’s following in a dialogue? How much dialogue is there and what is its nature? Here we need to recognize, though, that authority is contextual and topical. One might be an authority on PR measurement but not on 18th century Russian literature.

Promoter / Loyal Advocate

How many of the followers/commentators active contributors advocating, endorsing, advancing (or the opposite) your position? Are they adding links, tags. Is the nature of the language they are using inter-connective, expanded, clarifying, reinterpreting? RTs, digs, fans, votes, buzz-ups, up/downloads, shares, likes, invites, favorites, embeds. (More active than the metrics in popularity)

Profitable (conversion / value)

Someone who drives others to some form of value or conversion

10



‘REACH SHOULD BE A RELATIVE NOT AN ABSOLUTE!’NOT ALL CONVERSATIONS ARE RELVANT


EverybodyUnsegmented fans, followers, etc.

The people we really care about

InfluentialAuthoritativeAdvocatesCascaders

Wouldn’t you rather ‘reach’ 100% of only those that really matter vs. 10% of everybody?

B2B space: your influential community might only be 400 people


Objectives

• Visibility• Visits• Volume• Velocity• Awareness• Message delivery• Link love• Dialogue vs. Monologue• Generate dialogue / comments• Feedback• Using, sharing, pass-a-long• Embedding of interactive content• Diggs, Delicious tags, stumbles, etc.• Votes• Etc., etc., etc.

Campaign Types

• Push (get bloggers to review / write)• Pull (drive visitors on-line)• CSR• Product Launch• Product Engagement• Marketing• Blogger Outreach• Advocacy• Issues Management• Community building• Customer relations• Thought leadership • Executive profiling• Etc., etc., etc.

VARIETY OF OBJECTIVES & CAMPAIGN TYPES (NON-EXHAUSTIVE)

12



EXPERIEMENTAL THINKING ON AN APPROACH TO MEASURING SOCIAL MEDIAAN INDEX IS ONLY PART (BELOW-THE-LINE) OF THE ANSWER

13



COMPONENTS OF AN INDEX ?

• Based on an assumption that the components above are drivers of on-line influence, potential action

• They can be measured individually, but it’s important to measure them collectively as they are inter-related

• The index is intended as a measure of potential to influence not actual real influence—that is another research project all together.

• All components are important (but not equal) drivers of potential influence on-line

Popularity + Relevance + Authority + Presence + Engagement + Advocacy/Loyalty + Cascade

x (+/-) Sentiment=

Influence Potential

14



IMAGINE A DIFF’T. PYRAMID FOR DIFF’T. CAMPAIGN TYPES / OBJECTIVES

15



• Each of the above indicators can be adjusted to account for different types of campaigns & contexts

• The weights would change. The scores don’t.

• For an organization, new to social media, perhaps they might only want to establish an on-line presence

• So we’d dial or slide the weights of all but the popularity indicator down to zero

AN INDEX WITH MOVABLE INDICATORSFLEXIBLE ENOUGH TO ACCOUNT FOR DIFFERENT OBJECTIVES

16


Has a significantly scaled back / rudimentary version of this. This is inspired by but significantly expands on R6 thinking.

Indicator Popularity Relevance Authority Brand Engagement Advocacy Cascade SentimentPresence

Score 10 10 10 10 10 10 10 10Weight 1 2 3 4 5 6 7

(@ default)

Popularity + Relevance + Authority + Presence + Engagement + Advocacy + Cascade X Sentiment = Influence Potential


INDICATORS DEFINED

• Cascade• Citation analysis / ripple index• Network analysis• Volume & velocity of message spread across the

network• Inter-connectedness / inter-relationships among:

• all• Only the influentials

• Advocacy (active)• Are social media users advancing our cause?• % containing links• % active contributors• Tags added

• Engagement (more coming)• Dialogue? 2-way? How much? How good? • How many commentators? Repeats? Nature of

the language• Network analysis: inter-connected/related

• Entity Presence• Present? Prominent? Frequent?• # of + mentions• Competitive share of original post• Competitive share of resulting discussion• Attribution recognition

•Authority•Of both poster and commentators•Idea starters or spreaders?•Degree of inter-connectedness / inter-relatedness•Connected but respected?•How central is the influencer to the network?•Following: follower•RTs/1000 followers

•Relevance•On topic?•Author contribution: how often / prolific?•Topical / textual / temporary

•Sentiment (more coming)•Multi-level, multi-author tone•Post-level, theme level, sentence level•Gaps in tone in originating content and resulting discussion

•Popularity (passive) •‘Reach’ ?•Voice of the consumer stuff we can count•Links, votes, shares, recommends, subscribers, registrations, tweets, RTs, followers, fans, friends, diggs, shares, likes, fav’s, upvotes, buzzups, embeds, tags, wish list saves, forwards, invites, etc.

17



• It’s a relative measure of the quality of social media coverage:– One post relative to another and over time – Many posts relative to a number of other similar clusters of postings

• It is a measure of both output and outtake, but not outcome– Output: how much and how good?– Outtake: how engaged, inter-connected, inter-related, volume & velocity of network /

community spread– Outtake: advocacy– it does not claim to be a measure of outcome or impact

• It’s a proxy for on-line influence; it’s about potential to influence– to measure actual influence, we’ve have to survey

• In a diagnostic, benchmarking sense the index can help inform an organization’s social media engagement strategy if tracked over time

• It’s tactical on the fly for mid-campaign course correction and looking back over the long term

• It’s strategic looking forward: helps shape communications recommendations

• This data can be correlated with tangible outcomes

• This data can be plugged into a market mix model to show causality…isolate for PR’s unique contribution to the marketing mix.

WHAT DOES THE INDEX INDICATE?

18



(A QUICK ASIDE ON) ENGAGEMENT MONOLOGUE OR DIALOGUE?

Source: Olivier Blanchard, Social Media ROI

19


Better

Not-so-Good

Best

•Consumers vs. Prosumers

•But haven’t ‘consumers’ always been talking to one another?

•Public vs. private conversations

Isn’t this a ‘system’ or a network?How could you measure this without a systems / network approach?


(A QUICK ASIDE ON) SENTIMENT AS MARKET RESEARCH PROXY / PREDICTOR

Think here about social media monitoring and analysis as: •Real time situational awareness•Listening at the point of need•Listening close to the moment of influence

A real-time digital form of or proxy for market research and CRM:

Mining conversations to tease out how people ‘feel’ about products/services/issues/causes and perhaps even:

1) their intended (reported) behavior and what motivates them to buy, (as cited in the conversations), and

2) their actual behavior tracked via CRM 2.0 and Web 2.0 analytics

BUT!!: we’re only talking about people online and visible / vocal people online (not the casual lurkers) in public social media spaces

20


Source: Crimson Hexagon

SO WHAT?

WHAT ABOUT CONVERSION & ROI?



WE’RE BACK TO THE PYRAMID THE TOP THIS TIME

22



OUTCOMES: KINDS OF CONVERSIONSETTING THE STAGE FOR CORRELATION

• Conversion to a tangible like web traffic, sales, donations, awareness, opinion, usage, brand preference, likelihood to try/buy/recommend.

• Could also mean conversion toward any measurable marketing communication or public affairs, issues, advocacy-based objective.

• Here we could / should also be looking at the extent to which the quantity and quality of social media activity correlates with these outputs.

Micro (steps toward destination) Macro (Destination)

Usually an intermediate means to an end The end

Non-transactional (Engagement) Transactional

Non-financial Financial

Get to x clicks in a site, spend x time on site, download, lead generated, watch a video, created account, contacted us etc.

Sale, donation, (paid) subscription etc.

23



2 WEEKS IN THE SOCIAL MEDIA ECOSYSTEM…THEN…YOUR WEBSITE

Where were they? What drove them to your site? What did they do once on your site? What are they ‘worth’ to you?



Export Data to Your Web Analytics•#of posts by date•# posts with x or higher an influence index score

SOCIAL MEDIA PERFORMANCE DATA + WEB ANALYTICS

25


Or

Integrate web analytics into your social media monitoring platform


LEVELS OF LINKING OUTPUT TO OUTCOMES


Level of Linkage Notes

Just graph ‘er up Plot # of articles vs. web traffic on a graph

Don’t do it

•No indication of relationship other than lines that might trend together

•Could be a fluke

Correlation •“R score”•Some stats •You can do this in excel•Pearson’s Product Moment

•Better

•Showing there is some relationship beyond coincidence

•Not ‘proving’ media exclusively responsible for sales

Causality •Heavy duty stats•You so can’t do this in excel•Sophisticated modeling•Market Mix Modeling•Multivariate Regression Analysis

•Best

•Isolates (statistically ‘proves’) for PR’s unique contribution to the communications mix

•Ton ‘o data, time, $

26


Michael Kelly


UPPERCASE ‘ROI’ vs. lowercase ‘roi’ & Other Return-Offs

ROI roi

MBA PR

Financial Non-financial

Macro Micro (not all conversion events are ROI)

A mathematical equation that works

Usually conceptual only; an equation in words only; can’t actually be calculated; something the industry tosses around too liberally

A business metric All-too-often a media or audience metric

Direct, clear A loose proxy or an intermediate metric (return on expectation, return on objective, return on engagement)

27




SOCIAL MEDIA BOOK BAG

• Social Media Metrics: • Jim Sterne, (@jimsterne)

• Social Media ROI: • Olivier Blanchard (@thebrandbuilder)

• Social Media Analytics: Effective Tools for Building, Interpreting & Using Metrics: • Marshal Sponder (@webmetricsguru)

• Measure What Matters: • Katie Paine (@kdpaine)

• Using Web Analytics to Measure Impact of Earned Online Media…:• Seth Duncan (www.instituteforpr.org)

28


THANK YOU

QUESTIONS?


Alan Chumley



APPENDIX


Alan Chumley




How are these changing over time?

Might be part of many systems / networks?

Linking networks to networks


NETWORK ANALYSIS

Source: K. Ognyanova, UCLA, 2010

Measuring Social Media Prsa Digital Impact

Technology

Transcript of Measuring Social Media Prsa Digital Impact