Social Media is Dead! Long Live Social Media! at PRSA Houston #PRDay2013
Measure What Matters: Monitoring and Measurement in the New Media...
Transcript of Measure What Matters: Monitoring and Measurement in the New Media...
![Page 1: Measure What Matters: Monitoring and Measurement in the New Media …painepublishing.com/wp-content/uploads/2015/04/PRSA-NCC... · 2015-05-07 · Social Media isn’t media, it’s](https://reader033.fdocuments.us/reader033/viewer/2022050607/5fae60c192b42d396277cbf2/html5/thumbnails/1.jpg)
Measure What Matters: Monitoring and Measurement
in the New Media World
PRSA NCC Katie Delahaye Paine CEO Paine Publishing March 19, 2015
www.painepublishing.com
@queenofmetrics
![Page 2: Measure What Matters: Monitoring and Measurement in the New Media …painepublishing.com/wp-content/uploads/2015/04/PRSA-NCC... · 2015-05-07 · Social Media isn’t media, it’s](https://reader033.fdocuments.us/reader033/viewer/2022050607/5fae60c192b42d396277cbf2/html5/thumbnails/2.jpg)
Paine Publishing: Providing communications’
professionals the knowledge and information they need to navigate their path to perfect measurement
Newsletters
Training Courses
Consulting painepublishing.com
Katie Delahaye Paine: Helping communications professionals define and measure success for 25 years.
Founder of:
The Delahaye Group
KDPaine & Partners
Paine Publishing
Author of:
Measuring the Networked Non-Profit
Measure What Matters
Measuring Public Relationships [email protected]
![Page 3: Measure What Matters: Monitoring and Measurement in the New Media …painepublishing.com/wp-content/uploads/2015/04/PRSA-NCC... · 2015-05-07 · Social Media isn’t media, it’s](https://reader033.fdocuments.us/reader033/viewer/2022050607/5fae60c192b42d396277cbf2/html5/thumbnails/3.jpg)
Don’t Measure What’s Easy
3
Cartoon by Rob Cottingham
![Page 4: Measure What Matters: Monitoring and Measurement in the New Media …painepublishing.com/wp-content/uploads/2015/04/PRSA-NCC... · 2015-05-07 · Social Media isn’t media, it’s](https://reader033.fdocuments.us/reader033/viewer/2022050607/5fae60c192b42d396277cbf2/html5/thumbnails/4.jpg)
Measure What Matters
20/03/2015 News Group International 4
Cartoon by Rob Cottingham
![Page 5: Measure What Matters: Monitoring and Measurement in the New Media …painepublishing.com/wp-content/uploads/2015/04/PRSA-NCC... · 2015-05-07 · Social Media isn’t media, it’s](https://reader033.fdocuments.us/reader033/viewer/2022050607/5fae60c192b42d396277cbf2/html5/thumbnails/5.jpg)
Measuring Attribution in an era of all media everywhere
Web Analytics
eCommerce
iCloud &
Google+
Map My Run/Social Sharing
CRM/Convio
![Page 6: Measure What Matters: Monitoring and Measurement in the New Media …painepublishing.com/wp-content/uploads/2015/04/PRSA-NCC... · 2015-05-07 · Social Media isn’t media, it’s](https://reader033.fdocuments.us/reader033/viewer/2022050607/5fae60c192b42d396277cbf2/html5/thumbnails/6.jpg)
Conversations
$$$$$$$$$$$$$$$$$$$$$$$$$$
Savings, shorter cycles, more
renewals, better ideas,
Product
Support HR IR Sales CI R&D Mktg
Research
Prod.
Mktg
PR//Mkt
7
Social Media isn’t media, it’s your business
3/20/2015
![Page 7: Measure What Matters: Monitoring and Measurement in the New Media …painepublishing.com/wp-content/uploads/2015/04/PRSA-NCC... · 2015-05-07 · Social Media isn’t media, it’s](https://reader033.fdocuments.us/reader033/viewer/2022050607/5fae60c192b42d396277cbf2/html5/thumbnails/7.jpg)
We need a new Attribution Model
ROI
Other Paid
Marketing
Digital/On-line
Media Buy
Print/TV Media
Buy
ROI
Media Relations
/Social Media/PR
Digital/ Online Media
Buy
Print/TV
Media Buy
Media
Relations/
Social
Media/PR
![Page 8: Measure What Matters: Monitoring and Measurement in the New Media …painepublishing.com/wp-content/uploads/2015/04/PRSA-NCC... · 2015-05-07 · Social Media isn’t media, it’s](https://reader033.fdocuments.us/reader033/viewer/2022050607/5fae60c192b42d396277cbf2/html5/thumbnails/8.jpg)
What’s Changed?
Collapse of mass media
Growth of media
everywhere
Intolerance for messaging
The Barcelona Principles
9
“Viewers are more likely to stop watching commercials at the moment in which brand logos appear on the screen” - ARF Study
![Page 9: Measure What Matters: Monitoring and Measurement in the New Media …painepublishing.com/wp-content/uploads/2015/04/PRSA-NCC... · 2015-05-07 · Social Media isn’t media, it’s](https://reader033.fdocuments.us/reader033/viewer/2022050607/5fae60c192b42d396277cbf2/html5/thumbnails/9.jpg)
The Barcelona Principles, The Conclave &
Industry Standards
1. Importance of Goal Setting and Measurement
2. Measuring the Effect on Outcomes is Preferred to Measuring Outputs
3. The Effect on Business Results Can and Should Be Measured Where Possible
4. Media Measurement Requires Quantity and Quality
5. Earned Media Value/AVEs are not the value of Public Relations
6. Social Media Can and Should be Measured
7. Transparency and Replicability are Paramount to Sound Measurement
All standards are available
on
http://www.painepublishing.c
om/standards-central
The Conclave
![Page 10: Measure What Matters: Monitoring and Measurement in the New Media …painepublishing.com/wp-content/uploads/2015/04/PRSA-NCC... · 2015-05-07 · Social Media isn’t media, it’s](https://reader033.fdocuments.us/reader033/viewer/2022050607/5fae60c192b42d396277cbf2/html5/thumbnails/10.jpg)
Social Media Measurement Standards have been published for:
Content Sourcing & Methods 1
Reach and Impressions 2
Engagement 3
Influence & Relevance 4
Opinion & Advocacy 5
Impact & Value 6
3/20/2015
![Page 11: Measure What Matters: Monitoring and Measurement in the New Media …painepublishing.com/wp-content/uploads/2015/04/PRSA-NCC... · 2015-05-07 · Social Media isn’t media, it’s](https://reader033.fdocuments.us/reader033/viewer/2022050607/5fae60c192b42d396277cbf2/html5/thumbnails/11.jpg)
The Transparency Table:
![Page 12: Measure What Matters: Monitoring and Measurement in the New Media …painepublishing.com/wp-content/uploads/2015/04/PRSA-NCC... · 2015-05-07 · Social Media isn’t media, it’s](https://reader033.fdocuments.us/reader033/viewer/2022050607/5fae60c192b42d396277cbf2/html5/thumbnails/12.jpg)
Eyeball counting
HITS Outcomes
MSM Online Social Media
Impressions are not awareness. Where’s the “So What”?
13
![Page 13: Measure What Matters: Monitoring and Measurement in the New Media …painepublishing.com/wp-content/uploads/2015/04/PRSA-NCC... · 2015-05-07 · Social Media isn’t media, it’s](https://reader033.fdocuments.us/reader033/viewer/2022050607/5fae60c192b42d396277cbf2/html5/thumbnails/13.jpg)
Likes Are Not Engagement
14
Advocacy Commitment Trial/Consideration
Followers Likes Impressions
3/20/2015
![Page 14: Measure What Matters: Monitoring and Measurement in the New Media …painepublishing.com/wp-content/uploads/2015/04/PRSA-NCC... · 2015-05-07 · Social Media isn’t media, it’s](https://reader033.fdocuments.us/reader033/viewer/2022050607/5fae60c192b42d396277cbf2/html5/thumbnails/14.jpg)
• Influence happens when you are persuaded to
change behavior or opinion that would
otherwise not have changed.
• Must include:
• Reach
• Engagement around individual
• Relevance to topic
• Frequency of posts around the topic
• Audience impact as measured by the
ability to get the target audience to change
behavior or opinion.
• If an individual scores a 0 on one element, they
have no influence
• Influence cannot be expressed in a single score
or algorithm
#4: Influence & Relevance
![Page 15: Measure What Matters: Monitoring and Measurement in the New Media …painepublishing.com/wp-content/uploads/2015/04/PRSA-NCC... · 2015-05-07 · Social Media isn’t media, it’s](https://reader033.fdocuments.us/reader033/viewer/2022050607/5fae60c192b42d396277cbf2/html5/thumbnails/15.jpg)
#5 Standards for Measuring Tone/Sentiment
Sentiment is overrated
Messaging and Positioning &
Recommendations are more useful
Whatever process is defined and
applied, it must be used
consistently throughout any
analysis.
Validity testing is critical
Comparison of Tonality Between Vendors
33.12%
58.00%
7.84%
11.95%
26.53%
13.00%
0.32%
9.46%
40.35%
30.00%
92.11%
78.00%
SDL
Beyond
NetBase
UberVu
% positive %negative %neutral
![Page 16: Measure What Matters: Monitoring and Measurement in the New Media …painepublishing.com/wp-content/uploads/2015/04/PRSA-NCC... · 2015-05-07 · Social Media isn’t media, it’s](https://reader033.fdocuments.us/reader033/viewer/2022050607/5fae60c192b42d396277cbf2/html5/thumbnails/16.jpg)
Discussions of nature, specifically the celebrity’s Twitter fundraising efforts were
most successful in conveying positive relationships concepts
![Page 17: Measure What Matters: Monitoring and Measurement in the New Media …painepublishing.com/wp-content/uploads/2015/04/PRSA-NCC... · 2015-05-07 · Social Media isn’t media, it’s](https://reader033.fdocuments.us/reader033/viewer/2022050607/5fae60c192b42d396277cbf2/html5/thumbnails/17.jpg)
• Impact and value will always be
dependent on client objectives
• Need to define outcomes in advance
– will likely span multiple business
goals, especially for social (crosses
disciplines)
• “ROI” should be strictly limited to
measurable financial impact;
#6: Impact & Value
Type I love Zappos into Google,
and you find 1.19 million
references
Type Citibank and you get 21,000
references. Citibank spends 100
times more a year on advertising
than Zappos.
Cost per delegate acquired:
Obama: $6,024
Clinton: $147,058
Romney: $2,389,464
![Page 18: Measure What Matters: Monitoring and Measurement in the New Media …painepublishing.com/wp-content/uploads/2015/04/PRSA-NCC... · 2015-05-07 · Social Media isn’t media, it’s](https://reader033.fdocuments.us/reader033/viewer/2022050607/5fae60c192b42d396277cbf2/html5/thumbnails/18.jpg)
ROI
ROI is:
Cost savings
Greater efficiency
Lower legal costs
Faster time to implementation
Lower cost per impact vs. other
marketing channels
Lower cost per message
exposure vs. other channels
ROI is not:
Impressions
Earned media value or AVE
Number of shares, views or
likes
![Page 19: Measure What Matters: Monitoring and Measurement in the New Media …painepublishing.com/wp-content/uploads/2015/04/PRSA-NCC... · 2015-05-07 · Social Media isn’t media, it’s](https://reader033.fdocuments.us/reader033/viewer/2022050607/5fae60c192b42d396277cbf2/html5/thumbnails/19.jpg)
Custom Standards Standards for Media Reporting
Web Analytics
5
![Page 20: Measure What Matters: Monitoring and Measurement in the New Media …painepublishing.com/wp-content/uploads/2015/04/PRSA-NCC... · 2015-05-07 · Social Media isn’t media, it’s](https://reader033.fdocuments.us/reader033/viewer/2022050607/5fae60c192b42d396277cbf2/html5/thumbnails/20.jpg)
ACA Measurement Program Parameters
Goal: Move to Industry Standards Metrics and away from media value and ad value equivalency
Measure quality not just quantity
Began in January 2013
Established a benchmark: January-March 2013
Customized “Quality Score” that indexes message content, tone, and media importance on a scale of +10 to -10
Correlated with web traffic to measure outcomes
Correlated with longitudinal research to determine impact on preference and intent to visit
Data used continuously in strategic decision making
![Page 21: Measure What Matters: Monitoring and Measurement in the New Media …painepublishing.com/wp-content/uploads/2015/04/PRSA-NCC... · 2015-05-07 · Social Media isn’t media, it’s](https://reader033.fdocuments.us/reader033/viewer/2022050607/5fae60c192b42d396277cbf2/html5/thumbnails/21.jpg)
Definitions of “Success” Workshop defined the criteria
All criteria linked back to the goals:
High Quality Media Coverage
Intent to visit
Improved reputation
Visits/Non-gaming revenue
![Page 22: Measure What Matters: Monitoring and Measurement in the New Media …painepublishing.com/wp-content/uploads/2015/04/PRSA-NCC... · 2015-05-07 · Social Media isn’t media, it’s](https://reader033.fdocuments.us/reader033/viewer/2022050607/5fae60c192b42d396277cbf2/html5/thumbnails/22.jpg)
Reality: Negative Coverage of AC is Constant and Growing
0
100
200
300
400
500
600
700
Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec Jan Feb Mar Apr May Jun
Tone of Coverage about Atlantic City Over Time
Negative Neutral Positive
Miss America
![Page 23: Measure What Matters: Monitoring and Measurement in the New Media …painepublishing.com/wp-content/uploads/2015/04/PRSA-NCC... · 2015-05-07 · Social Media isn’t media, it’s](https://reader033.fdocuments.us/reader033/viewer/2022050607/5fae60c192b42d396277cbf2/html5/thumbnails/23.jpg)
Solution: ACA Programs generate predominantly positive
coverage
0
20
40
60
80
100
120
140
160
Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec Jan Feb Mar Apr May Jun
Sentiment of ACA Coverage Over Time
Negative Neutral Positive
![Page 24: Measure What Matters: Monitoring and Measurement in the New Media …painepublishing.com/wp-content/uploads/2015/04/PRSA-NCC... · 2015-05-07 · Social Media isn’t media, it’s](https://reader033.fdocuments.us/reader033/viewer/2022050607/5fae60c192b42d396277cbf2/html5/thumbnails/24.jpg)
Without ACA events, OCS Scores for Atlantic City would
have been significantly lower
4
3.25
2.75 2.99
3.65
2.96 3.36 3.24
2.34 2.37 2.43
1.30
-1.24
0.37 -0.05
0.28 0.28
-1.56
4.91
3.92
2.99
3.58
4.14 4.1 4.27 4.12 4.29
2.78 2.56
1.53
-0.29
0.61 0.20
1.44 1.77
0.63
-2
-1
0
1
2
3
4
5
6
Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec Jan Feb Mar Apr May June
Ave
rage
OC
S Sc
ore
Average OCS Score Over Time
Atlantic City OCS without ACA Atlantic City OCS with ACA
The red line represents coverage of Atlantic City minus all mentions of ACA and its programs
![Page 25: Measure What Matters: Monitoring and Measurement in the New Media …painepublishing.com/wp-content/uploads/2015/04/PRSA-NCC... · 2015-05-07 · Social Media isn’t media, it’s](https://reader033.fdocuments.us/reader033/viewer/2022050607/5fae60c192b42d396277cbf2/html5/thumbnails/25.jpg)
ACA has made a demonstrable difference in the quality of
media coverage
If an ACA or one of its
programs was mentioned
in a media story, it was:
More likely to contain
an endorsement
More likely to be
positive
More likely to contain
key messages
Less likely to be
negative
0.22
0.00
0.62
1.42
4.61
0.08
0.08
0.01
-0.61
-0.66
Endorsements
Positive Visual
Negative Visual
Tone
MessageCommunication
Average of OCS
Differences in Quality of Coverage with and without
ACA
Atlantic City Atlantic City Alliance
![Page 26: Measure What Matters: Monitoring and Measurement in the New Media …painepublishing.com/wp-content/uploads/2015/04/PRSA-NCC... · 2015-05-07 · Social Media isn’t media, it’s](https://reader033.fdocuments.us/reader033/viewer/2022050607/5fae60c192b42d396277cbf2/html5/thumbnails/26.jpg)
OCS scores for Atlantic City significantly improved thanks
to ACA events
36
278
458 334
537 432
342 225
995
329 324 247
139
419
260
476 538
925
4.89
3.92
2.99
3.58
4.14 4.10 4.27 4.12
4.29
2.78 2.56
1.55
-0.29
0.61
0.20
1.44 1.77
0.63
-1.00
0.00
1.00
2.00
3.00
4.00
5.00
6.00
Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec Jan Feb Mar Apr May June
0
200
400
600
800
1000
1200
OC
S Sc
ore
Nu
mb
er o
f it
ems
AC Mentions vs. OCS Over Time
AC Mentions AC OCS
Dead whale, non-gaming promotions, Caesar’s CEO quote,
Lady Antibellum
Revel Casino value, Electric
Jitney test, Fake Chips, Phil Ivey suit, Blake
Shelton
Showboat closing, Revel facing 2nd
bankruptcy
![Page 27: Measure What Matters: Monitoring and Measurement in the New Media …painepublishing.com/wp-content/uploads/2015/04/PRSA-NCC... · 2015-05-07 · Social Media isn’t media, it’s](https://reader033.fdocuments.us/reader033/viewer/2022050607/5fae60c192b42d396277cbf2/html5/thumbnails/27.jpg)
Communication of the "Making a Comeback" & "Something for Everyone" messages increased as ACA announced new programs
14 39
23 36
133
235
8 7 4 2 10 15 8 11 1 2 9 2 24
70
21
73 88
110
13
104 74
138
87
48
Jan Feb Mar Apr May Jun
Qtr1 Qtr2
2014
Atlantic City has something for everyone Atlantic City is a clean and safe place to visit
Atlantic City is a year-round destination spot Atlantic City is making a comeback
Atlantic City offers more than just gaming
![Page 28: Measure What Matters: Monitoring and Measurement in the New Media …painepublishing.com/wp-content/uploads/2015/04/PRSA-NCC... · 2015-05-07 · Social Media isn’t media, it’s](https://reader033.fdocuments.us/reader033/viewer/2022050607/5fae60c192b42d396277cbf2/html5/thumbnails/28.jpg)
Free entertainment generated the highest OCS Scores in
Q2
5.26 5.73
6.36 6.54
6.71 7.15
7.31 7.36
7.50 7.67
8.00 8.10
8.68 9.30
Miss AmericaMeet AC
DO ACJuly 4th Fireworks
Air ShowBlake Shelton
Miss'd America PageantSand Blast
Hello SummerBoardwalk Hall Light Show
Challenge TriathlonSand Sculpting World CupLady Antebellum Concert
Free Entertainment
Top Programs By OCS
![Page 29: Measure What Matters: Monitoring and Measurement in the New Media …painepublishing.com/wp-content/uploads/2015/04/PRSA-NCC... · 2015-05-07 · Social Media isn’t media, it’s](https://reader033.fdocuments.us/reader033/viewer/2022050607/5fae60c192b42d396277cbf2/html5/thumbnails/29.jpg)
When ACA programs received media coverage, traffic
followed
0
50
100
150
200
250
300
0
20000
40000
60000
80000
100000
120000
140000
Relationship between ACA Program Mentions and Site Visits
Site Visits Program Mentions
![Page 30: Measure What Matters: Monitoring and Measurement in the New Media …painepublishing.com/wp-content/uploads/2015/04/PRSA-NCC... · 2015-05-07 · Social Media isn’t media, it’s](https://reader033.fdocuments.us/reader033/viewer/2022050607/5fae60c192b42d396277cbf2/html5/thumbnails/30.jpg)
ACA programs drive higher OCS scores, which correlate
highly with web visits
0.41
0.44
0.47
AC items
ACA Items
ACA OCS Scores
Correlations between Web Visits and PR Metrics
Pearson r. value
![Page 31: Measure What Matters: Monitoring and Measurement in the New Media …painepublishing.com/wp-content/uploads/2015/04/PRSA-NCC... · 2015-05-07 · Social Media isn’t media, it’s](https://reader033.fdocuments.us/reader033/viewer/2022050607/5fae60c192b42d396277cbf2/html5/thumbnails/31.jpg)
PR is having a positive impact on preference and
perceptions
People who say they have recently heard news about Atlantic
City are very likely to perceive AC as fun – a key driver of
preference
Respondents who remember seeing news reports about
Atlantic City are also very likely to associate key messages
and positioning statements such as “place I am excited to go
to” and “fun place to hang out with friends.”
The same respondents are also more likely to recommend
Atlantic City to friends.
![Page 32: Measure What Matters: Monitoring and Measurement in the New Media …painepublishing.com/wp-content/uploads/2015/04/PRSA-NCC... · 2015-05-07 · Social Media isn’t media, it’s](https://reader033.fdocuments.us/reader033/viewer/2022050607/5fae60c192b42d396277cbf2/html5/thumbnails/32.jpg)
6 steps to standards-compliant
measurement
Step 1: Define your goal(s).
What outcomes is this strategy or tactic going to achieve?
What are your measurable objectives?
Step 2: Define the parameters
Who are you are trying to reach? How do your efforts
connect with those audiences to achieve the goal.
Step 3: Define your benchmarks.
Who or what are you going to compare your results to?
Step 4: Define your metrics.
What are the indicators to judge your progress?
Step 5: Select your data collection tool(s).
Step 6: Analyze your data.
Turn it into action, measure again
33
Six Steps
to Success
1
2
3
4
5
6
![Page 33: Measure What Matters: Monitoring and Measurement in the New Media …painepublishing.com/wp-content/uploads/2015/04/PRSA-NCC... · 2015-05-07 · Social Media isn’t media, it’s](https://reader033.fdocuments.us/reader033/viewer/2022050607/5fae60c192b42d396277cbf2/html5/thumbnails/33.jpg)
Definitions
Monitoring – process by which data are systematically and regularly collected about a program over time.
Measurement – a way of giving an activity a precise dimension, generally by comparison to some standard; usually done in a quantifiable or numerical manner; see also: data, scale
Metrics – an agreed upon number or percentage that indicates progress towards a goal
Evaluation –a form of research that determines the relative effectiveness of a public relations campaign or program by measuring program*
34
*IPR Dictionary of Measurement Terms, Dr. Donald Stacks www..instituteforpr.org
![Page 34: Measure What Matters: Monitoring and Measurement in the New Media …painepublishing.com/wp-content/uploads/2015/04/PRSA-NCC... · 2015-05-07 · Social Media isn’t media, it’s](https://reader033.fdocuments.us/reader033/viewer/2022050607/5fae60c192b42d396277cbf2/html5/thumbnails/34.jpg)
Why do we communicate?
3/20/2015 35
Outtakes
(Intermediary Effects)
• Awareness
• Knowledge/Education
• Understanding
Outcomes
(Target Audience Action)
• Revenue
• Leads
• Engagement
• Advocacy
Activities How does what you do
contribute to the bottom line?
![Page 35: Measure What Matters: Monitoring and Measurement in the New Media …painepublishing.com/wp-content/uploads/2015/04/PRSA-NCC... · 2015-05-07 · Social Media isn’t media, it’s](https://reader033.fdocuments.us/reader033/viewer/2022050607/5fae60c192b42d396277cbf2/html5/thumbnails/35.jpg)
Step 1: Define the goals
What return is expected? – Define in terms of the
mission.
Define your champagne moment. If you are
celebrating complete 100% success a year from
now, what is different about the organization?
36
![Page 36: Measure What Matters: Monitoring and Measurement in the New Media …painepublishing.com/wp-content/uploads/2015/04/PRSA-NCC... · 2015-05-07 · Social Media isn’t media, it’s](https://reader033.fdocuments.us/reader033/viewer/2022050607/5fae60c192b42d396277cbf2/html5/thumbnails/36.jpg)
Goals & Suggested Metrics
Goals Goal 1: Meet Sales Targets
Goal 2: Reduce Risk/Threats
Goal 3: Increase market share in
new market
Communications Contribution
• Expand the marketable universe
• Reduce sales cycle
• Increase Trust • Increase
advocacy
• Expand the marketable
universe
Metrics
• % Increase in desirable share of voice
• % in awareness
• % decrease in undesirable voice
• % increase in trust scores
• % increase in desirable voice in new market
• % increase in awareness of brand in new market
![Page 37: Measure What Matters: Monitoring and Measurement in the New Media …painepublishing.com/wp-content/uploads/2015/04/PRSA-NCC... · 2015-05-07 · Social Media isn’t media, it’s](https://reader033.fdocuments.us/reader033/viewer/2022050607/5fae60c192b42d396277cbf2/html5/thumbnails/37.jpg)
Step 2: Understand the parameters.
What management’s priorities?
Who are you are trying to reach?
How do your efforts connect with
those audiences to achieve the
goal?
What influences their decisions?
What’s important to them?
What makes them act?
38
Goal: Get the cat to stop howling
Options: Local? Cheap? Convenient?
Strategy: Buy cat food
![Page 38: Measure What Matters: Monitoring and Measurement in the New Media …painepublishing.com/wp-content/uploads/2015/04/PRSA-NCC... · 2015-05-07 · Social Media isn’t media, it’s](https://reader033.fdocuments.us/reader033/viewer/2022050607/5fae60c192b42d396277cbf2/html5/thumbnails/38.jpg)
Step 3: Establish benchmarks
Past Performance Over Time
Think 3
Whatever keeps your
C-suite up at night
39
![Page 39: Measure What Matters: Monitoring and Measurement in the New Media …painepublishing.com/wp-content/uploads/2015/04/PRSA-NCC... · 2015-05-07 · Social Media isn’t media, it’s](https://reader033.fdocuments.us/reader033/viewer/2022050607/5fae60c192b42d396277cbf2/html5/thumbnails/39.jpg)
Step 4: Pick your Kick-Butt Index
The Perfect KBI
Is actionable
Is there when you need it
Continuously improves your processes
& gets you where you want to go
You become what you measure, so
pick your KBI carefully
40
![Page 40: Measure What Matters: Monitoring and Measurement in the New Media …painepublishing.com/wp-content/uploads/2015/04/PRSA-NCC... · 2015-05-07 · Social Media isn’t media, it’s](https://reader033.fdocuments.us/reader033/viewer/2022050607/5fae60c192b42d396277cbf2/html5/thumbnails/40.jpg)
Criteria Result Score Result Score
Tonality Favorable +3 Unfavorable -3
Messaging Yes +3 No -1
Quotes Yes +1 No -2
Dominance Dominant +1
Not Dominant
-or-
Unfavorable &
Dominant -1
Visibility High Visibility +2
Low Visibility
-or-
Unfavorable High
Visibility -3
Final Score +10 -10
How to calculate your Kick Butt Index
41
![Page 41: Measure What Matters: Monitoring and Measurement in the New Media …painepublishing.com/wp-content/uploads/2015/04/PRSA-NCC... · 2015-05-07 · Social Media isn’t media, it’s](https://reader033.fdocuments.us/reader033/viewer/2022050607/5fae60c192b42d396277cbf2/html5/thumbnails/41.jpg)
Step 5: Pick the right measurement tools
If you want to measure messaging, positioning,
themes, sentiment:
Content analysis
If you want to measure awareness, perception,
relationships, preference:
Survey research
If you want to measure engagement, action,
purchase:
Web analytics
If you want predictions and correlations
you need two out of three
![Page 42: Measure What Matters: Monitoring and Measurement in the New Media …painepublishing.com/wp-content/uploads/2015/04/PRSA-NCC... · 2015-05-07 · Social Media isn’t media, it’s](https://reader033.fdocuments.us/reader033/viewer/2022050607/5fae60c192b42d396277cbf2/html5/thumbnails/42.jpg)
Step 5: Selecting a measurement tool
43
Objective KBI Tool
Increase message
communications
Increase percent of items containing one or
more messages
CyberAlert, Prime Research
Increase
awareness/preference
% of audience preferring your brand
to the competition
Survey Monkey, Survata
Engage
marketplace
% increase in engagement on website and/or
social sites
Simply Measured, Unmetric,
Google Analytics, Site Catalyst,
Network Analysis
Become a thought
leader on a specific
issue/market/topic
% increase in desirable share of voice on
issue/market/top
Little Bird, Trackr, CyberAlert
![Page 43: Measure What Matters: Monitoring and Measurement in the New Media …painepublishing.com/wp-content/uploads/2015/04/PRSA-NCC... · 2015-05-07 · Social Media isn’t media, it’s](https://reader033.fdocuments.us/reader033/viewer/2022050607/5fae60c192b42d396277cbf2/html5/thumbnails/43.jpg)
![Page 44: Measure What Matters: Monitoring and Measurement in the New Media …painepublishing.com/wp-content/uploads/2015/04/PRSA-NCC... · 2015-05-07 · Social Media isn’t media, it’s](https://reader033.fdocuments.us/reader033/viewer/2022050607/5fae60c192b42d396277cbf2/html5/thumbnails/44.jpg)
Testing the accuracy of coding
48.94%
68.31%
88.64%
0.00% 20.00% 40.00% 60.00% 80.00% 100.00%
SDL
UberVu
NetBase
% Agreement with human coding
SDL UberVu NetBase100.00%
62.32%
14.82%
72.36%
0.00%
37.68%
85.18%
27.64%
Mixed
Negative
Neutral
Positive
Degree to which agreement occurred
No Yes
![Page 45: Measure What Matters: Monitoring and Measurement in the New Media …painepublishing.com/wp-content/uploads/2015/04/PRSA-NCC... · 2015-05-07 · Social Media isn’t media, it’s](https://reader033.fdocuments.us/reader033/viewer/2022050607/5fae60c192b42d396277cbf2/html5/thumbnails/45.jpg)
3
Testing validity of criteria
![Page 46: Measure What Matters: Monitoring and Measurement in the New Media …painepublishing.com/wp-content/uploads/2015/04/PRSA-NCC... · 2015-05-07 · Social Media isn’t media, it’s](https://reader033.fdocuments.us/reader033/viewer/2022050607/5fae60c192b42d396277cbf2/html5/thumbnails/46.jpg)
Biggest Areas of Discrepancies
7
Geographies vs Languages
Lack of consistent competitors list
Different definitions of Products/Brands Covered
Time frames
Media Types
Search strings and filters
![Page 47: Measure What Matters: Monitoring and Measurement in the New Media …painepublishing.com/wp-content/uploads/2015/04/PRSA-NCC... · 2015-05-07 · Social Media isn’t media, it’s](https://reader033.fdocuments.us/reader033/viewer/2022050607/5fae60c192b42d396277cbf2/html5/thumbnails/47.jpg)
Step 6: Be Data Informed, not Data Driven
Rank order results from worst to best
Ask “So What?” at least three times
Put your data into an overall framework consistent with C-Suite expectations
Find your “Data Geek”
Compare to last month, last quarter, 13-month average
48 Page 48
![Page 48: Measure What Matters: Monitoring and Measurement in the New Media …painepublishing.com/wp-content/uploads/2015/04/PRSA-NCC... · 2015-05-07 · Social Media isn’t media, it’s](https://reader033.fdocuments.us/reader033/viewer/2022050607/5fae60c192b42d396277cbf2/html5/thumbnails/48.jpg)
Photo Event
High Message Content
Resource Use
Low
Hig
h
Med
ium
Ver
y h
igh
Level of Engagement
Ver
y H
igh
Med
ium
Hig
h
Low
High Resources
No Message
Low Resources
Webinar
Status update
Link
Ultimate Road Trip
Google + Chat
Media Day
Corporate Video
Advocacy vs. resource use
Resource Use
Low
Hig
h
Med
ium
Ver
y h
igh
Ver
y H
igh
Med
ium
Hig
h
Low
High Resources Low Resources
Webinar
Status update
Link
Ultimate Road Trip
Google + Chat
Media Day
Corporate Video
![Page 49: Measure What Matters: Monitoring and Measurement in the New Media …painepublishing.com/wp-content/uploads/2015/04/PRSA-NCC... · 2015-05-07 · Social Media isn’t media, it’s](https://reader033.fdocuments.us/reader033/viewer/2022050607/5fae60c192b42d396277cbf2/html5/thumbnails/49.jpg)
Charting KBI over time between divisions
-100%
0%
100%
200%
300%
400%
500%
Jul Aug Sep Oct Nov Dec Jan Feb Mar Apr May Jun Jul
2007 2008
% D
iffer
ence
Kick Butt Score Relative to other LOBs The Percent Difference Between Each Business Unit's Average KBI and the
KBI for each Business Unit
SAS
IDS
IIS
MSDIB
FBX-T
ALR-67(V)3
DDG-1000 MSE
APG-79, APG63
AESA
for F-15E,
VIIRS delays
Glory APS and VIIRS vs.
competitors' EPX
ASAT, Patriot
ERGM cancellation
Patriot (Korea)
RISS, GBS
MALD, AMRAAM
APG-63 (v3)
NPOESS; BOE B-52 jammer
RIS JPL
50
![Page 50: Measure What Matters: Monitoring and Measurement in the New Media …painepublishing.com/wp-content/uploads/2015/04/PRSA-NCC... · 2015-05-07 · Social Media isn’t media, it’s](https://reader033.fdocuments.us/reader033/viewer/2022050607/5fae60c192b42d396277cbf2/html5/thumbnails/50.jpg)
Connecting data yield greater insights
106 38 24
73 101 49 25
1087
1331
699 637
848
1331
699
10.25
35.03
29.13
8.73 8.40
27.16 27.96
0
5
10
15
20
25
30
35
40
0
200
400
600
800
1000
1200
1400
Jan Feb Mar April May June July
Relationship between interviews and placements
Number of Interviews Number of placements Interview per placement
![Page 51: Measure What Matters: Monitoring and Measurement in the New Media …painepublishing.com/wp-content/uploads/2015/04/PRSA-NCC... · 2015-05-07 · Social Media isn’t media, it’s](https://reader033.fdocuments.us/reader033/viewer/2022050607/5fae60c192b42d396277cbf2/html5/thumbnails/51.jpg)
968 1027
914
208
297 261 269
215 222 145
328
204
510
188 222 234
41 71 50 36 25 56 30 71 45
107
Share & Likes Over Time
Average of PostLikes Average of PShares
Not all engagement is equal
Mia Farrow
David Beckham & Dierks Bentley
Video
![Page 52: Measure What Matters: Monitoring and Measurement in the New Media …painepublishing.com/wp-content/uploads/2015/04/PRSA-NCC... · 2015-05-07 · Social Media isn’t media, it’s](https://reader033.fdocuments.us/reader033/viewer/2022050607/5fae60c192b42d396277cbf2/html5/thumbnails/52.jpg)
White House Volunteerism Office (CNCS) is able to connect
specific social outreach to registrations on serve.gov
53
ReTweets compared to Visitors to Serve.gov
![Page 53: Measure What Matters: Monitoring and Measurement in the New Media …painepublishing.com/wp-content/uploads/2015/04/PRSA-NCC... · 2015-05-07 · Social Media isn’t media, it’s](https://reader033.fdocuments.us/reader033/viewer/2022050607/5fae60c192b42d396277cbf2/html5/thumbnails/53.jpg)
MLK saw the highest correlation of coverage to
visits
0.133
0.156
0.355
0.379
0.427
0.848
Learn & Serve
SeniorCorps
CNCS
AmeriCorps
Get Involved
MLK
![Page 54: Measure What Matters: Monitoring and Measurement in the New Media …painepublishing.com/wp-content/uploads/2015/04/PRSA-NCC... · 2015-05-07 · Social Media isn’t media, it’s](https://reader033.fdocuments.us/reader033/viewer/2022050607/5fae60c192b42d396277cbf2/html5/thumbnails/54.jpg)
10 Ways to find the Money for Measurement
1. Don’t call it measurement – It’s Research
2. Consolidate - Social? IR? Agencies? Research?
3. Crowdsource – Who else has tools or surveys?
4. Spread the cost over 2 fiscal years
5. Reach out to universities
6. Take advantage of free stuff
7. Monitor only what matters
8. Narrow your Top Tier media list to what you CAN afford
9. Google analytics
10. Facebook Insights
11. Twitter Analytics
![Page 55: Measure What Matters: Monitoring and Measurement in the New Media …painepublishing.com/wp-content/uploads/2015/04/PRSA-NCC... · 2015-05-07 · Social Media isn’t media, it’s](https://reader033.fdocuments.us/reader033/viewer/2022050607/5fae60c192b42d396277cbf2/html5/thumbnails/55.jpg)
Remember These Points
It’s not about the media, it’s about the business and the customer 1
It’s not about Big Data, but about how you use it. 2
You need to be data informed, not data-driven 3
It’s not how loud you’re shouting it’s about relationships. 4
Standards are a reality not an excuse to hide behind 5
![Page 56: Measure What Matters: Monitoring and Measurement in the New Media …painepublishing.com/wp-content/uploads/2015/04/PRSA-NCC... · 2015-05-07 · Social Media isn’t media, it’s](https://reader033.fdocuments.us/reader033/viewer/2022050607/5fae60c192b42d396277cbf2/html5/thumbnails/56.jpg)
Thank You!
For more information on measurement, read my blog:
http://www.painepublishing.com/blog
For a copy of this presentation or to subscribe to our newsletter, give me
your card or email me at [email protected]
Follow me on Twitter: @queenofmetrics
Friend me on Facebook: Katie Paine
Or call me at 1-603-682-0735
57