Researching, measuring, and communicating PR impact and ROI · I am a data and analytics...
Transcript of Researching, measuring, and communicating PR impact and ROI · I am a data and analytics...
Researching, measuring, and communicating PR impact and ROI
Orion Stavre
Associate Director of Insight and Measurement Worcester Polytechnic Institute
@inquisitiv_mind
Worcester Polytechnic Institute
Who I am...
2
Worcester Polytechnic Institute
What I do..
I am a data and analytics professional who works with Marketing, Media Relations and PR professionals to help them engrain the use of data, measurement, and actionable insights as part of their decision-making process.
➢An integrated measurement structure across all marketing and communication channels (earned, paid, shared, and owned) used to better explain their relationship to brand perception, reputation, or revenue.
3
Worcester Polytechnic Institute4
Worcester Polytechnic Institute
World’s current Data Creation levels
http://www.vcloudnews.com/every-day-big-data-statistics-2-5-quintillion-bytes-of-data-created-daily/
Worcester Polytechnic Institute
Everyday we create and use a large amount of data
http://www.northeastern.edu/levelblog/2016/05/13/how-much-data-produced-every-day/
Data produced Yesterday is equivalent to:
Worcester Polytechnic Institute7
Worcester Polytechnic Institute
Data processing capability has increased
http://www.vcloudnews.com/every-day-big-data-statistics-2-5-quintillion-bytes-of-data-created-daily/
8
Worcester Polytechnic Institute
http://www.digitalnewsreport.org/publications/2016/editorial-analytics-2016/
9 Source: https://chiefmartec.com/2017/05/marketing-techniology-landscape-supergraphic-2017/
Markom Technology and Data Landscape
Worcester Polytechnic Institute
Data is used for decision making
https://www.pwc.com/us/en/analytics/publications/view-of-decisions.html10
Worcester Polytechnic Institute
And the demand keeps rising…
https://www.pwc.com/us/en/analytics/publications/view-of-decisions.html
11
How about PR?Is PR using data?
12
Worcester Polytechnic Institute
Is PR embracing or resisting the use of data?
https://mashable.com/2014/12/05/public-relations-industry/#rkSjg629AOqw13
“change becomes more manageable when you understand what's going on, how long it will last, and whether you are on course to conquer it”
Worcester Polytechnic Institute
Why PR may lag behind in the use of data?
Barriers:
1. Lack of Time, Skills/Training, Budgets
2. Need for “Better Data”
3. Limited demand
4. Responsibility overlap
5. Adapting the data-driven mindset
14http://www.themeasurementstandard.com/2017/08/5-common-barriers-pr-measurement-adoption/
http://www.prnewsonline.com/pr-needs-transform-mindset-data-focus-growth-analysis/
Worcester Polytechnic Institute
Lack of time/skill/training
- PR professionals are busy and may not have time to think about measuring their efforts
- Data collection requires tools and skills that PR professionals don't necessarily have time or budget to acquire
15
Worcester Polytechnic Institute
Need for Better Data
• Data used in evaluating core aspects of PR has become and continues to grow increasingly disconnected from the realities of the media environment
• There is a need for broader availability and adoption of content-level and audience behavioral data in the PR community.
• Guidelines for better metric choice, usage and mapping to goals should be developed
16
Worcester Polytechnic Institute
Limited demand/supply of data
• PR campaigns are often developed to “increase awareness,” but rarely there is any desire or budget dedicated to measure awareness levels before and after the campaigns.
• Most of the measurement is built around placement counts, AVE’s, and vanity metrics.
• Tools used for measurement and tracking may not be readily available or affordable.
17
Worcester Polytechnic Institute
Responsibility Overlap/Lack of Integration
• With the rise of the PESO model and last-click attribution, separation between PR, advertising, and marketing has been greatly reduced.
• Within an integrated campaign, who gets the credit for the resulting outcomes?
• What is the level of collaboration and measurement integration?
18
Worcester Polytechnic Institute
Adapting to data-driven Mindset
• For so long measurement was considered a performance tool in PR, therefore PR professionals are lagging to adopt as a planning and decision-informing tool.
• A shift in mindset is needed to start making measurement and analytics less of a reporting tool and more of a planning and opportunity identification tool.
19
Worcester Polytechnic Institute20
Worcester Polytechnic Institute
Data is YOUR Friend
http://www.prnewsonline.com/pr-needs-transform-mindset-data-focus-growth-analysis/21
Worcester Polytechnic Institute
Why measure PR efforts?
• Validate your PR efforts for achieving your organization’s objectives
• Make adjustments and corrections
• Plan for the future
• Eliminate confusion and misinterpretation
For more info: http://www.axiapr.com/blog/why-measure-pr-efforts22
Worcester Polytechnic Institute
Questions you can answer with data…
• Are you keeping your competition out of the media?
• Is your PR team forming valuable relationships?
• Is your team building your brand?
• Are you reaching your desired target market?
• Do you have the right endorsements?
For more info: https://www.forbes.com/sites/ilyapozin/2014/05/29/5-measurements-for-pr-roi/#542ef57e77d123
Worcester Polytechnic Institute
PR is already embracing the use of data
• PR Measurement Conferences
• Barcelona Principles
• IPR’s Measurement Commission
• The Coalition for Public Relations Research Standards
• IPR’s Organizational Communication Research Center
• IPR’s Social Media Research Center
• Barcelona Principles 2.0
24
How to understand the Barcelona Principles 2.0as a measurement implementation guide
25
Worcester Polytechnic Institute
Barcelona Principles
• The Barcelona Principles is a set of seven principles that provide the first overarching framework for effective public relations and communication measurement.
• The principles were originally adopted by about 200 delegates from over 30 countries at the 2nd Annual European Summit on Measurement in Barcelona in 2010, convened by the International Association for Measurement and Evaluation of Communication (AMEC)
• They were updated in 2015 by the same group.
26 Source : https://instituteforpr.org/barcelona-principles-2-0-updated-2015/
Worcester Polytechnic Institute
Barcelona Principles
The original Barcelona Principles were established in Barcelona in 2010.
In 2015 they were updated to better reflect the needs of the same PR professionals who were implementing those in their daily work.
27 Source : https://instituteforpr.org/barcelona-principles-2-0-updated-2015/
Worcester Polytechnic Institute
Barcelona Principles 2.0
PRINCIPLE 1: Goal Setting and Measurement are Fundamental to Communication and Public Relations
PRINCIPLE 2: Measuring Communication Outcomes is Recommended Versus Only Measuring Outputs
PRINCIPLE 3: The Effect on Organizational Performance Can and Should Be Measured Where Possible
PRINCIPLE 4: Measurement and Evaluation Require Both Qualitative and Quantitative Methods
PRINCIPLE 5: AVEs are Not the Value of Communication
PRINCIPLE 6: Social Media Can and Should be Measured Consistently with Other Media Channels
PRINCIPLE 7: Measurement and Evaluation Should be Transparent, Consistent and Valid
28 Source : https://instituteforpr.org/barcelona-principles-2-0-updated-2015/
Worcester Polytechnic Institute29
Worcester Polytechnic Institute
Creating Goals
• Goals are critical for every PR/Communication plan
• Goals can be quantitative or qualitative and should be aligned with measurable KPI’s
• Goals, measurement, and evaluation should be holistic in their approach
30 Source : https://instituteforpr.org/barcelona-principles-2-0-updated-2015/
Worcester Polytechnic Institute
Barcelona Principles 2.0
PRINCIPLE 1: Goal Setting and Measurement are Fundamental to Communication and Public Relations
PRINCIPLE 2: Measuring Communication Outcomes is Recommended Versus Only Measuring Outputs
PRINCIPLE 3: The Effect on Organizational Performance Can and Should Be Measured Where Possible
PRINCIPLE 4: Measurement and Evaluation Require Both Qualitative and Quantitative Methods
PRINCIPLE 5: AVEs are Not the Value of Communication
PRINCIPLE 6: Social Media Can and Should be Measured Consistently with Other Media Channels
PRINCIPLE 7: Measurement and Evaluation Should be Transparent, Consistent and Valid
31 Source : https://instituteforpr.org/barcelona-principles-2-0-updated-2015/
Worcester Polytechnic Institute
Measuring outputs and outcomes
• Outputs are helpful to PR professionals but often misunderstood/ignored by C-suite executives (especially when presented in isolation).
• Processes for measuring the effect of PR efforts are notably enhanced when the relationship between efforts - output - outcomes is established (tailored based on the goals/objectives of the communication plan.)
• Primary research and statistical quantitative techniques applied according to research best practices could be very helpful in measuring outcomes.
32
Worcester Polytechnic Institute
PR Outcomes vs. Output
Outputs Outcomes
Examples of outcomes:
• shifts in awareness/behavior
• advocacy related to purchase
• change in brand perception/affinity
• corporate reputation/employee
engagement,
• shift in public policy/investment
decisions
33
Examples of outputs:
• Press releases
• AVE’s
• Media mentions
• Media hits
• Social likes
Worcester Polytechnic Institute
Barcelona Principles 2.0
PRINCIPLE 1: Goal Setting and Measurement are Fundamental to Communication and Public Relations
PRINCIPLE 2: Measuring Communication Outcomes is Recommended Versus Only Measuring Outputs
PRINCIPLE 3: The Effect on Organizational Performance Can and Should Be Measured Where Possible
PRINCIPLE 4: Measurement and Evaluation Require Both Qualitative and Quantitative Methods
PRINCIPLE 5: AVEs are Not the Value of Communication
PRINCIPLE 6: Social Media Can and Should be Measured Consistently with Other Media Channels
PRINCIPLE 7: Measurement and Evaluation Should be Transparent, Consistent and Valid
34 Source : https://instituteforpr.org/barcelona-principles-2-0-updated-2015/
Worcester Polytechnic Institute
Can PR efforts be linked to Organizational Performance KPI’s?
How do we understand and measure the link between Communications outputs and Organizational outcomes?
• When measuring results from communication for an organization, integrated marketing and communications models should be used. Practitioners need to understand the value and implications of integrated marketing and communication models for accurate evaluation of each channel including PR.
• Inputs that convey quantity and quality of communication should be provided and used to identify their effect on organizational outputs, while accounting for other variables. There exists an ongoing need to develop PR measures that can provide reliable input into integrated marketing and communication models.
• Demand for models to evaluate the impact on target audiences of PR and communication activities in an integrated environment is significant and increasing.
35
Worcester Polytechnic Institute
Integrated Marketing Mix Modeling
Marketing Mix Modeling is a perfect example of an integrated model that can be used to better understand the value of PR on organizational outcomes.
➢ Marketing mix modeling is a technique that uses market-influencing metrics from disparate sources (internal or external, controllable or not) and applies advanced statistical analysis to provide insights on how various inputs affect organizational outcomes.
➢ Typical questions answered by MMM:○ How much is each input adding to (detracting from) outcome?
○ What is the ROI/effectiveness of each input?
○ Which marketing tactics are contributing more?
○ Are there any synergistic effects across channels?
○ Should marketing budgets/communication efforts be reallocated?
36Learn more about MMM: https://en.wikipedia.org/wiki/Marketing_mix_modeling
Worcester Polytechnic Institute
Barcelona Principles 2.0
PRINCIPLE 1: Goal Setting and Measurement are Fundamental to Communication and Public Relations
PRINCIPLE 2: Measuring Communication Outcomes is Recommended Versus Only Measuring Outputs
PRINCIPLE 3: The Effect on Organizational Performance Can and Should Be Measured Where Possible
PRINCIPLE 4: Measurement and Evaluation Require Both Qualitative and Quantitative Methods
PRINCIPLE 5: AVEs are Not the Value of Communication
PRINCIPLE 6: Social Media Can and Should be Measured Consistently with Other Media Channels
PRINCIPLE 7: Measurement and Evaluation Should be Transparent, Consistent and Valid
37 Source : https://instituteforpr.org/barcelona-principles-2-0-updated-2015/
Worcester Polytechnic Institute
Qualitative & Quantitative Measurement
Tracking tools can do a good job in measuring quantitative change in outcomes, but qualitative nuances are needed to better understand the whole picture.
• Total mention counts and impressions could be meaningless when it comes to understanding PR performance. Most recommended is the use or Mentions and Impressions from your specific target audience, crossed with quality identifiers like:
─ Tone/Sentiment
─ Credibility/Relevance
─ Message Prominence
Note: The assumption that results of a Communication/PR campaign are always positive / successful should never be made . A good measurement approach / methodology should allow for negative or inefficient results.
38
Worcester Polytechnic Institute
Typical PR Data Inputs for MMM
Both quantity and quality nuances of PR outputs can be used as inputs in a Marketing Mix Model to understand their relation to sales.
39 Source: https://instituteforpr.org/wp-content/uploads/BG_SmithKetchum.pdf
Worcester Polytechnic Institute
Barcelona Principles 2.0
PRINCIPLE 1: Goal Setting and Measurement are Fundamental to Communication and Public Relations
PRINCIPLE 2: Measuring Communication Outcomes is Recommended Versus Only Measuring Outputs
PRINCIPLE 3: The Effect on Organizational Performance Can and Should Be Measured Where Possible
PRINCIPLE 4: Measurement and Evaluation Require Both Qualitative and Quantitative Methods
PRINCIPLE 5: AVEs Are Not the Value of Communication
PRINCIPLE 6: Social Media Can and Should be Measured Consistently with Other Media Channels
PRINCIPLE 7: Measurement and Evaluation Should be Transparent, Consistent and Valid
40 Source : https://instituteforpr.org/barcelona-principles-2-0-updated-2015/
Worcester Polytechnic Institute
AVE’s
• Advertising Value Equivalents (AVEs) are not representative of, and do
not measure the value of PR.
• This metric should not be used as a value for communication, media
content, earned media, public relations, etc.
41
Worcester Polytechnic Institute
Barcelona Principles 2.0
PRINCIPLE 1: Goal Setting and Measurement are Fundamental to Communication and Public Relations
PRINCIPLE 2: Measuring Communication Outcomes is Recommended Versus Only Measuring Outputs
PRINCIPLE 3: The Effect on Organizational Performance Can and Should Be Measured Where Possible
PRINCIPLE 4: Measurement and Evaluation Require Both Qualitative and Quantitative Methods
PRINCIPLE 5: AVEs are Not the Value of Communication
PRINCIPLE 6: Social Media Can and Should be Measured Consistently with Other Media Channels
PRINCIPLE 7: Measurement and Evaluation Should be Transparent, Consistent and Valid
42 Source : https://instituteforpr.org/barcelona-principles-2-0-updated-2015/
Worcester Polytechnic Institute
Social Media is important & easily measured
According to a 2015 finding by Pew Research Center, almost two-thirds of American adults (65%) use social networking sites. This rapid popularity and adoption of Social Media, has made it a very important channel for PR professionals. Some of the ways that social media is useful for PR professionals:
• Integrate and amplify PR’s message and measurement
• Co-creation with influencers and immediate user feedback/reaction
• Real-time monitoring and risk mitigation
• Targeted and segmented messaging by audiences of interest
• Gathering of journalistic insight
43
Worcester Polytechnic Institute
Social Media Monitoring Tools in 2017
44 Source: https://chiefmartec.com/2017/05/marketing-techniology-landscape-supergraphic-2017/
Worcester Polytechnic Institute
Barcelona Principles 2.0
PRINCIPLE 1: Goal Setting and Measurement are Fundamental to Communication and Public Relations
PRINCIPLE 2: Measuring Communication Outcomes is Recommended Versus Only Measuring Outputs
PRINCIPLE 3: The Effect on Organizational Performance Can and Should Be Measured Where Possible
PRINCIPLE 4: Measurement and Evaluation Require Both Qualitative and Quantitative Methods
PRINCIPLE 5: AVEs are Not the Value of Communication
PRINCIPLE 6: Social Media Can and Should be Measured Consistently with Other Media Channels
PRINCIPLE 7: Measurement and Evaluation Should Be Transparent, Consistent and Valid
45 Source : https://instituteforpr.org/barcelona-principles-2-0-updated-2015/
Worcester Polytechnic Institute
Research Transparency & Validity is Important
• Valid research methods and trustworthy data sources should be
used for analysis and measurement
• Basic research guidelines to be followed:
─ Reference sources of content
─ Identify data collection criteria
─ Describe audience selection methodology
─ Define analysis approach and methodology
─ Defend choice of statistical methodology
─ Reveal known bias possibilities
46
Building the Measurement Process
47
Worcester Polytechnic Institute
PR Technology/Measurement Stack 2017
48 Source: https://chiefmartec.com/2017/05/marketing-techniology-landscape-supergraphic-2017/
Worcester Polytechnic Institute
Think Holistically around Marcom Integration
“You may measure individually, but always analyze holistically”
Why is integration important?
• Seamless Customer Experience
• Creative/Operational/Measurement Consistency
• Collaboration vs. Competition
• Cost Savings
• Efficiency Gains
• Acceptance of results
49
Worcester Polytechnic Institute
Integration based on PESO Model
https://mashable.com/2014/12/05/public-relations-industry/#stEEJ2dwQOqW50
Worcester Polytechnic Institute
Building Measurement Process
1. Set Baseline/Audit - Where are we now?
2. Create Goals/KPI’s - Where do we need to be?
3. Strategize and Plan - How do we get there?
4. Measure and Analytics - Are we on course?
5. Forecast/Course Correct - Should we keep/change approach?
6. Results and Evaluation - How did we do?
51
Worcester Polytechnic Institute
How do we decide where we want to go?
“You need to learn where you are and how you got here, then decide where you want to be…”
52
Worcester Polytechnic Institute
Using measurement to inform goal-setting
The Jim Macnamara model of measurement and evaluation
Source : https://www.prweek.com/article/1303749/emerging-models-pr-measurement#tEoJAoUC2WESSZor.9953
Worcester Polytechnic Institute
Where are we Now?
• Does our target audience know us?
• What do they know? What do they think of us?
• How do they perceive us compared to competition?
• How are our PR efforts currently affecting outcomes?
• What has been the success of recent process?(if available)
54
Worcester Polytechnic Institute
Creating Goals
• Goals are critical for every PR/Communication plan
• Goals can be quantitative or qualitative and should be aligned with measurable KPIs
• Goals, measurement and evaluation should be holistic in their approach
• Goals Should be SMART
55 Source : https://instituteforpr.org/barcelona-principles-2-0-updated-2015/
Worcester Polytechnic Institute
Goals should be SMART
Note: For more info on a Guidelines for Setting Measurable Public Relations Objectives: An Update http://painepublishing.com/wp-content/uploads/2014/04/Setting_PR_Objectives-1.pdf
Source:https://www.mindtools.com/pages/article/smart-goals.htm56
Worcester Polytechnic Institute
Deciding on KPIs - No Shortage of Metrics
57 https://metricsman.wordpress.com/
Worcester Polytechnic Institute
Deciding on KPIs
When deciding on KPIs, please remember that Measurement and
Management go together. What gets measured gets done.
• What is most important for your business/organization?
• What metrics better match/represent your strategic goals?
• Are they comprehensive? / Do they holistically represent your
goals?
• Can you realistically measure?
58
Worcester Polytechnic Institute
Building the Communication Plan
Communication Plans can vary:
• Strategic vs Opportunistic Mix
• Risk Tolerance
• Alignment to Organizational Strategic Plan
• Audience Specific vs Broad Range
• Omni-Channel vs Channel Specific
• Integrated with Marketing vs Separated Siloes
59
Worcester Polytechnic Institute
Measurement & Analytics - Are we on course?
Measurement should be:
• Holistic
• Granular
• Ongoing
• KPI specific
• Outcome bound
• Insightful/Actionable
• Accepted and used by all Marcom
60
Worcester Polytechnic Institute
Forecast and Course Correct
Use ongoing analytics to track:
• Is message perceived as intended?
• Is it being consumed by desired target market?
• What channels are performing better?
• Which approach provides highest synergistic effects?
─ Social-Digital-Print vs. Print-Digital-Social
• What outlets are most perceptive/enthusiastic?
• What (if anything) should we change to stay on course/outperform ?
61
Worcester Polytechnic Institute
Results and Performance Evaluation
Overall Performance Evaluation should:
• Be Accurate, Reliable and Transparent
• Be as Integrated as Possible Across Marcom
• Focus on Outcomes/Organizational Goals
• Inform Learning and Next Planning Cycle
• Celebrate Wins and Note Learnings
• Have Various Levels of Granularity
62
PR Measurement Examples/Dashboards
Example of Visibility Dashboards
63
Worcester Polytechnic Institute
PR Visibility Dashboard
64
Brand A - Share of Voice
Worcester Polytechnic Institute
PR Visibility Dashboard
65
Brand A - Share of Voice
Competitor 3
has biggest
share of
mentions
Worcester Polytechnic Institute
PR Visibility Dashboard
66
Brand A - Share of Voice
Brand A
dominated the
share of Online
Visibility
Worcester Polytechnic Institute
PR Visibility Dashboard
67
Brand A - Share of Voice
Competitor 2
mostly focused on
gaining share of
Print Reach
Worcester Polytechnic Institute
PR Visibility Dashboard
68
Brand A - Share of Voice
Message 1 had the
highest number of
positive mentions
Worcester Polytechnic Institute
PR Visibility Dashboard
69
Brand A - Share of Voice
Message 2
highest number
of people
reached
PR Measurement Examples/Dashboards
Example of Brand Awareness Measurement Report
70
Worcester Polytechnic Institute
Example of Awareness Tracking Results
71
Worcester Polytechnic Institute
Example of Awareness Tracking Results
72
Worcester Polytechnic Institute
Example of Awareness Comparison
73
Worcester Polytechnic Institute
What Topics/Characteristics need more support
74
PR Measurement Examples/Dashboards
Example of Outputs vs Outcomes Dashboard
75
Worcester Polytechnic Institute
PR campaigns outcomes
76
Worcester Polytechnic Institute
PR campaigns outcomes
77
Worcester Polytechnic Institute
PR campaigns outcomes
78
PR Measurement Examples/Dashboards
Example of Marketing Mix Outcomes Report
79
Worcester Polytechnic Institute
Typical MMM output visualization
80 Source: https://www.instituteforpr.org/wp-content/uploads/OptMarketingMixModeling.pdf
Worcester Polytechnic Institute
Typical MMM output visualization
81 Source: https://www.instituteforpr.org/wp-content/uploads/OptMarketingMixModeling.pdf
Worcester Polytechnic Institute
Typical MMM output visualization
82 Source: https://www.instituteforpr.org/wp-content/uploads/OptMarketingMixModeling.pdf
Worcester Polytechnic Institute
Typical MMM output visualization
83 Source: https://www.instituteforpr.org/wp-content/uploads/OptMarketingMixModeling.pdf
Thank You!
Any Questions?
84