The theory: evaluating PR

Evaluating PR Effectiveness21 July 2011, London

Web: www.charitycomms.org.ukTwitter: @CharityComms

© echo

Evaluation PR & Measurement in ActionCharity Communication PresentationSandra Macleod, Group CEO, Echo ResearchE. [email protected]

21 July 2011

© echo |

Outline

1. About Echo

2. Setting PR Objectives

3. Measuring PR

4. Case Studies

5. Echoes of the future …

© echo | Page 3

About Echo

Evaluation PR & Measurement in Action

“ The soul never thinks without a

picture”Aristotle

© echo |

22-year track record in communication research -

media content analysis, stakeholder

evaluation and reputation measurement.

Now part of Ebiquity plc for integrated above

& below line ROI

and insight.

50 staff and 150 analysts

Offices in London, Paris,

New York, Singapore

Members of

UN Global Compact

All Echo research complies with the MRS Code of

Conduct and

ISO 9001:2008; Registered Expert Witness on

image and reputation. Winner of 89 industry

awards for best practice in communications

and reputation research

Echo has supported 500 world-class clients incl:

About Echo Research

© echo |

Publicity-based Research

Echo shows IT plays major role in corporate ‘green’ initiatives

© echo | Page 6

© echo |

Thought Leadership

• Echo Chambers

(London, NY, Paris),

Annual Echo Summit

• White Papers (Web 2.0,

Communications

Research Wiki)

• Global reports on CSR

and Trust (freely

available to download

from our website)

© echo | Page 7

Setting PR Objectives


© echo |

The Organisation

values, objectives,

strategies

Activities

messages sent by

company

Output

messages received

by audience

media analysis

market research

Outcome

Audience choices / behaviours

Advocacy & 3rd party

endorsement

Recruitment / retention

Brand equity

Regulation

Driver Analysis

Values and

expectations

Outtake / Impact

audience perceptions

market research

Making the Intangibles Tangible

© echo | Page 9

© echo |

What Are Your Objectives?

© echo | Page 10

© echo |

•Building awareness / visibility

•Fundraising

•Membership recruitment and retention

•Loyalty building

•Networking and connecting

•Thought Leadership

What Are Your Objectives?

© echo | Page 11

© echo |

SETTING OBJECTIVES

• Replicable

• Clearly defined

audiences/

stakeholders

• Objectives support

organisational goals

• Clearly stated

business objectives

• Measurable

business objectives

RESEARCH & PLANNING

• Links outputs to

outcomes

• Identifies audience

needs

• Relevant/salient to

target audiences

• Research-based

programme

• Quantifiable

evaluation metrics

• Identifies obstacles

& barriers to launch

• Executable

programme

OUTPUTS

(CONTENT)

• Truthful/believable

• Communicates

clearly

• Flawless execution

• Addresses

stakeholder needs

• Consistent

implementation

• Single minded

• Integrated with

other marketing

disciplines

RESULTS

• Objectives set in the

planning stage were

achieved

• Measurable impact

on stakeholder

relationships

• Measurable impact

on behaviours

and/or attitudes

• Demonstrable ROI,

including non-

financial indicators

such as increased

awareness

OUTTAKES

(INTERPRETATION)

• Creates clear

brand/corporate

differentiation

• Communicates

desired information

• Creates a positive

image

© echo | Page 12

Why Setting Objectives Is Key

© echo |

“Companies and industries with reputation problems are more likely to

incur the wrath of legislators, regulators and the public.

“To prepare for and respond to serious reputational threats, companies should

emphasize three priorities.

1.Develop a rich understanding of key stakeholders

2.Focus on the actions that matter most to stakeholders

3.Influence stakeholders through techniques that go beyond traditional PR

approaches, with an emphasis on two-way dialogue.”

Sheila Bonini, David Court, Alberto Marchi

The McKinsey Quarterly

New Priorities: Stakeholder Engagement

© echo | Page 13

© echo |

Partners

Communities

Potentialcustomers

Organisation

Employees

Shareholders

Suppliers

Customers

Potentialemployees

Regulators

Consumergroups

GovernmentAnalysts

Trade unions&

professionalbodies

NGOs

The Media

Reputation: for WHAT and among WHOM?

© echo | Page 14

© echo |

Reputation Drivers On Scorecards

Management Performance

Financial Performance

Innovation

Workplace

Environmental Performance

Ethics

Marketing

© echo |

• Reputation affects attitudes (like satisfaction, commitment,

and trust) and drives behaviour (like loyalty, advocacy, support)

– THESE ARE THE THINGS THAT MUST BE MEASURED

• Reputation exists in the minds of a myriad of stakeholders and ‘influencers’

and is based on a summation of influences and

experiences over time through:

• Direct experiences

• Employee advocacy

• Direct Communication

• Indirect Communication – know who is influencing whom

Measurement that matters

© echo | Page 16


Measuring PR

© echo 17

© echo |

• Customers*

• Opinion Formers

• Recruits*

• Investors

• Suppliers

• NGO

• Community .....etc

Key stakeholder groups:

*current/potential/lapsed

• Personnel *

• Media/web 2.0

Content

Analysis

Identity

Image

Influence

Echo’s Integrated Reputation Research

© echo | Page 18

© echo |

Triangulation (I³ or I-Cubed)

© echo | Page 19

© echo |

Influence of media messagesuptake of messages in the media compared to awareness among target audience

Audience awareness

© echo | Page 20

© echo |

Identity vs Image

© echo | Page 21

= Identity

= ImagePerception Gap

© echo |

Media content evaluation

Quality Control

Report

Production

Ongoing Updates

and Finetuning of

issues & messages

Report

Delivery &

Review

Analyse, Code

& Rate Articles

Develop

Media Lists

Develop Issues

& Messages

© echo

© echo | Page 22

© echo |

Image on the Internet vs Other Media

© echo | Page 25

© echo |

Penetration of campaign messagesPenetration and favourability of messages in UK national media, against uptake of message in target audience .

© echo | Page 26

© echo |

Visualisation: From Diagnosis to Solutions

• Baseline measures: how are we perceived?

• What do we do particularly well / poorly?

• How do we compare to competitors?

• What is the perceived ideal reputation for a group like ours?

• How should we position and differentiate ourselves?

• What does ‘success’ look like?

• What should we do or say differently?

• How should we communicate and engage with key stakeholders?

• How do we manage the risks and opportunities?

• How do we measure success?

Discover Address & Enhance

© echo | Page 27


Case Studies

© echo 28

© echo |

Case Study: Aligning Research

• Echo has provided communications research support to

WWF for many years

� Campaign evaluation – media content analysis to

evaluate effectiveness of PR and media outreach e.g.

Earth Hour – the global initiative to encourage people to

switch off their lights for one hour.

� Journalist surveys to track awareness of WWF and its

activities and to drive improvements to the media

relations function.

� Research audit of existing WWF brand research held

across the organisation. To help WWF to

� align research to focus on three brand pillars

� develop consistent brand measurement metrics

� identify gaps in research

� maximise the value of the research.

© echo |

Monthly Ratioby volume

Favourable85 (76%)

Unfavourable1 (1%)

Neutral18 (23%)

Positive Points� Rating rose from 53.5 to 56.0 due to a high proportion of favourable coverage (54% in February to 81% in March). This was very much boosted by Fundraising campaign reviews and feedback from recipients. � Volume remained virtually the same as last month.�Interviews scored very highly in March, with an article about WWF’s Deputy CEO Leslie Jones achieving an impressive 70.0 rating.� Once again the issue of the proposed Lignite mine was the focus of many articles in Ireland with Mr Malachy Campbell quoted in the Northern and South Derry Constitution, 1/3, “In Northern Ireland we have fabulous rivers and they should be protected. Lignite mining should not go ahead”. � Message penetration remained the same at 1: 1.4 articles.� Campaigns proved very popular with Big Swim and Global Applause gaining good volume and high ratings.

Negative Points7 Negative comment was minimal this month with just 1% of coverage rating under 50.0.

7 Criticism came from the West Highland Free Press, who commented on the agreement between the WWF and Lafarge –Redland.

7Proactive coverage was also down on February from 25% to 19%.

7 Negative messages were down this month although ‘an organisation with unclear and unrealistic objectives’ did appear once in relation to the conflicting Lafarge agreement.

Media Relations Scorecard

Total Reach : 14.4 million (- 34%)

Leading Descriptors• WWF (67 mentions, + 56%)• World Wildlife Fund (52 mentions, + 63%)• Worldwide Fund for Nature (8 mentions, -20%)

• Charity (7 mentions, + 20%)

Leading Bylinesby volume & rating

With more than one article

Byline Volume Rating

Bryan Millin - Dunstable Gazette 2 55.0

Charles Glover - The Daily Telegraph 2 52.5

Ian Johnston - Scotland on Sunday 2 55.0

Sue Howlett - Clacton Gazette 2 65.0

Monthly Trendby volume, favourability & rating

0

20

40

60

80

100

120

140

160

Nov-02 Dec-02 Jan-03 Feb-03 Mar-03

# ar

ticle

s

45

50

55

60

65

70

ratin

g

Favourable Unfavourable Neutral Rating

Note: Qualitative analysis started in Jan 03

Publication Volume Rating Tonbridge Courier 4 52.5

Scotland on Sunday 3 55.0

The Scotsman 3 51.6

Bridgnorth Journal 2 55.0

Clacton Gazette 2 57.5

Dawlish Post 2 52.5

Harwich & Manningtree Standard 2 62.5

Rutland & Stamford Mercury 2 57.5

Teignmouth News 2 55.0

The Daily Telegraph 2 52.5

The Times 2 50.0

Times & Citizen 2 60.0

Woman's Weekly 2 57.5

Leading Mediaby volume & rating

With more than one article

© echo |

The latest data on stakeholder perceptions highlights inroads made with targets in Government

Perceptions and Supporter Perspective showing a continuing improvement on influence and action.

Of note, though, is the latest ‘Business & Industry’ survey indicates that overall perceptions are down

on the 2006 figures. Awareness is still low, and the ‘dip’ in perceptions of influence further

investigation.

Supporters Business & Industry

Reputation Measurement Scorecard (illustration only)

Government

© echo |

Case Study: Influencing the Media Agenda

• PneumoADIP was formed by the Global Alliance of Vaccines and

Immunisation (GAVI) to speed up the provision of vaccines for

pneumococcal diseases to the world’s poorest countries.

• Initial research showed that despite killing more children than AIDS,

malaria and measles combined, there was a significant lack of

awareness of pneumococcal disease among health policymakers

and in the media.

• Echo helped PneumoADIP evaluate its progress in moving

pneumococcal disease up the media agenda.

• In addition, the findings were presented to an influential audience

of pneumococcal disease experts at two pneumococcal disease

congresses.

Gold AMEC Award winner -

“easy to read, good analysis

and good coverage” AMEC

Judge

© echo |

Case Study: Media Landscape Analysis

• Detailed analysis of the media landscape in 15 countries in the

East Asia and Pacific region (including North Korea, Papua New

Guinea, Myanmar and Timor Leste).

• To support UNICEF in its communication activities and media

outreach on emerging infectious diseases.

• Desk research plus over 50 in-depth interviews with media

experts.

• Identified the most influential medium and media outlets for

reaching different target audiences in each country.

• Also included information on media freedom and legislative,

cultural and historical factors together with new media trends.

© echo |

Case Study: Motivating Volunteers

• To identify current and potential volunteer motivations and to

uncover any disconnect between RNLI volunteers and staff to

ensure a cohesive organisation focused on the same goals.

• Echo applied its I3 (i-cubed) research approach:

� IDENTITY - how the RNLI views itself - depth interviews with

senior executives, focus groups with staff and a review of RNLI

internal and external literature

� IMAGE - how key audiences view the RNLI - quantitative

research with existing and potential volunteers, staff and

general public

� INFLUENCE - how the media portrays the RNLI - media

content analysis benchmarked against other national charities.

• “The findings and recommendations will inform the future direction

of the RNLI’s relationship with volunteers for many years to come,

but there have been a number of ‘quick wins’ enabling us to show

volunteers that we have both listened to them and acted upon their

concerns.” Ginette Tessier, RNLI Market Research and

Development Manager

© echo |

Case Study: Advocacy

• To review reaction to RSPB advocacy delivery among

Whitehall and Westminster decision makers, with a view

to modifying or adapting its approach, as necessary, to

meet its objectives.

• Echo conducted 28 in-depth interviews with elected and

non-elected government officials, special advisers and

policymakers.

• Interviewees were considered, frank and thoughtful in

their responses thus providing the RSPB with rich

feedback.

• The findings enabled the RSPB to determine the most

effective way of working with these stakeholders in the

future.

© echo |

Case Study: Evaluating Partnership Activities

Integrated study to evaluate 2010

International Year of Biodiversity

�Interviews with partner organisations

�Media content analysis

�Social media analysis

© echo |

Case Study: Event evaluation

• Evaluation of Holocaust

Memorial Day 2011.

• To evaluate the success of

the event and identify key

learnings for 2012.

• Qualitative and quantitative interviews with partners and supporters including teachers, local authorities and faith groups.

Echoes of the future …


© echo |

� Educate & involve Management

� Link communications to goals & objectives of the organisation

� Use measurement & evaluation in plans

(you-can’t-manage-it-if-you-can’t measure-it)

� Focus on stakeholders and their influencers/influences

� Quantify the financial contribution of your reputation

� Set benchmarks – OWN THIS SPACE !

� Be proactive

Guidelines for Success

© echo | Page 40

© echo |

Last words ….

“Everything that we do gets measured and my bonus and remuneration are tied to my scorecard, to my performance. I don't think you can justify your seat at the top table unless you are accountable, unless you can demonstrate that you are contributing to the business.”

“I don't think you should resist the notion that communications is a deliverable and that there should be a constant quest to measure it and evaluate it in similar terms to which other parts of the organisation are also subject.”

“You must challenge constantly. But do it with substance.”

© echo | Page 41

© echo |

“ This is an INSIGHT process. It makes tangible the intangibles and gives us a precise roadmap of what we need to focus on; it's about the business understanding thisand owning it. It takes the risk out of marketing and communications”

Board Director

© echo | Page 42

The theory: evaluating PR

Business

Transcript of The theory: evaluating PR