Sigried Caspar European Commission, DG Employment & Social Affairs Moderator.

Sigried Caspar

European Commission, DG Employment & Social Affairs

Moderator

Workshop A7: how to evaluate your communication activities 2014-2020

December 10, 2013 – Brussels

Céline Mas

Occurrence est certifiée ISO 9001 depuis 2004

"Half the money I spend on advertising is wasted;

the trouble is I don’t know which half."

John WanamakerInventor of mass

retailing in the United States

Occurrence - Workshop A7 - 10/12/2013

Key issue

> Communications are under pressure:- How much does it cost?

> View it as an investment and not as an expense

> Provide the resources to prove your effectiveness: 5% to evaluation.

> "How much does overlooked inefficiency cost?“ : if you can not assess it, you can not improve it!

€How much

does it contribute?

Whatdoes it contribute?

4Occurrence - Workshop A7 - 10/12/2013

Communication plan

ActionsA goals and resources « contract »

with company management

Assessment

Most of the time, evaluation is occasional

or partial.

It rarely shapes dialogue between the

communications team and the other decision

makers.

The missing link in a virtuous circle

A goals and resources « contract » with institution or company management

Reporting on the achievement of the targets,or on the progress and effectiveness

of the implementation


Ongoing

improvement

Behaving like any other function

> Communications must not exclude itself from Quality and Operating Excellence systems

> Communications is a job and a skill; it must include ongoing improvement procedures

The Deming Wheel

Quality management system


The 4 main benefits of an evaluation 1. PROMOTING

• Circulating results and performances to other departments/teams

• Achieving investment choices that are based on targets, and not on expenditures on resources

2. MANAGING

• Allocating resources in accordancewith performance indicators

• Identifying the most effective initiativesfor achieving your various objectives

3. SAVING TIME

• Prioritizing/Sorting initiatives by orderof effectiveness

• Concentrating your efforts and budget on the most effective initiatives

4. SHARING

• Gathering all the activity and effectiveness data

• Highlighting best practices

7

AND SPECIFIC TO THE PUBLIC SECTOR

• Giving evidence of a sound use of public money

• Reinforce citizen’s trust


2 main categories of performance (KPI) indicators

> A third, highly practical approach is possible: assessing the satisfaction of (internal) customers

> Defining performance thresholds for each indicator

e.g. Number of initiatives/tools,type of initiatives/tools, assessment of the content issued(Press releases, and internal communications),Ressources: Who? How much? How long?

Activityincluding

Resources(What?)

e.g. Memorization, Understanding, Buy-in, Incentive, Transformation, Satisfaction, Improving the brand's image, and satisfying internal customersAudience: How many people attended? How many Likes ? How many readers? …

Effectiveness

including the Audiences(For what

purpose?)


Process : global picture

1. Define 2. Count

3. Contact details Yes

- "ABC respects environment” - …

60%

Indicators

Target audience Communications objectives

Segmentation

x N[Sample survey]

Opinions/perceptions to share

-- "ABC respects environment”

Yes :No : Don’t know :

Targeted score :90 %

Currentscore:60 %

x 1[File qualification]

Gap between results and objectives

OK!!!

Surveys, research media analysis,

observations

Do you agree with the following assertion ?

Management• Allocate • Understand• Adjust• Maintain• Etc…

Com

mun

icat

ions

Das

hbo

ard

co

mm

unic

atio

ns p

lan


Advice

> Efficiency means producing the desired effects on the desired target audiences

> Therefore, you need to define the target audiences that you want to reach with which effect, prior to the initiative, and ideally to define the performance threshold

> As a starting point: KISS (Keep It Simple, Stupid)- Keep things simple at the beginning

> BUT assess them regularly

> And don't change the assessment system for each evaluation

> Do not restrict the evaluation to the activity, in order not to limit communications to initiatives and tools

> Design SMART (Specific, Measurable, Achievable, Realistic, and Timed) tools and goals for each initiative

> Share the results and the decisions they help to take in order to enhance the value added of evaluation


Thank you

> Contact : Céline Mas

> Partner & Research Director Occurrence

> [email protected]


Brussels, 9-10 December

Workshop B7: How to evaluate communication activities 2014-2020

UK Government perspective

December 9, 2013 – Brussels

Paul NjokuCabinet Office, UK


Agenda

A• Why evaluate (context)?• Barriers to evaluation

B

• How to go about it - PROOF guiding principles • 4 stage evaluation process - The Big IDIA• Main performance categories - KPI indicators

C• Top tips

16

Why evaluate?

A1


Context • Austere times

Need to make every € count View as an investment not an expense

• Media landscape & consumption patters

Evidence of what works and what does not Optimise use of scarce resources

• The role of communications

How it supports achievement of policy outcomes Business planning & activity prioritisation

A2

COMMUNICATION OBJECTIVERole that communication will play in achieving departmental

objective

DEPARTMENTAL OBJECTIVETo address specific issue

Sub-objective Sub-objective Sub-objectiveCommunication plan

Specific activities, channels, target

audiences

Communication plan Specific activities, channels, target

audiences

Communication plan Specific activities, channels, target

audiences

Overall communication strategy - how communication will achieve its objective

Policy development, policy delivery, reputation management

Strategic alignmentA3

1919

Barriers – stopping it happening

Insufficient time / resource / budget

Lack of SMART or unrealistic policy

objectives / targets

Difficulty accessing the right data / tools

Culture & entrenched behaviours

Gaps in evaluation standards & capability

A4


How to go about it

Five key principles

Pragmatic – best available within budget, not best ever

Realistic – prove what you can, acknowledge what you can’t

Open – record and share as much as possible

Objective – be honest & constructive about results, to inform future learning

Fully integrated – integral part of planning & delivery, not an add-on

P

R

O

O

F

B1


Evaluation stages –The Big IDIAB2

IdentifyThe scope of your project

DevelopYour evaluation plan

ImplementGather data to measure

performance

Analyse & reportPerformance against plan

1

2

3

4

Task 1: Define what you need to evaluate by asking:• What activity am I evaluating?• What do I know & what factors could affect the outcome?• What is my evaluation expected to achieve?Output: Summary of your proposed evaluation approach

Task 2: Define how you’ll measure success:• Set SMART objectives & defining your target audience• Map out how activity will work• Set performance metrics (KPIs) & agreeing targets Output: Draft evaluation plan

Task 3: Identify and gather evaluation data:• Make most of existing data• Gather additional data (research, feedback & proxies) • Review data gaps (more budget ?) manage expectations Output: Completed evaluation plan

Task 4: Assess the success of your activity:• Analyse effectiveness & provide insights for future• Demonstrating efficiency and value for money • Demonstrating role of communications in supporting

the achievement of policy objective (outcome)Output: Final evaluation report


Key performance indicator categories B3

Activity Effectiveness Result


1. Strategic alignment – Ensure activity objectives are SMART and supports policy delivery.

2. Business impact – Aim to measure true business impact (outcome) rather than for example, the perceived quality of specific channels.

3. PROOF the big IDIA – Try to adopt the suggested guiding principles and follow the big IDIA stages.

4. Continuous improvement – Ensure results drive appropriate actions and any learnings inform future activities.

5. Best practice – Be objective, share results and make evaluation an integral part of your communications planning process.

Top tips

24

Thank you!

Contact: Paul NjokuEmail : [email protected] link to guide: https://gcn.civilservice.gov.uk/guidance/evaluation/


Appendix

ProductsCustomer Service

InvestmentsEmployment

BrandingPublic Relations

MarketingSocialResponsibility

MEDIA (Traditional, Social)

Topic Experts, Leaders,Friends/Family

Perceptions & expectations

Supportive Behaviour

Results

What your departmentSays/Does

Direct Experience

What Others Say

How reputation is created

The attributes and dimensions have different meanings and importance for different stakeholders. Beneath the 7 dimensions, 32 attributes underpin the individual dimension themes. Different stakeholder groups typically have unique attributes that are found more important than others (reputation drivers).

Reputation Attributes

Reputation Dimensions

The seven dimensions specify at a more operational level, which aspects are most important for stakeholders’ perceptions and expectations – i.e. what’s driving a company’s reputation

Reputation StrengthA measure of the emotional connection.

Reputation has a positive/negative impact on support. An increase in reputation = an increase in support. Support (such as buying products and services, saying something positive, giving the benefit of doubt in times of crisis (etc.) leads to increased business results

Supportive Behaviour

Reputation drivers & dimensions

Sigried Caspar European Commission, DG Employment & Social Affairs Moderator.

Documents

Transcript of Sigried Caspar European Commission, DG Employment & Social Affairs Moderator.