The Mid-Term Assessment as a tool to reorient the communication plan

Judit SZŰCS8 December 2011, Brussels

The goal of the evaluation is- to sum up and assess at mid-term the results of communication related to the implementation of OPs 2007-2013

- to present how the communication activities have contributed to raising awareness of the OPs and the role that

the EU plays in the developments;

- to contribute to the further development of communication and to make preparations for the 2014-2020 period.

The final results and recommendations will be presented - in the Evaluation Report and - in the Annual Implementation Reports 2011 of the OPs.

The evaluation is conducted by an external contractor, TNS Hoffmann under the commission of NDA.

Goal and schedule of the evaluation

In Hungary, communication is centralised:

1. NSRF-level communication is provided by the NDA Communication Department (ERDF, ESF, Cohesion Fund),

2. OP-level communication (intended for the beneficiaries) is managed by intermediate bodies in cooperation with the Communication Department of NDA and with the professional support of the MA,

3. project-level communication is provided by the beneficiaries to popularise the accomplished projects, according to the Image Manual developed by the Communication Department and the provisions applicable to the information supply obligations of beneficiaries.

The methodology for the evaluation has been designed accordingly:- quantitative data and facts (e.g. the number of advertisements, the number of workshops and events, etc.), - qualitative results (the evaluation of efficiency and cost- effectiveness, in-depth interviews, focus groups of beneficiaries, the evaluation of the results of surveys conducted among the general public and potential beneficiaries).

Evaluation methodology

The most important communication campaigns

(2007-2010)

Phase 2:Get the best out of yourself!

Phase 3:It is good to be

proud!

Phase 1: You are a part of it!

1. Image campaign 2. Crisis management campaign

3. Local campaign

Involvement – Crisis management – Local communication

In line with the EU trendsHierarchically structured campaigns allowing

flexible reaction to the changes of their environment

Media utilisation habits of (potential) beneficiaries

”How often do you……”

„

% rate to total population (TNS data)Lower:• TV (appr. 85%)

Higher:• press (appr. 40%)• internet (appr. 50%)• radio (appr. 60%)

Contribution of the respective media to be well-informed

”In your opinion, to what extent does …… contribute to be well-informed?”

The ratio of press users is lower, but the contribution of this medium to be well-informed is outstanding

Preference: other than the population average

• radio (population: well below TV)• TV (ranks 1st for the population)

Beside the mass media, professional days play a most important role

Opinions on the communication media based on avoidance of commercials

”How much do ads/commercials at the following places disturb you?”

• The message relay method is a key issue (ads reduce efficiency).

•Press: despite the decrease in use, beneficiaries consider it important due to authenticity and low avoidance of commercials.

• Media co-operations – diverse programmes on the most watched channels instead of disturbing TV ads.

• Internet – based on international best practices, instead of classical ads, it is worth shifting towards other (creative) solutions, e.g. social media.

Average (1 to 10)

potential loser winner

Interrupting TV programme 8.53 8.27 8.52

In between TV programmes 5.43 5.05 6.10

In the radio4.74 4.82 5.01

On the Internet5.6 5.54 5.64

In the press2.69 2.94 3.14

On fliers3.97 4.03 3.91

On billboards2.78 2.95 3.21

Put in the mailbox 6.19 6.45 6.13

In shops2.82 3.28 2.86

At events2.91 3.20 3.03

Most important channels for application-writing”Which of the items in the list below was the most useful, most efficient source for you?”

• The role and effect of the mass media is significant in the first phase – basic information, ”appetizing”, emergence of the needs.

• Specific action, participation in tenders or the application-writing process itself: depends on other, special, communication devices and the weighing of further pieces of information.

% loser winner

Press ads 0 1

Online news pages 8 3

Internet-based ads 8 6

Website of the National Development Agency (www.nfu.hu)

44 43

Website of the intermediary body 4 5

Events, workshops 2 2

Application-writer company 24 23

Fliers, brochures 1 0

Colleagues, friends, relatives 2 2

Billboard\public area commercials 1 0

NDA information line and customer service

2 4

Other 5 12

The website

• Awareness of the Applicant Information Surface is definitely high.

• The surface is not only well-known, but it is also considered efficient.

% Loser Winner

yes 86 92

no 14 8

% Loser Winner

1-not efficient at all 1 0

2 2 03 11 264 41 43

5-fully efficient 43 31

Does not know/Does not answer

1 0

”Do you know, have you heard about the Applicant Information surface on the website of the National Development Agency?”

”To what extent do you consider the NDA website and the Applicant Information Surface an efficient source of information for learning about the application opportunities?”

The most important results and indicators

Indicator

Target value (2013)

Base value (2007)

Population research May 2011

Potential beneficiari

esSept. 2011

Loser applicantsSept. 2011

Winner applicantsSept. 2011

Awareness of the general goals of the developments financed by the EU

70% 31% 61% 55% 55% 60%

Support for EU membership in Hungary 70% 41% 51% 34% 47% 38%

Support for EU cohesion policy (how many % regards developments as a positive thing)

70% 5% 39% 21% 31% 27%

The proportion of those who think that the support provided by the EU helps the country close the gap

60% n.a. 67% 34% 54% 56%

The most important results and indicators

IndicatorTarget value (2013)

Base value (2007)

Population research May 2011

Potential beneficiari

esSept. 2011

Loser applicantsSept. 2011

Winner applicantsSept. 2011

The proportion of those who think that the tendering system works in accordance with the regulations in the population aged 18+

40% 25% 45% 30% 52% 59%

The proportion of those who would submit a tender (or who submitted one) for EU funding in the population aged 18+

60% 10% 47% 63% 86% 88%

The proportion of those beneficiaries who think that the funding institutions are user-friendly among the beneficiaries

40% 39% 43% 15% 30% 23%

The most important results and indicators

Indicator

Target value (2013)

Base value (2007

)

Population

research May 2011

Potential beneficiari

esSept. 2011

Loser applicant

sSept. 2011

Winner applican

tsSept. 2011

% rate of population aged 18+ knowing of projects co-funded by the European Union

65% 46% 76% 75% 95% 99%

% rate of population aged 15+ knowing EU funded projects in the vicinity of their place of residence

80% 40% 93% 66% 95% 90%

% rate of social partners expressing their opinion to the involved ones

70% n.a. n.a. n.a. 46% 54%

Awareness of the OPs, in %

OP potential beneficiares

loser applicants

winner applicants

OP potential beneficiari

es

loser applicants

winner applicants

SROP 38 53 63 CTOP 67 80 82

SGPOP 54 85 95 EEOP 70 83 91

STOP 70 79 94 CHOP 41 67 89

NGPOP 73 85 88 TOP 51 64 79

EAOP 48 41 65 WTOP 75 88 86

NHOP 78 91 94 SROP 64 89 88

EDOP 68 84 88 SIOP 48 70 79

I. Conclusions, recommendations for NSRF-level communication, 1.

Observation Conclusion Recommendation

The indicator scores of the beneficiary target group are significantly different from those of the general public. The primary research has revealed that their communication habits do not always coincide with the general public either.

The information sources used by the beneficiaries are different from those of the general public. The communication mix that is used for the general public does not necessarily achieve the desired results in their case.

Adopting a communication strategy that is more segmented and takes into account the habits and needs of the beneficiary target group, while achieving better separation of the messages intended for the general public and the beneficiaries. Or: use of different channels is recommended, as well as the reconsideration of the synergies of the mainstream and the other media types.

I. Conclusions, recommendations for NSRF-level communication, 2

Observation Conclusion Recommendation

Except for TV, the beneficiary target group uses every medium type more intensively than the population. The avoidance of commercials is similar to that of the total population, and it is especially significant for TV and for the Internet.The mainstream media plays a major role in Level 1 information transfer; concrete action is affected by other channels.

The different communication channels carry different weights and roles. Online communication is quite strong and its role is expected to increase.Hierarchically structured (ATL and BTL) campaigns are needed.

Instead of classical ads, further development of media co-operations. TV: various short films. Internet: use of NDA website and, based of foreign experience, of the social media is recommended.Hierarchically structured creative concepts shall be discernible also in the hierarchically structured ATL and BTL campaigns

II. Conclusions, recommendations for OP-level communication, 1 (sectoral OPs)

Observation Conclusion Recommendation

The volume of press consumption is decreasing, but ads are less irritating than in the case of other media channels.

It is an important media channel that is losing significance. It can be used to increase effectiveness, due to its credibility.

If more sophisticated segmentation is achieved relevant contents should be published in the papers preferred by the beneficiaries instead of ads.

Information days and workshops contribute significantly (71%) to being well-informed.

Alongside the classic media channels it is also important to organise workshops that enable personal feedback.

In some OPs there were relatively fewer workshops, it is recommended to increase their number.

A high proportion of the beneficiaries use the internet on a daily basis (90%).

It plays a particularly important role in the media consumption of the target group. This should be taken into account in the communication too.

Increasing online presence, but due to high advertising rejection, primarily relevant contents, search marketing and social media (Facebook, Twitter) platforms are recommended.

II. Conclusions, recommendations for OP-

level communication, 2 (regional OPs)

Observation Conclusion Recommendation

Those who are familiar with ROPs in the western and central regions enjoy 90%+ internet penetration.

Online communication is even more important to them and every form of online communication can be used more intensively in their case.

Increasing online presence in the western and central regions of the country in the form of ads (to increase awareness, as it reaches a high proportion of people) and contents (to increase involvement).

In the central region (CHOP) a lower than average proportion read newspapers (49%).

Due to the heavy significance of Budapest there are no strong local papers, so the consumption printed press is also lower.

Re-considering the role of the press in the central region and decreasing its relative significance, segmentation.

In the eastern region information days and workshops contribute to being well-informed to a lesser extent compared to the total sample (68%).

Although these people are a less active audience the informational days and workshops cannot be skipped.

Increasing the relevance of information days and workshops, fine-tuning their content especially in the eastern region.

II. Conclusions, recommendations for OP-level communication, 3 (regional OPs)

Observation Conclusion Recommendation

In East Hungary the role of television is more significant. More people watch TV on a daily basis (86%), ads are less irritating and TV plays a more important role in keeping them well-informed (38%).

Classic, TV-based communication is more important. It could become even more important due to the digital switch (more channels). TV should play a more emphatic role in the media mix.

Increasing TV presence in East Hungary in the form of ads (to increase awareness, as it reaches a high proportion of people) and relevant contents (to increase involvement).

The intensity of press consumption in West Hungary is above the average (83%). According to 60% the press contributes to keeping them well-informed.

Due to the popular county papers the press plays a particularly important role in this region, which should be taken into account when placing ads and content.

Increasing press presence in the western part of the country in the form of ads (to increase awareness, as it reaches a high proportion of people) and contents (to increase involvement) with a better segmentation of papers.

Radio consumption is higher than the average in West and East Hungary (over 80%).

An important media channel that could primarily increase the frequency of being exposed to the communication.

In order to further increase awareness it is recommended to increase radio presence at popular, local radio channels.

III. Conclusions, recommendations for project-level communication

Observation Conclusion Recommendation

The beneficiaries identified as negative circumstances high fluctuation, the difficult and cumbersome set of requirements, and administration, lack of dialogue between the organizations and the impersonal nature of the system.

Internal organizational conditions and processes have a major influence on the operation and efficiency of communication, their improvement and development would in all probability improve also the effectiveness of communication.

Review of internal organizational structure and processes; if necessary, their transformation, fine-tuning based on research data.Multi-level communication between the beneficiaries and the institution system to promote execution.

The communication obligation reinforces the strategy and institutional communication of the beneficiaries, but the requirements are not always adequate.

The communication obligations are not always realistic.

Specification of more differentiated communication obligations matching the nature of the organization (e.g. a school should not be required to have press conferences).

Training needs related to application and project implementation.

The degree of efficiency of communication could be developed through the optimization of the use of inner resources.

Resources earmarked for training, which would also boost the efficiency of communication.

Fine-tuning of the NSRF Communication Strategy in view of

the results of the mid-term assessment

Fine-tuning of the NSRF Communication Strategy

Reasons for the change:

• The NSRF communication strategy 2007-2008 expected a prospering economy

• Economic outlook and objectives of economic policy have changed due to the crisis

• Hungary launched a comprehensive economic development programme in 2010, known as New Széchenyi Plan with the following aims:

– Dynamic increase of employment,– Maintaining financial stability,– Creating the conditions for economic

growth,– Enhancing Hungary’s

competitiveness.

• New actors have appeared in the institutional system of economic development; division of labour has changed among organizations.

Purpose of the change:

• Re-segmentation of the target group, using a different tool system for each OP

• Transformation of the system of information provision to the population

• Cost-efficient system management

• Transparent and unified action plans overarching the entire development policy for the last two years, involving IBs

• NSZP – prioritizing EU2020 objectives

• Adjusting communication with Beneficiaries to OPs and coordinating it with the institutional communications

– Comprehensive change based on the results of the mid-term assessment in terms of target groups and communication tools

– The supervision of the impact indicators adopted in 2008, the inclusion of new target indicators suitable for the new economic situation

– Rescheduling timing based on the results of the mid-term assessment

– Drafting and integration of annual communication action plans made with the involvement of IBs which promote the implementation of the modified communication strategy

Areas of modification of the Communication Strategy