Evaluating conferences and

events: new approaches and

initiatives

10th EES Biennial Conference

5 October 2012

Helsinki, Finland

Glenn O’Neil

oneil@owlre.com

www.owlre.com

The two case studies

• Both conferences undertake evaluation

• The conference evaluation differs in the

number and type of evaluation tools used, in

addition to aims and objectives

Approaches and ideas from two case

studies

The International AIDS Conference:

biannual, over 20,000 participants

The Lift technology conference:

annual, some 700 participants

Commonly used evaluation tools for

event/conference evaluation

• Face-to-face or phone individual interviews

• Focus groups

• Online/printed surveys

• Structured observations of key sessions and

conference areas

• Review of online resources/activities

• Review of statistical data on conference

registration

• Comparison with previous conferences

The question of impact

Some main questions posed by conference

organisers and funders are:

• What is the impact of a conference on

participants?

• What does it change in the way participants do

their jobs?

•Methods used: follow-up survey (online/face-to-face)

and action plans

1,195 AIDS 2008 delegates completed the survey

About 2/3 had learnt something new and had changed some

aspects of their work practice

Almost half reported that AIDS 2008 had directly influenced

their organizations’ HIV work

Almost 4 in 10 were aware of AIDS 2008’s influences on HIV

work, policies or advocacy in their countries

75% had kept in contact with at least 1 person met

Ex: AIDS 2008 follow-up survey (1.5 year after)

International AIDS Conference & impact

Source: http://www.aids2008.org/admin/images/upload/AIDS_2008_follow_up_survey_report.pdf

Lift Conference and impact

•Methods used: follow-up survey (online) and

testimonies

Lift Conference and impact

• Use of conference “instant” feedback

systems

• Use of the “mystery shopper” approach

• Use of network analysis and mapping

• Analysis of conference media coverage

• Analysis of social media activity

Other emerging evaluation tools

• Evaluation findings should be “very usable” as

conferences are often repeated

• Importance of “buy-in” of conference organizers

• The quality of content and format of evaluation

reports is crucial to attract readers and convince

them that evaluation results are reliable and

useable

• Dissemination of evaluation results should be

timely, and use a variety of channels depending on

the target audience

• Use of follow-up mechanisms with conference

organizers and relevant stakeholders is

recommended

Use of evaluation findings

Lessons from conference evaluation

• How to deal with over-positive feedback

• Evaluation findings can be used more for

accountability and marketing rather than for

learning and improvements

• Challenges in getting adequate resources for

conference evaluation

• Challenges in moving beyond the basic satisfaction

measures

• Factoring in the importance of participants’ views

12

Q&A

Download this presentation on my blog

oneil@owlre.com

glennoneil

www.owlre.com

@glenn_oneil

www.intelligentmeasurement.wordpress.com

Thanks to Laetitia Lienart, former evaluation manager, IAS

Join the Conference Evaluation Google Group:

http://groups.google.com/group/conference_evaluation