UKOLN is supported by:

Metrics and Social Web Services: Quantitative Evidence for their Use & Impact

Welcome

Brian Kelly

UKOLN

University of Bath

Bath, UK

http://www.ukoln.ac.uk/web-focus/events/workshops/eim-2011-07/http://www.ukoln.ac.uk/web-focus/events/workshops/eim-2011-07/

This work is licensed under a Attribution-NonCommercial-ShareAlike 2.0 licence (but note caveat)

Acceptable Use PolicyRecording this talk, taking photos, discussing the talk using Twitter, blogs, etc. is permitted but please try to minimise distractions to others.

Acceptable Use PolicyRecording this talk, taking photos, discussing the talk using Twitter, blogs, etc. is permitted but please try to minimise distractions to others.

Twitter:#ukolneim

Blog: Twitter:http://ukwebfocus.wordpress.com/ @briankelly

22

You are free to:

copy, share, adapt or re-mix;

photograph, film or broadcast;

blog, live-blog or post video of

this presentation provided that:You attribute the work to its author and respect the rights and licences associated with its components.

Idea from Cameron Neylon

Slide Concept by Cameron Neylon, who has waived all copyright and related or neighbouring rights. This slide only CCZero.Social Media Icons adapted with permission from originals by Christopher Ross. Original images are available under GPL at:http://www.thisismyurl.com/free-downloads/15-free-speech-bubble-icons-for-popular-websites

Off the record questions, comments should be flagged

Evidence, Impact, MetricsEvidence, Impact, Metrics (EIM):

• UKOLN activity funded by the JISC

Aims to:

• Explore ways of gathering evidence which can demonstrate the impact of services and devise appropriate metrics to support such work

By:

• Three one-day workshops

• Gathering of evidence and discussing approaches and issues on UK Web Focus blog

• A final report

3

4

Workshop Aims

By the end of the workshop we should:• Have an appreciation of the importance of

the need to gather and interpret evidence• Understand how metrics can be used to

demonstrate the value and ROI of services.• Have seen examples of how institutions are

gathering and using evidence• Be aware of limitations of such approaches• Discussed ways in which such approaches

can be used across the sector

Intr

od

uct

ion

The Context •The need to demonstrate value in light of Government cuts 5

NB 1D view: no comments on ROI, comparisons with print, TV, … costs

Open DataGovernment to ‘force’ Unis to publish data about courses, student experiences, …• Builds on Labour’s

Open Gov initiative• An opportunity to be

welcomed? • Can take initiative in

publishing open data about uses of social media

6

Numbers Matter

7

“Sales of 25 million iPads in just 14 months. 200 million iOS devices in total. 15 billion songs downloaded since 2003. 130 million books. 14 billion apps downloaded from a store that now runs to 425,000 apps.”

BBC News 7 Jun 11

JISC-funded OII Impact report: “The media and the public are influenced by numbers and metrics”

Prisoner’s DilemmaWe can all benefit by:

• Sharing best practices• Understanding limitations• Agreeing on ways forward

But concerns that:• We might be doing badly• We might not be doing as well

as our rivals• We don’t want to boast• We want a perfect solution,

not a good enough one• We want to learnt from others,

but not share our experiences

8

Prisoner’s dilemma:  why people might not cooperate even if it is in their best interests to do so.

Concerns

Your Interests

Your interests (from 15 survey responses):• “Hoping that the day will help formulate

future development and plans for effective use of third party services”

• “I have three major issues: 1. How to gather and process the data. 2. Data visualisations. 3. What does the data mean and what should I do about it?”

• “… we are looking into requirements and then will look at options/costs etc to decide what we need and what we can afford.”

9

New

Your ConcernsYou concerns:• “Hard to measure impact of the message and

whether the right audiences are being reached”• “My own data gathering is fairly ad hoc at the

moment”• “Accuracy, baseline data to compare to”• “I do not fully understand how Klout works or how

Facebook's algorithms work … would like to understand them better in order to become better informed and foster decision-making procedures.”

• “Turning data into strategy”

10

New

Your Current Approaches

What you’re using at present

11

New

Your Priorities

How much additional resources you expect to have in the future

12

New

Plans

How you expect to use your additional / existing resources

13

New

Spending Money?

14

New

Social Media services seem to be free when launched and then charge for ad-free use and for metrics (e.g. Slideshare)

Challenges:•How much would you pay (Slideshare is $19/month)?•Who pays?•Who pays for personal uses?…

Hootsuite recently introduced new charging plans - $6/month

ProgrammeTime Content

10.20-10.25 Introduction, Brian Kelly, UKOLN

10.25-10.45 Why Impact, ROI and Marketing are No Longer Dirty Words!, Amber Thomas, JISC

10.45-11.00 Surveying our Landscape from Top to Bottom Brian Kelly, UKOLN

11.00-11.20 Learning From Institutional Approaches, Ranjit Sidhu

11.20-11.40 Identity, Scholarship and Metrics,  Martin Weller, Open University

11.40-12.30 Breakout groups 1

12.30-13.30 Lunch

13.30-13.50 Experiences at the Open University, Andrew Law, Open University

13.50-14.10 The Script Kiddie's Perspective, Tony Hirst, Open University

14.10-15.00 Breakout groups 2

15.00-15.15 Coffee

15.15-15.45 Report Back & Action Plans

15.45-16.00 Conclusions 15

16

Questions

Any questions or comments?

Note that resources used in the workshop and notes from the workshop will be made available shortly. See ukwebfocus.wordpress.com for further information