Return On Contribution (ROC) ECSCW 2009 Muller Et Al
-
Upload
michael-muller -
Category
Spiritual
-
view
108 -
download
1
description
Transcript of Return On Contribution (ROC) ECSCW 2009 Muller Et Al
Return On Contribution (ROC):
A Metric for Enterprise Social Software
Michael Muller, Jill Freyne*, Casey Dugan,David R Millen, & Jennifer Thom-Santelli
IBM Research & IBM Center for Social Software
Cambridge MA USA
*Jill Freyne is now at Tasmanian ICT Center, CSIRO, Australia
1
Agenda
• How to measure the benefits of social software in
organizations?
• Return On Contribution (ROC)
• Applying ROC to
– Enterprise social software applications
– Types of resources in social software applications– Types of resources in social software applications
– Points of articulation in social software applications
– Individual users (with a few notes on privacy)
• Conclusion and Next Steps
2
Benefits of Social Software
• Informal arguments are known
– Better knowledge-sharing
– Better personal effectiveness
– Improved ability to manage one’s reputation
– In organizations, better satisfaction and retention
• There are few strong studies to support those claims• There are few strong studies to support those claims
• Management’s desire: Return On Investment
– Has been shown for niche social software applications
• Customer-support operations
• Customer communities
– Has been shown for advertising opportunities
• Social networking sites
3
Return On … What?
• Return On Investment
– ROI = Benefit / Cost � € / € (unitless economic ratio)
– We hope for ROI >> 1.0
• Social software benefits – and even costs – are
difficult to measure
– Is the purpose of social software to increase productivity?– Is the purpose of social software to increase productivity?
– How do you calculate ROI of a telephone? an IM product?
– How do you calculate ROI of a relationship?
4
Return On Contribution (ROC)
• A social ratio
– ROI = € / €
– ROI = Benefit / Cost
– ROC = Beneficiaries / Contributors (unitless social ratio)
= Consumers / Producers
• Return On Contribution• Return On Contribution
– A measure of social effectiveness – do more people benefit
(or consume) than contribute (or produce)?
– Rational Choice theory (Pirolli, 2007)
• Over time, people’s work-oriented decisions are beneficial to them
• Measure those choices and summarize them as a metric
5
• Beehive
– “Social networking
behind the firewa
– Overall usagell”
– Common goods
• Photos
ROC for Two Enterprise Services
• Dogear
– “Social bookmarking
behind the firewall”
– Overall usage
– Common goods
• Bookmarks Photos
• Lists (HiveFives)
• Events
• (Person-summaries)
Bookmarks
• Tags
• (Person-summaries)
6
• Beehive
– “Social networking
behind the firewall”
– Overall usage
– Common goods
• Photos
ROC for Two Enterprise Services
• Dogear
– “Social bookmarking
behind the firewall”
– Overall usage
– Common goods
• Bookmarks Photos
• Lists (HiveFives)
• Events
• (Person-summaries)
Bookmarks
• Tags
• (Person-summaries)
7
• Beehive
– “Social networking
behind the firewall”
– Overall usage
– Common goods
• Photos
ROC for Two Enterprise Services
• Dogear
– “Social bookmarking
behind the firewall”
– Overall usage
– Common goods
• Bookmarks Photos
• Lists (HiveFives)
• Events
• (Person-summaries)
Bookmarks
• Tags
• (Person-summaries)
8
• Beehive
ROC for Two Enterprise Services
• Dogear
9
• Beehive
ROC for Two Enterprise Services
• Dogear
10
• Beehive
– “Social networking
behind the firewall”
– Overall usage
– Common goods
• Photos
ROC for Types of Contributions
• Dogear
– “Social bookmarking
behind the firewall”
– Overall usage
– Common goods
• Bookmarks Photos
• Lists (HiveFives)
• Events
• (Person-summaries)
Bookmarks
• Tags
• (Person-summaries)
11
• Beehive
– “Social networking
behind the firewall”
– Overall usage
– Common goods
• Photos
ROC for Types of Contributions
• Dogear
– “Social bookmarking
behind the firewall”
– Overall usage
– Common goods
• Bookmarks
Monthly Social-Networking ROC for three media types
6
8
10
12
14
16
18
20
ROCC
Photo
List
Event
Data range of Figure 1
Monthly Social-Networking ROC for three media types
6
8
10
12
14
16
18
20
ROCC
Photo
List
Event
6
8
10
12
14
16
18
20
ROCC
Photo
List
Event
Photo
List
Event
Data range of Figure 1
Photos
• Lists (HiveFives)
• Events
• (Person-summaries)
Bookmarks
• Tags
• (Person-summaries)
12
0
2
4
6
Jun-07 Jul-07 Aug-07 Sep-07 Oct-07 Nov-07 Dec-07 Jan-08
Date
Data range of Figure 1
0
2
4
6
Jun-07 Jul-07 Aug-07 Sep-07 Oct-07 Nov-07 Dec-07 Jan-08
Date
0
2
4
6
Jun-07 Jul-07 Aug-07 Sep-07 Oct-07 Nov-07 Dec-07 Jan-08
Date
Data range of Figure 1
• Beehive
– “Social networking
behind the firewall”
– Overall usage
– Common goods
• Photos
ROC for Specific Contributions
• Dogear
– “Social bookmarking
behind the firewall”
– Overall usage
– Common goods
• Bookmarks Photos
• Lists (HiveFives)
• Events
• (Person-summaries)
Bookmarks
• Tags – specific tags
• (Person-summaries)
13
• Beehive
– “Social networking
behind the firewall”
– Overall usage
– Common goods
• Photos
ROC for Specific Contributions
• Dogear
– “Social bookmarking
behind the firewall”
– Overall usage
– Common goods
• Bookmarks Photos
• Lists (HiveFives)
• Events
• (Person-summaries)
Bookmarks
• Tags – specific tags
• (Person-summaries)
14
Tagging for audiences (Thom-Santelli
et al., 2008)
• Publishers
• Evangelists
• Beehive
– “Social networking
behind the firewall”
– Overall usage
– Common goods
• Photos
ROC for Specific Contributions
• Dogear
– “Social bookmarking
behind the firewall”
– Overall usage
– Common goods
• Bookmarks Photos
• Lists (HiveFives)
• Events
• (Person-summaries)
15
Bookmarks
• Tags – specific tags
– Publisher: podcast tag “Tag-City”
Tag ROC = 7.41 readers/contributor – Personal ROC = 63.00
– Evangelist: tag “web2.0”
Tag ROC = 1.95 readers/contributor – Personal ROC = 1245.00
• (Person-summaries)
• Beehive
– “Social networking
behind the firewall”
– Overall usage
– Common goods
• Photos
ROC for Specific Contributions
• Dogear
– “Social bookmarking
behind the firewall”
– Overall usage
– Common goods
• Bookmarks Photos
• Lists (HiveFives)
• Events
• (Person-summaries)
16
Bookmarks
• Tags – specific tags
– Publisher: podcast tag “Tag-City”
Tag ROC = 7.41 readers/contributor – Personal ROC = 63.00
– Evangelist: tag “web2.0”
Tag ROC = 1.95 readers/contributor – Personal ROC = 1245.00
• (Person-summaries)
ROC for Other Social Applications
Service Beneficiaries Contributors ROC C
Dogear 10896 4213 2.59
Beehive 21453 8397 2.55
Wiki server 238838 36377 6.57
Discussion server 150000 23000 6.52
Person-tagging 20973 3102 6.76
File-sharing 68762 11276 6.19
17
Implications for Design or Potential Use
• Track the development of organizational value of an application
over time
– Does it increase? Does it stabilize?
• Compare the organizational value of different types of
contributions over time
• Compare the organizational value of specific contribution
instancesinstances
• Assist the development of individual contributors, especially in
assigned roles such as “evangelist” or “publisher”, by providing
private views of her/his personal ROC
• Monitor, on an anonymous basis, the development of social
capital through aggregate, summary ROC measures across all
beneficiaries and contributors
18
Unanswered Questions about ROC
• Are there characteristic “signature” ROC values for
different types of applications?
• How to determine “stabilization” of ROC over time?
• What should the “target” ROC be for a discussion
forum?
– ROC >> 6.0 for some applications looked very nice– ROC >> 6.0 for some applications looked very nice
– However, ROC=1.0 suggests full democratic participation
– When are different values of ROC desirable?
• What should the “target” ROC be for a type of object,
or a particular object (e.g., a tag)?
• Can ROC help to show the value of “lurkers”? When is
it permissible (under privacy rules) to study “lurking”?19
Summary of Contributions
• Lurkers as non-public participants (Nonnecke and
Preece, 2001) and as altruists (Takahashi et al., 2003)
– Employees in some jobs are “paid to lurk”
– Lurkers’ “consumption” of shared objects is a test of the
organizational value of those objects
• ROC provides a “social value” metric for• ROC provides a “social value” metric for
– Social software applications (Dogear, Beehive)
– Types of contributions (Photos, Lists, Events in Beehive)
– Specific contributions (Tags in Dogear)
– (where permitted) specific contributors (Taggers in Dogear)
• ROC can help organizations and researchers to assess
and study the value of social media20
• Beehive
ROC for Two Enterprise Services
• Dogear
22
• Beehive
ROC for Two Enterprise Services
• Dogear
Two ways to think about benefit
• All users (including contributors)
ROC C = AllUsers/Contributors
• Lurkers only
ROC = Lurkers/Contributors
23
ROC L = Lurkers/Contributors
• Beehive
ROC C and ROC L
• Dogear
Two ways to think about benefit
• All users (including contributors)
ROC C = AllUsers/Contributors
• Lurkers only
ROC = Lurkers/Contributors
24
ROC L = Lurkers/Contributors
• Beehive
ROC for Two Enterprise Services
• Dogear
25