1. A reputation system to model expertise in online
communities
Doctoral Consortium UMAP2011
+ some social mechanisms
2. From reputation to social mechanisms
Peer-based learning in online communities
Challenges
Motivation
Quality
Initial research direction: reputation
3. Reputationprinciples
Linkedwith trust
Reciprocity & Goodbehaviour
Someoneelses story about me
Linkedwithidentity; long-lived
A currency, a resource
Narrative, dynamic
Based on claims, transactions, opinion, rating, endorsements,
Based on indirect information
Context of community
Contextual
Trade-off between trust & privacy
Regular assessment of reputationquality
Windley, Phillip J., Kevin Tew, and Devlin Daley. A Framework for
Building Reputation Systems.
In WWW2007.
4. Main challenge
What:
Aggregating and interpreting meaningfulinteractions between &
among objects* in online communities
Why:
to give insight in the value on the object level based on user
feedback and usage.
* Objects are people and information objects
5. So what did I do?
Literature
Learning theories, knowledge management
Trust and reputation
Look at different successful reputation systems/techniques
Google PageRank, eBay, StackOverflow, Guru, etc.
How?
value flow
context integration
sustainability
Hennis, T., Lukosch, S., & Veen, W. (2011). Reputation in
peer-based learning environments. In O. C. Santos & J. G.
Boticario (Eds.), Educational Recommender Systems and Technologies.
IGI.
6. Value flow
Source objects
People, organizations,
Target objects
Blog posts, articles,
Claims (value statements)
Rates, links, recommendations, etc.
Implicit, explicit
(Farmer & Glass, 2009)
7. Context integration
8. Sustainability, i.e. StackOverflow
9.
10.
11. Concept reputation model
12. Concept reputation model 1/6claim weight
claim weight ~ expressiveness of the claim type
(i.e. rating versus click)
13. Concept reputation model 2/6target object weight
claim weight ~ expressiveness of the claim type (i.e. rating versus
click)
target object weight ~ importance of the contribution type
(i.e. article versus comment)
14. Concept reputation model 3/6affiliated keyword weight
apples (0.8)
pears (0.2)
claim weight ~ expressiveness of the claim type (i.e. rating versus
click)
target object weight ~ importance of the contribution type (i.e.
article versus comment)
affiliated keyword weight ~ expressiveness of tag about the target
object
15. Concept reputation model 4/6source object weight =
authority
match keywords with reputation!
apples (64)
pears (33)
kiwis (0)
apples (0.6)
pears (0.2)
kiwis (0.2)
claim weight ~ expressiveness of the claim type (i.e. rating versus
click)
target object weight ~ importance of the contribution type (i.e.
article versus comment)
affiliated keyword weight ~ expressiveness of tag about the target
object
source object weight = (source objects reputation for keyword) /
(global rep. value for that keyword)
DIFFERENT WEIGHTS FOR DIFFERENT KEYWORDS
16. Concept reputation model 5/6claim value
apples (64)
pears (33)
kiwis (0)
apples (0.6)
pears (0.2)
kiwis (0.2)
claim weight ~ expressiveness of the claim type (i.e. rating versus
click)
target object weight ~ importance of the contribution type (i.e.
article versus comment)
affiliated keyword weight ~ expressiveness of tag about the target
object
source object weight = (source objects reputation for keyword) /
(global rep. value for that keyword)
claim value = rating (implicit/explicit), i.e. 4/5 stars
17. Concept reputation model 6/6claim value
apples (64)
pears (33)
kiwis (0)
apples (0.6)
pears (0.2)
kiwis (0.2)
claim weight ~ expressiveness of the claim type (i.e. rating versus
click)
target object weight ~ importance of the contribution type (i.e.
article versus comment)
affiliated keyword weight ~ expressiveness of tag about the target
object
source object weight = (source objects reputation for keyword) /
(global rep. value for that keyword)
claim value = rating (implicit/explicit)
3 claims (one for each affiliate keyword)
18. Why is this a useful approach?
Target object weight
Affiliate keyword weight
Claim for keyword k
Claim weight
Authority
Claim value (rating)
19. Why is this a useful approach?
Target object weight
Affiliate keyword weight
Account for:
Several relevant context factors, such as authority
20. Extensible & configurable
21. including other weights and metrics (i.e. trust value,
network centrality, etc.)
22. integrating formal ontologies
23. taking into account all relevant interactions
24. Starting point for the design of such a system
25. Rich profiles
Requirements
Sufficient interactions & contributions
26. Rather large distributed online network or community
Claim for keyword k
Claim weight
Authority
Claim value (rating)
27. Reputation clouds (object model)
28. Peer Support Community
Trying to get funding (50k) for first prototype
Context
Blackboard apps & Tags
Comment & Rating functionality
Application of reputation model
Source: Teacher or Student
Target: comment, answer
Claims: like, page visit, follow
Reuse of reputation
Award, status, social comparison, gaming mechanisms
(competition)
29. Contextualized support: content and user support
30. But
Not smart to bet on 1 horse, when 3 already dropped out of the
race
31. So.. New scope (since 2 weeks)
Social mechanisms to design incentive structures to support
informal learning in online communities
Hennis, T. A., & Kolfschoten, G. L. (2010). Understanding
Social Mechanisms in Online Communities. In G. D. Vreede (Ed.),
Group Decision and Negotiation 2010. Delft, the Netherlands.
Hennis, T. A., & Lukosch, H. (2011). Social Mechanisms to
Motivate Learning with Remote Experiments - Design choices to
foster online peer-based learning. CSEDU 2011.
Veen, W., Staalduinen, J.-P. V., & Hennis, T. A. (2010).
Informal self-regulated learning in corporate organizations. In G.
Dettori & D. Persico (Eds.), Fostering Self-regulated learning
through ICTs. Genova, Italy: Institute for Educational Technologies
Italys National Research Council.
32. Bouwman et al. (2007)
We argue that social software systems should trigger mechanisms
that allow us to associate with or form social groups, whether
online or in the real world.
Such mechanisms would acknowledge human motivations, like eagerness
for exploration, curiosity, inquisitiveness, civilization,
valuation of belonging, achieving self-realization, enjoying
one-self.
Bouman, W., Hoogenboom, T., Jansen, R., Schoondorp, M., Bruin, B.
de, & Huizing, A. (2007). The realm of sociality: notes on the
design of social software. Amsterdam.
33. Research objectives
Designing incentives: which mechanism to apply when and how?
Design of processes
Supportive technologies
Focused on
informal learning
in organizations
during initial phase startup phase
Cases
Philips Lighting
Mediamatic (various communities)
34. Step 1 improve list of mechanisms (literature)
Matching objectives
Organizational objectives
User models
Fit / Embedding in practice
Rhythm
Leadership and roles
Heterogeneity & Diversity
Learning & Networking
Reputation & Identity
Reciprocity & Feedback
Common Ground & Privacy
Self-efficacy & Social comparison
Autonomy? Empowerment?
Curiosity & Provocation
IMPROVE
35. Step 2 Design & Evaluate
Philips Lighting
3 communities by December
Mediamatic Design team
Design & Test new things
Evaluate existing communities using
the Anymeta platform
Qualitative
Quantitative 50+ small-medium sized online communities
Blogging communities
Storytelling
Event communities & Professional networks
36. Rating & Reputation
Reciprocity & Feedback
Matching online and offline networks through RFID
Notifications & Activity
38. Keyword based
Recommend content, people, projects,
39.
40. Definitions
A social mechanism is a plausible hypothesis, or set of plausible
hypotheses, that could be the explanation of some social
phenomenon.
An incentive is any factor (financial or non-financial) that
enables or motivates a particular course of action, or counts as a
reason for preferring one choice to the alternatives.
41. Overall picture
Typicalvaluejudgements/claims
Context parameters
aggregateandinherit/inference
repute
value judgements:
use/rate/recommend
context parameters:
tag
contribute
42. 1: ContributionI write a blog post
Knowledge (topic)
What kind of contribution?
What kind of topic?
contributions
Competencies (process)
What kind of action?
Which competencies involved?
43. 2: Value + context (cont.)people rate, comment, tag
etc
value statement: