8/26/2011 www.insemtives.eu 1

Collaborative knowledge creation is not a game

Elena SimperlTalk at the Semantic Web meet-up, Cluj Napoca, Romania

Insemtives in a nutshell• Many aspects of semantic content authoring naturally rely on human

contribution

• Motivating users to contribute is essential for semantic technologies to reach critical mass and ensure sustainable growth

• Insemtives works on – Best practices and guidelines for incentives-compatible technology design– Enabling technology to realize incentivized semantic applications– Showcased in three case studies: enterprise knowledge management;

services marketplace; multimedia management within virtual worlds

www.insemtives.eu 2

Incentives and motivators

• Motivation is the driving force that makes humans achieve their goals

• Incentives are ‘rewards’ assigned by an external ‘judge’ to a performer for undertaking a specific task– Common belief (among

economists): incentives can be translated into a sum of money for all practical purposes

• Incentives can be related to both extrinsic and intrinsic motivations

• Extrinsic motivation if task is considered boring, dangerous, useless, socially undesirable, dislikable by the performer

• Intrinsic motivation is driven by an interest or enjoyment in the task itself

Extrinsic vs intrinsic motivations• Successful volunteer crowdsourcing is difficult

to predict or replicate– Highly context-specific– Not applicable to arbitrary tasks

• Reward models often easier to study and control*– Different models: pay-per-time, pay-per-unit, winner-

takes-it-all…– Not always easy to abstract from social aspects (free-

riding, social pressure…)– May undermine intrinsic motivation

* in cases when performance can be reliably measured

Examples

www.insemtives.eu 5

Games with a purpose (GWAP)

• „ a human-based computation technique in which a computational process performs its function by outsourcing certain steps to humans in an entertaining way”

*Wikipedia

Gamification

• “use of game play mechanics for non-game applications […] in order to encourage people to adopt the applications”*

*WikipediaImage from http://gapingvoid.com/2011/06/07/pixie-dust-the-mountain-of-mediocrity/

Gamification features*

• Accelerated feedback cycles. – Annual performance appraisals vs immediate

feedback to maintain engagement.

• Clear goals and rules of play. – Players feel empowered to achieve goals vs fuzzy,

complex system of rules in real-world.

• Compelling narrative. – Gamification builds a narrative that engages players to

participate and achieve the goals of the activity.

*http://www.gartner.com/it/page.jsp?id=1629214

How to implement gamification*

• Cosmetic: adding game-like visual elements or copy (usually visual design or copy driven)

• Accessory: wedging in easy-to-add-on game elements, such as badges or adjacent products (usually marketing driven)

• Integrated: more subtle, deeply integrated elements like % complete (usually interaction design driven)

• Basis: making the entire offering a game (usually product driven)

* http://uxmag.com/design/a-gamification-framework-for-interaction-designers

What tasks can be gamified?*

• Decomposable into simpler tasks.• Nested tasks.• Performance is measurable.• Obvious rewarding scheme.• Skills can be arranged in a smooth learning

curve.

*http://www.lostgarden.com/2008/06/what-actitivies-that-can-be-turned-into.html

What is different about semantic systems?

• Semantic Web toolsvs applications– Intelligent (specialized)

Web sites (portals) with improved (local) search based on vocabularies and ontologies

– X2X integration (often combined with Web services)

– Knowledge representation, communication and exchange

What do you want your users to do?

• Semantic applications– Context of the actual application– Need to involve users in knowledge acquisition and

engineering tasks?• Incentives are related to organizational and social factors• Seamless integration of new features

• Semantic tools– Game mechanics– Paid crowdsourcing (integrated)

• Using results of games with a purpose

Knowledge engineering tasks• Granularity of ontology

engineering activities is toobroad; further splitting isneeded

• Crowdsource very specifictasks that are (highly) divisible– Labeling (in different

languages)– Finding relationships– Populating the ontology– Aligning and interlinking– Ontology-based annotation– Validating the results of

automatic methods– …

www.insemtives.eu 13

Example: ontology alignment

www.insemtives.eu 14

Example: relationship finding

Example: video annotation

www.insemtives.eu 16

www.insemtives.eu

Example: image annotation

8/26/2011 17

Example: ontology population

8/26/2011 www.insemtives.eu 18

Example: ontology evaluation

www.insemtives.eu 19

OntoGame API

• API that provides several methods that are shared by the OntoGame games, such as– Different agreement types (e.g. selection agreement)– Input matching (e.g. , majority)– Game modes (multi-player, single player)– Player reliability evaluation– Player matching (e.g., finding the optimal partner to play)– Resource (i.e., data needed for games) management– Creating semantic content

• http://insemtives.svn.sourceforge.net/viewvc/insemtives/generic-gaming-toolkit

8/26/2011 www.insemtives.eu 20

Lessons learned• Tasks which can be subject to games

– Definition of vocabulary– Conceptualization

• Based on competency questions• Identifying instances, classes, attributes, relationships

– Documentation• Labeling and definitions• Localization

– Evaluation and quality assurance• Matching conceptualization to documentation

– Alignment– Validating the results of automatic methods

• But, the approach is per design less applicable because– Knowledge-intensive tasks that are not easily nestable– Repetitive tasks players‘ retention?

www.insemtives.eu 21

Lessons learned (ii)• Approach is feasible for mainstream domains, where a

knowledge corpus is available• Knowledge corpus has to be large-enough to allow for

a rich game experience– But you need a critical mass of players to validate the

results• Advertisement is essential• Game design vs useful content

– Reusing well-kwown game paradigms– Reusing game outcomes and integration in existing

workflows and tools• Cost-benefit analysis

elena.simperl@kit.edu

http://www.ontogame.orghttp://apps.facebook.com/ontogame

http://www.insemtives.eu/iswc2011-tutorial/