Copyright 2010 Digital Enterprise Research Institute. All rights reserved.

Digital Enterprise Research Institute

www.deri.ie

Building a Standpoints Web to Support Decision-Making in

Wikipedia

Jodi Schneider

Doctoral Colloquium at CSCW 20122012-02-12Seattle, Washington

Digital Enterprise Research Institute

www.deri.ie

What I’m looking for

Scoping & focus Detailed mentoring on CSCW/HCC methodologies Interviewing Qualitative Research Statistics

Suggestions for evaluating my work

2

Digital Enterprise Research Institute

www.deri.ie

3

Digital Enterprise Research Institute

www.deri.ie

Should we delete this article?

4

Digital Enterprise Research Institute

www.deri.ie

Improving deletion discussions

Main problems: Newcomers who don’t know how to argue Overwhelm of long discussions Discussions that happen over and over again

Deletion as quality control Large number of discussions - ~500/week

5

Digital Enterprise Research Institute

www.deri.ie

Deletion argument

[Delete the article]...hasn't played since 2008. His 66-73 record is far from stellar and, in my opinion, does not merit an article.

>>He pitched last month and plays for the Venezuelan League. This meets our article criteria.

6

Digital Enterprise Research Institute

www.deri.ie

Goals

Newcomers who don’t know how to argue Characterize the “good” and “bad” arguments Develop argument templates Provide guidance and support for new users in properly structuring arguments according to Wikipedia’s rhetorical standards

Overwhelm of long discussions Develop a claims/argument explorer

Discussions that happen over and over again Prototype an argument bot Populate argument maps with mixed-initiative claims extraction

7

Digital Enterprise Research Institute

www.deri.ie

Overview

Corpus: All Wikipedia deletion discussions from January 29, 2011

Perspectives/approaches: Argumentation CSCW/HCC Text analytics Ontologies/Social Semantic Web

8

Digital Enterprise Research Institute

www.deri.ie

Current work

Analysis of the corpus Argument schemes (e.g. expert opinion) Factors (e.g. notability, uniqueness) Newcomer’s arguments

Interviews Administrators Experienced users

Argument exploration Text mining cue words (‘however’, ‘therefore’,…)

Architecture Ontology development “Standpoints Web”

9

Digital Enterprise Research Institute

www.deri.ie

Standpoint

[Delete the article]...hasn't played since 2008. His 66-73 record is far from stellar and, in my opinion, does not merit an article.

Proposition: does not merit an article

Justification: hasn’t played since 2008, bad record

10

Digital Enterprise Research Institute

www.deri.ie

Opposing standpoint

>>He pitched last month and plays for the Venezuelan League. This meets our article criteria.

Proposition: keep the article

Justification: meets our article criteria

11

Digital Enterprise Research Institute

www.deri.ie

Possible applications

Visualize decision-making Highlight controversies Query opinions and arguments Discuss arguments interactively with a bot Calculate the “best” options Analyze, extract, and represent disagreement

13

Digital Enterprise Research Institute

www.deri.ie

Thanks!

jodi.schneider@deri.orghttp://jodischneider.com/jodi.html@jschneider

Digital Enterprise Research Institute

www.deri.ie

Acknowledgments

Thanks to our collaborators! Katie Atkinson, Trevor Bench-Capon, Adam Wyner (Liverpool)

DERI Social Software Unit Rhetorical Structure, W3C Health Care and Life Sciences

Funding Science Foundation Ireland Grant No. SFI/08/CE/I1380 (Líon-2)

Short-term scientific mission (STSM 1868) from the COST Action ICO801 on Agreement Technologies

15

Digital Enterprise Research Institute

www.deri.ie

Digital Enterprise Research Institute

www.deri.ie

“ELIZA for arguments”

17

Snaith, Lawrence, & Reed, “Mixed initiative argument in public deliberation,” ODET 2010

Digital Enterprise Research Institute

www.deri.ie

Highlight Controversies

Ennals, R., Trushkowsky, B., & Agosta, J. M. (2010). Highlighting Disputed Claims on the Web. In WICOW at WWW 2010.

Digital Enterprise Research Institute

www.deri.ie

Transform Debates into Argument Frameworks

(1) Households should pay tax for their garbage.

(4) (1) Paying tax for garbage increases recycling, so households should pay.

(3) (1)Recycling more is good, so people should pay tax for their garbage.

19

Arrow: premise

Wyner, van Engers, & Bahreini. From Policy-making Statements to First-order Logic. EGOV 2010

Digital Enterprise Research Institute

www.deri.ie

Calculate best options (non-contradictory opinions)

20

Wyner, van Engers, & Bahreini. From Policy-making Statements to First-order Logic.

EGOV 2010

Digital Enterprise Research Institute

www.deri.ie

Claims Extraction

Cue words (“Hence Jaffa Cakes are cakes.”) [Marcu]

Rhetorical Structure Theory

Claims Extraction

Digital Enterprise Research Institute

www.deri.ie

Important Relationships

Attacks Supports

23

Digital Enterprise Research Institute

www.deri.ie

Case Study

Understand: interviews, observation, and content analysis

Intervene: Implement & test the Standpoints Web architecture on Wikipedia deletion discussions

Evaluation:Community feedback

Ontology fitness-for-purpose Precision & recall?

25

Digital Enterprise Research Institute

www.deri.ie

Problem Possible Uses of a Knowledge Representation

Concrete Examples Some Current Directions

26

Digital Enterprise Research Institute

www.deri.ie

The Problem

The Web is full of opinions & commentary. A lot of it disagrees. How do we learn from other people, when they disagree?

27

Digital Enterprise Research Institute

www.deri.ie

My Approach

Identify peoples’ views Collect the explanations people give Create a hypertext web of these views & explanations

28

Digital Enterprise Research Institute

www.deri.ie

Two Persuasive Messages

29

Digital Enterprise Research Institute

www.deri.ie

Why: Walton’s dialogue types

30

Digital Enterprise Research Institute

www.deri.ie

Knowledge-based Claims Vary

Use of statistics & impersonal information

31

Digital Enterprise Research Institute

www.deri.ie

Versus personal appeals…

Where opinions and personal values are explicit

32

Digital Enterprise Research Institute

www.deri.ie

Why: Knowledge, Emotion, Values as a Proxy

33

Digital Enterprise Research Institute

www.deri.ie

Purpose-related keywords

Knowledge statistics

Values truth secret

Rhetoric you can thank

Judgment/Opinion eradicate tough rejecting

34

Digital Enterprise Research Institute

www.deri.ie

Purpose matters

Knowledge-oriented discussions are straightforward to reuse

Opinion-oriented discussion types may require caveating or balancing emotion makes a discussion more interesting can also indicate the potential for bias.

35

Digital Enterprise Research Institute

www.deri.ie

Twitter: Standpoint

Difference between cakes and biscuits? When stale, cakes go hard, biscuits go soft. Hence Jaffa Cakes are cakes. (Was official EU ruling).

View: Jaffa Cakes are cakesJustification: official EU ruling; go hard when stale

36

Digital Enterprise Research Institute

www.deri.ie

Visualize

bCisive