Building a-standpoints-web-to-support-decision-making-in-wikipedia--cscw2012-doctoral-colloquium

Copyright 2010 Digital Enterprise Research Institute. All rights reserved.

Digital Enterprise Research Institute

www.deri.ie

Building a Standpoints Web to Support Decision-Making in

Wikipedia

Jodi Schneider

Doctoral Colloquium at CSCW 20122012-02-12Seattle, Washington


www.deri.ie

What I’m looking for

Scoping & focus Detailed mentoring on CSCW/HCC methodologies Interviewing Qualitative Research Statistics

Suggestions for evaluating my work

2


www.deri.ie

3


www.deri.ie

Should we delete this article?

4


www.deri.ie

Improving deletion discussions

Main problems: Newcomers who don’t know how to argue Overwhelm of long discussions Discussions that happen over and over again

Deletion as quality control Large number of discussions - ~500/week

5


www.deri.ie

Deletion argument

[Delete the article]...hasn't played since 2008. His 66-73 record is far from stellar and, in my opinion, does not merit an article.

>>He pitched last month and plays for the Venezuelan League. This meets our article criteria.

6


www.deri.ie

Goals

Newcomers who don’t know how to argue Characterize the “good” and “bad” arguments Develop argument templates Provide guidance and support for new users in properly structuring arguments according to Wikipedia’s rhetorical standards

Overwhelm of long discussions Develop a claims/argument explorer

Discussions that happen over and over again Prototype an argument bot Populate argument maps with mixed-initiative claims extraction

7


www.deri.ie

Overview

Corpus: All Wikipedia deletion discussions from January 29, 2011

Perspectives/approaches: Argumentation CSCW/HCC Text analytics Ontologies/Social Semantic Web

8


www.deri.ie

Current work

Analysis of the corpus Argument schemes (e.g. expert opinion) Factors (e.g. notability, uniqueness) Newcomer’s arguments

Interviews Administrators Experienced users

Argument exploration Text mining cue words (‘however’, ‘therefore’,…)

Architecture Ontology development “Standpoints Web”

9


www.deri.ie

Standpoint

[Delete the article]...hasn't played since 2008. His 66-73 record is far from stellar and, in my opinion, does not merit an article.

Proposition: does not merit an article

Justification: hasn’t played since 2008, bad record

10


www.deri.ie

Opposing standpoint

>>He pitched last month and plays for the Venezuelan League. This meets our article criteria.

Proposition: keep the article

Justification: meets our article criteria

11


www.deri.ie

Possible applications

Visualize decision-making Highlight controversies Query opinions and arguments Discuss arguments interactively with a bot Calculate the “best” options Analyze, extract, and represent disagreement

13


www.deri.ie

Thanks!

[email protected]://jodischneider.com/jodi.html@jschneider

mailto:[email protected]

http://jodischneider.com/jodi.html


www.deri.ie

Acknowledgments

Thanks to our collaborators! Katie Atkinson, Trevor Bench-Capon, Adam Wyner (Liverpool)

DERI Social Software Unit Rhetorical Structure, W3C Health Care and Life Sciences

Funding Science Foundation Ireland Grant No. SFI/08/CE/I1380 (Líon-2)

Short-term scientific mission (STSM 1868) from the COST Action ICO801 on Agreement Technologies

15


www.deri.ie


www.deri.ie

“ELIZA for arguments”

17

Snaith, Lawrence, & Reed, “Mixed initiative argument in public deliberation,” ODET 2010


www.deri.ie

Highlight Controversies

Ennals, R., Trushkowsky, B., & Agosta, J. M. (2010). Highlighting Disputed Claims on the Web. In WICOW at WWW 2010.


www.deri.ie

Transform Debates into Argument Frameworks

(1) Households should pay tax for their garbage.

(4) (1) Paying tax for garbage increases recycling, so households should pay.

(3) (1)Recycling more is good, so people should pay tax for their garbage.

19

Arrow: premise

Wyner, van Engers, & Bahreini. From Policy-making Statements to First-order Logic. EGOV 2010


www.deri.ie

Calculate best options (non-contradictory opinions)

20

Wyner, van Engers, & Bahreini. From Policy-making Statements to First-order Logic.

EGOV 2010


www.deri.ie

Claims Extraction

Cue words (“Hence Jaffa Cakes are cakes.”) [Marcu]

Rhetorical Structure Theory

Claims Extraction


www.deri.ie

Important Relationships

Attacks Supports

23


www.deri.ie

Case Study

Understand: interviews, observation, and content analysis

Intervene: Implement & test the Standpoints Web architecture on Wikipedia deletion discussions

Evaluation:Community feedback

Ontology fitness-for-purpose Precision & recall?

25


www.deri.ie

Problem Possible Uses of a Knowledge Representation

Concrete Examples Some Current Directions

26


www.deri.ie

The Problem

The Web is full of opinions & commentary. A lot of it disagrees. How do we learn from other people, when they disagree?

27


www.deri.ie

My Approach

Identify peoples’ views Collect the explanations people give Create a hypertext web of these views & explanations

28


www.deri.ie

Two Persuasive Messages

29


www.deri.ie

Why: Walton’s dialogue types

30


www.deri.ie

Knowledge-based Claims Vary

Use of statistics & impersonal information

31


www.deri.ie

Versus personal appeals…

Where opinions and personal values are explicit

32


www.deri.ie

Why: Knowledge, Emotion, Values as a Proxy

33


www.deri.ie

Purpose-related keywords

Knowledge statistics

Values truth secret

Rhetoric you can thank

Judgment/Opinion eradicate tough rejecting

34


www.deri.ie

Purpose matters

Knowledge-oriented discussions are straightforward to reuse

Opinion-oriented discussion types may require caveating or balancing emotion makes a discussion more interesting can also indicate the potential for bias.

35


www.deri.ie

Twitter: Standpoint

Difference between cakes and biscuits? When stale, cakes go hard, biscuits go soft. Hence Jaffa Cakes are cakes. (Was official EU ruling).

View: Jaffa Cakes are cakesJustification: official EU ruling; go hard when stale

36


www.deri.ie

Visualize

bCisive

Building a-standpoints-web-to-support-decision-making-in-wikipedia--cscw2012-doctoral-colloquium

Technology

Transcript of Building a-standpoints-web-to-support-decision-making-in-wikipedia--cscw2012-doctoral-colloquium