Arguments about deletion cscw2013 how experience improves the acceptability of arguments in ad hoc...
-
Upload
jodischneider -
Category
Technology
-
view
534 -
download
0
description
Transcript of Arguments about deletion cscw2013 how experience improves the acceptability of arguments in ad hoc...
Copyright 2011 Digital Enterprise Research Institute. All rights reserved.
Digital Enterprise Research Institute www.deri.ie
Enabling Networked Knowledge
Arguments about Deletion
How Experience Improves the Acceptability of Arguments in Ad-hoc Online Task Groups
Jodi Schneider, Krystian Samp, Alexandre Passant, and Stefan Decker @jschneider
Tuesday 26th February 2013
1
CSCW 2013: Computer Supported Cooperative Work and Social Computing
San Antonio, Texas
Ad-hoc online task groups• Open source software development• Collaborative writing groups• Standardization bodies
Let’s do something together at the banquet!
How’s the talk length?
Making an argument
Arguments give a POSITION & RATIONALE
Position: Add more question time
Rationale: the flow of people between sessions can be distracting
Arguments are used in decision-making
Position: Add more question time Attack position
Rationale: the flow of people between sessions can be distractingAttack rationale
Attack inference
John Danaher
Case Study of Argumentation in Wikipedia Deletion Discussions
1. What arguments are given in content deletion discussions?
2. Differences in novices’ and experts’ arguments?
3. Which argumentation schemes are accepted?
Should we delete this Wikipedia article?
[Delete the article]...hasn't played since 2008. His 66-73 record is far from stellar and, in my opinion, does not merit an article.
>>He pitched last month and plays for the Venezuelan League. This meets our article criteria.
[Delete the article]...hasn't played since 2008. His 66-73 record is far from stellar and, in my opinion, does not merit an article.
>>He pitched last month and plays for the Venezuelan League. This meets our article criteria.
“Rule” Argumentation Scheme
“Evidence” Argumentation Scheme
Evidence + Rule -> Conclusion
Corpus Analysis• English Wikipedia deletion discussions• Representative “typical” day: 72 discussions• Iterative Manual Annotation w/ multiple coders
Which arguments are given?
Compared to general arguing, Wikipedia arguing involves more:
• Rules• Values• Bias• Precedent• Avoiding Waste
Novices often misunderstand policy
“if you folks had been around actively working on the Web in 2000, you would know when the Dot Com Bubble burst, many, many companies went out of business. Servers with information about me... were taken off line.”
Problems with novices’ arguments
• Confusion about what “reliable source” (RS) means• Confusion about establishing notability with RS’s• Confusion about verification and need for RS’s
Problems with experts’ arguments
• Vagueness or lack of justification• Boilerplate messages lacking in detail• Sourcing (verification vs. importance of topic)• Mixing case-debate with policy development
Copyright 2011 Digital Enterprise Research Institute. All rights reserved.
Enabling Networked Knowledge
Potentially problematic argument
Typical Argumentation Schemes
Wikipedia terminology
Personal preference Values ILIKEIT
Few search engine hits Ignorance Google test
Many search engine hits Cause to Effect Google test
Requesting a favorValues or Waste or Practical Reasoning
PLEASEDONT
Analogy to other cases Analogy OTHER
No harm in keeping an article Values or Waste NOHARM
Topic will be notable in the future
Practical Reasoning CRYSTALBALL
Novices are more likely to use some problematic arguments• Valueso“obviously of interest to the public in general”
• Analogyo“just as special as an article on a breed of dog
or something similar”
Emotion• Emotional involvement vs. actiono“I know Wikipedia has a dislike for all things
[article topic]”o“I and others have added several third-party
reliable sources to the article.”• Understanding the process helpso “I believe that (much as it would break my
heart based on the no of hours I have put in over the years working on the article) it is perhaps sensible that the piece is deleted.”
Future Work
• Templates for effective arguments • Semi-automatic argument identification• Reusing argument analysis methods • Incentivizing social sensitivity• Classifying emotional needs and triggers• Understanding impact on newcomers &
article creators
Future Work
• Templates for effective arguments • Semi-automatic argument identification• Reusing argument analysis methods • Incentivizing social sensitivity• Classifying emotional needs and triggers• Understanding impact on newcomers &
article creators
Thanks! @jschneider [email protected]
Acknowledgements
• Science Foundation Ireland Grant No. SFI/09/CE/I1380 (Líon2)
• Annotators: Laura O’Connor and Lyndia Peters• Trevor Bench-Capon, Luigina Ciolfi,
Bernie Hogan, David Randall, Mark Snaith, Adam Wyner
• Thanks to reviewers and CSCW revisions process!
Research Questions• [RQ1] What arguments are given?
• [RQ2] Do people with different levels of experience with Wikipedia editing or the Wikipedia deletion process provide different types of arguments?
• [RQ3] Which argumentation schemes are accepted?
Previous ResearchShallow analysis of large datasets
• Redacted content • West & Lee, “What Wikipedia deletes” WikiSym 2011
• Vote sequencing • Taraborelli & Ciampaglia “Beyond notability” SASOW 2011
• Decision quality • Lam, Karim & Riedl “The effects of group composition on decision quality in a
social production community”, GROUP 2010
• Who participates, what & how much gets deleted• Priedhorsky, Chen, Lam, Panciera, Terveen, & Riedl. “Creating, destroying, and
restoring value in Wikipedia”, GROUP 2007• Geiger & Ford “Participation in Wikipedia’s article deletion processes”, WikiSym
2011
30
Argument from Rules - From Established Rule
Major Premise: If carrying out types of actions includingA is the established rule for x, then (unless the case is anexception), a must carry out A.
Minor Premise: Carrying out types of actions including A isthe established rule for a.
Conclusion: Therefore, a must carry out A.
Find counterarguments with “critical questions”
1. Does the rule require carrying out this type of action?
2. Are there other established rules that might conflict with or override this one?
3. Are there extenuating circumstances or an excuse for noncompliance?
Instantiating
If stopping at a red light is the established rule for driving a vehicle, then (unless the case is an exception), drivers must stop at a red light.
Stopping at a red light is the established rule for drivers.
Therefore, drivers must stop at a red light.
1. Were you driving a vehicle?
2. Did a police officer direct you to continue without stopping?
3. Were you driving an ambulance with its siren on?
Differences in Novices vs. Experts
• Experts may read all debates
Case Study of Argumentation in Wikipedia Deletion Discussions
RQ 1: What arguments are given in content deletion discussions?
RQ 2: Differences in novices’ and experts’ arguments?
RQ 3: Which argumentation schemes are accepted?