Identifying arguments for evaluation using an argument explorer - London Argumentation Forum -...

Identifying Arguments for Evaluation using an Argument Explorer

Jodi Schneider1, Adam Wyner2, Katie Atkinson2, Trevor Bench-Capon2

1Digital Enterprise Research Institute, National University of Ireland2Department of Computer Science, University of Liverpool

April 20, 2012

London Argumentation Forum


Argumentation is everywhere!

April 20, 2012


Identifying arguments is hard.

April 20, 2012


Goals

• Extract arguments from source texts so they can be evaluated with formal automated tools

• Speed the work of human analysts• Make argument identification more objective

April 20, 2012


Strategy & Issues

• Decompose the complexity of a text– What are the parts of an argument?– What kind of domain knowledge do we need?– How are the parts of the argument related?– What are the contrasts and negations from which

we can derive attack relationships?

April 20, 2012


Use case: Which camera should I buy?

April 20, 2012


Value-based Practical Reasoning Argumentation Scheme

Premises:

Before doing action A, the current circumstances are R;

After doing action A, the new circumstances are S;

G is a goal of the agent Ag, where S implies G;

Doing action A in R and achieving G promotes value V;

Conclusion:

We should perform action A.

April 20, 2012


Consumer Argumentation Scheme

Premises:

Camera X has property P.

Property P promotes value V for agent A.

Conclusion:

Agent A should Action1 Camera X.

April 20, 2012


Critical Questions

• Does Camera X have property P? • Does property P promote value V for agent A?• Is value V more important than value V’ for

agent A?

April 20, 2012


Analyst’s goal: instantiate

Premises: The Canon SX220 has good video quality.Good video quality promotes image quality for

casual photographers.

Conclusion: Casual photographers should buy the Canon

SX220.

April 20, 2012


… starting from this

April 20, 2012


Highlight parts of the argument

• Does Camera X have property P? • Does property P promote value V for agent A?• Is value V more important than value V’ for

agent A?

April 20, 2012


Highlight parts of the argument

• Argumentative indicators• Property – with camera terminology• Value for agent – with sentiment, user models• Value V more important – with comparisons

April 20, 2012


Implementing with a Text Analysis Tool

• GATE “General Architecture for Text Engineering”• Environment for text analysis• Adds annotation to text

– Highlight annotations with colours– Search for annotations– Can work with large corpora of text– Coarse or fine-grained annotations

April 20, 2012


Help analysts find relevant passages

• Indicators of premise after, as, because, for, since, when, .... • Indicators of conclusion therefore, in conclusion, consequently, .... • Indicators of contrast but, except, not, never, no, ....

April 20, 2012


Rhetorical terminology

April 20, 2012


Domain terminology

• Has a flash• Number of megapixels• Scope of the zoom• Lens size• The warranty

April 20, 2012


Domain terminology

April 20, 2012


Sentiment terminology

• The flash worked poorly• The flash worked flawlessly

April 20, 2012


Sentiment terminology

April 20, 2012


Agents: User Models

• User’s parametersAge, gender, education, previous camera experience, ....

• User’s context of useParty, indoors, sport, travel, desired output format, ....

• User’s constraintsCost, portability, size, richness or flexibility of features, ....

• User’s quality expectations Colour quality, information density, reliability, ....

April 20, 2012


Instantiating the CAS

Premises: The Canon SX220 camera has property P.Property P promotes value V for agent A.

Conclusion: Agent A should buy the Canon SX220.

April 20, 2012


Domain properties, positive sentiment,

premises

April 20, 2012


Query for patterns

April 20, 2012


An argument for buying the camera

Premises: The pictures are perfectly exposed. The pictures are well-focused. No camera shake. Good video quality.Each of these properties promotes image quality.

Conclusion: (You, the reader,) should buy the CanonSX220.April 20, 2012


An argument for NOT buying the camera

Premises: The colour is poor when using the flash.The images are not crisp when using the flash.The flash causes a shadow. Each of these properties demotes image quality.

Conclusion: (You, the reader,) should NOT buy the CanonSX220.

April 20, 2012


Counterarguments to the premises of “Don’t buy”

The colour is poor when using the flash. For good colour, use the colour setting, not the flash.

The images are not crisp when using the flash.No need to use flash even in low light.

The flash causes a shadow. There is a corrective video about the flash shadow.

April 20, 2012


Future Work

• Tool refinement• Add terminology modules to the tool• User models – how do they play a role• More complicated query patterns, what results do

we get?• More elaborate examples• Disambiguation issues for rhetorical terminology –

must deal with it step-by-step, what are the indicators we can use to disambiguate

April 20, 2012

Thanks to our funders!

• FP7-ICT-2009-4 Programme, IMPACT Project, Grant Agreement Number 247228.

• Science Foundation Ireland Grant No. SFI/08/CE/I1380 (Líon-2)

• COST Action ICO801 on Agreement TechnologiesShort-term scientific mission (STSM 1868)

• Upcoming: SFI Travel Supplement

31


Thanks for your attention!

• Questions?• Contacts:

– Jodi Schneider [email protected]– Adam Wyner [email protected]– Katie Atkinson [email protected]– Trevor Bench-Capon [email protected]

April 20, 2012

Identifying arguments for evaluation using an argument explorer - London Argumentation Forum -...

Technology

Transcript of Identifying arguments for evaluation using an argument explorer - London Argumentation Forum -...