Argumentation ExtensionsEnumeration as a Constraint

Satisfaction Problem: aPerformance Overview

Mauro Vallati, Federico Cerutti, Massimiliano Giacomin

DARe-2014 — Tuesday 19th August, 2014

Implementations for Enumerating PreferredExtensions

Two main approaches:1. Ad-hoc:

– NAD-Alg [Nofal et al., 2014];2. Reduction of enumerating preferred extensions into a:

– ASP AspartixM [Dvořák et al., 2011];– CSP CONArg2 [Bistarelli et al., 2014];– SAT (+ maximisation process) PrefSAT

[Cerutti et al., 2013].

BackgroundImplementationsEmpirical Results

Conclusions

Background

Definition

Given an AF Γ = 〈A,R〉, with R ⊆ A×A:- a set S ⊆ A is conflict–free if @ a,b ∈ S s.t. a→ b;- an argument a ∈ A is acceptable with respect to a set S ⊆ A if ∀b ∈ As.t. b→ a, ∃ c ∈ S s.t. c→ b;

- a set S ⊆ A is admissible if S is conflict–free and every element of S isacceptable with respect to S;

- a set S ⊆ A is a complete extension, i.e. S ∈ ECO(Γ), iff S is admissibleand ∀a ∈ A s.t. a is acceptable w.r.t. S, a ∈ S;

- a set S ⊆ A is a preferred extension, i.e. S ∈ EPR(Γ), iff S is a maximal(w.r.t. set inclusion) complete set.

Background

Definition

Let 〈A,R〉 be an AF : Lab : A 7→ {in, out, undec} is a complete labelling iff∀a ∈ A:

- Lab(a) = in⇔ ∀b ∈ a−Lab(b) = out;- Lab(a) = out⇔ ∃b ∈ a− : Lab(b) = in.

Let S ⊆ A a conflict–free set: the corresponding labelling isExt2Lab(S) ≡ Lab, where

- Lab(a) = in⇔ a ∈ S- Lab(a) = out⇔ ∃ b ∈ S s.t. b→ a- Lab(a) = undec⇔ a /∈ S ∧ @ b ∈ S s.t. b→ a

Proposition ([Caminada, 2006])Given an an AF Γ = 〈A,R〉, Lab is a complete (grounded, preferred)labelling of Γ if and only if there is a complete (grounded, preferred)extension S of Γ such that Lab = Ext2Lab(S).

An Example

An Example

Answer Set Programming

– Answer Set Programming is a recent problem solving approach;– It has roots in KR, logic programming, and nonmonotonic

reasoning;– The idea: stop trying to prove something, represent solutions, or

models (Answer Sets)!– Normal logic program P is a finite set of rules of the form:

a← b1, . . . , bm, not c1, . . . , not cn

where a, bi, cj are literals of the form p or ¬p (strong negation,also written as “-”) where p is a first-order atom from a classicalFOL signature.

– An answer set is a set of ground atoms that are “collectivelyacceptable”

Constraint Satisfaction Programming[Rossi et al., 2008]

DefinitionA Constraint Satisfaction Problem (CSP) P is a triple P = 〈X,D,C〉such that:

– X = 〈x1, . . . , xn〉 is a tuple of variables;– D = 〈D1, . . . , Dn〉 a tuple of domains such that ∀i, xi ∈ Di;– C = 〈C1, . . . , Ct〉 is a tuple of constraints, where∀j, Cj = 〈RSj , Sj〉, Sj ⊆ {xi|xi is a variable}, RSj ⊆ (SD

j )n

where SDj = {Di|Di is a domain, and xi ∈ Sj}.

DefinitionA solution to the CSP P is A = 〈a1, . . . , an〉 where ∀i, ai ∈ Di and∀j, RSj holds on the projection of A onto the scope Sj . If the set ofsolutions is empty, the CSP is unsatisfiable.

Propositional Satisfiability Problems

SATsolver

Φ1

Φ2

Φ3

SAT

UNSAT

SAT problem

– The SAT problem is a formula in conjunctive normal form(CNF):

Φi = (u1 ∨ ¬u2 ∨ u3) ∧ (u1 ∨ u2) ∧ (¬u1 ∨ ¬u2 ∨ u3)

– A solver searches a solution for the CNF, viz. a variableassignment satisfying the formula.

u1 = V , u2 = F , u3 = V

Background

ImplementationsEmpirical Results

Conclusions

NAD-Alg: [Nofal et al., 2014]

AspartixM: [Dvořák et al., 2011]

– Expresses argumentation semantics in Answer Set Programming(ASP);

– Tests for subset-maximality exploiting the metasp optimisationfrontend for the ASP-package gringo/claspD;

– Database of the form:

{arg(a) | a ∈ A} ∪ {defeat(a,b) | 〈a,b〉 ∈ R}

– Example of program for checking the conflict–freeness:

πcf = { in(X)← not out(X), arg(X);out(X)← not in(X), arg(X);← in(X), in(Y ),defeat(X,Y )}.

CONArg2: [Bistarelli and Santini, 2012,Bistarelli et al., 2014]

Given an AF 〈A,R〉:1. create a variable for each argument whose domain is always {0, 1}

— ∀ai ∈ A,∃xi ∈ X such that Di = {0, 1};2. describe constraints associated to different definitions of Dung’s

argumentation framework: e.g.{a,b} ⊆ A is conflict–free iff ¬(x1 = 1 ∧ x2 = 1);

3. solve the CSP problem.

PrefSAT: [Cerutti et al., 2013]

Given an AF Γ = 〈A,R〉, ΠΓ is a boolean formula (complete labellingformula) such that each satisfying assignment of the formulacorresponds to a complete labelling:

– k = |A|– φ : {1, . . . , k} 7→ A is a bijection (the inverse map is φ−1)– For each argument φ(i) we define three boolean variables:

– Ii, which is true when argument φ(i) is labelled in, falseotherwise;

– Oi, which is true when argument φ(i) is labelled out, falseotherwise;

– Ui, which is true when argument φ(i) is labelled undec, falseotherwise;

– V(Γ) , ∪1≤i≤|A|{Ii, Oi, Ui} (set of variables for the AF Γ)

PrefSAT: [Cerutti et al., 2013]

– Lab is a total function;– If a is not attacked, Lab(a) = in;– Lab(a) = in⇔ ∀b ∈ a−Lab(b) = out;– Lab(a) = out⇔ ∃b ∈ a− : Lab(b) = in;– Lab(a) = undec⇔ ∀b ∈ a−Lab(b) 6= in ∧ ∃c ∈ a− : Lab(c) =undec.

PrefSAT: [Cerutti et al., 2013]

∧i∈{1,...,k}

((Ii ∨ Oi ∨ Ui) ∧ (¬Ii ∨ ¬Oi)∧(¬Ii ∨ ¬Ui) ∧ (¬Oi ∨ ¬Ui)

)∧

∧{i|φ(i)−=∅}

(Ii ∧ ¬Oi ∧ ¬Ui) ∧∧

{i|φ(i)−6=∅}

Ii ∨

∨{j|φ(j)→φ(i)}

(¬Oj)

∧

∧{i|φ(i)−6=∅}

∧{j|φ(j)→φ(i)}

¬Ii ∨ Oj

∧∧

{i|φ(i)−6=∅}

∧{j|φ(j)→φ(i)}

¬Ij ∨ Oi

∧

∧{i|φ(i)−6=∅}

¬Oi ∨

∨{j|φ(j)→φ(i)}

Ij

∧

∧{i|φ(i)−6=∅}

∧{k|φ(k)→φ(i)}

Ui ∨ ¬Uk ∨

∨{j|φ(j)→φ(i)}

Ij

∧

∧{i|φ(i)−6=∅}

∧{j|φ(j)→φ(i)}

(¬Ui ∨ ¬Ij)

∧

¬Ui ∨

∨{j|φ(j)→φ(i)}

Uj

BackgroundImplementations

Empirical ResultsConclusions

The Experimental hypothesis

There will be a strict ordering — under any configuration— regarding the performance of the software measured in (1)CPU-time needed to enumerate all the preferred extensionsgiven an AF and in (2) percentage of successfulenumeration. Such an ordering should see the ad-hocapproach NAD-Alg as the best one, followed by PrefSAT,CONArg2, and finally AspartixM.

Empirical Evaluation: the Experiment

– Random generated 720 AF s divided in different classes accordingto two dimensions:

– |A|: ranging from 25 to 225 with a step of 25;– generation of the attack relations:

– fixing the probability patt ∈ {0.25, 0.5, 0.75} that there isan attack for each ordered pair of arguments: 10 AF sforbidding self-attacks, 10 AF s allowing self-attacks;

– selecting randomly the number natt of attacks in it: 20AF s.

Analysis Using the International PlanningCompetition (IPC) Score

– For each test case (in our case, each test AF ) let T ∗ be the bestexecution time among the compared systems (if no systemproduces the solution within the time limit, the test case is notconsidered valid and ignored).

– For each valid case, each system gets a score of1/(1 + log10(T/T ∗)), where T is its execution time, or a score of 0if it fails in that case. Runtimes below 1 sec get by default themaximal score of 1.

– The (non normalized) IPC score for a system is the sum of itsscores over all the valid test cases. The normalised IPC scoreranges from 0 to 100 and is defined as(IPC/# of valid cases) ∗ 100.

IPC score w.r.t. number of arguments

0

20

40

60

80

100

25 50 75 100 125 150 175 200 225

CONArg2AspartixM

PrefSATNAD-Alg

Normalised IPC score (y axis) w.r.t. the number of arguments (x axis)of each considered system.

Average runtime w.r.t. number of arguments

Average CPU-Time25 50 75 100 125 150 175 200 225

CONArg2 0.25 0.27 0.65 2.15 5.48 14.98 73.78 86.62 187.11AspartixM 0.18 0.67 1.44 3.26 6.02 15.70 27.99 87.46 117.18PrefSAT 0.04 0.11 0.23 0.44 0.81 1.67 3.76 6.41 16.21NAD-Alg 0.01 0.02 0.06 0.99 10.23 12.74 60.35 42.78 75.07

Average runtime for each of the considered solvers, according to thenumber of arguments of the AF s.

IPC score w.r.t. probability of attacks

40

60

80

100

25 50 75 RAND

CONArg2AspartixM

PrefSATNAD-Alg

Normalised IPC score (y axis) w.r.t. the probability of attacks (xaxis) of each considered system.

Average runtime w.r.t. probability of attacks

% Solved Average CPU-Time25 50 75 RAND 25 50 75 RAND

CONArg2 97.8 100.0 100.0 97.2 87.4 11.0 7.1 59.6AspartixM 98.3 100.0 100.0 98.9 56.5 14.7 10.0 34.0PrefSAT 100.0 100.0 100.0 100.0 5.1 1.6 2.2 4.2NAD-Alg 100.0 100.0 100.0 93.9 18.9 0.2 0.2 70.6

Percentage of solved AF s and average runtime for each of theconsidered solvers, according to the percentages of attacks.

BackgroundImplementationsEmpirical Results

Conclusions

Concluding Remarks

– First comparison of state-of-the-art approaches which transformthe preferred enumeration problem into a CSP (CONArg2),ASP (AspartixM) and SAT (PrefSAT) with the bestargumentation-dedicated approach NAD-Alg;

– Experimental hypothesis partially true: most of the cases thisorder: NAD-Alg, PrefSAT, CONArg2 and AspartixM. Butthere are several cases in which:1. PrefSAT has been the best approach — and it is also the

only one implementation that solved all the AF s consideredin the experiment;

2. AspartixM performed significantly — according to theFriedman statistic test confirmed by a post-hoc analysis withthe Wilcoxon signed rank with a Bonferroni correctionapplied — better than CONArg2.

Future Works

– Larger experimental evaluation;– Exploitation of a white-box approach: looking at the design of

the solvers.

Acknowledgement

The authors would like to acknowledge the use of the University ofHuddersfield Queensgate Grid in carrying out this work.

Advert

References I

[Bistarelli et al., 2014] Bistarelli, S., Rossi, F., and Santini, F. (2014).Enumerating Extensions on Random Abstract-AFs with ArgTools, Aspartix, ConArg2, andDung-O-Matic.In Bulling, N., van der Torre, L., Villata, S., Jamroga, W., and Vasconcelos, W., editors,Computational Logic in Multi-Agent Systems, volume 8624 of Lecture Notes in ComputerScience, pages 70–86. Springer International Publishing.

[Bistarelli and Santini, 2012] Bistarelli, S. and Santini, F. (2012).Modeling and solving afs with a constraint-based tool: Conarg.In Modgil, S., Oren, N., and Toni, F., editors, Theorie and Applications of FormalArgumentation, volume 7132 of Lecture Notes in Computer Science, pages 99–116. SpringerBerlin Heidelberg.

[Caminada, 2006] Caminada, M. (2006).On the issue of reinstatement in argumentation.In Proceedings of JELIA 2006, pages 111–123.

[Cerutti et al., 2013] Cerutti, F., Dunne, P. E., Giacomin, M., and Vallati, M. (2013).Computing preferred extensions in abstract argumentation: A sat-based approach.In Proceedings of Theory and Applications of Formal Argumentation (TAFA 2013), pages176–193.

References II

[Dvořák et al., 2011] Dvořák, W., Gaggl, S. A., Wallner, J., and Woltran, S. (2011).Making Use of Advances in Answer-Set Programming for Abstract Argumentation Systems.In Proceedings of the 19th International Conference on Applications of DeclarativeProgramming and Knowledge Management (INAP 2011).

[Nofal et al., 2014] Nofal, S., Atkinson, K., and Dunne, P. E. (2014).Algorithms for decision problems in argument systems under preferred semantics.Artificial Intelligence, 207:23–51.

[Rossi et al., 2008] Rossi, F., van Beek, P., and Walsh, T. (2008).Chapter 4 constraint programming.In Frank van Harmelen, V. L. and Porter, B., editors, Handbook of Knowledge Representation,volume 3 of Foundations of Artificial Intelligence, pages 181 – 211. Elsevier.